andrewf's questions

Okay, I’m going just about insane here.

Trying to set up an OS X (10.8.2) NFS client to connect to Debian (6.0.6) nfs server. I’ve mostly been following instructions here: https://help.ubuntu.com/community/NFSv4Howto which are very thorough. However, two issues:

• When I don’t use Kerberos, the performance sucks (I think something must be timing-out), and all the files are apparently owned by nobody:nobody. Doing an ls of the mounted directory takes tens of seconds.
• When I do try to use Kerberos, with a principal created for the server and for the client, I cannot mount the share. The client complains: mount_nfs: can't mount /mnt from servername.domain onto /home: Permission denied

I’ve set up two principals, nfs/servername@REALM and nfs/clientname@REALM, and copied into /etc/krb5.keytab on client and server.

The server is (according to Kerberos logs) getting tickets for nfs/servername.REALM so my suspicion is that the OS X NFS client isn't using its principal somehow.

Any ideas?

Update: /var/log/daemon.log reports:

Oct  7 19:12:43 servername rpc.svcgssd[19635]: ERROR: GSS-API: error in handle_nullreq: gss_accept_sec_context(): Unspecified GSS failure.  Minor code may provide more information - No supported encryption types (config file error?)

As per this page: http://permalink.gmane.org/gmane.comp.encryption.kerberos.heimdal.general/5551 I’ve removed all but the DES encryption options in both client and server krb5.keytab files. And added allow_weak_crypto in the /etc/krb5.conf file. And restarted hfs-kernel-server and it still doesn’t work. Sigh.

Actually, now it reports:

Oct  7 19:26:55 servername rpc.svcgssd[19721]: ERROR: GSS-API: error in handle_nullreq: gss_accept_sec_context(): Unspecified GSS failure.  Minor code may provide more information - Wrong principal in request

Does that “Wrong principal in request” mean that the client is passing the wrong krb5 principal? Can I control that in OS X?

I've started seeing errors reported by LVM on certain Logical Volumes (and by Xen when attempting to create virtual machines on these LVs). But I've run tests on the disk, and can't see any hardware problems.

We're running a XEN/Linux (Debian Lenny) box here, running off a single SATA disk managed with LVM2. It's been in and running for more than a year, with the only major changes being recent apt-get upgrade of the kernel.

# uname -a
Linux hostname 2.6.26-2-xen-amd64 #1 SMP Thu Sep 16 16:32:15 UTC 2010 x86_64 GNU/Linux

The errors appear like this:

# vgck
/dev/dm-20: read failed after 0 of 4096 at 0: Input/output error

And then when I try to start the VM which uses that LV for its C-drive (it's a Windows virtual machine), the VM refuses to start and I see this at the end of the /var/log/xen/qemu-dm-*.log log file:

...
Register xen platform.
Done register platform.
raw_read(6:/dev/vgroup/newvm-cdrive, 0, 0x7fff02bca520, 512) [20971520] read failed -1 : 5 = Input/output error
I/O request not ready: 0, ptr: 0, port: 0, data: 0, count: 0, size: 0
raw_read(6:/dev/vgroup/newvm-cdrive, 0, 0x12dfff0, 512) [20971520] read failed -1 : 5 = Input/output error

This first happened on 2 VMs whose disk was based on a snapshot of a third, original VM. I nuked the 2 LVs and recreated them (again by snapshotting the same, original VM's LV), and they've been fine since.

However, today I attempted to create a new VM. I snapshotted the same, orginal VM's LV (lvcreate -L500M --snapshot --name newvm-cdrive /dev/vgroup/original-cdrive), and created the new VM. It initially worked, but after shutting down the VM once, it refuses to start up again, with the errors shown above.

My obvious first guess would be physical problems with the drive, but smartmon does not report anything:

# smartctl -t long /dev/sda
# [later]
# smartctl -l selftest /dev/sda
smartctl version 5.38 [x86_64-unknown-linux-gnu] Copyright (C) 2002-8 Bruce Allen
Home page is http://smartmontools.sourceforge.net/

=== START OF READ SMART DATA SECTION ===
SMART Self-test log structure revision number 1
Num  Test_Description    Status                  Remaining  LifeTime(hours)  LBA_of_first_error
# 1  Extended offline    Completed without error       00%         1         -
# 2  Short offline       Completed without error       00%         0         -

Also, not getting any errors from badblocks.

I've tried running vgck and pvck:

# vgck vgroup -v
    Using volume group(s) on command line
    Finding volume group "vgroup"
  /dev/dm-20: read failed after 0 of 4096 at 0: Input/output error

# pvck /dev/sda2
  Found label on /dev/sda2, sector 1, type=LVM2 001
  Found text metadata area: offset=4096, size=192512

Have found a few references to this error message ("read failed after 0 of 4096 at...") on the Interwebs, but nothing which seems to apply to my situation.

Any ideas?

Update: As requested, below is output of lvdisplay and ls -l. Running out of COW space is plausible. How do I tell?

# lvdisplay /dev/vgroup/newvm-cdrive
  /dev/dm-20: read failed after 0 of 4096 at 0: Input/output error
  --- Logical volume ---
  LV Name                /dev/vgroup/newvm-cdrive
  VG Name                vgroup
  LV UUID                jiarxt-q2NO-SyIf-5FrW-I9iq-mNEQ-iwS4EH
  LV Write Access        read/write
  LV snapshot status     INACTIVE destination for /dev/vgroup/original-cdrive
  LV Status              available
  # open                 0
  LV Size                10.00 GB
  Current LE             2560
  COW-table size         200.00 MB
  COW-table LE           50
  Snapshot chunk size    4.00 KB
  Segments               1
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     256
  Block device           254:20

# ls -l /dev/dm-20
brw-rw---- 1 root disk 254, 20 2010-10-11 15:02 /dev/dm-20

And here's fdisk -l.

# fdisk -l /dev/sda

Disk /dev/sda: 160.0 GB, 160000000000 bytes
255 heads, 63 sectors/track, 19452 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x00000080

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *           1          31      248976   83  Linux
/dev/sda2              32       19452   155999182+  8e  Linux LVM