MegaBrutal's questions

I have an NFS mount over a Strongswan IPSec tunnel, which is encapsulated in a 6to4 tunnel. The IPSec is because I need encryption for NFS traffic, the 6to4 is because the VPS provider won't assign a native IPv6 prefix to my server. Because I had MTU problems with the 6to4 tunnel, I had to lower the MTU on the tunnel interface to the minimum (1280 – if I try to set anything lower, I get an "Error: mtu less than device minimum." message).

NFS still wants to send packets over MTU. I know this, because I have an nftables rule to log ESP packets:

    chain output {
    type filter hook output priority filter; policy accept;
    ip6 nexthdr esp counter packets 303367 bytes 323173696 log accept
}

Thus I see these packets logged in syslog/journal:

Jan 29 21:41:18 nfsclient kernel: IN= OUT=he-ipv6 SRC=fd48:2b50:6a95:a6db:0000:0000:0000:0004 DST=fdc8:d5f9:cbbf:b206:0000:0000:0000:2001 LEN=1316 TC=0 HOPLIMIT=64 FLOWLBL=155038

(IPs are changed to private for privacy reasons.)

Now I can't see the logged packet with tcpdump because supposedly they get dropped by the kernel due to being over MTU. I assume NFS tries to adhere to the MTU setting, but it doesn't know that its packages will be encapsulated in IPSec. So even if NFS generates a packet under 1280 bytes, the ESP header added to it gets it over the set MTU. I also suspect that NFS sets the DF flag on its packets, because otherwise fragmentation would work. (I tested it with ping6 -M want and fragmented packets went through.) So I can't lower the MTU, NFS insists to send packets those will be over MTU when encrypted and even sets the DF flag.

What can I do now? The following things I thought about, but don't know how to implement:

1. Set a maximum packet size for NFS, e.g. with a mount option, but I don't think there is such a setting, I already searched for it.
2. Configure Strongswan to deal with the situation, but I didn't find such option either.
3. Set an nftables rule that somehow notifies NFS that it should generate smaller packets. E.g. report an even lower MTU for NFS when it looks for it – don't know if it's even possible.
4. Remove the DF flag from the packets to force fragmentation. I don't know how to do it either, or whether it's possible.

I think nftables is the way to go, but if it could be solved on NFS level, it would be even better. I'd also appreciate solutions with iptables, I could look up what's the nftables equivalent.

Because it's asked in comments, I provide information about my interfaces.

2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT group default qlen 1000
    link/ether aa:00:11:4d:f7:01 brd ff:ff:ff:ff:ff:ff
4: he-ipv6@NONE: <POINTOPOINT,NOARP,UP,LOWER_UP> mtu 1280 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
    link/sit 192.168.32.84 peer 216.66.84.42

And here are the tunnels (ip tunnel):

sit0: ipv6/ip remote any local any ttl 64 nopmtudisc 6rd-prefix 2002::/16
he-ipv6: ipv6/ip remote 216.66.84.42 local 192.168.32.84 ttl 255 6rd-prefix 2002::/16

(Changed my public IPv4 to a private address, but in reality I have a globally routable address for local; 216.66.84.42 is the HE 6to4 tunnel gateway, which is well-known so left here.)

And here is the default route that applies for the traffic:

default via fd48:2b50:6a95:a6db::1 dev he-ipv6 metric 1024 onlink pref medium

So applications believe their packets will go out on he-ipv6, which has an MTU of 1280. But their packets first get encapsulated in IPSec ESP, and then sent through the he-ipv6 tunnel. The result is an IPSec-encrypted NFS data packet encapsulated in a 6to4 packet which itself goes out on the eth0 interface towards 216.66.84.42 (HE gateway).

I exported an NFS share from a server and mounted it from a client. I got permission denied errors when I tried to access it, wondered why. Made the export world-writable, created a file there, and it turned out that the client accesses it with a different user than I would expect.

My export:

/export             fd48:2b50:6a95:a6db::4(ro,no_subtree_check,fsid=0,all_squash)
/export/backup-fs   fd48:2b50:6a95:a6db::4(rw,no_subtree_check,root_squash,anonuid=34,anongid=34)

(Note: public IP addresses changed to private for privacy reasons.)

The intention here is to do root_squash, BUT the client's root user should be squashed to the local UID 34 (backup user); this is the purpose of anonuid, anongid. Thus, if the export is owned by UID/GID 34, the client's root user should be treated as it would be an owner of it. Yet when I create a file on the share, it gets owned by nobody/nogroup. Why? Everything matches between the two systems: both has UID/GID 34 and both named backup.

Also I had to change /export to rw, which doesn't really make sense to me, because I want the export root to be read-only and only the subdir backup-fs writable.

Server system:

Ubuntu 20.04.3 LTS (GNU/Linux 5.4.0-81-generic x86_64)

This is how the mount looks like on the client:

my-nfs-server:/backup-fs on /var/backups/fs/data type nfs4 (rw,relatime,vers=4.2,rsize=65536,wsize=65536,namlen=255,hard,proto=tcp6,timeo=600,retrans=2,sec=sys,clientaddr=fd48:2b50:6a95:a6db::4,local_lock=none,addr=fdc8:d5f9:cbbf:b206::2001) 

What particularly puzzles me is that I have THIS EXACT SAME config on another NFS server and I mount it WITH THE VERY SAME client and it works as intended. I can't find any difference between the two configurations, albeit I have an older Ubuntu and associated kernel version on the other system.

I have a virtio disk mounted to a KVM guest, backed by an LVM logical volume. I increased the size of the logical volume with lvresize on the KVM host machine.

Then I had to issue virsh pool-refresh to make the KVM host aware of the volume size change. Now if I check the volume in virt-manager, I see the correct disk size. But the guest OS still sees the disk by its old size no matter what.

How can I force the Linux kernel to rescan virtio disks without rebooting the guest?

The other solutions I found were not specific to virtio and they didn't work. Like echo "- - -" > /sys/class/scsi_host/host0/scan should work for SCSI disks, but not virtio. Another tip, echo 1 > /sys/block/vda/device/rescan doesn't work because the rescan file doesn't exist.

So do you know about any virtio specific tip to have my virtio disk rescanned?