Home / server / Questions / 153996
Accepted
Dan Carley
Asked: 2010-06-24 04:41:27 +0800 CST

Why is rpc.lockd obscured from netstat/lsof output?

  • 772

Prologue:

On a number of machines, which happen to act as NFS clients, netstat reports two ports that are open with no PID listed for an associated daemon. Ordinarily this might be a bit concerning.

# netstat -lnp | egrep -- '- +$'
tcp        0      0 0.0.0.0:57448           0.0.0.0:*               LISTEN     -
udp        0      0 0.0.0.0:48933           0.0.0.0:*                          -

Additionally netcat confirms that the TCP port really is open.

# nc -v localhost 57448
localhost [127.0.0.1] 57448 (?) open
^C

Yet lsof reports nothing for these two ports. Intrigue grows.

# lsof -i TCP:57448 -i UDP:48933

However rpcinfo finally points us in the right direction. It's being held open by nlockmgr, aka lockd for NFS. Call off the search.

# rpcinfo -p | egrep '57448|48933'
    100021    1   udp  48933  nlockmgr
    100021    3   udp  48933  nlockmgr
    100021    4   udp  48933  nlockmgr
    100021    1   tcp  57448  nlockmgr
    100021    3   tcp  57448  nlockmgr
    100021    4   tcp  57448  nlockmgr

It becomes clear that lockd/rpc.lockd is called when mounting an NFS export. This is a kernel thread (is it always?) which binds itself to one TCP and one UDP port in the ephemeral range. The ports are typically reconfigurable with the fs.nfs.nlm_tcpport and fs.nfs.nlm_udpport sysctls.

Questions:

I am intrigued though. Would love some kernel-internals insight as to..

  1. Why isn't the kernel thread's PID visible from netstat?

  2. Why aren't the bound ports visible from lsof?

linux nfs rpc netstat lsof

1 Answers

  1. Best Answer

    netstat and lsof both crawl /proc/<pid>/fd to associate processes to ports/sockets, and /proc/<pid>/fd doesn't get populated for kernel threads AFAIK.

    lockd is always a kernel thread - at least on modern (newer than say 2.2) kernels.

    • 7