I have a weird situation going on with an open network port. My main question is, why would there not be a program associated with an open TCP port:
netstat -ln --program
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 0.0.0.0:5666 0.0.0.0:* LISTEN -
For my specific case, there is supposed to be an nrpe daemon (opsview install) listening on port 5666, but there is no nrpe daemon running. If I try to start it, it exits immediately.
lsof -i :5666
doesn't show any output either. There is no (x)inetd running on my system.
UPDATE
Yes, I was running those commands as root. Telnet would could, but there was never any response.
After further investigation, I found a kernel error in dmesg
: this was an EC2 instance (actually several of them) running an older kernel (2.6.16 is apparently unstable). The fix to stop the crash was to upgrade kernels.
It looks like the way the kernel crashed caused the process to go away and leave the port open.
Have you run netstat and lsof as root or with sudo? Notice the last column:
From the netstat manpage:
How do you know there isn't one running? If the port is in use it makes sense that it would exit immediately with a 'socket in use' error. what happens when you telnet to the port?
Ports open by the kernel won't show up with program name. Some NFS and OCFS stuff come to mind. Maybe it's something like that?
Or it could be a kernel bug. Check kernel logs for OOPS and BUG.
execute 'netstat --tcp --udp --listening --program' as root user. other wise you it won't give PID/Program Name
then use kill -9 PID command
I actually wrote a small shell script to help id these occassional questions:
save as /usr/local/bin/tracer; output:
You will need root privileges to use it
I was able to track down the process by getting its inode via netstat and then using that inode with lsof. See my more detailed answer in https://serverfault.com/a/847910/94376.
Sometimes, nfs related programs cannot be seen on the program list.
Also, LDAP pam modules and libnss_ldap open connections to ldap servers, but there is not an actual process holding the connection open, so netstat -tnp shows an active connection without a process.