I have noticed that a Solaris 9 server keeps sending ARP requests for an IP address that is owned by our DNS server.
The DNS server answers with its MAC address but Solaris 9 does not record the address in the ARP table. You can ping the DNS server just fine from it but for every packet that goes out, it needs to request the MAC address again and again.
Here is the snoop output:
10.0.0.12 -> (broadcast) ARP C Who is 10.0.0.4, 10.0.0.4 ?
10.0.0.4 -> 10.0.0.12 ARP R 10.0.0.4, 10.0.0.4 is 0:14:4f:ac:cf:2e
10.0.0.12 -> (broadcast) ARP C Who is 10.0.0.4, 10.0.0.4 ?
10.0.0.4 -> 10.0.0.12 ARP R 10.0.0.4, 10.0.0.4 is 0:14:4f:ac:cf:2e
10.0.0.12 -> (broadcast) ARP C Who is 10.0.0.4, 10.0.0.4 ?
10.0.0.4 -> 10.0.0.12 ARP R 10.0.0.4, 10.0.0.4 is 0:14:4f:ac:cf:2e
Even though it's answering, 10.0.0.4 never shows in the ARP table but ping works all the time (presumably because it requests the MAC address for each request and thus can send the ICMP packet).
I've confirmed the source MAC address in the requests are indeed from the Solaris 9 server and the replies also ahve the correct source address from the DNS server. Basically everything seems to be working fine unles... ARP poisoning?
What makes this even more weird is that it works fine for all other IP address in the 10.0.0.0/24 network. Their MAC addresses are cached in the ARP table but this one doesn't.
Is there anything special about IP addresses appointed as DNS servers in /etc/resolv.conf that would make ARP behave like this? The other 2 DNS servers are not on the same subnet so I cannot test if it happens with other addresses.
# ndd -get /dev/arp arp_cleanup_interval
60000
Turned out the system was missing some important patches: