I've recently been trying to set up unicast DNS-SD to make printing easier for our Mac users, but I've hit an odd snag.
We use two Windows Server 2008 R2 machines as the DNS servers between end-users and the Internet, but for some reason they return a SERVFAIL response when forwarding printers' TXT records. Other DNS servers (including those running Windows Server 2003) have no problem at all.
I even set up a fresh Windows Server 2008 R2 instance (on EC2) to rule out firewall and network configuration issues. The issue still occurs. My research revealed some issues with 2008 R2 and EDNS, but the command that others claim fixed the issue hasn't helped.
Has anyone else set up unicast DNS-SD sharing that works through a Windows 2008 R2 DNS server? Any idea what could be causing this?
It looks like the query message (0x3766) is being sent over UDP without EDNS (there would be an Opt RR in the additional section if it were using EDNS) which limits the maximum response to 512 bytes. The TXT data alone is 546 bytes so that query can't be answered in 512 bytes. The server should be replying with a truncated (TC bit set) response which would trigger the client to retry the query over TCP.
I'm not that familiar with Windows so all I can suggest is to reduce the size of the TXT record. Personally, I'd try setting the record data to "1" and see how that affects the client. (A TXT record is required for a DNS-SD service and some DNS servers/clients choke on empty TXT records so a single character is the minimum you can expect to get away with.)