I have an interesting situation.
I'm trying to us a Linux based machine to allow Mac's to Netboot (similiar to PXE boot) by running a DHCP service in parallel with the "global" DHCP server.
The local DHCP server hands out IPs in a private subnet, e.g., 10.168.0.10-10.168.254-254, while the "global" DHCP server hands out IPs from the IP range 10.0.0.1 - 10.0.1.254.
The local DHCP range is only supposed to be used in Preboot Execution Environment and Netboot. The local DHCP server is something I have control over, but I do not have access to the global DHCP server.
I have a filter to only allow members with the vendor strings "AAPLBSDPC/i386" and "PXEClient".
PXE works fine, but Netboot has a quirk.
The Apple systems that haven't been connected to the network yet can Netboot fine. But once it grabs a "real" IP address from the global DHCP server, it will "save" it and request it the next time we want it to netboot (which the local dhcp server won't give it).
This is what I want:
Mar 30 10:52:28 dev01 dhcpd: DHCPDISCOVER from 34:15:xx:xx:xx:xx via eth1
Mar 30 10:52:29 dev01 dhcpd: DHCPOFFER on 10.168.222.46 to 34:15:xx:xx:xx:xx via eth1
Mar 30 10:52:31 dev01 dhcpd: DHCPREQUEST for 10.168.222.46 (10.168.0.1) from 34:15:xx:xx:xx:xx via eth1
Mar 30 10:52:31 dev01 dhcpd: DHCPACK on 10.168.222.46 to 34:15:xx:xx:xx:xx via eth1
Mar 30 10:52:32 dev01 in.tftpd[5890]: tftp: client does not accept options
Mar 30 10:52:53 dev01 in.tftpd[5891]: tftp: client does not accept options
Mar 30 10:52:53 dev01 in.tftpd[5893]: tftp: client does not accept options
Mar 30 10:52:54 dev01 in.tftpd[5895]: tftp: client does not accept options
This is what I get when it already has a "stored" IP:
Mar 30 10:51:29 dev01 dhcpd: DHCPDISCOVER from 00:25:xx:xx:xx:xx via eth1
Mar 30 10:51:30 dev01 dhcpd: DHCPOFFER on 10.168.222.45 to 00:25:xx:xx:xx:xx via eth1
Mar 30 10:51:31 dev01 dhcpd: DHCPREQUEST for 10.0.0.61 (10.0.0.1) from 00:25:xx:xx:xx:xx via eth1: ignored (not authoritative).
Do you have any suggestions? It would be much appreciated.
EDIT: I think the DHCP server should be NACK'ing the request if it's in the Apple class... could I just stick the 'authoritative' statement inside the class that filters out Apple Netbooting systems?
[Removed tcpdump from local DHCP server]
I tried zapping the pram with the key combo, but it didn't work. It still reports the same thing in the DHCP logs. I'm looking into additional random options in the DHCP config for now.
Edit:
What seems interesting, is if I go into the operating system, turn off en0, then restart and try NetBooting (therefore releasing the IP?). Netboot will pickup an IP from the local server and Netboot correctly.
Do you have any ideas to why this works?
(I wanted to thank you for all your effort so far, you have been really helpful.)
Here's the port traces via a mirrored port of the Netboot Client.
Legend (Just in case):
IP Addr
- 10.0.* is the Global IP Range [Public LAN]
- 10.168.* is the Local IP Range [Private LAN/For Netboot/PXE]
MAC
- 34:15:xx:... is the Netboot Client
- 00:1e:xx:... is the Local DHCP Server
- 00:24:xx:... is the Global DHCP Server
Trace when it doesn't work:
tcpdump: listening on en0, link-type EN10MB (Ethernet), capture size 65535 bytes
15:01:10.765615 34:15:xx:xx:xx:xx > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 590: (tos 0x0, ttl 16, id 163, offset 0, flags [none], proto UDP (17), length 576)
0.0.0.0.68 > 255.255.255.255.67: [udp sum ok] BOOTP/DHCP, Request from 34:15:xx:xx:xx:xx, length 548, xid 0x2b93, secs 5, Flags [none] (0x0000)
Client-Ethernet-Address 34:15:xx:xx:xx:xx
Vendor-rfc1048 Extensions
Magic Cookie 0x63825363
DHCP-Message Option 53, length 1: Discover
Parameter-Request Option 55, length 5:
Subnet-Mask, Default-Gateway, BF, Vendor-Option
Vendor-Class
Client-ID Option 61, length 7: ether 34:15:xx:xx:xx:xx
Vendor-Class Option 60, length 28: "AAPLBSDPC/i386/MacBookPro5,3"
Vendor-Option Option 43, length 4: 2.2.1.1
END Option 255, length 0
PAD Option 0, length 0, occurs 252
15:01:10.784087 00:24:xx:xx:xx:xx > 34:15:xx:xx:xx:xx, ethertype 802.1Q (0x8100), length 346: vlan 1, p 0, ethertype IPv4, (tos 0x0, ttl 255, id 17248, offset 0, flags [none], proto UDP (17), length 328)
10.0.129.254.67 > 10.0.128.63.68: [udp sum ok] BOOTP/DHCP, Reply, length 300, xid 0x2b93, Flags [none] (0x0000)
Your-IP 10.0.128.63
Server-IP 10.0.178.10
Gateway-IP 10.0.129.254
Client-Ethernet-Address 34:15:xx:xx:xx:xx
Vendor-rfc1048 Extensions
Magic Cookie 0x63825363
DHCP-Message Option 53, length 1: Offer
Subnet-Mask Option 1, length 4: 255.255.254.0
RN Option 58, length 4: 1296000
RB Option 59, length 4: 2268000
Lease-Time Option 51, length 4: 2592000
Server-ID Option 54, length 4: 10.0.178.10
Default-Gateway Option 3, length 4: 10.0.129.254
END Option 255, length 0
PAD Option 0, length 0, occurs 20
15:01:11.527910 00:1e:xx:xx:xx:xx > 34:15:xx:xx:xx:xx, ethertype 802.1Q (0x8100), length 416: vlan 1, p 0, ethertype IPv4, (tos 0x10, ttl 128, id 0, offset 0, flags [none], proto UDP (17), length 398)
10.168.0.1.67 > 10.168.0.11.68: [udp sum ok] BOOTP/DHCP, Reply, length 370, xid 0x2b93, secs 5, Flags [none] (0x0000)
Your-IP 10.168.0.11
Server-IP 10.168.0.1
Client-Ethernet-Address 34:15:xx:xx:xx:xx
sname "10.168.0.1"
file "macnbi-i386/booter"
Vendor-rfc1048 Extensions
Magic Cookie 0x63825363
DHCP-Message Option 53, length 1: Offer
Server-ID Option 54, length 4: 10.168.0.1
Lease-Time Option 51, length 4: 86400
Subnet-Mask Option 1, length 4: 255.255.0.0
Default-Gateway Option 3, length 4: 10.168.0.1
RP Option 17, length 76: "http://10.0.128.1/Netboot/NetBootSP0/NetRestore.nbi/NetInstall-Restore.dmg"
Vendor-Option Option 43, length 6: 8.4.129.0.0.103
Vendor-Class Option 60, length 14: "AAPLBSDPC/i386"
END Option 255, length 0
15:01:12.865888 34:15:xx:xx:xx:xx > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 590: (tos 0x0, ttl 16, id 39430, offset 0, flags [none], proto UDP (17), length 576)
0.0.0.0.68 > 255.255.255.255.67: [udp sum ok] BOOTP/DHCP, Request from 34:15:xx:xx:xx:xx, length 548, xid 0x2b93, secs 5, Flags [none] (0x0000)
Client-Ethernet-Address 34:15:xx:xx:xx:xx
Vendor-rfc1048 Extensions
Magic Cookie 0x63825363
DHCP-Message Option 53, length 1: Request
Parameter-Request Option 55, length 5:
Subnet-Mask, Default-Gateway, BF, Vendor-Option
Vendor-Class
Client-ID Option 61, length 7: ether 34:15:xx:xx:xx:xx
Vendor-Class Option 60, length 28: "AAPLBSDPC/i386/MacBookPro5,3"
Requested-IP Option 50, length 4: 10.0.128.63
Server-ID Option 54, length 4: 10.0.178.10
Vendor-Option Option 43, length 4: 2.2.1.1
END Option 255, length 0
PAD Option 0, length 0, occurs 240
15:01:12.868182 00:24:xx:xx:xx:xx > 34:15:xx:xx:xx:xx, ethertype 802.1Q (0x8100), length 346: vlan 1, p 0, ethertype IPv4, (tos 0x0, ttl 255, id 17251, offset 0, flags [none], proto UDP (17), length 328)
10.0.129.254.67 > 10.0.128.63.68: [udp sum ok] BOOTP/DHCP, Reply, length 300, xid 0x2b93, Flags [none] (0x0000)
Your-IP 10.0.128.63
Gateway-IP 10.0.129.254
Client-Ethernet-Address 34:15:xx:xx:xx:xx
Vendor-rfc1048 Extensions
Magic Cookie 0x63825363
DHCP-Message Option 53, length 1: ACK
RN Option 58, length 4: 1296000
RB Option 59, length 4: 2268000
Lease-Time Option 51, length 4: 2592000
Server-ID Option 54, length 4: 10.0.178.10
Subnet-Mask Option 1, length 4: 255.255.254.0
Default-Gateway Option 3, length 4: 10.0.129.254
END Option 255, length 0
PAD Option 0, length 0, occurs 20
15:01:12.868185 34:15:xx:xx:xx:xx > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 60: Ethernet (len 6), IPv4 (len 4), Request who-has 10.0.128.63 tell 0.0.0.0, length 46
15:01:13.367995 34:15:xx:xx:xx:xx > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 60: Ethernet (len 6), IPv4 (len 4), Request who-has 10.0.128.63 tell 10.0.128.63, length 46
15:01:13.868312 34:15:xx:xx:xx:xx > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 60: Ethernet (len 6), IPv4 (len 4), Request who-has 10.0.129.254 tell 10.0.128.63, length 46
15:01:13.868854 00:24:xx:xx:xx:xx > 34:15:xx:xx:xx:xx, ethertype 802.1Q (0x8100), length 64: vlan 1, p 0, ethertype ARP, Ethernet (len 6), IPv4 (len 4), Reply 10.0.129.254 is-at 00:24:xx:xx:xx:xx, length 46
15:01:13.868857 34:15:xx:xx:xx:xx > 00:24:xx:xx:xx:xx, ethertype IPv4 (0x0800), length 89: (tos 0x0, ttl 16, id 39236, offset 0, flags [none], proto UDP (17), length 75)
10.0.128.63.15789 > 10.168.0.1.69: [udp sum ok] 47 RRQ "macnbi-i386/booter" octet blksize 512 tsize 0
15:01:18.968010 34:15:xx:xx:xx:xx > 00:24:xx:xx:xx:xx, ethertype IPv4 (0x0800), length 89: (tos 0x0, ttl 16, id 41750, offset 0, flags [none], proto UDP (17), length 75)
10.0.128.63.15790 > 10.168.0.1.69: [udp sum ok] 47 RRQ "macnbi-i386/booter" octet blksize 512 tsize 0
15:01:24.067221 34:15:xx:xx:xx:xx > 00:24:xx:xx:xx:xx, ethertype IPv4 (0x0800), length 89: (tos 0x0, ttl 16, id 30380, offset 0, flags [none], proto UDP (17), length 75)
10.0.128.63.15791 > 10.168.0.1.69: [udp sum ok] 47 RRQ "macnbi-i386/booter" octet blksize 512 tsize 0
It looks like you're right; It does receive multiple replys, but I'm not sure if that's the reason why it's not selecting one over the other.
Here's a tcpdump of a successful netboot attempt:
tcpdump: listening on en0, link-type EN10MB (Ethernet), capture size 65535 bytes
15:31:26.287342 34:15:xx:xx:xx:xx > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 590: (tos 0x0, ttl 16, id 44354, offset 0, flags [none], proto UDP (17), length 576)
0.0.0.0.68 > 255.255.255.255.67: [udp sum ok] BOOTP/DHCP, Request from 34:15:xx:xx:xx:xx, length 548, xid 0x32cc, secs 5, Flags [none] (0x0000)
Client-Ethernet-Address 34:15:xx:xx:xx:xx
Vendor-rfc1048 Extensions
Magic Cookie 0x63825363
DHCP-Message Option 53, length 1: Discover
Parameter-Request Option 55, length 5:
Subnet-Mask, Default-Gateway, BF, Vendor-Option
Vendor-Class
Client-ID Option 61, length 7: ether 34:15:xx:xx:xx:xx
Vendor-Class Option 60, length 28: "AAPLBSDPC/i386/MacBookPro5,3"
Vendor-Option Option 43, length 4: 2.2.1.1
END Option 255, length 0
PAD Option 0, length 0, occurs 252
15:31:26.289057 00:24:xx:xx:xx:xx > 34:15:xx:xx:xx:xx, ethertype 802.1Q (0x8100), length 64: vlan 1, p 0, ethertype IPv4, (tos 0x0, ttl 127, id 1530, offset 0, flags [none], proto ICMP (1), length 39)
10.0.178.10 > 10.0.128.63: ICMP echo request, id 512, seq 22420, length 19
15:31:26.624305 00:1e:xx:xx:xx:xx > 34:15:xx:xx:xx:xx, ethertype 802.1Q (0x8100), length 416: vlan 1, p 0, ethertype IPv4, (tos 0x10, ttl 128, id 0, offset 0, flags [none], proto UDP (17), length 398)
10.168.0.1.67 > 10.168.0.11.68: [udp sum ok] BOOTP/DHCP, Reply, length 370, xid 0x32cc, secs 5, Flags [none] (0x0000)
Your-IP 10.168.0.11
Server-IP 10.168.0.1
Client-Ethernet-Address 34:15:xx:xx:xx:xx
sname "10.168.0.1"
file "macnbi-i386/booter"
Vendor-rfc1048 Extensions
Magic Cookie 0x63825363
DHCP-Message Option 53, length 1: Offer
Server-ID Option 54, length 4: 10.168.0.1
Lease-Time Option 51, length 4: 86400
Subnet-Mask Option 1, length 4: 255.255.0.0
Default-Gateway Option 3, length 4: 10.168.0.1
RP Option 17, length 76: "http://10.0.128.1/Netboot/NetBootSP0/NetRestore.nbi/NetInstall-Restore.dmg"
Vendor-Option Option 43, length 6: 8.4.129.0.0.103
Vendor-Class Option 60, length 14: "AAPLBSDPC/i386"
END Option 255, length 0
15:31:27.301638 00:24:xx:xx:xx:xx > 34:15:xx:xx:xx:xx, ethertype 802.1Q (0x8100), length 64: vlan 1, p 0, ethertype IPv4, (tos 0x0, ttl 127, id 1532, offset 0, flags [none], proto ICMP (1), length 39)
10.0.178.10 > 10.0.128.63: ICMP echo request, id 512, seq 22676, length 19
15:31:28.387589 34:15:xx:xx:xx:xx > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 590: (tos 0x0, ttl 16, id 29575, offset 0, flags [none], proto UDP (17), length 576)
0.0.0.0.68 > 255.255.255.255.67: [udp sum ok] BOOTP/DHCP, Request from 34:15:xx:xx:xx:xx, length 548, xid 0x32cc, secs 5, Flags [none] (0x0000)
Client-Ethernet-Address 34:15:xx:xx:xx:xx
Vendor-rfc1048 Extensions
Magic Cookie 0x63825363
DHCP-Message Option 53, length 1: Request
Parameter-Request Option 55, length 5:
Subnet-Mask, Default-Gateway, BF, Vendor-Option
Vendor-Class
Client-ID Option 61, length 7: ether 34:15:xx:xx:xx:xx
Vendor-Class Option 60, length 28: "AAPLBSDPC/i386/MacBookPro5,3"
Requested-IP Option 50, length 4: 10.168.0.11
Server-ID Option 54, length 4: 10.168.0.1
Vendor-Option Option 43, length 4: 2.2.1.1
END Option 255, length 0
PAD Option 0, length 0, occurs 240
15:31:28.802414 00:24:xx:xx:xx:xx > 34:15:xx:xx:xx:xx, ethertype 802.1Q (0x8100), length 346: vlan 1, p 0, ethertype IPv4, (tos 0x0, ttl 255, id 19737, offset 0, flags [none], proto UDP (17), length 328)
10.0.129.254.67 > 10.0.128.63.68: [udp sum ok] BOOTP/DHCP, Reply, length 300, xid 0x32cc, Flags [none] (0x0000)
Your-IP 10.0.128.63
Server-IP 10.0.178.10
Gateway-IP 10.0.129.254
Client-Ethernet-Address 34:15:xx:xx:xx:xx
Vendor-rfc1048 Extensions
Magic Cookie 0x63825363
DHCP-Message Option 53, length 1: Offer
Subnet-Mask Option 1, length 4: 255.255.254.0
RN Option 58, length 4: 1296000
RB Option 59, length 4: 2268000
Lease-Time Option 51, length 4: 2592000
Server-ID Option 54, length 4: 10.0.178.10
Default-Gateway Option 3, length 4: 10.0.129.254
END Option 255, length 0
PAD Option 0, length 0, occurs 20
15:31:28.899055 00:1e:xx:xx:xx:xx > 34:15:xx:xx:xx:xx, ethertype 802.1Q (0x8100), length 392: vlan 1, p 0, ethertype IPv4, (tos 0x10, ttl 128, id 0, offset 0, flags [none], proto UDP (17), length 374)
10.168.0.1.67 > 10.168.0.11.68: [udp sum ok] BOOTP/DHCP, Reply, length 346, xid 0x32cc, secs 5, Flags [none] (0x0000)
Your-IP 10.168.0.11
Server-IP 10.168.0.1
Client-Ethernet-Address 34:15:xx:xx:xx:xx
sname "10.168.0.1"
file "macnbi-i386/booter"
Vendor-rfc1048 Extensions
Magic Cookie 0x63825363
DHCP-Message Option 53, length 1: ACK
Server-ID Option 54, length 4: 10.168.0.1
Lease-Time Option 51, length 4: 86400
Subnet-Mask Option 1, length 4: 255.255.0.0
Default-Gateway Option 3, length 4: 10.168.0.1
RP Option 17, length 76: "http://10.0.128.1/Netboot/NetBootSP0/NetRestore.nbi/NetInstall-Restore.dmg"
END Option 255, length 0
15:31:28.899058 34:15:xx:xx:xx:xx > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 60: Ethernet (len 6), IPv4 (len 4), Request who-has 10.168.0.11 tell 0.0.0.0, length 46
15:31:29.398941 34:15:xx:xx:xx:xx > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 60: Ethernet (len 6), IPv4 (len 4), Request who-has 10.168.0.11 tell 10.168.0.11, length 46
15:31:29.899254 34:15:xx:xx:xx:xx > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 60: Ethernet (len 6), IPv4 (len 4), Request who-has 10.168.0.1 tell 10.168.0.11, length 46
15:31:29.899257 00:1e:xx:xx:xx:xx > 34:15:xx:xx:xx:xx, ethertype 802.1Q (0x8100), length 64: vlan 1, p 0, ethertype ARP, Ethernet (len 6), IPv4 (len 4), Reply 10.168.0.1 is-at 00:1e:xx:xx:xx:xx, length 46
15:31:29.899259 34:15:xx:xx:xx:xx > 00:1e:xx:xx:xx:xx, ethertype IPv4 (0x0800), length 89: (tos 0x0, ttl 16, id 38655, offset 0, flags [none], proto UDP (17), length 75)
10.168.0.11.17638 > 10.168.0.1.69: [udp sum ok] 47 RRQ "macnbi-i386/booter" octet blksize 512 tsize 0
15:31:29.899924 00:1e:xx:xx:xx:xx > 34:15:xx:xx:xx:xx, ethertype 802.1Q (0x8100), length 73: vlan 1, p 0, ethertype IPv4, (tos 0x0, ttl 64, id 25574, offset 0, flags [DF], proto UDP (17), length 55)
10.168.0.1.43349 > 10.168.0.11.17638: [udp sum ok] UDP, length 27
15:31:29.900216 34:15:xx:xx:xx:xx > 00:1e:xx:xx:xx:xx, ethertype IPv4 (0x0800), length 60: (tos 0x0, ttl 16, id 59278, offset 0, flags [none], proto UDP (17), length 33)
10.168.0.11.17638 > 10.168.0.1.43349: [udp sum ok] UDP, length 5
15:31:34.900598 00:1e:xx:xx:xx:xx > 34:15:xx:xx:xx:xx, ethertype 802.1Q (0x8100), length 64: vlan 1, p 0, ethertype ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 10.168.0.11 tell 10.168.0.1, length 46
15:31:35.900833 00:1e:xx:xx:xx:xx > 34:15:xx:xx:xx:xx, ethertype 802.1Q (0x8100), length 64: vlan 1, p 0, ethertype ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 10.168.0.11 tell 10.168.0.1, length 46
15:31:36.901071 00:1e:xx:xx:xx:xx > 34:15:xx:xx:xx:xx, ethertype 802.1Q (0x8100), length 64: vlan 1, p 0, ethertype ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 10.168.0.11 tell 10.168.0.1, length 46
What are your thoughts upon seeing this log?
Edit / Update:
I've just added more options so my Local DHCP Option Count is 11 vs the Global DHCP Option Count of 9. And it still won't grab the IP address from my local DHCP server. Not sure what I can do, I shouldn't need to release the IP every time I need to NetBoot.
So it looks as if it will take the first DHCP offer, is there anyway to make sure that the local DHCP server responds first?
Do you have some insight you could share with us?
These look like logs from your local DHCP server, is that correct? If so, then it would be interesting to see an actual packet trace* to see what's really going on. It may be that the client isn't "storing" the lease at all; it may have gotten an Offer from the global DHCP server, and selected that.
I know Mac OS X's DHCP client, when presented with multiple DHCP Offers from multiple servers, tends to choose the one that has more DHCP Options defined in it. (This is generally a good heuristic for selecting a site's "real" DHCP server instead of being fooled by an accidental rogue DHCP server on the same network -- when someone accidentally enables DHCP service on a box, they generally haven't taken the time to configure it to include a bunch of DHCP Options.) HOWEVER, I don't know what the DHCP client in the Mac's EFI bootrom does when presented multiple offers. I doubt the EFI DHCP client (the DHCP client in ROM) is all that similar to the Mac OS X DHCP client (the DHCP client on disk).
If it turns out that the EFI DHCP client is choosing based on number of DHCP options, you could possibly work around the issue by padding your DHCP server with a bunch of options, like telling the clients what WINS server and NTP server and LDAP server and whatever else to use. You could probably browse the list of registered DHCP options and configure your DHCP server to provide a bunch that seem like they can't hurt.
If it really turns out the client is storing the DHCP lease in NVRAM, you should be able to could clear NVRAM by holding down Cmd-Opt-P-R at boot until you hear a second boot chime. (That's the old "Zap the PRAM" key combo that used to be occasionally necessary on 80's and 90's Macs, but nowadays is almost never useful, but still gets recommended way too often. In this case, however, you have a reasonable expectation that something stored in NVRAM is causing a problem for you, so this might be one of the rare times when it's reasonable.)
*For the packet trace, I'd recommend doing something like this on an independent machine hooked to a switch port that's configured to do port mirroring of the netboot client's switch port:
Or maybe:
(where '$MACOfEn0' is the MAC address of the netboot client's en0, or whatever interface you're doing this netboot attempt over)
Can you put these MAC's into their own network with a router to route the 10.168 addresses through to the Global network? The router would then restrict the passing of DHCP requests.
Option 2: Put the MAC's into their own VLAN with the DHCP server.
Use a DHCP server that will not answer for certain configured mac addresses. If there is such a thing. If it's not already there you could probably patch the linux dhcp server to support this feature. It wouldn't be that difficult to add for an average programmer.