Kevin Keane's questions

I am using firewalld on RHEL 8, and need to add a few nftable rules as well.

(The nftable rules are based on the answer to CentOS 8 as NAT router with nft and firewalld - how to get it to pass TFTP?)

In a running firewall, this works well with the nft -f command.

However, this gets lost on a reboot.

The RedHat documentation (behind paywall) suggests using the nftables.service service to load rules on reboot, but this does not work in conjunction with firewalld. The two services are listed as conflicting, and even if they weren't, firewalld would likely flush the nftable rules.

Is there another way to get the nftable rules to load on restart?

I am trying to set up PXE booting (which requires TFTP) on one of my networking that is hiding behind a NAT router.

My question is similar to many others around the 'Net, but all the answers I found applied to CentOS 7 with iptables. I need to do this with CentOS 8 with firewalld and nft as the backend.

Unable to NAT TFTP traffic because iptables is not forwarding the return connection to the client despite TFTP helper creating an expectation https://unix.stackexchange.com/questions/579508/iptables-rules-to-forward-tftp-via-nat

Here is my simplified network diagram:

     Outside NAT               Inside NAT
10.0.10.10  10.0.10.11->192.168.1.1  192.168.1.2
TFTP server --------> NAT ---------> PXE/TFTP client

TFTP is not working. With tcpdump, I see that the RRQ message travels successfully from 192.168.1.2 to 10.0.10.10. The response arrives at the router, but is not properly NATed to arrive at the client.

I tried it with both settings for sysctl net.netfilter.nf_contrack_helper (rebooted after changing the setting):

# sysctl -a | grep conntrack_helper
net.netfilter.nf_conntrack_helper = 0

With nf_contrack_helper=0:

tcpdump: listening on any, link-type LINUX_SLL (Linux cooked), capture size 262144 bytes

Initial RRQ:

14:02:27.842563 IP (tos 0x0, ttl 64, id 64642, offset 0, flags [DF], proto UDP (17), length 54)
    192.168.1.2.36799 > 10.0.10.10.69: [udp sum ok]  26 RRQ "grub2/grubx64.efi" octet

Initial RRQ after NAT:

14:02:27.842619 IP (tos 0x0, ttl 63, id 64642, offset 0, flags [DF], proto UDP (17), length 54)
    10.0.10.11.36799 > 10.0.10.10.69: [udp sum ok]  26 RRQ "grub2/grubx64.efi" octet

Response from TFTP server to NAT router:

14:02:27.857924 IP (tos 0x0, ttl 63, id 60000, offset 0, flags [none], proto UDP (17), length 544)
    10.0.10.10.60702 > 10.0.10.11.36799: [udp sum ok] UDP, length 516

(repeated several times until timeout)

With nf_contrack_helper=1, the outgoing packet is not even NATed at all:

tcpdump: listening on any, link-type LINUX_SLL (Linux cooked), capture size 262144 bytes

Initial RRQ:

14:02:27.842563 IP (tos 0x0, ttl 64, id 64642, offset 0, flags [DF], proto UDP (17), length 54)
    192.168.1.2.36799 > 10.0.10.10.69: [udp sum ok]  26 RRQ "grub2/grubx64.efi" octet

(repeated several times until timeout)

The nf_*_tftp helpers are both loaded (regardless of the nf_contrack_helper setting):

# lsmod | grep tftp
nf_nat_tftp            16384  0
nf_conntrack_tftp      16384  3 nf_nat_tftp
nf_nat                 36864  3 nf_nat_ipv6,nf_nat_ipv4,nf_nat_tftp
nf_conntrack          155648  10 nf_conntrack_ipv6,nf_conntrack_ipv4,nf_nat,nf_conntrack_tftp,nft_ct,nf_nat_ipv6,nf_nat_ipv4,nf_nat_tftp,nft_masq,nft_masq_ipv4

One of the article linked above suggests the following using iptables (which makes sense):

iptables -A PREROUTING -t raw -p udp --dport 69 -s 192.168.11.0/24 -d 172.16.0.0/16 -j CT --helper tftp

How would I do the equivalent with firewalld with an nft backend.

Update:

The firewalld configuration is fairly complex, so I'm only adding the relevant zones:

The external zone:

<?xml version="1.0" encoding="utf-8"?>
<zone>
  <source address="10.0.10.0/24"/>
  <service name="tftp-client"/>
  <service name="ssh"/>
  <masquerade/>
</zone>

And the internal zone:

<?xml version="1.0" encoding="utf-8"?>
<zone>
  <source address="192.168.1.0/24"/>
  <service name="dhcp"/>
  <service name="ssh"/>
  <service name="dns"/>
  <service name="tftp"/>
  <masquerade/>
</zone>

Note: the Masquerade on the internal zone is a mistake. I removed it, but the behavior is not changing.

Zone drifting is disabled.

Update 2:

To answer a request from a commenter:

DHCP configuration

The DHCP server is running on the same system as the NAT router (192.168.1.1 in the network diagram). It is standard ISC DHCP, handing out IP addresses (as fixed-address; there is no pool involved), mask, gateway, DNS server, etc., as well as the PXE Boot next-server and filename options.

All this obviously works. tcpdump shows that the client sends the correct RRQ packet to the server.

The response arrives back at the NAT router, but does not get sent to the behind-the-NAT side.

Details about how TFTP works and how it breaks with NAT

If you understand the TFTP protocol, it is fairly clear what is happening; I just do not know how to handle it with firewalld/nft/CentOS 8.

Fundamentally, the problem is that the TFTP protocol uses UDP ports in a non-standard way. In "standard" UDP-based protocols such as DNS, the response comes from the same port that the server listens on.

Request: client:54321 -> server:53
Response: server:53 -> client:54321

(where 54321 can be any random ephemeral port number picked by the client).

NAT matches up those IP addresses and ports to identify which response belongs to which request.

TFTP does it differently; the responses do not come from port 69, but some other random port.

Request (RRQ): client54321 -> server:69
Response (Data): server:12345 -> client:54321

Where 54321 is again a random ephemeral port the client chooses, and 12345 is a random ephemeral port the server chooses.

As a result, standard NAT behavior will not find a connection matching an origin server:12345, and drops the packet.

The solution to this problem involves using a helper - the nf_nat_tftp kernel module that understands this quirk.

I just have not been able to figure out how to implement this using CentOS 8, nftables and firewalld.

An answer that uses nftables is perfectly acceptable for me, as long as it does not break any firewalld rules.

In an Ansible role, I'm looking for a way to copy a remote file to a different location, while also replacing the first two lines of the file.

I'm open to other approaches, but my solution involves using slurp to retrieve the file, somehow convert it into a list of individual lines, and then write it back using a template.

I'm stuck at the step of splitting the string returned by slurp into lines.

Ansible 2.9, both the controller and the remote hosts are running RHEL 7.8.

The input file is already on the remote host as /etc/sample/inputfile.txt

Line1 something
Line2 something
Line3 stays untouched
Line4 stays untouched

Desired output in /etc/sample/outputfile.txt

Line1 has changed
Line2 has also changed
Line3 stays untouched
Line4 stays untouched

The effect I want to replicate is what the following sequence would produce, but idempotently without making three changes to outputfile.txt on every run.

- copy:
    src: /etc/sample/inputfile.txt
    dest: /etc/sample/outputfile.txt
    remote_src: True

- lineinfile:
    path: /etc/sample/outputfile.txt
    regexp: '^Line1'
    line: 'Line1 has changed'

- lineinfile:
    path: /etc/sample/outputfile.txt
    regexp: '^Line2'
    line: 'Line2 has also changed'

To make this idempotent, my current idea is to use slurp to retrieve inputfile.txt, then (somehow) strip the first two lines of that variable, and create outputfile.txt using a template.

- slurp:
    src: /etc/sample/inputfile.txt
  register: r_input
- set_fact:
    intermediate_var: '{{ r_input.content | b64decode }}'
- ??? How would I convert intermediate_var into a list of lines?
- template:
  vars:
    line1: 'Line1 has changed'
    line2: 'Line2 has also changed'
    body:  '??? intermediate_var without the first two lines'

However, I haven't figured out a way to strip the first two lines off the slurped variable. I'm aware of the lookup plugin "lines" but have not figured out how to apply it in this scenario.

Using RedHat Satellite Server 6.7, is there a way to provision bare-metal CentOS systems?

Satellite Server generally supports CentOS for updates.

Satellite Server also supports a very simple method to provision new RedHat Enterprise Linux systems using PXE (or a boot ISO image) and auto-generated Kickstart files. I use it frequently.

But I have not been able to find a way for Satellite Server to provision CentOS the same way.

Is there a way to get this to work?

I am looking for a way to modify a configuration file while preserving local modifications. The format of the configuration file is similar to this:

entry1: This is a local
entry2: modification.
entry3:

The file consists of a variable number of keys (entry1, entry2, entry3, and possibly later entry4, etc. - it could go up to 100, and in the future maybe even up to 2000 or so) that should be set up by Ansible, followed by additional options that another program will add to the configuration file.

I want Ansible to be able to add new keys to the file, while preserving any of the existing local modifications.

The most natural fit seems lineinfile - but unfortunately it only gives me the option of either preserving the local modifications (using backref=yes), or adding new keys (backref=yes will not add new lines).

Is there a good way to accomplish what I need?

I can modify the existing entries with lineinfile and backref enabled:

- lineinfile:
  path: myfile.conf
  regexp: "^{{ item }}: (.*)"
  backref: yes
  line: "{{ item }}: \1"
  with_items:
    - entry1
    - entry2
    - entry3
    - entry4

But this will not add entry4 to my file.

Or I can use backref: no

- lineinfile:
  path: myfile.conf
  regexp: "^{{ item }}: (.*)"
  backref: no
  line: "{{ item }}:"
  with_items:
    - entry1
    - entry2
    - entry3
    - entry4

But this will clobber local modifications for entry1, entry2 and entry3.

Or I could change the regexp:

- lineinfile:
  path: myfile.conf
  regexp: "^"
  backref: no
  line: "{{ item }}:"
  with_items:
    - entry1
    - entry2
    - entry3
    - entry4

But this will of course add each key on every run.

I've also looked at using a template (but have not found an easy way to use it to manipulate an existing file).

Of course I can write my own Python module, but there must be an easier way to do this?

In IPv6, you are not supposed to subnet to anything smaller than a /64 (RFC 5375). Among other things, SLAAC does not work with smaller subnets, and apparently also some other features will break.

What are the workarounds for situations where ISPs will only give you a single /64 but you need multiple subnets internally? The common advice seems to be to just find another ISP who will hand out a /56 or /48. In some parts of the world, that may work, but in our area (USA), that's not feasible due to a lack of competition. Most of my clients are lucky if they have a single ISP serving their area. Many people here are still on dial-up.

My clients won't qualify for their own /48 from ARIN.