When does iptable's conntrack module track states of packets?

Introductory presentation of Netfilter and conntrack

First the mandatory schematic about Packet flow in Netfilter and General Networking:

Netfilter is the packet filtering framework inserting itself over the rest of the network stack (represented by "routing decision" and other white round-edged box parts). Netfilter provides hooks and APIs for other subsystems and "clients". Among these parts are conntrack (the connection tracker) and iptables (or nftables). The separation between Netfilter and conntrack is quite fuzzy. You can just consider conntrack as an integrated part of Netfilter.

In the schematic describing the various steps a packet traverses you can see that at some point (between raw/PREROUTING and mangle/PREROUTING, or between raw/OUTPUT and mangle/OUTPUT) the packet traverses conntrack.

At this point, conntrack will search in its own lookup tables (a mini lookup database kept in kernel memory):

• if this packet's charateristics are not found (and if not declared UNTRACKED in the raw table), a new conntrack bidirectional tuple entry (protocol, then specific family and protocol informations: initial source and port, initial destination and port, reply source and port, reply destination and port (those two last are usually the reverse, unless NAT or some strange protocols are involved, like echo reply matching echo request for ICMP)) describing the flow is created with state NEW.
• if it matches (in any direction) a previous entry and is compatible with this flow's state, the flow state might be altered (eg: changing from NEW to ESTABLISHED if that wasn't the case before).
• if for some specific reason the packet can't match an existing flow despite it having its characteristics (eg: a late TCP packet received after a retransmissions already kicked in successfully, so being out of window with regard to the sequence and SACK values) the packet will be tagged INVALID.
• there are a few other cases like RELATED: this is about packets not part of the flow itself but related to a new flow that can be associated with an other existing (ie: in the database) flow. Two examples are an ICMP error created from receiving a packet (eg: UDP port unreachable) or when a special protocol helper like the kernel module nf_conntrack_ftp , which is a plugin to the conntrack subsystem, detects that a packet is part of the separate data flow associated with the FTP commands PASV/EPSV or PORT/EPRT done on the command flow (on port 21).

Addressing the question

All this being said, here are the answers to your two bullets:

• in the main network namespace conntrack starts tracking connections as soon as its modules (including possible relevant protocol-specific sub-modules) are loaded. For non-initial network namespaces (containers...) this also requires that some other subsystem references it (such as OP's iptables's conntrack module or using once the command conntrack described later). This is the default and a packet must be specifically marked as UNTRACKED before the conntrack subsystem sees it for this packet to not be tracked. On Linux there are only a few cases where not tracking would be needed, but then of course stateful firewalling and stateful/dynamic NAT won't be available anymore (steless NAT which might even require to use UNTRACKED in the first place, can still be done, but not with iptables. tc or nftables can). To avoid conntrack handling some packets, this kind of iptables rule can be used (eg: port 80/tcp):

  iptables -t raw -A PREROUTING -p tcp --dport 80 -j CT --notrack
  iptables -t raw -A OUTPUT -p tcp --sport 80 -j CT --notrack
  

• When the packet traverses filter/INPUT and reaches this rule:

   iptables -A INPUT -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
  

  The iptables's specific kernel module xt_conntrack queries the conntrack subsystem (handled by the various relevant kernel modules nf_conntrack*) and asks about the state of this packet in its lookup database. If the answer is RELATED or ESTABLISHED the packet matches and proceeds to the ACCEPT verdict. Actually the result is already cached in the packet the first time the lookup was done (usually by conntrack) so this is a cheap "lookup". This is thus a generic rule to handle flows already accepted before. Those flows can be initially accepted in rules explicitly mentioning -m conntrack --ctstate NEW or simply rules not mentioning it but placed after this generic rule (but keep in mind the INVALID state, which should usually be DROPed before doing so).

• adding a bullet: the handling of incoming packets and outgoing packets is quite symmetrical between PREROUTING and OUTPUT (even if those don't look symmetrical): conntrack interfaces in PREROUTING as well as in OUTPUT (and in a few other places, considering NAT is working with conntrack, except for its first packet in state NEW traversing iptables's nat table). This might be slightly different from the description you wrote about IPFW. If a server running applications is also restricting outgoing flows, then it most likely needs this same generic iptables rule both in filter/OUTPUT and in filter/INPUT, to allow outgoing reply packets of already accepted incoming traffic to pass.

Additional informations

There are dedicated tools to interact with the conntrack subsystem's lookup tables from conntrack-tools.

• conntrack: to query, delete or update the contents of the lookup tables handled by conntrack.

  Some examples.

  You can list all tracked entries (which can be big without additional filter) with:

  conntrack -L
  

  If your system is doing NAT (eg a router in front of a private LAN, or running VMs and containers) you can use --any-nat, --src-nat or --dst-nat to only display resp. all NAT, all source NAT (masquerade), or all destination NAT (typically for forwarded ports):

  Real-time monitoring of conntrack events:

  conntrack -E
  

• conntrackd: a daemon whose two main purposes are (conntrack) flow accounting and statistics, or high-availability stateful firewall cluster state synchronization.

Connection tracking is a separate function of Netfilter, and it is not configured with IPTables.

In the picture, there are two conntrack steps in INPUT path and one in OUTPUT path. These steps associate individual packets with existing connections tracked in the connection tracking table, or create new connection tracking entries in the table.

Conntrack functionality is a Linux kernel module, and it is often included in the kernel in default configuration.

Conntrack operation can be tuned by adjusting net.netfilter.nf_conntrack sysctl values.

Your second alternative is what happens. The state information is recorded by the Conntrack function, and the IPTables rule simply consults the Conntrack table for information.