Home / server / Questions / 162558
Accepted
parkamark
Asked: 2010-07-22 01:29:44 +0800 CST

Inconsistencies with Linux tc filter u32 matching on packet data payload - can someone explain?

  • 772

I would just like some heads up on this as I really don't understand -why- this is the case.

Further down is the TCPDUMP output (tcpdump -s0 -XXnni eth0 tcp port 80) of the 'HTTP' response a webserver gives when a normal GET request is made to it. What I am trying to do is use Linux u32 tc filter to match on the contents of the TCP ack packet, looking for the string 'HTTP/1.[01] 200' in the data payload of the TCP ack packet (in otherwords, looking for a typical 'HTTP/1.0 200 OK' response or, 'HTTP/1.1 200 OK' response.

Here's a snippet of the tc filter command - this will probably help put things in context:

tc filter add dev eth0 parent ffff: protocol ip u32 \
  match ip protocol 6 0xff \
  match ip sport 80 0xffff \
  match u8 0x10 0xff at 33 \
  match u32 0x48545450 0xffffffff at 52 \
  match u32 0x2f312e31 0xfffffffe at 56 \
  match u32 0x20323030 0xffffffff at 60 \
<do something>

The last 3 'match u32' lines will match 'HTTP/1.0 200' or 'HTTP/1.1 200' while the u8 one matches the TCP ack flag, and the others match source port 80 protocol TCP.

My query is - why, on two different Linux boxes, do I have to change the numbers 52, 56 and 60 to 40, 44 and 48 ? (subtract 12 from the offset). It seams on my Slackware Linux box at home, I have to use 52, 56 and 60, where as on a RedHat/CentOS server, I have to use 40, 44 and 48.

The reason for this is simple; compare these two TCPDUMPs from each server:

Slackware: 0x0000:  0040 63c9 c3a0 0018 7d05 dd11 0800 4500  .@c.....}.....E.
           0x0010:  05be d41c 0000 3606 9ea2 4266 0963 c0a8  ......6...Bf.c..
           0x0020:  000a 0050 a278 e948 dcdb fa41 ac84 8010  ...P.x.H...A....
           0x0030:  0059 3cb2 0000 0101 080a 9380 9172 0008  .Y<..........r..
           0x0040:  3bea 4854 5450 2f31 2e31 2032 3030 204f  ;.HTTP/1.1.200.O

RedHat or: 0x0000:  0016 3e32 3fcf 0010 dbff 2050 0800 4500  ..>2?......P..E.
CentOs:    0x0010:  0554 cf64 0000 3706 e08b 4266 0969 0a64  .T.d..7...Bf.i.d
           0x0020:  7881 0050 b316 c917 2062 b4a8 cff4 5018  x..P.....b....P.
           0x0030:  005a a17f 0000 4854 5450 2f31 2e31 2034  .Z....HTTP/1.1.4
           0x0040:  3034 204e 6f74 2046 6f75 6e64 0d0a 436f  04.Not.Found..Co

As you can see, the offset point where the 'HTTP' part starts (the packet data payload) is different in both cases. Why is this? And what would cause it?

Thanks in advance to anyone who can explain this mystery to me.

linux networking pcap tc filter

2 Answers

  1. Best Answer

    Found the answer. Having TCP timestamps ON introduces 12 bytes extra header information, resulting in the offset difference.

    You can toggle them on/off in Linux by doing this:

    echo 0 > /proc/sys/net/ipv4/tcp_timestamps
echo 1 > /proc/sys/net/ipv4/tcp_timestamps

    Also see if they are on/off by doing:

    cat /proc/sys/net/ipv4/tcp_timestamps
    • 0

  2. I found this may solve the problem.

    http://linux-tc-notes.sourceforge.net/tc/doc/cls_u32.txt

    Header Offsets

    The IP header (and other headers) are variable length. This creates a problem if you are trying to use "match" to look at a value in a header that follows - you don't know where it is. It is not an impossible problem because every header in an IP packet contains a length field. The "header offsets" feature of u32 allows you to extract that length from the packet, and then add it to the offset specified in the "match" option.

    Here is how it works. Recall that the match option looks like this:

    match u32 VALUE MASK at OFFSET

    I said earlier that OFFSET tells the kernel which word in the packet to compare to VALUE. That statement was a simplification. Two other values can be added to OFFSET to determine which word to use. Both those values start off as 0, but they can be modified when a "link" option calls another filter list. Any modification made only applies while called filter list is being executed as the old values are restored if the called filter list fails to classify the packet. Here are the two values and the names I call them:

    permoff     This value is unconditionally added to every OFFSET
        that is done in the destination link, ie that one
    that is called.  This includes calculations of new
    permoff's and tempoff's.  Permoff's are cumulative
    in that if the destination link calls another link
    and calculates a new permoff, the result is added to
    this one.

tempoff A "match" option in the destination link can
        optionally add this value its OFFSET.  Tempoff's are
    temporary, in that it does not apply to any links the
    destination link calls.  It also does not effect the
    calculation of OFFSET's for new permoff's and
    tempoff's.

    Time for an example. Consider this command:

    # tc filter add dev eth0 parent 999:0 protocol ip prio 99 u32 \
    link 1: offset at 0 mask 0f00 shift 6 plus 0 eat \
match ip protocol 6 ff

    The match expression selects tcp packets (which is IP protocol 6). If we have protocol 6 we execute filter 1:0. Now for the rest of it:

    offset  This signals that we want to modify permoff or tempoff
        if the link is executed.  If this is not present,
    neither permoff nor tempoff are effected - in other
    words the target of the link inherits the current
    permoff and tempoff.

at 0    This says the 16 bit word that contains the value we
        are going to use to calculate permoff or tempoff lives
    offset 0 the IP packet - ie at the start of the packet.
    This offset must be even.  If not specified 0 is used.

mask 0f00   This mask (which is in hex) is bit-wise anded with the
        16 bit word extracted from the packet header.  It
    isolates the header length from the rest of the
    information in the word.  If not specified 0 is used
    for the extracted value.

shift 6 This says the word extracted is to be divided by 64
        after being masked.  If not present the value is not
    shifted.

plus 0  After extracting the word, masking it and dividing it by
        64, this value is now added to it.  If not present is
    assumed to be 0.

eat     If this is present we are calculating permoff, and the
        result of the calculation above is added to it.  Tempoff
    is set to 0 in this case.  If this is not present we are
    calculating tempoff, and the result of the calculation
    becomes tempoff's new value.  Permoff is not altered in
    this case.
    • 0