I am using Traffic Flow with pmacct (nfacct) to do IP Accounting.

I've noticed that if a flow exceeds ~4GBytes in less than a minute (which is my active-flow-timeout) the exported flow Octets counter wraps around losing a significant amount of total data measured.

I believe this is an issue with the Octet counter being 32bit unsigned and if the traffic is over that threshold (4294967296) then the exporter wraps around the counter without first sending the flow to the collector (I am not sure how other vendors handle this).

This is quite serious since it results in very wrong traffic totals!

Here is my traffic flow configuration:

/ip traffic-flow
set active-flow-timeout=1m cache-entries=1k enabled=yes interfaces=sfp1
/ip traffic-flow target
add dst-address=X.X.X.X v9-template-refresh=60 v9-template-timeout=1m

And here are a couple of flow captures from wireshark.

Flow 3
    [Duration: 59.590000000 seconds (switched)]
    Packets: 5700194
    Octets: 4255323704
    InputInt: 16
    OutputInt: 0
    SrcAddr: 31.X.X.254
    DstAddr: 185.X.X.254
    Protocol: UDP (17)
    IP ToS: 0x00
    SrcPort: 2043 (2043)
    DstPort: 2299 (2299)
    NextHop: 185.X.X.X
    DstMask: 0
    SrcMask: 0
    TCP Flags: 0x00
    Destination Mac Address: Routerbo_XX:XX:XX (d4:ca:6d:XX:XX:XX)
    Post Source Mac Address: 00:00:00_00:00:00 (00:00:00:00:00:00)
    Post NAT Source IPv4 Address: 31.X.X.254
    Post NAT Destination IPv4 Address: 185.X.X.254
    Post NAPT Source Transport Port: 0
    Post NAPT Destination Transport Port: 0

Second capture:

Flow 3
    [Duration: 59.590000000 seconds (switched)]
    Packets: 5532208
    Octets: 4003344704
    InputInt: 16
    OutputInt: 0
    SrcAddr: 31.X.X.254
    DstAddr: 185.X.X.254
    Protocol: UDP (17)
    IP ToS: 0x00
    SrcPort: 2043 (2043)
    DstPort: 2299 (2299)
    NextHop: 185.X.X.X
    DstMask: 0
    SrcMask: 0
    TCP Flags: 0x00
    Destination Mac Address: Routerbo_XX:XX:XX (d4:ca:6d:XX:XX:XX)
    Post Source Mac Address: 00:00:00_00:00:00 (00:00:00:00:00:00)
    Post NAT Source IPv4 Address: 31.X.X.254
    Post NAT Destination IPv4 Address: 185.X.X.254
    Post NAPT Source Transport Port: 0
    Post NAPT Destination Transport Port: 0

At the time of those captures, a bandwidth test (UDP, 1500bytes, 1Gbit, receive) was running for quite some time. So running at 1gbit for 60seconds (active-flow-timeout) it should have measured at least ~7864320000 Octets (~7.3GB)

If I reduce the bandwidth test to 460mbit then the exported flows seem to report the traffic properly since the Octets counter does not exceed the 32bit unsigned maximum. Though I see quite a lot of overhead and I wonder why that is. At 460mbit sustained traffic, in 60seconds it should measure ~3617587200 octets (=3.36GB). But instead it measured 4269160500 (=3.9GB) I am not sure where the extra ~600MB came from.

Flow 6
    [Duration: 59.590000000 seconds (switched)]
    Packets: 2846107
    Octets: 4269160500
    InputInt: 16
    OutputInt: 0
    SrcAddr: 31.X.X.254
    DstAddr: 185.X.X.254
    Protocol: UDP (17)
    IP ToS: 0x00
    SrcPort: 2058 (2058)
    DstPort: 2314 (2314)
    NextHop: 185.X.X.X
    DstMask: 0
    SrcMask: 0
    TCP Flags: 0x00
    Destination Mac Address: Routerbo_0d:95:72 (d4:ca:6d:XX:XX:XX)
    Post Source Mac Address: 00:00:00_00:00:00 (00:00:00:00:00:00)
    Post NAT Source IPv4 Address: 31.X.X.254
    Post NAT Destination IPv4 Address: 185.X.X.254
    Post NAPT Source Transport Port: 0
    Post NAPT Destination Transport Port: 0

But if I increase the bandwidth test to 480mbit for example, then the exported flow has its counter wrapped around losing a significant amount of data (ie: ~4GBytes of data)

Flow 3
    [Duration: 59.590000000 seconds (switched)]
    Packets: 2865308
    Octets: 2994704 <-- Only 2.8MB?! Even with 64byte packets,
                    based on the measured packets above, 
                    it should have measured > 174MBytes of data!
    InputInt: 16
    OutputInt: 0
    SrcAddr: 31.X.X.254
    DstAddr: 185.X.X.254
    Protocol: UDP (17)
    IP ToS: 0x00
    SrcPort: 2055 (2055)
    DstPort: 2311 (2311)
    NextHop: 185.X.X.X
    DstMask: 0
    SrcMask: 0
    TCP Flags: 0x00
    Destination Mac Address: Routerbo_0d:95:72 (d4:ca:6d:XX:XX:XX)
    Post Source Mac Address: 00:00:00_00:00:00 (00:00:00:00:00:00)
    Post NAT Source IPv4 Address: 31.X.X.254
    Post NAT Destination IPv4 Address: 185.X.X.254
    Post NAPT Source Transport Port: 0
    Post NAPT Destination Transport Port: 0

The above tests were made on a CCR1036-8G-2S+ running version 6.32.1 (I cannot upgrade since this is a production system).

Doing the same tests on a x86 installation (running 6.29 - also cannot upgrade because it's in production) the results are even worse! There it appears that the Octets counter wraps around at 2147483647 which suggests that either in versions < 6.32.1 or in non Tilera builds the Octets counter is 32bit Signed.

The whole situation is pretty much the same with when you monitor a Gbit interface with v1 SNMP (32bit counters). The solution in SNMP is very simple. Use SNMP v2 that supports 64bit counters. But I cannot find any solution for Netflow.

Can anyone else confirm this issue? Does anyone know a workaround for it? Is this a limitation of the netflow protocol or a bug in RouterOS? How do other vendors handle this (I don't have any other equipment at the moment to test this out) ?

I have a Nexus 1000v installation running for well over a year now without any problems.

Recently I noticed that I keep getting a warning every few minutes on CLI.

2015 Sep  7 15:13:53 n1kv ipqosmgr[3116]: %IPQOSMGR-4-QOSMGR_WARNING_MSG: Session error status from PPF for client: 0xff01022c, node_id: 0x0, status: 0x41170014  0
2015 Sep  7 15:20:33 n1kv ipqosmgr[3116]: %IPQOSMGR-4-QOSMGR_WARNING_MSG: Session error status from PPF for client: 0xff01022c, node_id: 0x0, status: 0x41170014  0

Searching for this warning did not yield any results. Only a cisco manual about Nexus 5000 series and it mentions that no action is required.

Even if this nothing to worry about, the question remains!
What does that warning mean and why did it start now?

I have some equipment that will be moved to a new datacenter soon.

At the current datacenter, the switches are mounted in the back of the racks so the air flow for the switches is reverse in comparison to the rest of the equipment on the racks.

Since on the new datacenter they strictly follow the cold/hot aisle setup, I have been asked to move the switches on the front of the racks, something which entails a lot more downtime and I wish to avoid, if possible.

The switches are standard Cisco Catalyst 2960(G).

Is it possible to reverse the airflow of the switches so that can still be left on the back of the racks?
Do the fans and IOS support something like that, or if I mount the fans in reverse on the chassis, would that be ok?

I need to block all dynamic dns services update requests from my network so that no dyndns client be able to update its IP.

Do the major dyndns providers use a standard protocol or each one uses its own proprietary one?

Is it possible to match the update requests globally or do I need to find out each provider's IPs to block them manually?

The last few days I noticed some servers being hammered with unknown requests.

Most of them are like the following:

60.246.*.* - - [03/Jan/2015:20:59:16 +0200] "GET /announce.php?info_hash=%80%85%8e%9bu%cfJ.%85%82%e9%25%bf%8e%9e%d7%bf%c5%b0%12&peer_id=-UT3420-v%8bN%aa%60%60%fd%5d%d1%b0Ux&port=15411&uploaded=48588531&downloaded=0&left=0&corrupt=0&key=9E124668&numwant=200&compact=1&no_peer_id=1 HTTP/1.1" 200 -

After a bit of logging and searching I found out that some Chinese ISP (probably CERNET according to the results of whatsmydns.net ) and some Turkish ISP (probably TTNET) respond to dns queries such as a.tracker.thepiratebay.org with various IPs that have nothing to do with piratebay or torrents. In other words they seem to do some kind of DNS Cache Poisoning for some bizarre reason.

So hundreds (if not thousands) of bittorrent clients on those countries make tons of 'announces' to my webservers which result pretty much in a DDoS attack filling up all Apache's connections.

At the moment I blocked China and Turkey altogether and it does the job, but I would like to find a better way to block those requests.

I was thinking of blocking those requests with mod_security based on the HTTP Host header.

All those requests include an HTTP Host header like a.tracker.thepiratebay.org (or many other subdomains of thepiratebay.org domain).

Here's a dump of the request headers via PHP's $_SERVER variable.

DOCUMENT_ROOT: /usr/local/apache/htdocs
GATEWAY_INTERFACE: CGI/1.1
HTTP_ACCEPT_ENCODING: gzip
HTTP_CONNECTION: Close
HTTP_HOST: a.tracker.thepiratebay.org
HTTP_USER_AGENT: uTorrent/342(109415286)(35702)
PATH: /bin:/usr/bin
QUERY_STRING: info_hash=%80%85%8e%9bu%cfJ.%85%82%e9%25%bf%8e%9e%d7%bf%c5%b0%12&peer_id=-UT3420-v%8bN%aa%60%60%fd%5d%d1%b0Ux&port=15411&uploaded=48588531&downloaded=0&left=0&corrupt=0&key=9E124668&numwant=200&compact=1&no_peer_id=1
REDIRECT_STATUS: 200
REMOTE_ADDR: 60.246.*.*
REMOTE_PORT: 3445
REQUEST_METHOD: GET
REQUEST_URI: /announce.php?info_hash=%80%85%8e%9bu%cfJ.%85%82%e9%25%bf%8e%9e%d7%bf%c5%b0%12&peer_id=-UT3420-v%8bN%aa%60%60%fd%5d%d1%b0Ux&port=15411&uploaded=48588531&downloaded=0&left=0&corrupt=0&key=9E124668&numwant=200&compact=1&no_peer_id=1
SCRIPT_FILENAME: /usr/local/apache/htdocs/announce.php
SCRIPT_NAME: /announce.php
SERVER_ADDR: *.*.*.*
SERVER_ADMIN: *@*.*
SERVER_NAME: a.tracker.thepiratebay.org
SERVER_PORT: 80
SERVER_PROTOCOL: HTTP/1.1
SERVER_SIGNATURE: 
SERVER_SOFTWARE: Apache/2.2.29 (Unix) mod_ssl/2.2.29 OpenSSL/1.0.1e-fips mod_bwlimited/1.4 mod_perl/2.0.8 Perl/v5.10.1
UNIQUE_ID: VKg8BJBMIPQAD01XYzgAAAAD
PHP_SELF: /announce.php
REQUEST_TIME_FLOAT: 1420311556.43
REQUEST_TIME: 1420311556
argv: Array
argc: 1

So my question is, how can I block incoming requests to Apache based on the request domain (HTTP Host header) ? Keep in mind that the requests are on various URLs not just /announce.php so blocking by URL is not useful.

Also is that approach viable or will it cause too much load and I should keep dropping those requests before they even reach Apache?

Update:

It turns out this issue has affected many people in many countries around the globe.
There have been numerous reports and blogposts about it and various solutions to block this traffic.

I've collected some of the reports to help anyone coming here searching on a solution to block this.

Mysterious misdirected Chinese traffic : How can I find out what DNS server an HTTP request used?
Strange Bittorrent Log On My Server
http://blog.devops.co.il/post/108740168304/torrent-ddos-attack
https://www.webhostingtalk.com/showthread.php?t=1443734
http://torrentfreak.com/zombie-pirate-bay-tracker-fuels-chinese-ddos-attacks-150124/
https://isc.sans.edu/forums/diary/Are+You+Piratebay+thepiratebayorg+Resolving+to+Various+Hosts/19175/
http://furbo.org/2015/01/22/fear-china/
http://www.jwz.org/blog/2015/01/chinese-bittorrent-the-gift-that-keeps-on-giving/

I have an LSI MegaRAID SAS 9260-4i Controller with two drives attached to it with a simple RAID1 array (virtual drive).

One of the disks failed (complete disappeared from the system) so I asked the datacenter to replace the failed drive.

After they replaced the drive, the controller lost all configuration regarding the RAID1 array it had configured. Now during POST it shows:

After entering the controller configuration utility it shows both drives (old working one and newly replaced one) as unconfigured good. It does not show though the 'Foreign' prefix so I cannot import the old configuration and recreate the missing RAID1 Array.

Doing a 'Scan Devices' did not find any foreign configuration for me to import.

How can I recreate the missing RAID1 array without losing any data?

Is there a way to measure the monthly traffic per VM without having direct access (eg: SNMP) on each VM?

I am aware that there is VMware vCenter Chargeback Manager software which claims to provide this type of measurements but it's not quite what we need (and it seems there is not a public demo available to try it).

Ideally a solution which involves SNMP through vCenter will be the best so we can integrate with our existing statistics system. Or something like Cisco Catalyst switches which provide port statistics via SNMP.

But since most likely there won't be such solution (at least that's what all my google searches seem to suggest), any direction to accomplish this (via API maybe?) would be most welcome.

We are using ESXi 5.5, vCenter 5.5 and Virtual Distributed Switches. Though we do not have access to our gateway router on the datacenter. This is managed by the datacenter itself.

Update:

The public switch (c2960G) on which the ESXi hosts connect to is ours but managed by the datacenter netadmin (I found it that way when I came to the company). So at the moment I don't have access to it.

The gateway router to the public internet is the datacenter's and it's managed by them. No access to it also.

We could arrange to have access to our public switch if that helps somehow.

The datacenter measures our total traffic every month by SNMP on our switch (on the uplink port).