traffic shaping on OSX 10.10 with pfctl and dnctl

I am trying to do traffic shaping (throttling) on Mac OS X 10.10 via pfctl and dnctl.

I have implemented a simple test server and client with netcat (nc) and a synthetic random payload file to verify if my dummynet pipe throttling is working correctly. So far, attempting to configure the dummynet pipe using the murus firewall GUI app does not seem to correctly throttle traffic (64MB transfer completes in ~200ms).

Here's an OSX bash shell script (requires brew install coreutils for gdate) that is a complete example. If you run it in one terminal as:

./throttle-test.sh server

and another one as

./throttle-test.sh client

They will try to transfer a 64MB payload over your en0 interface (not using lo0 because its huge MTU is not analogous to WAN traffic).

I have also tested transferring the file to a remote linux laptop to see if the source and destination IP both being local IPs was bypassing the throttling, but even to a remote machine on my LAN/wifi the speed is much faster than the throttled limit.

My question is what would be a correct script to configure pfctl and dnctl to throttle this file transfer to a given bandwidth limit (say 8mbps for example). The scope of the throttling can be a specific TCP port.

Note OS X 10.10 no longer includes ipfw so I'm looking for something using pfctl and dnctl.

Here's my throttle-test.sh file:

#!/bin/bash
set -o errexit    # always exit on error
set -o errtrace   # trap errors in functions as well
set -o pipefail   # don't ignore exit codes when piping output
set -o posix      # more strict failures in subshells
# set -x          # enable debugging

IFS="$(printf "\n\t")"

setup_payload() {
  local payload_path="$1"
  local size_kbytes="$2"
  mkdir -p $(basename "${payload_path}")

  if [[ -f "${payload_path}" ]]; then
    local on_disk=$(wc -c < "${payload_path}")
  fi
  if [[ ${on_disk} -ne $((${size_kbytes} * 1024)) ]]; then
    echo "creating payload file ${payload_path}"
    dd if=/dev/urandom of="${payload_path}" \
      bs=1024 count="${size_kbytes}" &> /dev/null
  fi
}

start_server() {
  local payload_path="$1"
  local ip="$2"
  local port="$3"
  while true; do
    echo "Starting netcat server for ${payload_path} on ${ip}:${port}"
    nc -l "${ip}" "${port}" < "${payload_path}"
    sleep 1
  done
}

hash() {
  shasum -a 256 "$1" | cut -d " " -f 1
}

verify_hashes() {
  # Sanity check no funny business
  from_hash=$(hash "$1")
  to_hash=$(hash "$2")
  if [[ "${from_hash}" != "${to_hash}" ]]; then
    echo "checksums did not match ${from_hash} ${to_hash}" 1>&2
    exit 10
  fi
}

client() {
  local payload_path="$1"
  local ip="$2"
  local port="$3"

  # time how long it takes to transfer the payload file
  start=$(gdate +%s%3N)
  nc -d "${ip}" "${port}" > "${payload_path}.client"
  stop=$(gdate +%s%3N)

  verify_hashes "${payload_path}" "${payload_path}.client"

  local duration=$((${stop} - ${start}))
  echo "${duration}"
}

main() {
  local size_kbytes=$((64 * 1024)) # 64 MB
  local payload_path="/tmp/throttle-test/data-${size_kbytes}.bin"
  local port="${PORT-9112}"
  # You may need to change this if you are on linux
  local interface="${INTERFACE-en0}"
  local ip=$(ipconfig getifaddr "${interface}")

  setup_payload "${payload_path}" "${size_kbytes}"
  case "$1" in
    server)
      start_server "${payload_path}" "${ip}" "${port}"
    ;;
    client)
      local duration=$(client "${payload_path}" "${ip}" "${port}")
      echo "Transfered ${size_kbytes} kbytes in ${duration} ms"
    ;;
    *)
      echo "Usage: $0 <server|client>"
    ;;
  esac
}

main "$@"

Update

Here is what I have so far. This seems to work correctly for the download direction, but not throttle at all in the upload direction.

throttle_start() {
  local down_mbps="$1"
  local up_mbps="$2"
  local delay=$(($3 / 2))
  sudo dnctl pipe 1 config bw "${down_mbps}Mbit/s" delay "${delay}"
  sudo dnctl pipe 2 config bw "${up_mbps}Mbit/s" delay "${delay}"
  (cat /etc/pf.conf && \
    echo 'dummynet-anchor "throttle"' && \
    echo 'anchor "throttle"') | sudo pfctl -f -
  cat << EOF | sudo pfctl -a throttle -f -
dummynet in quick proto tcp from any port = 9112 to any pipe 1
dummynet out quick proto tcp from any to any port = 9112 pipe 2
EOF
  sudo pfctl -q -e
}