Home / server / Questions / 11024
Accepted
deadprogrammer
Asked: 2009-05-22 14:07:57 +0800 CST

examining http headers in shell scripts

  • 772

How do I write a script that sends a get request to a web page and returns nicely formatted http headers?

scripting http shell

6 Answers

  1. One option might be to use curl with the --dump-header option.

    • 3

  2. There are quite a few modules in many different languages that will retrieve HTTP headers for you.

    etc, etc.

    • 2
  3. Best Answer

    try this nasty hack:

    wget -O - -o /dev/null --save-headers http://google.com | \
awk 'BEGIN{skip=0}{ if (($0)=="\r") {skip=1;}; if (skip==0) print $0 }'
    • 1

  4. curl has support to view the headers when you're downloading, or you can use the -I option to save the headers to a file.

    • 1

  5. If you just want to view the headers.. (Not programatically) just use [live http headers][1] plugin for mozilla firefox.

    https://addons.mozilla.org/en-US/firefox/addon/3829

    • 0

  6. This bash function will take an URL + method, or Server + path + method + port. If you use "HEAD" method, it will return the headers, if you use GET, it will return all headers and the whole reply. Supports https through openssl.

    #!/bin/bash
function httpreq ()
{
    if [ $# -eq 0 ]; then echo -e "httpreq SERVER PATH [GET/HEAD] [PORT]\nOR\nhttpreq URL [GET/HEAD]"; return 1; fi
    if echo $1 | grep -q "://"
    then
        SUNUCU=$(echo $1 |cut -d '/' -f3 | cut -d':' -f1)
        YOL=/$(echo $1 |cut -d '/' -f4-)
        PROTO=$(echo $1 |cut -d '/' -f1)
        METHOD=${2:-GET}
        PORT=$(echo $1| sed -n 's&^.*://.*:\([[:digit:]]*\)/.*&\1&p')
        if [ -z $PORT ]
        then
            if [ $PROTO == "https:" ]; then PORT=443; else PORT=80; fi
        fi
    else
        SUNUCU=$1
        YOL=$2
        METHOD=${3:-GET}
        PORT=${4:-80}
    fi
    if [ $PROTO == "https:" ];
    then
     echo -e "$METHOD $YOL HTTP/1.1\r\nHOST:$SUNUCU\r\n\r\n" | openssl s_client -quiet -connect $SUNUCU:$PORT
    else
     echo -e "$METHOD $YOL HTTP/1.1\r\nHOST:$SUNUCU\r\n\r\n" | nc $SUNUCU $PORT
    fi
}
    • 0