pkaeding's questions

I have a pcap file (~2.3G) containing HTTP requests. I need to extract the body of each request in some way that I can further process it. Each request in its own file would work well, but I can be flexible on that.

I found something promising in tshark, as this command does almost what I need:

tshark -r capture.pcap --export-objects "http,data"

I get a folder with a bunch of files in it, each one containing one request body.

However, it only outputs the first 1000 requests. How can I get the rest of the requests?

I know that I can find the latest Ubuntu AMI builds at http://cloud-images.ubuntu.com/locator/ec2/, but is there a mailing list, or RSS feed, or some way to be notified of new builds? I want to be sure that my AMI builds are based on the latest patches, but I don't want to have to visit that page every day to see if there is something new.

Ideally, I would only be notified of LTS releases.

I have a VPC using the network 10.10.0.0/16, and a VPN server in the public subnet (10.10.0.0/24). The VPN uses addresses in the 10.11.254.0/24 range. When I connect to the VPN, I can access hosts in both the public subnet and the private subnet (10.10.1.0/24), so I believe I have the routing table set up properly to send packets bound for the VPN back through the VPN server. I have disabled source/destination checks on the VPN server (which is also needed to be able to access other hosts in the network).

The AmazonProvidedDNS server seems to be working properly inside the VPC; I can run dig @10.10.0.2 ip-10-10-1-215.ec2.internal from a host in the VPC, and I get the expected response.

However, if I run that same dig command from may laptop, connected to the VPN, I get no response.

From running tcpdump on the vpn server, I see the A? packets coming from my VPN address, going to 10.10.0.2, but I don't see any response coming back. Is there something I need to do to enable the DNS server to answer requests from outside the VPC address range?

I am trying to add an EC2 instance to an Elasitic Load Balancer using an Ansible playbook, with the ec2_elb module. This is the task that should do this:

- name: "Add host to load balancer {{ load_balancer_name }}"
  sudo: false
  local_action:
    module: ec2_elb
    state: present
    wait: true
    region: "{{ region }}"
    ec2_elbs: ['{{ load_balancer_name }}']
    instance_id: "{{ ec2_id }}"

However, it routinely fails, with this output (verbosity turned up):

TASK: [Add host to load balancer ApiELB-staging] ****************************** 
<127.0.0.1> REMOTE_MODULE ec2_elb region=us-east-1 state=present instance_id=i-eb7e0cc7
<127.0.0.1> EXEC ['/bin/sh', '-c', 'mkdir -p $HOME/.ansible/tmp/ansible-tmp-1409156786.81-113716163813868 && chmod a+rx $HOME/.ansible/tmp/ansible-tmp-1409156786.81-113716163813868 && echo $HOME/.ansible/tmp/ansible-tmp-1409156786.81-113716163813868']
<127.0.0.1> PUT /var/folders/d4/17fw96k107d5kbck6fb2__vc0000gn/T/tmpki4HPF TO /Users/pkaeding/.ansible/tmp/ansible-tmp-1409156786.81-113716163813868/ec2_elb
<127.0.0.1> EXEC ['/bin/sh', '-c', u'LANG=en_US.UTF-8 LC_CTYPE=en_US.UTF-8 /usr/bin/python /Users/pkaeding/.ansible/tmp/ansible-tmp-1409156786.81-113716163813868/ec2_elb; rm -rf /Users/pkaeding/.ansible/tmp/ansible-tmp-1409156786.81-113716163813868/ >/dev/null 2>&1']
failed: [10.0.115.149 -> 127.0.0.1] => {"failed": true}
msg: The instance i-eb7e0cc7 could not be put in service on LoadBalancer:ApiELB-staging. Reason: Instance has not passed the configured HealthyThreshold number of health checks consecutively.

FATAL: all hosts have already failed -- aborting

I have my ELB configuration defined like this (also via Ansible):

- name: "Ensure load balancer exists: {{ load_balancer_name }}"
  sudo: false
  local_action:
    module: ec2_elb_lb
    name: "{{ load_balancer_name }}"
    state: present
    region: "{{ region }}"
    subnets: "{{ vpc_public_subnet_ids }}"
    listeners:
      - protocol: https
        load_balancer_port: 443
        instance_protocol: http
        instance_port: 8888
        ssl_certificate_id: "{{ ssl_cert }}"
    health_check:
        ping_protocol: http # options are http, https, ssl, tcp
        ping_port: 8888
        ping_path: "/internal/v1/status"
        response_timeout: 5 # seconds
        interval: 30 # seconds
        unhealthy_threshold: 10
        healthy_threshold: 10
  register: apilb

When I access the status resource from either my laptop or from the server itself (as localhost) I get a 200 response as expected. I also added a command task to the Ansible playbook, right before adding the instance to the ELB, to confirm that the application is booted up and serving requests properly (and it is):

- command: /usr/bin/curl -v --fail http://localhost:8888/internal/v1/status

I don't see any non-200 responses for the status check resource in the logs for my application (but of course, if the requests never made it as far as my application, they would not be logged).

The other weird thing is that the instance does get added to the ELB, and it seems to work properly. So I know that at some point, at least, the load balancer can access the application properly (for both the status check resource, and other resources). The AWS console shows the instance is healthy, and the Cloudwatch charts don't show any failed health checks.

Any ideas?

I am running nginx/1.1.19:

# nginx -V
nginx version: nginx/1.1.19
TLS SNI support enabled
configure arguments: --prefix=/etc/nginx --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-client-body-temp-path=/var/lib/nginx/body --http-fastcgi-temp-path=/var/lib/nginx/fastcgi --http-log-path=/var/log/nginx/access.log --http-proxy-temp-path=/var/lib/nginx/proxy --http-scgi-temp-path=/var/lib/nginx/scgi --http-uwsgi-temp-path=/var/lib/nginx/uwsgi --lock-path=/var/lock/nginx.lock --pid-path=/var/run/nginx.pid --with-debug --with-http_addition_module --with-http_dav_module --with-http_geoip_module --with-http_gzip_static_module --with-http_image_filter_module --with-http_realip_module --with-http_stub_status_module --with-http_ssl_module --with-http_sub_module --with-http_xslt_module --with-ipv6 --with-sha1=/usr/include/openssl --with-md5=/usr/include/openssl --with-mail --with-mail_ssl_module --add-module=/build/buildd/nginx-1.1.19/debian/modules/nginx-auth-pam --add-module=/build/buildd/nginx-1.1.19/debian/modules/nginx-echo --add-module=/build/buildd/nginx-1.1.19/debian/modules/nginx-upstream-fair --add-module=/build/buildd/nginx-1.1.19/debian/modules/nginx-dav-ext-module

I am using it to reverse proxy a service that can stream data as a chunked response. Each chunk from the backend service is a valid JSON document, but when accessed through the nginx proxy, the chunks are repackaged into 'round' sizes, like 0x1000, 0x2000, 0x4000, etc. This makes the chunks no longer valid JSON blobs, since each chunk will have the end of one document at the beginning, and the beginning of another document at the end.

My location block is configured like so:

    location / {
            proxy_pass http://backend;
            proxy_read_timeout 600;
            proxy_buffering off;
    }

I thought that the proxy_buffering off; part should avoid the behavior I am seeing. Is there some other setting I need to change?

I am trying to monitor the temp sensors on a server, and plot them using Cacti. I have lm-sensors installed and working correctly. For example, here is the output from sensors:

% sensors                                 
acpitz-virtual-0
Adapter: Virtual device
temp1:       +26.8 C  (crit = +100.0 C)                  
temp2:       +32.0 C  (crit = +60.0 C)                  

coretemp-isa-0000
Adapter: ISA adapter
Core 0:      +36.0 C  (high = +105.0 C, crit = +105.0 C)  

coretemp-isa-0001
Adapter: ISA adapter
Core 1:      +42.0 C  (high = +105.0 C, crit = +105.0 C)  

However, when I try to get this data via SNMP, I get only one sensor's temperature correctly, and another one always returns 100.000 C:

% snmpwalk -Os -c public -v 1 10.8.0.18 -m ALL lmTempSensors     
lmTempSensorsIndex.1 = INTEGER: 0
lmTempSensorsIndex.2 = INTEGER: 1
lmTempSensorsDevice.1 = STRING: temp1
lmTempSensorsDevice.2 = STRING: temp1
lmTempSensorsValue.1 = Gauge32: 26800
lmTempSensorsValue.2 = Gauge32: 100000

So, my question is two-fold:

1. Why is the second sensor that is returned by SNMP giving a value of 100 C (when it should be 32 C)
2. Why are my CPU core sensors not being returned by SNMP?

UPDATE: I noticed that it seems the other sensors are present as lmMiscSensors, so I can see the other data:

% snmpwalk -Os -c public -v 1 10.8.0.18 -m ALL sensor
lmTempSensorsIndex.1 = INTEGER: 0
lmTempSensorsIndex.2 = INTEGER: 1
lmTempSensorsDevice.1 = STRING: temp1
lmTempSensorsDevice.2 = STRING: temp1
lmTempSensorsValue.1 = Gauge32: 26800
lmTempSensorsValue.2 = Gauge32: 100000
lmMiscSensorsIndex.1 = INTEGER: 0
lmMiscSensorsIndex.2 = INTEGER: 1
lmMiscSensorsIndex.3 = INTEGER: 2
lmMiscSensorsIndex.4 = INTEGER: 3
lmMiscSensorsIndex.5 = INTEGER: 4
lmMiscSensorsIndex.6 = INTEGER: 5
lmMiscSensorsIndex.7 = INTEGER: 6
lmMiscSensorsIndex.8 = INTEGER: 7
lmMiscSensorsDevice.1 = STRING: Core 0
lmMiscSensorsDevice.2 = STRING: Core 0
lmMiscSensorsDevice.3 = STRING: Core 0
lmMiscSensorsDevice.4 = STRING: Core 0
lmMiscSensorsDevice.5 = STRING: Core 1
lmMiscSensorsDevice.6 = STRING: Core 1
lmMiscSensorsDevice.7 = STRING: Core 1
lmMiscSensorsDevice.8 = STRING: Core 1
lmMiscSensorsValue.1 = Gauge32: 34000
lmMiscSensorsValue.2 = Gauge32: 105000
lmMiscSensorsValue.3 = Gauge32: 105000
lmMiscSensorsValue.4 = Gauge32: 0
lmMiscSensorsValue.5 = Gauge32: 39000
lmMiscSensorsValue.6 = Gauge32: 105000
lmMiscSensorsValue.7 = Gauge32: 105000
lmMiscSensorsValue.8 = Gauge32: 0

I am still wondering how to read this. Is lmTempSensorsValue.2 just giving the critical temperature point for temp1? If so, where is the temp2 reading?

I have two Grandstream GXP2000 hardphones that power on (the LEDs all come on green), but the display is blank, and the switch doesn't recognize the phones as being connected.

In the past, this has happened, except the phones still responded to the network, so I was able to reboot them using the web interface, and they worked again. Simply unplugging it, and plugging it back in doesn't help. I can't reset it to the factory settings, because the display is blank.

I am fairly certain that the phone is the problem, since I have plugged known good phones into the same power and network connections, and they worked. Likewise, the bad phones connected to known-good power and network don't work.

From what I have read online, it sounds like this might be a common problem, but I thought I'd ask here to see if anyone knows any tricks.

I want to be able to see how many minutes of use each user in my asterisk uses. Right now, I am using a POTS interface to the outside wold, but I want to know how much the system is used, so I can compare an IP-based service.

Ideally, I want to be able to see how much each POTS line is used, in minutes per month.

It seems that Cacti would be great to be able to visualize this usage, but I can't find anything online. The only things I can find will graph the number of open channels at the time of the polling, which I believe would not reflect calls that were started after one poll and ended before the next.

Any ideas?

I have Apache set up to serve several Virtual Hosts, and I would like to see how much bandwidth each site uses. I can see how much the entire server uses, but I would like more detailed reports.

Most of the things I have found out there are for limiting bandwidth to virtual hosts, but I don't want to do that; I just want to see which sites are using how much bandwidth.

This isn't for billing purposes, just for information.

Is there an apache module I should use? Or is there some other way to do this?

I have two DD-WRT routers in different locations. One of them is running the VPN version of the firmware, and the other is running the standard version. I am on location with the VPN one, and I want to upgrade the other one to run the VPN firmware so I can wire them together. Is this possible or advisable? Of course, I don't want to lose the settings on the remote one, since then I would lose my remote access (currently through SSH) and not be able to configure it.

Is it possible to set up a Hardy domU on an existing Xen dom0 running Dapper? I have found many postings talking about the reverse (Dapper domU on a Hardy dom0), but I have the existing server set up with a Dapper dom0 and a few Dapper domUs. I need to set up a new domU, so I want to use a more modern LTS release. I am hesitant to upgrade the dom0, on the "if it ain't broke, don't fix it" principle.

Has anyone tried this? Can I just use a 2.6.24 Xen kernel? Do I need to recompile the domU kernel to match anything in my dom0?

Or is it necessary to upgrade the dom0?

Thanks!