iMil's questions

In order to dynamically build a configuration file, I need to query EC2 from an instance to retrieve information on existing instances. I use the aws CLI with the ec2 describe-instances flag. The instance I'm running the command from is attached to a role that permits such queries:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Action": [
                "ec2:DescribeAvailabilityZones",
                "ec2:DescribeInstances",
                "ec2:DescribeRegions",
                "ec2:DescribeSecurityGroups",
                "ec2:DescribeTags"
            ],
            "Resource": "*",
            "Effect": "Allow"
        }
    ]
}

This works perfectly fine when the instance is started, but when I try to fetch data from a user-data file, I get:

A client error (AuthFailure) occurred when calling the DescribeInstances operation: AWS was not able to validate the provided access credentials

I read that this could be caused by a not-time-synchronized instance, so I added openntpd to the user-data file, before calling awscli, but still get the same error. Within the userdata, I successfully access to s3 and route53 using awscli:

aws s3 cp s3://s3test/foobar.yml playbook.yml

And just to be sure the IAM policy was ok, I also tried with AWS's EC2ReadOnly policy, and got the same result.

The query is done this way:

region="eu-central-1"

for ip in $(aws ec2 describe-instances --debug --filters 'Name=tag:Name,Values=rabbitmq' --region "$region"|jq -r '.Reservations[].Instances[].NetworkInterfaces[].PrivateIpAddresses[].PrivateIpAddress')
do
    # [stuff to be done]
done

Please note this is not an authentication failure, I am using roles to allow the instance to query EC2.

Anyone around using an awscli EC2 access and willing to share experience?

I have set up an IPsec over GRE connection with a remote host, both are NetBSD 6.1 based. The "client" is connected to the Internet through a 400Mbps fiber connection. The "server" is located on a 10Gbps network. Both machines have 1Gbps NICs which behave perfectly, meaning they both reach the link speed limit when transferring data outside the IPsec tunnel. When doing a transfer through the tunnel, speed drops at a factor 5 to 10:

direct connection: /dev/null 27%[====> ] 503.19M 45.3MB/s eta 83s IPsec connection: /dev/null 2%[ ] 47.76M 6.05MB/s eta 5m 3s

The tunnel is setup this way:

On the server, which is a NetBSD domU running on a debian/amd64 dom0:

$ cat /etc/ifconfig.xennet0
# server interface
up
inet 192.168.1.2 netmask 255.255.255.0
inet 172.16.1.1 netmask 0xfffffffc alias
$ cat /etc/ifconfig.gre0 
create
tunnel 172.16.1.1 172.16.1.2 up
inet 172.16.1.5 172.16.1.6 netmask 255.255.255.252

IPsec traffic is forwarded from dom0's public IP to the domU's xennet0 interfacethrough iptables NAT rules:

-A PREROUTING -i eth0 -p udp -m udp --dport 500 -j DNAT --to-destination 192.168.1.2:500 
-A PREROUTING -i eth0 -p esp -j DNAT --to-destination 192.168.1.2 
-A PREROUTING -i eth0 -p ah -j DNAT --to-destination 192.168.1.2

On the client:

$ cat /etc/ifconfig.vlan8 
# client public interface
create
vlan 8 vlanif re0
!dhcpcd -i $int
inet 172.16.1.2 netmask 0xfffffffc alias
$ cat /etc/ifconfig.gre1 
create
tunnel 172.16.1.2 172.16.1.1 up
inet 172.16.1.6 172.16.1.5 netmask 255.255.255.252

On racoon's side, I tried various hash / encryption algorithms combinations, even enc_null, but nothing changes really, transfer is still stuck at a 6MB/s max.

remote node.public.ip {
     exchange_mode main;
     lifetime time 28800 seconds;
     proposal {
         encryption_algorithm blowfish;
         hash_algorithm sha1;
         authentication_method pre_shared_key;
         dh_group 2;
     }
     generate_policy off;
}

sainfo address 172.16.1.1/30 any address 172.16.1.2/30 any {
     pfs_group 2;
     encryption_algorithm blowfish;
     authentication_algorithm hmac_sha1;
     compression_algorithm deflate;
     lifetime time 3600 seconds;
}

On the client:

remote office.public.ip {
     exchange_mode main;
     lifetime time 28800 seconds;
     proposal {
         encryption_algorithm blowfish;
         hash_algorithm sha1;
         authentication_method pre_shared_key;
         dh_group 2;
     }
     generate_policy off;
}

sainfo address 172.16.1.2/30 any address 172.16.1.1/30 any {
     pfs_group 2;
     encryption_algorithm blowfish;
     authentication_algorithm hmac_sha1;
     compression_algorithm deflate;
     lifetime time 3600 seconds;
}

The tunnel establishes with no issue, the only problem here is transfer drop. Again, when transferring from / to the server from / to the client without tunnel, speed is optimal, drop occurs only through IPsec.

Both machines are intel-based CPUs running at 2+GHz, plenty of memory and very little CPU time consumed by anything else than forwarding / NAT.

Has anyone witnessed such a behaviour? Any idea on where to look further?

Thanks,