I have two systems, one running Ubuntu 11.04 and one running 10.04 LTS. The same configuration on both machine yields different results. The 10.04LTS machine is unable to find my AWS credential files.
On 11.04
export | grep EC2
declare -x EC2_CERT="/home/jdw/.ec2/cert-PJU5MW2PCN24SKPOIVFTWCMBUXWWZO7U.pem"
declare -x EC2_HOME="/opt/ec2-ami-tools-1.3-66634"
declare -x EC2_PRIVATE_KEY="/home/jdw/.ec2/pk-PJU5MW2PCN24SKPOIVFTWCMBUXWWZO7U.pem"
ls -al /home/jdw/.ec2/
total 16
drwxr-xr-x 2 jdw jdw 4096 2011-08-18 13:31 .
drwxr-xr-x 58 jdw jdw 4096 2011-09-23 10:24 ..
-rw-r--r-- 1 jdw jdw 916 2011-08-18 13:30 cert-PJU5MW2PCN24SKPOIVFTWCMBUXWWZO7U.pem
-rw-r--r-- 1 jdw jdw 928 2011-08-18 13:30 pk-PJU5MW2PCN24SKPOIVFTWCMBUXWWZO7U.pem
/usr/bin/ec2-consistent-snapshot --description "Autosnapshot root" vol-283d5246 --region us-west-1 snap-0c679a62
On 10.04 LTS
export | grep EC2
declare -x EC2_CERT="/root/.ec2/cert-PJU5MW2PCN24SKPOIVFTWCMBUXWWZO7U.pem"
declare -x EC2_HOME="/mnt/ec2-api-tools-1.4.4.2"
declare -x EC2_PRIVATE_KEY="/root/.ec2/pk-PJU5MW2PCN24SKPOIVFTWCMBUXWWZO7U.pem"
ls -al /root/.ec2
total 16
drwxr-xr-x 2 root root 4096 2011-09-23 10:50 .
drwx------ 8 root root 4096 2011-09-23 10:50 ..
-rw-r--r-- 1 root root 916 2011-09-23 10:11 cert-PJU5MW2PCN24SKPOIVFTWCMBUXWWZO7U.pem
-rw-r--r-- 1 root root 928 2011-09-23 10:47 pk-PJU5MW2PCN24SKPOIVFTWCMBUXWWZO7U.pem
/usr/bin/ec2-consistent-snapshot --description "Autosnapshot root" vol-283d5246 --region us-west-1 ec2-consistent-snapshot: ERROR: Can't find AWS access key or secret access key at /usr/bin/ec2-consistent-snapshot line 97.
I assume I am missing some dependency on the 10.04 box but have not had any luck finding what it might me. The only dependency I have run across is the PERL_MM_USE_DEFAULT=1 cpan Net::Amazon::EC2 which is present on both machines.
Further information.
It seems that despite having my environment variables set correctly, ec2-consistent-snapshot is looking for my key in a file named .awssecret
open("/root/.awssecret", O_RDONLY) = -1 ENOENT (No such file or directory) write(2, "ec2-consistent-snapshot: ERROR: "..., 124) = 124 exit_group(2) = ? ~
And it appears that providing the credentials from the command line truncates the credential names:
execve("/usr/bin/ec2-consistent-snapshot", ["/usr/bin/ec2-consistent-snapshot", "--aws-access-key-id-file", "/root/.ec2/cert-PJU5MW2PCN44GRAV"..., "--aws-secret-access-key-file", "/root/.ec2/pk-PJU5MW2PCN44GRAVIV"..., "--description", "Autosnapshot SMC root ", "vol-283d5446", "--region", "us-west-1"], [/* 21 vars */]) = 0
I have attempted using shorter names and the name .awssecret for the pem files and this removes the 'cannot find' condition, but it results in a 'cannot validate the credentials' error. I therefore feel that resolving the main error will quite likely resolve the secondary one.
I'm positive this is some missing library or something on my 10.04 system.
I see what the confusion is. It's very common on AWS and I didn't even notice the first time I read your description. You're passing the wrong credential objects in the command line options.
The ec2-consistent-snapshot program needs the "Access Key ID" and "Secret Access Key". These are completely different values from the "Certificate" and "Private Key", even though they are used to accomplish the same purpose.
Here's an article I wrote which tries to explain many of the different credentials and where to find them:
I am the author/maintainer of ec2-consistent-snapshot (with thanks to other contributors of patches) and will be the first to admit that it tries too hard to find your AWS credentials by searching in too many places. Unfortunately, there is little consistency in where Amazon tools and other third party tools tell you to put your credentials. My goal was to avoid adding to this problem, so I figured it would be good to look in several of the most common places and use the first credentials found.
This works well for most scenarios, but it can cause problems if you have multiple, different AWS credentials stored in your file system, environment variables, and environment variables pointing to the file system.
The software makes a couple poor choices about prioritizing which values should override other values. For example, it looks like you are specifying --aws-access-key-id-file on the command line, but this will be overridden by the envariable $AWS_ACCESS_KEY_ID if it is set. Command line options should override environment or file system settings, but in this case it's getting confused because it's a specific value specified in the environment vs. a file path specified in the command line and it chooses specific values over values stored in a file somewhere.
Turn on the --debug command line option and look for debug output with the information like:
If this is the right access key id, then you might want to edit the program and add in a debug line directly afterwards that outputs the secret access key. If it is not the right access key id, then you'll want to look through the different places where the software looks for AWS credentials (see the manpage) and find where it is getting that information.
Note: In your debug line, I don't think the credentials are actually getting truncated in the software. I believe that is just the debugger saying that the actual values passed were longer than it cares to show.
This is resolved. Thank you to @eric for walking me through the credentials.
The whole issue of environment variables vice command-line variables was a red-herring for me. It turns out that the working 11.04 box was working because, at some point, I had put my AWS credentials into an .awssecret files in my home directory. The fact that I also had a variety of environment variables set was irrelevant.
Once I set my .awssecret file to the same values on the 10.04 box, all was well.
It turns out that with a properly formatted .awssecret file, there is no need at for the following environment variables:
EC2-CERT, EC2_PRIVATE_KEY, AWS_ACCESS_KEY_ID or AWS_SECRET_ACCESS_KEY set at all.