Questions[aws-cli](server)

We have a bucket with more than 500,000 objects in it.

I'm assigned a job where I've to delete files which have a specific prefix. There are around 300,000 files with the given prefix in the bucket.

For eg If there are 3 files

abc_1file.txt
abc_2file.txt
abc_1newfile.txt

I've to delete the files with abc_1 prefix only. I didn't find much in AWS documentation related to this.

Any suggestions on how can I automate this?

I have an IAM user adminuser which is in the Administrators group (having AdministratorAccess).

I have used aws configure to set up the settings.

~/.aws/config

[default]
region = us-east-1
output = json

~/.aws/credentials

[default]
aws_access_key_id = ************
aws_secret_access_key = ************

But when I run aws lambda list-functions --profile adminuser, it says

The config profile (adminuser) could not be found

Once I run export AWS_DEFAULT_PROFILE=adminuser, and run aws lambda list-functions --profile adminuser again, it shows:

botocore.exceptions.ProfileNotFound: The config profile (adminuser) could not be found

I read AWS CLI Config profile not found. So I try to changed to

~/.aws/config

[adminuser]
region = us-east-1
output = json

~/.aws/credentials

[adminuser]
aws_access_key_id = ************
aws_secret_access_key = ************

and run aws lambda list-functions --profile adminuser again, then it shows:

You must specify a region. You can also configure your region by running "aws configure".

I also tried this like the official AWS document demo

~/.aws/config

[default]
region = us-east-1
output = json

[adminuser]
region = us-east-1
output = json

~/.aws/credentials

[default]
aws_access_key_id = ************
aws_secret_access_key = ************

[profile adminuser]
aws_access_key_id = ************
aws_secret_access_key = ************

But still failed.

How can I run aws lambda list-functions --profile adminuser successfully? Thanks

I have hundreds of instances in the EC2 web console.

Some are shutdown. I can see that they were created many months or years ago. They seem abandoned and forgotten for a while but it's hard to tell if terminating them is safe.

e.g. They could be some old backup, someone could have just migrated a load to new instances and keep the old ones for a few weeks as precaution, etc...

We end up keeping most of them as it's the safest choice.

We are still paying for the EBS volumes when the instance is shutdown (but not terminated). The price can escalate over time with many unused instances (EBS is $140 per month per TB). So I'd like to do some cleanup.

I'm thinking that if I could find the time when the instance was shutdown, that would be a good indicator of how "abandoned" it is.

How can I find when an instance was last shutdown?

The closest I have get is using the following commands.

This command manage to lists all name of instances.

aws ec2 describe-instances --filters Name=vpc-id,Values=vpc-e2f17e8b --query 'Reservations[].Instances[].Tags[?Key==`Name`].Value[]'

This command manage to list all private ip address, instance id and ALL tags which I don't need. I just need the name.

aws ec2 describe-instances --filters Name=vpc-id,Values=vpc-e2f17e8b | jq '.Reservations[].Instances[] | {PrivateIpAddress, InstanceId, Tags}'

I'm not sure why I can't execute command like this way:

aws ec2 describe-instances | jq '.["Reservations"]|.[]|.Instances|.[]|.PrivateIpAddress + " " + .InstanceId + " " + .Tags[?Key==`Name`].Value[]'

This command works but its showing all the Tags Key names.

aws ec2 describe-instances | jq '.["Reservations"]|.[]|.Instances|.[]|.PrivateIpAddress + " " + .InstanceId + " " + .Tags'

I am running "Amazon Linux AMI release 2010.11.2". I noticed that the AWS CLI tools are outdated.

I have tried to use yum update and yum upgrade but it's telling me that the packages are already at their latest version.

How would I update the CLI tools, and how would I update the distribution itself? E.g. to a newer version of AWS Linux.

Thanks!