Home / server / Questions / 769069
Accepted
Ernest Mueller
Asked: 2016-04-09 07:03:29 +0800 CST

Access S3 bucket from my EC2 instance using CF::Init via IAM role not working

  • 772

I have a CloudFormation template I'm using to set up an ECS cluster and I am trying to drop some config files onto the box using CloudFormation::Init on the ASG and pulling them out of S3.

"ECSASGLaunchConfiguration": {
  "Type": "AWS::AutoScaling::LaunchConfiguration",
  "Metadata": {
    "AWS::CloudFormation::Authentication": {
      "S3AccessCreds": {
        "type": "S3",
        "roleName": {
          "Ref": "ECSEC2InstanceIAMRole"
        }
      }
    },
    "AWS::CloudFormation::Init": {
      "config": {
        "packages": {
        },
        "groups": {
        },
        "users": {
        },
        "sources": {
        },
        "files": {
          "/etc/dd-agent/conf.d/nginx.yaml": {
            "source": "https://s3.amazonaws.com/foobar/scratch/nginx.yaml",
            "mode": "000644",
            "owner": "root",
            "group": "root"
          },
          "/etc/dd-agent/conf.d/docker_daemon.yaml": {
            "source": "https://s3.amazonaws.com/foobar/scratch/docker_daemon.yaml",
            "mode": "000644",
            "owner": "root",
            "group": "root"
          }
        },
        "commands": {
        },
        "services": {
        }
      }
    }
  },

To do so, I've added a policy inline to the role I'm creating for my EC2 instances that should allow all S3 access from the instances.

"ECSEC2InstanceIAMRole": {
  "Type": "AWS::IAM::Role",
  "Properties": {
    "AssumeRolePolicyDocument": {
      "Statement": [
        {
          "Effect": "Allow",
          "Principal": {
            "Service": [
              "ec2.amazonaws.com"
            ]
          },
          "Action": [
            "sts:AssumeRole"
          ]
        }
      ]
    },
    "ManagedPolicyArns": [
      "arn:aws:iam::aws:policy/service-role/AmazonEC2ContainerServiceforEC2Role",
      "arn:aws:iam::aws:policy/CloudWatchLogsFullAccess"
    ],
    "Path": "/",
    "Policies": [
      {
        "PolicyName": "otxS3access",
        "PolicyDocument": {
          "Version": "2012-10-17",
          "Statement": [
            {
              "Effect": "Allow",
              "Action": "s3:ListAllMyBuckets",
              "Resource": "arn:aws:s3:::*"
            },
            {
              "Effect": "Allow",
              "Action": [
                "s3:ListBucket",
                "s3:GetBucketLocation"
              ],
              "Resource": "arn:aws:s3:::foobar"
            },
            {
              "Effect": "Allow",
              "Action": [
                "s3:GetObject",
                "s3:PutObject",
                "s3:DeleteObject"
              ],
              "Resource": "arn:aws:s3:::foobar/*"
            }
          ]
        }
      }
    ]
  }
},
"ECSEC2InstanceIAMProfile": {
  "Type": "AWS::IAM::InstanceProfile",
  "Properties": {
    "Path": "/",
    "Roles": [
      {
        "Ref": "ECSEC2InstanceIAMRole"
      }
    ]
  }
},

But it doesn't work. I can't find an error (or anything else) in the logs. When I manually try to curl it from the instance it doesn't work either, "AccessDenied..."

The bucket itself in this case, "foobar", doesn't have any special permissions besides not being public, just the standard account-name single grantee.

Any idea what I'm doing wrong?

amazon-web-services

1 Answers

  1. Best Answer

    I discovered the problem. All that CloudFormation in the question is just fine and does what it's supposed to. The problem was with my run of cfn-init in the instance user-data set up in the ASG's launch configuration - that wasn't working so it wasn't executing the init stuff. Thanks to @Rob-d, your comments led me down the path to the executable-level.

    The magical other part that makes all this work:

        "UserData": {
      "Fn::Base64": {
        "Fn::Join": [
          "",
          [
            "#!/bin/bash\n",
            "cat > /etc/ecs/ecs.config <<EOF\n",
            "ECS_CLUSTER=",
            {
              "Ref": "ECSCluster"
            },
            "\n",
            "ECS_ENGINE_AUTH_TYPE=docker\n",
            "ECS_ENGINE_AUTH_DATA={REDACTED}\n",
            "EOF\n",
            "yum -y install aws-cfn-bootstrap\n",
            "# Install the files and packages from the metadata\n",
            "/opt/aws/bin/cfn-init -v ",
            "         --stack ",
            {
              "Ref": "AWS::StackName"
            },
            "         --resource ECSASGLaunchConfiguration ",
            "         --region ",
            {
              "Ref": "AWS::Region"
            },
            "\n",
            "/opt/aws/bin/cfn-signal -e -0 ",
            "         --stack ",
            {
              "Ref": "AWS::StackName"
            },
            "         --resource ECSAutoScalingGroup ",
            "         --region ",
            {
              "Ref": "AWS::Region"
            },
            "\n",
            "\n"
          ]
        ]
      }
    }
    • 1