Home / server / Questions / 725562
Accepted
Gabriel Bauman
Asked: 2015-09-30 07:37:08 +0800 CST

Recursively changing the content-type for files of a given extension on Amazon S3

  • 772

I have a large S3 bucket with a nested "folder" structure containing (among other things) static .json and .md files. Theses files are being served by S3 as text/plain rather than the correct application/json and text/markdown.

I have updated the bucket defaults so that new uploads will have the correct content type.

What is the best way to walk the "tree" and update the content type for files matching a certain extension?

amazon-s3

4 Answers

  1. Best Answer

    Here is an example how to do this with the aws cli tool. The cp tool allows the use of a recursive options, which I don't think the s3api tool can do. In this case, I'm fixing a bunch of SVGs. Remove the --dryrun options when you are ready to unleash it.

    aws s3 cp \
      --exclude "*" \
      --include "*.svg" \
      --content-type="image/svg+xml"  \
      --metadata-directive="REPLACE" \
      --recursive \
      --dryrun \
       s3://mybucket/static/ \
       s3://mybucket/static/
    • 32

  2. After some digging, I found that the s3cmd tool can do this. For example, to set JSON files to application/json:

    s3cmd --recursive modify --acl-public \
       --add-header='content-type':'application/json' \
       --exclude '' --include '.json' \
       s3://bucket/
    • 4

  3. Unless you want to get your hands dirty using boto/python or another AWS API, I suspect your best bet would be to use $ aws s3api copy-object with the --content-type flag to copy an object to itself, setting the new content type.

    Here's the documentation for s3api.

    It goes without saying that you should test this first, before running it recursively on your whole bucket.

    • 3

  4. As mentioned earlier, you can use aws tool by Amazon and use s3api to copy-object onto itself, and use metadata-directive=REPLACE to change the content-type.

    I am putting this here, because sometimes you would want to iterate on filenames that are stored in database, and this is how you can do it through cli. 

    aws s3api copy-object \
          --content-type="application/vnd.android.package-archive" \
          --metadata-directive="REPLACE" \
          --copy-source "MYBUCKET/FILE.apk" \
          --bucket "MYBUCKET" \
          --key "FILE.apk" \
          --acl public-read
    • 3