Using s3cmd to recursively update the meta data on a bucket

In our case, we needed to set the Expires and Cache-Control headers on all the files in one of our buckets for optimization purposes. Here’s what we did.

Only the newest version of s3cmd supports the modify command, so if you have an old version installed, run apt-get remove s3cmd to remove the old version.

1. Compile s3cmd directly from GitHub (https://github.com/s3tools/s3cmd)

apt-get install python-setuptools
mkdir /home/s3cmd
cd /home/s3cmd/
git clone https://github.com/s3tools/s3cmd .
sudo python setup.py install

2. Once the compile finishes, configure s3cmd. You must have your AWS Access Key and Secret available.
./s3cmd --configure

3. Execute the following commands. Replace with the name of your bucket. If you want to do this operation only on a particular folder inside a bucket, you can replace with the folder along with the bucketname.
./s3cmd --recursive modify --add-header="Expires:`date -u +"%a, %d %b %Y %H:%M:%S GMT" --date "next Year"`" s3:///
./s3cmd --recursive modify --add-header='Cache-Control:max-age=31536000, public' s3:///

Leave a Reply

Your email address will not be published. Required fields are marked *