9 Hidden Features of Amazon S3
Posted by Carlton Bale on 1st May 2008
Amazon S3 (Simple Storage Solution) is a great way to store (and optionally share) large amounts of data. It's cheap, fast, and reliable. If you're a casual user, you may not know that you do much more than just store and sever data of HTTP, as long as you use an advanced application such as Bucket Explorer (or read through the S3 documentation and code it yourself.) Here's a list of 9 features you may not have known about.
- Torrent Tracking and Seeding: Amazon S3 can server as an ultra-reliable torrent tracker; share/seed the files from your local PC and let S3 act as the tracker. Or let S3 handle both the seeding and the tracking. Here are the details on how to do it.
- Enable / Disable directory browsing: If you are sharing files of HTTP, you may or may not want people to be able to list the contents of a bucket (folder.) If you want the bucket contents to be listed when someone enters the bucket name (http://s3.amazonaws.com/bucket_name/), then edit the Access Control List and give the Everyone group the access level of Read (and do likewise with the contents of the bucket.) If you don't want the bucket contents list-able but do want to share the file within it, disable Read access for the Everyone group for the bucket itself, and then enable Read access for the individual files within the bucket.
- Prevent the contents of a bucket from being indexed by a search engine: If you don't want the contents of a bucket to be indexed by Google and company, place a file named Robots.txt in the bucket and share it with Everyone. (This trick works for any webserver.) The file needs to contain the following two entries to prevent indexing:
- User-agent: *
- Disallow: /
- Use your own domain name: You can use your own domain name (http://s3.carltonbale.com) instead of the default Amazon URL (http://s3.amazonaws.com/s3.carltonbale.com/). You just need to edit your DNS settings. Here are the details on how to do it. Bucket Explorer has an option to "Use bucket name as virtual host" to make sharing files in this manner even easier.
- Temporarily share a file using an auto-expire link: You many want to allow someone to download a file but prevent them from sharing the link and having 100's of people download it as well. With S3, it's possible to create a link that expires after a defined period of time expires. The easiest way to do this using Bucket Explorer. First, make sure that the file has permission set so that it is not readable by everyone, since temporary access is what you're going after in the first place. Then select the file, right-click, and select Generate Web URL. Select Signed URL and specify the expiration date. Then send the URL to the person with which you are sharing the file
- Share your bucket with a someone - whether they're an S3 user or not. The Third-party bucket features allows you to share the contents of a bucket. In Bucket Explorer, right-click on the bucket and select "Update bucket's access control list…" and set the permissions and add the other persons email address. See this documentation page for more details.
- Upload any file extension as an HTML file (to redirect to another location.) Say you have a picture shared on your account and it is being leeched by multiple, external sites. Instead of just deleting the picture, you can create a web page to replace it that directs people to your site. If you picture name was picture.jpg, create the HTML file and save it as it on your local computer. Then change the file name to be picture.jpg. Now if you just upload this file normally to S3, it will be treated as a jpg image and will not download properly. But you can use BucketExplorer to upload it as an HTML file, which specifically tells S3 to treat it as a HTML file and not a jpg file. Here is an example of picture that I converted to a HTML redirect: http://carltonbale.com.s3.amazonaws.com/distance_chart.png
- Store your media on Amazon S3 while running your WordPress blog on a different server. If you're like me, your web host has overly limited storage space restriction. If you have a bunch of small pictures or just a few large videos, it can quickly consume your available disk space. To combat this, you can use the Amazon S3 plugin for WordPress. Instead of uploading files to your web server, it instead uploads them to your Amazon S3 account and automatically links to them, all within the WordPress admin interface.
- Compare local and remote file versions using MD5 check sums. Wondering if version of a file on your local computer is the same as the one stored in your S3 account? Perform a MD5 check sum and see if the results match. On you local computer, you can use a program such as HkSFV to calculate the MD5 sum. To see the MD5 sum of the file on S3, right-click on it and select Properties within Bucket Explorer. If the sums match, the files are identical.
Those are all of the "hidden" features I've found, but I'm sure there are more. Please let me know of any features I've missed.
Posted in Web Development, Internet | No Comments »



