I am in the process of building a website that will distribute a large amount (>150GB) of small files (~25Mb Avg.) to a large number of people, since both bandwidth and disk space are expensive with my server company, and we are on a low budget, I'm thinking that we should go with Amazon's S3 service to host our content, are there any potential gotchas or specail precautions that should be taken when building a high demand website around Amazon's S3?
S3 is not technically a hosting service. Yes, you can put content up there but seemingly simple things that you may depend upon are not present. E.g.:
Another catch is that, no matter the bandwidth on the client side, the per-file transfer rate will be limited. (E.g.: Here in Europe, I measured 1-2mpbs from the US Amazon S3 service. We had some connection as bad as low 100s of kbps.) That said, cumulative bandwidth is effectively unlimited.
Have a look at CloudFront, which is Amazon's CDN that works with S3.
Note: I'd not call 150GB of data 'large', especially in the scope of Amazon's platform. It is peanuts for them.
The phrase "low budget" seems to be a red flag for me. Last I checked, Amazon S3 didn't have a way to limit the amount of $$$ you could be billed if a zillion people decide to check out your website.
You need to look at Amazon CloudFront.
Try the application from CloudBeryLabs to manage yoyur storage. There is a setting in there for CloudFront.
I am using it to host my deployment apps via Microsoft ClickOnce and I am getting excellent speed for all multiple downloads and updates. We are based in USA.