A time travel paradox in the title is a good place to start a blog post, don’t you think? You don’t yet know the things you need to know so you can’t wish you didn’t need to know them. There is a solution though – Read this blog post.
This all started because Plerion is trying to build a comprehensive risk model for the most severe data breaches occurring in AWS environments. At the top of the list is unauthorised access to S3 buckets. Here are some examples in the media.
As with many things AWS security, the more one digs into the details the more oddities one discovers. None of these oddities are new but they all get regularly rediscovered with great surprise. I thought I’d make a list to maybe lessen the surprise for people in the future, and probably future me as well.
S3 buckets are the S3 API
Amazon S3 was one of the first services released by AWS so its incredibly robust and well tested. It also means it has a history predating standardised design patterns, resulting in a quirky API relative to other services.
One of those quirks is that a relatively small part of the API requires HTTP requests to be sent to generic S3 endpoints (such as s3.us-east-2.amazonaws.com ), while the vast majority of requests must be sent to the URL of a target bucket. Under the hood it might all be the same but at least to the caller, this is how it works.
For example, to list the contents of a bucket, the HTTP request looks something like: