In How S3 Buckets Become Public, and the Fastest Way to Find Yours we reviewed the myriad of ways S3 buckets become public and where to look for them. Today I’ll show the easiest way to continuously monitor for public buckets using AWS Config. The good news is this is pretty easy to set up, the bad news is you need to configure it separately in every region in every account.
One of the advantages of using config is that it leverages Amazon’s Zelkova capability. This is a behind the scenes machine learning tool that can detect effectively every possible way a bucket can become public. Config is one of the few ways today that we can access this tool, which is also what decide if that yellow “Public” flag shows for an S3 bucket in the console.
Turning on Config
Config needs to be turned on separately for each region. For our examples we will show you what it looks like in Ohio. The first step is to navigate to config in the console and you will see the splash screen if you have not previously set it up in this region.
The next part is slightly annoying. If this is the first time you have set it up in any of your regions, the default settings will work fine. If not, you will probably want to choose an existing bucket and IAM role. Also, you should only select the global resources option once within any given account. We do not need that for our S3 example, but it is useful if you also want to use config for IAM monitoring.
Next, select the public read and public write rules for S3. If you type S3 into the search box it makes it a lot easier to find them.
The rest is merely clicking through the default options. The one exception is if you get an error message. If so it’s likely that you already have an S3 bucket with that name or previously created the Config service linked IAM role.
If so, when you go back it will now ask you to use the existing role. Don’t ask me why AWS doesn’t ask you the first time, the more you work with the console the more you just get used to these… quirks. Once everything is up and running your dashboard starts discovering and checking resources. Keep in mind that AWS charges you pre-resource per-rule in Config! Check the pricing page for the region if you have a lot in there.
Recently in a training class I had a student turn on ALL the recommended Config Rules. Most students racked up about $3.00 in charges for a 2 day training, but he was over $30.00 due completely to the high number of Config rules that got turned on. Amazon recently moved to tiered pricing to better manage costs for larger customers.
Your dashboard starts blank, but will then show findings as Config rolls through the region. This:
Turns into this, but only if you remember to refresh the page:
How it works, converting to a guardrail, and limitations
Config is now using Zelkova to evaluate the overall combination of your Bucket ACLs, Policies, and other settings to determine if the bucket is public. If so, Config will flag it. The default setting we used has Config evaluate the buckets on every resource change, so it isn’t only running on a schedule. That doesn’t mean it’s real time, but it should be pretty decent and keep your exposure windows to a minimum.
Except… right now we have a console to find bad things, but it isn’t really a guardrail.
For a simple guardrail the next step is to turn on notifications. This is built into Config, and we actually skipped that step during our initial configuration to keep this post concise. You have two options, both enabled under Config -> Settings.
If you stream SNS that includes all changes on all monitored resources. So, er, don’t send this to your email or phone. This is good for a SIEM and the SNS message will include an indicator if a resources is non-compliant that you can filter on.
What might be better if you want more-granular notifications or to tie things right to email/text message is to build a rule over in CloudWatch (full details are fodder for a future post). CloudWatch is where you will also build rules to hook into auto-remediation with Lambda. We will cover this more in-depth in future posts, but if you go into CloudWatch Rules and basically replicate these settings you can trigger, in near real time, either a notification (set SNS as the target) or remediation (set Lambda as the target).
And that’s all it takes to build a notification guardrail with Config. This does have some major limitations. The biggest is that you need to replicate this in each region (even for S3, which likes to pretend it isn’t region based). IAM is really the only global service you can build a single Config rule to monitor. Also, we skipped filtering down so that, for example, you only include buckets tagged a certain way. You can further scope down a Config Rule with names and tags in the settings, but it is tough to scale that out.
Needless to say, this process is tougher to scale as you add AWS accounts and regions, which is where automation comes into play. Infrastructure as code (CloudFormation/Terraform) can build the rules out across accounts and regions, or you can use Open Source or third party commercial tools (you know, like us) to scale more smoothly and consistently while getting more-granular filtering and more-flexible handling options.
Our objective today was to show you how to build a quick and dirty guardrail with Config, although we lied a bit and you also need to add in CloudWatch to make use of it. In the next post we will show you how to aggregate Config across regions or accounts for a unified view.