I recently worked with a customer who wanted to report on AWS flow logs within Scrutinizer. Scrutinizer requires only a bit of information in order to report on AWS flow logs; however, an S3 bucket needs to be configured with the right permissions within AWS. We walked through the process of creating an S3 bucket and the type of reports that can be viewed from within Scrutinizer. In this blog, I’ll share what that process is like.
Creating the S3 bucket
Within AWS, navigate to “Services” and select “S3 bucket.” At the top of the screen you’ll see the prompt to create an S3 bucket.
Select “Create Bucket.”
Now name the bucket and select the region.
Take note of the name and the region you select, as this will be the name and region you’ll use in the Scrutinizer settings.
Once done, click “Next.”
On the following screen, you’ll be asked to configure your preferred type of logging and to add any tags you might need.
Once done, click “Next” and you will be brought to permissions.
Select the permissions you would like the S3 bucket to have and click “Next” when done.
Take a moment to review the configuration of your S3 bucket, then select “Create bucket.”
Create the flow log
Now that the S3 bucket has been created, we’ll create the flow logs. From within AWS, navigate to Services > VPC > Your VPCs and select the VPC you want to monitor. From here, select the “Flow Logs” tab and select “Create flow log.”
When creating the flow log, select “Accept” as the filter. For the destination, select “Send to an S3 bucket.” In the “S3 bucket ARN” field, specify the name of the S3 bucket that we just created.
Use arn:aws:s3::: as the prefix to the S3 bucket’s name.
Make sure the format is “AWS default format” and select “Create.”
Create the AWS user
Now we’ll create an AWS user and grant them AmazonS3FullAccess permissions to the S3 bucket. From within AWS, navigate to Services > IAM > Users and select “Add User.”
From here, create the username and set the access type to “Programmatic access.”
Once complete, select “Next.”
Here we’ll need to set permissions. Choose “Attach existing policies directly” and search for “AmazonS3FullAccess.” Choose the “AmazonS3FullAccess” permission and select “Next.”
It’s worth noting that with AmazonS3FullAccess permissions, flow logs will be deleted from the S3 bucket as Scrutinizer ingests them.
On the following screen, add any tags that you have and select “Next.”
Review your current settings and when ready, select “Create User.”
This will display your access key ID and your secret access key. Copy these fields to later add them to the Scrutinizer integration.
Scrutinizer integration
Now that we created the S3 bucket, set permissions, and created the user, we’ll configure Scrutinizer to ingest the logs.
From Scrutinizer navigate to Admin > Settings > AWS Flowlogs S3. Select the “Add” option.
In the “Bucket” field, enter the name of the S3 bucket we just created. The name of the S3 bucket will be used as the name of the exporter in Scrutinizer. Next, use the Access Key ID and your Secret Access Key that we copied early as the “AWS access key ID” and “AWS secret access key.”
Select “Test.” If we’ve answered everything correctly, the test should be successful.
Reporting
After a few minutes, you should see your S3 bucket display as a new exporter in Scrutinizer. From here, you can begin pulling reports and monitoring AWS with NetFlow.
This type of reporting will show you if an action was accepted or rejected and the sources responsible. You can then filter the report on individual sources or actions to narrow down a particular event to report on. Curious about your own traffic? Review your AWS data in Scrutinizer.