Optimizing definition of inbound rules for security groups in AWS


11 Apr 2021


In a perfect world, we should always protect the network by avoiding unnecessary open network ports. Protecting a resource (such as an EC2 instance) in AWS can be done by using a security group, which acts as a virtual firewall to control incoming and outgoing traffic.

However, there are cases where defining the inbound rules of a security group can be challenging and a compromise is needed. Some applications may use thousands of ports; the Ops team may have limited documentation on what ports are actually needed and should be accessible. We also have to consider certain limits regarding security groups (e.g. default limit of 60 maximum inbound rules per security group).

Therefore, there is a need to better understand the application requirements and identify what ports are being used by a certain workload. This is where we can leverage VPC Flow Logs. It is a feature that captures logs about the IP traffic going to and from network interfaces in a VPC. Flow logs data can be published to Amazon CloudWatch Logs or to Amazon S3. You can create a flow log for a VPC, a subnet, or a network interface. Here is an example of a flow log record :

2 123456789010 eni-1235b8ca123456789 20641 22 6 20 4249 1418530010 1418530070 ACCEPT OK

In this example, we can read that SSH traffic (destination port 22, TCP protocol) to network interface eni-1235b8ca123456789 in account 123456789010 was allowed.

VPC Flow Logs Analyzer is a little experiment that I worked on (an open source tool written in Python). It analyzes a VPC Flow Logs and suggest a set of optimized port ranges and individual ports that will cover all the source ports used for a specific ENI (Network interface). This result can serve as a base to create the inbound rules of your security groups.

I experimented two different implementations. The first leverages Amazon Athena, which is an interactive query services that makes it easy to analyze data in Amazon S3 using standard SQL. The second implementation uses Amazon CloudWatch Logs, which can centralize logs from different systems and AWS services; it can be queried using CloudWatch Logs Insights.

Let’s see it in action

Let’s assume that the VPC Flow Logs returned the following source port used by a specific ENI :


Running the tool with maxInboundRules of 3 and maxOpenPorts of 3 (maximum number of open ports left within a single range or inbound rule) gives the following result :

ranges : 
- from 80 to 86
- from 1001 to 1002

individual ports :
- 1020

extra/unused ports (ports that are open and shouldnt) :
- 83, 84

In the previous example, we end up with two ranges and one individual port. There are 2 ports that would be open and shouldn’t (ports 83 and 84). This is a compromise that we may decide to accept if we want to only have a maximum of 3 inbound rules defined in the security group.

If we decide to run the optimizer with maxInboundRules = 10 and maxOpenPorts of 3, we get the following result :

ranges : 
- from 80 to 82
- from 85 to 86
- from 1001 to 1002

individual ports :
- 1020

extra/unused ports (ports that are open and shouldnt) :
- (none)

In this case, the optimizer is suggesting three ranges and one individual port. This means a total of four inbound rules (which respects our constraints of being below 10 rules). There is no extra port left open.

If we decide to run the optimizer with maxInboundRules = 1 and maxOpenPorts of 3, we get the following result :

Couldnt find a combination - you may want to consider increasing 
the values for maxInboundRules and/or maxOpenPorts

The optimizer has no magic power (at least yet) and is simply unable to come up with a solution with those constraints.

If we decide to run the optimizer with maxInboundRules = 1 and maxOpenPorts of 1000, we get the following result :

ranges : 
- from 80 to 1020

individual ports :
- (none)

extra/unused ports (ports that are open and shouldnt) :
- 83, 84, 87, 88, 89, 90, ..., ..., 999, 1000, ..., ..., 1018, 1019

In this example, the optimizer is giving one single range (as requested). However, there will be 933 ports left open that shouldn’t (which is below the value of maxOpenPorts of 1000).

What’s next?

We saw different ways to leverage and query VPC Flow Logs, using Amazon Athena and AWS CloudWatch Logs Insights. It becomes quite easy to identify traffic usage and build tools such as the vpc-flowlogs-analyzer to better understand the network requirements of a specific workload.

From a cost perspective, you may want to consider the following to keep your cost low:

I believe there is value to run such analysis to improve the security posture of a workload running on AWS. I am already thinking of some of the enhancements, such as :

  • Analyzing VPC flow logs along with existing security groups inbound rules and trying to come up with recommendations (e.g. there is no traffic on port X, however, security_group_1 has port X opened).
  • Analyzing existing security groups and provide recommendation to merge some of them or reduce their numbers (less code is always better).


Source code and documentation can be found here : https://github.com/alfallouji/AWS-SAMPLES/tree/master/vpc-flowlogs-analyzer

Be Sociable, Share!

Comment Form

Who am I?

My name is Bashar Al-Fallouji, I work as a Enterprise Solutions Architect at Amazon Web Services.

I am particularly interested in Cloud Computing, Web applications, Open Source Development, Software Engineering, Information Architecture, Unit Testing, XP/Agile development.

On this blog, you will find mostly technical articles and thoughts around PHP, OOP, OOD, Unit Testing, etc. I am also sharing a few open source tools and scripts.

  • Trinzia: Well done, my friend! [...]
  • vivek raj: Hello Bashar, It's really good that you wrote this code. but I'm confused some part. can you suppor [...]
  • irfan: I saw watch your youtube talk on clean and testable code. By the way very good talk. I was wondering [...]
  • Mohamed: Hello bashar, I hope you are doing well. Thank you for your hard work, and thank you for sharing [...]
  • alex davila: Hi Bashar is there any pick up example?? Regards Alex Davila [...]