Security September: Fun with Fn::Cidr

This is the first of a 5-part series on AWS exploits and similar findings discovered over the course of 2020. All findings discussed in this series have been disclosed to the AWS security team and had patches rolled out to all affected regions, where necessary. A big thanks to my friend and fellow Australian Aidan Steele for co-authoring this series with me. Check out parts 2 and 4 for his work!

This first part covers an issue which could lead to a denial-of-service on some CloudFormation actions within a single AWS account using the Fn::Cidr CloudFormation intrinsic function.

How Fn::Cidr works

Fn::Cidr (also written as !Cidr) is an intrinsic function within CloudFormation that allows template authors to have an array of CIDR address block returned, usually used during programmatic creation of VPC structures where the base CIDR range is dynamic. The function handles both IPv4 and IPv6 range calculations.

After generating this for a VPC CIDR, developers would typically use Fn::Select to choose an appropriate subset of the base CIDR range for the creation of VPC subnets.

Breaking things

During a time when I was debugging a VPC creation template, I used CloudFormation outputs to determine the expected values from an IPv6 range. I noticed that the length of the output was very long and was curious to see what the limits of this field was. I checked the published limits and didn’t find what I was looking for, so decided to find out directly. I slowly incremented the field until finally I ended up consuming the full 450kb available to me within the template, which looked like this:

AWSTemplateFormatVersion: "2010-09-09"
Description: A very long CFN template
Resources:

    VPC:
        Type: AWS::EC2::VPC
        Properties:
            CidrBlock: 10.0.0.0/24

    VPCCidrBlockIpv6:
        Type: AWS::EC2::VPCCidrBlock
        Properties:
            VpcId: !Ref VPC
            AmazonProvidedIpv6CidrBlock: true

Outputs:

    VeryLongOutput:
        Value: !Join ["##", [
            !Join ["::", !Cidr [!Select [0, !GetAtt VPC.Ipv6CidrBlocks], 256, 64] ],
            !Join ["::", !Cidr [!Select [0, !GetAtt VPC.Ipv6CidrBlocks], 256, 64] ],
            
            ...

            !Join ["::", !Cidr [!Select [0, !GetAtt VPC.Ipv6CidrBlocks], 256, 64] ]
        ]]

In this template, I am creating a simple VPC and generating an output using Fn::Cidr and Fn::Join that is as long as possible. At the time, my rough calculations was that this would generate an output string roughly 30MB long.

As I watched the stack creation in the management console, I suddenly stopped receiving updates. I refreshed the page and saw that my stack refused to show any updates or the output I had constructed.

I moved over to the CLI and entered:

aws cloudformation describe-stacks --stack-name mystack

and waited. The CLI didn’t return any results and simply hanged. I tried the following command:

aws cloudformation describe-stacks

and had the same issue. The same thing happened when I used a different user. I realized I had found a way to deny the ability to describe stacks to other users so decided to raise this with the AWS security team.

Disclosure and Remediation

As this was effectively a denial-of-service to some stack actions within the containing account, this issue was raised with the AWS security team via email. Additionally, as I am a threat to friends with some leaders of the CloudFormation team, I was able to connect with them directly to work through the issue.

Once patched, the following error occurred when attempting to replicate the issue:

Essentially, they’ve placed a limit on the size any generated outputs and will rollback the stack if this limit is exceeded.

Closing out

This issue wasn’t always present within CloudFormation. I believe this issue comes as a result of an assumption made (which was correct for a certain time) that a users raw template had limits on it that meant the size of the outputs data structures and/or API responses would not exceed a certain value. As new features came along, like the introduction of Fn::Cidr, these assumptions no longer held true. It’s a reminder to us developers that new features sometimes have unintended consequences and we need to consider all of the previous assumptions made to see if they still hold.

Thanks to the AWS security and CloudFormation team members who worked with me to help remediate this issue.