AWS AutoScaling Group & dynamic Route53 records

Dynamic DNS for AWS AutoScaling Group

AutoScaling group and Route53 record

When you are deploying your application on AWS, you might want to use an AutoScaling Group to launch your EC2 instances. It groups logically the instance and has a lot of feature such as auto healing, scalability according to metrics, load balancer integration, etc. A good ptractice is to always run your instance in such group. Everything seems great with AutoScaling Group, except when you want to have a DNS record pointing on your instances. With a single EC2, it is possible to automatically add a Route53 record pointing to the instance with CloudFormation. But if you use an AutoScaling Group, AWS does not provide an automatic method to add each instance IP to a Route53 record. One well known workaround is to deploy an Elastic Load Balancer in front of the ASG and then add a alias record pointing to the ELB generated domain name. It works in most of the cases but sometimes you don’t really need or want to have a load balancer in front if your instances. For instance if you deploy a Kafka cluster on an ASG, you don’t want a load balancer because Kafka is designed to distribute traffic between brokers itself. n this blog post I will present a solution to solve this problem, using a Lambda function.

AutoScaling Group dynamic DNS Lambda function

The AutoScaling Group will be configured to send a notification to a SNS topic when an instance is created or terminated. A Lambda function called awg_ddns will be triggered by those notifications and will update the Route53 record accordingly. This Lambda function will query the Amazon EC2 API to get the IP of the instances in the AutoScaling Group then it will update the Route53 record.

The code can be found on GitHub.

Usage

To use this lambda function, you will first need to deploy an AutoScaling group on your AWS account and a hosted zone with the domain name you want. You will need to have Python 3, Terraform and the AWS CLI installed with an access key configured. Then you can run make command to package the function in a zip file and deploy the SNS topic and Lambda function the required IAM role. A prompt will appear asking for the domain name you want to assign to the instances and for the hosted zone id.

make

Then, get the SNS topic ARN and the AutoScaling group name, you can test that the function is working using the test.sh script.

./test.sh launch <sns_topic_arn> <asg_name>

Check the Route53 hosted zone, the DNS record should appear with the instance IP. You can run the previous command with terminate instead of launch to remove the DNS record.

If the previous step is working, you can setup the AutoScaling group to send notification to the SNS topic on instance creation or termination. Once done, you can try to change the desired instance number of the ASG to check that the DNS record is being updated.

Alternative solution

An alternative design can be to configure a CloudWatch Event to trigger the Lambda if CloudTrail is receiving a CreateInstance API call. Or the Lambda function can be triggered by a lifecycle hook set on the AutoScaling Group. Another alternative solution that works without having to deploy anything more than the ASG can be to put the Lambda code as a script inside each instances and call it in the user-data script when the instance is booting. But this solution does not work if we loose an instance unexpectedly since it is the instance that update the Route53 record. In this case your record will be in an inconsistent state until a new instance is launched. And it will require to give the instance the permission to change Route53 records, with the Lambda solution, only the lambda has this permission.

At the end, all of those solutions are valid. I chose the one with Lambda and SNS but you might prefer one of the other, to each his own. I have been running this Lamdba every time I needed a dynamic DNS for an ASG for quite some time now and it never let me down. I prefer this than using ELB, it cost more and adds a middleman so more complexity and potential point of failure, I don’t think it is a good solution when you only want a domain name for your instances.

If you have a better solution for this, don’t hesitate to hit me up ! You can open an issue on Github if you encounter one or even contribute with a Pull Request.