Automatically Manage your AWS EC2 Instance Public IP Addresses with Route53

Our Goal: Easy access to our Instances by Name instead to locate them through EC2 Console after an IP change caused by a stop/start action.

Is quite tedious the need to open the AWS Console to find an instance Public IP after a stop/start action or if we forgot which previously it was. Here I show you a tool that consists in a script executed inside the instance that updates its DNS records in Route53 using the instance Tag "Name". This is and optional Tag we can use to store the "Host Name" when launching a new instance or edit it anytime we need afterwards. If this optional tag is not present, the script I show you here, will use the instance ID to update (or create) the corresponding DNS A Record. This way we will have always the instance accessible through its FQDN and it will be stable (It won't change overtime).
Example: My-Instance-Host-Name.ec2.My-Domain.com

$ ssh -i juankeys.pem ec2-user@webserver1.ec2.donatecpu.com Last login: Mon Nov 12 00:14:35 2012 from 77.224.98.33        __|  __|_  )        _|  (     /   Amazon Linux AMI       ___|\___|___| https://aws.amazon.com/amazon-linux-ami/2012.09-release-notes/  There are 4 total update(s) available Run "sudo yum update" to apply all updates. [ec2-user@webserver1 ~]$

Instance Tag Name
Configure your EC2 instance with a Tag Name using the Console. Usually the Instance Launch Wizard will ask you for it but if is empty, you can update it any time you want. In this example the Tag Name will be "webserver1".

Preparations
Log into your instance and make sure that the EC2 API is ready to run. Follow this previous post if you need help with that. You will need a IAM user with admin permissions on Route53.


Route53
Create a new zone in Route53 (if you don't have any created yet) and save the assigned Hosted Zone ID:

dnscurl.pl
dnscurl.pl is an AWS Perl tool that will help you to use the Route53 API. Unlike other AWS APIs, Route53's API uses REST methods. This means that is accessible using HTTP calls (similar to accessing instance metadata) which looks good but the authentication process is a painful. dnscurl.pl simplifies the authentication process to generate the calls (GET and POST) to the Route 53 API.

Create a directory called /root/bin/ to store our tools, download dnscurl.pl, and make it executable:

# cd /root # mkdir bin # cd bin # wget -q http://awsmedia.s3.amazonaws.com/catalog/attachments/dnscurl.pl # chmod u+x dnscurl.pl

Note: You can also download the dnscurl.pl from here using a browser.

Create in the same folder a file called ".aws-secrets" (note the dot at the begining of the file name) with the following content and make it only readable for root:

%awsSecretAccessKeys = (     '(your key name without parentheses)' => {         id => '(your access key without parentheses)',         key => '(your secret key without parentheses)',      }, );

# chmod go-rwx .aws-secrets

Test dnscurl.pl with a simple read-only call. If everything is good, you should see something like this:

# ./dnscurl.pl --keyfile ./.aws-secrets --keyname juan -- -v -H "Content-Type: text/xml; charset=UTF-8" https://route53.amazonaws.com/2012-02-29/hostedzone/Z1F5BRDVBM                                                                            0.0% * About to connect() to route53.amazonaws.com port 443 (#0) *   Trying 72.21.194.53... * connected * Connected to route53.amazonaws.com (72.21.194.53) port 443 (#0) * Initializing NSS with certpath: sql:/etc/pki/nssdb *   CAfile: /etc/pki/tls/certs/ca-bundle.crt   CApath: none * SSL connection using SSL_RSA_WITH_RC4_128_MD5 * Server certificate: * subject: CN=route53.amazonaws.com,O=Amazon.com Inc.,L=Seattle,ST=Washington,C=US * start date: Nov 05 00:00:00 2010 GMT * expire date: Nov 04 23:59:59 2013 GMT * common name: route53.amazonaws.com * issuer: CN=VeriSign Class 3 Secure Server CA - G3,OU=Terms of use at https://www.verisign.com/rpa (c)10,OU=VeriSign Trust Network,O="VeriSign, Inc.",C=US > GET /2012-02-29/hostedzone/Z1F5BRDVBM HTTP/1.1 > User-Agent: curl/7.24.0 (x86_64-redhat-linux-gnu) libcurl/7.24.0 NSS/3.13.5.0 zlib/1.2.5 libidn/1.18 libssh2/1.2.2 > Host: route53.amazonaws.com > Accept: */* > Content-Type: text/xml; charset=UTF-8 > Date: Sun, 11 Nov 2012 23:21:26 GMT > X-Amzn-Authorization: AWS3-HTTPS AWSAccessKeyId=AKIAJ5,Algorithm=HmacSHA1,Signature=/i+0d= >  < HTTP/1.1 200 OK < x-amzn-RequestId: 843632ca-2c56-11e2-94bf-3b3ef9a8f457 < Content-Type: text/xml < Content-Length: 582 < Date: Sun, 11 Nov 2012 23:21:26 GMT <  <?xml version="1.0"?> * Connection #0 to host route53.amazonaws.com left intact <GetHostedZoneResponse xmlns="https://route53.amazonaws.com/doc/2012-02-29/"><HostedZone><Id>/hostedzone/Z1F5BRDVBM</Id><Name>donatecpu.com.</Name><CallerReference>454848C9-18D1-2DDB-AC24-B629E</CallerReference><Config/><ResourceRecordSetCount>2</ResourceRecordSetCount></HostedZone><DelegationSet><NameServers><NameServer>ns-1146.awsdns-15.org</NameServer><NameServer>ns-1988.awsdns-56.co.uk</NameServer><NameServer>ns-228.awsdns-28.com</NameServer><NameServer>ns-783.awsdns-33.net</NameServer></NameServers></DelegationSet></GetHostedZoneResponse>* Closing connection #0

You should see a correctly created AWSAccessKeyId and Signature, no error messages and at the bottom and XML output showing the DNS Servers for you Zone.


start-up-names.sh
Download my script start-up-names.sh and make it executable:

# wget -q http://www.domenech.org/files/start-up-names.sh  # chmod u+x start-up-names.sh

Or copy and paste the following text into a file called start-up-names.sh

#!/bin/bash # start-up-names.sh # http://blog.domenech.org logger start-up-name.sh Started #More environment variables than we need but... we always do that export AWS_CREDENTIAL_FILE=/opt/aws/apitools/mon/credential-file-path.template export AWS_CLOUDWATCH_HOME=/opt/aws/apitools/mon export AWS_IAM_HOME=/opt/aws/apitools/iam export AWS_PATH=/opt/aws export AWS_AUTO_SCALING_HOME=/opt/aws/apitools/as export AWS_ELB_HOME=/opt/aws/apitools/elb export AWS_RDS_HOME=/opt/aws/apitools/rds export EC2_AMITOOL_HOME=/opt/aws/amitools/ec2 export EC2_HOME=/opt/aws/apitools/ec2 export JAVA_HOME=/usr/lib/jvm/jre export PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/opt/aws/bin:/root/bin # *** Configure these values with your settings *** #API Credentials AWSSECRETS="/root/bin/.aws-secrets" KEYNAME="juan" #Hosted Zone ID obtained from Route53 Console once the zone is created HOSTEDZONEID="Z1F5BRDVBM" #Domain name configured in Route53 and used to store our server names DOMAIN="ec2.donatecpu.com" # *** Configuration ends here *** #Let's get the Credentials that EC2 API needs from .aws-secrets dnscurl.pl file ACCESSKEY=`cat $AWSSECRETS | grep id | cut -d\' -f2` SECRETKEY=`cat $AWSSECRETS | grep key | cut -d\' -f2` #InstanceID Obtained from MetaData INSTANCEID=`wget -q -O - http://169.254.169.254/latest/meta-data/instance-id` #Public Instance IP obtained from MetaData PUBLICIP=`wget -q -O - http://169.254.169.254/latest/meta-data/public-ipv4` #IP Currently configured in the DNS server (if exists) CURRENTDNSIP=`dig $INSTANCEID"."$DOMAIN A | grep -v ^\; | sort | tail -1 | awk '{print $5}'` #Instance Name obtained from the Instance Custom Tag NAME WGET="`wget -q -O - http://169.254.169.254/latest/meta-data/instance-id`" INSTANCENAME=`ec2-describe-instances -O $ACCESSKEY -W $SECRETKEY $WGET --show-empty-fields | grep TAG | grep Name | awk '{ print $5 }'` echo $INSTANCEID $PUBLICIP $CURRENTDNSIP $INSTANCENAME logger $INSTANCEID $PUBLICIP $CURRENTDNSIP $INSTANCENAME #Set the new Hostname using the Instance Tag OR the Instance ID if [ -n "$INSTANCENAME" ]; then hostname $INSTANCENAME logger Hostname from InstanceName set to $INSTANCENAME else hostname $INSTANCEID logger Hostname from InstanceID set to $INSTANCEID fi #dnscurl.pl Delete Current InstanceID Public IP A Record to allow Later Update COMMAND="<?xml version=\"1.0\" encoding=\"UTF-8\"?><ChangeResourceRecordSetsRequest xmlns=\"https://route53.amazonaws.com/doc/2012-02-29/\"><ChangeBatch><Changes><Change><Action>"DELETE"</Action><ResourceRecordSet><Name>"$INSTANCEID"."$DOMAIN".</Name><Type>A</Type><TTL>600</TTL><ResourceRecords><ResourceRecord><Value>"$CURRENTDNSIP"</Value></ResourceRecord></ResourceRecords></ResourceRecordSet></Change></Changes></ChangeBatch></ChangeResourceRecordSetsRequest>" /root/bin/dnscurl.pl --keyfile $AWSSECRETS --keyname $KEYNAME -- -v -H "Content-Type: text/xml; charset=UTF-8" -X POST https://route53.amazonaws.com/2012-02-29/hostedzone/$HOSTEDZONEID/rrset -d "$COMMAND" #dnscurl.pl Create InstanceID Public IP A Record COMMAND="<?xml version=\"1.0\" encoding=\"UTF-8\"?><ChangeResourceRecordSetsRequest xmlns=\"https://route53.amazonaws.com/doc/2012-02-29/\"><ChangeBatch><Changes><Change><Action>"CREATE"</Action><ResourceRecordSet><Name>"$INSTANCEID"."$DOMAIN".</Name><Type>A</Type><TTL>600</TTL><ResourceRecords><ResourceRecord><Value>"$PUBLICIP"</Value></ResourceRecord></ResourceRecords></ResourceRecordSet></Change></Changes></ChangeBatch></ChangeResourceRecordSetsRequest>" /root/bin/dnscurl.pl --keyfile $AWSSECRETS --keyname $KEYNAME -- -v -H "Content-Type: text/xml; charset=UTF-8" -X POST https://route53.amazonaws.com/2012-02-29/hostedzone/$HOSTEDZONEID/rrset -d "$COMMAND" logger Entry $INSTANCEID.$DOMAIN sent to Route53 #Create DNS A record for Instance Name (if exists) if [ -n "$INSTANCENAME" ]; then #dnscurl.pl Delete Current Instance Name Public IP A Record to allow Later Update COMMAND="<?xml version=\"1.0\" encoding=\"UTF-8\"?><ChangeResourceRecordSetsRequest xmlns=\"https://route53.amazonaws.com/doc/2012-02-29/\"><ChangeBatch><Changes><Change><Action>"DELETE"</Action><ResourceRecordSet><Name>"$INSTANCENAME"."$DOMAIN".</Name><Type>A</Type><TTL>600</TTL><ResourceRecords><ResourceRecord><Value>"$CURRENTDNSIP"</Value></ResourceRecord></ResourceRecords></ResourceRecordSet></Change></Changes></ChangeBatch></ChangeResourceRecordSetsRequest>" /root/bin/dnscurl.pl --keyfile $AWSSECRETS --keyname $KEYNAME -- -v -H "Content-Type: text/xml; charset=UTF-8" -X POST https://route53.amazonaws.com/2012-02-29/hostedzone/$HOSTEDZONEID/rrset -d "$COMMAND" #dnscurl.pl Create Instance Name Public IP A Record COMMAND="<?xml version=\"1.0\" encoding=\"UTF-8\"?><ChangeResourceRecordSetsRequest xmlns=\"https://route53.amazonaws.com/doc/2012-02-29/\"><ChangeBatch><Changes><Change><Action>"CREATE"</Action><ResourceRecordSet><Name>"$INSTANCENAME"."$DOMAIN".</Name><Type>A</Type><TTL>600</TTL><ResourceRecords><ResourceRecord><Value>"$PUBLICIP"</Value></ResourceRecord></ResourceRecords></ResourceRecordSet></Change></Changes></ChangeBatch></ChangeResourceRecordSetsRequest>" /root/bin/dnscurl.pl --keyfile $AWSSECRETS --keyname $KEYNAME -- -v -H "Content-Type: text/xml; charset=UTF-8" -X POST https://route53.amazonaws.com/2012-02-29/hostedzone/$HOSTEDZONEID/rrset -d "$COMMAND" logger Entry $INSTANCENAME.$DOMAIN sent to Route53 fi logger start-up-names.sh Ended

Edit the script and adapt the variables from the "*** Configure these values with your settings ***" section with your parameters.

Test it:

# ./start-up-names.sh (text output) # tail /var/log/messages Nov 11 23:30:57 ip-10-29-30-48 ec2-user: start-up-name.sh StartedNov 11 23:30:59 ip-10-29-30-48 ec2-user: i-87eef4e1 54.242.191.68 ns-1146.awsdns-15.org. webserver1 Nov 11 23:30:59 ip-10-29-30-48 ec2-user: Hostname from InstanceName set to webserver1 Nov 11 23:31:00 ip-10-29-30-48 ec2-user: Entry i-87eef4e1.ec2.donatecpu.com sent to Route53 Nov 11 23:31:00 ip-10-29-30-48 ec2-user: Entry webserver1.ec2.donatecpu.com sent to Route53 Nov 11 23:31:00 ip-10-29-30-48 ec2-user: start-up-names.sh Ended

Reading /var/log/messages you should have something like this output. First the script gathers the Instance ID and the Public IP reading the Instance Metadata. Then the current IP ($CURRENTDNSIP) configured at the DNS (if any) using dig and the Instance Tag Name using the ec2-describe-instances command. The first change to happen is the Host Name. If the Instance Tag Name is present it will become the machine Host Name and if not, the Instance ID will play this role. One way or the other we will have a stable way to identify our servers. The Instance ID is unique and won't change over time. Then we call the Route53 API using dnscurl.pl four times. There is no API call to "overwrite" and existing DNS record so we need to Delete it first and Create it afterwards. The Delete call has to include the exact values the current entry has (quite silly if you ask me...) so that is why the scripts needs the current Public IP configured. We Delete using the old values and Create using the new ones. One dnscurl execution for the Instance ID (that always exists) and again for the Instance Tag Name (if present).

Two entries should have been automatically created in your Hosted Zoned and present at Route53 console for our Instance:

Those entries are ready to use and now you can forget its Instance ID or volatile Public IP and just ping or ssh to the name. Example: webserver1.ec2.donatecpu.com.


Auto Start
The main purpose is to maintain our servers IPs automatically updated in our DNS so we need that the main script is executed every time the machine starts. Once we've verified that it works fine is time to edit /etc/rc.local and add start-up-names.sh full path to it:

#!/bin/sh # # This script will be executed *after* all the other init scripts. # You can put your own initialization stuff in here if you don't # want to do the full Sys V style init stuff. touch /var/lock/subsys/local /root/bin/start-up-names.sh

And that is it. I suggest you to manually stop and start your instance and verify that its new assigned Public IP is updated in the DNS. All AMIs you generate from this Instance will include this described  configuration and therefore they will dynamically maintain their IPs. Cool!

Note: When playing with changes in DNS Records their TTL value matters. In this exercise we've used a value of 600 seconds so a change could take up to 10 minutes to be available in your local area network if your DNS server has cached it.