Wednesday, April 20, 2016

Outbound IPv6 on AWS EC2 Amazon Linux How-To (Not Production Grade)




It is well known that we can deploy an application in AWS and be fully IPv6 compliant thanks to the AAAA DNS records that every EC2 Elastic Load Balancer have at our disposal, but this does not apply to Outbound Internet connections (connections that are originated in our EC2 boxes).
The arrival of IPv6 to EC2 could be near but meanwhile there is a way to provide outbound IPv6 connectivity to our servers thanks to Hurricane Electric tunnel broker service.

I call this solution "Not Production Grade" because it is provided for free for experimentation purposes. Please read the Terms of Service (I have to say that is pretty fast and stable though).



Important Security Note:
With no additional measures in place, the configuration described here will open your TCP/IP services to Internet. Deploying a TCP tunnel will bypass the EC2 Security Group security layer.
IPv6 has no Network Address Translation (NAT) and your server will be directly connected to Internet to all effects.
Enabling and configuring ip6tables is advised.




Register:

- Get your free IPv6 tunnel at https://www.tunnelbroker.net




- Open your EC2 Security Group to receive ICMP traffic from Hurricane Electric (This is a requisite for this tunnel provider).




- Fill the field "IPv4 Endpoint (Your side)" with the Public IP of your instance.

- Select an IPv4 tunnel endpoint close to your AWS region.




- Once the tunnel is created we can access its details. No other changes are required, the tunnel is ready to use.




Configure:

- Click on "Example Configurations" to obtain the configuration guidelines for our Operative System (In our case: "Linux-net-tools" option).




Important Security Note:
With no additional measures in place, the configuration described here will open your TCP/IP services to Internet. Deploying a TCP tunnel will bypass the EC2 Security Group security layer.
IPv6 has no Network Address Translation (NAT) and your server will be directly connected to Internet to all effects.
Enabling and configuring ip6tables is advised.


(with sudo)


sudo ifconfig sit0 up
sudo ifconfig sit0 inet6 tunnel ::216.66.88.98
sudo ifconfig sit1 up
sudo ifconfig sit1 inet6 add 2001:470:1f1c:666::2/64
sudo route -A inet6 add ::/0 dev sit1


Note: In your case these IP addresses will vary.

- At this point the new interface and the tunnel are ready.



Test:

- Check our new interface sit1 and its IPv6 configuration. In this example the IP 2001:470:1f1c:666::2 is our Public IPv6 address for this server.


$ ifconfig sit1

sit1      Link encap:IPv6-in-IPv4  
          inet6 addr: 2001:470:1f1c:666::2/64 Scope:Global
          inet6 addr: fe80::a52:b404/64 Scope:Link
          UP POINTOPOINT RUNNING NOARP  MTU:1480  Metric:1
          RX packets:111 errors:0 dropped:0 overruns:0 frame:0
          TX packets:110 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1 
          RX bytes:14575 (14.2 KiB)  TX bytes:11293 (11.0 KiB)




- ping6 (against Google IPv6 DNS server)


$ ping6 -c 5 2001:4860:4860::8888

PING 2001:4860:4860::8888(2001:4860:4860::8888) 56 data bytes
64 bytes from 2001:4860:4860::8888: icmp_seq=1 ttl=56 time=18.9 ms
64 bytes from 2001:4860:4860::8888: icmp_seq=2 ttl=56 time=19.0 ms
64 bytes from 2001:4860:4860::8888: icmp_seq=3 ttl=56 time=19.0 ms
64 bytes from 2001:4860:4860::8888: icmp_seq=4 ttl=56 time=19.1 ms
64 bytes from 2001:4860:4860::8888: icmp_seq=5 ttl=56 time=19.0 ms

--- 2001:4860:4860::8888 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 4034ms
rtt min/avg/max/mdev = 18.993/19.062/19.125/0.131 ms



- route


$ route -n -A inet6

Kernel IPv6 routing table
Destination                     Next Hop                                Flags Metric Ref    Use Iface
::/96                              ::                                      U     256    0        0 sit0
2001:470:1f1c:666::/64             ::                                      U     256    0        0 sit1
fe80::/64                          ::                                      U     256    0        0 eth0
fe80::/64                          ::                                      U     256    0        0 sit1
::/0                               ::                                      U     1      3469     1 sit1
::1/128                            ::                                      U     0      22       2 lo
::10.82.180.4/128                  ::                                      U     0      0        1 lo
::127.0.0.1/128                    ::                                      U     0      0        1 lo
2001:470:1f1c:666::2/128           ::                                      U     0      3397     2 lo
fe80::a52:b404/128                 ::                                      U     0      0        1 lo
fe80::2000:aff:fe52:b404/128       ::                                      U     0      0        1 lo
ff00::/8                           ::                                      U     256    0        0 eth0

ff00::/8                           ::                                      U     256    0        0 sit1


::/0 is the Default route in IPv6 (equivalent to 0.0.0.0/0 in IPv4).
::1 host is our localhost interface (equivalent to 127.0.0.1). 

In IPv6 one or more leading zeroes from any groups of hexadecimal digits are removed and consecutive sections of zeroes are replaced with a double colon (::). 
This 0000:0000:0000:0000:0000:0000:0000:0001 is equal to ::1


- netstat


$ telnet www.google.com 80 &
$ netstat -nat -A inet6

Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address               Foreign Address              State      
tcp        0      0 :::22                       :::*                         LISTEN      
tcp        0      0 :::38030                    :::*                         LISTEN      
tcp        0      0 :::111                      :::*                         LISTEN      
tcp        0      0 2001:470:1f1c:666::2:35960  2a00:1450:400b:802::2004:80  ESTABLISHED 


Notice that our telnet command has created an IPv6 connection. We didn't specify any IPv6 parameter. How come? More about that later. Check DNS Considerations at the bottom of this article to know more.



- traceroute


$ traceroute -6 2001:4860:4860::8888

traceroute to 2001:4860:4860::8888 (2001:4860:4860::8888), 30 hops max, 80 byte packets
 1  juandomenech-1.tunnel.tserv1.lon2.ipv6.he.net (2001:470:1f1c:666::1)  13.331 ms  13.703 ms  14.143 ms
 2  ge3-20.core1.lon2.he.net (2001:470:0:320::1)  42.622 ms  75.028 ms  75.002 ms
 3  2001:7f8:4::3b41:1 (2001:7f8:4::3b41:1)  15.652 ms  15.614 ms  15.386 ms
 4  2001:4860::1:0:ab9e (2001:4860::1:0:ab9e)  14.648 ms 2001:4860::1:0:ab9d (2001:4860::1:0:ab9d)  14.697 ms 2001:4860::1:0:9914 (2001:4860::1:0:9914)  15.386 ms
 5  2001:4860::8:0:aba0 (2001:4860::8:0:aba0)  26.167 ms 2001:4860::8:0:ab9f (2001:4860::8:0:ab9f)  26.139 ms 2001:4860::8:0:aba0 (2001:4860::8:0:aba0)  26.113 ms
 6  2001:4860::8:0:83d2 (2001:4860::8:0:83d2)  26.079 ms 2001:4860::8:0:507c (2001:4860::8:0:507c)  18.276 ms 2001:4860::8:0:83d2 (2001:4860::8:0:83d2)  17.891 ms
 7  2001:4860::2:0:7a79 (2001:4860::2:0:7a79)  18.432 ms 2001:4860::2:0:79fb (2001:4860::2:0:79fb)  20.376 ms  20.369 ms

 8  google-public-dns-a.google.com (2001:4860:4860::8888)  19.208 ms  18.261 ms  17.545 ms



Notice Hop#1. It is the other site of the tunnel. The address :666::1 is the gateway of our network. 




- Type in your IPv6 Address, hit Submit and wait for 10 seconds. 

Do you see something interesting? Yes, as mentioned before, the ports 22 and 111 are open to the network over IPv6 bypassing the security provided by the EC2 Security Groups. 
Creating a TCP/IP tunnel has the same effect as adding another Internet connection to our instance. That traffic is encapsulated over TCP/IP and is out of control of the traditional EC2 Security Group firewall layer. 
Configuring ip6tables is advised.



DNS considerations:

We have added new interfaces to this box and we are routing IPv6 through a tunnel but we haven't changed its DNS configuration. It has the standard EC2 DNS configuration unchanged (EC2-Classic):



$ cat /etc/resolv.conf

; generated by /sbin/dhclient-script
search eu-west-1.compute.internal
options timeout:2 attempts:5
nameserver 172.16.0.23



Despite that, our previous telnet (telnet www.google.com 80) is connecting to Google's IPv6. Let's take a look to the dialog between our box and the EC2-Classic DNS server 172.16.0.23 to understand why:



$ sudo tcpdump -i eth0 -nn -s0 -A port 53

13:31:52.961142 IP 10.104.229.189.47624 > 172.16.0.23.53: 61300+ A? www.google.com. (32)
E..<.c@...1.
h.........5.(...t...........www.google.com.....
13:31:52.961158 IP 10.104.229.189.47624 > 172.16.0.23.53: 48977+ AAAA? www.google.com. (32)
E..<.d@...0.
h.........5.(...Q...........www.google.com.....
13:31:52.962279 IP 172.16.0.23.53 > 10.104.229.189.47624: 48977 1/0/0 AAAA 2a00:1450:400b:802::2004 (60)
E..X....@..N....
h...5...Ds3.Q...........www.google.com..............w..*..P@......... .
13:31:52.963683 IP 172.16.0.23.53 > 10.104.229.189.47624: 61300 6/0/0 A 209.85.203.104, A 209.85.203.105, A 209.85.203.106, A 209.85.203.147, A 209.85.203.99, A 209.85.203.103 (128)
E.......@.. ....
h...5....u..t...........www.google.com..............,...U.h.........,...U.i.........,...U.j.........,...U...........,...U.c.........,...U.g




- Packets #1 and #2 are our requests and packets #3 and #4 are the answers coming back from the DNS server.
- Our Linux box is resolving www.google.com twice. First with IPv4 (A) and second with IPv6 (AAAA).
- Each request receives a different answer. The A record receives a list of IPv4 addresses and the record AAAA receives a single IPv6 address (2a00:1450:400b:802::2004).  This is the address our box has decided to use.

In other words, during the DNS resolution our system determines whether this host is accessible using IPv6 or not. The way to do that is asking for the AAAA DNS record and use it when present.

We can do the same using dig.


- dig


$ dig AAAA www.linkedin.com

;; ANSWER SECTION:
www.linkedin.com. 88 IN CNAME glb-any-eu.www.linkedin.com.
glb-any-eu.www.linkedin.com. 88 IN CNAME any-eu.www.linkedin.com.
any-eu.www.linkedin.com. 1869 IN AAAA 2a04:f540:1::b93f:930a


Migrated = Yes



$ dig AAAA www.facebook.com


;; ANSWER SECTION:
www.facebook.com. 14 IN CNAME star-mini.c10r.facebook.com.
star-mini.c10r.facebook.com. 41 IN AAAA 2a03:2880:2130:7f20:face:b00c:0:25de


Migrated = Yes



$ dig AAAA github.com


;; AUTHORITY SECTION:
github.com. 60 IN SOA ns1.p16.dynect.net. hostmaster.github.com. 1461162636 3600 600 604800 60


Not migrated yet.