The internet is out of space. Not for people, but for addresses. The system that has served us since the early days of the internet is reaching its limits, causing AWS to charge more for public IPv4 addresses. So what are the options? Do we just have to reduce the number of IP addresses we use and cope with the higher costs? Fortunately, the answer is “no”. We recently set up a new Kubernetes cluster on AWS, to simplify deploying and managing our clients’ applications, using the “new” IPv6 support (not that “new”, since it was devised in the 90’s and standardised in 2017). Here’s what we learned, and why it might make sense for your next cluster too.
The Internet Protocol, Version 4 (IPv4), the current way we give every host an address on the internet gives us about 4 billion addresses. This would have sounded a lot in the 1980s, but even in the 1990s it was clear that we would outgrow it. Enter IPv6, with 340,282,366,920,938,463,463,374,607,431,768,211,456 (340 undecillion, apparently) possible addresses. That’s really big. It’s enough to assign 100 million addresses to every cubic millimetre of the Earth!
IPv6 has been available since I was at university (many years ago!), but it wasn't ratified as a standard until 2017 and still hasn’t gained widespread adoption. Why? Mostly because IPv4 with NAT still works and has become so commonplace. Nevertheless, address space is becoming exhausted, and as a result, providers such as AWS are raising the cost of using the legacy standard. So whether you want to save money or build for the future, IPv6 deserves a serious look.
Kubernetes gained experimental IPv6 support in version 1.9, and it became Generally Available in 1.23. Even then, not all AWS services supported dual-stack or native IPv6. Thankfully, today there is good support across VPC, RDS, S3 and almost all other services. Elastic Kubernetes Service (EKS) and Elastic Load Balancer (ELB) are currently listed as “partially supported”, but we have found it entirely possible to run a reliable cluster in a dual-stack VPC and connect it to dual-stack RDS databases using IPv6.
The EKS cluster must use a dual-stack VPC, as the control plane still requires IPv4 access to nodes. This means the size of the cluster is still limited at the time of writing, yet it is still a solid step towards full IPv6-native networking. Outbound traffic from private subnets still requires NAT Gateways for services that don't support IPv6. However, native IPv6 traffic uses Egress-only Internet Gateways, which reduces the need for costly public IPv4 addresses and may let you reduce the number of expensive NAT Gateways significantly. Inbound traffic is handled by dual-stack Application Load Balancers (ALBs), which can accept both IPv4 and IPv6 traffic and forward it to IPv6 targets. This keeps your services reachable by all clients, both those using IPv4 and IPv6. Internally, the cluster is configured to use the IPv6 address family, so:
The first challenge (and disappointment) was that we couldn’t go fully native IPv6. But this makes sense, as some resources, such as RDS instances, only support Dual Stack mode. Another point to note is that standard DNS does not work in IPv6-only networks, as IPv6-only hosts cannot route IPv4 addresses! Fortunately, a system known as DNS64 has been developed to overcome this. DNS64 translates DNS requests for IPv4-only hosts by resolving them to a special IPv6 subnet, which the NAT Gateway maps to the correct IPv4 address. This is simply a tick-box option on the subnet configuration.
All of these systems mean that the cluster can use most of the benefits of an IPv6 network, and the only difference that you see is the IP address format:
❯ kubectl get services -n newredo-com-prod NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE newredo-com-cache ClusterIP fd87:cec5:a64d::5870 <none> 80/TCP 261d newredo-com-web ClusterIP fd87:cec5:a64d::e5d5 <none> 80/TCP 258d strapi-cms ClusterIP fd87:cec5:a64d::dcf8 <none> 80/TCP 257d
The reality is that the IPv6 configuration was only part of the challenge of setting up our multi-tenant cluster. We also share the cluster across our entire AWS Organisation. That is a story for another day!
So why should you consider setting up an IPv6 cluster? The first benefit is clear: you will no longer be limited by the IP addresses you can use. But the real impact is that you will be future-proof and not beholden to the costs associated with increased competition for “legacy” IPv4 addresses. And finally, you will get used to seeing and dealing with IPv6 addresses, which, as they become more widely adopted, will become increasingly common.
IPv6 isn’t the future: it’s here. All of the pieces are in place for IPv6 adoption, and services keep getting better support. So why not update your infrastructure when you get a chance? And if you need any help, then NewRedo are here, with the experience you need.
We're software development specialists boosting our client's business with bespoke systems. We're practical, pragmatic and commercially minded, which means we do the right thing for you when it comes to building software.
New
