Episode 0x03: Networking - Part 2: IPv4 to IPv6 and back

Intro

In Episode 1, we discussed the challenges posed by Vodafone’s lack of IPv4 port-forwarding and GitHub’s lack of IPv6 support. To work around these limitations, I’ve rented a VPS from Digital Ocean. This VPS will forward IPv4 traffic to our device that’s exposed over IPv6. Then, within our server, we’ll convert this IPv6 traffic back to IPv4 before passing it on to our Kubernetes cluster.

If you can port-forward IPv4 traffic, you can skip this part and forward the external traffic to your pfSense modem instead. The diagram below illustrates our overall network architecture.

In part 1 of this episode, we did set up pfSense. For this part, we’ll set up the Edge virtual PC, expose it through our Vodafone modem, and configure our Digital Ocean VPS for traffic tunneling using Nginx. Note that this setup will limit us to HTTP and HTTPS traffic but should suffice for most web applications.

Edge Virtual PC

First, let’s create a virtual machine called Edge Services. This VM will have two network interfaces: one connected to the Vodafone Modem (with an IPv6 address) and another connected to pfSense. This configuration enables traffic forwarding from the Vodafone modem to pfSense.

1. Follow the instructions in Episode 2 to install a Debian cloud image on this VM.
2. Configure the VM with the following settings:

Install and configure Qemu Guest Agent (optional)

To enable Proxmox to detect the IP addresses assigned to each network interface, install the Qemu guest agent. SSH into your Edge VM as root and execute the following commands:

apt install qemu-guest-agent
systemctl enable qemu-guest-agent
systemctl start qemu-guest-agent

This should display the IP address on the Edge VM status page in Proxmox.

While both network interfaces should automatically receive IP addresses via DHCP, setting static IPs is advisable for consistent Nginx configuration. Use the cloud-init method described in Episode 2. In our example, the IPs are 192.168.0.201 for the external interface and 10.100.0.2 for the internal interface.

Run ifconfig to verify your configuration:

eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.0.201  netmask 255.255.255.0  broadcast 192.168.0.255
        inet6 2aXX:XXXX:XXXX:XXXX::d298  prefixlen 128  scopeid 0x0<global>
        inet6 2aXX:XXXX:XXXX:XXXX:be24:11ff:fe9b:ad64  prefixlen 64  scopeid 0x0<global>
        inet6 fe80::be24:11ff:fe9b:ad64  prefixlen 64  scopeid 0x20<link>
        ether bc:24:11:9b:ad:64  txqueuelen 1000  (Ethernet)
        RX packets 1569868  bytes 188403125 (179.6 MiB)
        RX errors 0  dropped 40946  overruns 0  frame 0
        TX packets 199260  bytes 89691614 (85.5 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

eth1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 10.100.0.2  netmask 255.255.255.0  broadcast 10.100.0.255
        inet6 fe80::be24:11ff:fea5:db0a  prefixlen 64  scopeid 0x20<link>
        ether bc:24:11:a5:db:0a  txqueuelen 1000  (Ethernet)
        RX packets 1684756  bytes 1048786998 (1000.2 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 1749093  bytes 819816787 (781.8 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 1000  (Local Loopback)
        RX packets 86  bytes 6908 (6.7 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 86  bytes 6908 (6.7 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

The IPv6 address on eth0 is the one we are going to expose to outside later.

Install and configure Nginx

1. Install Nginx using the following commands:

apt install nginx
apt install libnginx-mod-stream # if not installed by above

2. Configure Nginx to forward incoming traffic from geekembly.com to your Kubernetes cluster. Edit /etc/nginx/sites-available/default to include:

server {
        listen 80;
        listen [::]:80;

        server_name _;

        location /.well-known/acme-challenge/ {
                proxy_pass http://10.100.0.128;
                proxy_set_header Host $host;
                proxy_set_header X-Real-IP $remote_addr;
                proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
                proxy_set_header X-Forwarded-Proto $scheme;
        }

        location / {
                deny all;
        }
}

Above, forwards the acme challenge to our kubernetes ingress. Remember that the only http traffic we want to allow is the acme challenge. Everything else should use https. In episode 8 we will explore handling of https connection further.

If your are wondering what is IP address 10.100.0.128 for, well, if you remember from the previous part, the DHCP pool we made available on KubernetesVLAN100 was from 10.100.0.10 to 10.100.0.127, and I mentioned that the higher address from 128 are reserved for metallb. 10.100.0.128 is the first address in that range, used by metallb to expose our k8s nginx ingress. More on this later, for us, at the moment, it just the gate to our k8s cluster.

3. Forward HTTPS traffic by editing /etc/nginx/nginx.conf:

# At the beginning of config file
load_module /usr/lib/nginx/modules/ngx_stream_module.so;

# At the end of config file
stream {
        server {
                listen 443;
                listen [::]:443;
                proxy_pass 10.100.0.128:443;
        }
}

Since we are going to use cert-manager in our kuberenets cluster, and our https connections will terminate there, all we need to do is to just stream https traffic to the kubernetes cluster.

Exposing Edge VM

Expose the Edge VM using the IPv6 Host Exposure feature of your Vodafone modem. Log in to your modem’s admin panel and navigate to the IPv6 Host Exposure settings. Expose ports 80 and 443 of your Edge VM.

Test connectivity with the following command:

curl http://\[<exposed-ipv6-address-here>\]

You should see an Nginx error page.

Configuring our VPS

As mentioned previously, we need to configure a VPS to use it as our host for geekembly.com traffic. For that, I’ve utilized a Digital Ocean Droplet. You can use any cloud provider you prefer. Setting up that VPS is almost idential to configuring our Edge service, just in reverse: We need to tunnel IPv4 traffic to IPv6.

On /etc/nginx/sites-available/default we have:

server {
        listen 80;
        listen [::]:80;
        server_name _;

        location /.well-known/acme-challenge/ {
                proxy_pass http://[<ipv6-address-of-edge-service>];
                proxy_set_header Host $host;
                proxy_set_header X-Real-IP $remote_addr;
                proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
                proxy_set_header X-Forwarded-Proto $scheme;
        }

        location / {
                deny all;
        }
}

and on /etc/nginx/nginx.conf we have:

# At the beginning of config file
load_module /usr/lib/nginx/modules/ngx_stream_module.so;

# At the end of config file
stream {
        server {
                listen 443;
                listen [::]:443;
                proxy_pass [<ipv6-address-of-edge-service>]:443;
        }
}

The only diff is that we proxy pass to exposed ipv6 address of the Edge VM.

NOTE: When using IPv6 addresses, you have to put them inside square brackets like [].

Configure DNS records

Finally, configure the DNS records for your domain. Access your domain registrar’s control panel and set up the following records:

Include an A record for each subdomain you plan to use. Above are just some examples.

With this setup in place, our network configuration is ready. In the next episode, we’ll provision our Kubernetes cluster using Terraform! 🚀