Root server with IPv6-only KVM guests (II): NAT64, DNS64, and KVM 18

2018-11-20 in English / Linux

This article is outdated, find its replacement in the IPv6-first guide

I’ve updated this article between 2018 and 2020 constantly but finally decided that I need another format for this information. So I wrote the IPv6-first guide which you can read online as HTML or PDF document. It’s CC-BY-SA-licensed and its sources can be found on Github

Root server with IPv6-only KVM guests

(1) The basic setup – (2) NAT64, DNS64, and KVM – (3) Mail server with Postfix, Dovecot – and IPv4 – (4) The IPv6-first guide

In the first part of this article series, I’ve described how to set up a root server which can serve KVM guests which only have IPv6 connectivity. I use the German hosting provider Hetzner as an example and describe a bridged setup which fits into that provider’s IPv6 infrastructure.

Network switch

After the basic setup, we need some more services on the host to be able to install the first guest systems.

Create a non-root user on the system.

Hetzner’s installation process generates a system with only the root user. To make it more compliant with usual Ubuntu behaviour, add a non-root user:

Create user with adduser.
Put your ssh public key into .ssh/authorized_keys of that user.

You can now either perform the following steps as that user with sudo -s or continue as directly logged in root via ssh.

NAT64 with Tayga

As I wrote, our guest systems shall have IPv6-only internet. That implies that they cannot access systems which are IPv4-only. Unfortunately, even in 2018, there are quite popular sites like github.com which do not have any IPv6 connectivity at all. To make such systems accessible from the guest systems, we set up a NAT64 service which performs a network address translation for exactly this case.

I decided to go with the „Tayga” server. Its scope is limited to exactly perform NAT64. This makes it necessary to add further services to make all this really useable but it also minimizes configuration complexity. This is what I did:

Install the tayga service by the usual

apt install tayga

1

apt install tayga
In /etc/tayga.conf, enable the disabled ipv6-addr directive as this is needed for working with the well-known prefix. You should set the IPv6 address to something random in your IPv6 subnet:

ipv6-addr 2a01:4f8:1:3:135d:6:4b27:5f

1

ipv6-addr 2a01:4f8:1:3:135d:6:4b27:5f

Additionally, switch the prefix directive from the activated 2001... one to the 64:ff9b::/96 one::

# prefix 2001:db8:1:ffff::/96 prefix 64:ff9b::/96

1
2

# prefix 2001:db8:1:ffff::/96
prefix 64:ff9b::/96

The whole Tayga configuration reads like this:

# Minimum working NAT64 Tayga configuration for KVM host with IPv6-only guests

# (A) Basic setup
# Device name, this is the default
tun-device nat64
# Data dir for stateful NAT information
data-dir /var/spool/tayga

# (B) IPv6 setup
# The "well-known" prefix for NAT64
prefix 64:ff9b::/96
# IPv6 address, from the official ::/64 network
ipv6-addr 2a01:4f8:X:Y:14a5:69be:7e23:89

# (C) IPv4 setup
# Pool of dynamic addresses
dynamic-pool 192.168.255.0/24
# IPv4 address, not to be used otherwise in the network
ipv4-addr 192.168.255.1

# Minimum working NAT64 Tayga configuration for KVM host with IPv6-only guests

# (A) Basic setup

# Device name, this is the default

tun-device nat64

# Data dir for stateful NAT information

data-dir /var/spool/tayga

# (B) IPv6 setup

# The „well-known” prefix for NAT64

prefix 64:ff9b::/96

# IPv6 address, from the official ::/64 network

ipv6-addr 2a01:4f8:X:Y:14a5:69be:7e23:89

# © IPv4 setup

# Pool of dynamic addresses

dynamic-pool 192.168.255.0/24

# IPv4 address, not to be used otherwise in the network

ipv4-addr 192.168.255.1

Test the new setup by starting tayga once in foreground:

systemctl stop tayga  <-- Disable if already started
tayga -d --nodetach

1 2	systemctl stop tayga <– Disable if already started tayga -d –nodetach

This should give something like this:

starting TAYGA 0.9.2
Using tun device nat64 with MTU 1500
TAYGA's IPv4 address: 192.168.255.1
TAYGA's IPv6 address: 2a01:4f8:1:3:135d:6:4b27:5f
NAT64 prefix: 64:ff9b::/96
Note: traffic between IPv6 hosts and private IPv4 addresses (i.e. to/from 64:ff9b::10.0.0.0/104, 64:ff9b::192.168.0.0/112, etc) will be dropped.  Use a translation prefix within your organization's IPv6 address space instead of 64:ff9b::/96 if you need your IPv6 hosts to communicate with private IPv4 addresses.
Dynamic pool: 192.168.255.0/24

starting TAYGA 0.9.2

Using tun device nat64 with MTU 1500

TAYGA’s IPv4 address: 192.168.255.1

TAYGA’s IPv6 address: 2a01:4f8:1:3:135d:6:4b27:5f

NAT64 prefix: 64:ff9b::/96

Note: traffic between IPv6 hosts and private IPv4 addresses (i.e. to/from 64:ff9b::10.0.0.0/104, 64:ff9b::192.168.0.0/112, etc) will be dropped. Use a translation prefix within your organization’s IPv6 address space instead of 64:ff9b::/96 if you need your IPv6 hosts to communicate with private IPv4 addresses.

Dynamic pool: 192.168.255.0/24

Stop the manually started instance and edit /etc/default/tayga. Set RUN to yes:

# Change this to "yes" to enable tayga RUN="yes"

1
2

# Change this to „yes” to enable tayga
RUN=„yes”
Launch the service

systemctl start tayga

1

systemctl start tayga

systemctl status tayga should say the Active state is active (running), the log should end with

... systemd[1]: Started LSB: userspace NAT64.

1

... systemd[1]: Started LSB: userspace NAT64.
If the Active state is active (exited) and the protocol says something about „set RUN to yes”, you have forgotten to enable the RUN option in /etc/default/tayga. If you correct that, you have to perform a stop and a start command afterwards to launch tayga:

systemctl stop tayga systemctl start tayga

1
2

systemctl stop tayga
systemctl start tayga

DNS64 with bind

NAT64 is usually used together with a so-called „DNS64” name server. This is a specially configured name server. If a client asks it for an IPv6 name resolution, i.e. an AAAA name service record and there is only an IPv4 A record for the requested name, the DNS64 name server „mocks up” an AAAA record munging the IPv4 address and a „well-known prefix” to a synthetical IPv6 address. This address – surprise, surprise – points directly to a nicely prepared NAT64 server so that the IPv6 system talks to an IPv4 system transparently hidden behind the NAT64 proxy.

DNS64 and NAT64 translate betwwen IP protocols

DNS64 and NAT64 translate between IP protocols

Go on like this:

Install bind9 on the host:

apt install bind9

1

apt install bind9

Our bind is a forwarding only-server only for our own virtual machines. On Debian-derived systems, the bind options needed for this setup are located in /etc/bind/named.conf.options. Edit that file and enter the following entries:

options {
        directory "/var/cache/bind";

        forwarders {
                2a01:4f8:0:1::add:1010;  # Hetzner name servers
                2a01:4f8:0:1::add:9999;
                2a01:4f8:0:1::add:9898;
        };

        dnssec-validation auto;

        auth-nxdomain no;    # conform to RFC1035
        listen-on {};
        listen-on-v6 {
                <IPv6 network assigned by provider>::/64;
        };
        allow-query { localnets; };
        dns64 64:ff9b::/96 {
                clients { any; };
        };
};

options {

directory „/var/cache/bind”;

forwarders {

2a01:4f8:0:1::add:1010; # Hetzner name servers

2a01:4f8:0:1::add:9999;

2a01:4f8:0:1::add:9898;

};

dnssec-validation auto;

auth-nxdomain no; # conform to RFC1035

listen-on {};

listen-on-v6 {

<IPv6 network assigned by provider>::/64;

};

allow-query { localnets; };

dns64 64:ff9b::/96 {

clients { any; };

};

The actual important definition is the dns64 section at the bottom of the options definitions. It enables the DNS64 mode of bind and defines the IPv6 address range into which the addresses should be converted.
It also important to define listen-on {}; to disable listening on the IPv4 port altogether – we do not need it. Restricting allow-query to the localnets is also important to prevent the server from becoming an open DNS relay. We only need it for our internal network.
Check the network in listen-on-v6 and also check the forwarders. You whole IP address resolution will not work if one of these is wrong.
Restart the daemon and check that it is enabled and running:

systemctl restart bind9 systemctl status bind9

1
2

systemctl restart bind9
systemctl status bind9

After these steps, you have a working DNS64 server which you can use for all your virtual machines on the system.

Using an external DNS64 server

So far, the name server is only used for DNS64. You can also use the Google servers 2001:4860:4860::6464 and 2001:4860:4860::64 offering this service. Their replies are compatible with our NAT64 setup. However, having an own server reduces external dependencies and allows for additional services lateron.

Include NAT64/DNS64 into radvd advertisements

We now have the NAT64/DNS64 service pair up and running. We will inform the virtual machines by the router advertisement daemon we have already set up. This simplifies installation and administration tremendously:

Extend /etc/radvd.conf:

interface br0 { [...] RDNSS <IPv6 address of host with DNS64 bind> { }; route 64:ff9b::/96 { }; };

1
2
3
4
5
6
7
8
9

interface br0
{
[...]
RDNSS <IPv6 address of host with DNS64 bind>
{
};
route 64:ff9b::/96 {
};
};
The RDNSS definition describes which DNS server to access. Use the host’s IP address. If you opted for the Google servers to do the job, write instead

RDNSS 2001:4860:4860::6464 2001:4860:4860::64 { };

1
2
3
4

RDNSS 2001:4860:4860::6464
2001:4860:4860::64
{
};
The route section advertises that this system routes the 64:ff9b:: network. Only with this definition the virtual servers know where to send the packets for the emulated IPv6 addresses for the IPv4-only servers to.
Restart radvd and check its output with radvdump. It should contain both the DNS server and the NAT64 route.

The nasty Hetzner pitfall

In their own documentation, Hetzner also describes how to setup radvd. For the DNS servers, however, they use IPv6 example addresses from the 2001:db8 realm. It took me three days and severe doubts about Hetzner’s IPv6 setup to find out, that my only mistake was to copy these wrong IP addresses for the DNS server into the configuration. Don’t make the same mistake…

You have now prepared everything for the IPv6-only virtual machines to come: They get their network configuration through the centrally administrated radvd. The advertised setup includes a name server with DNS64 and a NAT64 route to access IPv4-only systems.

About non-virtual network setups

In my series, I describe how to set up a root server with virtual machines. Especially NAT64/DNS64 is completely independent of that. If you administrate a (real) computer network and want to lay the ground for IPv6-only machines in that, do exactly the same with your physical machines: Install Tayga and the DNS64-capable Bind9 on the router behind which the IPv6-only systems reside. This might be the „firewall” of classical setups. Then, your actual computers play the role of the virtual machines in this guide.

Install KVM

We’re now ready for the final steps! Our network is configured far enough so that we really can start installing virtual machines on our system. For this, we, of course, need KVM. For Ubuntu 18.04 I followed the first steps of this guide::

Check that the system supports virtualization at all. Issue

egrep -c '(vmx|svm)' /proc/cpuinfo

1

egrep -c ‚(vmx|svm)’ /proc/cpuinfo

and verify that the result is greater than 0.
Then, apply

apt install cpu-checker kvm-ok

1
2

apt install cpu-checker
kvm-ok

and check that the result is

INFO: /dev/kvm exists KVM acceleration can be used

1
2

INFO: /dev/kvm exists
KVM acceleration can be used

If not, the BIOS settings of the system must be corrected. Contact the hosting provider to sort that out.
Install KVM and the required helper packages:

apt install qemu qemu-kvm libvirt-bin bridge-utils virt-manager

1

apt install qemu qemu-kvm libvirt-bin bridge-utils virt-manager

This will install a rather large number of new packages on your host. Finally, it will be capable to server virtual machines.
The libvirtd daemon should already be up and running at this point. If this is, for any reason, not the case, start and enable it with the usual systemctl commands or whatever the init system of your host server requires to do this.
To simplify installation and administration of your virtual machines, add the „normal” user you created above to the libvirt user group. I prefer doing this by simply adding the user name to the definition in /etc/group:

libvirt:x:<groupid>:USERNAME

1

libvirt:x:<groupid>:USERNAME

Well, that’s it! Our system can get its first virtual machine!

The first virtual machine

Installing a virtual machine might sound difficult to you if you have never done this before. In fact, it is not. After all, it is even simpler than the remote installation on a hosted system once you get used to it. On the physical machine, you are dependent on what the hosting provider offers as installation procedures. KVM offers you more or less a complete virtualized graphical console which allows acting just as you were sitting in front of the (virtual) computer’s monitor. This way, you can install whatever you like.

Furthermore, if you make a configuration mistake on the physical host, you might end with a broken machine. If this happens in a virtual machine, you have several ways to solve the problem: You can connect to the console and log in directly without network access. If the machine does not boot anymore, you can even mount the virtual hard disk into the physical machine and try to fix. And if the machine is for any reason broken beyond repair, you can just throw it away and start over with a fresh installation.

I suggest starting with a throw-away virtual machine. It will not contain any „real” services but only show any remaining problems in the setup. Further, it allows to test and learn the whole process of installing a virtual machine in the setup. Prepare the setup with the following steps:

Copy the installation ISO image for the system to install onto the host.
Connect to the KVM system using virt-manager. Of course, you might also use another client, but I find this rather easy.
Use the ssh connection of the normal user created on the host.
Start the host creation.

From here on, things become rather standard. We’re now in the process of installing a guest system in a KVM guest. My best practices are these:

Assign as many CPUs to the virtual machine as the hardware has. Only if you suspect the virtual machine to grab too many resources, reduce the CPU number.
Use cow2 image file for the virtual hard disk. It is the most flexible way once it comes to migrating the virtual machine and today’s file systems can cope with the double indirection quite well.
Give the virtual machine definition the same name the system will later have.

The one interesting point is the network configuration at the very end of the definition process. Here, enter the „Network selection” before creating the machine. Select „Name of common device” and give the name of bridge explicitly. Here it is br0.

Network setup for virtual machine in virt-manager

If you are ready, press „Create” and summon your first virtual system.

Ubuntu 18.04 as guest system

We’re now at a stage where you can install any operating system which is installable in KVM virtual machines. I give some advises for the Ubuntu 18.04 network installer:

Install by simply pressing the „Install” button. I never needed any additional kernel parameters.
Select correct keyboard or it will drive you nuts.
Network autodetection will idle around when looking for the non-existing DHCP server. Keep calm. Apart from that, it will simply setup everything correctly.
Enter the hostname, preferrably the same as the name of the virtual machine to keep it simple…
Check whether you provider has their own mirror for the installation server. Hetzner has, therefore you can save download time:
- Go to top of mirror list and press enter information manually
- For Hetzner: mirror.hetzner.de. This server also works also with IPv6. But even IPv4 servers would be possible due to our NAT64/DNS64 setup.
- Set directory for the Hetzner server to /ubuntu/packages/
You do not need a HTTP proxy.
Install should start.
I suggest to not partition the virtual hard disk in any way. It is not needed.
Everything else is as usual. In the software selection, you should at least select the „OpenSSH server” so that you can log into the system after installation.

Ubuntu 18.04 within KVM virt-manager

As this is Ubuntu 18.04, this machine uses netplan for network configuration. It has a very simple definition file in /etc/netplan/01-netcfg.yaml:

# This file describes the network interfaces available on your system
# For more information, see netplan(5).
network:
  version: 2
  renderer: networkd
  ethernets:
    ens3:
      dhcp6: yes

# This file describes the network interfaces available on your system

# For more information, see netplan(5).

network:

version: 2

renderer: networkd

ethernets:

ens3:

dhcp6: yes

Netplan summarizes router advertisement in the „dhcp6” statement.

Note that after (re-)booting the virtual machine, it may take some seconds until it has configured its network interface. Once it has done so, everything should work without problems.

Entering the virtual machine in DNS

To work with the long IPv6 addresses conveniently, DNS is almost mandatory. You should enter the virtual machine now into the domain.

Find the virtual machine’s IP address, e.g. through „ip a”.
Create an entry in the DNS zone of the system. Note that you only enter a AAAA record, not an old-fashioned A record for an IPv4 address. The system just has no IPv4 address…
In my opinion, it makes also sense to always create a reverse IP entry for IPv6 hosts. If for any reason your DNS AAAA entry vanishes, you still have the reverse IP entry which assigns name and IP address. Reverse IP entries are always managed in the DNS realm of the IP network owner. In my case, they are edited in Hetzner’s robot interface.

Reverse IP entries in Hetzner’s robot interface

The SLAAC mechanism derives the IP address from the MAC address of the virtual machine. So, it will be static even though it has nowhere been configured explicitly. This whole mechanism works smoothly with any modern operating system. I even made a test installation of Windows 10 which went perfectly without any IPv4 address.

IPv6-only network settings in Windows 10

We can now install any number of virtual machines on our server. In the next articles of this series, I describe how I set up some services on this infrastructure. The most important ones are, of course, e‑mail and web servers.