Configuring Gentoo with Xen
1.
Introduction
The Xen technology allows you to run
multiple operating systems on a single physical system, govern resource
consumption and even migrate domains (which are the virtual environments in
which a guest operating system runs) from one Xen-powered system to another. Xen
requires the host operating system to support Xen (which, in this case, will be
a Linux kernel) but guest operating systems can run unmodified if your
hardware supports Intel Virtualization Technology (VT-x) or AMD Virtualization
Technology (SVM). Otherwise your guest operating systems must also support Xen.
This guide will talk you through the configuration steps necessary to get Xen up
and running on Gentoo Linux. We will not discuss Xen itself (the Xen project has
decent
documentation available) nor will we talk about specialized setups that
might be very interesting for Xen setups but are not Xen-related (like exporting
Portage through NFS, booting Linux using PXE, etc.)
2.
Preparing Domain0
Introduction
Domain0 is the primary domain under Xen, hosting the host operating
system which governs all other domains. In this chapter we will prepare an
existing Gentoo installation to become the host operating system in this domain
and build the Xen-powered kernel so that Gentoo is ready to host other Xen
domains.
Rebuilding the Gentoo Installation ?
A dramatic change that might be necessary is to rebuild the entire Gentoo
installation with a different CFLAGS setting. Guest operating systems
running under Xen might otherwise see major performance degradation. If you,
however, are planning on checking out Xen rather than installing it for
production use and are not terribly fond of rebuilding all programs, you can
skip this step. In this case you will notice performance degradation but you
will still be able to use Xen.
Important:
It is advised that, if you change your CFLAGS and build your system with
a gcc lower than version 4, you do not have -Os set as it has been
reported to produce broken code.
|
Code Listing 2.1: Editing the CFLAGS and rebuild the Gentoo installation |
~# nano -w /etc/make.conf
CFLAGS="-O2 -march=pentium4 -pipe -mno-tls-direct-seg-refs"
~# emerge -e world
|
If you boot your system using an initial ramdisk (initrd) you need to
rebuild the initrd as well (which is best done by running all steps you would do
when you rebuild your kernel).
Installing Xen
Xen actually contains many components, so you'll need to install a couple of
packages. Because it is still ~arch
masked you first need to unmask it by adding the necessary lines to
/etc/portage/package.keywords and then install them.
Code Listing 2.2: Unmasking and Installing Xen |
~# nano -w /etc/portage/package.keywords
app-emulation/xen
app-emulation/xen-tools
sys-kernel/xen-sources
~# emerge xen xen-tools xen-sources
|
Building the Kernel
Next we'll build the Linux kernel with Xen support. This kernel, whose sources
are available at /usr/src/linux-2.6.x.z-xen, will be our main
running kernel (i.e. the one running domain 0). In the XEN section you'll
find drivers for all kinds of input/output, each driver having a backend
and frontend implementation available. For the domain 0 kernel you need
to select the backend implementation: these are used by the other
domains (who use the frontend drivers) to communicate directly with
the hardware.
Of course, don't forget to select Xen-compatible at Processor type and
features. If you're wondering about networking: each interface in a domain
has a point-to-point link to an interface on domain 0 (called
vifX.Y where X is the domain number and Y the Yth interface of that
domain), so you can configure your network the way you want (bridging, NAT,
etc.)
Code Listing 2.3: Enabling Xen Support for i386 Kernels |
Processor type and features --->
Subarchitecture Type (Xen-compatible)
|
Code Listing 2.4: Enabling Xen Support for x86_64 Kernels |
Processor type and features --->
Subarchitecture Type (PC-compatible)
[*] Enable Xen compatible kernel
|
Code Listing 2.5: Domain-0 Kernel Config |
Bus options (PCI etc.) --->
[*] PCI support
[ ] Xen PCI Frontend Debugging
Networking --->
Networking options --->
<*> 802.1d Ethernet Bridging
XEN --->
[*] Privileged Guest (domain 0)
<*> Backend driver support
<*> Block-device backend driver
<*> Network-device backend driver
<*> PCI-device backend driver
PCI Backend Mode (Virtual PCI) --->
[*] Scrub memory before freeing it to Xen
[*] Disable serial port drivers
Xen version compatibility (3.0.4 and later)
|
Code Listing 2.6: Domain-U Kernel Config |
Bus options (PCI etc.) --->
[ ] PCI support
Device Drivers --->
SCSI device support --->
< > SCSI device support
XEN --->
[ ] Privileged Guest (domain 0)
<*> Block-device frontend driver
<*> Network-device frontend driver
[*] Scrub memory before freeing it to Xen
[*] Disable serial port drivers
Xen version compatibility (3.0.4 and later)
|
A nice hint is to have the kernel make process store its intermediate object
files elsewhere so that you can reuse the same kernel tree to build different
configurations:
Code Listing 2.7: Building the Kernel |
~# mkdir -p ~/build/dom0 ~/build/domU
~# make O=~/build/dom0 menuconfig
~# make O=~/build/dom0 && make O=~/build/dom0 modules_install
|
Once the kernel is built you'll find the kernel image immediately in the
build directory (not inside arch/ or any other directory) called
vmlinuz. Copy it to /boot and then configure your
bootloader to use the Xen hypervisor (one of the components installed
previously) which is stored as /boot/xen.gz. In the bootloader
configuration, add your newly built kernel as the kernel that Xen should
boot. For instance, for GRUB:
Code Listing 2.8: GRUB Configuration for Xen |
title Xen 3.0 / Gentoo Linux 2.6.x.y
root (hd0,0)
kernel /boot/xen.gz
module /boot/kernel-2.6.x.y-xen0 root=/dev/hda3
|
Now reboot your system into Xen. Once you are booted, you need to load the Xen
daemon:
Code Listing 2.9: Loading the Xen daemon |
~# /etc/init.d/xend start
|
Now check if you can do whatever you normally do on your system. If this is the
case, you can edit your bootloader configuration to always boot into Xen and add
the Xen deamon to the default runlevel so that it is started automatically
next time you boot.
Note:
If you wish to start guest domains automatically on boot add xendomains
to the default runlevel as well and create a symlink in
/etc/xen/auto/ to the Xen configuration files for the domains
you wish to start.
|
3.
Creating an Unpriviledged Domain
Building the Kernel
Go to the Xen-powered Linux kernel source and update the configuration. It is
wise to keep as many topics as possible similar to the main kernel except the
XEN settings where drivers should now have their frontend
implementation selected instead of the backend. Then build the kernel
and place the resulting vmlinuz file where you want (we assume this
is /mnt/data/xen/kernel):
Code Listing 3.1: Building the guest kernel |
~# make O=~/build/domU
~# cp ~/build/domU/vmlinuz /mnt/data/xen/kernel/kernel-2.6.x.y-xen
|
It is also possible to create a single kernel image for both the administrative
domain and the unpriviledged domain. More information about this can be found
in the Xen user manual.
Creating the Domain Disks
For best performance, it is best to dedicate a partition (or logical volume) to
a domain rather than a file based filesystem. However, if you are going to use
Xen primarily for tests using a file based filesystem does have its advantages
(especially regarding maintenance).
You can create a file based filesystem using dd and mke2fs (or
any other file system creation tool). For instance, to create a 2Gbyte ext3
filesystem:
Code Listing 3.2: Creating a file based filesystem |
~# dd if=/dev/zero of=/mnt/data/xen/disks/ext3root.img bs=1M count=2048
~# mke2fs -j /mnt/data/xen/disks/ext3root.img
|
Configuring a Domain
Next we create a Xen configuration file for a domain. You can store these
configuration files where you want, for instance at
/mnt/data/xen/configs. As an example, we create a configuration
file for a small Gentoo environment which uses the disk image we created
previously:
Code Listing 3.3: Creating a domain configuration file |
~# nano -w /mnt/data/xen/configs/gentoo
kernel = "/mnt/data/xen/kernel/kernel-2.6.x.y-xen"
memory = 512
name = "gentoo"
disk = ['file:/mnt/data/xen/disks/ext3root.img,sda1,w']
root = "/dev/sda1 ro"
|
If you are using a block device (such as an lvm volume or partition) for
the disk use 'phy:' instead of 'file:' and leave off /dev. For example:
Code Listing 3.4: Using a block device |
disk = [ 'phy:lvm/xen-guest-root,sda1,w' ]
disk = [ 'phy:sdb6,sda1,w' ]
|
You can find example configuration files in /etc/xen.
Launching the New Domain
Now we're all set and we can launch the new domain. If the disk image contained
an operating system, we could just create and attach the domain using the
xm command (Xen manager):
Code Listing 3.5: Creating and starting a new domain |
~# xm create /mnt/data/xen/configs/gentoo -c
|
The domain would be booted inside the terminal in which you executed the
command. However, in our case, the disk image is empty so the domain won't boot
up in anything useful. To fix this, you can loop-mount the image and install
Gentoo as you're used to.
If you want to disconnect from the domain, press Ctrl+]. You can
always reconnect to the domains' console using xm console gentoo.
However, there is only one console per domain, so only use it when you can't
access the domain otherwise (for instance, through SSH).
4.
Networking on Unpriviledged Domains
Introduction
Xen supports at least two ways of configuring your (virtual) network:
routed and bridged.
When selecting the routed approach, the interface inside your
unpriviledged domain is connected to the virtual interface on your
administrative domain. On your administrative domain (domain 0), the virtual
interface is linked together with eth0. The
interface inside your unpriviledged domain should have an IP address on the same
network as the interface on the administrative domain. Any communication to
that IP address can only occur from the administrative domain, unless you set
up specific routing rules.
When selecting the bridged approach, your default network interface on
the administrative domain becomes a bridge which accepts connections to the
virtual domains as well as to the IP address your administrative domain has.
Regular Routed Interfaces
Before you set up the interface on your unpriviledged domain, make sure that
Xen's netloop and netbk drivers are loaded. A quick
hint: if you have netloop as a module, load it with
nloopbacks=0 so that it doesn't create pointless interfaces to the
loopback device. Then, edit your domain configuration file and add a vif
instruction to it.
Code Listing 4.1: Configuring a virtual interface |
~# nano -w /mnt/data/xen/configs/gentoo
vif = [ 'ip=192.168.1.101, vifname=veth1' ]
|
In the above example, the interface will be created for the unpriviledged domain
(in which it will be called eth0) and Xen will ensure that address
192.168.1.101 will be reachable from the administrative domain through interface
veth1.
This doesn't mean that the virtual eth0 interface will
automatically have IP 192.168.1.101 assigned to it, but rather that, if you
don't give it that IP, it will not be connected with the administrative domain
and thus cannot be reached.
Now edit /etc/xen/xend-config.sxp as follows to select routed
network configuration:
Code Listing 4.2: Editing xend-config.sxp |
~# nano -w /etc/xen/xend-config.sxp
#(network-script network-bridge)
#(vif-script vif-bridge)
(network-script network-route)
(vif-script vif-route)
|
Bridged Interfaces
Unlike the routed interfaces you now need to load the netloop
driver with nloopbacks=1 (or higher) as the additional loopback devices
are used to create the bridge. For the other modules you still need the
netbk module as well as briding functionality (bridge
module if build as such).
Now edit your virtual domain and add the vif construct:
Code Listing 4.3: Configuring a virtual interface |
~# nano -w /mnt/data/xen/configs/gentoo
vif = [ 'ip=192.168.1.101, vifname=veth0' ]
|
Next edit /etc/xen/xend-config.sxp as follows to select bridged
network configuration:
Code Listing 4.4: Editing xend-config.sxp |
~# nano -w /etc/xen/xend-config.sxp
(network-script network-bridge)
(vif-script vif-bridge)
# (network-script network-route)
# (vif-script vif-route)
|
By default, the bridge will contain whatever interface is configured to be the
default interface (the device that is listed under the default route through
ip route list). If you want to alter this behavior, edit the
xend-config.sxp as follows:
Code Listing 4.5: Editing xend-config.sxp to change bridge configuration |
~# nano -w /etc/xen/xend-config.sxp
(network-script 'network-bridge netdev=eth0 bridge=xenbr0 vifnum=0')
|
Once the configuration is done, restart the xend init script to have Xen
build the bridge:
Code Listing 4.6: Restarting the xend daemon |
~# /etc/init.d/xend restart
|
5.
Further Resources
Xen Documentation
Xen Tools
The contents of this document are licensed under the Creative Commons -
Attribution / Share Alike license.
|