Xen

From Smop.co.uk

Contents 1 Initial setup 2 Building my own kernel 3 Bridging 4 Binding adapters 5 prexen 6 Miscellaneous

Initial setup

I use xen-tools to create new Xen domains:

xen-create-image --lvm rootvg --size=2Gb --hostname xen1 --kernel /boot/vmlinuz-xen --initrd /boot/initrd.img-xen  --memory 128 --debootstrap --dist etch --noswap --dhcp

I actually then edit the generated config (/etc/xen/xen1.cfg) like this:

vif  = [ 'mac=00:16:3E:XX:YY:00,bridge=dmzbr0' ]

Where XX is the zone (00 for DMZ, 01 for internal for me) and YY is the primary IP address in hexadecimal. I just find this nicer than a randomly generated MAC address. My interior firewall for example has this settting:

vif  = [ 'mac=00:16:3E:00:02:00,bridge=dmzbr0',
         'mac=00:16:3E:01:01:00,bridge=intbr0' ]

Building my own kernel

I needed to build a new kernel (dom0 as well as domU):

• I needed a hostap patch
• set CONFIG_HOSTAP_FIRMWARE_NVRAM so that prism2_srec can load primary firmware
• and also to increase HZ to 1000 for zaptel
• download and extract linux-2.6_2.6.18 package from debian (nothing later is supported as a dom0)
• apply above patch
• edit debian/arch/config:

# CONFIG_HZ_250 is not set
CONFIG_HZ_1000=y 
CONFIG_HZ=1000

• dch -i (mention changes)
• thanks to this page for the next steps:
  • fakeroot debian/rules debian/control
  • fakeroot debian/rules debian/build debian/stamps
  • fakeroot make -f debian/rules.gen binary-arch-i386-xen-686 binary-arch-i386-xen-real
    • first rule makes the "useful" packages, the second creates the headers packages

Bridging

Some hints and tips on Xen briding:

• use brctl to create the bridge device before xen wants to use it (brctl add xenbr0)
• add the external interface (if any - e.g. eth0, wlan0) to the bridge (brctl addif xenbr0 wlan0)
• put your IP address/dhcp the bridge, not the external device (dhcpcd xenbr0)
• ensure all devices (xenbr0, eth0, vif4.1 etc) are marked "up" by ifconfig
• wait 30secs for STP to kick in and detect any changes

My /etc/network/interfaces looks like this:

# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface
#allow-hotplug eth0
auto intbr0
iface intbr0 inet static
        address 192.168.101.3
        netmask 255.255.255.0
        gateway 192.168.101.1
        # dns-* options are implemented by the resolvconf package, if installed
        dns-nameservers 192.168.100.3
        dns-search smop.co.uk
        bridge_ports wlan0

# we just want this up, but no IP
# done in /etc/init.d/prexen now
#auto dmzbr0
#iface dmzbr0 inet manual

Binding adapters

I bind the onboard ethernet to "ferro". This requires the device to be unbound from the Dom0 domain (see prexen below) and then the following line must be added to /etc/xen/ferro.cfg:

pci = [ '00:12.0' ]

prexen

I use this script to set up dom0 ready for the domUs to run. It's all self-explanatory:

#!/bin/sh

# stuff to do before Xen starts up

main()
{
  # unbind onboard ethernet so that perimeter firewall gets it
  onboard_eth=`lspci | awk '/Ethernet/ { print $1 }'`
  pci_unbind $onboard_eth

  # add DMZ bridge (can't do in /etc/network/interfaces without IP)
  brctl addbr dmzbr0
  # need to mark bridge as up to allow forwarding
  ifconfig dmzbr0 up

  # reduce memory allocate to Dom0 to prevent:
  #   xen_net: Memory squeeze in netback driver
  # this has to be done _post_ xend
  # but we set this in /boot/grub/menu.lst instead:
  # dom0_mem=131072
  #xm mem-set 0 128

  # because of /etc/udev/rules.d/z25_persistent-net.rules
  # up with eth2 and eth2ap_rename which we could later rename
  # removed rule (forcing wifi0, wlan0, wlanap0 _all_ to eth2
  # which broken (non-unique name) 
  # NB: we moved /etc/rc2.d/S20hostapd to S21hostapd
  #echo | ifrename -c - -i eth2ap_rename -n eth2ap
}

pci_unbind()
{
  pci_id=0000:$1
  echo -n $pci_id > /sys/bus/pci/devices/$pci_id/driver/unbind
  echo -n $pci_id > /sys/bus/pci/drivers/pciback/new_slot
  echo -n $pci_id > /sys/bus/pci/drivers/pciback/bind
}

main "$@"

Miscellaneous

• domUs should slave time from master (default)
  • /proc/sys/xen/independent_wallclock should be 0
• "Driver tried to write to a read-only configuration space field"
  • /etc/xen/xend-pxi-permissive.sxp needed editing (when wlan0 was in domU?)