Table of Contents
Xen - Dedibox
On a hosted server, with only a small install scenario, and not using a console, it is sometime hard to install everything like you'd like. Here is a small step by stop on how I used the lack-of-options install and moved my way up to a nice LVM and Xen setup.
Minimal installation
Web installation allows to install a debian, but does not support LVM. That's bad.
I decided to do a minimal installation. Here is how I partitionned the disk :
- /boot - 200MB (enough for alot of kernels, initrd, … don't need much else)
- / - 2GB (does not allow to install with less for /)
- swap - 1GB (does not allow to install without swap)
- /mnt - everything else (install is buggy, would be included on / of left empty)
Once we have our minimal system up, we quickly configure it (accounts, ssh keys, sudo, …). That minimal installation will stay around, and be our “rescue” OS. It takes 2GB, but our disk have alot of space, 2GB for some rescue full OS is not that much to pay.
Swap and /mnt partition are useless, remove them :
# swapoff -a # umount /mnt # sed -i '/swap/d;/mnt/d' /etc/fstab
LVM
Now that the swap is not used anymore, we can re-partition our disk, reclaim the swap partition, and create our LVM.
# fdisk /dev/sda
Command (m for help): p
[...]
Device Boot Start End Blocks Id System
/dev/sda1 * 2048 391167 194560 83 Linux
/dev/sda2 391168 4390911 1999872 83 Linux
/dev/sda3 4390912 6391807 1000448 82 Linux swap / Solaris
/dev/sda4 6391808 3906963455 1950285824 83 Linux
Command (m for help): d
Partition number (1-4): 3
Command (m for help): d
Partition number (1-4): 4
Command (m for help): n
Partition type:
p primary (2 primary, 0 extended, 2 free)
e extended
Select (default p): p
Partition number (1-4, default 3):
Using default value 3
First sector (4390912-3906963631, default 4390912):
Using default value 4390912
Last sector, +sectors or +size{K,M,G} (4390912-3906963631, default 3906963631):
Using default value 3906963631
Command (m for help): t
Partition number (1-4): 3
Hex code (type L to list codes): 8e
Changed system type of partition 3 to 8e (Linux LVM)
Command (m for help): p
[...]
Device Boot Start End Blocks Id System
/dev/sda1 * 2048 391167 194560 83 Linux
/dev/sda2 391168 4390911 1999872 83 Linux
/dev/sda3 4390912 3906963631 1951286360 8e Linux LVM
Command (m for help): w
The partition table has been altered!
Calling ioctl() to re-read partition table.
WARNING: Re-reading the partition table failed with error 16: Device or resource busy.
The kernel still uses the old table. The new table will be used at
the next reboot or after you run partprobe(8) or kpartx(8)
Syncing disks.
# reboot
For some reason the kernel still has the old partitions? Well, we don't have anything running, and it's the last time we edit the partitions anyway… reboot.
Now that we have the partition, we can start creating our LVM! I tend to call the VG with the hostname. My host will be named krikkit.
# apt-get install -y lvm2 [...] # pvcreate /dev/sda3 WARNING: swap signature detected on /dev/sda3. Wipe it? [y/n] y Wiping swap signature on /dev/sda3. Writing physical volume data to disk "/dev/sda3" Physical volume "/dev/sda3" successfully created # vgcreate krikkit /dev/sda3 Volume group "krikkit" successfully created # lvcreate -L 4G -n root krikkit Logical volume "root" created # lvcreate -L 1G -n swap krikkit Logical volume "swap" created # mkswap -f /dev/krikkit/swap Setting up swapspace version 1, size = 1048572 KiB no label, UUID=23effdae-53b7-4129-90a4-cf043e8cb775 # vgchange -ay 0 logical volume(s) in volume group "krikkit" now active # mkfs.ext3 /dev/krikkit/root [...] # mount /dev/krikkit/root /mnt
Ok, we have a nice to-be root partition. Time to get a Debian on it!
Bootstrap
Install a Debian, and configure to be able to boot it, and log in.
# apt-get install debootstrap # debootstrap stable /mnt http://http.debian.net/debian
Note: http://http.debian.net/debian is a way to get a nice “local mirror” automagicaly!
Get a nice place to chroot… remount some directories that will be usefull.
# for i in /sys /proc /dev /dev/pts /boot ; do mount -o bind $i /mnt$i ; done
Install packages that might be usefull to start with, and configure it.
# cp /etc/network/interfaces /mnt/etc/network/interfaces # chroot /mnt # cat <<EOF > /etc/fstab /dev/sda1 /boot ext3 defaults 0 2 /dev/krikkit/root / ext3 errors=remount-ro 0 1 EOF # echo krikkit > /etc/hostname # echo 127.0.1.1 krikkit >> /etc/hosts # passwd [...] # adduser ze [...] # apt-get install --purge nano- vim sudo openssh-server grub-pc linux-image-amd64 lvm2 less # sed -i /^PermitRootLogin/s/yes/no/ /etc/ssh/sshd_config # adduser ze sudo
Well, everything seems in place : - grub - kernel - hostname - fstab - network - ssh server - accounts
Just to make sure…
# cat /proc/mounts > /etc/mtab # update-grub Found linux image: /boot/vmlinuz-3.2.0-4-amd64 Found initrd image: /boot/initrd.img-3.2.0-4-amd64 done #
Ok, we can leave the chroot, and reboot the box!
Xen
Nice, we have a system on LVM. That's a very good base to get Xen and have some fun.
# apt-get install xen-system-amd64 [...]
Now, we have Xen, but default install tend to get some nasty stuffs in place. Like integrating eth0 in a bridge, some stupid iptables rules whenever you start a VM, … Let's just get rid of those, and handle the bridge ourselve.
# cat <<EOF >> /etc/network/interfaces
# br0 - bridge for VMs
auto br0
iface br0 inet static
address 10.0.0.1
netmask 255.255.255.0
bridge_ports none
bridge_maxwait 0
bridge_fd 0
EOF
Edit /etc/xen/xend-config.sxp, with :
## need to add (network-script network-route-manual) ## need to comment/add or update value # (vif-script vif-bridge) (vif-script vif-bridge-manual)
Now get those scripts ready:
# cd /etc/xen/scripts/ && cp network-route network-route-manual && cp vif-bridge vif-bridge-manual # cat <<EOF > network-route-manual #! /bin/sh echo 1 >/proc/sys/net/ipv4/ip_forward EOF # sed -i 's/handle_iptables/: &/' vif-bridge-manual
Now, on start, it will just enable ip_forward, and when a new VM is started, the interface will get in the correct bridge, but will NOT add any iptables rules.
Now, edit grub configuration for xen… Edit /etc/default/grub, and add:
GRUB_CMDLINE_XEN="dom0_mem=512M dom0_max_vcpus=1 dom0_vcpus_pin"
That will make sure the dom0 gets only 512MB, only 1 CPU, and the VCPU stays on the same CPU all the time.
Now, make grub default boot on xen:
# dpkg-divert --divert /etc/grub.d/08_linux_xen --rename /etc/grub.d/20_linux_xen # update-grub [...]
Ok, everything is ready!
# reboot
Once rebooted, you can check:
# xm list Name ID Mem VCPUs State Time(s) Domain-0 0 511 1 r----- 4.8
Ok… running xen !
create a VM
Ok, time to get a VM up!
First, make sure we don't try to save VM state between reboots (would need alot of diskspace for not much. Better shut it down, and start it again later!)
# sed -i 's/SAVE=.*/SAVE=/' /etc/default/xendomains
Get xen-tools. Those are great to create a new VM!
# apt-get install --no-install-recommends xen-tools
Now, edit the config: /etc/xen-tools/xen-tools.conf (only showing usefull modifications)
lvm = krikkit gateway = 10.0.0.1 netmask = 255.255.255.0
Ok, guess we can try to create one:
# xen-create-image --host vmtest --ip 10.0.0.2 --bridge br0 [...] Installation Summary --------------------- Hostname : vmtest Distribution : stable IP-Address(es) : 10.0.0.2 RSA Fingerprint : ca:aa:18:d8:d4:ad:bf:bb:c1:48:6f:ca:32:20:64:16 Root Password : cudeokMaun0 #
Not specifying the bridge might work. But if later you get multiple bridges, it would just get a “random” bridge, and not the one you'd expect. Better specify it, just to be sure :)
If you loose the output with the root password, it is logged in /var/log/xen-tools/vmtest.log.
# xm create -c /etc/xen/vmtest.cfg [...] vmtest login:
Ok, the VM is up and running. We can log in, could log in via network… would just need a little help to get some connectivity to internet. Like some nat…
On our dom0:
# apt-get install iptables-persistent # iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE # /etc/init.d/iptables-persistent save
ok, now we have a working VM able to talk to the net!
