ze's sandcastle

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
====== Xen - Dedibox ======

On a hosted server, with only a small install scenario, and not using a console, it is sometime hard to install everything like you'd like. Here is a small step by stop on how I used the lack-of-options install and moved my way up to a nice LVM and Xen setup.

===== Minimal installation =====

Web installation allows to install a debian, but does not support LVM. That's bad.

I decided to do a minimal installation. Here is how I partitionned the disk :
  * /boot - 200MB (enough for alot of kernels, initrd, ... don't need much else)
  * / - 2GB (does not allow to install with less for /)
  * swap - 1GB (does not allow to install without swap)
  * /mnt - everything else (install is buggy, would be included on / of left empty)

Once we have our minimal system up, we quickly configure it (accounts, ssh keys, sudo, ...). That minimal installation will stay around, and be our "rescue" OS. It takes 2GB, but our disk have alot of space, 2GB for some rescue full OS is not that much to pay.

Swap and /mnt partition are useless, remove them :
<code>
# swapoff -a
# umount /mnt
# sed -i '/swap/d;/mnt/d' /etc/fstab
</code>

===== LVM =====

Now that the swap is not used anymore, we can re-partition our disk, reclaim the swap partition, and create our LVM.

<code>
# fdisk /dev/sda

Command (m for help): p

[...]
   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *        2048      391167      194560   83  Linux
/dev/sda2          391168     4390911     1999872   83  Linux
/dev/sda3         4390912     6391807     1000448   82  Linux swap / Solaris
/dev/sda4         6391808  3906963455  1950285824   83  Linux

Command (m for help): d
Partition number (1-4): 3

Command (m for help): d
Partition number (1-4): 4

Command (m for help): n
Partition type:
   p   primary (2 primary, 0 extended, 2 free)
   e   extended
Select (default p): p
Partition number (1-4, default 3): 
Using default value 3
First sector (4390912-3906963631, default 4390912): 
Using default value 4390912
Last sector, +sectors or +size{K,M,G} (4390912-3906963631, default 3906963631): 
Using default value 3906963631

Command (m for help): t
Partition number (1-4): 3
Hex code (type L to list codes): 8e
Changed system type of partition 3 to 8e (Linux LVM)

Command (m for help): p

[...]
   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *        2048      391167      194560   83  Linux
/dev/sda2          391168     4390911     1999872   83  Linux
/dev/sda3         4390912  3906963631  1951286360   8e  Linux LVM

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.

WARNING: Re-reading the partition table failed with error 16: Device or resource busy.
The kernel still uses the old table. The new table will be used at
the next reboot or after you run partprobe(8) or kpartx(8)
Syncing disks.
# reboot
</code>

For some reason the kernel still has the old partitions? Well, we don't have
anything running, and it's the last time we edit the partitions anyway...
reboot.

Now that we have the partition, we can start creating our LVM! I tend to call the VG with the hostname. My host will be named krikkit.

<code>
# apt-get install -y lvm2
[...]
# pvcreate /dev/sda3 
WARNING: swap signature detected on /dev/sda3. Wipe it? [y/n] y
  Wiping swap signature on /dev/sda3.
  Writing physical volume data to disk "/dev/sda3"
  Physical volume "/dev/sda3" successfully created
# vgcreate krikkit /dev/sda3 
  Volume group "krikkit" successfully created
# lvcreate -L 4G -n root krikkit
  Logical volume "root" created
# lvcreate -L 1G -n swap krikkit
  Logical volume "swap" created
# mkswap -f /dev/krikkit/swap 
Setting up swapspace version 1, size = 1048572 KiB
no label, UUID=23effdae-53b7-4129-90a4-cf043e8cb775
# vgchange -ay
  0 logical volume(s) in volume group "krikkit" now active
# mkfs.ext3 /dev/krikkit/root 
[...]
# mount /dev/krikkit/root /mnt
</code>

Ok, we have a nice to-be root partition. Time to get a Debian on it!

===== Bootstrap =====

Install a Debian, and configure to be able to boot it, and log in.

<code>
# apt-get install debootstrap
# debootstrap stable /mnt http://http.debian.net/debian
</code>

Note: http://http.debian.net/debian is a way to get a nice "local mirror" automagicaly!

Get a nice place to chroot... remount some directories that will be usefull.

<code>
# for i in /sys /proc /dev /dev/pts /boot ; do mount -o bind $i /mnt$i ; done
</code>

Install packages that might be usefull to start with, and configure it.

<code>
# cp /etc/network/interfaces /mnt/etc/network/interfaces 
# chroot /mnt
# cat <<EOF > /etc/fstab
/dev/sda1         /boot ext3 defaults          0 2
/dev/krikkit/root /     ext3 errors=remount-ro 0 1
EOF
# echo krikkit > /etc/hostname
# echo 127.0.1.1 krikkit >> /etc/hosts
# passwd
[...]
# adduser ze
[...]
# apt-get install --purge nano- vim sudo openssh-server grub-pc linux-image-amd64 lvm2 less
# sed -i /^PermitRootLogin/s/yes/no/ /etc/ssh/sshd_config 
# adduser ze sudo
</code>

Well, everything seems in place :
- grub
- kernel
- hostname
- fstab
- network
- ssh server
- accounts

Just to make sure...
<code>
# cat /proc/mounts > /etc/mtab 
# update-grub
Found linux image: /boot/vmlinuz-3.2.0-4-amd64
Found initrd image: /boot/initrd.img-3.2.0-4-amd64
done
#
</code>

Ok, we can leave the chroot, and reboot the box!

===== Xen =====

Nice, we have a system on LVM. That's a very good base to get Xen and have some fun.

<code>
# apt-get install xen-system-amd64
[...]
</code>

Now, we have Xen, but default install tend to get some nasty stuffs in place.
Like integrating eth0 in a bridge, some stupid iptables rules whenever you
start a VM, ... Let's just get rid of those, and handle the bridge ourselve.

<code>
# cat <<EOF >> /etc/network/interfaces
# br0 - bridge for VMs
auto br0
iface br0 inet static
        address 10.0.0.1
        netmask 255.255.255.0
        bridge_ports none
        bridge_maxwait 0
        bridge_fd 0
EOF
</code>

Edit /etc/xen/xend-config.sxp, with :
<code>
## need to add
(network-script network-route-manual)
## need to comment/add or update value
# (vif-script vif-bridge)
(vif-script vif-bridge-manual)
</code>

Now get those scripts ready:
<code>
# cd /etc/xen/scripts/ && cp network-route network-route-manual && cp vif-bridge vif-bridge-manual
# cat <<EOF > network-route-manual
#! /bin/sh
echo 1 >/proc/sys/net/ipv4/ip_forward
EOF
# sed -i 's/handle_iptables/: &/' vif-bridge-manual 
</code>

Now, on start, it will just enable ip_forward, and when a new VM is started, the interface will get in the correct bridge, but will NOT add any iptables rules.

Now, edit grub configuration for xen... Edit /etc/default/grub, and add:
<code>

GRUB_CMDLINE_XEN="dom0_mem=512M dom0_max_vcpus=1 dom0_vcpus_pin"
</code>

That will make sure the dom0 gets only 512MB, only 1 CPU, and the VCPU stays on the same CPU all the time.

Now, make grub default boot on xen:
<code>
# dpkg-divert --divert /etc/grub.d/08_linux_xen --rename /etc/grub.d/20_linux_xen
# update-grub
[...]
</code>

Ok, everything is ready!

<code>
# reboot
</code>

Once rebooted, you can check:
<code>
# xm list
Name                                        ID   Mem VCPUs      State   Time(s)
Domain-0                                     0   511     1     r-----      4.8
</code>

Ok... running xen !

===== create a VM =====

Ok, time to get a VM up!

First, make sure we don't try to save VM state between reboots (would need alot of diskspace for not much. Better shut it down, and start it again later!)

<code>
# sed -i 's/SAVE=.*/SAVE=/' /etc/default/xendomains 
</code>

Get xen-tools. Those are great to create a new VM!

<code>
# apt-get install --no-install-recommends xen-tools
</code>

Now, edit the config: /etc/xen-tools/xen-tools.conf (only showing usefull modifications)
<code>
lvm = krikkit
gateway    = 10.0.0.1
netmask    = 255.255.255.0
</code>

Ok, guess we can try to create one:
<code>
# xen-create-image --host vmtest --ip 10.0.0.2 --bridge br0
[...]

Installation Summary
---------------------
Hostname        :  vmtest
Distribution    :  stable
IP-Address(es)  :  10.0.0.2 
RSA Fingerprint :  ca:aa:18:d8:d4:ad:bf:bb:c1:48:6f:ca:32:20:64:16
Root Password   :  cudeokMaun0

#
</code>

Not specifying the bridge might work. But if later you get multiple bridges, it would just get a "random" bridge, and not the one you'd expect. Better specify it, just to be sure :)

If you loose the output with the root password, it is logged in /var/log/xen-tools/vmtest.log.

<code>
# xm create -c /etc/xen/vmtest.cfg 
[...]
vmtest login: 
</code>

Ok, the VM is up and running. We can log in, could log in via network... would just need a little help to get some connectivity to internet. Like some nat...

On our dom0:
<code>
# apt-get install iptables-persistent
# iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
# /etc/init.d/iptables-persistent save
</code>

ok, now we have a working VM able to talk to the net!
ze's sandcastle

User Tools

Site Tools

Page Tools
