By default, when you get 2 DomUs running, they both get an interface frontend within the domU (usualy called “eth0”), and an interface backend in the dom0 (usualy called vifX.0 – X is the dom id, and 0 is the interface number for that domU).

.---------------.        .--------------------.
|     Dom0      |        |     DomU: foo      |
|---------------|        |--------------------|
|      .---------.      .--------.            |
|      | vifX.0  |<-----|  eth0  |            |
|      '---------'      '--------'            |
|               |        '--------------------'
|      .---------.
'------| vifY.0  |       .--------------------.
       '---------'       |     DomU: bar      |
            ^            |--------------------|
            |           .--------.            |
            '-----------|  eth0  |            |
                        '--------'            |
                         '--------------------'

To avoid having too much network configuration each time you start/stop a domU, the easiest way is to have a bridge ready on the dom0, and use some script to add the backend interface to.

When /etc/init.d/xen starts, it runs the configured network-script. The configuration is made in /etc/xen/xend-config.sxp, script is searched in /etc/xen/scripts.

That network script is supposed to do an initial setup for the incoming domUs network backends. Prepare to forward, create the bridges, and whatever else you need. Most could be done directly in the rest of your system (/etc/network/interfaces, /etc/sysctl.conf, and your firewall configuration)

When a new backend is presented, the kernel will create the interface, and xen will run the configured vif-script. That script should do what's necessary to get your interface usable. It's really like if you inserted a new network card, with a cable already plugged to a single remote host.

As you may have multiple of such, and usualy don't want starting/stopping your domUs messing with your dom0 network configuration, stable bridges, and simply having the domU plugging itself to it, bridge seems the easiest way.

When you tend to have multiple DomU that should be split into different networks, the usual configuration is to have a bridge on the Dom0 for each network segment, and the DomUs to join them.

If you want to also include a backend between two of your DomUs, you would have to get a bridge just for the two of them, and try to avoid Dom0's firewalling those packets.

.-----------------.
|      Dom0       |
|-----------------|               .-----------.
|  .----------.   |               | DomU: foo |
|  | br0      |   |               |-----------|
|  |      .--------.            .------.      |
|  |      | vifX.0 |<---------->| eth0 |      |
|  |      '--------'            '------'      |
|  |      .--------.            .------.      |
|  |      | vifY.0 |<-.      .->| eth1 |      |
|  |      '--------'   \    /   '------'      |
|  '----------'   |     \  /      '-----------'
|                 |      \/       .-----------.
|  .----------.   |      /\       | DomU: bar |
|  | br1      |   |     /  \      |-----------|
|  |      .--------.   /    \   .------.      |
|  |      | vifX.1 |<-'      `->| eth0 |      |
|  |      '--------'            '------'      |
|  |      .--------.            .------.      |
|  |      | vifY.1 |<---------->| eth1 |      |
|  |      '--------'            '------'      |
|  '----------'   |               '-----------'
'-----------------'

• br1: only used for foo/bar communications.
• br0: used for external network connectivity

Xen configuration files:

foo.cfg:

      vif = [
	'mac=00:16:3E:00:00:02,bridge=br0',
	'mac=00:16:3E:00:01:02,bridge=br1',
      ]


bar.cfg:

      vif = [
	'mac=00:16:3E:00:00:03,bridge=br0',
	'mac=00:16:3E:00:01:03,bridge=br1',
      ]

On br1, you would have to take very extra care not to firewall anything. When both DomU talks to each other through br1, Dom0 would copy the data around, and send it back to the other DomU.

Lets say we would like foo and bar to talk directly to each other. Network connections should look something like :

.-----------------.
|      Dom0       |
|-----------------|               .-----------.
|  .----------.   |               | DomU: foo |
|  | br0      |   |               |-----------|
|  |      .--------.            .------.      |
|  |      | vifX.0 |<---------->| eth0 |      |
|  |      '--------'            '------'      |
|  |      .--------.            .---------.   |
|  |      | vifY.0 |<-.     .-->| vif.bar |   |
|  |      '--------'   \    |   '---------'   |
|  '----------'   |     \   |     '-----------'
|                 |      \  |     .-----------.
|                 |       \ |     | DomU: bar |
|                 |        \|     |-----------|
|                 |         \   .------.      |
|                 |         |`->| eth0 |      |
|                 |         |   '------'      |
|                 |         |   .------.      |
|                 |         '-->| eth1 |      |
|                 |             '------'      |
|                 |               '-----------'
'-----------------'

Well, it is possible. You just have to get the network backend in a different domain (not the dom0, but an other existing domain!)

Here is an example of configuration for such:

foo.cfg:

      vif = [
	'mac=00:16:3E:00:00:02,bridge=br0',
      ]


bar.cfg:

      vif = [
	'mac=00:16:3E:00:00:03,bridge=br0',
	'mac=00:16:3E:00:01:03,backend=bar,script=vif-bridge-manual,bridge=brbar',
      ]

Lets see the different options :

• ip - supposed IP of the DomU. Can be used for documentation, accessible by the vif script if it require to deal stupid firewall rules (As you can see, I don't use that option)
• mac - mac address of the interface on the DomU
• backend - name of the domain that should get the backend. Default is the dom0
• script - name of the script to launch on the backend domain when the interface is added. The script must be present on the specified domain
• bridge - option passed to the vif script, usually the name of the bridge the interface backend should be added to.

With all that, we can get a whole network between our DomUs, without the dom0 seeing any of those interfaces!