Xen Networking

Cloudmin
1

So i use CSF to manage iptables on my servers and its a nice bonus that it actually has a webmin module so it can be managed from webin webmin as well. Anyway, my servers (dom0’s) are also setup to be their own gateways. Not a big deal except that when running xen guests on it, you have to get a bit tricky. Normally i just run a csfpost.sh script that adds the following rules:

iptables -A FORWARD -m physdev --physdev-in peth0 --physdev-out viflnca0 -j ACCEPT

iptables -A FORWARD -m physdev --physdev-out peth0 --physdev-in viflnca0 -j ACCEPT

What stinks though is that when xen instances are created, no actual vif[] settings are created in the instances xen config and it would only have something like: vif = [ ‘’ ]

So what do i do now? Right now, since i have no generic interface to set the rules to, my instances cant communicate with the outside work. Any tips?

2

HI MACscr,

I think i saw some “Actions on Xen state changes” somewhere in the XEN create system screen.

If somewhere in Cloudmin, for this particular instance, there would be an “Scripts to execute on Xen state changes” (Start, Stop, Pause, Delete) , then it could be possible to create a script that would read the .cfg file, find the “vif = [’’]” line and read between the brackets

Till then i made my own try and dig.

While the creation was being done, i’ve made a sneak peek to the new machine .cfg file

[root@centos5 xen]# cat centos5.cfg
kernel = ‘/xen/vmlinuz-vm2-xenU’
ramdisk = ‘/xen/initrd.vm2.xenU.img’
memory = 512
name = ‘centos5’
vif = [ ‘’ ]
address = ‘192.168.1.180’
netmask = ‘255.255.255.0’
disk = [‘file:/xen/centos5.img,sda1,w’,‘file:/xen/centos5.swap,sda2,w’]
root = ‘/dev/sda1 ro’

And it was like that even after i’ve started the VM. No vif info…

I did give another go and, this time i’ve created a second machine but, i decided to give a name to the interface. When creating a new Xen System, you are able to go to Advanced options and give the interface name at “Virtual interface name” (choose custom name). I gave the name flc4 (not flc4.4 or flc4.0 … it gaved me a some errors).

So after the machine was created, i could see its .cfg file and guess what !
[root@centos5 xen]# cat centos5dois.cfg
kernel = ‘/xen/vmlinuz-vm2-xenU’
ramdisk = ‘/xen/initrd.vm2.xenU.img’
memory = 256
name = ‘centos5dois’
vif = [ ‘vifname=flc4’ ]
address = ‘192.168.1.181’
netmask = ‘255.255.255.0’
disk = [‘file:/xen/centos5dois.img,sda1,w’]
root = ‘/dev/sda1 ro’
[root@centos5 xen]#

I could find a vifname on this line, the one I’ve configured. So, after I started the VM, making command bellow:
#>ifconfig

flc4 Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF
inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
UP BROADCAST RUNNING NOARP MTU:1500 Metric:1
RX packets:35 errors:0 dropped:0 overruns:0 frame:0
TX packets:1838 errors:0 dropped:330 overruns:0 carrier:0
collisions:0 txqueuelen:32
RX bytes:3526 (3.4 KiB) TX bytes:280119 (273.5 KiB)

This is just another interface that i believe that you can now control with iptables.

I do now that it is possible to configure ip’s and mac address’s also on the vif line.

So, in the end, just personalize your interface and add it to your iptable’s rules.

Scripting might help to charge rules on the fly on iptables, reading the .cfg file, extracting the vif content , and adding it to iptables configuration, on start up VMs. Doing exactly the same when Shutting down VMs. But that only when Cloudmin allows to run scripts on Xen Change Status.

Hope it helps,

Fil

3

I changed the rule to the following and this seems to fix things without any other changes:

iptables -A FORWARD -m physdev --physdev-in peth0 --physdev-out vif+ -j ACCEPT
iptables -A FORWARD -m physdev --physdev-out peth0 --physdev-in vif+ -j ACCEPT

4

Nice ! It seems vif+ is considered a wildcard then. That sound’s good.
F

5

yep, thats exactly what it is, + is a wildcard. Ive created a few instances with it so far and seems to work well so far.

6

Now that i think of it though, if were going to allow clients to create multiple instances, we need the ability to easily setup private lans for them as well. Hmm.

7

Interesting… maybe there should be a way of knowing the address’s in use, or provide a “private” pools of address’s for clients.

8

Well, cant xen even do virtual vlans?

9

I do believe so, anf after digging a bit, i’ve found a possible answer in renial.net. Here is the link:
http://renial.net/weblog/2007/02/27/xen-vlan/

I think, we might achieve vlan’s with this approach, by tristanb.

You do want to isolate each machine right? I do think that this approach can make isolation on machines, but if you want to configure a vlan for several VMs, probably would be just using the same vlan for all of them…

Also at pixelchaos.net, http://www.pixelchaos.net/2008/07/16/vlan-bridging-in-xen/ is another approach to the problem. And a nice comment afterwords

"It’s possible to name your Xen interfaces so that your output from various commands looks a little nicer and makes it easier to tell what’s what when you’re looking at interface listings.
I like to use the primary function of the domU as my interface names. So I end up with interfaces like ‘www0′ or ‘mail0′.

You can do this by using ‘vifname=somename’ in the ‘vif = [ ]‘ config for each of your domUs."

Maybe a vlan for each client instead for each machine ? Dont know… i still have to test it though…

Regards,
F

10

LOL, you wont believe it, but i found out those rules didnt work for HVM’s and also guests werent able to communicate with each other, so here are the updated rules:

iptables -A FORWARD -m physdev --physdev-in peth1 --physdev-out vif+ -j ACCEPT
iptables -A FORWARD -m physdev --physdev-out peth1 --physdev-in vif+ -j ACCEPT
iptables -A FORWARD -m physdev --physdev-in peth0 --physdev-out vif+ -j ACCEPT
iptables -A FORWARD -m physdev --physdev-out peth0 --physdev-in vif+ -j ACCEPT
iptables -A FORWARD -m physdev --physdev-in peth1 --physdev-out tap+ -j ACCEPT
iptables -A FORWARD -m physdev --physdev-out peth1 --physdev-in tap+ -j ACCEPT
iptables -A FORWARD -m physdev --physdev-in peth0 --physdev-out tap+ -j ACCEPT
iptables -A FORWARD -m physdev --physdev-out peth0 --physdev-in tap+ -j ACCEPT
iptables -A FORWARD -m physdev --physdev-out vif+ --physdev-in vif+ -j ACCEPT
iptables -A FORWARD -m physdev --physdev-in vif+ --physdev-out vif+ -j ACCEPT
iptables -A FORWARD -m physdev --physdev-out tap+ --physdev-in tap+ -j ACCEPT
iptables -A FORWARD -m physdev --physdev-in tap+ --physdev-out tap+ -j ACCEPT
iptables -A FORWARD -m physdev --physdev-out vif+ --physdev-in tap+ -j ACCEPT
iptables -A FORWARD -m physdev --physdev-in vif+ --physdev-out tap+ -j ACCEPT
iptables -A FORWARD -m physdev --physdev-out tap+ --physdev-in vif+ -j ACCEPT
iptables -A FORWARD -m physdev --physdev-in tap+ --physdev-out vif+ -j ACCEPT

11

I have two nics in that box for xen (then two more for iscsi), so thats why you see the peth0 and peth1

12

this work well for my DOMU

iptables -A FORWARD -i vif+ -j ACCEPT


iptables -A FORWARD -o vif+ -j ACCEPT