I already blogged about my experiments with mcollective & xen but I had something a little bigger in my mind. A friend had sent me a video showing some vmware neat features (DRS mainly) with VMs migrating through hypervisors automatically.

So I wrote a “proof of concept” of what you can do with an awesome tool like mcollective. The setup of this funny game is the following :

• 1 box used a iSCSI target that serves volumes to the world
• 2 xen hypervisors (lenny packages) using open-iscsi iSCSI initiator to connect to the target. VMs are stored in LVM, nothing fancy

The 3 boxens are connected on a 100Mb network and the hypervisors have an additionnal gigabit network card with a crossover cable to link them (yes, this is a lab setup). You can find a live migration howto here.

For the mcollective part I used my Xen agent (slightly modified from the previous post to support migration), which is based on my xen gem. The client is the largest part of the work but it’s still less than 200 lines of code. It can (and will) be improved because all the config is hardcoded. It would also deserve a little DSL to be able to handle more “logic” than “if load is superior to foo” but as I said before, it’s a proof of concept.

Let’s see it in action :

test3 is a VM that is “artificially” loaded, as is the machine “hypervisor3” (to trigger migration)

Let’s see our hypervisors :

A little word about configuration options :

• interval : the poll time in seconds.  this should not be too low, let the machine some time and avoid load peeks to distort the logic.
• load_threshold : where you consider the machine load is too high and that it is time to move some stuff away (tampered with max_over, see below)
• daemonize : not used yet
• max_over : maximum time (in minutes) where load should be superior to the limit. When reached, it’s time, really. Don’t set it too low and at least 2*interval or sampling will not be efficient
• debug : well….
• max_vm_per_host : the maximum VMs a host can handle. If a host already hit this limit it will not be candidate for receiving a VM
• max_load_candidate : same thing as above, but for the load
• host_mapping : a simple CSV file to handle non-DNS destinations (typically my crossover cable address have no DNS entries)

What is left to do :

• Add some barriers to avoid migration madness to let load go down after a migration or to avoid migrating a VM permanently
• Add a DSL to insert some more logic
• Write a real client, not a big fat loop

Enjoy the tool !

Files :

• the xen agent (updated)
• the mc-xen-balancer client