[one-users] Message from CERN

[sebastien goasguen]

Dear opennebula team,

we have been testing opennebula, along with other solutions, extensively over
the last 6 months at CERN, and we would like to give you a quick
update of where
we are with it now. Over the whole time period, we've experienced an
excellent and
competent support by the developers.

We have demonstrated that we can manage more than 1,000 virtual
machines with it, and have reached up to 7,500 Virtual Machines running
on approximately 400 physical hosts. Our hosts run Xen and are managed
by our proven configuration management system Quattor; we added
functionality to our hosts so that when they go through their own
lifecycle (testing, production, maintenance etc..), they automatically
notify opennebula and disable or enable themselves using the One XML-RPC
server. This has added some autonomic property to our environment with
self-notificiation of hosts in opennebula. We also use the XML-RPC server
to self-delete VMs and autonomically manage the shares of various VM
types. Programming using your API has proven quite useful and easy.

We are also using the MAC pinning solution that you developed for us.
Each host maintains a list of available MAC adresses for VMs running on
that particular host. A One probe is setup to check this list and
schedule VMs only on hosts that have obtained a MAC lease that matches
one of the MAC in the list. This was required by our networking
infrastructure to be able to track the VMs properly.

In terms of image transfer we are using your LVM transfer scripts that
we helped to develop and test. These are working quite nicely and allow us
to start VMs extremely quickly. We pre-stage a base image on all the
nodes, and the transfer script creates LVM snapshots. This way we
demonstrated that we can handle 20 snapshots on each host easily, and ONE
cleans up everything when VMs are deleted. Since we don't have a shared
file system that is usable for this, we have tested a simple parallel scp
program to push the images to all the hosts. This enables us to push a
~10GB image to ~500 hosts in about one hour.
We are also currently testing a bittorrent network between all our
hosts, early testing is very promising and we are confident to have some
performance numbers soon for this system.

Currently we are pushing your VM scheduler quite hard, trying to
dispatch over 8,000 VMs every 3 hours, it seems that the scheduler can
achieve 1VM/sec which is quite good but not yet fast enough for us. We look
forward for improvements in this area, and count on your continued support
to improve and harden the nice bit of free software.

-Sebastien Goasguen for the CERN batch virtualization team

-- 
---
Sebastien Goasguen
Assistant Professor and CERN scientific associate
School of Computing
Clemson University
864-553-4734
http://sites.google.com/site/runseb/