[one-users] Transfer manager
Alexandre di Costanzo
adc at csse.unimelb.edu.au
Wed Nov 19 16:12:25 PST 2008
Hi all,
Thank you for all your answers.
I am working with Marcos and we have noticed that you have implemented
a disk image transfer mechanism in Open Nebula. Copying by NFS or SSH
the disk image each time the user submits a VM.
Within the system that we are currently working on, we have already
implemented a similar mechanism to address this need. It is done by
our system because we don't want to modify too much Open Nebula, which
is already doing a lot things. Hence, we have done some experiments
and what we observed is that it takes a very long time to copy the
disk image each time (specially with NFS), about 1 minute and half for
a file of 2GB. It is certainly due to our cheap testbed: 4 linux
workstation on a 100Mb network.
In order to hide this lag, we have implemented in our system a cache
that pre-copy disk images before submitting a VM, and start a new copy
each time a VM is started for having a disk image ready for a the next
submission. We can do that because we manage a template directory that
the user can choose which template to submit. I will sent more details
later if you are interested (I am writing a technical report
describing what we are doing).
Thus, our wonder is may you set the use of Transfer manager optional?
In other word, let the user decide if he wants to use the TM or not.
Because it seams to be mandatory when we add a host to set the TM, or
I missed something?
However, it is great a news for us that you handle the disk image
transfer.
Alex
--
Dr Alexandre di Costanzo
Research Fellow - Grid Computing and Distributed Systems (GRIDS)
Laboratory
Department of Computer Science and Software Engineering
The University of Melbourne, Australia
T: +61 3 8344 1346
E: adc at csse.unimelb.edu.au
W: www.csse.unimelb.edu.au/~adc
On 19/11/2008, at 10:27 PM, Javier Fontan wrote:
>
> Hello Marcos,
>
> You are correct, the problem is that the deployment file is not
> being copied to that directory. The place where the deployment file
> is copied to the "images" directory is in VMM Xen driver,
> specifically in http://trac.opennebula.org/browser/one/trunk/src/vmm_mad/xen/one_vmm_xen.rb#L55
>
> I have updated the driver to log the copy to the vm log file. If you
> update your files the copy should appear logged in that file.
> Anyhow, it is good do have debugging enabled for drivers when errors
> happen, you have more information when things fail. Edit etc/
> defaultrc and set ONE_MAD_DEBUG to 1. After you restart oned there
> will be more log files in var directory, one for each driver. I hope
> you will get the information needed to make it work. Ask again if
> there are still problems.
>
> Concerning the VirtualNetworkingManager, we are still in the process
> of writing documentation. We will let you know when something is
> ready.
>
> Bye
>
>
> On Nov 19, 2008, at 8:07 AM, Marcos Dias de Assuncao wrote:
>
>>
>> Hi Tino,
>>
>> thanks for your e-mail. We have updated OpenNebula on our testbed,
>> but we had a few problems and could not test the case below.
>>
>> We have noticed that you have introduced a transfer manager that
>> copies the file system images and the deployment file to the remote
>> hosts. It looks interesting, but we have a problem with that. We
>> are using NFS and noticed that OpenNebula is cloning the images and
>> creating a subdirectory called $ONE_LOCATION/var/vmid/images, where
>> the disk image is placed. Also, I looked at the source code and
>> found that the Xen driver now expects a remote deployment file.
>> However, it seems that OpenNebula is not copying the deployment
>> file to the remote location. I could not find where that is done in
>> the source code. The error we get is the following:
>>
>> cat ../OpenNebula/var/2/vm.log
>>
>> Wed Nov 19 17:14:52 2008 [DiM][I]: New VM state is ACTIVE.
>> Wed Nov 19 17:14:52 2008 [LCM][I]: New VM state is PROLOG.
>> Wed Nov 19 17:14:52 2008 [TM][I]: tm_clone.sh: gieseking:/home/
>> oneadmin/vm/domains/xen01.localhost/disk.img billabong:/home/
>> oneadmin/OpenNebula/var/2/images/disk.0
>> Wed Nov 19 17:14:52 2008 [TM][I]: tm_clone.sh: DST: /home/oneadmin/
>> OpenNebula/var/2/images/disk.0
>> Wed Nov 19 17:14:52 2008 [TM][I]: tm_clone.sh: Creating directory /
>> home/oneadmin/OpenNebula/var/2/images
>> Wed Nov 19 17:14:52 2008 [TM][I]: tm_clone.sh: Executed "mkdir -p /
>> home/oneadmin/OpenNebula/var/2/images".
>> Wed Nov 19 17:14:52 2008 [TM][I]: tm_clone.sh: Executed "chmod a+w /
>> home/oneadmin/OpenNebula/var/2/images".
>> Wed Nov 19 17:14:52 2008 [TM][I]: tm_clone.sh: Cloning /home/
>> oneadmin/vm/domains/xen01.localhost/disk.img
>> Wed Nov 19 17:16:35 2008 [TM][I]: tm_clone.sh: Executed "cp /home/
>> oneadmin/vm/domains/xen01.localhost/disk.img /home/oneadmin/
>> OpenNebula/var/2/images/disk.0".
>> Wed Nov 19 17:16:35 2008 [TM][I]: tm_clone.sh: Executed "chmod a+w /
>> home/oneadmin/OpenNebula/var/2/images/disk.0".
>> Wed Nov 19 17:16:36 2008 [LCM][I]: New VM state is BOOT
>> Wed Nov 19 17:16:36 2008 [VMM][I]: Generating deployment file: /
>> home/oneadmin/OpenNebula/var/2/deployment.0
>> Wed Nov 19 17:16:37 2008 [VMM][E]: Error deploying virtual machine:
>> Unable to open config file: /home/oneadmin/OpenNebula/var/2/images/
>> deployment.0
>> Wed Nov 19 17:16:37 2008 [LCM][I]: Fail to boot VM.
>> Wed Nov 19 17:16:37 2008 [DiM][I]: New VM state is FAILED
>> Wed Nov 19 17:29:04 2008 [DiM][I]: New VM state is DONE.
>>
>> The content os the deployment directory is:
>> ls ../OpenNebula/var/2
>>
>> deployment.0 images transfer.0 vm.log
>>
>> And the images subdirectory is:
>> ls ../OpenNebula/var/2/images/
>>
>> disk.0
>>
>> Our deployment file is:
>>
>> cat xen.template
>> NAME = xen01
>> CPU = 0.5
>> MEMORY = 128
>> OS = [kernel="/home/oneadmin/vm/kernels/vmlinuz-2.6.24-21-xen",
>> initrd="/home/oneadmin/vm/kernels/
>> initrd.img-2.6.24-21-xen",
>> root="hda1" ]
>> DISK = [source="/home/oneadmin/vm/domains/xen01.localhost/
>> disk.img",
>> target="hda1",
>> readonly="no"]
>> NIC = [ mac="00:16:3e:01:01:01"]
>> GRAPHICS = [type="vnc",listen="127.0.0.1",port="5900"]
>>
>>
>> In addition, we have also found that you guys are implementing some
>> features that are useful to us, such as virtual networking. We were
>> wondering whether we could have access to use or design
>> documentation as well as the timeline for the implementation?
>>
>> Regards,
>>
>> Marcos
>>
>>
>> On 19/11/2008, at 1:26 AM, Tino Vazquez wrote:
>>
>>> Hi there,
>>>
>>> We've opened a ticket to keep track of the bug:
>>>
>>> http://trac.opennebula.org/ticket/52
>>>
>>> Also, I would like to let you know it has been already fixed in
>>> the Trunk of OpenNebula SVN:
>>>
>>> http://trac.opennebula.org/changeset/245
>>> http://trac.opennebula.org/changeset/246
>>>
>>> Now it should work as Rubén suggested in his last email. You are
>>> invited to give it a try and see if it works for you.
>>>
>>> Best regards,
>>>
>>> -Tino
>>>
>>> On Mon, Nov 17, 2008 at 6:32 AM, Marcos Dias de Assuncao <marcosd at csse.unimelb.edu.au
>>> > wrote:
>>>
>>> Hi guys,
>>>
>>> I am working with Alexandre Di Costanzo, who has sent some
>>> messages to the list before.
>>>
>>> We are using OpenNebula with OpenSuse 11, Ubuntu 8.04 and Xen 3.2.
>>> Some of our hosts run OpenSuse whereas others run Ubuntu. We have
>>> created Ubuntu paravirtualized images and OpenSuse images to run
>>> as guests. We do not have problems scheduling the Ubuntu virtual
>>> machines, but we have encountered some issues with OpenSuse. So,
>>> we thought that it would be interesting to post something about
>>> the problem to the list and check if other people have had the
>>> similar problems and get some help.
>>>
>>> When OpenNebula generates the deployment file (i.e. xen guest
>>> description file), it appends 'ro' to the description of the file
>>> system to be mounted as root. In this way, if I specify an image
>>> to be mapped to hda1, the deployment file will contain something
>>> like "root = '/dev/hda1 ro'". We believe this is to force fsck to
>>> check the file system at boot time. Ubuntu images are pretty ok
>>> with that, but our OpenSuse images do not seem very happy in
>>> mounting the file system as read-only during boot.
>>>
>>> We have performed some tests with some Xen guest description files
>>> for OpenSuse and it works only if we specify 'rw' as the boot
>>> parameter for 'root=' or nothing, which will assume 'rw' I guess.
>>> For example, one description file that works is the following:
>>>
>>>
>>>
>>> kernel = '/boot/vmlinuz-2.6.25.18-0.2-xen'
>>> ramdisk = '/boot/initrd-2.6.25.18-0.2-xen'
>>> #root = '/dev/hda1'
>>> root = '/dev/hda1 rw'
>>> name="susebox"
>>> memory=256
>>> vcpus=1
>>> on_poweroff="destroy"
>>> on_reboot="restart"
>>> on_crash="destroy"
>>> disk=[ 'tap:aio:/path/to/file/OpenSuse11.img,hda1,w']
>>> vif=[ 'mac=00:16:3e:01:01:06' ]
>>> extra = '3 console=xvc0'
>>>
>>>
>>>
>>> The configuration above works fine, but if we change (root = '/dev/
>>> hda1 rw') to (root = '/dev/hda1 ro'), we get OpenSuse complaining
>>> about file permissions and finally hangs at the initialisation of
>>> HAL daemon. Part of the output is shown below:
>>>
>>>
>>>
>>> rm: cannot remove `/success': Read-only file system
>>> failed
>>> Mounting local file systems...
>>> mount: according to mtab, proc is already mounted on /proc
>>>
>>> mount: according to mtab, sysfs is already mounted on /sys
>>>
>>> mount: according to mtab, debugfs is already mounted on /sys/
>>> kernel/debug
>>>
>>> mount: according to mtab, udev is already mounted on /dev
>>>
>>> loop: module loaded
>>> mount: according to mtab, devpts is already mounted on /dev/pts
>>>
>>> nothing was
>>> mounted failed
>>> mv: cannot move `/var/log/boot.msg' to `/var/log/boot.omsg': Read-
>>> only file system
>>> Creating /var/log/boot.msg
>>> Can't open or create /var/run/
>>> klogd.pid. done
>>> /etc/init.d/boot.klog: line 39: /var/log/boot.msg: Read-only file
>>> system
>>> touch: cannot touch `/var/run/utmp': Read-only file system
>>> chmod: changing permissions of `/var/run/utmp': Read-only file
>>> system
>>> chown: changing ownership of `/var/run/utmp': Read-only file system
>>> chown: changing ownership of `/tmp': Read-only file system
>>> chown: changing ownership of `/tmp/.X11-unix': Read-only file system
>>> chown: changing ownership of `/tmp/.ICE-unix': Read-only file system
>>> chown: changing ownership of `/var/tmp': Read-only file system
>>> chown: changing ownership of `/var/tmp/vi.recover': Read-only file
>>> system
>>> chown: changing ownership of `/var/run/uscreens': Read-only file
>>> system
>>> chown: changing ownership of `/var/run/screens': Read-only file
>>> system
>>> Setting up hostname
>>> 'opensuse' done
>>> Setting up loopback interface lo
>>> lo IP address: 127.0.0.1/8
>>> IP address: 127.0.0.2/8
>>> done
>>> Serial: 8250/16550 driver $Revision: 1.90 $ 4 ports, IRQ sharing
>>> enabled
>>> Configuring serial ports...
>>> Configured serial
>>> ports done
>>> Activating remaining swap-devices in /etc/
>>> fstab... done
>>> Setting current sysctl status from /etc/sysctl.conf
>>> net.ipv4.icmp_echo_ignore_broadcasts = 1
>>> net.ipv4.conf.all.rp_filter = 1
>>> fs.inotify.max_user_watches = 65536
>>> net.ipv4.conf.default.promote_secondaries = 1
>>> net.ipv4.conf.all.promote_secondaries = 1
>>> done
>>> Enabling syn flood
>>> protection done
>>> Disabling IP
>>> forwarding done
>>> done
>>> System Boot Control: The system has
>>> been set up
>>> Skipped
>>> features
>>> :
>>> boot.cycle
>>> System Boot Control: Running /etc/init.d/
>>> boot.local done
>>> blogd: no message logging because /var file system is not accessible
>>> INIT: Entering runlevel: 3
>>> blogd: can not open /var/run/blogd.pid: Read-only file system
>>> Boot logging started on /dev/xvc0(/dev/console) at Tue Nov 18
>>> 03:22:02 2008
>>> Master Resource Control: previous runlevel: N, switching to
>>> runlevel: 3
>>> Starting D-Bus daemonFailed to start message bus: Failed to bind
>>> socket "/var/run/dbus/system_bus_socket": Read-only file system
>>> failed
>>> Starting
>>> ConsoleKit done
>>> Loading CPUFreq modules (CPUFreq not supported)
>>> Starting HAL daemon
>>>
>>>
>>> If we try to start the same OpenSuse image with an Ubuntu's
>>> kernel, it actually boots, but then we have other problems with
>>> some modules. I am just wondering if anybody has faced a similar
>>> problem?
>>>
>>> Also, just one last thing. About the NIC option in the VM
>>> template. I am just wondering, it might be interesting to add a
>>> description to the documentation saying which drivers use the
>>> SCRIPT option.
>>>
>>> Thanks for your support,
>>>
>>> Marcos
>>>
>>>
>>> Marcos Dias de Assuncao
>>> Grid Computing and Distributed Systems (GRIDS) Laboratory
>>> Department of Computer Science and Software Engineering
>>> The University of Melbourne, Australia
>>> Email: marcosd at csse.unimelb.edu.au
>>>
>>> -------------
>>> "The bankers own the earth. Take it away from them, but leave them
>>> the power to create money, and with the flick of the pen they will
>>> create enough money to buy it back again.
>>> However, take away from them the power to create money, and all
>>> the great fortunes like mine will disappear and they ought to
>>> disappear, for this would be a happier and better world to live
>>> in. But, if you wish to remain the slaves of bankers and pay the
>>> cost of your own slavery, let them continue to create money."
>>>
>>> Sir Josiah Stamp
>>> Former Director of The Bank of England
>>>
>>> _______________________________________________
>>> Users mailing list
>>> Users at lists.opennebula.org
>>> http://lists.opennebula.org/listinfo.cgi/users-opennebula.org
>>>
>>>
>>>
>>> --
>>> Constantino Vázquez, Grid Technology Engineer/Researcher: http://www.dsa-research.org/doku.php?id=people:tinova
>>> DSA Research Group: http://dsa-research.org
>>> Globus GridWay Metascheduler: http://www.GridWay.org
>>> OpenNebula Virtual Infrastructure Engine: http://www.OpenNebula.org
>>> _______________________________________________
>>> Users mailing list
>>> Users at lists.opennebula.org
>>> http://lists.opennebula.org/listinfo.cgi/users-opennebula.org
>>
>> Marcos Dias de Assuncao
>> Grid Computing and Distributed Systems (GRIDS) Laboratory
>> Department of Computer Science and Software Engineering
>> The University of Melbourne, Australia
>> Email: marcosd at csse.unimelb.edu.au
>>
>> -------------
>> "The bankers own the earth. Take it away from them, but leave them
>> the power to create money, and with the flick of the pen they will
>> create enough money to buy it back again.
>> However, take away from them the power to create money, and all the
>> great fortunes like mine will disappear and they ought to
>> disappear, for this would be a happier and better world to live in.
>> But, if you wish to remain the slaves of bankers and pay the cost
>> of your own slavery, let them continue to create money."
>>
>> Sir Josiah Stamp
>> Former Director of The Bank of England
>>
>> _______________________________________________
>> Users mailing list
>> Users at lists.opennebula.org
>> http://lists.opennebula.org/listinfo.cgi/users-opennebula.org
>
> --
> Javier Fontan, Grid & Virtualization Technology Engineer/Researcher
> DSA Research Group: http://dsa-research.org
> Globus GridWay Metascheduler: http://www.GridWay.org
> OpenNebula Virtual Infrastructure Engine: http://www.OpenNebula.org
>
> _______________________________________________
> Users mailing list
> Users at lists.opennebula.org
> http://lists.opennebula.org/listinfo.cgi/users-opennebula.org
More information about the Users
mailing list