[one-users] RFC: Feature idea - Image preparation

Thu May 30 05:46:09 PDT 2013

Hi,

On Wed, May 29, 2013 at 6:01 AM, Jon <three18ti at gmail.com> wrote:

> Hello Carlos,
>
> You definitely read my mind.  :)  My next question is going to be about
> how to use a "golden image", I'm thinking this will become a two step
> process.  However, there are some migration implications that I'm not sure
> I understand yet, which I think may make the "golden image" hook a bit more
> complicated than the "finalize" or "prepare golden" hook.
>
> First I'd like to look at preparing the VM image for use as a golden
> image, or use as a base, non-persistent image.  Since the vm can
> be specified and cloned without any modification of the vm drivers (I'm
> currently doing this for my vms, copying 500MBs vs 2Gbs saves a ton of time
> on my "economy" server).  According to the docs, there are hooks for create
> and delete of images, is there a hook for cloning an image?
>

No, hooks are triggered only when a resource changes state. Instead of a
hook, you need to edit the clone driver script.

> Is it possible to only execute hooks under certain conditions?  I like
> your Idea of adding a custom attribute "PREPARE_IMAGE = Yes" to the image,
> then when you clone it, it performs all of the sys-prep and compressing
> steps as it's cloning the image.  Also, I didn't see a way to test for
> attributes in the hooks documentation, is it possible to do so?
>
> I'm thinking along the lines of:
>
> IMAGE_HOOK = [
>    name        =  "prep_image",
>    on             = "CLONE if $IMAGE['PREPARE_IMAGE'] = YES"
>    command  = "/var/lib/one/myhooks/prep_image"
>    arguments = "$BASE_IMAGE $NEW_IMAGE_ID"
> ]
>
>
You can't set conditions in oned.conf, but it is easy to check for that
condition inside the hook.
Hooks are executed as oneadmin, so you can query opennebula using the ruby
oca, or the cli. Or you can also use the $TEMPLATE attribute, and use xpath
to look for values. For example, [1] uses the xpath.rb script [2] and onevm
show -x.

> If the hook executes, does it stop other scripts from executing?  I
> Imagine the clone driver has a copy function, wold that also execute if my
> hook executed?
>
>
Just edit the clone script to make sure it is doing exactly what you expect.

> Finally, Is there a reason the page you linked [1] says: "*Development
> version of OpenNebula 4.2*. The material on this page needs to be
> reviewed for completeness and accuracy." in the heading?
> will these hooks work in 4.0?
>
>
Oops, looks like we copied the 4.2 hook guide to the 4.0 workspace, thanks!

Regards,
Carlos

[1]
http://dev.opennebula.org/projects/opennebula/repository/revisions/one-4.0/entry/src/tm_mad/ceph/mvds
[2]
http://dev.opennebula.org/projects/opennebula/repository/revisions/one-4.0/entry/src/datastore_mad/remotes/xpath.rb

> Thanks,
> Jon A
>
> [1] http://opennebula.org/documentation:rel4.0:hooks
>
> On Tue, May 28, 2013 at 7:07 AM, Carlos Martín Sánchez <
> cmartin at opennebula.org> wrote:
>
>> Hi,
>>
>> I just want to give you some general advice here: use hooks and drivers
>> as much as possible.
>>
>> For example, instead of adding a new xml-rpc call, oca method, CLI and
>> sunstone, you could simply add a new attribute to the Image or VM template.
>> MAKE_GOLDEN = YES or similar.
>> Doing this, you can forget about the user interface.
>>
>> Then you can look for that attribute when the VM is shutdown.
>> This could be done editing the 'mvds' TM script [1], the one executed for
>> the epilog phase. Or implement a hook [2] to be triggered after a VM is
>> shut down.
>>
>>
>> Good luck, and keep us posted with your progress!
>>
>>
>> [1] http://opennebula.org/documentation:rel4.0:sd
>> [2] http://opennebula.org/documentation:rel4.0:hooks
>>
>> --
>> Join us at OpenNebulaConf2013 <http://opennebulaconf.com> in Berlin,
>> 24-26 September, 2013
>> --
>> Carlos Martín, MSc
>> Project Engineer
>> OpenNebula - The Open-source Solution for Data Center Virtualization
>> www.OpenNebula.org | cmartin at opennebula.org | @OpenNebula<http://twitter.com/opennebula><cmartin at opennebula.org>
>>
>>
>> On Thu, May 23, 2013 at 7:13 AM, Jon <three18ti at gmail.com> wrote:
>>
>>> Hello All,
>>>
>>> Before I was using Open Nebula, my general virtual machine workflow was
>>> as follows:
>>>
>>> 1)  Install OS to qcow image, this creates an image between 500-1500MB
>>> 2)  "Unconfigure" virtual machine, e.g. remove any system specific
>>> config: udev rules, user accounts, ssh host keys, SELinux relabel etc.
>>> 3)  Zero out the file system by creating a large file of all zeros,
>>> e.g.: dd if=/dev/zero of=./delme; rm delme
>>> 4)  Compress and (qemu-img reclaims unused space while compressing so
>>> the "sparsify" is implied) the image, to create a "master" image (I've
>>> heard this referred to as a "golden" image)
>>>            this typically results in an image between 300-500MB,
>>> e.g. qemu-img convert -c -f qcow2 master.qcow2 -O qcow2
>>> master.compressed.qcow2
>>> 5)  Create a "slave" copy on write image off the master, e.g.:  qemu-img
>>> create -b ../master/master.compressed.qcow2 -f qcow2 ./virtualmachine.qcow2
>>>           This results in all of the changes being written to
>>> ./virtualmachine.qcow2
>>>
>>> Fortunately, virt-sysprep and virt-sparsify handle steps 2 and 3 and the
>>> sparcify part of step 4 more or less mindlessly.
>>> qemu-img is still required (as far as I know) to compress the image, but
>>> there is no longer a need to be in the vm to zero out the image,
>>> so this whole process can be automated through a management layer like
>>> OpenNebula.
>>>
>>> I would like to automate this work flow through OpenNebula.  I think it
>>> would be extremely useful to be able to prepare and "finalize" a virtual
>>> machine from OpenNebula.
>>> Finalizing an image in this manner allows for maximization of disk space
>>> as well as use as a "master" or "golden" image.
>>> In my mind, I think it makes sense to use the least amount of space for
>>> the common filesystem pieces.
>>>
>>> Basically, when I create a new image in OpenNebula, I create a template
>>> with the iso and a blank persistent datablock, then instantiate the vm,
>>> install the iso,
>>> shutdown the vm, and flip the image to a nonpersistent OS.
>>>
>>> What I am proposing is a feature that would add an additional step that
>>> would allow a user to "finalize" the vm by "virt-sysprep"ing,
>>> "virt-sparsify"ing,
>>> and compressing the image, to maximize space usage.
>>>
>>> -- My goal is to have a cli command, e.g.: `onevm finalize 38` or `onevm
>>> finilize ubuntu-server-13.04` as well as an option in Sunstone to finalize
>>> the image.
>>> It would effectively finalize and clone the vm so, if necessary, the
>>> original can still be modified and finalized again.
>>>
>>> Does any one have any other ideas, thoughts, comments, or suggestions
>>> for an image "finalization" function?  Is there anything I'm not thinking
>>> about that I should be?
>>>
>>> As far as development of the feature is concerned, where would I create
>>> this feature for a commandline function, can you point me at some useful
>>> documentation for developing this kind of feature?
>>> I'm not ready to tackle the sunstone piece of this yet I don't think, so
>>> I think my focus will be on the command line function for now.
>>>
>>> Thanks for your input,
>>> Jon A
>>>
>>> _______________________________________________
>>> Users mailing list
>>> Users at lists.opennebula.org
>>> http://lists.opennebula.org/listinfo.cgi/users-opennebula.org
>>>
>>>
>>
>
--
Join us at OpenNebulaConf2013 <http://opennebulaconf.com/> in Berlin, 24-26
September, 2013
--
Carlos Martín, MSc
Project Engineer
OpenNebula - The Open-source Solution for Data Center Virtualization
www.OpenNebula.org <http://www.opennebula.org/> | cmartin at opennebula.org |
@OpenNebula <http://twitter.com/opennebula> <cmartin at opennebula.org>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.opennebula.org/pipermail/users-opennebula.org/attachments/20130530/53a6af91/attachment-0002.htm>