[one-users] RFC: Feature idea - Image preparation

[Jon]

Hello Carlos,

You definitely read my mind.  :)  My next question is going to be about how
to use a "golden image", I'm thinking this will become a two step process.
 However, there are some migration implications that I'm not sure I
understand yet, which I think may make the "golden image" hook a bit more
complicated than the "finalize" or "prepare golden" hook.

First I'd like to look at preparing the VM image for use as a golden image,
or use as a base, non-persistent image.  Since the vm can be specified and
cloned without any modification of the vm drivers (I'm currently doing this
for my vms, copying 500MBs vs 2Gbs saves a ton of time on my "economy"
server).  According to the docs, there are hooks for create and delete of
images, is there a hook for cloning an image?  Is it possible to only
execute hooks under certain conditions?  I like your Idea of adding a
custom attribute "PREPARE_IMAGE = Yes" to the image, then when you clone
it, it performs all of the sys-prep and compressing steps as it's cloning
the image.  Also, I didn't see a way to test for attributes in the hooks
documentation, is it possible to do so?

I'm thinking along the lines of:

IMAGE_HOOK = [
   name        =  "prep_image",
   on             = "CLONE if $IMAGE['PREPARE_IMAGE'] = YES"
   command  = "/var/lib/one/myhooks/prep_image"
   arguments = "$BASE_IMAGE $NEW_IMAGE_ID"
]

If the hook executes, does it stop other scripts from executing?  I Imagine
the clone driver has a copy function, wold that also execute if my hook
executed?

Finally, Is there a reason the page you linked [1] says: "*Development
version of OpenNebula 4.2*. The material on this page needs to be reviewed
for completeness and accuracy." in the heading?
will these hooks work in 4.0?

Thanks,
Jon A

[1] http://opennebula.org/documentation:rel4.0:hooks

On Tue, May 28, 2013 at 7:07 AM, Carlos Martín Sánchez <
cmartin at opennebula.org> wrote:

> Hi,
>
> I just want to give you some general advice here: use hooks and drivers as
> much as possible.
>
> For example, instead of adding a new xml-rpc call, oca method, CLI and
> sunstone, you could simply add a new attribute to the Image or VM template.
> MAKE_GOLDEN = YES or similar.
> Doing this, you can forget about the user interface.
>
> Then you can look for that attribute when the VM is shutdown.
> This could be done editing the 'mvds' TM script [1], the one executed for
> the epilog phase. Or implement a hook [2] to be triggered after a VM is
> shut down.
>
>
> Good luck, and keep us posted with your progress!
>
>
> [1] http://opennebula.org/documentation:rel4.0:sd
> [2] http://opennebula.org/documentation:rel4.0:hooks
>
> --
> Join us at OpenNebulaConf2013 <http://opennebulaconf.com> in Berlin,
> 24-26 September, 2013
> --
> Carlos Martín, MSc
> Project Engineer
> OpenNebula - The Open-source Solution for Data Center Virtualization
> www.OpenNebula.org | cmartin at opennebula.org | @OpenNebula<http://twitter.com/opennebula><cmartin at opennebula.org>
>
>
> On Thu, May 23, 2013 at 7:13 AM, Jon <three18ti at gmail.com> wrote:
>
>> Hello All,
>>
>> Before I was using Open Nebula, my general virtual machine workflow was
>> as follows:
>>
>> 1)  Install OS to qcow image, this creates an image between 500-1500MB
>> 2)  "Unconfigure" virtual machine, e.g. remove any system specific
>> config: udev rules, user accounts, ssh host keys, SELinux relabel etc.
>> 3)  Zero out the file system by creating a large file of all zeros, e.g.:
>> dd if=/dev/zero of=./delme; rm delme
>> 4)  Compress and (qemu-img reclaims unused space while compressing so the
>> "sparsify" is implied) the image, to create a "master" image (I've heard
>> this referred to as a "golden" image)
>>            this typically results in an image between 300-500MB,
>> e.g. qemu-img convert -c -f qcow2 master.qcow2 -O qcow2
>> master.compressed.qcow2
>> 5)  Create a "slave" copy on write image off the master, e.g.:  qemu-img
>> create -b ../master/master.compressed.qcow2 -f qcow2 ./virtualmachine.qcow2
>>           This results in all of the changes being written to
>> ./virtualmachine.qcow2
>>
>> Fortunately, virt-sysprep and virt-sparsify handle steps 2 and 3 and the
>> sparcify part of step 4 more or less mindlessly.
>> qemu-img is still required (as far as I know) to compress the image, but
>> there is no longer a need to be in the vm to zero out the image,
>> so this whole process can be automated through a management layer like
>> OpenNebula.
>>
>> I would like to automate this work flow through OpenNebula.  I think it
>> would be extremely useful to be able to prepare and "finalize" a virtual
>> machine from OpenNebula.
>> Finalizing an image in this manner allows for maximization of disk space
>> as well as use as a "master" or "golden" image.
>> In my mind, I think it makes sense to use the least amount of space for
>> the common filesystem pieces.
>>
>> Basically, when I create a new image in OpenNebula, I create a template
>> with the iso and a blank persistent datablock, then instantiate the vm,
>> install the iso,
>> shutdown the vm, and flip the image to a nonpersistent OS.
>>
>> What I am proposing is a feature that would add an additional step that
>> would allow a user to "finalize" the vm by "virt-sysprep"ing,
>> "virt-sparsify"ing,
>> and compressing the image, to maximize space usage.
>>
>> -- My goal is to have a cli command, e.g.: `onevm finalize 38` or `onevm
>> finilize ubuntu-server-13.04` as well as an option in Sunstone to finalize
>> the image.
>> It would effectively finalize and clone the vm so, if necessary, the
>> original can still be modified and finalized again.
>>
>> Does any one have any other ideas, thoughts, comments, or suggestions for
>> an image "finalization" function?  Is there anything I'm not thinking about
>> that I should be?
>>
>> As far as development of the feature is concerned, where would I create
>> this feature for a commandline function, can you point me at some useful
>> documentation for developing this kind of feature?
>> I'm not ready to tackle the sunstone piece of this yet I don't think, so
>> I think my focus will be on the command line function for now.
>>
>> Thanks for your input,
>> Jon A
>>
>> _______________________________________________
>> Users mailing list
>> Users at lists.opennebula.org
>> http://lists.opennebula.org/listinfo.cgi/users-opennebula.org
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.opennebula.org/pipermail/users-opennebula.org/attachments/20130528/2067a6e4/attachment-0002.htm>