[one-users] TransportManager for SAN storage, VMs in clvm, templates in GFS2 volume

[jan horacek]

Hello,

i already published the updated driver. now all images not uploaded by
administrator to the GFS2 volume are stored inside lvm volumes (inside
clvm)

https://github.com/jhrcz/opennebula-tm-gfs2clvm/tree/v0.0.20120303.0

basic tests are already done, and everything seems to work fine ;o) it
looks it could realy be added to one ecosystem ;o)

Regards,
J.Horacek

On Thu, Mar 1, 2012 at 9:01 AM, jan horacek <jahor.jhr.cz at gmail.com> wrote:
> Hi Steve,
>
> The complete writeup about seting all the things up "is on my todo".
>
> clvm is the minimal form on centos. Just a shared storage (sas
> infrastructure exactly, but could be anything else like drbd
> primary/primary, iscsi etc). The driver currently does not use
> exclusive locks, but i'm tempted not to use it in close future. the
> global volume group for all the cloud-related things is created as
> clustered (--clustered yes).
>
> The GFS2 volume in one of the LV is for /var/lib/one on worker nodes,
> context.sh contextualisation isos, vm checkpoints, deployment.X and
> disk.X symlinks are here. All the files for oned are on the management
> node, in /var/lib/one. This storage is NOT shared betwen management
> node and worker nodes.
>
> i'm currently rewriting the driver and related (remotes/fs) to support
> the next level for this setup - having all the images created
> dynamicaly by one on clvm too.
>
> so in the VG, there is
>  * LV for gfs2
>  * LVs "lv-one-XXX-X" for nonpresistent, dynamicaly created volumes
>  * LVs "lv-oneimg-XXXXXXXXXXX for volumes created in one (by saveas,
> cloning etc - replacement of "hash-like" named files)
>
> this brings possibility to use persistent volumes from lvm and not
> gfs2 filesystem, future possibility for using snapshosts for cloning
> (and even live snapshoting, without suspending the machine). Currently
> it is able to create new volumes as a copy of volumes from machines in
> suspend state, no need to wait for shutdown, just suspend it and
> create copy. For my this gives me the proof, that something working is
> cloned and checked and then the source could be suspended - no risk of
> saveas failure with work losk.
>
> the last changes are not in my git repo yet, i hope the will be in
> very short time.
>
> To your questions...
>
> ad 1... yes, the one management/head node is virtual machine in my
> installation. It is on some other physical machine, not directly
> connected in the cluster tools. this is why i challenged the
> disconnection of the node from shared filesystem, to make safe fencing
> in the cluster possible. all fencing could be directly to ipmi and no
> need to use libvirt fencing for the management node (management node
> shares the physical machine with some other critical systems not
> realted in the cloud and so ipmi fencing is not the best way to fence
> it)
>
> ad 2... some sort of description is in the text above. the driver in
> the form of v0.0.20120221.0 as initialy pushed to github, has
> master-templates (isos, hand made sys images) on the lvm, all created
> volumes (files with "hash-like filename"), vm images are in the lvm,
> so every time you deploy a machine, it dd's the from gfs2 to lvm.
> persistent images are used from gfs2. but as i wrote above, this will
> change, because i want to minize the usage of the gfs2 storage.
>
> i hope i answered the question sufficiently ;o)
>
> Regards,
> Jan Horacek
>
> On Mon, Feb 27, 2012 at 11:37 PM, Steven Timm <timm at fnal.gov> wrote:
>> This is very interesting work.  Jan, do you have any write-up on how
>> you were able to set up the gfs and clvm setup to work with this driver?
>> It's a use case very similar to what we are considering for FermiCloud.
>>
>> Two other questions that weren't immediately obvious from looking
>> at the code:
>>
>> 1) with this driver, could the OpenNebula head node itself be
>> a (static) virtual machine?  Looks like yes, but I want to be sure.
>>
>> 2) How is the notion of an image repository handled--does
>> openNebula copy the OS image from a separate image repository
>> every time the VM is instantiated, or is the repository defined
>> to be the place that the OS image lives on disk?
>>
>> Steve Timm
>>
>>
>>
>>
>>
>> On Thu, 23 Feb 2012, Borja Sotomayor wrote:
>>
>>> Hi Jan,
>>>
>>>> I call this transfer manager drive **gfs2clvm** and made it (even in
>>>> current development state - but most of the functions works
>>>> already) available on github:
>>>> https://github.com/jhrcz/opennebula-tm-gfs2clvm
>>>>
>>>> if anyone is interrested, wants to contribute and
>>>> help, please contact me.
>>>
>>>
>>> A good way to get more people involved and interested would be to add
>>> it to our ecosystem catalog:
>>>
>>>   http://www.opennebula.org/software:ecosystem
>>>
>>> gfs2clvm definitely sounds like a good candidate for inclusion in the
>>> ecosystem. If you are interested, you can find instructions here:
>>>
>>>   http://www.opennebula.org/community:ecosystem
>>>
>>> You're also welcome to write about gfs2clvm on our blog
>>> (http://blog.opennebula.org/). If you're interested, just drop me a
>>> line off-list and I'll set you up with an account.
>>>
>>> Cheers!
>>> --
>>> Borja Sotomayor
>>>
>>>  Researcher, Computation Institute
>>>  Lecturer, Department of Computer Science
>>>  University of Chicago
>>>  http://people.cs.uchicago.edu/~borja/
>>>
>>>  Community Manager, OpenNebula project
>>>  http://www.opennebula.org/
>>> _______________________________________________
>>> Users mailing list
>>> Users at lists.opennebula.org
>>> http://lists.opennebula.org/listinfo.cgi/users-opennebula.org
>>>
>>
>> ------------------------------------------------------------------
>> Steven C. Timm, Ph.D  (630) 840-8525
>> timm at fnal.gov  http://home.fnal.gov/~timm/
>> Fermilab Computing Division, Scientific Computing Facilities,
>> Grid Facilities Department, FermiGrid Services Group, Group Leader.
>> Lead of FermiCloud project.