[one-users] Storage architecture question

Thu Aug 16 04:23:41 PDT 2012

Hi Gary,

sounds great, could you contribute the code at some point?

cheers,
Jaime

On Mon, Aug 6, 2012 at 2:52 PM, Gary S. Cuozzo <gary at isgsoftware.net> wrote:

> Here's an update on this thread...
>
> So I did originally setup a "local" datastore by using NFS from the host
> to the controller as a shared DS.  It worked fine and performed as
> expected.  I was able to get the speed of local storage without having
> bloated capacity requirements and delays in deployment due to having large
> images be copied across the network.  I also tested rebooting the
> controller node and the host node and the NFS share would come back up
> reliably.
>
> Having to setup the NFS share was the main drawback.  I figured I could
> get around that by writing a custom DS/TS driver.  That is the route I
> took.  Over the weekend I created a 'localfs' set of drivers and they seem
> to work great and don't require me to setup the NFS share.  This seems to
> give me exactly what I was looking for.
>
> The only requirement for my driver is that I create the datastore first so
> that I can get the ID.  Then, I mount my local disks (I use an LVM volume)
> at the correct mount point according to the DS ID.
>
> I use a DS template variable to specify the hostname of the host so that I
> can use the driver on multiple hosts.  The images get created as
> hostname:/path/to/image.
>
> So far so good.
>
> Just FYI,
> gary
>
>
> ------------------------------
> *From: *"Ruben S. Montero" <rsmontero at opennebula.org>
>
> *To: *"Gary S. Cuozzo" <gary at isgsoftware.net>
> *Sent: *Thursday, August 2, 2012 12:46:28 PM
>
> *Subject: *Re: [one-users] Storage architecture question
>
> OK, keep us updated with the performance of your final solution
>
>
> Good Luck
>
> Ruben
>
> On Thu, Aug 2, 2012 at 5:58 PM, Gary S. Cuozzo <gary at isgsoftware.net>wrote:
>
>> Hi Ruben,
>> Thank you for the reply & information.
>>
>> I had previously thought about a similar setup to what you have
>> outlined.  One drawback I was trying to work around is that I will be
>> duplicating storage requirement as I will have to store images on both my
>> SAN/NAS and the local system.  As my SAN/NAS is fully mirrored, I will
>> actually incur a fairly substantial increase in the overhead per GB.  I was
>> trying to avoid that.
>>
>> My other main concern was time to copy the image files to/from the
>> hosts.  A few of the images will be TB's and the copies will take a long
>> time I think, even over GB links.
>>
>> I do intend to create a separate cluster and datastore for these few
>> hosts.  I'll just have to try out a few different setups and see what works
>> best.
>>
>> You guys have given me some good ideas, thank you.
>>
>> gary
>>
>> ------------------------------
>> *From: *"Ruben S. Montero" <rsmontero at opennebula.org>
>>
>> *To: *"Gary S. Cuozzo" <gary at isgsoftware.net>
>> *Cc: *users at lists.opennebula.org
>> *Sent: *Tuesday, July 31, 2012 5:44:39 PM
>>
>> *Subject: *Re: [one-users] Storage architecture question
>>
>> Hi,
>>
>> You can set the Images Datastore of type FS to use the shared TM and your
>> fast NAS, but the system datastore to use the ssh drivers. This  will copy
>> the images from the Datastore which uses NFS to the local storage area.
>>
>> Note 1: This will only work for non-persistent images. Persistent images
>> will be linked and thus used directly from the NAS server.
>>
>> Note 2: you cannot mix both setups in the same cluster. This is, if your
>> system datastore is shared the ssh transfer will copy the images to the NFS
>> volume in the remote host.
>>
>> You can:
>>
>> 1.- Create a dedicated cluster for this images that uses ssh as the
>> system datastore (so the hosts do not mount the NFS export). You just need
>> to add a few host and the datastore of the high demaning I/O images.The 3.6
>> scheduler will only use the resources of that cluster for that VMs [2]
>>
>> 2.- Modify the clone/mv/mvds scripts from a TM shared so it copy some of
>> your images to a local disk and link them to the path expected by
>> OpenNebula.
>>
>>
>> Cheers
>>
>> Ruben
>>
>> [1]
>> http://opennebula.org/documentation:rel3.6:system_ds#using_the_ssh_transfer_driver
>>
>> [2]http://opennebula.org/documentation:rel3.6:cluster_guide
>>
>>
>> On Mon, Jul 30, 2012 at 7:36 PM, Gary S. Cuozzo <gary at isgsoftware.net>wrote:
>>
>>> Hi Tino,
>>> Thanks for the reply.
>>>
>>> Yes, I think you understand correctly.  My goal is to be able to utilize
>>> storage which is local to a particular vm host node without incurring the
>>> overhead of duplicated storage on the controller node and transfer time
>>> from controller to vm host.
>>>
>>> I do understand that the images will only be accessible from the
>>> particular vm host which they reside on, but that is ok as it would be the
>>> trade-off for local disk performance.  I have a great iSCSI/NFS SAN which
>>> is used for shared storage, but it will never be able to offer the same
>>> level of performance as local storage.  So I'm looking to be able to have
>>> that local option for the few cases it's required for I/O intensive
>>> applications.
>>>
>>> I have not actually had the chance to try it out yet, but I think it
>>> will give me what I'm looking for.
>>>
>>> Thanks again,
>>> gary
>>>
>>>
>>> ----- Original Message -----
>>> From: "Tino Vazquez" <tinova at opennebula.org>
>>> To: "Gary S. Cuozzo" <gary at isgsoftware.net>
>>> Cc: users at lists.opennebula.org
>>> Sent: Monday, July 30, 2012 12:40:06 PM
>>> Subject: Re: [one-users] Storage architecture question
>>>
>>> Dear Gary,
>>>
>>> I am not sure I understand 100% your desired set-up, but if I grasped
>>> it correctly, I think the problem you may found is the images would
>>> only be local to the node that is exporting the NFS share. Otherwise I
>>> think it will work as you expect.
>>>
>>> Regards,
>>>
>>> -Tino
>>>
>>> --
>>> Constantino Vázquez Blanco, MSc
>>> Project Engineer
>>> OpenNebula - The Open-Source Solution for Data Center Virtualization
>>> www.OpenNebula.org | @tinova79 | @OpenNebula
>>>
>>>
>>> On Fri, Jul 27, 2012 at 11:44 PM, Gary S. Cuozzo <gary at isgsoftware.net>
>>> wrote:
>>> > Hi Users,
>>> > I am running ONE 3.6 and would like to be able to run a combination of
>>> > shared storage (via iSCSI) and local storage (to take advantage of
>>> local
>>> > disk performance for certain applications).  My question is related to
>>> the
>>> > local storage aspect.
>>> >
>>> > From what I've seen, I can use a local datastore and the ssh TM to
>>> > accomplish local storage.  The drawback that I see is that I need 2x
>>> the
>>> > amount of disk space because I need storage for the permanent image on
>>> the
>>> > controller node, then storage on the local host for the running image
>>> when
>>> > it is deployed.  A secondary issue for me is that the images have to be
>>> > transferred between the datastore and the host machine, which will
>>> take some
>>> > time with larger images.
>>> >
>>> > To get around the problem, I thought I could set the datastore up as a
>>> > shared filesystem, except the sharing would actually be from the host
>>> > machine to the controller machine via NFS.  Is there any particular
>>> > reason(s) that would be a bad idea?  On the surface it seems like it
>>> should
>>> > work just fine, but I'm somewhat new to ONE and want to be sure I'm not
>>> > going down a bad path since I plan to do this with several host
>>> machines.
>>> >
>>> > Thanks in advance for any advice.
>>> >
>>> > gary
>>> >
>>> >
>>> > _______________________________________________
>>> > Users mailing list
>>> > Users at lists.opennebula.org
>>> > http://lists.opennebula.org/listinfo.cgi/users-opennebula.org
>>> >
>>> _______________________________________________
>>> Users mailing list
>>> Users at lists.opennebula.org
>>> http://lists.opennebula.org/listinfo.cgi/users-opennebula.org
>>>
>>> --
>>> Ruben S. Montero, PhD
>>> Project co-Lead and Chief Architect
>>> OpenNebula - The Open Source Solution for Data Center Virtualization
>>> <http://lists.opennebula.org/listinfo.cgi/users-opennebula.org>
>>> www.OpenNebula.org | rsmontero at opennebula.org | @OpenNebula
>>>
>>>
>>
>
>
> --
> Ruben S. Montero, PhD
> Project co-Lead and Chief Architect
> OpenNebula - The Open Source Solution for Data Center Virtualization
> www.OpenNebula.org | rsmontero at opennebula.org | @OpenNebula
>
>
> _______________________________________________
> Users mailing list
> Users at lists.opennebula.org
> http://lists.opennebula.org/listinfo.cgi/users-opennebula.org
>
>

-- 
Jaime Melis
Project Engineer
OpenNebula - The Open Source Toolkit for Cloud Computing
www.OpenNebula.org | jmelis at opennebula.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.opennebula.org/pipermail/users-opennebula.org/attachments/20120816/5dd99b58/attachment-0003.htm>