<meta http-equiv="content-type" content="text/html; charset=utf-8"><span class="Apple-style-span" style="border-collapse: collapse; color: rgb(68, 68, 68); font-family: arial, sans-serif; font-size: 13px; "><div>Hi Viven,</div>
<div><br></div><div>The use of different TM mechanism is quite interesting, as you propose probably the best way to handle this is a TM clever enough to choose the corresponding storage backend for different disks. In fact, that is the spirit of the example at [1], although simplier than the one you propose.</div>
<div><br></div><div>I'd say that the preferred alternatives are:</div><div><br></div><div>* Include the TM options in the Image template. Define an image "Debian base system", with the proper TM options to handle it. The TM_OPTIONS will be stored in the image definition and could be accessed trough the driver by getting the VM id from the SRC_PATH and from there 'onevm show <id>', and get the disk that match de disk_id (also from the SRC_PATH, note that disks are named disk.<disk_id>) to get the IMAGE and the TM options from 'oneimage show'.</div>
<div><br></div><div>* Use special URLs for your images so the TM knows how to handle them. For example, iscsi_filer://debian. URLs are not modified by OpenNebula so you get the that URL. In the TM you can have the TM options for different URLs</div>
<div><br></div><div>NOTE: There is a "bug" in 2.2 that removes the custom attributes (like TM_OPTIONS you propose) [2]. This will be solved in 2.4. Storing the TM_OPTIONS have some advantages as you do not have to include them in all the templates that uses that OPTIONS. Also, if you want to change them you only have to do it in the Image Pool. </div>
<div><br></div><div>I think that this approach is pretty close to the Storage Pool proposed in your email. </div><div><br></div><div>However I think that two queries (onevm show + oneimage show) is a bit cumbersome, my proposal here is to include a special Image attribute (TM_OPTIONS or DISK_ATTRIBUTES) that will be copied form the IMAGE to the DISK attribute in the VM template (when the VM is created), so a simple onevm show will give you those values. </div>
<div><br></div><div>What do you think?</div><div><br></div><div>REFS</div><div>[1] <a href="http://www.opennebula.org/documentation:rel2.2:sd" target="_blank" style="color: rgb(34, 34, 34); ">http://www.opennebula.org/documentation:rel2.2:sd</a></div>
<div>[2] <a href="http://dev.opennebula.org/issues/559" target="_blank" style="color: rgb(34, 34, 34); ">http://dev.opennebula.org/issues/559</a></div><div><br></div><div>Cheers,</div><div>Jaime</div></span><br><div class="gmail_quote">
On Fri, Apr 15, 2011 at 7:15 PM, Vivien Bernet-Rollande <span dir="ltr"><<a href="mailto:vivien.bernet-rollande@nexen.alterway.fr">vivien.bernet-rollande@nexen.alterway.fr</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
Hi.<br>
I'm new to the OpenNebula Project. I'm currently testing if it can do
what we need. So far, it looks promising, but I have a problem.<br>
<br>
Currently, Open Nebula assumes there is a single type of storage for
each hypervisor. It lets us write custom drivers, but only for one
single storage type at a time.<br>
<br>
I have read
<a href="http://lists.opennebula.org/htdig.cgi/users-opennebula.org/2011-April/004678.html" target="_blank">http://lists.opennebula.org/htdig.cgi/users-opennebula.org/2011-April/004678.html</a>,
but this is not what I am trying to do.<br>
<br>
I want to mix several types of storage on a single virtual machine. For instance :<br>
- swap on local disks<br>
- data on NFS<br>
or :<br>
- persistent data on shared block devices (iscsi, gluster, ceph, you name it)<br>
- temporary OS on local storage.<br>
- CD images on a shared filesystem (NFS)<br>
or even :<br>
- local SSD for a fast cache<br>
- local sata drive for the OS<br>
<br>
I'm totally fine with the idea of writing (and sharing) a custom
transfer manager driver for this purpose. The question is : will it be
enough, or will I need do add some way to dispatch calls to various<br>
<br>
I've come up with several ideas regarding how to deal with this from a configuration standpoint.<br>
<br>
The idea I like most is fairly straightforward : we specify some options
for the TM driver . Of course, the TM has to be custom written and do
all the work. For example :<br>
<br>
DISK = [ IMAGE = "Debian base system",<br>
TM_OPTIONS = "storage=iscsi iscsi_type=solaris_zfs iscsi_filer=1.2.3.4 iscsi_volume=vm_images" ]<br>
<br>
With these options, the transfer manager could CLONE the image :<br>
- check the size of the source image<br>
- connect to the filer<br>
- create a new LUN in the volume with the right size<br>
- make sure the node sees the LUN<br>
- create a link in /srv/cloud/one/5/images/disk.0 pointing to that lun (with some /dev/disk/by-path/ magic)<br>
- copy the data from wherever the image is stored onto the LUN.<br>
<br>
With this in place, MKSWAP, MKIMAGE, LN and DELETE operations are straightforward.<br>
<br>
MV is a bit more complex if we want to be able to move data from one
storage backend to another, but can probably be figured out. Then I need
to figure where I want to store data, how, make sure everything
interacts well. But that's just a matter of scripting (and testing), and
I can handle it.<br>
<br>
The key operation is to have the TM_OPTIONS variable given to the
transfer manager. I've checked, it's possible to add variables in the VM
template, they are simply ignored.<br>
<br>
The best way I can think of is to have the transfer manager take that
field, and add it an environment variable. This would be 100% compatible
with existing drivers, and allow me to do what I need.<br>
<br>
<br>
The thing is, I'm not quite sure about how to implement this.<br>
<br>
If I'm correct, I can get the value for TM_OPTIONS easily in
src/tm/TransferManager.cc with a simple
disk->vector_value("TM_OPTIONS"). The problem is the commands are
then simply written to a file, and it's kind of hard to set an
environment variable that would be forwarded to the driver's script.<br>
<br>
So I came up with another idea : instead of adding TM_OPTIONS as an
environment variable, append it to the line in the command files. For
instance, instead of :<br>
<br>
LN opennebula:/srv/cloud/images/debian.img 1.1.1.1:/var/lib/one/25/images/disk.0<br>
<br>
We would have : <br>
<br>
LN opennebula:/srv/cloud/images/debian.img
1.1.1.1:/var/lib/one/25/images/disk.0 storage=iscsi iscsi_type=nexenta
iscsi_filer=1.2.3.4 iscsi_volume=vm_images<br>
<br>
Then it would be the driver's work to parse the command line.<br>
<br>
Since most drivers use $1 and $2 to handle their parameters, this would
be perfectly transparent to existing code. one_tm simply forwards '$*'
to the right driver. However, it would enable building much more
powerful transfer manager drivers.<br>
<br>
Oh, and "CONTEXT" could take a similar TM_OPTIONS argument.<br>
<br>
Now, a few more practical questions :<br>
- is there interest for this feature from the community ?<br>
- is the way I propose to add it to the VM template compliant to ONE's
way of doing things ? Is another name preferred (rather than TM_OPTIONS)
?<br>
- is appending those options at the end of the command the right way, or will it break something further down the line ?<br>
- any chances to get this in 2.2 ? Or will I have to maintain my own
patched version ? (I'm using the .deb you guys provide by the way).<br>
<br>
<br>
The other method I thought of is to have a StoragePool database object,
with a set of options. One would register such a StoragePool in a
similar way to Virtual Networks (ie, with a template and a command). The
user could then say "I want this disk to be in storage pool X". This
moves the responsability of managing storage from the TM driver to the
core. It has some nice implications (for instance we could attach a
default StoragePool to a cluster or a host, we could monitor available
space in a StoragePool, a pool could be marked as persistant or not,
etc.), but is much more complex to implement, and might break existing
setups. Morever this would break existing drivers, because we need a way
to transfer images between storage pools. But I like the idea of
declaring "I have an NFS share here. Oh, and some AoE there, and SCSI
other here. And all those machines have two 1T drives they can use", and
having the VM select the type of storage they need.<br>
<br>
For me, the most important difference between the two solutions is that I
can have the first one up and running in a matter of days, while the
other might require several weeks. I will go and try the first one. I do
share the second one with you, however, because I think it would be an
interesting, even more extensible, long-term solution.<br><font color="#888888">
<br>
<br>
-- <br>
Vivien Bernet-Rollande<br>
Systems & Networking Engineer<br>
Alter Way Hosting<br>
</font><br>_______________________________________________<br>
Users mailing list<br>
<a href="mailto:Users@lists.opennebula.org">Users@lists.opennebula.org</a><br>
<a href="http://lists.opennebula.org/listinfo.cgi/users-opennebula.org" target="_blank">http://lists.opennebula.org/listinfo.cgi/users-opennebula.org</a><br>
<br></blockquote></div><br><br clear="all"><br>-- <br>Jaime Melis, Cloud Technology Engineer/Researcher<br>Major Contributor<br>OpenNebula - The Open Source Toolkit for Cloud Computing<br><a href="http://www.OpenNebula.org" target="_blank">www.OpenNebula.org</a> | <a href="mailto:jmelis@opennebula.org" target="_blank">jmelis@opennebula.org</a><br>