[one-users] use of IMAGE_ID and its ilk in 3.0

[Thomas Higdon]

On Fri, Nov 04, 2011 at 07:00:21AM -0500, Carlos Martín Sánchez wrote:
> Hi Chris,
> 
> I think adding the owner is a good idea. However, the examples you provide rely
> on the '/' separator, and currently that's a valid character for both user and
> resource names.
> 
> This is our suggested syntax to implement this:
> 
> DISK = [ IMAGE        = "Ubuntu",
>          IMAGE_UNAME  = "oneadmin" ]
> 
> DISK = [ IMAGE        = "Ubuntu",
>          IMAGE_UID    = 7 ]
> 
> DISK = [ IMAGE        = "Ubuntu" ]
> 
> DISK = [ IMAGE_ID     = 23 ]
> 
> 1: use the Image named ubuntu, owned by the user named oneadmin.
> 2: use the ubuntu Image owned by user with ID 7.
> 3: use the ubuntu Image owned by the user instantiating the VM.
> 4: use image with ID 23
> 
> 
> Thomas, what do you mean by the following? Could you elaborate a bit more?
> If you are referreing to VMs, you can feed template files to the "onevm create
> filename" command.

I've thought about this a little further, and I've come to the
conclusion that the template system is just not that useful to me. The
only advantage that I can see of having templates in the database is
that you can instantiate them via Sunstone.

My original thoughts were that the templating system would be completely
done away with in favor of having a daemon that tracks the state of a
filesystem hierarchy that contains template files and populates the
database with them, allowing them to be instantiated by CLI users as
well as Sunstone users. For CLI users, I realize (and you imply) that
this simply boils down to maintaining your own file hierarchy with
template files in it, and instantiating VMs (and images, networks)
directly from those files, and ignoring the 3.0 template-storage system
entirely. The only missing piece in that solution is access to these
templates from Sunstone, which is very desirable. However, it seems much
simpler to add a feature to Sunstone that allows it access to a certain
directory hierarchy that contains templates that it can instantiate, as
well as allowing it to add templates to this hierarchy. <- Consider this
a feature request.

> 2011/11/3 Thomas Higdon <thigdon at akamai.com>
> 
>     Another more radical suggestion is to do away with the template
>     management system entirely, and switch to a system where the templates
>     are derived directly from files that exist on the filesystem. This has a
>     number of advantages I can see (if you're interested I'll explain
>     further), but it might be more change than you're willing to go through
>     at this point.
> 
> 
> Regards.
> --
> Carlos Martín, MSc
> Project Engineer
> OpenNebula - The Open Source Toolkit for Data Center Virtualization
> www.OpenNebula.org | cmartin at opennebula.org | @OpenNebula 
> 
> 
> 2011/11/3 Chris Johnston <chjohnston at rim.com>
> 
>     This issue has caused myself much frustration as well. Personally I prefer
>     the suggestion of using both the owner and the resource name for specifying
>     a resource. That should work for just about any use case and allows names
>     to be used at any level with no confusion as to what resource is really
>     being referenced. For example...
> 
>     IMAGE = "oneadmin/generic"
>     IMAGE = "dev01/virtapp"
>     IMAGE = "user01/virtapp"
>     IMAGE = "user01/custom"
> 
>     ...and the same for NETWORK. It does add some overhead but it's a lot
>     simpler than maintaining lists of IDs and constantly updating scripts.
> 
>     -----Original Message-----
>     From: users-bounces at lists.opennebula.org [mailto:
>     users-bounces at lists.opennebula.org] On Behalf Of Thomas Higdon
>     Sent: Thursday, November 03, 2011 12:36 PM
>     To: Carlos Martín Sánchez
>     Cc: users at lists.opennebula.org
>     Subject: Re: [one-users] use of IMAGE_ID and its ilk in 3.0
> 
>     Hi, it's good to know the ONE developers are so open to feedback about
>     the way the system works. ONE seems great from what I've used so far.
> 
>     The solution that you would propose would work fine for me, mostly
>     because it looks like that for a single user, the templating system
>     would work the same as it did for 2.2, which was ok with me.
> 
>     I'm a little leery of the idea of ever forcing users to use IMAGE_ID or
>     NETWORK_ID parameters. In my mind, forcing users to specify these in
>     static template strings/files is equivalent to some Unix command-line
>     tool asking a user to specify the inode number of a file rather than its
>     name.
> 
>     Here's a suggestion. Your database is basically a key-value store. In
>     2.2, the only key you allowed is the name (in 3.0 it's only ID). I'd
>     suggest that you allow more freedom in the specification of keys. You
>     might ask users to specify a name for resources that they own. If they
>     are asking for resources that they don't own, then they must also
>     specify which user owns that resource, or perhaps which group. If
>     there's ambiguity, then error on that.
> 
>     Another suggestion I would make is to remove the NAME attribute from all
>     templates (or make it optional) and allow users to specify a name on the
>     command-line, defaulting to something reasonable, as is done now for
>     template instantiation. That way, users who don't care what the name is
>     don't need to specify it, and users who do care don't need to edit a
>     text file to instantiate multiple instances of the same resource. I see
>     this as being useful in a scenario where I want to create <x> read/write
>     250GB disks for <x> VMs, but not necessarily create <x> template files,
>     or edit one file <x> times.
> 
>     At the end of the day, desirable behavior for me is to be able to create
>     a set of static template files for networks, images and VMs, and be able
>     to instantiate all of them in the proper order and have it just work.
>     ONE 3.0 does not allow me to do that.
> 
>     Another more radical suggestion is to do away with the template
>     management system entirely, and switch to a system where the templates
>     are derived directly from files that exist on the filesystem. This has a
>     number of advantages I can see (if you're interested I'll explain
>     further), but it might be more change than you're willing to go through
>     at this point.
> 
>     On Thu, Nov 03, 2011 at 08:55:30AM -0500, Carlos Martín Sánchez wrote:
>     > Hi Thomas,
>     >
>     > Some users requested to let new resources use repeated names, see for
>     instance
>     > this thread [1]. This makes sense in deployments with a large number of
>     users,
>     > or in a multi-tenant scenario, where you don't want users having to try a
>     > resource creation several times until they find a free name.
>     >
>     > We implemented this allowing repeated names, but only if they are not
>     owned by
>     > the same user. It was done this way so users can reference resources by
>     name in
>     > CLI commands, e.g. 'oneimage show my_img'.
>     >
>     > Since we released OpenNebula 3.0, some of you have complained about this
>     > change, so let me try to explain why we decided to drop the NAME
>     reference from
>     > VM templates:
>     >
>     >
>     > Lets say you want to use the VNet named "blue". In the worst case
>     scenario, you
>     > will have one "blue" network owned by you, several "blue" networks owned
>     by
>     > other people in your group, and several other "blue" networks owned by
>     people
>     > from other groups. You may, or may not, have rights to use the latter
>     "blue"
>     > networks outside your group.
>     >
>     > How does OpenNebula decide which "blue" network do you want to use? At
>     first
>     > sight, you could try to arrange them by priority: your VNet has comes
>     first,
>     > then VNets from your group, and then VNets from other groups. This
>     presents a
>     > lot of problems:
>     >
>     > You can have at most one "blue" network, but what happens if you are
>     > instantiating a public template created by other user in your group?
>     > Maybe the template was prepared by "boss_user", who owns a "blue" VNet
>     with
>     > addresses in the 192.169.10.0 network. But you own another "blue" VNet
>     with
>     > base address 192.168.30.0. The user will expect his VNets to have greater
>     > priority, but then the VM will be created in a different VNet than the
>     one
>     > intended by "boss_user".
>     >
>     > If you don't own any "blue" network, you may create a template that uses
>     a
>     > public one owned by other user in your group. If it is the only "blue"
>     network
>     > in the group, then you template will work fine, until somebody else
>     decides to
>     > publish another "blue" network in the same group. From that moment,
>     OpenNebula
>     > would have to guess, or just refuse to instantiate that template that was
>     > working fine before.
>     >
>     >
>     > In the end, we could implement a priority and do our best to document it,
>     but
>     > we though that would cause a lot of confusion. The easiest solution was
>     to
>     > force the usage of IDs, what didn't look to us like a really limiting
>     > requirement.
>     >
>     >
>     >
>     > After reading your feedback, we are considering to bring back the NAME
>     > reference in VM templates for version 3.2.
>     > We think the most robust and easier to understand behaviour is the
>     following
>     > one:
>     >
>     > * Allow to use NAME to reference only resources owned by the user
>     instantiating
>     > the VM.
>     > * If the template is intended to be shared with other users, then it must
>     use
>     > the IMAGE_ID and NETWORK_ID attributes.
>     >
>     >
>     > To all community members intersested in this issue, please share your
>     thoughts.
>     > What do you think about this? Would it be enough for your use-cases?
>     Would you
>     > address this issue differently?
>     >
>     >
>     > Regards.
>     >
>     > [1] http://lists.opennebula.org/pipermail/users-opennebula.org/
>     2010-October/
>     > 002924.html
>     >
>     > --
>     > Carlos Martín, MSc
>     > Project Engineer
>     > OpenNebula - The Open Source Toolkit for Data Center Virtualization
>     > www.OpenNebula.org | cmartin at opennebula.org | @OpenNebula
>     >
>     >
>     > On Tue, Nov 1, 2011 at 6:12 PM, Thomas Higdon <thigdon at akamai.com> wrote:
>     >
>     >     In 2.2, in a VM template, I could specify an image for a disk that
>     was
>     >     in the image repository by using IMAGE = <name>. This behavior
>     appears
>     >     to have been removed in 3.0, in favor of using IMAGE_ID = <id>, where
>     >     <id> is an arbitrary number assigned by the opennebula system.
>     >
>     >     This change in behavior seems kind of limiting. Before, I could
>     create
>     >     an image with a certain name, and then instantiate a VM template that
>     >     had a disk that used that name. Now, in order to get that same
>     behavior,
>     >     I must instantiate the image, note the ID returned, and then rewrite
>     my
>     >     VM template to use that ID in its DISK attribute, then instantiate
>     it.
>     >     This is also true of virtual networks.
>     >
>     >     Is there something I'm missing with respect to how VM templates are
>     >     instantiated with respect to images? Is there any workaround that
>     will
>     >     allow the old behavior? Why was this change made?
>     >     _______________________________________________
>     >     Users mailing list
>     >     Users at lists.opennebula.org
>     >     http://lists.opennebula.org/listinfo.cgi/users-opennebula.org
>     >
>     >
>     _______________________________________________
>     Users mailing list
>     Users at lists.opennebula.org
>     http://lists.opennebula.org/listinfo.cgi/users-opennebula.org
> 
>     ---------------------------------------------------------------------
>     This transmission (including any attachments) may contain confidential
>     information, privileged material (including material protected by the
>     solicitor-client or other applicable privileges), or constitute non-public
>     information. Any use of this information by anyone other than the intended
>     recipient is prohibited. If you have received this transmission in error,
>     please immediately reply to the sender and delete this information from
>     your system. Use, dissemination, distribution, or reproduction of this
>     transmission by unintended recipients is not authorized and may be
>     unlawful.
> 
>