[one-users] econe-run-instances idempotent?
Daniel Molina
dmolina at opennebula.org
Wed Oct 22 09:23:06 PDT 2014
Hi Parag,
I have uploaded a first version of the client token support. You can apply
the commit included in the ticket. It will be included in the next release
Cheers
On 16 October 2014 11:39, Daniel Molina <dmolina at opennebula.org> wrote:
>
>
> On 15 October 2014 18:57, Parag Mhashilkar <parag at fnal.gov> wrote:
>
>> Hi Daniel
>>
>> Such a feature would be really useful when provisioning VMs and avoiding
>> leaks. To be useful, this also requires support for client token in
>> ec2-describe-instances equivalent in open nebula
>>
>> http://docs.aws.amazon.com/AWSEC2/latest/CommandLineReference/ApiReference-cmd-DescribeInstances.html
>>
>>
> I have opened a ticker in our dev portal, and we will try to implement it.
>
>
>> AWS uses client token to achieve idempotency in few other calls too, but
>> I will leave that upto you on if and where you want to support such a
>> functionality.
>>
>
> The easiest way to implement this is only for VMs as I explained before.
> If we want to support this for any call we would have to generate a client
> token pool and it will be trickier.
>
> Cheers
>
>>
>> Thanks & Regards
>> +==========================================================
>> | Parag Mhashilkar
>> | Fermi National Accelerator Laboratory, MS 120
>> | Wilson & Kirk Road, Batavia, IL - 60510
>> |----------------------------------------------------------
>> | Phone: 1 (630) 840-6530 Fax: 1 (630) 840-3109
>> |----------------------------------------------------------
>> | Wilson Hall, 806E (Nov 8, 2012 - To date)
>> | Wilson Hall, 867E (Nov 17, 2010 - Nov 7, 2012)
>> | Wilson Hall, 863E (Apr 24, 2007 - Nov 16, 2010)
>> | Wilson Hall, 856E (Mar 21, 2005 - Apr 23, 2007)
>> +==========================================================
>>
>> On Oct 15, 2014, at 11:44 AM, Daniel Molina wrote:
>>
>> > I think tthat could be implemented in the econe server.
>> > 1. Include an EC2_CLIENT_TOKEN in the vm.allocate method containing
>> the ClientToken param provided in the CreateInstance command.
>> > 2. When a new CreateInstance is requested and contains a ClientToken,
>> the vmpool is retrieved to check if any vm contains that token
>> >
>> > On 15 October 2014 18:34, Parag Mhashilkar <parag at fnal.gov> wrote:
>> > That won't help. If the middle ware like HTcondor or run-instance
>> commands starts putting anything in the user data, users are deprived of
>> the functionality of user data.
>> >
>> > AWS achieves idempotency with --client-token option, which to me seems
>> like a tagging the request from the client side.
>> >
>> >
>> http://docs.aws.amazon.com/AWSEC2/latest/CommandLineReference/ApiReference-cmd-RunInstances.html
>> >
>> >
>> > Thanks & Regards
>> > +==========================================================
>> > | Parag Mhashilkar
>> > | Fermi National Accelerator Laboratory, MS 120
>> > | Wilson & Kirk Road, Batavia, IL - 60510
>> > |----------------------------------------------------------
>> > | Phone: 1 (630) 840-6530 Fax: 1 (630) 840-3109
>> > |----------------------------------------------------------
>> > | Wilson Hall, 806E (Nov 8, 2012 - To date)
>> > | Wilson Hall, 867E (Nov 17, 2010 - Nov 7, 2012)
>> > | Wilson Hall, 863E (Apr 24, 2007 - Nov 16, 2010)
>> > | Wilson Hall, 856E (Mar 21, 2005 - Apr 23, 2007)
>> > +==========================================================
>> >
>> > On Oct 15, 2014, at 11:27 AM, Daniel Molina wrote:
>> >
>> > >
>> > >
>> > > On 15 October 2014 18:18, Parag Mhashilkar <parag at fnal.gov> wrote:
>> > > Hi Daniel,
>> > >
>> > > Let me rephrase what we meant.
>> > >
>> > > As Steve mentioned in one of his emails, we use HTCondor to launch
>> VMs. There is always a possibility that something can go wrong after a
>> run-instance command is issued and before the ec2 server gets back to you
>> with a valid instance id or an error message. Note that in this case the
>> server may serve the request, but there is no way for the client to know
>> the exact instance name that resulted from the request. Life is much easier
>> if the client crashes after it gets back the instance name.
>> > >
>> > > One way for the client to know the exact status of the request even
>> after a crash (and before it gets the instance id) is to assign a unique
>> identifier of its own like a tag name or something. This way client will
>> never lose track of the request and result in the leaked VMs.
>> > >
>> > > The way HTCondor is implementing this is by creating a new key pair
>> and giving it a unique name and tracking the requests based on the key
>> pair. But then we are hit by other limit ~300 that we observed.
>> > >
>> > > Ideally, we would like to use the --keypair option and use the
>> existing pre registered key. But than we cannot track the results of the
>> create-instance in case something goes wrong before the instance id is
>> issued.
>> > >
>> > > And what about including a tag in the UserData instead of using the
>> unique keypar?
>> > >
>> > >
>> > >
>> > > Thanks & Regards
>> > > +==========================================================
>> > > | Parag Mhashilkar
>> > > | Fermi National Accelerator Laboratory, MS 120
>> > > | Wilson & Kirk Road, Batavia, IL - 60510
>> > > |----------------------------------------------------------
>> > > | Phone: 1 (630) 840-6530 Fax: 1 (630) 840-3109
>> > > |----------------------------------------------------------
>> > > | Wilson Hall, 806E (Nov 8, 2012 - To date)
>> > > | Wilson Hall, 867E (Nov 17, 2010 - Nov 7, 2012)
>> > > | Wilson Hall, 863E (Apr 24, 2007 - Nov 16, 2010)
>> > > | Wilson Hall, 856E (Mar 21, 2005 - Apr 23, 2007)
>> > > +==========================================================
>> > >
>> > > On Oct 15, 2014, at 11:02 AM, Daniel Molina wrote:
>> > >
>> > > >
>> > > >
>> > > > On 15 October 2014 17:57, Steven Timm <timm at fnal.gov> wrote:
>> > > > If you call the same CreateInstances command more than once
>> > > > is there any way that it will create the instance twice or not.
>> > > >
>> > > > If you call the command twice it will create 2 instances. You can
>> also provide de MaxCount param in the CreateInstance command to create more
>> than one instance at once.
>> > > >
>> > > >
>> > > > Steve Timm
>> > > >
>> > > >
>> > > >
>> > > >
>> > > > On Wed, 15 Oct 2014, Daniel Molina wrote:
>> > > >
>> > > > Hi,
>> > > > What do you mean with "idempotent"? As long as the client
>> implements the ec2 API, it should work
>> > > > Cheers
>> > > >
>> > > > On 13 October 2014 20:33, Parag Mhashilkar <parag at fnal.gov> wrote:
>> > > > Hi,
>> > > >
>> > > > We are using HTCondor to launch VMs in OpenNebula using ec2
>> interface and would like to know if the
>> > > > submit calls are idempotent.
>> > > >
>> > > > Thanks & Regards
>> > > > +==========================================================
>> > > > | Parag Mhashilkar
>> > > > | Fermi National Accelerator Laboratory, MS 120
>> > > > | Wilson & Kirk Road, Batavia, IL - 60510
>> > > > |----------------------------------------------------------
>> > > > | Phone: 1 (630) 840-6530 Fax: 1 (630) 840-3109
>> > > > |----------------------------------------------------------
>> > > > | Wilson Hall, 806E (Nov 8, 2012 - To date)
>> > > > | Wilson Hall, 867E (Nov 17, 2010 - Nov 7, 2012)
>> > > > | Wilson Hall, 863E (Apr 24, 2007 - Nov 16, 2010)
>> > > > | Wilson Hall, 856E (Mar 21, 2005 - Apr 23, 2007)
>> > > > +==========================================================
>> > > >
>> > > >
>> > > > _______________________________________________
>> > > > Users mailing list
>> > > > Users at lists.opennebula.org
>> > > > http://lists.opennebula.org/listinfo.cgi/users-opennebula.org
>> > > >
>> > > >
>> > > >
>> > > >
>> > > > --
>> > > > --
>> > > > Daniel Molina
>> > > > Project Engineer
>> > > > OpenNebula - Flexible Enterprise Cloud Made Simple
>> > > > www.OpenNebula.org | dmolina at opennebula.org | @OpenNebula
>> > > >
>> > > >
>> > > >
>> > > > ------------------------------------------------------------------
>> > > > Steven C. Timm, Ph.D (630) 840-8525
>> > > > timm at fnal.gov http://home.fnal.gov/~timm/
>> > > > Fermilab Scientific Computing Division, Scientific Computing
>> Services Quad.
>> > > > Grid and Cloud Services Dept., Associate Dept. Head for Cloud
>> Computing
>> > > >
>> > > >
>> > > >
>> > > > --
>> > > > --
>> > > > Daniel Molina
>> > > > Project Engineer
>> > > > OpenNebula - Flexible Enterprise Cloud Made Simple
>> > > > www.OpenNebula.org | dmolina at opennebula.org | @OpenNebula
>> > >
>> > >
>> > >
>> > >
>> > > --
>> > > --
>> > > Daniel Molina
>> > > Project Engineer
>> > > OpenNebula - Flexible Enterprise Cloud Made Simple
>> > > www.OpenNebula.org | dmolina at opennebula.org | @OpenNebula
>> >
>> >
>> >
>> >
>> > --
>> > --
>> > Daniel Molina
>> > Project Engineer
>> > OpenNebula - Flexible Enterprise Cloud Made Simple
>> > www.OpenNebula.org | dmolina at opennebula.org | @OpenNebula
>>
>>
>
>
> --
> --
> Daniel Molina
> Project Engineer
> OpenNebula - Flexible Enterprise Cloud Made Simple
> www.OpenNebula.org | dmolina at opennebula.org | @OpenNebula
>
--
--
Daniel Molina
Project Engineer
OpenNebula - Flexible Enterprise Cloud Made Simple
www.OpenNebula.org | dmolina at opennebula.org | @OpenNebula
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.opennebula.org/pipermail/users-opennebula.org/attachments/20141022/0ebac5f6/attachment-0001.htm>
More information about the Users
mailing list