[one-users] Bulk delete of vCenter VM's leaves stray VM's

Thu Jan 8 02:57:05 PST 2015

The deletion code worked like this:

* Get the list of VMs
* Search the list of a VM with an specific UUID

When several VMs are being deleted it could happen that one VM marked for
deletion is in the list of all VMs but by the time it checks its UUID it is
already deleted. There is another patch for that problem [1] that you can
apply to fix it.

Still, we have made some other changes to deal with VMs and templates in
folders so I think it is better to use the version 4.10.1 that comes with
all these fixes.

Cheers

[1]
http://dev.opennebula.org/projects/opennebula/repository/revisions/e2886255464e666c6a62d91cc1d016da6ab1a643/diff/

On Mon Jan 05 2015 at 12:56:49 PM Sebastiaan Smit <bas at echelon.nl> wrote:

>  Hi Javier,
>
>
>
> The bug concerning the bulk creation of VM’s works as expected now. Do you
> have an idea of what the problem is while bulk deleting vm’s?
>
>
>
> Best regards,
>
>
>
> Sebastiaan Smit
>
>
>
> *Van:* Javier Fontan [mailto:jfontan at opennebula.org]
> *Verzonden:* vrijdag 14 november 2014 15:44
> *Aan:* Sebastiaan Smit; users at lists.opennebula.org
> *Onderwerp:* Re: [one-users] Bulk delete of vCenter VM's leaves stray VM's
>
>
>
> There was a bug in the driver that caused error when deploying several VMs
> at the same time. To fix it change the file
> /var/lib/one/remotes/vmm/vcenter/vcenter_driver.rb at line 120 from this
> code:
>
>
>
>     def find_vm_template(uuid)
>
>         vms = @dc.vmFolder.childEntity.grep(RbVmomi::VIM::VirtualMachine)
>
>
>
>         return vms.find{ |v| v.config.uuid == uuid }
>
>     end
>
>
>
> to this other one:
>
>
>
>     def find_vm_template(uuid)
>
>         vms = @dc.vmFolder.childEntity.grep(RbVmomi::VIM::VirtualMachine)
>
>
>
>         return vms.find{ |v| v.config && v.config.uuid == uuid }
>
>     end
>
>
>
> We are still looking into the problem when deleting several VMs.
>
>
>
> Thanks for telling us.
>
>
>
> On Thu Nov 13 2014 at 12:59:55 PM Javier Fontan <jfontan at opennebula.org>
> wrote:
>
> Hi,
>
>
>
> We have opened an issue to track this problem:
>
>
>
> http://dev.opennebula.org/issues/3334
>
>
>
> Meanwhile you can decrease the number of actions sent changing in
> /etc/one/oned.conf the parameter -t (number of threads) for VM driver. For
> example:
>
>
>
> VM_MAD = [
>
>     name       = "vcenter",
>
>     executable = "one_vmm_sh",
>
>     arguments  = "-p -t 2 -r 0 vcenter -s sh",
>
>     type       = "xml" ]
>
>
>
> Cheers
>
>
>
> On Wed Nov 12 2014 at 5:40:00 PM Sebastiaan Smit <bas at echelon.nl> wrote:
>
> Hi list,
>
> We're testing the vCenter functionality in version 4.10 and see some
> strange behaviour while doing bulk actions.
>
> Deleting VM's sometimes leave stray VM's on our cluster. We see the
> following in de VM log:
>
> Sun Nov  9 15:51:34 2014 [Z0][LCM][I]: New VM state is RUNNING
> Wed Nov 12 17:30:36 2014 [Z0][LCM][I]: New VM state is CLEANUP.
> Wed Nov 12 17:30:36 2014 [Z0][VMM][I]: Driver command for 60 cancelled
> Wed Nov 12 17:30:36 2014 [Z0][DiM][I]: New VM state is DONE
> Wed Nov 12 17:30:41 2014 [Z0][VMM][W]: Ignored: LOG I 60 Command execution
> fail: /var/lib/one/remotes/vmm/vcenter/cancel
> '423cdcae-b6b3-07c1-def6-96b9f3f4b7b3' 'demo-01' 60 demo-01
> Wed Nov 12 17:30:41 2014 [Z0][VMM][W]: Ignored: LOG I 60 Cancel of VM
> 423cdcae-b6b3-07c1-def6-96b9f3f4b7b3 on host demo-01 failed due to
> "ManagedObjectNotFound: The object has already been deleted or has not been
> completely created"
> Wed Nov 12 17:30:41 2014 [Z0][VMM][W]: Ignored: LOG I 60 ExitCode: 255
> Wed Nov 12 17:30:41 2014 [Z0][VMM][W]: Ignored: LOG I 60 Failed to execute
> virtualization driver operation: cancel.
> Wed Nov 12 17:30:41 2014 [Z0][VMM][W]: Ignored: LOG I 60 Successfully
> execute network driver operation: clean.
> Wed Nov 12 17:30:41 2014 [Z0][VMM][W]: Ignored: CLEANUP SUCCESS 60
>
> We see it in a different manner while bulk creating VM's (20+ at a time):
>
> Sun Nov  9 16:01:34 2014 [Z0][DiM][I]: New VM state is ACTIVE.
> Sun Nov  9 16:01:34 2014 [Z0][LCM][I]: New VM state is PROLOG.
> Sun Nov  9 16:01:34 2014 [Z0][LCM][I]: New VM state is BOOT
> Sun Nov  9 16:01:34 2014 [Z0][VMM][I]: Generating deployment file:
> /var/lib/one/vms/81/deployment.0
> Sun Nov  9 16:01:34 2014 [Z0][VMM][I]: Successfully execute network driver
> operation: pre.
> Sun Nov  9 16:01:36 2014 [Z0][VMM][I]: Command execution fail:
> /var/lib/one/remotes/vmm/vcenter/deploy '/var/lib/one/vms/81/deployment.0'
> 'demo-01' 81 demo-01
> Sun Nov  9 16:01:36 2014 [Z0][VMM][I]: Deploy of VM 81 on host demo-01
> with /var/lib/one/vms/81/deployment.0 failed due to "undefined method
> `uuid' for nil:NilClass"
> Sun Nov  9 16:01:36 2014 [Z0][VMM][I]: ExitCode: 255
> Sun Nov  9 16:01:36 2014 [Z0][VMM][I]: Failed to execute virtualization
> driver operation: deploy.
> Sun Nov  9 16:01:36 2014 [Z0][VMM][E]: Error deploying virtual machine
> Sun Nov  9 16:01:36 2014 [Z0][DiM][I]: New VM state is FAILED
> Wed Nov 12 17:30:19 2014 [Z0][DiM][I]: New VM state is DONE.
> Wed Nov 12 17:30:19 2014 [Z0][LCM][E]: epilog_success_action, VM in a
> wrong state
>
>
> I think these have two different root causes. The cluster is not under
> load.
>
>
> Has anyone else seen this behaviour?
>
> Best regards,
> --
> Sebastiaan Smit
> Echelon BV
>
> E: bas at echelon.nl
> W: www.echelon.nl
> T: (088) 3243566 (gewijzigd nummer)
> T: (088) 3243505 (servicedesk)
> F: (053) 4336222
>
> KVK: 06055381
>
>
> _______________________________________________
> Users mailing list
> Users at lists.opennebula.org
> http://lists.opennebula.org/listinfo.cgi/users-opennebula.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.opennebula.org/pipermail/users-opennebula.org/attachments/20150108/9aaaee37/attachment-0001.htm>