[one-users] VM removed from one, but still running

[Carlos Martín Sánchez]

Hi,

Looks like you executed a onevm delete. The delete operation is tricky,
because it's a fire and forget: the cancel command failed but ONE ignored
it.
It's better to use onevm shutdown --hard. It will use the same "cancel"
driver command, but waiting for the success/error message from the driver.

Best regards.

--
Carlos Martín, MSc
Project Engineer
OpenNebula - Flexible Enterprise Cloud Made Simple
www.OpenNebula.org | cmartin at opennebula.org | @OpenNebula
<http://twitter.com/opennebula> <cmartin at opennebula.org>


On Sun, Jul 6, 2014 at 8:33 AM, Fabian Zimmermann <dev.faz at gmail.com> wrote:

> Hi,
>
> just wondered, why one of my VMs had a completely broken filesystem.
> After some research I found an old instance of my template (I'm using
> persistent storage) running on one of my nodes. It looks like OpenNebula
> didn't handle the broken auth correctly (caused by NFS-issues) and
> marked the storage as free even if the VM is still running, isn't it?
>
> --
> Sat Jun 14 20:20:13 2014 [LCM][I]: New VM state is RUNNING
> Sat Jun 28 14:11:07 2014 [LCM][I]: New VM state is UNKNOWN
> Sat Jun 28 14:11:08 2014 [LCM][I]: New VM state is CLEANUP.
> Sat Jun 28 14:11:08 2014 [VMM][I]: Driver command for 74 cancelled
> Sat Jun 28 14:11:09 2014 [VMM][I]: Command execution fail:
> /var/tmp/one/vmm/kvm/cancel 'one-74' 'node1' 74 node1
> Sat Jun 28 14:11:09 2014 [VMM][I]: Warning: Permanently added
> 'node1,192.168.20.21' (ECDSA) to the list of known hosts.
> Sat Jun 28 14:11:09 2014 [VMM][I]: Permission denied, please try again.
> Sat Jun 28 14:11:09 2014 [VMM][I]: Permission denied, please try again.
> Sat Jun 28 14:11:09 2014 [VMM][I]: Permission denied (publickey,password).
> Sat Jun 28 14:11:09 2014 [VMM][I]: ExitSSHCode: 255
> Sat Jun 28 14:11:09 2014 [VMM][E]: Error connecting to node1
> Sat Jun 28 14:11:09 2014 [VMM][I]: Failed to execute virtualization
> driver operation: cancel.
> Sat Jun 28 14:11:09 2014 [VMM][I]: Command execution fail:
> /var/tmp/one/vnm/dummy/clean
>
> PFZNPjxJRD43NDwvSUQ+PERFUExPWV9JRD5vbmUtNzQ8L0RFUExPWV9JRD48VEVNUExBVEU+PE5JQz48QlJJREdFPjwhW0NEQVRBW2JyMF1dPjwvQlJJREdFPjxJUD48IVtDREFUQVs4OS4yMzguODkuMzZdXT48L0lQPjxJUDZfTElOSz48IVtDREFUQVtmZTgwOjoyMTY6M2VmZjpmZWI5OjU2ZTBdXT48L0lQNl9MSU5LPjxNQUM+PCFbQ0RBVEFbMDA6MTY6M2U6Yjk6NTY6ZTBdXT48L01BQz48TkVUV09SSz48IVtDREFUQVs4OS4yMzguODkuMzItMjldXT48L05FVFdPUks+PE5FVFdPUktfSUQ+PCFbQ0RBVEFbNF1dPjwvTkVUV09SS19JRD48TkVUV09SS19VTkFNRT48IVtDREFUQVtvbmVhZG1pbl1dPjwvTkVUV09SS19VTkFNRT48TklDX0lEPjwhW0NEQVRBWzBdXT48L05JQ19JRD48VkxBTj48IVtDREFUQVtOT11dPjwvVkxBTj48L05JQz48L1RFTVBMQVRFPjxISVNUT1JZX1JFQ09SRFM+PEhJU1RPUlk+PEhPU1ROQU1FPm5vZGUxPC9IT1NUTkFNRT48L0hJU1RPUlk+PC9ISVNUT1JZX1JFQ09SRFM+PC9WTT4=
> Sat Jun 28 14:11:09 2014 [VMM][I]: Warning: Permanently added
> 'node1,192.168.20.21' (ECDSA) to the list of known hosts.
> Sat Jun 28 14:11:09 2014 [VMM][I]: Permission denied, please try again.
> Sat Jun 28 14:11:09 2014 [VMM][I]: Permission denied, please try again.
> Sat Jun 28 14:11:09 2014 [VMM][I]: Permission denied (publickey,password).
> Sat Jun 28 14:11:09 2014 [VMM][I]: ExitSSHCode: 255
> Sat Jun 28 14:11:09 2014 [VMM][E]: Error connecting to node1
> Sat Jun 28 14:11:09 2014 [VMM][I]: Failed to execute network driver
> operation: clean.
> Sat Jun 28 14:11:09 2014 [VMM][I]: Successfully execute transfer manager
> driver operation: tm_delete.
> Sat Jun 28 14:11:09 2014 [VMM][I]: Command execution fail:
> /var/lib/one/remotes/tm/shared/delete
> node1:/var/lib/one//datastores/0/74 74 0
> Sat Jun 28 14:11:09 2014 [VMM][I]: delete: Deleting
> /var/lib/one/datastores/0/74
> Sat Jun 28 14:11:09 2014 [VMM][E]: delete: Command "[ -e
> "/var/lib/one/datastores/0/74" ] || exit 0
> Sat Jun 28 14:11:09 2014 [VMM][I]:
> Sat Jun 28 14:11:09 2014 [VMM][I]: times=10
> Sat Jun 28 14:11:09 2014 [VMM][I]: function="rm -rf
> /var/lib/one/datastores/0/74"
> Sat Jun 28 14:11:09 2014 [VMM][I]:
> Sat Jun 28 14:11:09 2014 [VMM][I]: count=1
> Sat Jun 28 14:11:09 2014 [VMM][I]:
> Sat Jun 28 14:11:09 2014 [VMM][I]: ret=$($function)
> Sat Jun 28 14:11:09 2014 [VMM][I]: error=$?
> Sat Jun 28 14:11:09 2014 [VMM][I]:
> Sat Jun 28 14:11:09 2014 [VMM][I]: while [ $count -lt $times -a "$error"
> != "0" ]; do
> Sat Jun 28 14:11:09 2014 [VMM][I]: sleep 1
> Sat Jun 28 14:11:09 2014 [VMM][I]: count=$(( $count + 1 ))
> Sat Jun 28 14:11:09 2014 [VMM][I]: ret=$($function)
> Sat Jun 28 14:11:09 2014 [VMM][I]: error=$?
> Sat Jun 28 14:11:09 2014 [VMM][I]: done
> Sat Jun 28 14:11:09 2014 [VMM][I]:
> Sat Jun 28 14:11:09 2014 [VMM][I]: [ "x$error" = "x0" ]" failed:
> Warning: Permanently added 'node1,192.168.20.21' (ECDSA) to the list of
> known hosts.
> Sat Jun 28 14:11:09 2014 [VMM][I]: Permission denied, please try again.
> Sat Jun 28 14:11:09 2014 [VMM][I]: Permission denied, please try again.
> Sat Jun 28 14:11:09 2014 [VMM][I]: Permission denied (publickey,password).
> Sat Jun 28 14:11:09 2014 [VMM][E]: Error deleting
> /var/lib/one/datastores/0/74
> Sat Jun 28 14:11:09 2014 [VMM][I]: ExitCode: 255
> Sat Jun 28 14:11:09 2014 [VMM][I]: Failed to execute transfer manager
> driver operation: tm_delete.
> Sat Jun 28 14:11:09 2014 [VMM][I]: Host successfully cleaned.
> Sat Jun 28 14:11:09 2014 [DiM][I]: New VM state is PENDING
> Sat Jun 28 14:11:33 2014 [DiM][I]: New VM state is ACTIVE.
> Sat Jun 28 14:11:33 2014 [LCM][I]: New VM state is PROLOG.
> Sat Jun 28 14:32:57 2014 [LCM][I]: New VM state is CLEANUP.
> Sat Jun 28 14:32:57 2014 [DiM][I]: New VM state is DONE
> Sat Jun 28 14:32:57 2014 [TM][W]: Ignored: LOG I 74 Driver command for
> 74 cancelled
> Sat Jun 28 14:32:59 2014 [TM][W]: Ignored: TRANSFER SUCCESS 74 -
>
> --
>
> Fabian
> _______________________________________________
> Users mailing list
> Users at lists.opennebula.org
> http://lists.opennebula.org/listinfo.cgi/users-opennebula.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.opennebula.org/pipermail/users-opennebula.org/attachments/20140707/b41b0a1d/attachment.htm>