[one-users] OpenNebula 3.8.3 Failed to Delete VM on ESXi Host

[Tino Vazquez]

Hi Chen,

Let's check the permissions on the ESX. Disks should be created under
Administrator ownership, so they can be deleted properly afterwards.
The logs suggests otherwise:

---
Ignored: TRANSFER FAILURE 104 Cannot delete
[103]104/disk.0/disk-flat.vmdk in vmw01
---

Could you ssh into vmw01 and send through the output of:

$ ls -lrtah /vmfs/volumes/103/104

$ ls -lrtah /vmfs/volumes/103/104/disk.0

Regards,

-Tino

--
Constantino Vázquez Blanco, PhD, MSc
Senior Infrastructure Architect at C12G Labs
www.c12g.com | @C12G | es.linkedin.com/in/tinova

--
Confidentiality Warning: The information contained in this e-mail and
any accompanying documents, unless otherwise expressly indicated, is
confidential and privileged, and is intended solely for the person
and/or entity to whom it is addressed (i.e. those identified in the
"To" and "cc" box). They are the property of C12G Labs S.L..
Unauthorized distribution, review, use, disclosure, or copying of this
communication, or any part thereof, is strictly prohibited and may be
unlawful. If you have received this e-mail in error, please notify us
immediately by e-mail at abuse at c12g.com and delete the e-mail and
attachments and any copy from your system. C12G thanks you for your
cooperation.


On Wed, Jul 17, 2013 at 12:57 AM,  <chenxiang at aquala-tech.com> wrote:
> Hi Tino,
>
> Thanks a lot for the pointer. I replaced the ~/var/remotes/tm/vmfs
> directory with the one from 4.0.1, and now it is trying to delete files in
> the correct datastore. However, it still can not delete the files on ESXi.
>
> I do have the correct oneadmin user and permissions on ESXi.
>
> I am using the API instead of SSH to communicate with VMFS.  When trying
> to delete the VM, I got the following message in vm.log:
>
> Wed Jul 17 06:43:39 2013 [VMM][W]: Ignored: LOG I 104 Driver command for
> 104 cancelled
> Wed Jul 17 06:43:41 2013 [TM][W]: Ignored: LOG I 104 Command execution
> fail: /srv/cloud/one/var/remotes/tm/vmfs/delete
> vmw01:/vmfs/volumes/103/104/disk.0 104 102
> Wed Jul 17 06:43:41 2013 [TM][W]: Ignored: LOG I 104 delete: Deleting
> /vmfs/volumes/103/104/disk.0
> Wed Jul 17 06:43:41 2013 [TM][W]: Ignored: LOG I 104
> /srv/cloud/one/var/remotes/tm/vmfs/delete: line 88: disk_type: command not
> found
> Wed Jul 17 06:43:41 2013 [TM][W]: Ignored: LOG E 104 delete: Command "vifs
> --server vmw01 --username Administrator --password MyVMWarePassword
> --force --rm [103]104/disk.0/disk-flat.vmdk" failed: Unable to delete
> [103] 104/disk.0/disk-flat.vmdk:
> Wed Jul 17 06:43:41 2013 [TM][W]: Ignored: LOG I 104 Error caused by file
> /vmfs/volumes/51dd0f09-355621d0-6d8c-001b21cfaf45/104/disk.0/disk-flat.vmdk
> Wed Jul 17 06:43:41 2013 [TM][W]: Ignored: LOG E 104 Cannot delete
> [103]104/disk.0/disk-flat.vmdk in vmw01
> Wed Jul 17 06:43:41 2013 [TM][W]: Ignored: LOG I 104 ExitCode: 1
> Wed Jul 17 06:43:41 2013 [TM][W]: Ignored: TRANSFER FAILURE 104 Cannot
> delete [103]104/disk.0/disk-flat.vmdk in vmw01
> Wed Jul 17 06:43:44 2013 [VMM][W]: Ignored: LOG D 104 cancel: Successfully
> canceled domain one-104.
> Wed Jul 17 06:43:44 2013 [VMM][W]: Ignored: LOG I 104 ExitCode: 0
> Wed Jul 17 06:43:44 2013 [VMM][W]: Ignored: LOG I 104 Successfully execute
> virtualization driver operation: cancel.
> Wed Jul 17 06:43:44 2013 [VMM][W]: Ignored: LOG I 104 ExitCode: 0
> Wed Jul 17 06:43:44 2013 [VMM][W]: Ignored: LOG I 104 Successfully execute
> network driver operation: clean.
> Wed Jul 17 06:43:44 2013 [VMM][W]: Ignored: CANCEL SUCCESS 104 -
>
> As you can see, in ~/etc/vmwarerc I have Administrator user (instead of
> oneadmin) to communicate with ESXi. (But I do have the oneadmin user on
> ESXi) Is this a problem?
>
> Thanks a lot.
>
> Chen Xiang
>
>> Hi Chen Xiang,
>
>> You are right , there are several bugs in the 3.8.3 VMFS drivers. I
>> recommend updating to 3.8.4 at least, and ideally to 4.0.1.
>>
>> About the permissions problems, you should create an ESX user
>> 'oneadmin' with the same UID as 'oneadmin' in the front-end.
>>
>> Hope it helps,
>>
>> -Tino
>> --
>> Constantino Vázquez Blanco, PhD, MSc
>> Senior Infrastructure Architect at C12G Labs
>> www.c12g.com | @C12G | es.linkedin.com/in/tinova
>>
>> --
>> Confidentiality Warning: The information contained in this e-mail and
>> any accompanying documents, unless otherwise expressly indicated, is
>> confidential and privileged, and is intended solely for the person
>> and/or entity to whom it is addressed (i.e. those identified in the
>> "To" and "cc" box). They are the property of C12G Labs S.L..
>> Unauthorized distribution, review, use, disclosure, or copying of this
>> communication, or any part thereof, is strictly prohibited and may be
>> unlawful. If you have received this e-mail in error, please notify us
>> immediately by e-mail at abuse at c12g.com and delete the e-mail and
>> attachments and any copy from your system. C12G thanks you for your
>> cooperation.
>>
>>
>> On Tue, Jul 16, 2013 at 4:48 PM,  <chenxiang at aquala-tech.com> wrote:
>>> I looked into the related source code in 4.0. It seems that the DSID
>>> problem was fixed in 4.0. How about permission problem?
>>>
>>> Can I simple overwrite the 3.8.4 version ~/var/remotes/tm/vmfs directory
>>> with the one from 4.0?
>>>
>>> Chen Xiang
>>>
>>>> Hi,
>>>>
>>>> I am running OpenNebula 3.8.3 with VMFS. However, I am not able to
>>>> delete
>>>> the VM files from the ESXi Host, although the VM seems to be deleted
>>>> from
>>>> OpenNebula.
>>>>
>>>> My setup is that I have an image DS 102, and a system DS 103. When
>>>> trying
>>>> to delete a VM, I can see that OpenNebula is trying to delete the VM
>>>> image
>>>> from DS 102 rather than 103. I look into ~/var/remotes/tm/vmfs/delete,
>>>> and
>>>> manually change the DSID to 103 (this is not a good fix), and now
>>>> OpenNebula is trying to delete the VM images from DS 103, which is
>>>> correct. However, the delete script was not able to delete the files,
>>>> still!
>>>>
>>>> I look into ~/var/vms/$VMID/vm.log, and the log says that it was not
>>>> able
>>>> to delete disk-flat.vmdk on the ESXi box.
>>>>
>>>> I look into /vmfs/volumes/103/$VMID/disk.0, and found that the two VMDK
>>>> files have rw permissions for root user and no permissions for others,
>>>> and
>>>> other files have rw permissons for root user and r permission for group
>>>> members and guest.
>>>>
>>>> I log into the ESXi box, and was able to delete the files with root
>>>> users.
>>>>
>>>> In ~/etc/vmwarerc I have the root username and password.
>>>>
>>>> What could be wrong? Do I miss some executable permissions on the ESXi
>>>> box?
>>>>
>>>> Please help.
>>>>
>>>> Chen Xiang
>>>>
>>>> _______________________________________________
>>>> Users mailing list
>>>> Users at lists.opennebula.org
>>>> http://lists.opennebula.org/listinfo.cgi/users-opennebula.org
>>>>
>>>
>>>
>>> _______________________________________________
>>> Users mailing list
>>> Users at lists.opennebula.org
>>> http://lists.opennebula.org/listinfo.cgi/users-opennebula.org
>>
>
>
> _______________________________________________
> Users mailing list
> Users at lists.opennebula.org
> http://lists.opennebula.org/listinfo.cgi/users-opennebula.org