[one-users] OpenNebula 3.8.3 and VMWare ESXi 5.0: internal error HTTP response code 403 for upload
Tino Vazquez
tinova at opennebula.org
Wed Apr 24 02:31:11 PDT 2013
On Wed, Apr 24, 2013 at 7:39 AM, <chenxiang at aquala-tech.com> wrote:
> I double checked the VNC issue. I did set the VNC stuff according to the
> instructions, but that setting got lost across reboots, so I was not able
> to connect to VNC.
>
> I searched on the web, and found a solution. That is to copy the modified
> version of service.xml to the localdisk of ESXi as a backup, and then
> modified /etc/rc.local to recover that file from the backup version, and
> to refresh the firewall settings. (I need to do the same for the oneadmin
> SSH key, otherwise the front end won't be able to connect to the ESXi box
> when the box reboots.) Then VNC worked.
Glad you got it working. A similar approach is defined here [1]
(search for "persistency")
[1] http://opennebula.org/documentation:rel3.8:evmwareg
>
> Regarding the disk.vmdk. I checked into the oned.log. No I did not find
> the error message you mentioned.
Can you send us the relevant files in oned.log at the time of
registering a VMware disk folder _without_ the manual renaming to
disk.vmdk?
Thanks a lot for your feedback,
-T
>
> Chen Xiang
>
>
>> Hi,
>>
>> comments inline,
>>
>> On Tue, Apr 23, 2013 at 8:54 AM, <chenxiang at aquala-tech.com> wrote:
>>> Now I am able to get VM's running with ESXi nodes. I would like to share
>>> what I did to make things work, and would like you guys to help me
>>> further.
>>>
>>> From Marketplace I imported the ttylinux-vmware image to local
>>> infrastructure. When the images becomes READY, login to the front end
>>> server and browse to var/datastores/1 and further browse to the
>>> directory
>>> holding that particular image. Below was what I had when things did not
>>> work:
>>>
>>> oneadmin at opennebula:~/var/datastores/1/10fc21f21a452add3838d76d63052457$
>>> ls -l
>>> total 104864
>>> -rw------- 1 oneadmin cloud 107374080 Jul 3 2012 ttylinux-flat.vmdk
>>> -rw------- 1 oneadmin cloud 509 Jul 3 2012 ttylinux.vmdk
>>>
>>> Base on the error message I got from VM log, I decided to copy
>>> ttylinux.vmdk to disk.vmdk. So I did that and then I had the following:
>>>
>>> oneadmin at opennebula:~/var/datastores/1/10fc21f21a452add3838d76d63052457$
>>> cp ttylinux.vmdk disk.vmdk
>>> oneadmin at opennebula:~/var/datastores/1/10fc21f21a452add3838d76d63052457$
>>> ls -l
>>> total 104868
>>> -rw------- 1 oneadmin cloud 509 Apr 23 14:33 disk.vmdk
>>> -rw------- 1 oneadmin cloud 107374080 Jul 3 2012 ttylinux-flat.vmdk
>>> -rw------- 1 oneadmin cloud 509 Jul 3 2012 ttylinux.vmdk
>>>
>>> Now I went back to the same template, and instantiated a VM instance,
>>> now
>>> it worked.
>>
>> The vmware/cp script should automatically rename the file. Can you see
>> any line in /var/log/one/oned.log similar to:
>>
>> --
>> Error renaming disk file $BASE_DISK_FILE to disk.vmdk
>> --
>>
>>
>>>
>>> Now I have a new problem. This front-end has been tested for KVM with
>>> hundreds of VM instances before I tried ESXi. Now I am trying VM-ID
>>> above
>>> 100. Now the VM instances were running, but I was not able to connect to
>>> the VM console via VNC from SunStone. What should I do? (VM Console in
>>> vSphere Client still works.)
>>
>>
>> Have you configured the ESX host to allow VNC connections? See
>> http://opennebula.org/documentation:rel3.8:evmwareg#vnc
>>
>> Regards,
>>
>> -Tino
>>
>>>
>>> Thanks a lot for your assistance.
>>>
>>> Chen Xiang
>>>
>>>
>>>
>>>> With the proper NFS settings, I was able to define the domain, but fail
>>>> to
>>>> create the VM. The error message says "File [0] 120/disk.0/disk.vmdk
>>>> was
>>>> not found".
>>>>
>>>> I login to the ESXi box, and verified that I do have /vmfs/volumes/0
>>>> pointing to the right NFS mount. And inside /vmfs/volumes/0 I do have
>>>> 120/disk.0/disk.vmdk .
>>>>
>>>> Any ideas?
>>>>
>>>> Below is a copy of the VM log.
>>>>
>>>> Tue Apr 23 13:12:58 2013 [DiM][I]: New VM state is ACTIVE.
>>>> Tue Apr 23 13:12:58 2013 [LCM][I]: New VM state is PROLOG.
>>>> Tue Apr 23 13:12:58 2013 [VM][I]: Virtual Machine has no context
>>>> Tue Apr 23 13:13:12 2013 [TM][I]: clone: Cloning
>>>> /vmfs/volumes/1/10fc21f21a452add3838d76d63052457 in
>>>> vmware02:/vmfs/volumes/0/120/disk.0
>>>> Tue Apr 23 13:13:12 2013 [TM][I]: ExitCode: 0
>>>> Tue Apr 23 13:13:12 2013 [LCM][I]: New VM state is BOOT
>>>> Tue Apr 23 13:13:12 2013 [VMM][I]: Generating deployment file:
>>>> /srv/cloud/one/var/vms/120/deployment.0
>>>> Tue Apr 23 13:13:12 2013 [VMM][I]: ExitCode: 0
>>>> Tue Apr 23 13:13:12 2013 [VMM][I]: Successfully execute network driver
>>>> operation: pre.
>>>> Tue Apr 23 13:13:24 2013 [VMM][I]: Command execution fail:
>>>> /srv/cloud/one/var/remotes/vmm/vmware/deploy
>>>> /srv/cloud/one/var/vms/120/deployment.0 vmware02 120 vmware02
>>>> Tue Apr 23 13:13:24 2013 [VMM][D]: deploy: Successfully defined domain
>>>> one-120.
>>>> Tue Apr 23 13:13:24 2013 [VMM][E]: deploy: Error executing: virsh -c
>>>> 'esx://vmware02/?no_verify=1&auto_answer=1' start one-120 err:
>>>> ExitCode: 1
>>>> Tue Apr 23 13:13:24 2013 [VMM][I]: out:
>>>> Tue Apr 23 13:13:24 2013 [VMM][I]: error: Failed to start domain
>>>> one-120
>>>> Tue Apr 23 13:13:24 2013 [VMM][I]: error: internal error Could not
>>>> start
>>>> domain: FileNotFound - File [0] 120/disk.0/disk.vmdk was not found
>>>> Tue Apr 23 13:13:24 2013 [VMM][I]:
>>>> Tue Apr 23 13:13:24 2013 [VMM][I]: ExitCode: 1
>>>> Tue Apr 23 13:13:24 2013 [VMM][I]: Failed to execute virtualization
>>>> driver
>>>> operation: deploy.
>>>> Tue Apr 23 13:13:24 2013 [VMM][E]: Error deploying virtual machine
>>>> Tue Apr 23 13:13:24 2013 [DiM][I]: New VM state is FAILED
>>>>
>>>>
>>>>
>>>> Chen Xiang
>>>>
>>>>> Hi,
>>>>>
>>>>> Please use root_squash instead of no_root_squash
>>>>>
>>>>> --
>>>>> /srv/cloud/one/var/datastores/0
>>>>> *(rw,sync,no_subtree_check,no_root_squash,anonuid=10000,anongid=10000)
>>>>> /srv/cloud/one/var/datastores/1
>>>>> *(rw,sync,no_subtree_check,no_root_squash,anonuid=10000,anongid=10000)
>>>>> --
>>>>>
>>>>> You will need to force the nfs server to re-read the conf file.
>>>>>
>>>>> Regards,
>>>>>
>>>>> -Tino
>>>>> --
>>>>> Constantino Vázquez Blanco, PhD, MSc
>>>>> Project Engineer
>>>>> OpenNebula - The Open-Source Solution for Data Center Virtualization
>>>>> www.OpenNebula.org | @tinova79 | @OpenNebula
>>>>>
>>>>>
>>>>> On Mon, Apr 22, 2013 at 3:46 PM, <chenxiang at aquala-tech.com> wrote:
>>>>>> Here is what I got when trying to define the VM via virsh:
>>>>>>
>>>>>> oneadmin at opennebula:~/images$ virsh -c
>>>>>> 'esx://vmware02/?no_verify=1&auto_answer=1'
>>>>>> Enter username for vmware02 [root]:
>>>>>> Enter root's password for vmware02:
>>>>>> Welcome to virsh, the virtualization interactive terminal.
>>>>>>
>>>>>> Type: 'help' for help with commands
>>>>>> 'quit' to quit
>>>>>>
>>>>>> virsh # define /srv/cloud/one/var/vms/111/deployment.0
>>>>>> 2013-04-22 13:24:49.391+0000: 17332: info : libvirt version: 0.9.10
>>>>>> 2013-04-22 13:24:49.391+0000: 17332: warning : virVMXFormatVNC:3224 :
>>>>>> VNC
>>>>>> port 6011 it out of [5900..5964] range
>>>>>> error: Failed to define domain from
>>>>>> /srv/cloud/one/var/vms/111/deployment.0
>>>>>> error: internal error HTTP response code 403 for upload to
>>>>>> 'https://vmware02:443/folder/111%2fdisk%2e0/one%2d111.vmx?dcPath=ha%2ddatacenter&dsName=0'
>>>>>>
>>>>>> virsh # exit
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> I am running OpenNebula 3.8.3 on Ubuntu 12.04 (front end) with
>>>>>>> VMWare
>>>>>>> ESXi
>>>>>>> 5.0 (node). I am able to create a VMWare node in SunStone, register
>>>>>>> the
>>>>>>> ttylinux images (ttylinux.vmdk.tar download from C12G.com), but
>>>>>>> failed
>>>>>>> to
>>>>>>> instantiate a VM.
>>>>>>>
>>>>>>> Here is what I have as the NFS exports:
>>>>>>>
>>>>>>> /srv/cloud/one/var/datastores/0
>>>>>>> *(rw,sync,no_subtree_check,no_root_squash,anonuid=10000,anongid=10000)
>>>>>>> /srv/cloud/one/var/datastores/1
>>>>>>> *(rw,sync,no_subtree_check,no_root_squash,anonuid=10000,anongid=10000)
>>>>>>>
>>>>>>> One the ESXi node I mounted the NFS exports to /vmfs/volumes/0 and
>>>>>>> /vmfs/volumes/1 respectively.
>>>>>>>
>>>>>>> Yes I configured libvirt-0.9.10 and make it worked with VMWare,
>>>>>>> verified
>>>>>>> by command lines such as the following command (where vmware02 is my
>>>>>>> ESXi
>>>>>>> hostname):
>>>>>>>
>>>>>>> virsh -c 'esx://vmware02/?no_verify=1&auto_answer=1'
>>>>>>>
>>>>>>> Below is my VM template:
>>>>>>>
>>>>>>> CPU="1"
>>>>>>> DISK=[
>>>>>>> IMAGE="tty_vmdk",
>>>>>>> IMAGE_UNAME="oneadmin" ]
>>>>>>> GRAPHICS=[
>>>>>>> LISTEN="0.0.0.0",
>>>>>>> TYPE="vnc" ]
>>>>>>> MEMORY="512"
>>>>>>> NAME="ttylinux"
>>>>>>>
>>>>>>> Below is what I got when trying to instantiate a VM:
>>>>>>>
>>>>>>>
>>>>>>> Mon Apr 22 21:27:11 2013 [DiM][I]: New VM state is ACTIVE.
>>>>>>> Mon Apr 22 21:27:11 2013 [LCM][I]: New VM state is PROLOG.
>>>>>>> Mon Apr 22 21:27:11 2013 [VM][I]: Virtual Machine has no context
>>>>>>> Mon Apr 22 21:27:17 2013 [TM][I]: clone: Cloning
>>>>>>> /vmfs/volumes/1/43352fb75cee9bbc1da3c1e7ff474e26 in
>>>>>>> vmware02:/vmfs/volumes/0/112/disk.0
>>>>>>> Mon Apr 22 21:27:17 2013 [TM][I]: ExitCode: 0
>>>>>>> Mon Apr 22 21:27:17 2013 [LCM][I]: New VM state is BOOT
>>>>>>> Mon Apr 22 21:27:17 2013 [VMM][I]: Generating deployment file:
>>>>>>> /srv/cloud/one/var/vms/112/deployment.0
>>>>>>> Mon Apr 22 21:27:17 2013 [VMM][I]: ExitCode: 0
>>>>>>> Mon Apr 22 21:27:17 2013 [VMM][I]: Successfully execute network
>>>>>>> driver
>>>>>>> operation: pre.
>>>>>>> Mon Apr 22 21:27:22 2013 [VMM][I]: Command execution fail:
>>>>>>> /srv/cloud/one/var/remotes/vmm/vmware/deploy
>>>>>>> /srv/cloud/one/var/vms/112/deployment.0 vmware02 112 vmware02
>>>>>>> Mon Apr 22 21:27:22 2013 [VMM][E]: deploy: Error executing: virsh -c
>>>>>>> 'esx://vmware02/?no_verify=1&auto_answer=1' define
>>>>>>> /srv/cloud/one/var/vms/112/deployment.0 err: ExitCode: 1
>>>>>>> Mon Apr 22 21:27:22 2013 [VMM][I]: out:
>>>>>>> Mon Apr 22 21:27:22 2013 [VMM][I]: 2013-04-22 13:27:21.858+0000:
>>>>>>> 17586:
>>>>>>> info : libvirt version: 0.9.10
>>>>>>> Mon Apr 22 21:27:22 2013 [VMM][I]: 2013-04-22 13:27:21.858+0000:
>>>>>>> 17586:
>>>>>>> warning : virVMXFormatVNC:3224 : VNC port 6012 it out of
>>>>>>> [5900..5964]
>>>>>>> range
>>>>>>> Mon Apr 22 21:27:22 2013 [VMM][I]: error: Failed to define domain
>>>>>>> from
>>>>>>> /srv/cloud/one/var/vms/112/deployment.0
>>>>>>> Mon Apr 22 21:27:22 2013 [VMM][I]: error: internal error HTTP
>>>>>>> response
>>>>>>> code 403 for upload to
>>>>>>> 'https://vmware02:443/folder/112%2fdisk%2e0/one%2d112.vmx?dcPath=ha%2ddatacenter&dsName=0'
>>>>>>> Mon Apr 22 21:27:22 2013 [VMM][I]:
>>>>>>> Mon Apr 22 21:27:22 2013 [VMM][I]: ExitCode: 255
>>>>>>> Mon Apr 22 21:27:22 2013 [VMM][I]: Failed to execute virtualization
>>>>>>> driver
>>>>>>> operation: deploy.
>>>>>>> Mon Apr 22 21:27:22 2013 [VMM][E]: Error deploying virtual machine
>>>>>>> Mon Apr 22 21:27:22 2013 [DiM][I]: New VM state is FAILED
>>>>>>>
>>>>>>> What might be wrong? This looks like that I did not have write
>>>>>>> access
>>>>>>> to
>>>>>>> some place. So I tried both the oneadmin and root acount in
>>>>>>> etc/vmwarerc,
>>>>>>> with not much luck.
>>>>>>>
>>>>>>> On the front end the oneadmin user belongs to the following groups:
>>>>>>> cloud,
>>>>>>> adm, sudo, libvirtd.
>>>>>>>
>>>>>>> Best regards,
>>>>>>>
>>>>>>> Chen Xiang
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Users mailing list
>>>>>>> Users at lists.opennebula.org
>>>>>>> http://lists.opennebula.org/listinfo.cgi/users-opennebula.org
>>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> Users mailing list
>>>>>> Users at lists.opennebula.org
>>>>>> http://lists.opennebula.org/listinfo.cgi/users-opennebula.org
>>>>>
>>>>
>>>> _______________________________________________
>>>> Users mailing list
>>>> Users at lists.opennebula.org
>>>> http://lists.opennebula.org/listinfo.cgi/users-opennebula.org
>>>>
>>>
>>> _______________________________________________
>>> Users mailing list
>>> Users at lists.opennebula.org
>>> http://lists.opennebula.org/listinfo.cgi/users-opennebula.org
>>
>
>
> _______________________________________________
> Users mailing list
> Users at lists.opennebula.org
> http://lists.opennebula.org/listinfo.cgi/users-opennebula.org
More information about the Users
mailing list