[one-users] OpenNebula 3.8.3 and VMWare ESXi 5.0: internal error HTTP response code 403 for upload
chenxiang at aquala-tech.com
chenxiang at aquala-tech.com
Tue Apr 23 22:39:46 PDT 2013
I double checked the VNC issue. I did set the VNC stuff according to the
instructions, but that setting got lost across reboots, so I was not able
to connect to VNC.
I searched on the web, and found a solution. That is to copy the modified
version of service.xml to the localdisk of ESXi as a backup, and then
modified /etc/rc.local to recover that file from the backup version, and
to refresh the firewall settings. (I need to do the same for the oneadmin
SSH key, otherwise the front end won't be able to connect to the ESXi box
when the box reboots.) Then VNC worked.
Regarding the disk.vmdk. I checked into the oned.log. No I did not find
the error message you mentioned.
Chen Xiang
> Hi,
>
> comments inline,
>
> On Tue, Apr 23, 2013 at 8:54 AM, <chenxiang at aquala-tech.com> wrote:
>> Now I am able to get VM's running with ESXi nodes. I would like to share
>> what I did to make things work, and would like you guys to help me
>> further.
>>
>> From Marketplace I imported the ttylinux-vmware image to local
>> infrastructure. When the images becomes READY, login to the front end
>> server and browse to var/datastores/1 and further browse to the
>> directory
>> holding that particular image. Below was what I had when things did not
>> work:
>>
>> oneadmin at opennebula:~/var/datastores/1/10fc21f21a452add3838d76d63052457$
>> ls -l
>> total 104864
>> -rw------- 1 oneadmin cloud 107374080 Jul 3 2012 ttylinux-flat.vmdk
>> -rw------- 1 oneadmin cloud 509 Jul 3 2012 ttylinux.vmdk
>>
>> Base on the error message I got from VM log, I decided to copy
>> ttylinux.vmdk to disk.vmdk. So I did that and then I had the following:
>>
>> oneadmin at opennebula:~/var/datastores/1/10fc21f21a452add3838d76d63052457$
>> cp ttylinux.vmdk disk.vmdk
>> oneadmin at opennebula:~/var/datastores/1/10fc21f21a452add3838d76d63052457$
>> ls -l
>> total 104868
>> -rw------- 1 oneadmin cloud 509 Apr 23 14:33 disk.vmdk
>> -rw------- 1 oneadmin cloud 107374080 Jul 3 2012 ttylinux-flat.vmdk
>> -rw------- 1 oneadmin cloud 509 Jul 3 2012 ttylinux.vmdk
>>
>> Now I went back to the same template, and instantiated a VM instance,
>> now
>> it worked.
>
> The vmware/cp script should automatically rename the file. Can you see
> any line in /var/log/one/oned.log similar to:
>
> --
> Error renaming disk file $BASE_DISK_FILE to disk.vmdk
> --
>
>
>>
>> Now I have a new problem. This front-end has been tested for KVM with
>> hundreds of VM instances before I tried ESXi. Now I am trying VM-ID
>> above
>> 100. Now the VM instances were running, but I was not able to connect to
>> the VM console via VNC from SunStone. What should I do? (VM Console in
>> vSphere Client still works.)
>
>
> Have you configured the ESX host to allow VNC connections? See
> http://opennebula.org/documentation:rel3.8:evmwareg#vnc
>
> Regards,
>
> -Tino
>
>>
>> Thanks a lot for your assistance.
>>
>> Chen Xiang
>>
>>
>>
>>> With the proper NFS settings, I was able to define the domain, but fail
>>> to
>>> create the VM. The error message says "File [0] 120/disk.0/disk.vmdk
>>> was
>>> not found".
>>>
>>> I login to the ESXi box, and verified that I do have /vmfs/volumes/0
>>> pointing to the right NFS mount. And inside /vmfs/volumes/0 I do have
>>> 120/disk.0/disk.vmdk .
>>>
>>> Any ideas?
>>>
>>> Below is a copy of the VM log.
>>>
>>> Tue Apr 23 13:12:58 2013 [DiM][I]: New VM state is ACTIVE.
>>> Tue Apr 23 13:12:58 2013 [LCM][I]: New VM state is PROLOG.
>>> Tue Apr 23 13:12:58 2013 [VM][I]: Virtual Machine has no context
>>> Tue Apr 23 13:13:12 2013 [TM][I]: clone: Cloning
>>> /vmfs/volumes/1/10fc21f21a452add3838d76d63052457 in
>>> vmware02:/vmfs/volumes/0/120/disk.0
>>> Tue Apr 23 13:13:12 2013 [TM][I]: ExitCode: 0
>>> Tue Apr 23 13:13:12 2013 [LCM][I]: New VM state is BOOT
>>> Tue Apr 23 13:13:12 2013 [VMM][I]: Generating deployment file:
>>> /srv/cloud/one/var/vms/120/deployment.0
>>> Tue Apr 23 13:13:12 2013 [VMM][I]: ExitCode: 0
>>> Tue Apr 23 13:13:12 2013 [VMM][I]: Successfully execute network driver
>>> operation: pre.
>>> Tue Apr 23 13:13:24 2013 [VMM][I]: Command execution fail:
>>> /srv/cloud/one/var/remotes/vmm/vmware/deploy
>>> /srv/cloud/one/var/vms/120/deployment.0 vmware02 120 vmware02
>>> Tue Apr 23 13:13:24 2013 [VMM][D]: deploy: Successfully defined domain
>>> one-120.
>>> Tue Apr 23 13:13:24 2013 [VMM][E]: deploy: Error executing: virsh -c
>>> 'esx://vmware02/?no_verify=1&auto_answer=1' start one-120 err:
>>> ExitCode: 1
>>> Tue Apr 23 13:13:24 2013 [VMM][I]: out:
>>> Tue Apr 23 13:13:24 2013 [VMM][I]: error: Failed to start domain
>>> one-120
>>> Tue Apr 23 13:13:24 2013 [VMM][I]: error: internal error Could not
>>> start
>>> domain: FileNotFound - File [0] 120/disk.0/disk.vmdk was not found
>>> Tue Apr 23 13:13:24 2013 [VMM][I]:
>>> Tue Apr 23 13:13:24 2013 [VMM][I]: ExitCode: 1
>>> Tue Apr 23 13:13:24 2013 [VMM][I]: Failed to execute virtualization
>>> driver
>>> operation: deploy.
>>> Tue Apr 23 13:13:24 2013 [VMM][E]: Error deploying virtual machine
>>> Tue Apr 23 13:13:24 2013 [DiM][I]: New VM state is FAILED
>>>
>>>
>>>
>>> Chen Xiang
>>>
>>>> Hi,
>>>>
>>>> Please use root_squash instead of no_root_squash
>>>>
>>>> --
>>>> /srv/cloud/one/var/datastores/0
>>>> *(rw,sync,no_subtree_check,no_root_squash,anonuid=10000,anongid=10000)
>>>> /srv/cloud/one/var/datastores/1
>>>> *(rw,sync,no_subtree_check,no_root_squash,anonuid=10000,anongid=10000)
>>>> --
>>>>
>>>> You will need to force the nfs server to re-read the conf file.
>>>>
>>>> Regards,
>>>>
>>>> -Tino
>>>> --
>>>> Constantino Vázquez Blanco, PhD, MSc
>>>> Project Engineer
>>>> OpenNebula - The Open-Source Solution for Data Center Virtualization
>>>> www.OpenNebula.org | @tinova79 | @OpenNebula
>>>>
>>>>
>>>> On Mon, Apr 22, 2013 at 3:46 PM, <chenxiang at aquala-tech.com> wrote:
>>>>> Here is what I got when trying to define the VM via virsh:
>>>>>
>>>>> oneadmin at opennebula:~/images$ virsh -c
>>>>> 'esx://vmware02/?no_verify=1&auto_answer=1'
>>>>> Enter username for vmware02 [root]:
>>>>> Enter root's password for vmware02:
>>>>> Welcome to virsh, the virtualization interactive terminal.
>>>>>
>>>>> Type: 'help' for help with commands
>>>>> 'quit' to quit
>>>>>
>>>>> virsh # define /srv/cloud/one/var/vms/111/deployment.0
>>>>> 2013-04-22 13:24:49.391+0000: 17332: info : libvirt version: 0.9.10
>>>>> 2013-04-22 13:24:49.391+0000: 17332: warning : virVMXFormatVNC:3224 :
>>>>> VNC
>>>>> port 6011 it out of [5900..5964] range
>>>>> error: Failed to define domain from
>>>>> /srv/cloud/one/var/vms/111/deployment.0
>>>>> error: internal error HTTP response code 403 for upload to
>>>>> 'https://vmware02:443/folder/111%2fdisk%2e0/one%2d111.vmx?dcPath=ha%2ddatacenter&dsName=0'
>>>>>
>>>>> virsh # exit
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> I am running OpenNebula 3.8.3 on Ubuntu 12.04 (front end) with
>>>>>> VMWare
>>>>>> ESXi
>>>>>> 5.0 (node). I am able to create a VMWare node in SunStone, register
>>>>>> the
>>>>>> ttylinux images (ttylinux.vmdk.tar download from C12G.com), but
>>>>>> failed
>>>>>> to
>>>>>> instantiate a VM.
>>>>>>
>>>>>> Here is what I have as the NFS exports:
>>>>>>
>>>>>> /srv/cloud/one/var/datastores/0
>>>>>> *(rw,sync,no_subtree_check,no_root_squash,anonuid=10000,anongid=10000)
>>>>>> /srv/cloud/one/var/datastores/1
>>>>>> *(rw,sync,no_subtree_check,no_root_squash,anonuid=10000,anongid=10000)
>>>>>>
>>>>>> One the ESXi node I mounted the NFS exports to /vmfs/volumes/0 and
>>>>>> /vmfs/volumes/1 respectively.
>>>>>>
>>>>>> Yes I configured libvirt-0.9.10 and make it worked with VMWare,
>>>>>> verified
>>>>>> by command lines such as the following command (where vmware02 is my
>>>>>> ESXi
>>>>>> hostname):
>>>>>>
>>>>>> virsh -c 'esx://vmware02/?no_verify=1&auto_answer=1'
>>>>>>
>>>>>> Below is my VM template:
>>>>>>
>>>>>> CPU="1"
>>>>>> DISK=[
>>>>>> IMAGE="tty_vmdk",
>>>>>> IMAGE_UNAME="oneadmin" ]
>>>>>> GRAPHICS=[
>>>>>> LISTEN="0.0.0.0",
>>>>>> TYPE="vnc" ]
>>>>>> MEMORY="512"
>>>>>> NAME="ttylinux"
>>>>>>
>>>>>> Below is what I got when trying to instantiate a VM:
>>>>>>
>>>>>>
>>>>>> Mon Apr 22 21:27:11 2013 [DiM][I]: New VM state is ACTIVE.
>>>>>> Mon Apr 22 21:27:11 2013 [LCM][I]: New VM state is PROLOG.
>>>>>> Mon Apr 22 21:27:11 2013 [VM][I]: Virtual Machine has no context
>>>>>> Mon Apr 22 21:27:17 2013 [TM][I]: clone: Cloning
>>>>>> /vmfs/volumes/1/43352fb75cee9bbc1da3c1e7ff474e26 in
>>>>>> vmware02:/vmfs/volumes/0/112/disk.0
>>>>>> Mon Apr 22 21:27:17 2013 [TM][I]: ExitCode: 0
>>>>>> Mon Apr 22 21:27:17 2013 [LCM][I]: New VM state is BOOT
>>>>>> Mon Apr 22 21:27:17 2013 [VMM][I]: Generating deployment file:
>>>>>> /srv/cloud/one/var/vms/112/deployment.0
>>>>>> Mon Apr 22 21:27:17 2013 [VMM][I]: ExitCode: 0
>>>>>> Mon Apr 22 21:27:17 2013 [VMM][I]: Successfully execute network
>>>>>> driver
>>>>>> operation: pre.
>>>>>> Mon Apr 22 21:27:22 2013 [VMM][I]: Command execution fail:
>>>>>> /srv/cloud/one/var/remotes/vmm/vmware/deploy
>>>>>> /srv/cloud/one/var/vms/112/deployment.0 vmware02 112 vmware02
>>>>>> Mon Apr 22 21:27:22 2013 [VMM][E]: deploy: Error executing: virsh -c
>>>>>> 'esx://vmware02/?no_verify=1&auto_answer=1' define
>>>>>> /srv/cloud/one/var/vms/112/deployment.0 err: ExitCode: 1
>>>>>> Mon Apr 22 21:27:22 2013 [VMM][I]: out:
>>>>>> Mon Apr 22 21:27:22 2013 [VMM][I]: 2013-04-22 13:27:21.858+0000:
>>>>>> 17586:
>>>>>> info : libvirt version: 0.9.10
>>>>>> Mon Apr 22 21:27:22 2013 [VMM][I]: 2013-04-22 13:27:21.858+0000:
>>>>>> 17586:
>>>>>> warning : virVMXFormatVNC:3224 : VNC port 6012 it out of
>>>>>> [5900..5964]
>>>>>> range
>>>>>> Mon Apr 22 21:27:22 2013 [VMM][I]: error: Failed to define domain
>>>>>> from
>>>>>> /srv/cloud/one/var/vms/112/deployment.0
>>>>>> Mon Apr 22 21:27:22 2013 [VMM][I]: error: internal error HTTP
>>>>>> response
>>>>>> code 403 for upload to
>>>>>> 'https://vmware02:443/folder/112%2fdisk%2e0/one%2d112.vmx?dcPath=ha%2ddatacenter&dsName=0'
>>>>>> Mon Apr 22 21:27:22 2013 [VMM][I]:
>>>>>> Mon Apr 22 21:27:22 2013 [VMM][I]: ExitCode: 255
>>>>>> Mon Apr 22 21:27:22 2013 [VMM][I]: Failed to execute virtualization
>>>>>> driver
>>>>>> operation: deploy.
>>>>>> Mon Apr 22 21:27:22 2013 [VMM][E]: Error deploying virtual machine
>>>>>> Mon Apr 22 21:27:22 2013 [DiM][I]: New VM state is FAILED
>>>>>>
>>>>>> What might be wrong? This looks like that I did not have write
>>>>>> access
>>>>>> to
>>>>>> some place. So I tried both the oneadmin and root acount in
>>>>>> etc/vmwarerc,
>>>>>> with not much luck.
>>>>>>
>>>>>> On the front end the oneadmin user belongs to the following groups:
>>>>>> cloud,
>>>>>> adm, sudo, libvirtd.
>>>>>>
>>>>>> Best regards,
>>>>>>
>>>>>> Chen Xiang
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> Users mailing list
>>>>>> Users at lists.opennebula.org
>>>>>> http://lists.opennebula.org/listinfo.cgi/users-opennebula.org
>>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Users mailing list
>>>>> Users at lists.opennebula.org
>>>>> http://lists.opennebula.org/listinfo.cgi/users-opennebula.org
>>>>
>>>
>>> _______________________________________________
>>> Users mailing list
>>> Users at lists.opennebula.org
>>> http://lists.opennebula.org/listinfo.cgi/users-opennebula.org
>>>
>>
>> _______________________________________________
>> Users mailing list
>> Users at lists.opennebula.org
>> http://lists.opennebula.org/listinfo.cgi/users-opennebula.org
>
More information about the Users
mailing list