[one-users] Images from Images (Ceph) Datastore Not Deploying
Steve @ MyEvolve
steve at myevolve.com
Mon Apr 14 12:25:53 PDT 2014
SOLVED!
Hi Bill & Ruben,
I went ahead and upgraded to 4.6RC. While carefully examining the template
for my ceph datastore, I realized that "CEPH_SECRET" was incorrect, it was
listed as "CEPH_SERCRET" -- which explains why ONE wasn't adding it to the
deployment xml.
Fixed issue and now deploying from ceph correctly. :-)
Thanks so much for your help and I look forward to contributing to the
community moving forward!
Steve
On Fri, Apr 11, 2014 at 2:44 PM, Steve @ MyEvolve <steve at myevolve.com>wrote:
> 4.4.1
>
>
>
>
> On Fri, Apr 11, 2014 at 7:57 AM, Campbell, Bill <
> bcampbell at axcess-financial.com> wrote:
>
>> That's a very good question. Just to validate, which version of
>> OpenNebula are you using?
>>
>>
>>
>> *From:* Steve @ MyEvolve [mailto:steve at myevolve.com]
>> *Sent:* Thursday, April 10, 2014 9:33 PM
>> *To:* Campbell, Bill
>> *Cc:* Ruben S. Montero; users at lists.opennebula.org
>>
>> *Subject:* Re: [one-users] Images from Images (Ceph) Datastore Not
>> Deploying
>>
>>
>>
>> Also, why is ONE not adding the <auth> to the deployment.0 file?
>>
>>
>>
>>
>>
>> On Thu, Apr 10, 2014 at 6:32 PM, Steve @ MyEvolve <steve at myevolve.com>
>> wrote:
>>
>> Hi Bill & Ruben,
>>
>>
>>
>> 1.) I have confirmed that CEPH_HOST, CEPH_USER, and CEPH_SECRET are
>> properly set in my datastore.
>>
>>
>>
>> 2.) Bill you bring up a good point about why:
>>
>>
>>
>> <auth username='libvirt'>
>>
>> <secret type='ceph' uuid='34afabcd-8ee0-45fd-a880-09df24fef603'/>
>>
>> </auth>
>>
>> is not added to my deployment.0 file. I added it to the end just before
>> </domain>.
>>
>>
>>
>> 3.) I have tried to run the following command to manually test, but it
>> doesn't work for me. What am I doing wrong?
>>
>> [root at cloud2 54]# virsh create -c qemu:///system deployment.0
>>
>> error: unexpected data 'qemu:///system'
>>
>> [root at cloud2 54]#
>>
>>
>>
>> Thank you so much, I know we have to be close!
>>
>> Steve
>>
>>
>>
>> On Wed, Apr 9, 2014 at 11:56 AM, Campbell, Bill <
>> bcampbell at axcess-financial.com> wrote:
>>
>> A couple of things:
>>
>> · You shouldn't need the port, that should be generated
>> appropriately
>>
>> · I'm not seeing your secret definition within the deployment.0
>> file. That must exist if using the newer versions of Libvirt w/CephX
>> authentication (Libvirt assumes if you are specifying hosts that a secret
>> should be there). You should have an entry in that file like so (right
>> after the </source>):
>>
>>
>>
>> <auth username='libvirt'>
>>
>> <secret type='ceph'
>> uuid='ebc9c744-016d-40af-ae73-621f558398f0'/>
>>
>> </auth>
>>
>>
>>
>> · You've defined the secret/user on the Datastore within
>> OpenNebula using the appropriate configuration attributes CEPH_HOST,
>> CEPH_USER, and CEPH_SECRET?
>>
>> o And also defined the appropriate secret in Libvirt within each
>> hypervisor in your configuration?
>>
>>
>>
>> Outside of verifying the above, I'm not sure what else can be attempted.
>> You could try disabling CephX authentication, removing the CEPH_HOST
>> configuration attributes, and copying the /etc/ceph/ceph.conf and admin
>> keyring to each hypervisor and see if you can connect that way (however
>> newer versions of libvirt will probably prevent that from happening).
>>
>>
>>
>>
>>
>> *From:* Steve @ MyEvolve [mailto:steve at myevolve.com]
>> *Sent:* Wednesday, April 09, 2014 2:44 PM
>> *To:* Ruben S. Montero
>> *Cc:* users at lists.opennebula.org; Bill Campbell
>>
>>
>> *Subject:* Re: [one-users] Images from Images (Ceph) Datastore Not
>> Deploying
>>
>>
>>
>> Any suggestions on this?
>>
>>
>>
>> On Tue, Apr 8, 2014 at 7:37 PM, Steve @ MyEvolve <steve at myevolve.com>
>> wrote:
>>
>> Hi Ruben,
>>
>>
>>
>> So I've been trying to experiment a bit. I started over. Re-imported
>> ttylinux into ceph datastore. Created new template and new vm and I logged
>> in as oneadmin and defined the secret as instructed on all hosts.
>>
>>
>>
>> In creating the new VM, the process failed as before.
>>
>>
>>
>> I tried as you suggested:
>>
>>
>>
>> [oneadmin at cloud2 54]$ virsh create -c qemu:///system deployment.0
>>
>> error: unexpected data 'qemu:///system'
>>
>> But as you can see I got an error message I'm not sure what to do with.
>>
>>
>>
>> Also, I have a new log that looks different -- seems adding the port to
>> the CEPH_HOST was the distinguishing factor.
>>
>>
>>
>> Tue Apr 8 19:30:04 2014 [DiM][I]: New VM state is ACTIVE.
>> Tue Apr 8 19:30:04 2014 [LCM][I]: New VM state is PROLOG.
>> Tue Apr 8 19:30:09 2014 [LCM][I]: New VM state is BOOT
>> Tue Apr 8 19:30:10 2014 [VMM][I]: Generating deployment file:
>> /var/lib/one/vms/54/deployment.0
>> Tue Apr 8 19:30:10 2014 [VMM][I]: ExitCode: 0
>> Tue Apr 8 19:30:10 2014 [VMM][I]: Successfully execute network driver
>> operation: pre.
>> Tue Apr 8 19:30:10 2014 [VMM][I]: Command execution fail: cat << EOT |
>> /var/tmp/one/vmm/kvm/deploy '/var/lib/one//datastores/125/54/deployment.0'
>> 'cloud2' 54 cloud2
>> Tue Apr 8 19:30:10 2014 [VMM][I]: error: Failed to create domain from
>> /var/lib/one//datastores/125/54/deployment.0
>> Tue Apr 8 19:30:10 2014 [VMM][I]: error: internal error process exited
>> while connecting to monitor: qemu-kvm: -drive
>> file=rbd:one/one-28-54-0:auth_supported=none:mon_host=cloudweb12\:6789,if=none,id=drive-ide0-0-0,format=raw,cache=none:
>> error connecting
>> Tue Apr 8 19:30:10 2014 [VMM][I]: qemu-kvm: -drive
>> file=rbd:one/one-28-54-0:auth_supported=none:mon_host=cloudweb12\:6789,if=none,id=drive-ide0-0-0,format=raw,cache=none:
>> could not open disk image
>> rbd:one/one-28-54-0:auth_supported=none:mon_host=cloudweb12\:6789:
>> Operation not supported
>> Tue Apr 8 19:30:10 2014 [VMM][I]:
>> Tue Apr 8 19:30:10 2014 [VMM][E]: Could not create domain from
>> /var/lib/one//datastores/125/54/deployment.0
>> Tue Apr 8 19:30:10 2014 [VMM][I]: ExitCode: 255
>> Tue Apr 8 19:30:10 2014 [VMM][I]: Failed to execute virtualization driver
>> operation: deploy.
>> Tue Apr 8 19:30:10 2014 [VMM][E]: Error deploying virtual machine: Could
>> not create domain from /var/lib/one//datastores/125/54/deployment.0
>> Tue Apr 8 19:30:10 2014 [DiM][I]: New VM state is FAILED
>>
>>
>>
>> And also here is deployment.0:
>>
>> <domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0
>> '>
>>
>> <name>one-54</name>
>>
>> <cputune>
>>
>> <shares>1024</shares>
>>
>> </cputune>
>>
>> <memory>524288</memory>
>>
>> <os>
>>
>> <type arch='x86_64'>hvm</type>
>>
>> <boot dev='hd'/>
>>
>> </os>
>>
>> <devices>
>>
>> <emulator>/usr/libexec/qemu-kvm</emulator>
>>
>> <disk type='network' device='disk'>
>>
>> <source protocol='rbd' name='one/one-28-54-0'>
>>
>> <host name='cloudweb12' port='6789'/>
>>
>> </source>
>>
>> <target dev='hda'/>
>>
>> <driver name='qemu' type='raw' cache='none'/>
>>
>> </disk>
>>
>> <disk type='file' device='cdrom'>
>>
>> <source
>> file='/var/lib/one//datastores/125/54/disk.1'/>
>>
>> <target dev='hdb'/>
>>
>> <readonly/>
>>
>> <driver name='qemu' type='raw'/>
>>
>> </disk>
>>
>> <interface type='bridge'>
>>
>> <source bridge='br0'/>
>>
>> <mac address='02:00:26:6b:bb:98'/>
>>
>> </interface>
>>
>> <graphics type='vnc' listen='0.0.0.0' port='5954'/>
>>
>> </devices>
>>
>> <features>
>>
>> <acpi/>
>>
>> </features>
>>
>> </domain>
>>
>>
>>
>> Thank you!
>>
>> Steve
>>
>>
>>
>>
>>
>> On Tue, Apr 8, 2014 at 10:13 AM, Ruben S. Montero <
>> rsmontero at opennebula.org> wrote:
>>
>> Hi
>>
>> I've tested with the very same environment and it should work. Could you
>> try to define the secret with the oneadmin account and:
>>
>> -c qemu:///system
>>
>> option for virsh command.
>>
>> Also to manually test it, once the vm is in failed state go to the vm
>> directory in the host and simply execute:
>>
>> virsh create -c qemu:///system deployment.0
>>
>> Cheers
>>
>> Ruben
>>
>> On Apr 8, 2014 7:05 PM, "Steve @ MyEvolve" <steve at myevolve.com> wrote:
>>
>> Hi Bill,
>>
>>
>>
>> I am using Centos 6.5 and CephX. I installed the following rpm versions
>> from Ceph on the nodes and I also followed the procedure for authentication
>> :
>>
>>
>>
>> rpm -Uvh
>> http://ceph.com/packages/ceph-extras/rpm/centos6.4/x86_64/qemu-kvm-0.12.1.2-2.415.el6.3ceph.x86_64.rpm
>> http://ceph.com/packages/ceph-extras/rpm/centos6.4/x86_64/qemu-img-0.12.1.2-2.415.el6.3ceph.x86_64.rpm
>> http://ceph.com/packages/ceph-extras/rpm/centos6.4/x86_64/qemu-guest-agent-0.12.1.2-2.415.el6.3ceph.x86_64.rpm
>> http://ceph.com/packages/ceph-extras/rpm/centos6.4/x86_64/qemu-kvm-tools-0.12.1.2-2.415.el6.3ceph.x86_64.rpm
>>
>>
>>
>> CephX
>>
>> virsh secret-define --file secret.xml; virsh secret-set-value --secret
>> 34afabcd-8ee0-45fd-a880-09df24fef603 --base64 $(cat client.libvirt.key)
>>
>>
>>
>> Also, Ruben -- how do I:
>>
>> "1.- could you try manually to add port to the <source attribute, and
>> check if the VM boots with virsh create"
>>
>>
>>
>> Thank you,
>>
>> Steve
>>
>>
>>
>>
>>
>>
>>
>> On Tue, Apr 8, 2014 at 6:20 AM, Campbell, Bill <
>> bcampbell at axcess-financial.com> wrote:
>>
>> Are you using CentOS/RHEL? Are you using CephX Authentication?
>>
>>
>>
>> There have been a few changes in how Libvirt handles the configurations
>> of Ceph images, and I'm not sure if that version of Qemu is compiled with
>> RBD support.
>>
>>
>>
>> *From:* users-bounces at lists.opennebula.org [mailto:
>> users-bounces at lists.opennebula.org] *On Behalf Of *Ruben S. Montero
>> *Sent:* Tuesday, April 08, 2014 6:00 AM
>> *To:* Steve @ MyEvolve
>> *Cc:* users at lists.opennebula.org
>> *Subject:* Re: [one-users] Images from Images (Ceph) Datastore Not
>> Deploying
>>
>>
>>
>> Well
>>
>>
>>
>> The deployment file seems to be properly generated, so given the error
>> message complaining about the missing port:
>>
>>
>>
>> 1.- could you try manually to add port to the <source attribute, and
>> check if the VM boots with virsh create
>>
>>
>>
>> 2.- I've not seen this required by ceph, here it is working with the
>> libvirt packages from the ceph guys (libvirt 0.10.2 and qemu-kvm-0.12.1),
>> Could it be a version problem?
>>
>>
>>
>> Cheers
>>
>>
>>
>>
>>
>>
>>
>> On Tue, Apr 8, 2014 at 11:09 AM, Steve @ MyEvolve <steve at myevolve.com>
>> wrote:
>>
>> Hi Ruben,
>>
>>
>>
>> Contents of /var/lib/one//datastores/125/32/deployment.0
>>
>>
>>
>> <domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0
>> '>
>>
>> <name>one-32</name>
>>
>> <cputune>
>>
>> <shares>1024</shares>
>>
>> </cputune>
>>
>> <memory>524288</memory>
>>
>> <os>
>>
>> <type arch='x86_64'>hvm</type>
>>
>> <boot dev='hd'/>
>>
>> </os>
>>
>> <devices>
>>
>> <emulator>/usr/libexec/qemu-kvm</emulator>
>>
>> <disk type='network' device='disk'>
>>
>> <source protocol='rbd' name='one/one-18-32-0'>
>>
>> <host name='cloudweb12' port='6789'/>
>>
>> <host name='cloudweb13' port='6789'/>
>>
>> <host name='cloudweb14' port='6789'/>
>>
>> </source>
>>
>> <target dev='hda'/>
>>
>> <driver name='qemu' type='raw' cache='none'/>
>>
>> </disk>
>>
>> <disk type='file' device='cdrom'>
>>
>> <source
>> file='/var/lib/one//datastores/125/32/disk.1'/>
>>
>> <target dev='hdb'/>
>>
>> <readonly/>
>>
>> <driver name='qemu' type='raw'/>
>>
>> </disk>
>>
>> <interface type='bridge'>
>>
>> <source bridge='br0'/>
>>
>> <mac address='02:00:26:6b:bb:96'/>
>>
>> </interface>
>>
>> <graphics type='vnc' listen='0.0.0.0' port='5947'/>
>>
>> </devices>
>>
>> <features>
>>
>> <acpi/>
>>
>> </features>
>>
>> </domain>
>>
>>
>>
>>
>>
>> Ceph Datastore:
>>
>> CEPH_HOST = cloudweb12:6789 cloudweb13:6789 cloudweb14:6789
>>
>>
>>
>>
>>
>> Thanks,
>>
>> Steve
>>
>>
>>
>> On Tue, Apr 8, 2014 at 1:18 AM, Ruben S. Montero <
>> rsmontero at opennebula.org> wrote:
>>
>> Hi
>>
>>
>>
>> Could you send /var/lib/one//datastores/125/32/deployment.0? Also could
>> you double check you ceph datastore configuration, specially the CEPH_HOST
>> attribute to read something like
>>
>>
>>
>> CEPH_HOST="cephmon1:6789 cephmon2:6789"
>>
>>
>>
>> Cheers
>>
>>
>>
>> Ruben
>>
>>
>>
>> On Mon, Apr 7, 2014 at 2:02 AM, Steve @ MyEvolve <steve at myevolve.com>
>> wrote:
>>
>> Hi,
>>
>>
>>
>> I've added successfully images from the marketplace to the default image
>> datastore and deployed without any issues.
>>
>>
>>
>> I've continued to add a Ceph datastore and retrieved an image from
>> marketplace to it successfully as well.
>>
>>
>>
>> However, when I try to deploy this same image, I get the following
>> failures:
>>
>>
>>
>> Sun Apr 6 16:01:41 2014 [DiM][I]: New VM state is ACTIVE.
>>
>> Sun Apr 6 16:01:41 2014 [LCM][I]: New VM state is PROLOG.
>>
>> Sun Apr 6 16:01:43 2014 [LCM][I]: New VM state is BOOT
>>
>> Sun Apr 6 16:01:43 2014 [VMM][I]: Generating deployment file:
>> /var/lib/one/vms/32/deployment.0
>>
>> Sun Apr 6 16:01:43 2014 [VMM][I]: ExitCode: 0
>>
>> Sun Apr 6 16:01:43 2014 [VMM][I]: Successfully execute network driver
>> operation: pre.
>>
>> Sun Apr 6 16:01:43 2014 [VMM][I]: Command execution fail: cat << EOT |
>> /var/tmp/one/vmm/kvm/deploy '/var/lib/one//datastores/125/32/deployment.0'
>> 'cloud4' 32 cloud4
>>
>> Sun Apr 6 16:01:43 2014 [VMM][I]: error: Failed to create domain from
>> /var/lib/one//datastores/125/32/deployment.0
>>
>> Sun Apr 6 16:01:43 2014 [VMM][I]: error: XML error: missing port for host
>>
>> Sun Apr 6 16:01:43 2014 [VMM][E]: Could not create domain from
>> /var/lib/one//datastores/125/32/deployment.0
>>
>> Sun Apr 6 16:01:43 2014 [VMM][I]: ExitCode: 255
>>
>> Sun Apr 6 16:01:43 2014 [VMM][I]: Failed to execute virtualization
>> driver operation: deploy.
>>
>> Sun Apr 6 16:01:43 2014 [VMM][E]: Error deploying virtual machine: Could
>> not create domain from /var/lib/one//datastores/125/32/deployment.0
>>
>> Sun Apr 6 16:01:43 2014 [DiM][I]: New VM state is FAILED
>>
>>
>>
>> Any suggestions/help would be greatly appreciated!
>>
>>
>>
>> Thank you,
>>
>> Steve
>>
>>
>>
>>
>>
>> _______________________________________________
>> Users mailing list
>> Users at lists.opennebula.org
>> http://lists.opennebula.org/listinfo.cgi/users-opennebula.org
>>
>>
>>
>>
>>
>> --
>>
>> --
>>
>> Ruben S. Montero, PhD
>> Project co-Lead and Chief Architect
>>
>> OpenNebula - Flexible Enterprise Cloud Made Simple
>> www.OpenNebula.org | rsmontero at opennebula.org | @OpenNebula
>>
>>
>>
>>
>>
>>
>>
>> --
>>
>> --
>>
>> Ruben S. Montero, PhD
>> Project co-Lead and Chief Architect
>>
>> OpenNebula - Flexible Enterprise Cloud Made Simple
>> www.OpenNebula.org | rsmontero at opennebula.org | @OpenNebula
>>
>>
>>
>> *NOTICE: Protect the information in this message in accordance with the
>> company's security policies. If you received this message in error,
>> immediately notify the sender and destroy all copies.*
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> *NOTICE: Protect the information in this message in accordance with the
>> company's security policies. If you received this message in error,
>> immediately notify the sender and destroy all copies.*
>>
>>
>>
>>
>>
>>
>>
>> *NOTICE: Protect the information in this message in accordance with the
>> company's security policies. If you received this message in error,
>> immediately notify the sender and destroy all copies.*
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.opennebula.org/pipermail/users-opennebula.org/attachments/20140414/d04510c0/attachment-0002.htm>
More information about the Users
mailing list