[one-users] Images from Images (Ceph) Datastore Not Deploying
Steve @ MyEvolve
steve at myevolve.com
Fri Apr 11 14:44:35 PDT 2014
4.4.1
On Fri, Apr 11, 2014 at 7:57 AM, Campbell, Bill <
bcampbell at axcess-financial.com> wrote:
> That's a very good question. Just to validate, which version of
> OpenNebula are you using?
>
>
>
> *From:* Steve @ MyEvolve [mailto:steve at myevolve.com]
> *Sent:* Thursday, April 10, 2014 9:33 PM
> *To:* Campbell, Bill
> *Cc:* Ruben S. Montero; users at lists.opennebula.org
>
> *Subject:* Re: [one-users] Images from Images (Ceph) Datastore Not
> Deploying
>
>
>
> Also, why is ONE not adding the <auth> to the deployment.0 file?
>
>
>
>
>
> On Thu, Apr 10, 2014 at 6:32 PM, Steve @ MyEvolve <steve at myevolve.com>
> wrote:
>
> Hi Bill & Ruben,
>
>
>
> 1.) I have confirmed that CEPH_HOST, CEPH_USER, and CEPH_SECRET are
> properly set in my datastore.
>
>
>
> 2.) Bill you bring up a good point about why:
>
>
>
> <auth username='libvirt'>
>
> <secret type='ceph' uuid='34afabcd-8ee0-45fd-a880-09df24fef603'/>
>
> </auth>
>
> is not added to my deployment.0 file. I added it to the end just before
> </domain>.
>
>
>
> 3.) I have tried to run the following command to manually test, but it
> doesn't work for me. What am I doing wrong?
>
> [root at cloud2 54]# virsh create -c qemu:///system deployment.0
>
> error: unexpected data 'qemu:///system'
>
> [root at cloud2 54]#
>
>
>
> Thank you so much, I know we have to be close!
>
> Steve
>
>
>
> On Wed, Apr 9, 2014 at 11:56 AM, Campbell, Bill <
> bcampbell at axcess-financial.com> wrote:
>
> A couple of things:
>
> · You shouldn't need the port, that should be generated
> appropriately
>
> · I'm not seeing your secret definition within the deployment.0
> file. That must exist if using the newer versions of Libvirt w/CephX
> authentication (Libvirt assumes if you are specifying hosts that a secret
> should be there). You should have an entry in that file like so (right
> after the </source>):
>
>
>
> <auth username='libvirt'>
>
> <secret type='ceph'
> uuid='ebc9c744-016d-40af-ae73-621f558398f0'/>
>
> </auth>
>
>
>
> · You've defined the secret/user on the Datastore within
> OpenNebula using the appropriate configuration attributes CEPH_HOST,
> CEPH_USER, and CEPH_SECRET?
>
> o And also defined the appropriate secret in Libvirt within each
> hypervisor in your configuration?
>
>
>
> Outside of verifying the above, I'm not sure what else can be attempted.
> You could try disabling CephX authentication, removing the CEPH_HOST
> configuration attributes, and copying the /etc/ceph/ceph.conf and admin
> keyring to each hypervisor and see if you can connect that way (however
> newer versions of libvirt will probably prevent that from happening).
>
>
>
>
>
> *From:* Steve @ MyEvolve [mailto:steve at myevolve.com]
> *Sent:* Wednesday, April 09, 2014 2:44 PM
> *To:* Ruben S. Montero
> *Cc:* users at lists.opennebula.org; Bill Campbell
>
>
> *Subject:* Re: [one-users] Images from Images (Ceph) Datastore Not
> Deploying
>
>
>
> Any suggestions on this?
>
>
>
> On Tue, Apr 8, 2014 at 7:37 PM, Steve @ MyEvolve <steve at myevolve.com>
> wrote:
>
> Hi Ruben,
>
>
>
> So I've been trying to experiment a bit. I started over. Re-imported
> ttylinux into ceph datastore. Created new template and new vm and I logged
> in as oneadmin and defined the secret as instructed on all hosts.
>
>
>
> In creating the new VM, the process failed as before.
>
>
>
> I tried as you suggested:
>
>
>
> [oneadmin at cloud2 54]$ virsh create -c qemu:///system deployment.0
>
> error: unexpected data 'qemu:///system'
>
> But as you can see I got an error message I'm not sure what to do with.
>
>
>
> Also, I have a new log that looks different -- seems adding the port to
> the CEPH_HOST was the distinguishing factor.
>
>
>
> Tue Apr 8 19:30:04 2014 [DiM][I]: New VM state is ACTIVE.
> Tue Apr 8 19:30:04 2014 [LCM][I]: New VM state is PROLOG.
> Tue Apr 8 19:30:09 2014 [LCM][I]: New VM state is BOOT
> Tue Apr 8 19:30:10 2014 [VMM][I]: Generating deployment file:
> /var/lib/one/vms/54/deployment.0
> Tue Apr 8 19:30:10 2014 [VMM][I]: ExitCode: 0
> Tue Apr 8 19:30:10 2014 [VMM][I]: Successfully execute network driver
> operation: pre.
> Tue Apr 8 19:30:10 2014 [VMM][I]: Command execution fail: cat << EOT |
> /var/tmp/one/vmm/kvm/deploy '/var/lib/one//datastores/125/54/deployment.0'
> 'cloud2' 54 cloud2
> Tue Apr 8 19:30:10 2014 [VMM][I]: error: Failed to create domain from
> /var/lib/one//datastores/125/54/deployment.0
> Tue Apr 8 19:30:10 2014 [VMM][I]: error: internal error process exited
> while connecting to monitor: qemu-kvm: -drive
> file=rbd:one/one-28-54-0:auth_supported=none:mon_host=cloudweb12\:6789,if=none,id=drive-ide0-0-0,format=raw,cache=none:
> error connecting
> Tue Apr 8 19:30:10 2014 [VMM][I]: qemu-kvm: -drive
> file=rbd:one/one-28-54-0:auth_supported=none:mon_host=cloudweb12\:6789,if=none,id=drive-ide0-0-0,format=raw,cache=none:
> could not open disk image
> rbd:one/one-28-54-0:auth_supported=none:mon_host=cloudweb12\:6789:
> Operation not supported
> Tue Apr 8 19:30:10 2014 [VMM][I]:
> Tue Apr 8 19:30:10 2014 [VMM][E]: Could not create domain from
> /var/lib/one//datastores/125/54/deployment.0
> Tue Apr 8 19:30:10 2014 [VMM][I]: ExitCode: 255
> Tue Apr 8 19:30:10 2014 [VMM][I]: Failed to execute virtualization driver
> operation: deploy.
> Tue Apr 8 19:30:10 2014 [VMM][E]: Error deploying virtual machine: Could
> not create domain from /var/lib/one//datastores/125/54/deployment.0
> Tue Apr 8 19:30:10 2014 [DiM][I]: New VM state is FAILED
>
>
>
> And also here is deployment.0:
>
> <domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0
> '>
>
> <name>one-54</name>
>
> <cputune>
>
> <shares>1024</shares>
>
> </cputune>
>
> <memory>524288</memory>
>
> <os>
>
> <type arch='x86_64'>hvm</type>
>
> <boot dev='hd'/>
>
> </os>
>
> <devices>
>
> <emulator>/usr/libexec/qemu-kvm</emulator>
>
> <disk type='network' device='disk'>
>
> <source protocol='rbd' name='one/one-28-54-0'>
>
> <host name='cloudweb12' port='6789'/>
>
> </source>
>
> <target dev='hda'/>
>
> <driver name='qemu' type='raw' cache='none'/>
>
> </disk>
>
> <disk type='file' device='cdrom'>
>
> <source
> file='/var/lib/one//datastores/125/54/disk.1'/>
>
> <target dev='hdb'/>
>
> <readonly/>
>
> <driver name='qemu' type='raw'/>
>
> </disk>
>
> <interface type='bridge'>
>
> <source bridge='br0'/>
>
> <mac address='02:00:26:6b:bb:98'/>
>
> </interface>
>
> <graphics type='vnc' listen='0.0.0.0' port='5954'/>
>
> </devices>
>
> <features>
>
> <acpi/>
>
> </features>
>
> </domain>
>
>
>
> Thank you!
>
> Steve
>
>
>
>
>
> On Tue, Apr 8, 2014 at 10:13 AM, Ruben S. Montero <
> rsmontero at opennebula.org> wrote:
>
> Hi
>
> I've tested with the very same environment and it should work. Could you
> try to define the secret with the oneadmin account and:
>
> -c qemu:///system
>
> option for virsh command.
>
> Also to manually test it, once the vm is in failed state go to the vm
> directory in the host and simply execute:
>
> virsh create -c qemu:///system deployment.0
>
> Cheers
>
> Ruben
>
> On Apr 8, 2014 7:05 PM, "Steve @ MyEvolve" <steve at myevolve.com> wrote:
>
> Hi Bill,
>
>
>
> I am using Centos 6.5 and CephX. I installed the following rpm versions
> from Ceph on the nodes and I also followed the procedure for authentication
> :
>
>
>
> rpm -Uvh
> http://ceph.com/packages/ceph-extras/rpm/centos6.4/x86_64/qemu-kvm-0.12.1.2-2.415.el6.3ceph.x86_64.rpm
> http://ceph.com/packages/ceph-extras/rpm/centos6.4/x86_64/qemu-img-0.12.1.2-2.415.el6.3ceph.x86_64.rpm
> http://ceph.com/packages/ceph-extras/rpm/centos6.4/x86_64/qemu-guest-agent-0.12.1.2-2.415.el6.3ceph.x86_64.rpm
> http://ceph.com/packages/ceph-extras/rpm/centos6.4/x86_64/qemu-kvm-tools-0.12.1.2-2.415.el6.3ceph.x86_64.rpm
>
>
>
> CephX
>
> virsh secret-define --file secret.xml; virsh secret-set-value --secret
> 34afabcd-8ee0-45fd-a880-09df24fef603 --base64 $(cat client.libvirt.key)
>
>
>
> Also, Ruben -- how do I:
>
> "1.- could you try manually to add port to the <source attribute, and
> check if the VM boots with virsh create"
>
>
>
> Thank you,
>
> Steve
>
>
>
>
>
>
>
> On Tue, Apr 8, 2014 at 6:20 AM, Campbell, Bill <
> bcampbell at axcess-financial.com> wrote:
>
> Are you using CentOS/RHEL? Are you using CephX Authentication?
>
>
>
> There have been a few changes in how Libvirt handles the configurations of
> Ceph images, and I'm not sure if that version of Qemu is compiled with RBD
> support.
>
>
>
> *From:* users-bounces at lists.opennebula.org [mailto:
> users-bounces at lists.opennebula.org] *On Behalf Of *Ruben S. Montero
> *Sent:* Tuesday, April 08, 2014 6:00 AM
> *To:* Steve @ MyEvolve
> *Cc:* users at lists.opennebula.org
> *Subject:* Re: [one-users] Images from Images (Ceph) Datastore Not
> Deploying
>
>
>
> Well
>
>
>
> The deployment file seems to be properly generated, so given the error
> message complaining about the missing port:
>
>
>
> 1.- could you try manually to add port to the <source attribute, and check
> if the VM boots with virsh create
>
>
>
> 2.- I've not seen this required by ceph, here it is working with the
> libvirt packages from the ceph guys (libvirt 0.10.2 and qemu-kvm-0.12.1),
> Could it be a version problem?
>
>
>
> Cheers
>
>
>
>
>
>
>
> On Tue, Apr 8, 2014 at 11:09 AM, Steve @ MyEvolve <steve at myevolve.com>
> wrote:
>
> Hi Ruben,
>
>
>
> Contents of /var/lib/one//datastores/125/32/deployment.0
>
>
>
> <domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0
> '>
>
> <name>one-32</name>
>
> <cputune>
>
> <shares>1024</shares>
>
> </cputune>
>
> <memory>524288</memory>
>
> <os>
>
> <type arch='x86_64'>hvm</type>
>
> <boot dev='hd'/>
>
> </os>
>
> <devices>
>
> <emulator>/usr/libexec/qemu-kvm</emulator>
>
> <disk type='network' device='disk'>
>
> <source protocol='rbd' name='one/one-18-32-0'>
>
> <host name='cloudweb12' port='6789'/>
>
> <host name='cloudweb13' port='6789'/>
>
> <host name='cloudweb14' port='6789'/>
>
> </source>
>
> <target dev='hda'/>
>
> <driver name='qemu' type='raw' cache='none'/>
>
> </disk>
>
> <disk type='file' device='cdrom'>
>
> <source
> file='/var/lib/one//datastores/125/32/disk.1'/>
>
> <target dev='hdb'/>
>
> <readonly/>
>
> <driver name='qemu' type='raw'/>
>
> </disk>
>
> <interface type='bridge'>
>
> <source bridge='br0'/>
>
> <mac address='02:00:26:6b:bb:96'/>
>
> </interface>
>
> <graphics type='vnc' listen='0.0.0.0' port='5947'/>
>
> </devices>
>
> <features>
>
> <acpi/>
>
> </features>
>
> </domain>
>
>
>
>
>
> Ceph Datastore:
>
> CEPH_HOST = cloudweb12:6789 cloudweb13:6789 cloudweb14:6789
>
>
>
>
>
> Thanks,
>
> Steve
>
>
>
> On Tue, Apr 8, 2014 at 1:18 AM, Ruben S. Montero <rsmontero at opennebula.org>
> wrote:
>
> Hi
>
>
>
> Could you send /var/lib/one//datastores/125/32/deployment.0? Also could
> you double check you ceph datastore configuration, specially the CEPH_HOST
> attribute to read something like
>
>
>
> CEPH_HOST="cephmon1:6789 cephmon2:6789"
>
>
>
> Cheers
>
>
>
> Ruben
>
>
>
> On Mon, Apr 7, 2014 at 2:02 AM, Steve @ MyEvolve <steve at myevolve.com>
> wrote:
>
> Hi,
>
>
>
> I've added successfully images from the marketplace to the default image
> datastore and deployed without any issues.
>
>
>
> I've continued to add a Ceph datastore and retrieved an image from
> marketplace to it successfully as well.
>
>
>
> However, when I try to deploy this same image, I get the following
> failures:
>
>
>
> Sun Apr 6 16:01:41 2014 [DiM][I]: New VM state is ACTIVE.
>
> Sun Apr 6 16:01:41 2014 [LCM][I]: New VM state is PROLOG.
>
> Sun Apr 6 16:01:43 2014 [LCM][I]: New VM state is BOOT
>
> Sun Apr 6 16:01:43 2014 [VMM][I]: Generating deployment file:
> /var/lib/one/vms/32/deployment.0
>
> Sun Apr 6 16:01:43 2014 [VMM][I]: ExitCode: 0
>
> Sun Apr 6 16:01:43 2014 [VMM][I]: Successfully execute network driver
> operation: pre.
>
> Sun Apr 6 16:01:43 2014 [VMM][I]: Command execution fail: cat << EOT |
> /var/tmp/one/vmm/kvm/deploy '/var/lib/one//datastores/125/32/deployment.0'
> 'cloud4' 32 cloud4
>
> Sun Apr 6 16:01:43 2014 [VMM][I]: error: Failed to create domain from
> /var/lib/one//datastores/125/32/deployment.0
>
> Sun Apr 6 16:01:43 2014 [VMM][I]: error: XML error: missing port for host
>
> Sun Apr 6 16:01:43 2014 [VMM][E]: Could not create domain from
> /var/lib/one//datastores/125/32/deployment.0
>
> Sun Apr 6 16:01:43 2014 [VMM][I]: ExitCode: 255
>
> Sun Apr 6 16:01:43 2014 [VMM][I]: Failed to execute virtualization driver
> operation: deploy.
>
> Sun Apr 6 16:01:43 2014 [VMM][E]: Error deploying virtual machine: Could
> not create domain from /var/lib/one//datastores/125/32/deployment.0
>
> Sun Apr 6 16:01:43 2014 [DiM][I]: New VM state is FAILED
>
>
>
> Any suggestions/help would be greatly appreciated!
>
>
>
> Thank you,
>
> Steve
>
>
>
>
>
> _______________________________________________
> Users mailing list
> Users at lists.opennebula.org
> http://lists.opennebula.org/listinfo.cgi/users-opennebula.org
>
>
>
>
>
> --
>
> --
>
> Ruben S. Montero, PhD
> Project co-Lead and Chief Architect
>
> OpenNebula - Flexible Enterprise Cloud Made Simple
> www.OpenNebula.org | rsmontero at opennebula.org | @OpenNebula
>
>
>
>
>
>
>
> --
>
> --
>
> Ruben S. Montero, PhD
> Project co-Lead and Chief Architect
>
> OpenNebula - Flexible Enterprise Cloud Made Simple
> www.OpenNebula.org | rsmontero at opennebula.org | @OpenNebula
>
>
>
> *NOTICE: Protect the information in this message in accordance with the
> company's security policies. If you received this message in error,
> immediately notify the sender and destroy all copies.*
>
>
>
>
>
>
>
>
>
>
>
> *NOTICE: Protect the information in this message in accordance with the
> company's security policies. If you received this message in error,
> immediately notify the sender and destroy all copies.*
>
>
>
>
>
>
>
> *NOTICE: Protect the information in this message in accordance with the
> company's security policies. If you received this message in error,
> immediately notify the sender and destroy all copies.*
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.opennebula.org/pipermail/users-opennebula.org/attachments/20140411/8b337335/attachment-0002.htm>
More information about the Users
mailing list