[one-users] Images from Images (Ceph) Datastore Not Deploying
Campbell, Bill
bcampbell at axcess-financial.com
Fri Apr 11 07:57:23 PDT 2014
That's a very good question. Just to validate, which version of
OpenNebula are you using?
From: Steve @ MyEvolve [mailto:steve at myevolve.com]
Sent: Thursday, April 10, 2014 9:33 PM
To: Campbell, Bill
Cc: Ruben S. Montero; users at lists.opennebula.org
Subject: Re: [one-users] Images from Images (Ceph) Datastore Not Deploying
Also, why is ONE not adding the <auth> to the deployment.0 file?
On Thu, Apr 10, 2014 at 6:32 PM, Steve @ MyEvolve <steve at myevolve.com
<mailto:steve at myevolve.com> > wrote:
Hi Bill & Ruben,
1.) I have confirmed that CEPH_HOST, CEPH_USER, and CEPH_SECRET are
properly set in my datastore.
2.) Bill you bring up a good point about why:
<auth username='libvirt'>
<secret type='ceph' uuid='34afabcd-8ee0-45fd-a880-09df24fef603'/>
</auth>
is not added to my deployment.0 file. I added it to the end just before
</domain>.
3.) I have tried to run the following command to manually test, but it
doesn't work for me. What am I doing wrong?
[root at cloud2 54]# virsh create -c qemu:///system deployment.0
error: unexpected data 'qemu:///system'
[root at cloud2 54]#
Thank you so much, I know we have to be close!
Steve
On Wed, Apr 9, 2014 at 11:56 AM, Campbell, Bill
<bcampbell at axcess-financial.com <mailto:bcampbell at axcess-financial.com> >
wrote:
A couple of things:
* You shouldn't need the port, that should be generated
appropriately
* I'm not seeing your secret definition within the deployment.0
file. That must exist if using the newer versions of Libvirt w/CephX
authentication (Libvirt assumes if you are specifying hosts that a secret
should be there). You should have an entry in that file like so (right
after the </source>):
<auth username='libvirt'>
<secret type='ceph'
uuid='ebc9c744-016d-40af-ae73-621f558398f0'/>
</auth>
* You've defined the secret/user on the Datastore within
OpenNebula using the appropriate configuration attributes CEPH_HOST,
CEPH_USER, and CEPH_SECRET?
o And also defined the appropriate secret in Libvirt within each
hypervisor in your configuration?
Outside of verifying the above, I'm not sure what else can be attempted.
You could try disabling CephX authentication, removing the CEPH_HOST
configuration attributes, and copying the /etc/ceph/ceph.conf and admin
keyring to each hypervisor and see if you can connect that way (however
newer versions of libvirt will probably prevent that from happening).
From: Steve @ MyEvolve [mailto:steve at myevolve.com
<mailto:steve at myevolve.com> ]
Sent: Wednesday, April 09, 2014 2:44 PM
To: Ruben S. Montero
Cc: users at lists.opennebula.org <mailto:users at lists.opennebula.org> ; Bill
Campbell
Subject: Re: [one-users] Images from Images (Ceph) Datastore Not Deploying
Any suggestions on this?
On Tue, Apr 8, 2014 at 7:37 PM, Steve @ MyEvolve <steve at myevolve.com
<mailto:steve at myevolve.com> > wrote:
Hi Ruben,
So I've been trying to experiment a bit. I started over. Re-imported
ttylinux into ceph datastore. Created new template and new vm and I
logged in as oneadmin and defined the secret as instructed on all hosts.
In creating the new VM, the process failed as before.
I tried as you suggested:
[oneadmin at cloud2 54]$ virsh create -c qemu:///system deployment.0
error: unexpected data 'qemu:///system'
But as you can see I got an error message I'm not sure what to do with.
Also, I have a new log that looks different -- seems adding the port to
the CEPH_HOST was the distinguishing factor.
Tue Apr 8 19:30:04 2014 [DiM][I]: New VM state is ACTIVE.
Tue Apr 8 19:30:04 2014 [LCM][I]: New VM state is PROLOG.
Tue Apr 8 19:30:09 2014 [LCM][I]: New VM state is BOOT
Tue Apr 8 19:30:10 2014 [VMM][I]: Generating deployment file:
/var/lib/one/vms/54/deployment.0
Tue Apr 8 19:30:10 2014 [VMM][I]: ExitCode: 0
Tue Apr 8 19:30:10 2014 [VMM][I]: Successfully execute network driver
operation: pre.
Tue Apr 8 19:30:10 2014 [VMM][I]: Command execution fail: cat << EOT |
/var/tmp/one/vmm/kvm/deploy '/var/lib/one//datastores/125/54/deployment.0'
'cloud2' 54 cloud2
Tue Apr 8 19:30:10 2014 [VMM][I]: error: Failed to create domain from
/var/lib/one//datastores/125/54/deployment.0
Tue Apr 8 19:30:10 2014 [VMM][I]: error: internal error process exited
while connecting to monitor: qemu-kvm: -drive
file=rbd:one/one-28-54-0:auth_supported=none:mon_host=cloudweb12\:6789,if=
none,id=drive-ide0-0-0,format=raw,cache=none: error connecting
Tue Apr 8 19:30:10 2014 [VMM][I]: qemu-kvm: -drive
file=rbd:one/one-28-54-0:auth_supported=none:mon_host=cloudweb12\:6789,if=
none,id=drive-ide0-0-0,format=raw,cache=none: could not open disk image
rbd:one/one-28-54-0:auth_supported=none:mon_host=cloudweb12\:6789:
Operation not supported
Tue Apr 8 19:30:10 2014 [VMM][I]:
Tue Apr 8 19:30:10 2014 [VMM][E]: Could not create domain from
/var/lib/one//datastores/125/54/deployment.0
Tue Apr 8 19:30:10 2014 [VMM][I]: ExitCode: 255
Tue Apr 8 19:30:10 2014 [VMM][I]: Failed to execute virtualization driver
operation: deploy.
Tue Apr 8 19:30:10 2014 [VMM][E]: Error deploying virtual machine: Could
not create domain from /var/lib/one//datastores/125/54/deployment.0
Tue Apr 8 19:30:10 2014 [DiM][I]: New VM state is FAILED
And also here is deployment.0:
<domain type='kvm'
xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>
<name>one-54</name>
<cputune>
<shares>1024</shares>
</cputune>
<memory>524288</memory>
<os>
<type arch='x86_64'>hvm</type>
<boot dev='hd'/>
</os>
<devices>
<emulator>/usr/libexec/qemu-kvm</emulator>
<disk type='network' device='disk'>
<source protocol='rbd' name='one/one-28-54-0'>
<host name='cloudweb12' port='6789'/>
</source>
<target dev='hda'/>
<driver name='qemu' type='raw' cache='none'/>
</disk>
<disk type='file' device='cdrom'>
<source
file='/var/lib/one//datastores/125/54/disk.1'/>
<target dev='hdb'/>
<readonly/>
<driver name='qemu' type='raw'/>
</disk>
<interface type='bridge'>
<source bridge='br0'/>
<mac address='02:00:26:6b:bb:98'/>
</interface>
<graphics type='vnc' listen='0.0.0.0' port='5954'/>
</devices>
<features>
<acpi/>
</features>
</domain>
Thank you!
Steve
On Tue, Apr 8, 2014 at 10:13 AM, Ruben S. Montero
<rsmontero at opennebula.org <mailto:rsmontero at opennebula.org> > wrote:
Hi
I've tested with the very same environment and it should work. Could you
try to define the secret with the oneadmin account and:
-c qemu:///system
option for virsh command.
Also to manually test it, once the vm is in failed state go to the vm
directory in the host and simply execute:
virsh create -c qemu:///system deployment.0
Cheers
Ruben
On Apr 8, 2014 7:05 PM, "Steve @ MyEvolve" <steve at myevolve.com
<mailto:steve at myevolve.com> > wrote:
Hi Bill,
I am using Centos 6.5 and CephX. I installed the following rpm versions
from Ceph on the nodes and I also followed the procedure for
authentication :
rpm -Uvh
http://ceph.com/packages/ceph-extras/rpm/centos6.4/x86_64/qemu-kvm-0.12.1.
2-2.415.el6.3ceph.x86_64.rpm
http://ceph.com/packages/ceph-extras/rpm/centos6.4/x86_64/qemu-img-0.12.1.
2-2.415.el6.3ceph.x86_64.rpm
http://ceph.com/packages/ceph-extras/rpm/centos6.4/x86_64/qemu-guest-agent
-0.12.1.2-2.415.el6.3ceph.x86_64.rpm
http://ceph.com/packages/ceph-extras/rpm/centos6.4/x86_64/qemu-kvm-tools-0
.12.1.2-2.415.el6.3ceph.x86_64.rpm
CephX
virsh secret-define --file secret.xml; virsh secret-set-value --secret
34afabcd-8ee0-45fd-a880-09df24fef603 --base64 $(cat client.libvirt.key)
Also, Ruben -- how do I:
"1.- could you try manually to add port to the <source attribute, and
check if the VM boots with virsh create"
Thank you,
Steve
On Tue, Apr 8, 2014 at 6:20 AM, Campbell, Bill
<bcampbell at axcess-financial.com <mailto:bcampbell at axcess-financial.com> >
wrote:
Are you using CentOS/RHEL? Are you using CephX Authentication?
There have been a few changes in how Libvirt handles the configurations of
Ceph images, and I'm not sure if that version of Qemu is compiled with RBD
support.
From: users-bounces at lists.opennebula.org
<mailto:users-bounces at lists.opennebula.org>
[mailto:users-bounces at lists.opennebula.org
<mailto:users-bounces at lists.opennebula.org> ] On Behalf Of Ruben S.
Montero
Sent: Tuesday, April 08, 2014 6:00 AM
To: Steve @ MyEvolve
Cc: users at lists.opennebula.org <mailto:users at lists.opennebula.org>
Subject: Re: [one-users] Images from Images (Ceph) Datastore Not Deploying
Well
The deployment file seems to be properly generated, so given the error
message complaining about the missing port:
1.- could you try manually to add port to the <source attribute, and check
if the VM boots with virsh create
2.- I've not seen this required by ceph, here it is working with the
libvirt packages from the ceph guys (libvirt 0.10.2 and qemu-kvm-0.12.1),
Could it be a version problem?
Cheers
On Tue, Apr 8, 2014 at 11:09 AM, Steve @ MyEvolve <steve at myevolve.com
<mailto:steve at myevolve.com> > wrote:
Hi Ruben,
Contents of /var/lib/one//datastores/125/32/deployment.0
<domain type='kvm'
xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>
<name>one-32</name>
<cputune>
<shares>1024</shares>
</cputune>
<memory>524288</memory>
<os>
<type arch='x86_64'>hvm</type>
<boot dev='hd'/>
</os>
<devices>
<emulator>/usr/libexec/qemu-kvm</emulator>
<disk type='network' device='disk'>
<source protocol='rbd' name='one/one-18-32-0'>
<host name='cloudweb12' port='6789'/>
<host name='cloudweb13' port='6789'/>
<host name='cloudweb14' port='6789'/>
</source>
<target dev='hda'/>
<driver name='qemu' type='raw' cache='none'/>
</disk>
<disk type='file' device='cdrom'>
<source
file='/var/lib/one//datastores/125/32/disk.1'/>
<target dev='hdb'/>
<readonly/>
<driver name='qemu' type='raw'/>
</disk>
<interface type='bridge'>
<source bridge='br0'/>
<mac address='02:00:26:6b:bb:96'/>
</interface>
<graphics type='vnc' listen='0.0.0.0' port='5947'/>
</devices>
<features>
<acpi/>
</features>
</domain>
Ceph Datastore:
CEPH_HOST = cloudweb12:6789 cloudweb13:6789 cloudweb14:6789
Thanks,
Steve
On Tue, Apr 8, 2014 at 1:18 AM, Ruben S. Montero <rsmontero at opennebula.org
<mailto:rsmontero at opennebula.org> > wrote:
Hi
Could you send /var/lib/one//datastores/125/32/deployment.0? Also could
you double check you ceph datastore configuration, specially the CEPH_HOST
attribute to read something like
CEPH_HOST="cephmon1:6789 cephmon2:6789"
Cheers
Ruben
On Mon, Apr 7, 2014 at 2:02 AM, Steve @ MyEvolve <steve at myevolve.com
<mailto:steve at myevolve.com> > wrote:
Hi,
I've added successfully images from the marketplace to the default image
datastore and deployed without any issues.
I've continued to add a Ceph datastore and retrieved an image from
marketplace to it successfully as well.
However, when I try to deploy this same image, I get the following
failures:
Sun Apr 6 16:01:41 2014 [DiM][I]: New VM state is ACTIVE.
Sun Apr 6 16:01:41 2014 [LCM][I]: New VM state is PROLOG.
Sun Apr 6 16:01:43 2014 [LCM][I]: New VM state is BOOT
Sun Apr 6 16:01:43 2014 [VMM][I]: Generating deployment file:
/var/lib/one/vms/32/deployment.0
Sun Apr 6 16:01:43 2014 [VMM][I]: ExitCode: 0
Sun Apr 6 16:01:43 2014 [VMM][I]: Successfully execute network driver
operation: pre.
Sun Apr 6 16:01:43 2014 [VMM][I]: Command execution fail: cat << EOT |
/var/tmp/one/vmm/kvm/deploy '/var/lib/one//datastores/125/32/deployment.0'
'cloud4' 32 cloud4
Sun Apr 6 16:01:43 2014 [VMM][I]: error: Failed to create domain from
/var/lib/one//datastores/125/32/deployment.0
Sun Apr 6 16:01:43 2014 [VMM][I]: error: XML error: missing port for host
Sun Apr 6 16:01:43 2014 [VMM][E]: Could not create domain from
/var/lib/one//datastores/125/32/deployment.0
Sun Apr 6 16:01:43 2014 [VMM][I]: ExitCode: 255
Sun Apr 6 16:01:43 2014 [VMM][I]: Failed to execute virtualization driver
operation: deploy.
Sun Apr 6 16:01:43 2014 [VMM][E]: Error deploying virtual machine: Could
not create domain from /var/lib/one//datastores/125/32/deployment.0
Sun Apr 6 16:01:43 2014 [DiM][I]: New VM state is FAILED
Any suggestions/help would be greatly appreciated!
Thank you,
Steve
_______________________________________________
Users mailing list
Users at lists.opennebula.org <mailto:Users at lists.opennebula.org>
http://lists.opennebula.org/listinfo.cgi/users-opennebula.org
--
--
Ruben S. Montero, PhD
Project co-Lead and Chief Architect
OpenNebula - Flexible Enterprise Cloud Made Simple
www.OpenNebula.org <http://www.OpenNebula.org> | rsmontero at opennebula.org
<mailto:rsmontero at opennebula.org> | @OpenNebula
--
--
Ruben S. Montero, PhD
Project co-Lead and Chief Architect
OpenNebula - Flexible Enterprise Cloud Made Simple
www.OpenNebula.org <http://www.OpenNebula.org> | rsmontero at opennebula.org
<mailto:rsmontero at opennebula.org> | @OpenNebula
NOTICE: Protect the information in this message in accordance with the
company's security policies. If you received this message in error,
immediately notify the sender and destroy all copies.
NOTICE: Protect the information in this message in accordance with the
company's security policies. If you received this message in error,
immediately notify the sender and destroy all copies.
NOTICE: Protect the information in this message in accordance with the company's security policies. If you received this message in error, immediately notify the sender and destroy all copies.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.opennebula.org/pipermail/users-opennebula.org/attachments/20140411/602502f9/attachment-0002.htm>
More information about the Users
mailing list