<div dir="ltr">Hi Steven,<div><br></div><div>Sorry, but this email fell through the cracks.</div><div><br></div><div>Did you ever manage to launch a Ceph VM? Or you are still stumped by this issue here?</div><div><br></div><div>if it's still not working for you, can you send us the output of "onedatastore show -x <id>" where id is the ceph's ds id.</div><div><br></div><div>Regards,<br>Jaime</div></div><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Sep 24, 2014 at 4:49 PM, Steven Timm <span dir="ltr"><<a href="mailto:timm@fnal.gov" target="_blank">timm@fnal.gov</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Now have upgraded to opennebula 4.8.0 and still struggling<br>
with successfully launching a Ceph VM. We have worked out<br>
all permissions issues on the client host and in the Ceph datastore<br>
and have gotten to the point where opennebula can deploy<br>
the VM from RBD and the virtual machine starts.. but we get "Geom Error"<br>
on the console of the VM and that is all. Has anyone seen this error before and have any idea how to deal with it? I presume that<br>
it means that the virtual machine can not even find the boot sector of<br>
the disk that it sees as /dev/vda but I can't find any information<br>
on this error anywhere. Any help is appreciated.<br>
<br>
We are trying to replace an old san-gfs file store with Ceph<br>
but need better success than that if this is going to work.<br>
<br>
Steve Timm<br>
<br>
<br>
On Wed, 10 Sep 2014, Steven Timm wrote:<br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
The first and most obvious problem below was that we were running an<br>
old version of qemu-img and qemu-kvm that ships with RHEL6/Centos6/SL6<br>
that doesn't support the "rbd" format. We were able to find<br>
a modified version that the Ceph people had back-ported and now<br>
we can import an image into the datastore and have<br>
gotten as far to getting a deployment.0 written on the hypervisor.<br>
It can't contact Ceph as yet, is getting connection refused but<br>
we think that is an authentication issue.<br>
<br>
<a href="http://ceph.com/packages/qemu-kvm/redhat/x86_64/" target="_blank">http://ceph.com/packages/qemu-<u></u>kvm/redhat/x86_64/</a><br>
is where these packages for 6.2 live.<br>
<br>
Hopefully this all gets easier pretty soon now that RedHat has bought Ceph<br>
and all the right packages will be in RHEL7. Or will they be only proprietarily available in Redhat Enterprise Virtualization? Has anyone tried yet?<br>
<br>
Steve<br>
<br>
<br>
<br>
On Wed, 10 Sep 2014, Steven Timm wrote:<br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
<br>
I have configured a Ceph datastore on one 4.6 and have gotten as<br>
far as to get opennebula to accept the datastore. But when we<br>
try to do the first oneimage create into the datastore we get the<br>
following error in oned.log :<br>
<br>
Wed Sep 10 13:10:44 2014 [ImM][I]: Command execution fail:<br>
/var/lib/one/remotes/<u></u>datastore/ceph/cp<br>
PERTX0RSSVZFUl9BQ1RJT05fREFUQT<u></u>48SU1BR0U+<u></u>PElEPjE4PC9JRD48VUlEPjA8L1VJRD<u></u>48R0lEPjA8L0dJRD48VU5BTUU+<u></u>b25lYWRtaW48L1VOQU1FPjxHTkFNRT<u></u>5vbmVhZG1pbjwvR05BTUU+PE5BTUU+<u></u>Y2VwaHRlc3Q8L05BTUU+<u></u>PFBFUk1JU1NJT05TPjxPV05FUl9VPj<u></u>E8L09XTkVSX1U+PE9XTkVSX00+<u></u>MTwvT1dORVJfTT48T1dORVJfQT4wPC<u></u>9PV05FUl9BPjxHUk9VUF9VPjA8L0dS<u></u>T1VQX1U+PEdST1VQX00+<u></u>MDwvR1JPVVBfTT48R1JPVVBfQT4wPC<u></u>9HUk9VUF9BPjxPVEhFUl9VPjA8L09U<u></u>SEVSX1U+PE9USEVSX00+<u></u>MDwvT1RIRVJfTT48T1RIRVJfQT4wPC<u></u>9PVEhFUl9BPjwvUEVSTUlTU0lPTlM+<u></u>PFRZUEU+<u></u>MDwvVFlQRT48RElTS19UWVBFPjM8L0<u></u>RJU0tfVFlQRT48UEVSU0lTVEVOVD4w<u></u>PC9QRVJTSVNURU5UPjxSRUdUSU1FPj<u></u>E0MTAzNzI1NTU8L1JFR1RJTUU+<u></u>PFNPVVJDRT48L1NPVVJDRT48UEFUSD<u></u>4vY2xvdWQvaW1hZ2VzL3RpbW0vNDBn<u></u>Yi5xY293MjwvUEFUSD48RlNUWVBFPj<u></u>wvRlNUWVBFPjxTSVpFPjQwOTYwPC9T<u></u>SVpFPjxTVEFURT40PC9TVEFURT48Ul<u></u>VOTklOR19WTVM+<u></u>MDwvUlVOTklOR19WTVM+<u></u>PENMT05JTkdfT1BTPjA8L0NMT05JTk<u></u>dfT1BTPjxDTE9OSU5HX0lEPi0xPC9D<u></u>TE9OSU5HX0lEPjxEQVRBU1RPUkVfSU<u></u>Q+MTAzPC9EQVRBU1RPUkVfSUQ+<u></u>PERBVEFTVE9SRT5jZXBoX2RhdGFzdG<u></u>9yZTwvREFUQVNUT1JFPjxWTVM+<u></u>PC9WTVM+<u></u>PENMT05FUz48L0NMT05FUz48VEVNUE<u></u>xBVEU+PERFU0NSSV<br>
BUS<br>
U9OPjwhW0NEQVRBW3Rlc3QgY2VwaCB<u></u>nb2xkZW4gaW1nXV0+<u></u>PC9ERVNDUklQVElPTj48REVWX1BSRU<u></u>ZJWD48IVtDREFUQVtoZF1dPjwvREVW<u></u>X1BSRUZJWD48RFJJVkVSPjwhW0NEQV<u></u>RBW3Fjb3cyXV0+PC9EUklWRVI+<u></u>PC9URU1QTEFURT48L0lNQUdFPjxEQV<u></u>RBU1RPUkU+PElEPjEwMzwvSUQ+<u></u>PFVJRD4wPC9VSUQ+<u></u>PEdJRD4wPC9HSUQ+<u></u>PFVOQU1FPm9uZWFkbWluPC9VTkFNRT<u></u>48R05BTUU+<u></u>b25lYWRtaW48L0dOQU1FPjxOQU1FPm<u></u>NlcGhfZGF0YXN0b3JlPC9OQU1FPjxQ<u></u>RVJNSVNTSU9OUz48T1dORVJfVT4xPC<u></u>9PV05FUl9VPjxPV05FUl9NPjE8L09X<u></u>TkVSX00+PE9XTkVSX0E+<u></u>MDwvT1dORVJfQT48R1JPVVBfVT4xPC<u></u>9HUk9VUF9VPjxHUk9VUF9NPjA8L0dS<u></u>T1VQX00+PEdST1VQX0E+<u></u>MDwvR1JPVVBfQT48T1RIRVJfVT4wPC<u></u>9PVEhFUl9VPjxPVEhFUl9NPjA8L09U<u></u>SEVSX00+PE9USEVSX0E+<u></u>MDwvT1RIRVJfQT48L1BFUk1JU1NJT0<u></u>5TPjxEU19NQUQ+<u></u>Y2VwaDwvRFNfTUFEPjxUTV9NQUQ+<u></u>Y2VwaDwvVE1fTUFEPjxCQVNFX1BBVE<u></u>g+<u></u>L3Zhci9saWIvb25lLy9kYXRhc3Rvcm<u></u>VzLzEwMzwvQkFTRV9QQVRIPjxUWVBF<u></u>PjA8L1RZUEU+<u></u>PERJU0tfVFlQRT4zPC9ESVNLX1RZUE<u></u>U+PENMVVNURVJfSUQ+<u></u>LTE8L0NMVVNURVJfSUQ+<u></u>PENMVVNURVI+<u></u>PC9DTFVTVEVSPjxUT1RBTF9NQj42MT<u></u>AyNzMyODwvVE9UQUxfTUI+<u></u>PEZSRUVfTUI+<u></u>NjA4NTE1NTI8L0ZSRUVfTUI+<u></u>PFVTRURfTUI+<u></u>MTc1Nzc2PC9VU0VEX01CPjxJTUFHRV<u></u>M+PC9JTUF<br>
HRV<br>
M+PFRFTVBMQVRFPjxCQVNFX1BBVEg+<u></u>PCFbQ0RBVEFbL3Zhci9saWIvb25lLy<u></u>9kYXRhc3RvcmVzL11dPjwvQkFTRV9Q<u></u>QVRIPjxCUklER0VfTElTVD48IVtDRE<u></u>FUQVtvbmU0ZGV2XV0+<u></u>PC9CUklER0VfTElTVD48Q0VQSF9IT1<u></u>NUPjwhW0NEQVRBW3N0a2VuZGNhMDFh<u></u>IHN0a2VuZGNhMDRhIHN0a2VuZGNhMD<u></u>JhXV0+PC9DRVBIX0hPU1Q+<u></u>PENFUEhfU0VDUkVUPjwhW0NEQVRBWy<u></u>9ldGMvY2VwaC9jZXBoLmNsaWVudC5h<u></u>ZG1pbi5rZXlyaW5nXV0+<u></u>PC9DRVBIX1NFQ1JFVD48Q0xPTkVfVE<u></u>FSR0VUPjwhW0NEQVRBW1NFTEZdXT48<u></u>L0NMT05FX1RBUkdFVD48REFUQVNUT1<u></u>JFX0NBUEFDSVRZX0NIRUNLPjwhW0NE<u></u>QVRBW3llc11dPjwvREFUQVNUT1JFX0<u></u>NBUEFDSVRZX0NIRUNLPjxESVNLX1RZ<u></u>UEU+PCFbQ0RBVEFbUkJEXV0+<u></u>PC9ESVNLX1RZUEU+<u></u>PERTX01BRD48IVtDREFUQVtjZXBoXV<u></u>0+PC9EU19NQUQ+<u></u>PExOX1RBUkdFVD48IVtDREFUQVtOT0<u></u>5FXV0+PC9MTl9UQVJHRVQ+<u></u>PFBPT0xfTkFNRT48IVtDREFUQVtvbm<u></u>VdXT48L1BPT0xfTkFNRT48U1RBR0lO<u></u>R19ESVI+<u></u>PCFbQ0RBVEFbL3Zhci9saWIvb25lL2<u></u>NlcGgtdG1wXV0+<u></u>PC9TVEFHSU5HX0RJUj48VE1fTUFEPj<u></u>whW0NEQVRBW2NlcGhdXT48L1RNX01B<u></u>RD48L1RFTVBMQVRFPjwvREFUQVNUT1<u></u>JFPjwvRFNfRFJJVkVSX0FDVElPTl9E<u></u>QVRBPg==<br>
18<br>
Wed Sep 10 13:10:44 2014 [ImM][I]: cp: Copying local image<br>
/cloud/images/timm/40gb.qcow2 to the image repository<br>
Wed Sep 10 13:10:44 2014 [ImM][E]: cp: Command " set -e<br>
Wed Sep 10 13:10:44 2014 [ImM][I]:<br>
Wed Sep 10 13:10:44 2014 [ImM][I]: if [ "" = "2" ]; then<br>
Wed Sep 10 13:10:44 2014 [ImM][I]: FORMAT=$(qemu-img info<br>
/var/lib/one/ceph-tmp/<u></u>68bec7e25cb73c98a31a48117022d7<u></u>2c | grep "^file<br>
format:" | awk '{print }')<br>
Wed Sep 10 13:10:44 2014 [ImM][I]:<br>
Wed Sep 10 13:10:44 2014 [ImM][I]: if [ "$FORMAT" != "raw" ]; then<br>
Wed Sep 10 13:10:44 2014 [ImM][I]: qemu-img convert -O raw<br>
/var/lib/one/ceph-tmp/<u></u>68bec7e25cb73c98a31a48117022d7<u></u>2c<br>
/var/lib/one/ceph-tmp/<u></u>68bec7e25cb73c98a31a48117022d7<u></u>2c.raw<br>
Wed Sep 10 13:10:44 2014 [ImM][I]: mv<br>
/var/lib/one/ceph-tmp/<u></u>68bec7e25cb73c98a31a48117022d7<u></u>2c.raw<br>
/var/lib/one/ceph-tmp/<u></u>68bec7e25cb73c98a31a48117022d7<u></u>2c<br>
Wed Sep 10 13:10:44 2014 [ImM][I]: fi<br>
Wed Sep 10 13:10:44 2014 [ImM][I]:<br>
Wed Sep 10 13:10:44 2014 [ImM][I]: rbd import --format 2<br>
/var/lib/one/ceph-tmp/<u></u>68bec7e25cb73c98a31a48117022d7<u></u>2c one/one-18<br>
Wed Sep 10 13:10:44 2014 [ImM][I]: else<br>
Wed Sep 10 13:10:44 2014 [ImM][I]: qemu-img convert<br>
/var/lib/one/ceph-tmp/<u></u>68bec7e25cb73c98a31a48117022d7<u></u>2c rbd:one/one-18<br>
Wed Sep 10 13:10:44 2014 [ImM][I]: fi<br>
Wed Sep 10 13:10:44 2014 [ImM][I]:<br>
Wed Sep 10 13:10:44 2014 [ImM][I]: # remove original<br>
Wed Sep 10 13:10:44 2014 [ImM][I]: rm -f<br>
/var/lib/one/ceph-tmp/<u></u>68bec7e25cb73c98a31a48117022d7<u></u>2c" failed: Unknown<br>
protocol 'rbd:one/one-18'<br>
Wed Sep 10 13:10:44 2014 [ImM][E]: Error registering one/one-18 in one4dev<br>
Wed Sep 10 13:10:44 2014 [ImM][I]: ExitCode: 1<br>
Wed Sep 10 13:10:44 2014 [ImM][E]: Error copying image in the datastore:<br>
Error registering one/one-18 in one4dev<br>
<br>
<br>
<br>
<br>
---<br>
<br>
Clear we are afailed to register the rbd, but not clear why.. any<br>
or clues on why we failed are helpful.<br>
Several places in the docs refer to libvirt 1.x. Has anyone<br>
made this work on RHEL6/Centos 6 (we are running the newer kernel so<br>
we do have the rbd.o kernel module available, and rbd import/export<br>
works from the command line.)<br>
<br>
Steve Timm<br>
<br>
<br>
------------------------------<u></u>------------------------------<u></u>------<br>
Steven C. Timm, Ph.D <a href="tel:%28630%29%20840-8525" value="+16308408525" target="_blank">(630) 840-8525</a><br>
<a href="mailto:timm@fnal.gov" target="_blank">timm@fnal.gov</a> <a href="http://home.fnal.gov/~timm/" target="_blank">http://home.fnal.gov/~timm/</a><br>
Fermilab Scientific Computing Division, Scientific Computing Services<br>
Quad.<br>
Grid and Cloud Services Dept., Associate Dept. Head for Cloud Computing<br>
<br>
</blockquote>
<br>
------------------------------<u></u>------------------------------<u></u>------<br>
Steven C. Timm, Ph.D <a href="tel:%28630%29%20840-8525" value="+16308408525" target="_blank">(630) 840-8525</a><br>
<a href="mailto:timm@fnal.gov" target="_blank">timm@fnal.gov</a> <a href="http://home.fnal.gov/~timm/" target="_blank">http://home.fnal.gov/~timm/</a><br>
Fermilab Scientific Computing Division, Scientific Computing Services Quad.<br>
Grid and Cloud Services Dept., Associate Dept. Head for Cloud Computing<br>
<br>
</blockquote>
<br>
------------------------------<u></u>------------------------------<u></u>------<br>
Steven C. Timm, Ph.D <a href="tel:%28630%29%20840-8525" value="+16308408525" target="_blank">(630) 840-8525</a><br>
<a href="mailto:timm@fnal.gov" target="_blank">timm@fnal.gov</a> <a href="http://home.fnal.gov/~timm/" target="_blank">http://home.fnal.gov/~timm/</a><br>
Fermilab Scientific Computing Division, Scientific Computing Services Quad.<br>
Grid and Cloud Services Dept., Associate Dept. Head for Cloud Computing<br>
______________________________<u></u>_________________<br>
Users mailing list<br>
<a href="mailto:Users@lists.opennebula.org" target="_blank">Users@lists.opennebula.org</a><br>
<a href="http://lists.opennebula.org/listinfo.cgi/users-opennebula.org" target="_blank">http://lists.opennebula.org/<u></u>listinfo.cgi/users-opennebula.<u></u>org</a><br>
</blockquote></div><br></div><br clear="all"><div><br></div>-- <br><div class="gmail_signature"><div dir="ltr"><div>OpenNebula - Flexible Enterprise Cloud Made Simple<br></div><div>--</div><div>Jaime Melis</div><div>Senior Infrastructure Architect at OpenNebula Systems (formerly C12G Labs)</div><div>jmelis@opennebula.systems | @OpenNebula</div><div>--</div><div>Confidentiality Warning: The information contained in this e-mail and any accompanying documents, unless otherwise expressly indicated, is confidential and privileged, and is intended solely for the person and/or entity to whom it is addressed (i.e. those identified in the "To" and "cc" box). They are the property of OpenNebula.Systems S.L.. Unauthorized distribution, review, use, disclosure, or copying of this communication, or any part thereof, is strictly prohibited and may be unlawful. If you have received this e-mail in error, please notify us immediately by e-mail at abuse@opennebula.systems and delete the e-mail and attachments and any copy from your system. OpenNebula's thanks you for your cooperation.</div></div></div>