<div dir="ltr">Oh ok, that makes sense.<div><br></div><div>Thanks,</div><div>Jaime</div></div><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Nov 19, 2014 at 4:59 PM, Steven Timm <span dir="ltr"><<a href="mailto:timm@fnal.gov" target="_blank">timm@fnal.gov</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi Jaime..yes we did figure it out eventually. Turned out the image I had<br>
stored in CEPH was not a good image. Once I loaded another image into ceph and booted that, I was fine. the version of libvirt I was running only<br>
supported RAW images over ceph, not qcow2, and my first raw image had been<br>
truncated in some bad way.<br>
<br>
Steve Timm<div><div class="h5"><br>
<br>
<br>
On Wed, 19 Nov 2014, Jaime Melis wrote:<br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Hi Steven,<br>
Sorry, but this email fell through the cracks.<br>
<br>
Did you ever manage to launch a Ceph VM? Or you are still stumped by this issue here?<br>
<br>
if it's still not working for you, can you send us the output of "onedatastore show -x <id>" where<br>
id is the ceph's ds id.<br>
<br>
Regards,<br>
Jaime<br>
<br>
On Wed, Sep 24, 2014 at 4:49 PM, Steven Timm <<a href="mailto:timm@fnal.gov" target="_blank">timm@fnal.gov</a>> wrote:<br>
Now have upgraded to opennebula 4.8.0 and still struggling<br>
with successfully launching a Ceph VM. We have worked out<br>
all permissions issues on the client host and in the Ceph datastore<br>
and have gotten to the point where opennebula can deploy<br>
the VM from RBD and the virtual machine starts.. but we get "Geom Error"<br>
on the console of the VM and that is all. Has anyone seen this error before and have<br>
any idea how to deal with it? I presume that<br>
it means that the virtual machine can not even find the boot sector of<br>
the disk that it sees as /dev/vda but I can't find any information<br>
on this error anywhere. Any help is appreciated.<br>
<br>
We are trying to replace an old san-gfs file store with Ceph<br>
but need better success than that if this is going to work.<br>
<br>
Steve Timm<br>
<br>
<br>
On Wed, 10 Sep 2014, Steven Timm wrote:<br>
<br>
The first and most obvious problem below was that we were running an<br>
old version of qemu-img and qemu-kvm that ships with RHEL6/Centos6/SL6<br>
that doesn't support the "rbd" format. We were able to find<br>
a modified version that the Ceph people had back-ported and now<br>
we can import an image into the datastore and have<br>
gotten as far to getting a deployment.0 written on the hypervisor.<br>
It can't contact Ceph as yet, is getting connection refused but<br>
we think that is an authentication issue.<br>
<br>
<a href="http://ceph.com/packages/qemu-kvm/redhat/x86_64/" target="_blank">http://ceph.com/packages/qemu-<u></u>kvm/redhat/x86_64/</a><br>
is where these packages for 6.2 live.<br>
<br>
Hopefully this all gets easier pretty soon now that RedHat has bought Ceph<br>
and all the right packages will be in RHEL7. Or will they be only<br>
proprietarily available in Redhat Enterprise Virtualization? Has anyone<br>
tried yet?<br>
<br>
Steve<br>
<br>
<br>
<br>
On Wed, 10 Sep 2014, Steven Timm wrote:<br>
<br>
<br>
<br>
I have configured a Ceph datastore on one 4.6 and have gotten<br>
as<br>
far as to get opennebula to accept the datastore. But when we<br>
try to do the first oneimage create into the datastore we get<br>
the<br>
following error in oned.log :<br>
<br>
Wed Sep 10 13:10:44 2014 [ImM][I]: Command execution fail:<br>
/var/lib/one/remotes/<u></u>datastore/ceph/cp<br>
<u></u>PERTX0RSSVZFUl9BQ1RJT05fREFUQT<u></u>48SU1BR0U+<u></u>PElEPjE4PC9JRD48VUlEPjA8L1VJRD<u></u>48R0lEPjA8L0dJRD48VU5BTUU+b<br>
25lYWRtaW48L1VOQU1FPjxHTkFNRT5<u></u>vbmVhZG1pbjwvR05BTUU+PE5BTUU+<u></u>Y2VwaHRlc3Q8L05BTUU+<u></u>PFBFUk1JU1NJT05TPjx<br>
PV05FUl9VPjE8L09XTkVSX1U+<u></u>PE9XTkVSX00+<u></u>MTwvT1dORVJfTT48T1dORVJfQT4wPC<u></u>9PV05FUl9BPjxHUk9VUF9VPjA8L0dS<u></u>T<br>
1VQX1U+PEdST1VQX00+<u></u>MDwvR1JPVVBfTT48R1JPVVBfQT4wPC<u></u>9HUk9VUF9BPjxPVEhFUl9VPjA8L09U<u></u>SEVSX1U+PE9USEVSX00<br>
+<u></u>MDwvT1RIRVJfTT48T1RIRVJfQT4wPC<u></u>9PVEhFUl9BPjwvUEVSTUlTU0lPTlM+<u></u>PFRZUEU+<u></u>MDwvVFlQRT48RElTS19UWVBFPjM8L<br>
0RJU0tfVFlQRT48UEVSU0lTVEVOVD4<u></u>wPC9QRVJTSVNURU5UPjxSRUdUSU1FP<u></u>jE0MTAzNzI1NTU8L1JFR1RJTUU+<u></u>PFNPVVJDRT4<br>
8L1NPVVJDRT48UEFUSD4vY2xvdWQva<u></u>W1hZ2VzL3RpbW0vNDBnYi5xY293Mjw<u></u>vUEFUSD48RlNUWVBFPjwvRlNUWVBFP<u></u>jxTSVpFP<br>
jQwOTYwPC9TSVpFPjxTVEFURT40PC9<u></u>TVEFURT48UlVOTklOR19WTVM+<u></u>MDwvUlVOTklOR19WTVM+<u></u>PENMT05JTkdfT1BTPjA8L0N<br>
MT05JTkdfT1BTPjxDTE9OSU5HX0lEP<u></u>i0xPC9DTE9OSU5HX0lEPjxEQVRBU1R<u></u>PUkVfSUQ+<u></u>MTAzPC9EQVRBU1RPUkVfSUQ+PERBV<br>
EFTVE9SRT5jZXBoX2RhdGFzdG9yZTw<u></u>vREFUQVNUT1JFPjxWTVM+PC9WTVM+<u></u>PENMT05FUz48L0NMT05FUz48VEVNUE<u></u>xBVEU+PER<br>
FU0NSSV<br>
BUS<br>
<u></u>U9OPjwhW0NEQVRBW3Rlc3QgY2VwaCB<u></u>nb2xkZW4gaW1nXV0+<u></u>PC9ERVNDUklQVElPTj48REVWX1BSRU<u></u>ZJWD48IVtDREFUQVtoZF<br>
1dPjwvREVWX1BSRUZJWD48RFJJVkVS<u></u>PjwhW0NEQVRBW3Fjb3cyXV0+<u></u>PC9EUklWRVI+<u></u>PC9URU1QTEFURT48L0lNQUdFPjxEQV<u></u>RB<br>
U1RPUkU+PElEPjEwMzwvSUQ+<u></u>PFVJRD4wPC9VSUQ+<u></u>PEdJRD4wPC9HSUQ+<u></u>PFVOQU1FPm9uZWFkbWluPC9VTkFNRT<u></u>48R05BTUU+b2<br>
5lYWRtaW48L0dOQU1FPjxOQU1FPmNl<u></u>cGhfZGF0YXN0b3JlPC9OQU1FPjxQRV<u></u>JNSVNTSU9OUz48T1dORVJfVT4xPC9P<u></u>V05FUl9V<br>
PjxPV05FUl9NPjE8L09XTkVSX00+<u></u>PE9XTkVSX0E+<u></u>MDwvT1dORVJfQT48R1JPVVBfVT4xPC<u></u>9HUk9VUF9VPjxHUk9VUF9NPjA8L0<br>
dST1VQX00+PEdST1VQX0E+<u></u>MDwvR1JPVVBfQT48T1RIRVJfVT4wPC<u></u>9PVEhFUl9VPjxPVEhFUl9NPjA8L09U<u></u>SEVSX00+PE9USEVS<br>
X0E+<u></u>MDwvT1RIRVJfQT48L1BFUk1JU1NJT0<u></u>5TPjxEU19NQUQ+<u></u>Y2VwaDwvRFNfTUFEPjxUTV9NQUQ+<u></u>Y2VwaDwvVE1fTUFEPjxCQV<br>
NFX1BBVEg+<u></u>L3Zhci9saWIvb25lLy9kYXRhc3Rvcm<u></u>VzLzEwMzwvQkFTRV9QQVRIPjxUWVBF<u></u>PjA8L1RZUEU+PERJU0tfVFlQRT4z<br>
PC9ESVNLX1RZUEU+<u></u>PENMVVNURVJfSUQ+<u></u>LTE8L0NMVVNURVJfSUQ+<u></u>PENMVVNURVI+<u></u>PC9DTFVTVEVSPjxUT1RBTF9NQj42MT<u></u>AyNz<br>
MyODwvVE9UQUxfTUI+PEZSRUVfTUI+<u></u>NjA4NTE1NTI8L0ZSRUVfTUI+<u></u>PFVTRURfTUI+<u></u>MTc1Nzc2PC9VU0VEX01CPjxJTUFHRV<u></u>M+<br>
PC9JTUF<br>
HRV<br>
M+<u></u>PFRFTVBMQVRFPjxCQVNFX1BBVEg+<u></u>PCFbQ0RBVEFbL3Zhci9saWIvb25lLy<u></u>9kYXRhc3RvcmVzL11dPjwvQkFTRV9Q<u></u>QVRIPjx<br>
CUklER0VfTElTVD48IVtDREFUQVtvb<u></u>mU0ZGV2XV0+<u></u>PC9CUklER0VfTElTVD48Q0VQSF9IT1<u></u>NUPjwhW0NEQVRBW3N0a2VuZGNhM<br>
DFhIHN0a2VuZGNhMDRhIHN0a2VuZGN<u></u>hMDJhXV0+PC9DRVBIX0hPU1Q+<u></u>PENFUEhfU0VDUkVUPjwhW0NEQVRBWy<u></u>9ldGMvY2VwaC9<br>
jZXBoLmNsaWVudC5hZG1pbi5rZXlya<u></u>W5nXV0+<u></u>PC9DRVBIX1NFQ1JFVD48Q0xPTkVfVE<u></u>FSR0VUPjwhW0NEQVRBW1NFTEZdXT48<u></u>L<br>
0NMT05FX1RBUkdFVD48REFUQVNUT1J<u></u>FX0NBUEFDSVRZX0NIRUNLPjwhW0NEQ<u></u>VRBW3llc11dPjwvREFUQVNUT1JFX0N<u></u>BUEFDSVR<br>
ZX0NIRUNLPjxESVNLX1RZUEU+<u></u>PCFbQ0RBVEFbUkJEXV0+<u></u>PC9ESVNLX1RZUEU+<u></u>PERTX01BRD48IVtDREFUQVtjZXBoXV<u></u>0+PC9EU<br>
19NQUQ+<u></u>PExOX1RBUkdFVD48IVtDREFUQVtOT0<u></u>5FXV0+PC9MTl9UQVJHRVQ+<u></u>PFBPT0xfTkFNRT48IVtDREFUQVtvbm<u></u>VdXT48L1B<br>
PT0xfTkFNRT48U1RBR0lOR19ESVI+<u></u>PCFbQ0RBVEFbL3Zhci9saWIvb25lL2<u></u>NlcGgtdG1wXV0+<u></u>PC9TVEFHSU5HX0RJUj48VE1fT<br>
UFEPjwhW0NEQVRBW2NlcGhdXT48L1R<u></u>NX01BRD48L1RFTVBMQVRFPjwvREFUQ<u></u>VNUT1JFPjwvRFNfRFJJVkVSX0FDVEl<u></u>PTl9EQVR<br>
BPg==<br>
18<br>
Wed Sep 10 13:10:44 2014 [ImM][I]: cp: Copying local image<br>
/cloud/images/timm/40gb.qcow2 to the image repository<br>
Wed Sep 10 13:10:44 2014 [ImM][E]: cp: Command " set -e<br>
Wed Sep 10 13:10:44 2014 [ImM][I]:<br>
Wed Sep 10 13:10:44 2014 [ImM][I]: if [ "" = "2" ]; then<br>
Wed Sep 10 13:10:44 2014 [ImM][I]: FORMAT=$(qemu-img info<br>
/var/lib/one/ceph-tmp/<u></u>68bec7e25cb73c98a31a48117022d7<u></u>2c | grep<br>
"^file<br>
format:" | awk '{print }')<br>
Wed Sep 10 13:10:44 2014 [ImM][I]:<br>
Wed Sep 10 13:10:44 2014 [ImM][I]: if [ "$FORMAT" != "raw" ];<br>
then<br>
Wed Sep 10 13:10:44 2014 [ImM][I]: qemu-img convert -O raw<br>
/var/lib/one/ceph-tmp/<u></u>68bec7e25cb73c98a31a48117022d7<u></u>2c<br>
/var/lib/one/ceph-tmp/<u></u>68bec7e25cb73c98a31a48117022d7<u></u>2c.raw<br>
Wed Sep 10 13:10:44 2014 [ImM][I]: mv<br>
/var/lib/one/ceph-tmp/<u></u>68bec7e25cb73c98a31a48117022d7<u></u>2c.raw<br>
/var/lib/one/ceph-tmp/<u></u>68bec7e25cb73c98a31a48117022d7<u></u>2c<br>
Wed Sep 10 13:10:44 2014 [ImM][I]: fi<br>
Wed Sep 10 13:10:44 2014 [ImM][I]:<br>
Wed Sep 10 13:10:44 2014 [ImM][I]: rbd import --format 2<br>
/var/lib/one/ceph-tmp/<u></u>68bec7e25cb73c98a31a48117022d7<u></u>2c<br>
one/one-18<br>
Wed Sep 10 13:10:44 2014 [ImM][I]: else<br>
Wed Sep 10 13:10:44 2014 [ImM][I]: qemu-img convert<br>
/var/lib/one/ceph-tmp/<u></u>68bec7e25cb73c98a31a48117022d7<u></u>2c<br>
rbd:one/one-18<br>
Wed Sep 10 13:10:44 2014 [ImM][I]: fi<br>
Wed Sep 10 13:10:44 2014 [ImM][I]:<br>
Wed Sep 10 13:10:44 2014 [ImM][I]: # remove original<br>
Wed Sep 10 13:10:44 2014 [ImM][I]: rm -f<br>
/var/lib/one/ceph-tmp/<u></u>68bec7e25cb73c98a31a48117022d7<u></u>2c"<br>
failed: Unknown<br>
protocol 'rbd:one/one-18'<br>
Wed Sep 10 13:10:44 2014 [ImM][E]: Error registering<br>
one/one-18 in one4dev<br>
Wed Sep 10 13:10:44 2014 [ImM][I]: ExitCode: 1<br>
Wed Sep 10 13:10:44 2014 [ImM][E]: Error copying image in the<br>
datastore:<br>
Error registering one/one-18 in one4dev<br>
<br>
<br>
<br>
<br>
---<br>
<br>
Clear we are afailed to register the rbd, but not clear why..<br>
any<br>
or clues on why we failed are helpful.<br>
Several places in the docs refer to libvirt 1.x. Has anyone<br>
made this work on RHEL6/Centos 6 (we are running the newer<br>
kernel so<br>
we do have the rbd.o kernel module available, and rbd<br>
import/export<br>
works from the command line.)<br>
<br>
Steve Timm<br>
<br>
<br>
-----------------------------<u></u>------------------------------<u></u>-------<br>
Steven C. Timm, Ph.D <a href="tel:%28630%29%20840-8525" value="+16308408525" target="_blank">(630) 840-8525</a><br>
<a href="mailto:timm@fnal.gov" target="_blank">timm@fnal.gov</a> <a href="http://home.fnal.gov/~timm/" target="_blank">http://home.fnal.gov/~timm/</a><br>
Fermilab Scientific Computing Division, Scientific Computing<br>
Services<br>
Quad.<br>
Grid and Cloud Services Dept., Associate Dept. Head for Cloud<br>
Computing<br>
<br>
<br>
------------------------------<u></u>------------------------------<u></u>------<br>
Steven C. Timm, Ph.D <a href="tel:%28630%29%20840-8525" value="+16308408525" target="_blank">(630) 840-8525</a><br>
<a href="mailto:timm@fnal.gov" target="_blank">timm@fnal.gov</a> <a href="http://home.fnal.gov/~timm/" target="_blank">http://home.fnal.gov/~timm/</a><br>
Fermilab Scientific Computing Division, Scientific Computing Services Quad.<br>
Grid and Cloud Services Dept., Associate Dept. Head for Cloud Computing<br>
<br>
<br>
------------------------------<u></u>------------------------------<u></u>------<br>
Steven C. Timm, Ph.D <a href="tel:%28630%29%20840-8525" value="+16308408525" target="_blank">(630) 840-8525</a><br>
<a href="mailto:timm@fnal.gov" target="_blank">timm@fnal.gov</a> <a href="http://home.fnal.gov/~timm/" target="_blank">http://home.fnal.gov/~timm/</a><br>
Fermilab Scientific Computing Division, Scientific Computing Services Quad.<br>
Grid and Cloud Services Dept., Associate Dept. Head for Cloud Computing<br>
______________________________<u></u>_________________<br>
Users mailing list<br>
<a href="mailto:Users@lists.opennebula.org" target="_blank">Users@lists.opennebula.org</a><br>
<a href="http://lists.opennebula.org/listinfo.cgi/users-opennebula.org" target="_blank">http://lists.opennebula.org/<u></u>listinfo.cgi/users-opennebula.<u></u>org</a><br>
<br>
<br>
<br>
<br>
--<br>
OpenNebula - Flexible Enterprise Cloud Made Simple<br>
--<br>
Jaime Melis<br>
Senior Infrastructure Architect at OpenNebula Systems (formerly C12G Labs)<br>
jmelis@opennebula.systems | @OpenNebula<br>
--<br>
Confidentiality Warning: The information contained in this e-mail and any accompanying documents,<br>
unless otherwise expressly indicated, is confidential and privileged, and is intended solely for<br>
the person and/or entity to whom it is addressed (i.e. those identified in the "To" and "cc" box).<br>
They are the property of OpenNebula.Systems S.L.. Unauthorized distribution, review, use,<br>
disclosure, or copying of this communication, or any part thereof, is strictly prohibited and may<br>
be unlawful. If you have received this e-mail in error, please notify us immediately by e-mail at<br>
abuse@opennebula.systems and delete the e-mail and attachments and any copy from your system.<br>
OpenNebula's thanks you for your cooperation.<br>
<br>
<br>
</blockquote>
<br>
------------------------------<u></u>------------------------------<u></u>------<br>
Steven C. Timm, Ph.D <a href="tel:%28630%29%20840-8525" value="+16308408525" target="_blank">(630) 840-8525</a><br>
<a href="mailto:timm@fnal.gov" target="_blank">timm@fnal.gov</a> <a href="http://home.fnal.gov/~timm/" target="_blank">http://home.fnal.gov/~timm/</a><br></div></div>
Office: Wilson Hall room 804<br>
Fermilab Scientific Computing Division,<br>
Currently transitioning from:<br>
Scientific Computing Services Quadrant<span class=""><br>
Grid and Cloud Services Dept., Associate Dept. Head for Cloud Computing<br>
<br></span>
To:<br>
Scientific Computing Facilities Quadrant.,<br>
Experimental Computing Facilities Dept.,<br>
Project Lead for Virtual Facility Project.<br>
<br>
</blockquote></div><br><br clear="all"><div><br></div>-- <br><div class="gmail_signature"><div dir="ltr"><div>Jaime Melis<br>Project Engineer<br>OpenNebula - Flexible Enterprise Cloud Made Simple<br><a href="http://www.OpenNebula.org" target="_blank">www.OpenNebula.org</a> | <a href="mailto:jmelis@opennebula.org" target="_blank">jmelis@opennebula.org</a></div></div></div>
</div>