[one-users] Configuring Ceph datastore 4.6

[Steven Timm]

I have configured a Ceph datastore on one 4.6 and have gotten as
far as to get opennebula to accept the datastore.  But when we
try to do the first oneimage create into the datastore we get the
following error in oned.log :

Wed Sep 10 13:10:44 2014 [ImM][I]: Command execution fail: 
/var/lib/one/remotes/datastore/ceph/cp 
PERTX0RSSVZFUl9BQ1RJT05fREFUQT48SU1BR0U+PElEPjE4PC9JRD48VUlEPjA8L1VJRD48R0lEPjA8L0dJRD48VU5BTUU+b25lYWRtaW48L1VOQU1FPjxHTkFNRT5vbmVhZG1pbjwvR05BTUU+PE5BTUU+Y2VwaHRlc3Q8L05BTUU+PFBFUk1JU1NJT05TPjxPV05FUl9VPjE8L09XTkVSX1U+PE9XTkVSX00+MTwvT1dORVJfTT48T1dORVJfQT4wPC9PV05FUl9BPjxHUk9VUF9VPjA8L0dST1VQX1U+PEdST1VQX00+MDwvR1JPVVBfTT48R1JPVVBfQT4wPC9HUk9VUF9BPjxPVEhFUl9VPjA8L09USEVSX1U+PE9USEVSX00+MDwvT1RIRVJfTT48T1RIRVJfQT4wPC9PVEhFUl9BPjwvUEVSTUlTU0lPTlM+PFRZUEU+MDwvVFlQRT48RElTS19UWVBFPjM8L0RJU0tfVFlQRT48UEVSU0lTVEVOVD4wPC9QRVJTSVNURU5UPjxSRUdUSU1FPjE0MTAzNzI1NTU8L1JFR1RJTUU+PFNPVVJDRT48L1NPVVJDRT48UEFUSD4vY2xvdWQvaW1hZ2VzL3RpbW0vNDBnYi5xY293MjwvUEFUSD48RlNUWVBFPjwvRlNUWVBFPjxTSVpFPjQwOTYwPC9TSVpFPjxTVEFURT40PC9TVEFURT48UlVOTklOR19WTVM+MDwvUlVOTklOR19WTVM+PENMT05JTkdfT1BTPjA8L0NMT05JTkdfT1BTPjxDTE9OSU5HX0lEPi0xPC9DTE9OSU5HX0lEPjxEQVRBU1RPUkVfSUQ+MTAzPC9EQVRBU1RPUkVfSUQ+PERBVEFTVE9SRT5jZXBoX2RhdGFzdG9yZTwvREFUQVNUT1JFPjxWTVM+PC9WTVM+PENMT05FUz48L0NMT05FUz48VEVNUExBVEU+PERFU0NSSVBUSU9OPjwhW0NEQVRBW3Rlc3QgY2VwaCBnb2xkZW4gaW1nXV0+PC9ERVNDUklQVElPTj48REVWX1BSRUZJWD48IVtDREFUQVtoZF1dPjwvREVWX1BSRUZJWD48RFJJVkVSPjwhW0NEQVRBW3Fjb3cyXV0+PC9EUklWRVI+PC9URU1QTEFURT48L0lNQUdFPjxEQVRBU1RPUkU+PElEPjEwMzwvSUQ+PFVJRD4wPC9VSUQ+PEdJRD4wPC9HSUQ+PFVOQU1FPm9uZWFkbWluPC9VTkFNRT48R05BTUU+b25lYWRtaW48L0dOQU1FPjxOQU1FPmNlcGhfZGF0YXN0b3JlPC9OQU1FPjxQRVJNSVNTSU9OUz48T1dORVJfVT4xPC9PV05FUl9VPjxPV05FUl9NPjE8L09XTkVSX00+PE9XTkVSX0E+MDwvT1dORVJfQT48R1JPVVBfVT4xPC9HUk9VUF9VPjxHUk9VUF9NPjA8L0dST1VQX00+PEdST1VQX0E+MDwvR1JPVVBfQT48T1RIRVJfVT4wPC9PVEhFUl9VPjxPVEhFUl9NPjA8L09USEVSX00+PE9USEVSX0E+MDwvT1RIRVJfQT48L1BFUk1JU1NJT05TPjxEU19NQUQ+Y2VwaDwvRFNfTUFEPjxUTV9NQUQ+Y2VwaDwvVE1fTUFEPjxCQVNFX1BBVEg+L3Zhci9saWIvb25lLy9kYXRhc3RvcmVzLzEwMzwvQkFTRV9QQVRIPjxUWVBFPjA8L1RZUEU+PERJU0tfVFlQRT4zPC9ESVNLX1RZUEU+PENMVVNURVJfSUQ+LTE8L0NMVVNURVJfSUQ+PENMVVNURVI+PC9DTFVTVEVSPjxUT1RBTF9NQj42MTAyNzMyODwvVE9UQUxfTUI+PEZSRUVfTUI+NjA4NTE1NTI8L0ZSRUVfTUI+PFVTRURfTUI+MTc1Nzc2PC9VU0VEX01CPjxJTUFHRVM+PC9JTUFHRVM+PFRFTVBMQVRFPjxCQVNFX1BBVEg+PCFbQ0RBVEFbL3Zhci9saWIvb25lLy9kYXRhc3RvcmVzL11dPjwvQkFTRV9QQVRIPjxCUklER0VfTElTVD48IVtDREFUQVtvbmU0ZGV2XV0+PC9CUklER0VfTElTVD48Q0VQSF9IT1NUPjwhW0NEQVRBW3N0a2VuZGNhMDFhIHN0a2VuZGNhMDRhIHN0a2VuZGNhMDJhXV0+PC9DRVBIX0hPU1Q+PENFUEhfU0VDUkVUPjwhW0NEQVRBWy9ldGMvY2VwaC9jZXBoLmNsaWVudC5hZG1pbi5rZXlyaW5nXV0+PC9DRVBIX1NFQ1JFVD48Q0xPTkVfVEFSR0VUPjwhW0NEQVRBW1NFTEZdXT48L0NMT05FX1RBUkdFVD48REFUQVNUT1JFX0NBUEFDSVRZX0NIRUNLPjwhW0NEQVRBW3llc11dPjwvREFUQVNUT1JFX0NBUEFDSVRZX0NIRUNLPjxESVNLX1RZUEU+PCFbQ0RBVEFbUkJEXV0+PC9ESVNLX1RZUEU+PERTX01BRD48IVtDREFUQVtjZXBoXV0+PC9EU19NQUQ+PExOX1RBUkdFVD48IVtDREFUQVtOT05FXV0+PC9MTl9UQVJHRVQ+PFBPT0xfTkFNRT48IVtDREFUQVtvbmVdXT48L1BPT0xfTkFNRT48U1RBR0lOR19ESVI+PCFbQ0RBVEFbL3Zhci9saWIvb25lL2NlcGgtdG1wXV0+PC9TVEFHSU5HX0RJUj48VE1fTUFEPjwhW0NEQVRBW2NlcGhdXT48L1RNX01BRD48L1RFTVBMQVRFPjwvREFUQVNUT1JFPjwvRFNfRFJJVkVSX0FDVElPTl9EQVRBPg== 
18
Wed Sep 10 13:10:44 2014 [ImM][I]: cp: Copying local image 
/cloud/images/timm/40gb.qcow2 to the image repository
Wed Sep 10 13:10:44 2014 [ImM][E]: cp: Command "    set -e
Wed Sep 10 13:10:44 2014 [ImM][I]:
Wed Sep 10 13:10:44 2014 [ImM][I]: if [ "" = "2" ]; then
Wed Sep 10 13:10:44 2014 [ImM][I]: FORMAT=$(qemu-img info 
/var/lib/one/ceph-tmp/68bec7e25cb73c98a31a48117022d72c | grep "^file 
format:" | awk '{print }')
Wed Sep 10 13:10:44 2014 [ImM][I]:
Wed Sep 10 13:10:44 2014 [ImM][I]: if [ "$FORMAT" != "raw" ]; then
Wed Sep 10 13:10:44 2014 [ImM][I]: qemu-img convert -O raw 
/var/lib/one/ceph-tmp/68bec7e25cb73c98a31a48117022d72c 
/var/lib/one/ceph-tmp/68bec7e25cb73c98a31a48117022d72c.raw
Wed Sep 10 13:10:44 2014 [ImM][I]: mv 
/var/lib/one/ceph-tmp/68bec7e25cb73c98a31a48117022d72c.raw 
/var/lib/one/ceph-tmp/68bec7e25cb73c98a31a48117022d72c
Wed Sep 10 13:10:44 2014 [ImM][I]: fi
Wed Sep 10 13:10:44 2014 [ImM][I]:
Wed Sep 10 13:10:44 2014 [ImM][I]: rbd import --format 2 
/var/lib/one/ceph-tmp/68bec7e25cb73c98a31a48117022d72c one/one-18
Wed Sep 10 13:10:44 2014 [ImM][I]: else
Wed Sep 10 13:10:44 2014 [ImM][I]: qemu-img convert 
/var/lib/one/ceph-tmp/68bec7e25cb73c98a31a48117022d72c rbd:one/one-18
Wed Sep 10 13:10:44 2014 [ImM][I]: fi
Wed Sep 10 13:10:44 2014 [ImM][I]:
Wed Sep 10 13:10:44 2014 [ImM][I]: # remove original
Wed Sep 10 13:10:44 2014 [ImM][I]: rm -f 
/var/lib/one/ceph-tmp/68bec7e25cb73c98a31a48117022d72c" failed: Unknown 
protocol 'rbd:one/one-18'
Wed Sep 10 13:10:44 2014 [ImM][E]: Error registering one/one-18 in one4dev
Wed Sep 10 13:10:44 2014 [ImM][I]: ExitCode: 1
Wed Sep 10 13:10:44 2014 [ImM][E]: Error copying image in the datastore: 
Error registering one/one-18 in one4dev




---

Clear we are afailed to register the rbd, but not clear why.. any
or clues on why we failed are helpful.
Several places in the docs refer to libvirt 1.x.  Has anyone
made this work on RHEL6/Centos 6 (we are running the newer kernel so
we do have the rbd.o kernel module available, and rbd import/export
works from the command line.)

Steve Timm


------------------------------------------------------------------
Steven C. Timm, Ph.D  (630) 840-8525
timm at fnal.gov  http://home.fnal.gov/~timm/
Fermilab Scientific Computing Division, Scientific Computing Services Quad.
Grid and Cloud Services Dept., Associate Dept. Head for Cloud Computing