[one-users] Ceph datastore monitoring in OpenNebula

[Javier Fontan]

CEPH_USER is used when generating the libvirt/kvm deployment file but
not for DS monitoring:

* Deployment file generation:
https://github.com/OpenNebula/one/blob/one-4.6/src/vmm/LibVirtDriverKVM.cc#L461
* Monitoring: https://github.com/OpenNebula/one/blob/one-4.6/src/datastore_mad/remotes/ceph/monitor#L64

Ceph is not may area of expertise but you may need to add those
parameters to the monitor script a maybe to other scripts that use the
"rados" command. It may also be possible to modify the RADOS command
to have those parameters instead of modifying all the scripts:

https://github.com/OpenNebula/one/blob/one-4.6/src/mad/sh/scripts_common.sh#L40

As I said I don't know much about Ceph and it may be those credentials
could be set in a config file or so.

On Tue, Jun 17, 2014 at 11:19 AM, Alvaro Simon Garcia
<Alvaro.SimonGarcia at ugent.be> wrote:
> Hi
>
> We have included the admin keyring instead of libvirt user and it works...
> that means that we can run rbd or qemu-img wihtout the libvirt id, but is
> not the best solution. We have included the user into datastore conf:
>
> CEPH_USER="libvirt"
>
> but it seems that is not used by opennebula at the end
>
> Cheers
> Alvaro
>
>
> On 2014-06-17 10:09, Alvaro Simon Garcia wrote:
>>
>> Hi all
>>
>>
>> We have included our ONE nodes into Ceph cluster, cephx auth is working
>> but OpenNebula is not able to detect the free space:
>>
>>
>>
>>> $ onedatastore show 103
>>> DATASTORE 103 INFORMATION
>>> ID             : 103
>>> NAME           : ceph
>>> USER           : oneadmin
>>> GROUP          : oneadmin
>>> CLUSTER        : -
>>> TYPE           : IMAGE
>>> DS_MAD         : ceph
>>> TM_MAD         : ceph
>>> BASE PATH      : /var/lib/one//datastores/103
>>> DISK_TYPE      : RBD
>>>
>>> DATASTORE CAPACITY
>>> TOTAL:         : 0M
>>> FREE:          : 0M
>>> USED:          : 0M
>>> LIMIT:         : -
>>>
>>> PERMISSIONS
>>> OWNER          : um-
>>> GROUP          : u--
>>> OTHER          : ---
>>
>>
>>
>>> $ onedatastore list
>>>    ID NAME                SIZE AVAIL CLUSTER      IMAGES TYPE DS       TM
>>>     0 system            114.8G 85%   -                 0 sys -
>>> shared
>>>     1 default           114.9G 84%   -                 2 img fs       ssh
>>>     2 files             114.9G 84%   -                 0 fil fs       ssh
>>>   103 ceph                  0M -     -                 0 img ceph
>>> ceph
>>
>>
>>
>> but if we run rados as oneadmin user:
>>
>>>
>>> $  rados df -p one --id libvirt
>>> pool name       category                 KB      objects clones
>>> degraded      unfound           rd        rd KB           wr        wr KB
>>> one             -                          0 0            0
>>> 0           0            0 0            0            0
>>>    total used         1581852           37
>>>    total avail   140846865180
>>>    total space   140848447032
>>
>>
>>
>> It's working correctly (we are using one pool and libvirt ceph id)
>>
>> the oned.log only shows this info:
>> Tue Jun 17 10:06:37 2014 [InM][D]: Monitoring datastore default (1)
>> Tue Jun 17 10:06:37 2014 [InM][D]: Monitoring datastore files (2)
>> Tue Jun 17 10:06:37 2014 [InM][D]: Monitoring datastore ceph (103)
>> Tue Jun 17 10:06:37 2014 [ImM][D]: Datastore default (1) successfully
>> monitored.
>> Tue Jun 17 10:06:37 2014 [ImM][D]: Datastore files (2) successfully
>> monitored.
>> Tue Jun 17 10:06:37 2014 [ImM][D]: Datastore ceph (103) successfully
>> monitored.
>>
>> Any clue about how to debug this issue?
>>
>> Thanks in advance!
>> Alvaro
>>
>>
>>
>> _______________________________________________
>> Users mailing list
>> Users at lists.opennebula.org
>> http://lists.opennebula.org/listinfo.cgi/users-opennebula.org
>
>
> _______________________________________________
> Users mailing list
> Users at lists.opennebula.org
> http://lists.opennebula.org/listinfo.cgi/users-opennebula.org



-- 
Javier Fontán Muiños
Developer
OpenNebula - Flexible Enterprise Cloud Made Simple
www.OpenNebula.org | @OpenNebula | github.com/jfontan