[one-users] Ceph datastore monitoring in OpenNebula

[Alvaro Simon Garcia]

Hola Javi

Thanks a lot for your feedback. Yes we will modify the current 
monitoring scripts to take into account this. This feature could be 
useful as well if you want to monitor several datastores that are using 
different ceph pools and users ids. You only have to include the id and 
pool info into the ONE datastore template and the monitoring script will 
use one or another depending on the DS conf.


Cheers and thanks!
Alvaro
On 2014-06-17 14:55, Javier Fontan wrote:
> CEPH_USER is used when generating the libvirt/kvm deployment file but
> not for DS monitoring:
>
> * Deployment file generation:
> https://github.com/OpenNebula/one/blob/one-4.6/src/vmm/LibVirtDriverKVM.cc#L461
> * Monitoring: https://github.com/OpenNebula/one/blob/one-4.6/src/datastore_mad/remotes/ceph/monitor#L64
>
> Ceph is not may area of expertise but you may need to add those
> parameters to the monitor script a maybe to other scripts that use the
> "rados" command. It may also be possible to modify the RADOS command
> to have those parameters instead of modifying all the scripts:
>
> https://github.com/OpenNebula/one/blob/one-4.6/src/mad/sh/scripts_common.sh#L40
>
> As I said I don't know much about Ceph and it may be those credentials
> could be set in a config file or so.
>
> On Tue, Jun 17, 2014 at 11:19 AM, Alvaro Simon Garcia
> <Alvaro.SimonGarcia at ugent.be> wrote:
>> Hi
>>
>> We have included the admin keyring instead of libvirt user and it works...
>> that means that we can run rbd or qemu-img wihtout the libvirt id, but is
>> not the best solution. We have included the user into datastore conf:
>>
>> CEPH_USER="libvirt"
>>
>> but it seems that is not used by opennebula at the end
>>
>> Cheers
>> Alvaro
>>
>>
>> On 2014-06-17 10:09, Alvaro Simon Garcia wrote:
>>> Hi all
>>>
>>>
>>> We have included our ONE nodes into Ceph cluster, cephx auth is working
>>> but OpenNebula is not able to detect the free space:
>>>
>>>
>>>
>>>> $ onedatastore show 103
>>>> DATASTORE 103 INFORMATION
>>>> ID             : 103
>>>> NAME           : ceph
>>>> USER           : oneadmin
>>>> GROUP          : oneadmin
>>>> CLUSTER        : -
>>>> TYPE           : IMAGE
>>>> DS_MAD         : ceph
>>>> TM_MAD         : ceph
>>>> BASE PATH      : /var/lib/one//datastores/103
>>>> DISK_TYPE      : RBD
>>>>
>>>> DATASTORE CAPACITY
>>>> TOTAL:         : 0M
>>>> FREE:          : 0M
>>>> USED:          : 0M
>>>> LIMIT:         : -
>>>>
>>>> PERMISSIONS
>>>> OWNER          : um-
>>>> GROUP          : u--
>>>> OTHER          : ---
>>>
>>>
>>>> $ onedatastore list
>>>>     ID NAME                SIZE AVAIL CLUSTER      IMAGES TYPE DS       TM
>>>>      0 system            114.8G 85%   -                 0 sys -
>>>> shared
>>>>      1 default           114.9G 84%   -                 2 img fs       ssh
>>>>      2 files             114.9G 84%   -                 0 fil fs       ssh
>>>>    103 ceph                  0M -     -                 0 img ceph
>>>> ceph
>>>
>>>
>>> but if we run rados as oneadmin user:
>>>
>>>> $  rados df -p one --id libvirt
>>>> pool name       category                 KB      objects clones
>>>> degraded      unfound           rd        rd KB           wr        wr KB
>>>> one             -                          0 0            0
>>>> 0           0            0 0            0            0
>>>>     total used         1581852           37
>>>>     total avail   140846865180
>>>>     total space   140848447032
>>>
>>>
>>> It's working correctly (we are using one pool and libvirt ceph id)
>>>
>>> the oned.log only shows this info:
>>> Tue Jun 17 10:06:37 2014 [InM][D]: Monitoring datastore default (1)
>>> Tue Jun 17 10:06:37 2014 [InM][D]: Monitoring datastore files (2)
>>> Tue Jun 17 10:06:37 2014 [InM][D]: Monitoring datastore ceph (103)
>>> Tue Jun 17 10:06:37 2014 [ImM][D]: Datastore default (1) successfully
>>> monitored.
>>> Tue Jun 17 10:06:37 2014 [ImM][D]: Datastore files (2) successfully
>>> monitored.
>>> Tue Jun 17 10:06:37 2014 [ImM][D]: Datastore ceph (103) successfully
>>> monitored.
>>>
>>> Any clue about how to debug this issue?
>>>
>>> Thanks in advance!
>>> Alvaro
>>>
>>>
>>>
>>> _______________________________________________
>>> Users mailing list
>>> Users at lists.opennebula.org
>>> http://lists.opennebula.org/listinfo.cgi/users-opennebula.org
>>
>> _______________________________________________
>> Users mailing list
>> Users at lists.opennebula.org
>> http://lists.opennebula.org/listinfo.cgi/users-opennebula.org
>
>