[one-users] Ceph w/ CephX Authentication

[Campbell, Bill]

So would like to share my experiences with the support of the new CephX authentication support in 4.4, and have a few questions for the development team. 

In short, it works, but the documentation is not very clear on what needs to be done to make it work, so here's what I did (and I'm certain this is the intent of development currently): 



    * A secret needs to be defined within Libvirt on ALL hypervisor nodes that you utilize with Ceph. This needs to be done manually prior to deploying any virtual instances within the cluster. 
    * Looks like development used the guideline set forth by the Ceph documentation. In this, it is required to create a client.libvirt user, as shown in the example in the Ceph documentation. Otherwise, that user is not validated via Ceph and will be unable to access RBDs in the cluster. 
    * The UUID can be specified or generated, however the secret.xml file must be defined: See my example here: http://dev.opennebula.org/issues/1796 and reference the Ceph documentation for defining the secret.xml within Libvirt. 
    * You must copy the key for the client.libvirt user and define the secret 


        * 'ceph auth list' will list the users and their appropriate keys. Copy the key for placement in the definition string. See the link to the opennebula issue 1796 above for an example: 

Now my questions for the OpenNebula devs: 



    * Since the Ceph_Secret UUID can be defined within a secret.xml file, can that UUID be automatically generated as part of the Ceph Datastore definition? 


        * I see why it's not based on the source, checking to see if the UUID variable exists to ensure the deployment file has the appropriate information 
        * Can there be an option for Ceph Datastore creation for whether or not you utilize CephX authentication on the Datastore (like a checkbox or YES/NO)? If so, then a UUID can be generated and the CEPH_SECRET attribute can be defined automatically. Otherwise, no CEPH_SECRET defined automatically. 
    * 
I think the UUID generation would be the most difficult part. The manual process for the administrator would need to be to define the KEY for injection into Libvirt on each hypervisor. I would imagine this key could be defined as a datastore attribute like CEPH_KEY or something? 
    * At that point, all other operations could be handled by the Transfer Manager 


        * Check to see if the UUID currently exists in the Libvirt secret list on the deployment host ("virsh secret-list | grep -c $CEPH_SECRET" should be an appropriate validation) 
        * If not, then generate the secret.xml file, injecting the CEPH_SECRET attribute 
        * Then define the secret and apply the CEPH_KEY value to the secret 
        * Continue with VM deployment. 

Thoughts on the above? 




Bill Campbell 
Infrastructure Architect 



Axcess Financial Services, Inc. 
7755 Montgomery Rd., Suite 400 
Cincinnati, OH 45236 


NOTICE: Protect the information in this message in accordance with the company's security policies. If you received this message in error, immediately notify the sender and destroy all copies.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.opennebula.org/pipermail/users-opennebula.org/attachments/20131121/02cedc47/attachment.htm>