[one-users] Ceph w/ CephX Authentication

Thu Nov 21 13:39:27 PST 2013

Thanks for the response. 

I'm looking at this from an automation level perspective. If we had the key defined within OpenNebula, that would provide one location where data related needs to be stored (vs. multiple, etc.). Trying to adjust for my use case systemically. 

As far as the key being public, can particular attributes only be seen or designated based on group ACLs? Or do ACLs only control access at the datastore level? I'd agree with storing the key would defeat the purpose, depending on use case (ours is internal with no 'users' per se, just administrative access). 

----- Original Message -----

From: "Ruben S. Montero" <rsmontero at opennebula.org> 
To: "Bill Campbell" <bcampbell at axcess-financial.com> 
Cc: "Users OpenNebula" <users at lists.opennebula.org> 
Sent: Thursday, November 21, 2013 4:23:03 PM 
Subject: Re: [one-users] Ceph w/ CephX Authentication 

Hi Bill 

Thanks for giving a try to 4.4beta and your great feedback on integrating cephx. So there are a couple of thing to clear out: 

1.- About the optional use of cephx. 

Currently ig CEPH_HOST is defined we generate the elements: 

<host name='$FIRST CEPHMON ENTRY' port='$CEPHPORT'/> 

But only if CEPH_SECRET (identified by UUID) is defined the <auth> element is included. 

Is this a good way to enable/disable cephx? 

2.- About the cephx user 

Yes, now it is hardcoded to libvirt but you should be able to change that. We will add a CEPH_USER attribute to the Datastore configuration. So 

<auth username='$CEPHUSER'> 

is generated as in 1796 

3.- About secret definition 

There are two options then: 

a) create and define secret.xml in all hypervisors, and add the UUID used to identify the secret to the datastore template. 

b) Let the admin define the key (and optionally a UUID for it) and then create and define the secret before deploying the VM. 

Currently we have a) 

My concern with b) is that any user able to register an image in the datastore, or deploying a VM using an image from the ceph datastore will have access to the KEY. I am not sure if that defeats the whole purpose of using cephx. If you are going to public the key, why bother? 

What do you think? 

On the other hand, with a) you need to setup the secret on the hypervisors but we're assuming this can be automated as part of the installation process 

Again, we really appreciate your feedback and contributions on this 

Cheers 

Ruben 

On Thu, Nov 21, 2013 at 5:29 PM, Campbell, Bill < bcampbell at axcess-financial.com > wrote: 

So would like to share my experiences with the support of the new CephX authentication support in 4.4, and have a few questions for the development team. 

In short, it works, but the documentation is not very clear on what needs to be done to make it work, so here's what I did (and I'm certain this is the intent of development currently): 

    * A secret needs to be defined within Libvirt on ALL hypervisor nodes that you utilize with Ceph. This needs to be done manually prior to deploying any virtual instances within the cluster. 
    * Looks like development used the guideline set forth by the Ceph documentation. In this, it is required to create a client.libvirt user, as shown in the example in the Ceph documentation. Otherwise, that user is not validated via Ceph and will be unable to access RBDs in the cluster. 
    * The UUID can be specified or generated, however the secret.xml file must be defined: See my example here: http://dev.opennebula.org/issues/1796 and reference the Ceph documentation for defining the secret.xml within Libvirt. 
    * You must copy the key for the client.libvirt user and define the secret 

        * 'ceph auth list' will list the users and their appropriate keys. Copy the key for placement in the definition string. See the link to the opennebula issue 1796 above for an example: 

Now my questions for the OpenNebula devs: 

    * Since the Ceph_Secret UUID can be defined within a secret.xml file, can that UUID be automatically generated as part of the Ceph Datastore definition? 

        * I see why it's not based on the source, checking to see if the UUID variable exists to ensure the deployment file has the appropriate information 
        * Can there be an option for Ceph Datastore creation for whether or not you utilize CephX authentication on the Datastore (like a checkbox or YES/NO)? If so, then a UUID can be generated and the CEPH_SECRET attribute can be defined automatically. Otherwise, no CEPH_SECRET defined automatically. 
    * 
I think the UUID generation would be the most difficult part. The manual process for the administrator would need to be to define the KEY for injection into Libvirt on each hypervisor. I would imagine this key could be defined as a datastore attribute like CEPH_KEY or something? 
    * At that point, all other operations could be handled by the Transfer Manager 

        * Check to see if the UUID currently exists in the Libvirt secret list on the deployment host ("virsh secret-list | grep -c $CEPH_SECRET" should be an appropriate validation) 
        * If not, then generate the secret.xml file, injecting the CEPH_SECRET attribute 
        * Then define the secret and apply the CEPH_KEY value to the secret 
        * Continue with VM deployment. 

Thoughts on the above? 

Bill Campbell 
Infrastructure Architect 

Axcess Financial Services, Inc. 
7755 Montgomery Rd., Suite 400 
Cincinnati, OH 45236 

NOTICE: Protect the information in this message in accordance with the company's security policies. If you received this message in error, immediately notify the sender and destroy all copies. 

_______________________________________________ 
Users mailing list 
Users at lists.opennebula.org 
http://lists.opennebula.org/listinfo.cgi/users-opennebula.org 

-- 
-- 
Ruben S. Montero, PhD 
Project co-Lead and Chief Architect 
OpenNebula - Flexible Enterprise Cloud Made Simple 
www.OpenNebula.org | rsmontero at opennebula.org | @OpenNebula 

NOTICE: Protect the information in this message in accordance with the company's security policies. If you received this message in error, immediately notify the sender and destroy all copies.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.opennebula.org/pipermail/users-opennebula.org/attachments/20131121/083947c1/attachment-0002.htm>