[one-users] ceph+flashcache datastore driver

[Stuart Longland]

On 24/03/14 17:57, Stuart Longland wrote:
> Well, I'm starting to have a stab at creating a driver to do local
> caching on hosts for RBD storage.
> 
> For those who want to follow progress, I've thrown a repository up here:
> http://git.longlandclan.yi.org/?p=opennebula-ceph-flashcache.git;a=summary

A heads up on the progress, I now have some basic cloning/linking of
images working.

Our cluster was pushing about 120MB/sec prior to me rebuilding the
compute nodes and embarking on this cache experiment.  The set up here is:

The Ceph nodes are combined MON/OSD hosts with two OSDs and a MON each.
 The machines each have two 1Gbps NICs, one of which connects to an
isolated unmanaged 1Gbps switch, the other connects to a second
unmanaged 1Gbps switch shared with the VM hosts.

The VM hosts themselves have 3 1Gbps NICs, one as a VLAN trunk for VMs,
the other two are bonded to form a 2Gbps link.  They each have a 60GB
SSD, of which 8GB is taken up by the OS.  The rest is LVM.

For testing I set up a VM with 100% of its disk cached to local SSD.
The initial read was slow, about 20MB/sec (dependent on overall
cluster/network load), however, subsequent reads is showing speeds of
approximately 218MB/sec.

So once the cache is filled up, there's a big jump in read performance.
-- 
Stuart Longland
Contractor
     _ ___
\  /|_) |                           T: +61 7 3535 9619
 \/ | \ |     38b Douglas Street    F: +61 7 3535 9699
   SYSTEMS    Milton QLD 4064       http://www.vrt.com.au