[one-users] ceph+flashcache datastore driver
stuartl at vrt.com.au
Tue Mar 25 02:03:06 PDT 2014
On 24/03/14 17:57, Stuart Longland wrote:
> Well, I'm starting to have a stab at creating a driver to do local
> caching on hosts for RBD storage.
> For those who want to follow progress, I've thrown a repository up here:
A heads up on the progress, I now have some basic cloning/linking of
Our cluster was pushing about 120MB/sec prior to me rebuilding the
compute nodes and embarking on this cache experiment. The set up here is:
The Ceph nodes are combined MON/OSD hosts with two OSDs and a MON each.
The machines each have two 1Gbps NICs, one of which connects to an
isolated unmanaged 1Gbps switch, the other connects to a second
unmanaged 1Gbps switch shared with the VM hosts.
The VM hosts themselves have 3 1Gbps NICs, one as a VLAN trunk for VMs,
the other two are bonded to form a 2Gbps link. They each have a 60GB
SSD, of which 8GB is taken up by the OS. The rest is LVM.
For testing I set up a VM with 100% of its disk cached to local SSD.
The initial read was slow, about 20MB/sec (dependent on overall
cluster/network load), however, subsequent reads is showing speeds of
So once the cache is filled up, there's a big jump in read performance.
\ /|_) | T: +61 7 3535 9619
\/ | \ | 38b Douglas Street F: +61 7 3535 9699
SYSTEMS Milton QLD 4064 http://www.vrt.com.au
More information about the Users