[sheepdog] [PATCH v0, RFC] sheep: writeback cache semantics in backend store

[Christoph Hellwig]

On Thu, Aug 16, 2012 at 08:45:57PM +0900, Hitoshi Mitake wrote:
> From: Hitoshi Mitake <mitake.hitoshi at lab.ntt.co.jp>
> 
> Hi sheepdog list, nice to meet you.
> 
> This patch implements writeback cache semantics in backend
> store of sheep. Current backend store farm calls open() with
> O_DSYNC, so every object write causes slow disk access. This incurs
> overhead and this overhead is not necessary. Because current qemu
> block driver invokes SD_OP_FLUSH_VDI explicitly for object
> cache. Flushing disk cache with the invocation of SD_OP_FLUSH_VDI
> instead of every object write is enough for current sheep.
> 
> For improving performance by reducing needless disk access, this patch
> adds new inter-sheep operation SD_OP_FLUSH_PEER. This operation is
> used in a situation like this:
> qemu sends SD_OP_FLUSH_VDI -> gateway sheep sends SD_OP_FLUSH_PEER ->
> other sheeps
> And sheeps which received SD_OP_FLUSH_PEER flush disk cache with
> syncfs() system call.
> 
> 
> Below is the evaluation result with dbench:
> 
>  Before applying this patch, without -s (O_SYNC) option:
>  Throughput 13.9269 MB/sec  1 clients  1 procs  max_latency=818.428 ms
>  Before applying this patch, with -s option:
>  Throughput 2.76792 MB/sec (sync open)  1 clients  1 procs
>  max_latency=291.670 ms
> 
>  After applying this patch, without -s option:
>  Throughput 29.7306 MB/sec  1 clients  1 procs  max_latency=1344.463 ms
>  After applying this patch, with -s option:
>  Throughput 4.29357 MB/sec (sync open)  1 clients  1 procs
>  max_latency=450.045 ms
> 
> 
> This patch adds new command line option -W to sheep. With -W, sheep
> uses writeback cache semantics in backend store. I added this new
> option mainly for easy testing and evaluation. If writeback cache
> semantics is more suitable than the previous writethrough semantics as
> default, I'll delete this option again.
> 
> This patch may contain lots of bad code because I'm new to sheepdog.
> I'd like to hear your comments.
> 
> Cc: MORITA Kazutaka <morita.kazutaka at lab.ntt.co.jp>
> Signed-off-by: Hitoshi Mitake <mitake.hitoshi at lab.ntt.co.jp>
> 
> ---
>  include/internal_proto.h |    1 +
>  sheep/farm/farm.c        |    3 +++
>  sheep/gateway.c          |    2 +-
>  sheep/ops.c              |   44 ++++++++++++++++++++++++++++++++++++++++++++
>  sheep/sheep.c            |    7 ++++++-
>  sheep/sheep_priv.h       |    5 +++++
>  6 files changed, 60 insertions(+), 2 deletions(-)
> 

> -static int gateway_forward_request(struct request *req)
> +int gateway_forward_request(struct request *req)

>  static int local_flush_vdi(struct request *req)
>  {
> +	if (sys->store_writeback)
> +		gateway_forward_request(req);
> +
>  	if (!sys->enable_write_cache)
>  		return SD_RES_SUCCESS;
>  	return object_cache_flush_vdi(req);

Instead of making gateway_forward_request non-static I'd rather
officially declare flush_vdi a gateway op and move it to gateway.c,
that's how it works anyway, except that right now it's a no-op
except for the object cache.

That moving can probably be a good preparatory patch before the
real changes.

> +int peer_flush_dcache(struct request *req)

I'd just call it peer_flush