[Sheepdog] [corosync] [announce] Accord, a coordination service for write-intensive workload

[MORITA Kazutaka]

At Tue, 04 Oct 2011 12:52:02 +0900,
OZAWA Tsuyoshi wrote:
> 
> CCing corosync ml since there may be some cluster developers interested
> in participating in this project there.
> 
> I am pleased to announce the release of Accord, a coordination service
> like Apache ZooKeeper that uses corosync as a total-order messaging
> infrastructure. Accord is a part of Sheepdog project, going to be in
> charge of cluster management of Sheepdog to get more scalability.
> 
> The current design of Sheepdog has a scalability problem because of
> corosync.
> It gets more scalability by picking out coordination features of
> corosync as an external system - Accord.

We already have a plan to abstract out Sheepdog membership management.
I posted a patch to support multiple coordination system last week.

I think ZooKeeper is a defacto standard coordination system, but
Accord is more suitable for Sheepdog because it is corosync-based and
works without JVM (Sheepdog used JGroups before, but moved to corosync
to get rid of the JVM dependency).  Moreover, Accord has a much
smaller code size than ZooKeeper and some of the codes are borrowed
from Sheepdog, so the code is easy to understand for Sheepdog
developers.

If no one opposes, I'd like to develop Accord in the Sheepdog mailing
list, and make it a default coordination system for Sheepdog.

Any thoughts on this?


Thanks,

Kazutaka


> Accord provides complex components of distributed systems such as
> distributed locking, message ordering, and membership management for a
> large number of clients, aiming to be a kernel of distributed systems.
> Concretely speaking, it features:
> 
> - Accord is a distributed, transactional, and fully-replicated (No SPoF)
> Key-Value Store with strong consistency.
> - Accord can be scale-out up to tens of nodes.
> - Accord servers can handle tens of thousands of clients.
> - Any clients can issue I/O requests, and any servers can accept and
> process I/O requests.
> - The changes for a write request from a client can be notified to the
> other clients.
> - Accord detects events of client's joining/leaving, and notifies
> joined/left client information to the other clients.
> 
> For instance, a distributed lock manager or a leader election feature
> for handling tens thousands of clients can be implemented easily by
> using the joining/leaving-clients notification and the KVS with strong
> consistency. In other words, Accord builds up the scalability of corosync.
> 
> These features are also provided in ZooKeeper in fact. The difference
> between Accord and ZooKeeper is :
> - Accord focused on write-intensive workloads unlike ZooKeeper.
> ZooKeeper forwards all write requests to a master server. It may be
> bottleneck in write-intensive workload. Benchmark demonstrates that the
> write-operation throughput of Accord is much higher than one of
> ZooKeeper (up to 20 times better throughput at persistent mode, and up
> to 18 times better throughput at in-memory mode).
> - More flexible transaction support. Not only write, del operations, but
> also value-based cmp, copy, read operations are supported in transaction
> operation.
> 
> As mentioned above, Accord is specific to write-intensive workload. It
> extends the application scope of Coordination service.
> Assumed applications are as follows, for example :
> - Distributed Lock Manager whose lock operations occur at a high
> frequency from thousands of clients.
> - Metadata management service for large-scale distributed storage,
> including Sheepdog, HDFS, etc.
> - Replicated Message Queue or logger (For instance, replicated RabbitMQ).
> and so on.
> 
> The other distributed systems can use Accord features easily because
> Accord provides general-purpose APIs (read/write/del/transaction).
> 
> More information including getting started, benchmarks, and API docs are
> available from our project page :
> http://www.osrg.net/accord
> 
> and all code is available from:
> https://github.com/collie/accord
> 
> Please try it out, and let me know any opinions or problems via Sheepdog ML.
> 
> Best regards,
> OZAWA Tsuyoshi
> <ozawa.tsuyoshi at lab.ntt.co.jp>
> 
> _______________________________________________
> discuss mailing list
> discuss at corosync.org
> http://lists.corosync.org/mailman/listinfo/discuss