[Sheepdog] [announce] Accord, a coordination service for write-intensive workload

[OZAWA Tsuyoshi]

CCing corosync ml since there may be some cluster developers interested
in participating in this project there.

I am pleased to announce the release of Accord, a coordination service
like Apache ZooKeeper that uses corosync as a total-order messaging
infrastructure. Accord is a part of Sheepdog project, going to be in
charge of cluster management of Sheepdog to get more scalability.

The current design of Sheepdog has a scalability problem because of
corosync.
It gets more scalability by picking out coordination features of
corosync as an external system - Accord.
Accord provides complex components of distributed systems such as
distributed locking, message ordering, and membership management for a
large number of clients, aiming to be a kernel of distributed systems.
Concretely speaking, it features:

- Accord is a distributed, transactional, and fully-replicated (No SPoF)
Key-Value Store with strong consistency.
- Accord can be scale-out up to tens of nodes.
- Accord servers can handle tens of thousands of clients.
- Any clients can issue I/O requests, and any servers can accept and
process I/O requests.
- The changes for a write request from a client can be notified to the
other clients.
- Accord detects events of client's joining/leaving, and notifies
joined/left client information to the other clients.

For instance, a distributed lock manager or a leader election feature
for handling tens thousands of clients can be implemented easily by
using the joining/leaving-clients notification and the KVS with strong
consistency. In other words, Accord builds up the scalability of corosync.

These features are also provided in ZooKeeper in fact. The difference
between Accord and ZooKeeper is :
- Accord focused on write-intensive workloads unlike ZooKeeper.
ZooKeeper forwards all write requests to a master server. It may be
bottleneck in write-intensive workload. Benchmark demonstrates that the
write-operation throughput of Accord is much higher than one of
ZooKeeper (up to 20 times better throughput at persistent mode, and up
to 18 times better throughput at in-memory mode).
- More flexible transaction support. Not only write, del operations, but
also value-based cmp, copy, read operations are supported in transaction
operation.

As mentioned above, Accord is specific to write-intensive workload. It
extends the application scope of Coordination service.
Assumed applications are as follows, for example :
- Distributed Lock Manager whose lock operations occur at a high
frequency from thousands of clients.
- Metadata management service for large-scale distributed storage,
including Sheepdog, HDFS, etc.
- Replicated Message Queue or logger (For instance, replicated RabbitMQ).
and so on.

The other distributed systems can use Accord features easily because
Accord provides general-purpose APIs (read/write/del/transaction).

More information including getting started, benchmarks, and API docs are
available from our project page :
http://www.osrg.net/accord

and all code is available from:
https://github.com/collie/accord

Please try it out, and let me know any opinions or problems via Sheepdog ML.

Best regards,
OZAWA Tsuyoshi
<ozawa.tsuyoshi at lab.ntt.co.jp>