[sheepdog] [PATCH v2 1/2] cluster: add the distributed lock implemented by zookeeper for object-storage

[Robin Dong]

2013/11/29 Liu Yuan <namei.unix at gmail.com>

> On Fri, Nov 29, 2013 at 05:10:31PM +0800, Robin Dong wrote:
> > Implement the distributed lock by zookeeper (refer:
> http://zookeeper.apache.org/doc/trunk/recipes.html)
> > The routine is:
> >         1. create a seq-ephemeral znode in lock directory (use lock-id
> as dir name)
> >         2. get smallest file path as owner of the lock; the other thread
> wait on a pthread_mutex_t
> >         3. if owner of the lock release it (or the owner is killed by
> accident), zookeeper will
> >            trigger zk_watch() which will wake up all waiting threads to
> compete new owner of the lock
> >
> > We add ->local_lock for dist_mutex to avoid many threads in one sheepdog
> daemon to create too many files
> > in a lock directory.
> >
> > Signed-off-by: Robin Dong <sanbai at taobao.com>
> > ---
> >  sheep/cluster.h           |   15 +++++
> >  sheep/cluster/zookeeper.c |  140
> ++++++++++++++++++++++++++++++++++++++++++++-
> >  2 files changed, 154 insertions(+), 1 deletions(-)
> >
> > diff --git a/sheep/cluster.h b/sheep/cluster.h
> > index 81b5ae4..0aa058c 100644
> > --- a/sheep/cluster.h
> > +++ b/sheep/cluster.h
> > @@ -23,6 +23,13 @@
> >  #include "sheep.h"
> >  #include "config.h"
> >
> > +struct dist_mutex {
> > +     uint32_t id;            /* id of this mutex */
> > +     pthread_mutex_t wait;
> > +     pthread_mutex_t local_lock;
> > +     char ephemeral_path[MAX_NODE_STR_LEN];
> > +};
> > +
> >  /*
> >   * maximum payload size sent in ->notify and ->unblock, it should be
> large
> >   * enough to support COROSYNC_MAX_NODES * struct sd_node
> > @@ -109,6 +116,14 @@ struct cluster_driver {
> >       int (*unblock)(void *msg, size_t msg_len);
> >
> >       /*
> > +      * A distributed mutex lock to avoid race condition when using
> swift
> > +      * interface to add/delete/list object.
>
> This is not a swift only operation. Instead it is now as a generic locking
> service.
>
> > +      */
> > +     int (*init_mutex)(struct dist_mutex *mutex, uint32_t id);
> > +     void (*lock_mutex)(struct dist_mutex *mutex);
> > +     void (*unlock_mutex)(struct dist_mutex *mutex);
>
> Please explain every function clearly as other interface does.
>
> Any reason to call it mutex? I'd suggest shorter name as
>

Because it is only mutex now, I will add dist_rwlock,dist_selock or some
other kind of locks in the future.


>
>  ->init_lock()
>  ->lock()
>  ->unlock()
>
> So struct dist_mutex -> struct cluster_lock
>
> Finally, I think you need to initialize local/cluster driver's lock
> interface's
> too, thought it would be currently empty operations for them.


> Thanks
> Yuan
>



-- 
--
Best Regard
Robin Dong
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.wpkg.org/pipermail/sheepdog/attachments/20131129/aa28d733/attachment-0004.html>