On Tue, Dec 03, 2013 at 06:15:53PM +0800, Robin Dong wrote: > From: Robin Dong <sanbai at taobao.com> > > Implement the distributed lock by zookeeper > (refer:http://zookeeper.apache.org/doc/trunk/recipes.html) > > The routine is: > 1. create a seq-ephemeral znode in lock directory > (use lock-id as dir name) > 2. get smallest file path as owner of the lock; the other thread wait > on a pthread_mutex_t (cluster_lock->wait) > 3. if owner of the lock release it (or the owner is killed by accident), > zookeeper will trigger zk_watch() which will wake up all waiting > threads to compete new owner of the lock > > We use dlock_array to store pointers of cluster_locks in this sheep daemon so > when receiving the event of ZOO_DELETED_EVENT the program will wake up all > waiters (in this sheep daemon) who is sleeping on the lock id and let them > compete for new owner. > > dlock_array is just a normal array using lock-id as index, so imaging a > scenario: two threads (A and B) in one sheep daemon call zk_lock() for same > lock-id, they will create two znodes in zookeeper but set dlock_array[lock_id] > to only one of them (for example, set to B). After that, when ZOO_DELETED_EVENT > comes, the zk_waiter() will only wake up thread B and thread A will sleep on > '->wait' forever becuase no one could wakeup him. > > We have two method to solve this problem: > A. using more complicated structure instead of dlock_array to store > both A and B 's lock handle. > B. adding a lock to avoid A and B call zk_lock() in the same time. > We prefer method B because it also avoid creating too many files in a directory > of zookeeper which will take too much pressure on zookeeper server if the > number of sheep deamons are huge. Therefore we add 'local_lock' in > 'struct cluster_lock'. > > v6 --> v7: > 1. change bucket number of lock table from 4097 to 1021 > 2. use sd_hash() for lock table Applied after chaning sd_hash as sd_hash_64(), thanks! Yuan |