[sheepdog] [PATCH] sheep: wait until there is no get_vdis work in wait_get_vdis_done()

[Liu Yuan]

On Wed, Aug 28, 2013 at 10:46:47AM +0900, MORITA Kazutaka wrote:
> From: MORITA Kazutaka <morita.kazutaka at lab.ntt.co.jp>
> 
> There is a race condition like as follows:
> 
>  1. Node A, B, and C start up.
>  2. Node A receives accept events from B and C, and calls
>     start_recovery().
>  3. Node A calls wait_get_vdis_done in the recovery thread, but there
>     are two get_vdis works in sys->block_wqueue.
>  4. When the first get_vdis_done() is called, is_vdi_list_ready will
>     be set to false.
>  5. Node A starts object recovery without receiving vdi state from C.
> 
> To fix the problem, this patch counts the number of get_vdi works in
> the block_wqueue.
> 
> Signed-off-by: MORITA Kazutaka <morita.kazutaka at lab.ntt.co.jp>
> ---
> 
> I think this should go into the stable branch, too.
> 
>  sheep/group.c |    8 ++++----
>  1 file changed, 4 insertions(+), 4 deletions(-)
> 
> diff --git a/sheep/group.c b/sheep/group.c
> index 450c71d..cd56cb6 100644
> --- a/sheep/group.c
> +++ b/sheep/group.c
> @@ -26,7 +26,7 @@ struct get_vdis_work {
>  
>  static pthread_mutex_t wait_vdis_lock = PTHREAD_MUTEX_INITIALIZER;
>  static pthread_cond_t wait_vdis_cond = PTHREAD_COND_INITIALIZER;
> -static bool is_vdi_list_ready = true;
> +static refcnt_t nr_get_vdis_works;
>  
>  static main_thread(struct vnode_info *) current_vnode_info;
>  static main_thread(struct list_head *) pending_block_list;
> @@ -498,7 +498,7 @@ static void get_vdis_done(struct work *work)
>  		container_of(work, struct get_vdis_work, work);
>  
>  	pthread_mutex_lock(&wait_vdis_lock);
> -	is_vdi_list_ready = true;
> +	refcount_dec(&nr_get_vdis_works);
>  	pthread_cond_broadcast(&wait_vdis_cond);
>  	pthread_mutex_unlock(&wait_vdis_lock);
>  
> @@ -581,7 +581,7 @@ static void get_vdis(const struct sd_node *nodes, size_t nr_nodes,
>  	w->nr_members = nr_nodes;
>  	memcpy(w->members, nodes, array_len);
>  
> -	is_vdi_list_ready = false;
> +	refcount_inc(&nr_get_vdis_works);
>  
>  	w->work.fn = do_get_vdis;
>  	w->work.done = get_vdis_done;
> @@ -593,7 +593,7 @@ void wait_get_vdis_done(void)
>  	sd_debug("waiting for vdi list");
>  
>  	pthread_mutex_lock(&wait_vdis_lock);
> -	while (!is_vdi_list_ready)
> +	while (refcount_read(&nr_get_vdis_works) > 0)
>  		pthread_cond_wait(&wait_vdis_cond, &wait_vdis_lock);
>  	pthread_mutex_unlock(&wait_vdis_lock);
>  
> -- 
> 1.7.9.5
> 

Applied thanks

Yuan