[sheepdog] [PATCH 1/2] sheep: use work queue for asynchronous local request

[Liu Yuan]

On Fri, Jan 17, 2014 at 12:35:31PM +0900, MORITA Kazutaka wrote:
> From: MORITA Kazutaka <morita.kazutaka at lab.ntt.co.jp>
> 
> We can use our work queue framework for handling asynchrouns requests.
> There is no need to create short living events.
> 
> Signed-off-by: MORITA Kazutaka <morita.kazutaka at lab.ntt.co.jp>
> ---
>  sheep/request.c    | 73 +++++++++++++++++++++++++++++-------------------------
>  sheep/sheep.c      |  4 ++-
>  sheep/sheep_priv.h |  2 +-
>  3 files changed, 43 insertions(+), 36 deletions(-)
> 
> diff --git a/sheep/request.c b/sheep/request.c
> index 161b7db..a15fd8c 100644
> --- a/sheep/request.c
> +++ b/sheep/request.c
> @@ -575,31 +575,11 @@ out:
>  	return ret;
>  }
>  
> -static void local_req_async_handler(int fd, int events, void *data)
> -{
> -	struct request *req = data;
> -	struct request_iocb *iocb = req->iocb;
> -
> -	if (events & EPOLLERR)
> -		sd_err("request handler error");
> -	eventfd_xread(fd);
> -
> -	if (unlikely(req->rp.result != SD_RES_SUCCESS))
> -		iocb->result = req->rp.result;
> -
> -	if (uatomic_sub_return(&iocb->count, 1) == 0)
> -		eventfd_xwrite(iocb->efd, 1);
> -
> -	unregister_event(req->local_req_efd);
> -	close(req->local_req_efd);
> -	free_local_request(req);
> -}
> -
>  worker_fn struct request_iocb *local_req_init(void)
>  {
>  	struct request_iocb *iocb = xzalloc(sizeof(*iocb));
>  
> -	iocb->efd = eventfd(0, 0);
> +	iocb->efd = eventfd(0, EFD_SEMAPHORE);
>  	if (iocb->efd < 0) {
>  		sd_err("eventfd failed, %m");
>  		free(iocb);
> @@ -613,7 +593,8 @@ worker_fn int local_req_wait(struct request_iocb *iocb)
>  {
>  	int ret;
>  
> -	eventfd_xread(iocb->efd);
> +	for (int i = 0; i < iocb->count; i++)

i should be uint32_t to avoid overflow

> +		eventfd_xread(iocb->efd);

Does this work for following case:

3 threads call eventfd_xwrite(efd), after that we call eventfd_xread(efd) here,
so we will only return once, not 3 times, no?

I guess you should use EFD_SEMAPHORE.

Thanks
Yuan