[Sheepdog] [PATCH v2 1/2] force some callbacks of cluster request running in worker thread

Liu Yuan namei.unix at gmail.com
Sun Apr 29 15:34:36 CEST 2012


On 04/29/2012 06:45 PM, Christoph Hellwig wrote:

> On Sat, Apr 28, 2012 at 03:33:53PM +0800, Liu Yuan wrote:
>> I think its time to retrofit process_main(). it doesn't necessarily run
>> in main thread. Actually, we should try to put it in the event queue as
>> possible as we can, to offload the main thread.
>>
>> I'd suggest a new name scheme:
>> process_work() -> process_top(), which runs only local node
>> process_main() -> process_bottom(), which runs in all nodes, either in
>> worker or in main thread.
> 
> How would we decide when to run it in the main or worker thread?
> 


Unfortunately, there is no specific rule. At least for now, there is
only some general practice:
1) push jobs as much as possible to worker threads
2) don't use lock in main thread (without a good reason)

> I'm getting more an more scared about the threading model - there's a
> lot of assumption that certain pieces of code only run in the main
> thread, or in the event_queue and thus can avoid locking, but there's no
> good way to veryify that this actually is the case.
> 


Ditto

> I think before this gets even more complicated an tid based assertation
> mechanism ala:
> 
>    main_tid = gettid();
> 
> static inline bool assert_mainthread(void)
> {
> 	assert(gettid() == main_tid);
> }
> 
> plus variants for other "optimized" queues needs to be added over all
> places that have this assumption.
> 


This looks good to me, but I think it would be useful only when we have
abstract out shared code in the main and worker thread as function helpers.

But I think this is hard to achieve, so for now we simply leave this
responsibility and freedom to programmers, mostly for implementing
lcoal&cluster operations.

> As for the naming of the execution function, why not name them
> descriptively as what they do:
> 
>   process_on_each_node()
> 
> and
> 
>   process_local()
> 
> or similar.
> 

It is not that easy. In a bigger picture, we have local and cluster
requests, which actually implement process_local() and
process_on_each_node() semantics.

I deliberately to suggest use top and bottom to differentiate it request
scope. top() and bottom() thus are only internal logic structure for
handling requests.

Thanks,
Yuan



More information about the sheepdog mailing list