At Sun, 01 Apr 2012 15:15:16 +0800, Liu Yuan wrote: > > On 04/01/2012 02:46 PM, MORITA Kazutaka wrote: > > > I don't think the approach can guarantee that all nodes process the > > membership events in the same order because some nodes may have > > outstanding I/Os but the others may not. > > > > > I think we need to clarify that there are two kind of outstanding IO > requests > a) queued in the cpg queue, not issued yet > b) already issued, being processed on the fly > > I don't think the a) cause any problem, because, > > when we get a confchg event, considering there are outstanding IO > reuqests in the cpg queue, if we flush them before confchg, most of them > will risk to be failed or wasteful operation because > 1) in case some nodes are down and they don't get a fresh knowledge of > that, so the operation targeted on those down nodes are doomed to be failed > 2) in case new nodes join, the targeted object location might be > migrated already > > and only b) might do. Yes, I meant b). > > Currently, with my path, 2) will lead the confchg to be queued instead > of being processed immediately. The queuing of more than one confchg > will cause problem. > > So this situation would be healed by awaiting infly IOs completion(spin > lock or aync notification), then insert confchg at the head of cpg > queue, that guaranteed to be processed. We cannot wait in start_cpg_event_work() because it is in the main thread. Thanks, Kazutaka |