On 07/13/2012 05:10 PM, Christoph Hellwig wrote: > On Thu, Jul 12, 2012 at 03:20:29PM +0800, Liu Yuan wrote: >> On 07/12/2012 02:18 PM, HaiTing Yao wrote: >>> 1, Can not guarantee I/O sequence of VDI. Many tx/rx threads, many io >>> threds. If two requests r/w the same sector of same VDI, data will be >>> corrupted. >>> >> >> I think we will never get two requests R/W the same sector, because VM's >> kernel should(might) guarantee this, at least for linux. > > At least Windows does not, we had a problem in qemu with this before, > note that even in this case there is no ordering guarantee so I don't > think it actually is a problem. > >>> 2, I did not test the performance of this patch, but I think it will >>> not be better than before. The time for creating/destroing threads may >>> be longer than the time for data translation. >> >> Yes, short threads for this tx/rx would be the performance bottleneck. I >> benchmark the write performance with my patch, seems that no noticeable >> boost. > > I really don't like either implementation. For one I'd really love to > figure out the root cause, that is where we spend all that time doing > epoll driven non-blocking I/O. Second if we go for a more threaded > network I/O model I'd like to design it based on possible contention > points, and avoiding creating threads all the time, which leads to a > per-connection thread model where listen_handler creates a RX and a TX > thread per connection which then uses EPOLL to handle all I/O on it. > The real benefit is, as I described before, that requests from different sources (be it from peer recovery req or io req, or gateway req) can overlap each other, I think this will improve the scalability and will work out better performance. Well, I think I should test the patch with several real boxes than on one box, because one disk shared by several sheep might hide the real benefit. I don't think per-connection thread model will work out. The big problem is that, we will use too many threads for one sheep: nr_threads = nr_node * 8 + nr_vm For e.g, for 1k node setup, we'll have 8000+ threads at most! This will be definitely a problem. Thanks, Yuan |