[sheepdog] [PATCH v4 0/5] work: implement a dynamically changing thread pool

[MORITA Kazutaka]

At Mon, 04 Mar 2013 19:10:41 +0800,
Liu Yuan wrote:
> 
> On 03/04/2013 08:08 AM, MORITA Kazutaka wrote:
> > This removes a short thread feature, and adds a worker thread pool
> > again with support of dymanic pool size.  The reasons we need this
> > series are:
> > 
> >  - Currently, sheep calls a pthread_create for every I/O request, but
> >    the overhead is not so cheap especially when the underlying disk is
> >    fast (e.g. the disk has non-volatile cache).
> > 
> >  - Some operations (e.g. object cache pusher) consumes too many
> >    threads and causes a problem.  To control the maximum number of
> >    threads, a thread pool model is better for us.
> 
> I have noticed that we'll have a lot of threads remain, in my case 99,
> after one installation of OS in a single VM even though there is no
> active VM after that.
> 
> So I suspect that work threads shrinking isn't well functional as
> expected. Do you test in a environment with massive active VMs in the
> cluster and see if nr of threads of one sheep is sane?

Yes, it works as I expected - if the number of running threads become
zero suddenly, the work queue keeps the number of threads in case that
the threads are required suddenly again.  If the number of running
threads decreases gradually, the work queue reduces the size of pool
gradually too.

The only problem is that the work queue cannot reduce the size of pool
when there is no running threads for a long time, but I think it is
okay as long as the remained threads don't consume many resources.

Thanks,

Kazutaka