[sheepdog] [PATCH] sheep: add simple_store driver again

[MORITA Kazutaka]

At Fri, 24 Aug 2012 00:03:09 +0800,
Liu Yuan wrote:
> 
> On 08/23/2012 10:58 PM, MORITA Kazutaka wrote:
> > To fix this problem, we need to move stale objects in the worker
> > thread.  I've spent several days trying to fix it, but it seems to
> > need a lot of work, and it looks difficult to stable the change in a
> > short time.
> > 
> 
> Last time your patches to fix this problem looks kind of easy, why 'a lot
> of work' now?

The patch was not enough.  Main problems are:

 - We cannot increment an epoch while deleting stale objects because
   the objects are not stale at the new epoch.
 - If objects are in the working directory, sheep assumes the objects
   are already recovered.  But it is not true if we unlink the objects
   in the worker thread.

> 
> 
> > $ ./farm.sh simple
> > using backend simple store
> >   Name        Id    Size    Used  Shared    Creation time   VDI id  Tag
> >   test         1  100 GB  100 GB  0.0 MB 2012-08-23 11:29   7c2b25
> > 
> > real    0m1.016s
> > user    0m0.092s
> > sys     0m0.000s
> 
> This looks much better, but I think not good enough. If we have 500G data, it seems
> that we still need around 5 minutes, which is unacceptable to guest VM too. For example
> Linux guests won't tolerate for more than 2 minutes.

I also tested with 500 GB, but the result was similar.  I think 1
second is the overhead of 'collie vdi list'.  On the other hand, farm
takes more than 1 hour...

> 
> I think we really need to put the moving stale objects job to worker thread as a long-term fix,
> which never block main thread. 

Yes, indeed.

Thanks,

Kazutaka