At Fri, 24 Aug 2012 00:03:09 +0800, Liu Yuan wrote: > > On 08/23/2012 10:58 PM, MORITA Kazutaka wrote: > > To fix this problem, we need to move stale objects in the worker > > thread. I've spent several days trying to fix it, but it seems to > > need a lot of work, and it looks difficult to stable the change in a > > short time. > > > > Last time your patches to fix this problem looks kind of easy, why 'a lot > of work' now? The patch was not enough. Main problems are: - We cannot increment an epoch while deleting stale objects because the objects are not stale at the new epoch. - If objects are in the working directory, sheep assumes the objects are already recovered. But it is not true if we unlink the objects in the worker thread. > > > > $ ./farm.sh simple > > using backend simple store > > Name Id Size Used Shared Creation time VDI id Tag > > test 1 100 GB 100 GB 0.0 MB 2012-08-23 11:29 7c2b25 > > > > real 0m1.016s > > user 0m0.092s > > sys 0m0.000s > > This looks much better, but I think not good enough. If we have 500G data, it seems > that we still need around 5 minutes, which is unacceptable to guest VM too. For example > Linux guests won't tolerate for more than 2 minutes. I also tested with 500 GB, but the result was similar. I think 1 second is the overhead of 'collie vdi list'. On the other hand, farm takes more than 1 hour... > > I think we really need to put the moving stale objects job to worker thread as a long-term fix, > which never block main thread. Yes, indeed. Thanks, Kazutaka |