[sheepdog] read/write during recovery

[Dietmar Maurer]

> > 1.) write 'obj' to C => write a journal, succeed
> 
> where is this journal stored? locally in node C?

yes.
 
> > 2.) write 'obj' to A => succeed
> >
> > Then node C recovers 'obj' from node A. After that, node C need to
> > apply above write journal to 'obj' - just to make sure that we do not
> > lose any writes.
> 
> This scheme might work, but I think with full implementation out, it would be
> not simpler than current implementation and need considerable lines of
> code. But before we see the real code, we can't reach any conclusion.

I am quite sure this is much simpler, and faster.

Main reason is because you only need to handle the write journal for
the current obj being recoverd. So you can also simply cache writes 
to ram (max 4MB ram usage). 

And you do not delay any request.

> This scheme is kind of radical change, so before this, we can try other means
> that need less work, such as beforehand mentioned only recovery of
> requested objects instead of all the objects in a given time window, which is
> supposed to be useful for maintenance work.

I thing you can dump about 50% of the recovery code that way (the complex parts), 
and you will get much better behavior (no delays). But to show that claim, someone need to implement 
a prototype, and I currently do not have the time to do that :-/

- Dietmar