[Sheepdog] support object recovery

[Piavlo]

MORITA Kazutaka wrote:
> On Fri, Jan 22, 2010 at 11:01 AM, Piavlo <piavka at cs.bgu.ac.il> wrote:
>   
>> some more questions regarding current recovery implementation:
>> With current recovery what is supposed to happen then one of the  nodes
>> goes down with --copies=N redundancy:
>> 1) Are the vm blocks get replicated to other nodes so that there are
>> still N copies of each vm block in active cluster?
>>     
>
> Yes.
>
>   
>> If yes does this happen in the background, it the recovery replication
>> triggered automatically?
>>     
>
> Yes, it is triggered automatically in the background.
>
>   
>> 2)Same questions regarding the vdi meta directories.
>>     
>
> Yes, too.
> One of sheepdog goals is `autonomously managed storage system'..
>
>   

 I've just tried simple test, with 3 nodes and copies=2
After cluster initialization and single image creation and booting it.
There were two nodes with the vdi metadirectory and third node with just
vm blocks and no metadata.
Next i stopped collie on one of the two nodes - which had the vdi
metadirectory.
After this the vm blocks were recovered/redistributed evenly between the
remaining two nodes.
However the vdi metadirectory was not created/recovered on the third node :(


Thanks
Alex