[sheepdog] [PATCH 0/2] don't remove objects when they might be necessary

[MORITA Kazutaka]

At Sat, 25 Aug 2012 20:57:10 +0200,
Bastian Scholz wrote:
> 
> 5 Sheeps in 5 zones (0-4) and a vdi with 2 copies and
> two object.
> 
> Obj 0 is on sheep 0 and 4
> Obj 1 is on sheep 3 and 4
> 
> Now I kill sheep 3 and 4, Obj 0 can be recovered,
> but Obj 1 will fail...
> 
> What happens to the VM in this situation, Obj 0
> can be read and write normally and Obj 1 will fail
> and give a kind of I/O Error to the VM, I guess?

Yes.

> 
> And when I restart sheep 3 and/or 4 later, Obj 1
> will be recovered (from the stale directory) so the
> VM can access it normally?

Yes.

I've confirmed it on my environment:

# run 5 sheeps
 $ sheep /store/0 -z 0 -p 7000
 $ sheep /store/1 -z 1 -p 7001
 $ sheep /store/2 -z 2 -p 7002
 $ sheep /store/3 -z 3 -p 7003
 $ sheep /store/4 -z 4 -p 7004
 $ sleep 1

# format with 2 copies
 $ collie cluster format -c 2
 using backend farm store

# create a image
 $ collie vdi create test 100M
 $ collie vdi write test < /dev/urandom
 $ collie vdi read test | md5sum
 895f6c27bf3b1cbcff18b7902049b721  -

# kill sheep 3 and 4
 $ pkill -f 'sheep /store/3'
 $ pkill -f 'sheep /store/4'

# cannot access obj 4 because it was stored on sheep 3 and 4
 $ collie vdi read test | md5sum
 Failed to read object 7c2b2500000004 No object found
 Failed to read VDI
 f622ee7c8f4a44538424ff46808b148d  -

# after restart sheep 3 and 4, obj 4 is recovered
 $ sheep /store/3 -z 3 -p 7003
 $ sheep /store/4 -z 4 -p 7004
 $ collie vdi read test | md5sum
 895f6c27bf3b1cbcff18b7902049b721  -

# Actually, obj 4 is stored on sheep 3 and 4
 $ collie vdi object -i 4 test
 Looking for the object 0x7c2b2500000004 (the inode vid 0x7c2b25 idx 4) with 5 nodes
 
 127.0.0.1:7000 doesn't have the object
 127.0.0.1:7001 doesn't have the object
 127.0.0.1:7002 doesn't have the object
 127.0.0.1:7003 has the object (should be 2 copies)
 127.0.0.1:7004 has the object (should be 2 copies)


Thanks,

Kazutaka