[sheepdog-users] monitor cluster to avoid corruption

[Liu Yuan]

On 12/18/2012 05:41 PM, Valerio Pachera wrote:
> collie vdi list
>   Name        Id    Size    Used  Shared    Creation time   VDI id  Copies  Tag
>   test         1   10 GB  1.5 GB  0.0 MB 2012-12-18 10:29   7c2b25     2
> 
> collie node info
> Id      Size    Used    Use%
>  0      982 MB  892 MB   90%
>  1      982 MB  672 MB   68%
>  2      10.0 GB 1.5 GB   15%
> Total   12 GB   3.0 GB   25%
> Total virtual image size        10 GB
> 
> 
> collie node list
> M   Id   Host:Port         V-Nodes       Zone
> -    0   192.168.2.41:7000      16  688040128
> -    1   192.168.2.42:7000      16  704817344
> -    2   192.168.2.43:7000      161  721594560

I think this is not easy to solve with a small change. To take it
concrete, let's start with current example. Sheepdog is taught to
distribute the *whole* image 'test' evenly on weighted node. So we write
full of test to 10GB, then all nodes will share the correct portion of
data. The problem is, it seldomly happens in real world and before it
reaches full, its data will never get evenly distributed.

For a quick thought, I think probably the right fix is to allocate
virtual nodes *dynamically* based on the *actual data written*, instead
of statically based on the node space.

Thanks,
Yuan