Hi, I was doing some tests about cache and write speed. I started the cluster with 3 nodes using md. sheep -w size=20000 /sheep/meta,/sheep/sda,/sheep/sdb Then added a 4th node that is not using md. sheep -w size=20000 /sheep/sda Done some tests with a guest. Stopped the cluster. Restarted sheep with the same options, just a smaller cache. Then one of the nodes (sheepdog004; id 0) is showing only 2 vdi instead of 4. root at test004:~# collie vdi list Name Id Size Used Shared Creation time VDI id Copies Tag empty02 0 10 GB 10 GB 0.0 MB 2013-05-27 12:27 366058 2 squeeze 0 10 GB 5.2 GB 0.0 MB 2013-05-27 12:10 a34c67 2 root at test005:~# collie vdi list Name Id Size Used Shared Creation time VDI id Copies Tag empty02 0 10 GB 10 GB 0.0 MB 2013-05-27 12:27 366058 2 empty03 0 10 GB 10 GB 0.0 MB 2013-05-27 12:28 36620b 2 empty01 0 10 GB 10 GB 0.0 MB 2013-05-27 12:27 366571 2 squeeze 0 10 GB 5.2 GB 0.0 MB 2013-05-27 12:10 a34c67 2 root at test005:~# collie node md info --all Id Used Free Path Node 0: 0 15 GB 35 GB /sheep/sda 1 1.0 GB 3.6 GB /sheep/sdb Node 1: 0 17 GB 33 GB /sheep/sda/obj Node 2: 0 19 GB 31 GB /sheep/sda 1 1.4 GB 3.2 GB /sheep/sdb Node 3: 0 16 GB 34 GB /sheep/sda 1 1.3 GB 3.3 GB /sheep/sdb root at test005:~# collie node info Id Size Used Use% 0 55 GB 16 GB 29% 1 50 GB 17 GB 33% 2 55 GB 21 GB 37% 3 55 GB 17 GB 30% Total 214 GB 70 GB 32% Total virtual image size 40 GB root at test004:~# tail -15 /sheep/meta/sheep.log May 27 12:33:09 [main] recover_object_main(612) done:3728 count:3731, oid:36620b00000602 May 27 12:33:09 [main] recover_object_main(612) done:3729 count:3731, oid:36657100000864 May 27 12:33:09 [main] recover_object_main(612) done:3730 count:3731, oid:36620b000006d4 May 27 12:33:09 [main] recover_object_main(612) done:3731 count:3731, oid:36620b00000952 May 27 13:47:44 [main] main(793) shutdown May 27 13:48:10 [main] md_add_disk(161) /sheep/sda, nr 1 May 27 13:48:10 [main] md_add_disk(161) /sheep/sdb, nr 2 May 27 13:48:10 [main] send_join_request(1100) IPv4 ip:192.168.2.44 port:7000 May 27 13:48:10 [main] for_each_object_in_stale(387) /sheep/sda/.stale May 27 13:48:10 [main] for_each_object_in_stale(387) /sheep/sdb/.stale May 27 13:48:10 [main] check_host_env(396) WARN: Allowed open files 1024 too small, suggested 1024000 May 27 13:48:10 [main] check_host_env(405) Allowed core file size 0, suggested unlimited May 27 13:48:10 [main] main(786) sheepdog daemon (version 0.5.5_432_g2e7656b) started May 27 13:48:10 [main] update_cluster_info(879) status = 4, epoch = 2, finished: 0 May 27 13:49:05 [main] update_cluster_info(879) status = 1, epoch = 2, finished: 1 root at test004:~# collie cluster info Cluster status: running, auto-recovery enabled Cluster created at Mon May 27 12:10:54 2013 Epoch Time Version 2013-05-27 12:33:08 2 [192.168.2.44:7000, 192.168.2.45:7000, 192.168.2.46:7000, 192.168.2.47:7000] 2013-05-27 12:10:54 1 [192.168.2.44:7000, 192.168.2.46:7000, 192.168.2.47:7000] sheep 0.5.5_432_g2e7656b |