The problem is repeatable. I've been also rebooting the hosts and formatting the cluster twice. root at test004:~# time qemu-img convert -f qcow2 wheezy_template.qcow2 sheepdog:wheezy_template real 0m38.084s user 0m6.756s sys 0m6.456s root at test004:~# time qemu-img convert -f qcow2 -t directsync wheezy_template.qcow2 sheepdog:wheezy_template2 real 3m4.104s user 0m6.192s sys 0m5.900s root at test004:~# time qemu-img convert -f qcow2 wheezy_template.qcow2 sheepdog:wheezy_template3 real 0m38.218s user 0m6.620s sys 0m6.456s qemu-system-x86_64 -name wheezy_template -enable-kvm -drive file=sheepdog:wheezy_template,cache=writeback,if=virtio (do not work) qemu-system-x86_64 -name wheezy_template2 -enable-kvm -drive file=sheepdog:wheezy_template2,cache=directsync (works) qemu-system-x86_64 -name wheezy_template3 -enable-kvm -drive file=sheepdog:wheezy_template3,cache=directsync (work) I shutdown wheezy_template3 and use cache qemu-system-x86_64 -name wheezy_template3 -enable-kvm -drive file=sheepdog:wheezy_template3,cache=writeback (do not work) sheep.log Jun 25 17:17:19 [main] main(797) shutdown Jun 26 08:32:22 [main] md_add_disk(161) /mnt/sheep/dsk02, nr 1 Jun 26 08:32:22 [main] send_join_request(1095) IPv4 ip:192.168.2.44 port:7000 Jun 26 08:32:22 [main] for_each_object_in_stale(403) /mnt/sheep/dsk02/.stale Jun 26 08:32:22 [main] check_host_env(405) Allowed core file size 0, suggested unlimited Jun 26 08:32:22 [main] main(790) sheepdog daemon (version 0.6.0_49_g00d4b07) started Jun 26 08:32:22 [main] update_cluster_info(871) status = 4, epoch = 8, finished: 0 Jun 26 08:32:26 [main] sd_check_join_cb(1055) 192.168.2.45:7000: ret = 0x0, cluster_status = 0x4 Jun 26 08:32:26 [main] update_cluster_info(871) status = 4, epoch = 8, finished: 1 Jun 26 08:32:29 [main] sd_check_join_cb(1055) 192.168.2.47:7000: ret = 0x0, cluster_status = 0x1 Jun 26 08:32:29 [main] update_cluster_info(871) status = 1, epoch = 8, finished: 1 Jun 26 08:34:27 [deletion] objlist_deletion_work(186) VDI (a34c67) is still in use, can not be deleted Jun 26 09:41:10 [main] for_each_object_in_stale(403) /mnt/sheep/dsk02/.stale Jun 26 09:41:10 [main] remove_epoch(230) failed to remove /mnt/sheep/meta/epoch/00000005: Operation not permitted Jun 26 09:41:10 [main] remove_epoch(230) failed to remove /mnt/sheep/meta/epoch/00000007: Operation not permitted Jun 26 11:34:46 [deletion] objlist_deletion_work(186) VDI (35af80) is still in use, can not be deleted Jun 26 11:34:51 [deletion] objlist_deletion_work(186) VDI (64be9f) is still in use, can not be deleted Jun 26 12:45:45 [main] for_each_object_in_stale(403) /mnt/sheep/dsk02/.stale Jun 26 18:20:46 [main] main(797) shutdown Jul 10 16:03:03 [main] md_add_disk(146) /mnt/sheep/dsk02, nr 1 Jul 10 16:03:03 [main] send_join_request(928) IPv4 ip:192.168.2.44 port:7000 Jul 10 16:03:03 [main] for_each_object_in_stale(388) /mnt/sheep/dsk02/.stale Jul 10 16:03:03 [main] main(780) sheepdog daemon (version 0.6.0_94_ge70a103) started Jul 10 16:49:27 [main] for_each_object_in_stale(388) /mnt/sheep/dsk02/.stale Jul 10 17:03:56 [main] main(787) shutdown Jul 11 08:36:19 [main] md_add_disk(146) /mnt/sheep/dsk02, nr 1 Jul 11 08:36:19 [main] send_join_request(928) IPv4 ip:192.168.2.44 port:7000 Jul 11 08:36:19 [main] for_each_object_in_stale(388) /mnt/sheep/dsk02/.stale Jul 11 08:36:19 [main] main(780) sheepdog daemon (version 0.6.0_94_ge70a103) started Jul 12 12:21:32 [gway 27528] push_cache_object(516) failed to push object Object is read-only Jul 12 12:21:32 [main] modify_event(151) event info for fd 23 not found Jul 12 18:29:07 [main] main(787) shutdown Jul 15 09:55:14 [main] md_add_disk(146) /mnt/sheep/dsk02, nr 1 Jul 15 09:55:14 [main] send_join_request(817) IPv4 ip:192.168.2.44 port:7000 Jul 15 09:55:14 [main] for_each_object_in_stale(388) /mnt/sheep/dsk02/.stale Jul 15 09:55:17 [main] main(871) sheepdog daemon (version 0.6.0_127_g085568a) started Jul 15 09:55:18 [main] recover_object_main(620) done:1 count:118447, oid:c8d12800006f80 Jul 15 09:55:18 [main] recover_object_main(620) done:2 count:118447, oid:c8d12800032a4c Jul 15 09:55:18 [main] recover_object_main(620) done:3 count:118447, oid:c8d1280001413d Jul 15 09:55:18 [main] recover_object_main(620) done:4 count:118447, oid:c8d12800015f3e Jul 15 09:55:18 [main] recover_object_main(620) done:5 count:118447, oid:c8d128000031cb |