[sheepdog-users] i/o stuck on recovery
terletskiy at emu.ru
Sun Sep 14 17:57:54 CEST 2014
Today I've been restarted sheep process on one of my nodes with data in
test cluster and noticed that I/O stuck on all VMs for almost 10 mins.
Initially I've tried to do "service sheepdog restart" to change sheep
settings(disable -D flag), but process doesn't started because of
"previous session in zookeeper found" or so. Waited for 30 secs,
executed "service sheepdog start", sheep started, "dog node recovery"
showed my 3 nodes recovering. I had "pv /dev/vda > /dev/null" in
terminal on one of my VMs. Right after restarting sheep pv stucked.
After 5 minutes when "dog node recovery" showed me only one node
recovering pv still was stucked. And only after 3-4 minutes later pv
continued to reading from disk. Is it normal behavior?
Have 3 nodes with data in the cluster, replication 2x, about 7Tb data
assuming to "dog node info". Sheep 0.8.0, options:
-c zookeeper:184.108.40.206:2181,220.127.116.11:2181,18.104.22.168:2181,timeout=30 -i
host=22.214.171.124,port=7001 -y 126.96.36.199
Additionally one node have -D flag in options.
More information about the sheepdog-users