[sheepdog] Redundancy policy via iSCSI
hujianyang
hujianyang at huawei.com
Tue Feb 3 09:54:00 CET 2015
Hi Hitoshi,
Thanks for your help!
Try to answer your questions.
On 2015/2/3 16:02, Hitoshi Mitake wrote:
>
> Hmm, seems strange. For diagnosing, I have some questions:
>
> 1. Can you see any error messages in log files of sheep?
log while performing dd on a formatted filesystem:
Feb 04 00:16:05 INFO [main] rx_main(830) req=0x7faf6c0008b0, fd=289, client=127.0.0.1:57302, op=DEL_VDI, data=(not string)
Feb 04 00:16:05 INFO [main] run_vid_gc(2147) all members of the family (root: 6e7762) are deleted
Feb 04 00:16:19 INFO [main] cluster_release_vdi_main(1431) node: IPv4 ip:130.1.0.147 port:7000 is unlocking VDI (type: shared): 6e7762
Feb 04 00:16:19 ERROR [main] vdi_unlock(717) no vdi state entry of 6e7762 found
Feb 04 00:16:41 INFO [main] rx_main(830) req=0x1b7fbf0, fd=221, client=127.0.0.1:57306, op=NEW_VDI, data=(not string)
Feb 04 00:16:41 INFO [main] post_cluster_new_vdi(133) req->vdi.base_vdi_id: 0, rsp->vdi.vdi_id: 6e7762
Feb 04 00:16:41 INFO [main] tx_main(882) req=0x1b7fbf0, fd=221, client=127.0.0.1:57306, op=NEW_VDI, result=00
Feb 04 00:17:21 INFO [main] cluster_lock_vdi_main(1408) node: IPv4 ip:130.1.0.147 port:7000 is locking VDI (type: shared): 6e776
> 2. Could you provide lists of obj/ directories of sheep servers?
original case: dd if=/dev/random of=/dev/sdg bs=2M
130.1.0.147
linux-rme9:/mnt/store/0/obj # ls
006e776200000000 006e77620000003d 006e776200000064 006e77620000008a 006e7762000000b2 006e7762000000db
006e776200000003 006e776200000044 006e776200000068 006e77620000008b 006e7762000000b5 006e7762000000dd
006e776200000008 006e776200000046 006e77620000006a 006e77620000008c 006e7762000000b6 006e7762000000e0
006e776200000010 006e776200000047 006e77620000006c 006e77620000008f 006e7762000000bc 006e7762000000e5
006e776200000012 006e776200000048 006e77620000006f 006e776200000091 006e7762000000bd 006e7762000000e8
006e77620000001b 006e77620000004a 006e776200000073 006e776200000094 006e7762000000c0 006e7762000000ea
006e77620000001c 006e77620000004b 006e776200000074 006e776200000095 006e7762000000c3 006e7762000000f2
006e776200000020 006e77620000004e 006e776200000078 006e77620000009a 006e7762000000c4 006e7762000000f4
006e776200000021 006e776200000054 006e77620000007c 006e77620000009e 006e7762000000c5 006e7762000000f5
006e776200000023 006e776200000056 006e77620000007d 006e7762000000a4 006e7762000000c7 006e7762000000f6
006e776200000029 006e776200000057 006e776200000080 006e7762000000a6 006e7762000000ca 006e7762000000f8
006e77620000002c 006e776200000058 006e776200000081 006e7762000000a7 006e7762000000d2 006e7762000000fd
006e776200000034 006e77620000005a 006e776200000085 006e7762000000aa 006e7762000000d3 806e776200000000
006e776200000035 006e77620000005f 006e776200000086 006e7762000000ac 006e7762000000d6
006e776200000037 006e776200000060 006e776200000087 006e7762000000ad 006e7762000000d7
006e776200000039 006e776200000063 006e776200000088 006e7762000000b0 006e7762000000d8
130.1.0.148
linux-2hp8:/mnt/store/0/obj # ls
006e776200000001 006e77620000001a 006e77620000004c 006e77620000007a 006e7762000000a5 006e7762000000d4
006e776200000002 006e77620000001e 006e77620000004d 006e77620000007b 006e7762000000ab 006e7762000000d9
006e776200000007 006e776200000024 006e776200000051 006e77620000007f 006e7762000000b1 006e7762000000da
006e776200000009 006e776200000025 006e776200000055 006e776200000082 006e7762000000b4 006e7762000000de
006e77620000000a 006e776200000026 006e776200000059 006e776200000084 006e7762000000b9 006e7762000000df
006e77620000000b 006e77620000002a 006e77620000005d 006e776200000089 006e7762000000ba 006e7762000000e2
006e77620000000d 006e77620000002f 006e77620000005e 006e77620000008e 006e7762000000bb 006e7762000000e7
006e77620000000e 006e776200000032 006e776200000061 006e776200000090 006e7762000000be 006e7762000000e9
006e77620000000f 006e776200000033 006e776200000067 006e776200000092 006e7762000000c1 006e7762000000ee
006e776200000011 006e776200000038 006e776200000069 006e776200000093 006e7762000000c2 006e7762000000ef
006e776200000013 006e77620000003b 006e77620000006b 006e776200000097 006e7762000000c8 006e7762000000f0
006e776200000014 006e77620000003e 006e77620000006e 006e776200000099 006e7762000000c9 006e7762000000f1
006e776200000016 006e776200000045 006e776200000071 006e77620000009d 006e7762000000cb 006e7762000000fa
006e776200000017 006e776200000049 006e776200000077 006e7762000000a3 006e7762000000cc 006e7762000000fc
130.1.0.149
linux-cjck:/mnt/store/0/obj # ls
006e776200000004 006e77620000002e 006e776200000053 006e77620000008d 006e7762000000b7 006e7762000000e6
006e776200000005 006e776200000030 006e77620000005b 006e776200000096 006e7762000000b8 006e7762000000eb
006e776200000006 006e776200000031 006e77620000005c 006e776200000098 006e7762000000bf 006e7762000000ec
006e77620000000c 006e776200000036 006e776200000062 006e77620000009b 006e7762000000c6 006e7762000000ed
006e776200000015 006e77620000003a 006e776200000065 006e77620000009c 006e7762000000cd 006e7762000000f3
006e776200000018 006e77620000003c 006e776200000066 006e77620000009f 006e7762000000ce 006e7762000000f7
006e776200000019 006e77620000003f 006e77620000006d 006e7762000000a0 006e7762000000cf 006e7762000000f9
006e77620000001d 006e776200000040 006e776200000070 006e7762000000a1 006e7762000000d0 006e7762000000fb
006e77620000001f 006e776200000041 006e776200000072 006e7762000000a2 006e7762000000d1 006e7762000000fe
006e776200000022 006e776200000042 006e776200000075 006e7762000000a8 006e7762000000d5 006e7762000000ff
006e776200000027 006e776200000043 006e776200000076 006e7762000000a9 006e7762000000dc
006e776200000028 006e77620000004f 006e776200000079 006e7762000000ae 006e7762000000e1
006e77620000002b 006e776200000050 006e77620000007e 006e7762000000af 006e7762000000e3
006e77620000002d 006e776200000052 006e776200000083 006e7762000000b3 006e7762000000e4
> 3. Is this reproducible even you make file system on the iSCSI target
> and put data on the file system?
Same problem.
# mount -t ext4 /dev/sdg /mnt
# dd if=/dev/random of=/mnt/test
dd: writing to `/mnt/test': No space left on device
0+12733259 records in
1963296+0 records out
1005207552 bytes (1.0 GB) copied, 517.707 s, 1.9 MB/s
linux-rme9:/mnt # dog node info
Id Size Used Avail Use%
0 261 GB 368 MB 260 GB 0%
1 261 GB 336 MB 261 GB 0%
2 261 GB 312 MB 261 GB 0%
Total 783 GB 1016 MB 782 GB 0%
Total virtual image size 1.0 GB
> 4. Is this reproducible even you append oflag=sync to the option of dd?
Too slow to test...
# dd if=/dev/random of=/dev/sdg bs=2M oflag=sync
^C^C0+97541 records in
0+97541 records out
7697935 bytes (7.7 MB) copied, 590.57 s, 13.0 kB/s
> Additionaly, could you provide options of sheep?
I'm not clear with this. Just run "sheep /mnt/store/0 -z 0 -p 7000"
on each host.
Corosync is used in my environment.
Thanks,
Hu
More information about the sheepdog
mailing list