[sheepdog] Redundancy policy via iSCSI

hujianyang hujianyang at huawei.com
Tue Feb 3 09:54:00 CET 2015

Hi Hitoshi,

Thanks for your help!

Try to answer your questions.

On 2015/2/3 16:02, Hitoshi Mitake wrote:
> Hmm, seems strange. For diagnosing, I have some questions:
> 1. Can you see any error messages in log files of sheep?

log while performing dd on a formatted filesystem:

Feb 04 00:16:05   INFO [main] rx_main(830) req=0x7faf6c0008b0, fd=289, client=, op=DEL_VDI, data=(not string)
Feb 04 00:16:05   INFO [main] run_vid_gc(2147) all members of the family (root: 6e7762) are deleted
Feb 04 00:16:19   INFO [main] cluster_release_vdi_main(1431) node: IPv4 ip: port:7000 is unlocking VDI (type: shared): 6e7762
Feb 04 00:16:19  ERROR [main] vdi_unlock(717) no vdi state entry of 6e7762 found
Feb 04 00:16:41   INFO [main] rx_main(830) req=0x1b7fbf0, fd=221, client=, op=NEW_VDI, data=(not string)
Feb 04 00:16:41   INFO [main] post_cluster_new_vdi(133) req->vdi.base_vdi_id: 0, rsp->vdi.vdi_id: 6e7762
Feb 04 00:16:41   INFO [main] tx_main(882) req=0x1b7fbf0, fd=221, client=, op=NEW_VDI, result=00
Feb 04 00:17:21   INFO [main] cluster_lock_vdi_main(1408) node: IPv4 ip: port:7000 is locking VDI (type: shared): 6e776

> 2. Could you provide lists of obj/ directories of sheep servers?

original case: dd if=/dev/random of=/dev/sdg bs=2M

linux-rme9:/mnt/store/0/obj # ls
006e776200000000  006e77620000003d  006e776200000064  006e77620000008a  006e7762000000b2  006e7762000000db
006e776200000003  006e776200000044  006e776200000068  006e77620000008b  006e7762000000b5  006e7762000000dd
006e776200000008  006e776200000046  006e77620000006a  006e77620000008c  006e7762000000b6  006e7762000000e0
006e776200000010  006e776200000047  006e77620000006c  006e77620000008f  006e7762000000bc  006e7762000000e5
006e776200000012  006e776200000048  006e77620000006f  006e776200000091  006e7762000000bd  006e7762000000e8
006e77620000001b  006e77620000004a  006e776200000073  006e776200000094  006e7762000000c0  006e7762000000ea
006e77620000001c  006e77620000004b  006e776200000074  006e776200000095  006e7762000000c3  006e7762000000f2
006e776200000020  006e77620000004e  006e776200000078  006e77620000009a  006e7762000000c4  006e7762000000f4
006e776200000021  006e776200000054  006e77620000007c  006e77620000009e  006e7762000000c5  006e7762000000f5
006e776200000023  006e776200000056  006e77620000007d  006e7762000000a4  006e7762000000c7  006e7762000000f6
006e776200000029  006e776200000057  006e776200000080  006e7762000000a6  006e7762000000ca  006e7762000000f8
006e77620000002c  006e776200000058  006e776200000081  006e7762000000a7  006e7762000000d2  006e7762000000fd
006e776200000034  006e77620000005a  006e776200000085  006e7762000000aa  006e7762000000d3  806e776200000000
006e776200000035  006e77620000005f  006e776200000086  006e7762000000ac  006e7762000000d6
006e776200000037  006e776200000060  006e776200000087  006e7762000000ad  006e7762000000d7
006e776200000039  006e776200000063  006e776200000088  006e7762000000b0  006e7762000000d8

linux-2hp8:/mnt/store/0/obj # ls
006e776200000001  006e77620000001a  006e77620000004c  006e77620000007a  006e7762000000a5  006e7762000000d4
006e776200000002  006e77620000001e  006e77620000004d  006e77620000007b  006e7762000000ab  006e7762000000d9
006e776200000007  006e776200000024  006e776200000051  006e77620000007f  006e7762000000b1  006e7762000000da
006e776200000009  006e776200000025  006e776200000055  006e776200000082  006e7762000000b4  006e7762000000de
006e77620000000a  006e776200000026  006e776200000059  006e776200000084  006e7762000000b9  006e7762000000df
006e77620000000b  006e77620000002a  006e77620000005d  006e776200000089  006e7762000000ba  006e7762000000e2
006e77620000000d  006e77620000002f  006e77620000005e  006e77620000008e  006e7762000000bb  006e7762000000e7
006e77620000000e  006e776200000032  006e776200000061  006e776200000090  006e7762000000be  006e7762000000e9
006e77620000000f  006e776200000033  006e776200000067  006e776200000092  006e7762000000c1  006e7762000000ee
006e776200000011  006e776200000038  006e776200000069  006e776200000093  006e7762000000c2  006e7762000000ef
006e776200000013  006e77620000003b  006e77620000006b  006e776200000097  006e7762000000c8  006e7762000000f0
006e776200000014  006e77620000003e  006e77620000006e  006e776200000099  006e7762000000c9  006e7762000000f1
006e776200000016  006e776200000045  006e776200000071  006e77620000009d  006e7762000000cb  006e7762000000fa
006e776200000017  006e776200000049  006e776200000077  006e7762000000a3  006e7762000000cc  006e7762000000fc

linux-cjck:/mnt/store/0/obj # ls
006e776200000004  006e77620000002e  006e776200000053  006e77620000008d  006e7762000000b7  006e7762000000e6
006e776200000005  006e776200000030  006e77620000005b  006e776200000096  006e7762000000b8  006e7762000000eb
006e776200000006  006e776200000031  006e77620000005c  006e776200000098  006e7762000000bf  006e7762000000ec
006e77620000000c  006e776200000036  006e776200000062  006e77620000009b  006e7762000000c6  006e7762000000ed
006e776200000015  006e77620000003a  006e776200000065  006e77620000009c  006e7762000000cd  006e7762000000f3
006e776200000018  006e77620000003c  006e776200000066  006e77620000009f  006e7762000000ce  006e7762000000f7
006e776200000019  006e77620000003f  006e77620000006d  006e7762000000a0  006e7762000000cf  006e7762000000f9
006e77620000001d  006e776200000040  006e776200000070  006e7762000000a1  006e7762000000d0  006e7762000000fb
006e77620000001f  006e776200000041  006e776200000072  006e7762000000a2  006e7762000000d1  006e7762000000fe
006e776200000022  006e776200000042  006e776200000075  006e7762000000a8  006e7762000000d5  006e7762000000ff
006e776200000027  006e776200000043  006e776200000076  006e7762000000a9  006e7762000000dc
006e776200000028  006e77620000004f  006e776200000079  006e7762000000ae  006e7762000000e1
006e77620000002b  006e776200000050  006e77620000007e  006e7762000000af  006e7762000000e3
006e77620000002d  006e776200000052  006e776200000083  006e7762000000b3  006e7762000000e4

> 3. Is this reproducible even you make file system on the iSCSI target
>    and put data on the file system?

Same problem.

# mount -t ext4 /dev/sdg /mnt
# dd if=/dev/random of=/mnt/test
dd: writing to `/mnt/test': No space left on device
0+12733259 records in
1963296+0 records out
1005207552 bytes (1.0 GB) copied, 517.707 s, 1.9 MB/s

linux-rme9:/mnt # dog node info
Id	Size	Used	Avail	Use%
 0	261 GB	368 MB	260 GB	  0%
 1	261 GB	336 MB	261 GB	  0%
 2	261 GB	312 MB	261 GB	  0%
Total	783 GB	1016 MB	782 GB	  0%

Total virtual image size	1.0 GB

> 4. Is this reproducible even you append oflag=sync to the option of dd?

Too slow to test...

# dd if=/dev/random of=/dev/sdg bs=2M oflag=sync
^C^C0+97541 records in
0+97541 records out
7697935 bytes (7.7 MB) copied, 590.57 s, 13.0 kB/s

> Additionaly, could you provide options of sheep?

I'm not clear with this. Just run "sheep /mnt/store/0 -z 0 -p 7000"
on each host.

Corosync is used in my environment.


More information about the sheepdog mailing list