[Sheepdog] Configuring simple cluster on CentOS 5.5 x86_64
Steven Dake
sdake at redhat.com
Thu Oct 21 16:57:14 CEST 2010
On 10/21/2010 07:47 AM, Yuriy Kohut wrote:
> As I know, fuse is ready to use filesystem. I would like to have block device mapped into existing VDI.
Yes, the fuse fs exports all VDIs into a block devices. Not sure what
the status of this work is though.
Regards
-steve
> ---
> Yura
>
> On Oct 21, 2010, at 5:33 PM, Steven Dake wrote:
>
>> Someone on the list was working on a fuse sheepdog daemon that exported
>> the vdi's via the fs.
>>
>> Regards
>> -steve
>>
>> On 10/21/2010 07:12 AM, Yuriy Kohut wrote:
>>> Hi,
>>>
>>> Is there a way translate Sheepdog VDIs into block devices on the boxes which belong to the cluster without using iSCSI ?
>>> ---
>>> Yura
>>>
>>> On Oct 20, 2010, at 2:23 PM, Yuriy Kohut wrote:
>>>
>>>> One quick question.
>>>> Is there any way to save/store created targets and logicalunits with 'tgtadm' ?
>>>> They all are "lost" after machine reboot.
>>>>
>>>> ---
>>>> Yura
>>>>
>>>> On Oct 19, 2010, at 6:40 PM, Yuriy Kohut wrote:
>>>>
>>>>> Got it working.
>>>>>
>>>>> The next step is to try all that on real hardware 3 node cluster.
>>>>>
>>>>>
>>>>> Thank you for help.
>>>>> ---
>>>>> Yura
>>>>>
>>>>> On Oct 19, 2010, at 3:02 PM, MORITA Kazutaka wrote:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> Your sheep.log says
>>>>>>
>>>>>> Oct 19 05:59:06 send_message(169) failed to send message, 2
>>>>>>
>>>>>> This means that the sheep daemon failed to communicate with corosync.
>>>>>> Unfortunatelly, I've never seen such an error...
>>>>>>
>>>>>> Try following things:
>>>>>> - restart corosync daemon
>>>>>> - disable iptable and restart corosync
>>>>>> - disable selinux and restart corosync
>>>>>>
>>>>>> Did sheepdog work fine when you tested it on debian?
>>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>> Kazutaka
>>>>>>
>>>>>> On 2010/10/19 19:56, Yuriy Kohut wrote:
>>>>>>> Attached.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> ------------------------------------------------------------------------
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Please feel free to kick me if anything else required.
>>>>>>>
>>>>>>> ---
>>>>>>> Yura
>>>>>>>
>>>>>>> On Oct 19, 2010, at 1:45 PM, MORITA Kazutaka wrote:
>>>>>>>
>>>>>>>> Could you send me a sheep.log in the store directory?
>>>>>>>> It would be helpful for debugging.
>>>>>>>>
>>>>>>>> Kazutaka
>>>>>>>>
>>>>>>>> On 2010/10/19 19:16, Yuriy Kohut wrote:
>>>>>>>>> The patch doesn't help.
>>>>>>>>>
>>>>>>>>> Probably I'm doing something wrong. but the following operation won't finish:
>>>>>>>>> # tgtadm --op new --mode logicalunit --tid 1 --lun 1 -b test0 --bstype sheepdog
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Attached please find the the operation/command strace log archived:
>>>>>>>>> strace.log.tar.gz
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> ------------------------------------------------------------------------
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Please advise.
>>>>>>>>>
>>>>>>>>> Thank you
>>>>>>>>> ---
>>>>>>>>> Yura
>>>>>>>>>
>>>>>>>>> On Oct 19, 2010, at 11:52 AM, Yuriy Kohut wrote:
>>>>>>>>>
>>>>>>>>>> Hi,
>>>>>>>>>>
>>>>>>>>>> Sure. I'll let you know results.
>>>>>>>>>>
>>>>>>>>>> Thank you.
>>>>>>>>>> ---
>>>>>>>>>> Yura
>>>>>>>>>>
>>>>>>>>>> On Oct 19, 2010, at 11:46 AM, MORITA Kazutaka wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi,
>>>>>>>>>>>
>>>>>>>>>>> At Fri, 15 Oct 2010 17:33:18 +0300,
>>>>>>>>>>> Yuriy Kohut wrote:
>>>>>>>>>>>> One more new issue with TGTd.
>>>>>>>>>>>>
>>>>>>>>>>>> Initially we have one sheepdog vdi (on which we would like to create iscsi unit) and no tgt targets/units:
>>>>>>>>>>>> [root at centos ~]# tgtadm --op show --mode target
>>>>>>>>>>>> [root at centos ~]# collie vdi list
>>>>>>>>>>>> name id size used shared creation time vdi id
>>>>>>>>>>>> ------------------------------------------------------------------
>>>>>>>>>>>> test0 1 4.0 GB 4.0 GB 0.0 MB 2010-10-15 17:55 fd34af
>>>>>>>>>>>> [root at centos ~]#
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Creating new target:
>>>>>>>>>>>> [root at centos ~]# tgtadm --op new --mode target --tid 1 -T some.vps:disk0
>>>>>>>>>>>> [root at centos ~]# tgtadm --op show --mode target
>>>>>>>>>>>> Target 1: some.vps:disk0
>>>>>>>>>>>> System information:
>>>>>>>>>>>> Driver: iscsi
>>>>>>>>>>>> State: ready
>>>>>>>>>>>> I_T nexus information:
>>>>>>>>>>>> LUN information:
>>>>>>>>>>>> LUN: 0
>>>>>>>>>>>> Type: controller
>>>>>>>>>>>> SCSI ID: IET 00010000
>>>>>>>>>>>> SCSI SN: beaf10
>>>>>>>>>>>> Size: 0 MB
>>>>>>>>>>>> Online: Yes
>>>>>>>>>>>> Removable media: No
>>>>>>>>>>>> Readonly: No
>>>>>>>>>>>> Backing store type: null
>>>>>>>>>>>> Backing store path: None
>>>>>>>>>>>> Backing store flags:
>>>>>>>>>>>> Account information:
>>>>>>>>>>>> ACL information:
>>>>>>>>>>>> [root at centos ~]#
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Try to create new logicalunit on existing tgt target and sheepdog vdi:
>>>>>>>>>>>> [root at centos ~]# tgtadm --op new --mode logicalunit --tid 1 --lun 1 -b test0 --bstype sheepdog
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> But the process never ends.
>>>>>>>>>>>> Please advise ...
>>>>>>>>>>> Thanks for your report.
>>>>>>>>>>>
>>>>>>>>>>> Can you try the following patch I sent minutes ago?
>>>>>>>>>>> http://lists.wpkg.org/pipermail/sheepdog/2010-October/000741.html
>>>>>>>>>>>
>>>>>>>>>>> Thanks,
>>>>>>>>>>>
>>>>>>>>>>> Kazutaka
>>>>>>>>>>>
>>>>>>>>>>>> ---
>>>>>>>>>>>> Yura
>>>>>>>>>>>>
>>>>>>>>>>>> On Oct 15, 2010, at 4:55 PM, Yuriy Kohut wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Cool, that works.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks
>>>>>>>>>>>>> ---
>>>>>>>>>>>>> Yura
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Oct 15, 2010, at 3:52 PM, MORITA Kazutaka wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> At Fri, 15 Oct 2010 13:38:16 +0300,
>>>>>>>>>>>>>> Yuriy Kohut wrote:
>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I'm using the following 'Getting Started' guide to configure simple cluster:
>>>>>>>>>>>>>>> http://www.osrg.net/sheepdog/usage.html
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I have configured cluster on 1 node/box, so the first questions are:
>>>>>>>>>>>>>>> Can I configure cluster on single node (1 box) under CentOS 5.5 x86_64 ?
>>>>>>>>>>>>>>> Is it required at least 3 nodes/boxes ... ?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I have faced with the following issue on my single-node cluster. I have rebooted the box after my first image creation. The following done for that:
>>>>>>>>>>>>>>> - corosync is up and running
>>>>>>>>>>>>>>> udp 0 0 192.168.128.195:5404 0.0.0.0:* 3541/corosync
>>>>>>>>>>>>>>> udp 0 0 192.168.128.195:5405 0.0.0.0:* 3541/corosync
>>>>>>>>>>>>>>> udp 0 0 226.94.1.1:5405 0.0.0.0:* 3541/corosync
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> - sheep is up and running
>>>>>>>>>>>>>>> tcp 0 0 0.0.0.0:7000 0.0.0.0:* LISTEN 3561/sheep
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> - cluster is formatted with 1 copy only
>>>>>>>>>>>>>>> #collie cluster format --copies=1
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> - the image with prelocated data is created
>>>>>>>>>>>>>>> # qemu-img create sheepdog:test0 -o preallocation=data 4G
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> So after such siple steps I got:
>>>>>>>>>>>>>>> # collie vdi list
>>>>>>>>>>>>>>> name id size used shared creation time vdi id
>>>>>>>>>>>>>>> ------------------------------------------------------------------
>>>>>>>>>>>>>>> test0 1 4.0 GB 4.0 GB 0.0 MB 2010-10-15 12:42 fd34af
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Then I rebooted the box, and no image(s) are available for me after box came back. The vdi list just show nothing:
>>>>>>>>>>>>>>> # collie vdi list
>>>>>>>>>>>>>>> name id size used shared creation time vdi id
>>>>>>>>>>>>>>> ------------------------------------------------------------------
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> and 'collie vdi list' never ends ...
>>>>>>>>>>>>>>> corosync and sheep are still running.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Could somebody assist me with that.
>>>>>>>>>>>>>> Sorry, the following patch will fixes the problem.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> =
>>>>>>>>>>>>>> From: MORITA Kazutaka<morita.kazutaka at lab.ntt.co.jp>
>>>>>>>>>>>>>> Subject: [PATCH] sheep: call start_recovery when cluster restarts with one node
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Sheepdog recovers objects before starting a storage service, and the
>>>>>>>>>>>>>> routine is called when nodes are joined. However If sheepdog consists
>>>>>>>>>>>>>> of only one node, no node doesn't send join messages, so
>>>>>>>>>>>>>> start_recovery doesn't called. This patch fixes the problem.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Signed-off-by: MORITA Kazutaka<morita.kazutaka at lab.ntt.co.jp>
>>>>>>>>>>>>>> ---
>>>>>>>>>>>>>> sheep/group.c | 3 +++
>>>>>>>>>>>>>> 1 files changed, 3 insertions(+), 0 deletions(-)
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> diff --git a/sheep/group.c b/sheep/group.c
>>>>>>>>>>>>>> index ba8cdfb..86cbdb8 100644
>>>>>>>>>>>>>> --- a/sheep/group.c
>>>>>>>>>>>>>> +++ b/sheep/group.c
>>>>>>>>>>>>>> @@ -1226,6 +1226,9 @@ static void __sd_confchg_done(struct cpg_event *cevent)
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> update_cluster_info(&msg);
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> + if (sys->status == SD_STATUS_OK) /* sheepdog starts with one node */
>>>>>>>>>>>>>> + start_recovery(sys->epoch, NULL, 0);
>>>>>>>>>>>>>> +
>>>>>>>>>>>>>> return;
>>>>>>>>>>>>>> }
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> --
>>>>>>>>>>>>>> 1.5.6.5
>>>>>>>>>>>>>>
>>>>>>>>>>>>> --
>>>>>>>>>>>>> sheepdog mailing list
>>>>>>>>>>>>> sheepdog at lists.wpkg.org
>>>>>>>>>>>>> http://lists.wpkg.org/mailman/listinfo/sheepdog
>>>>>>>>>>>> --
>>>>>>>>>>>> sheepdog mailing list
>>>>>>>>>>>> sheepdog at lists.wpkg.org
>>>>>>>>>>>> http://lists.wpkg.org/mailman/listinfo/sheepdog
>>>>>>>>>> --
>>>>>>>>>> sheepdog mailing list
>>>>>>>>>> sheepdog at lists.wpkg.org
>>>>>>>>>> http://lists.wpkg.org/mailman/listinfo/sheepdog
>>>>>>>
>>>>>
>>>>
>>>> --
>>>> sheepdog mailing list
>>>> sheepdog at lists.wpkg.org
>>>> http://lists.wpkg.org/mailman/listinfo/sheepdog
>>>
>>
>
More information about the sheepdog
mailing list