[sheepdog] Issues about setup

[Hitoshi Mitake]

At Mon, 19 Jan 2015 16:29:17 +0800,
hujianyang wrote:
> 
> Hi Hitoshi,
> 
> On 2015/1/16 17:40, Hitoshi Mitake wrote:
> >>
> > 
> > Hmm, could you show your yasm version? My yasm (1.2.0) can build
> > current master branch with no problems.
> > 
> > Thanks,
> > Hitoshi
> > 
> 
> Thanks for your advise. I've updated my yasm to 1.3.0 and found it's
> OK. I think maybe we should mark this requirement also in configure
> file.
> 
> Now I'm trying to use sheepdog backing device. Errors occur while
> compiling sbd kernel module:
> 
> linux-2hp8:/opt/sheepdog/sbd # make
> make -C /lib/modules/`uname -r`/build M=$PWD modules
> make[1]: Entering directory `/opt/linux-3.18.2'
>   CC [M]  /opt/sheepdog/sbd/sheep_block_device.o
> In file included from /opt/sheepdog/sbd/sbd.h:20,
>                  from /opt/sheepdog/sbd/sheep_block_device.c:48:
> /opt/sheepdog/sbd/../include/sheepdog_proto.h: In function ‘count_data_objs’:
> /opt/sheepdog/sbd/../include/sheepdog_proto.h:495: error: implicit declaration of function ‘UINT32_C’
> /opt/sheepdog/sbd/sheep_block_device.c: In function ‘sbd_remove’:
> /opt/sheepdog/sbd/sheep_block_device.c:293: error: implicit declaration of function ‘strict_strtoul’
> make[2]: *** [/opt/sheepdog/sbd/sheep_block_device.o] Error 1
> make[1]: *** [_module_/opt/sheepdog/sbd] Error 2
> make[1]: Leaving directory `/opt/linux-3.18.2'
> make: *** [default] Error 2
> 
> 
> My kernel version is 3.18.2. Seems UINT32_C is not used in kernel
> and strict_strtoul has been replaced by kstrtoul in my kernel.
> 
> diff --git a/include/sheepdog_proto.h b/include/sheepdog_proto.h
> index 3910bd5..0599ece 100644
> --- a/include/sheepdog_proto.h
> +++ b/include/sheepdog_proto.h
> @@ -493,7 +493,7 @@ static inline bool is_data_obj(uint64_t oid)
>  static inline size_t count_data_objs(const struct sd_inode *inode)
>  {
>         return DIV_ROUND_UP(inode->vdi_size,
> -                           (UINT32_C(1) << inode->block_size_shift));
> +                           (1UL << inode->block_size_shift));
>  }
> 
>  static inline size_t get_objsize(uint64_t oid, uint32_t object_size)
> diff --git a/sbd/sheep_block_device.c b/sbd/sheep_block_device.c
> index eaee932..d82ff43 100644
> --- a/sbd/sheep_block_device.c
> +++ b/sbd/sheep_block_device.c
> @@ -290,7 +290,7 @@ static ssize_t sbd_remove(struct bus_type *bus, const char *buf,
>         unsigned long ul;
>         int target_id, ret;
> 
> -       ret = strict_strtoul(buf, 10, &ul);
> +       ret = kstrtoul(buf, 10, &ul);
>         if (ret)
>                 return ret;
> 
> I did a little change and it compiles OK. But UINT*_C are widely
> used in sheepdog_proto.h so maybe a large-scale modification is
> needed?

Cc-ing Alexander. The above UINT*_C macro problem is solved by the
recent patch of Alexander. But the kstrtoul() problem still
remains. If you can create a patch for it, I'll apply it.

> 
> Also, kernel modules should be changed besides different kernel
> versions. How about separating kernel module code from userspace
> sheepdog package?
> 

The main interfaces for modules provided by linux kernel seem to be
(almost) stable. If we can avoid to use (few) unstable interface,
separating it from our repository isn't required, I think.

> Another interesting thing, A sbd device cannot be re-add after
> removing it.
> 
> Fs-Server:/opt/GIT/sheepdog/sbd # echo 130.1.0.147 7000 Hu0 > /sys/bus/sbd/add
> Fs-Server:/opt/GIT/sheepdog/sbd # cat /sys/bus/sbd/list
> 0 Hu0
> Fs-Server:/opt/GIT/sheepdog/sbd # echo 0 > /sys/bus/sbd/remove
> Fs-Server:/opt/GIT/sheepdog/sbd # echo 130.1.0.147 7000 Hu0 > /sys/bus/sbd/add
> -bash: echo: write error: Input/output error
> 
> Jan 19 15:52:35 Fs-Server kernel: [364284.639557] sbd: Associated to Hu0
> Jan 19 15:52:35 Fs-Server kernel: [364284.640389] sbd:sheep_aiocb_submit:516: submit off 0, len 4096
> Jan 19 15:52:35 Fs-Server kernel: [364284.640397] sbd:end_sheep_request:469: end oid 6e776200000000 off 0, len 4096, seq 1
> Jan 19 15:52:35 Fs-Server kernel: [364284.640400] sbd:aio_read_done:360: rdone off 0, len 4096
> Jan 19 15:52:35 Fs-Server kernel: [364284.640444] sbd:sheep_aiocb_submit:516: submit off 12288, len 4096
> Jan 19 15:52:35 Fs-Server kernel: [364284.640448] sbd:end_sheep_request:469: end oid 6e776200000000 off 12288, len 4096, seq 2
> Jan 19 15:52:35 Fs-Server kernel: [364284.640451] sbd:aio_read_done:360: rdone off 12288, len 4096
> Jan 19 15:52:35 Fs-Server kernel: [364284.640465]  sbd0: unknown partition table
> Jan 19 15:53:18 Fs-Server kernel: [364328.123575] Cannot get VDI for Hu0, -5
> Jan 19 15:53:18 Fs-Server kernel: [364328.124265] sbd: error adding device 130.1.0.147 7000 Hu0
> 
> If you have no idea about this problem, I'd like to take some
> time look at it.
> 
> By the way, there are too many log messages while using sheepdog
> device. Can we stop its printing?

I don't have time to maintain sbd. If you can post a patch for
reducing message, I'll apply.

BTW, what is the purpose of using sbd? AFAIK, it cannot handle a fault
of sheep process which is connected to sbd. If you want to use
sheepdog as an alternative of SAN, I'd like to suggest iSCSI
interface. Below is an instruction:
https://github.com/sheepdog/sheepdog/wiki/General-protocol-support-%28iSCSI-and-NBD%29#iscsi

> 
> 
> 
> One important thing, seems yasm are not used in v8.3.0 and are
> required since v9.0 because of erasure_code? Where can I get
> more information about erasure_code in sheepdog except of the
> wiki page in github?

It is required by isa-l, erasure coding library from intel used by
newer versions than v0.9.0.

Thanks,
Hitoshi