[sheepdog-users] 0.8.1 check broken?

Giovanni Bellac giovannib1979 at ymail.com
Wed Apr 9 18:29:30 CEST 2014 

Hello all,

we are currently using 0.8.0 and trying to update to 0.8.1. We have even tried the latest git version.

1) xattr INFOs
We are using ext4 with user_xattr parameter in /etc/fstab and have tried the defaults of ext4 also.


When starting with -u


Apr 09 16:32:28   INFO [main] convert_ecidx_xattr2path(447) object: /opt/sheep/storage/obj/0089b48c00000952 doesn't have its ec index in xattr: No data available
Apr 09 16:32:28   INFO [main] convert_ecidx_xattr2path(447) object: /opt/sheep/storage/obj/0072926800000103 doesn't have its ec index in xattr: No data available
Apr 09 16:32:28   INFO [main] convert_ecidx_xattr2path(447) object: /opt/sheep/storage/obj/007292680000053a doesn't have its ec index in xattr: No data available

2) Starting than without -u

Apr 09 16:37:53   INFO [main] md_add_disk(337) /opt/sheep/storage/obj, vdisk nr 3464, total disk 1
Apr 09 16:37:53  ERROR [main] init_config_file(99) This sheep version is not compatible with the existing data layout, 3
Apr 09 16:37:53  ERROR [main] init_config_file(118) use '-u' option to upgrade sheep store



How can we update our cluster ?

Thanks,
Giovanni

Hitoshi Mitake <mitake.hitoshi at gmail.com> schrieb am 19:30 Sonntag, 6.April 2014:
 
At Thu, 3 Apr 2014 06:02:50 +0200,
<richter at ecos.de> wrote:
> 
> Hi,
> 
> I have changed from corosync to zookeeper before. I have done a dog cluster
> shutdown and then changed all nodes and brought them up again. This worked
> fine. This was at 19:36:22.
> 
> There is nothing about the failed check in the log, as far as I can see. The
> message "Failed to find requested tag" was before my try to check the image

It seems that some of your sheep processes don't have its epoch log
and reading it from remote sheep process is failing. It would be a
cause of failing "dog vdi check". Can I see output of "dog cluster
info" on all nodes?

> 
> I was wondering why I get still the message about the upgrade from v3 to v4.
> I had already run sheep with -u in the past.

I think you will never see the log after rebooting sheep process. If
you see the log even after the reboot, it means that the updating
process has bug. Could you check it?

Thanks,
Hitoshi


> 
> Regards
> 
> Gerald
> 
> 
> #dog cluster info
> Cluster status: running, auto-recovery enabled
> 
> Cluster created at Fri Feb  7 23:17:20 2014
> 
> Epoch Time           Version
> 2014-04-02 09:01:56     99 [1.1.1.1:7000, 2.2.2.2:7000, 3.3.3.3:7000]
> 2014-04-01 16:57:48     98 [1.1.1.1:7000, 3.3.3.3:7000]
> 2014-04-01 12:54:46     97 [1.1.1.1:7000, 2.2.2.2:7000, 3.3.3.3:7000]
> 2014-04-01 12:54:41     96 [1.1.1.1:7000, 3.3.3.3:7000]
> 2014-03-29 15:00:20     95 [1.1.1.1:7000, 2.2.2.2:7000, 3.3.3.3:7000]
> 
> 
> Apr 02 19:36:22   INFO [main] md_add_disk(333)
> /var/lib/sheepdog//disc1/data, vdisk nr 1619, total disk 1
> Apr 02 19:36:22   INFO [main] md_add_disk(333)
> /var/lib/sheepdog//disc2/data, vdisk nr 1836, total disk 2
> Apr 02 19:36:22  ERROR [main] init_config_file(99) This sheep version is not
> compatible with the existing data layout, 3
> Apr 02 19:36:24   INFO [main] convert_ecidx_xattr2path(446) object:
> /var/lib/sheepdog//disc1/data/00f6b38a00005f9e doesn't have its ec index in
> xattr: No data available
> Apr 02 19:36:24   INFO [main] convert_ecidx_xattr2path(446) object:
> /var/lib/sheepdog//disc1/data/002b8afa00000679 doesn't have its ec index in
> xattr: No data available
> Apr 02 19:36:24   INFO [main] convert_ecidx_xattr2path(446) object:
> /var/lib/sheepdog//disc1/data/00f08a9300000da6 doesn't have its ec index in
> xattr: No data available
> ....
> Apr 02 19:36:26   INFO [main] convert_ecidx_xattr2path(446) object:
> /var/lib/sheepdog//disc2/data/00f6b38a0000c80a doesn't have its ec index in
> xattr: No dat
> a available
> Apr 02 19:36:26   INFO [main] convert_ecidx_xattr2path(446) object:
> /var/lib/sheepdog//disc2/data/00bb7a2500000c76 doesn't have its ec index in
> xattr: No dat
> a available
> Apr 02 19:36:26 NOTICE [main] get_local_addr(519) found IPv4 address
> Apr 02 19:38:17  ERROR [io 226735] sheep_exec_req(1096) failed Waiting for
> other nodes to join cluster
> Apr 02 19:38:17  ERROR [io 226735] sheep_exec_req(1096) failed Waiting for
> other nodes to join cluster
> Apr 02 19:38:17  ERROR [io 226735] sheep_exec_req(1096) failed Waiting for
> other nodes to join cluster
> Apr 02 19:38:17  ERROR [io 226735] sheep_exec_req(1096) failed Waiting for
> other nodes to join cluster
> Apr 02 19:38:17  ERROR [io 226735] sheep_exec_req(1096) failed Waiting for
> other nodes to join cluster
> Apr 02 19:38:17  ERROR [io 226735] sheep_exec_req(1096) failed Waiting for
> other nodes to join cluster
> Apr 02 19:38:17  ERROR [io 226735] sheep_exec_req(1096) failed Waiting for
> other nodes to join cluster
> Apr 02 19:38:17  ERROR [io 226735] sheep_exec_req(1096) failed Waiting for
> other nodes to join cluster
> Apr 02 19:38:17  ERROR [io 226735] sheep_exec_req(1096) failed Waiting for
> other nodes to join cluster
> Apr 02 19:38:17  ERROR [io 226735] sheep_exec_req(1096) failed Waiting for
> other nodes to join cluster
> Apr 02 19:38:17  ERROR [io 226735] sheep_exec_req(1096) failed Waiting for
> other nodes to join cluster
> Apr 02 19:38:17  ERROR [io 226735] sheep_exec_req(1096) failed Waiting for
> other nodes to join cluster
> Apr 02 19:38:17  ERROR [io 226735] sheep_exec_req(1096) failed Waiting for
> other nodes to join cluster
> Apr 02 19:38:17  ERROR [io 226735] sheep_exec_req(1096) failed Waiting for
> other nodes to join cluster
> Apr 02 19:38:17  ERROR [io 226735] sheep_exec_req(1096) failed Waiting for
> other nodes to join cluster
> Apr 02 19:38:17  ERROR [io 226735] sheep_exec_req(1096) failed Waiting for
> other nodes to join cluster
> Apr 02 19:38:17  ERROR [io 226735] sheep_exec_req(1096) failed Waiting for
> other nodes to join cluster
> Apr 02 19:39:53  ERROR [io 227060] sheep_exec_req(1096) failed Failed to
> find requested tag
> Apr 02 19:39:53  ERROR [io 227060] sheep_exec_req(1096) failed Failed to
> find requested tag
> Apr 02 19:39:53  ERROR [io 227060] sheep_exec_req(1096) failed Failed to
> find requested tag
> Apr 02 19:39:53  ERROR [io 227060] sheep_exec_req(1096) failed Failed to
> find requested tag
> Apr 02 19:39:53  ERROR [io 227060] sheep_exec_req(1096) failed Failed to
> find requested tag
> Apr 02 19:39:53  ERROR [io 227060] sheep_exec_req(1096) failed Failed to
> find requested tag
> Apr 02 19:39:53  ERROR [io 227060] sheep_exec_req(1096) failed Failed to
> find requested tag
> Apr 02 19:39:53  ERROR [io 227060] sheep_exec_req(1096) failed Failed to
> find requested tag
> Apr 02 19:39:53  ERROR [io 227060] sheep_exec_req(1096) failed Failed to
> find requested tag
> Apr 02 19:39:53  ERROR [io 227060] sheep_exec_req(1096) failed Failed to
> find requested tag
> Apr 02 19:39:53  ERROR [io 227060] sheep_exec_req(1096) failed Failed to
> find requested tag
> Apr 02 19:39:53  ERROR [io 227060] sheep_exec_req(1096) failed Failed to
> find requested tag
> Apr 02 19:39:53  ERROR [io 227060] sheep_exec_req(1096) failed Failed to
> find requested tag
> Apr 02 19:39:53  ERROR [io 227060] sheep_exec_req(1096) failed Failed to
> find requested tag
> Apr 02 19:39:53  ERROR [io 227060] sheep_exec_req(1096) failed Failed to
> find requested tag
> Apr 02 19:39:53  ERROR [io 227060] sheep_exec_req(1096) failed Failed to
> find requested tag
> Apr 02 19:39:53  ERROR [io 227060] sheep_exec_req(1096) failed Failed to
> find requested tag
> 
> 
> 
> > -----Ursprüngliche Nachricht-----
> > Von: Hitoshi Mitake [mailto:mitake.hitoshi at gmail.com]
> > Gesendet: Donnerstag, 3. April 2014 04:39
> > An: Gerald Richter - ECOS Technology
> > Cc: sheepdog-users at lists.wpkg.org
> > Betreff: Re: [sheepdog-users] 0.8.1 check broken?
> > 
> > Hi Gerald,
> > 
> > On Thu, Apr 3, 2014 at 2:50 AM, Gerald Richter - ECOS Technology
> > <gerald.richter at ecos.de> wrote:
> > > Hi,
> > >
> > >
> > >
> > > Running check (I tried with different images), I always get:
> > >
> > >
> > >
> > > #dog vdi check vm-12162-disk-1
> > >
> > >
> > >
> > > PANIC: can't find a valid vnode
> > >
> > > dog exits unexpectedly (Aborted).
> > >
> > > dog.c:351: crash_handler
> > >
> > > /lib/x86_64-linux-gnu/libpthread.so.0(+0xf02f) [0x7f880ff8d02f]
> > >
> > > /lib/x86_64-linux-gnu/libc.so.6(gsignal+0x34) [0x7f880fa0d474]
> > >
> > > /lib/x86_64-linux-gnu/libc.so.6(abort+0x17f) [0x7f880fa106ef]
> > >
> > > sheep.h:80: oid_to_vnodes
> > >
> > > vdi.c:1885: do_vdi_check
> > >
> > > dog.c:519: main
> > >
> > > /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xfc)
> > > [0x7f880f9f9eac]
> > >
> > > dog() [0x403fa8]
> > >
> > >
> > >
> > > This is a working image ( I can start th VM and it's working). So
> > > what's wrong here?
> > 
> > Thanks for reporting. Can I see an output of "dog cluster info" and log of
> > sheep process which runs on the host you executed the above dog vdi
> > check?
> > 
> > Thanks,
> > Hitoshi
> > 
> > >
> > >
> > >
> > > Thanks & Regards
> > >
> > >
> > >
> > > Gerald
> > >
> > >
> > >
> > >
> > > --
> > > sheepdog-users mailing lists
> > > sheepdog-users at lists.wpkg.org
> > > http://lists.wpkg.org/mailman/listinfo/sheepdog-users
> > >
> 
-- 
sheepdog-users mailing lists
sheepdog-users at lists.wpkg.org
http://lists.wpkg.org/mailman/listinfo/sheepdog-users
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.wpkg.org/pipermail/sheepdog-users/attachments/20140409/e2b1d6e6/attachment-0005.html>

More information about the sheepdog-users mailing list