[sheepdog-users] Failed to read object 80244e5600000000 No object found

[Struan Bartlett]

Hi Liu

Thanks for your help.

On 05/05/2014 04:23, Liu Yuan wrote:
> On Sun, May 04, 2014 at 08:11:02PM +0100, Struan Bartlett wrote:
>> Having been running sheepdog-0.8.0 successfully for a number of weeks,
>> earlier last month I suddenly found that my cluster would not longer
>> launch. After reattempting launch this evening, I finally got the
>> cluster launched but now to make matters worse it now looks like
>> sheepdog has deleted all underlying objects! Here is some data:
>>
> There might be some bug in 0.8.0 version. Probably latest v0.8.1 solve your
> problems.
Ok, I might try that. But when I last tried sheepdog (a year or two ago,
so I think it was 0.5.3) I experienced a similar problem - my object
store being mysteriously wiped - so I fear this bug might have been
lurking for a long time.
> Can anyone explain what has happened, and why sheepdog has just now
> deleted all the objects associated with my cluster, I assume rendering
> it completely unrecoverable? Please let me know if there are other
> investigations I should perform.
> Hmmm, this looks a fatal problem. Sheepdog will try to put the objects into stale
> directory temprarily if your cluster wasn't shutdown normally beforhand. Then
> all the nodes will try to do a recovery.
I've checked and the stale directories are now all empty.
>
> Is there any method to reproduce the problem?
Good question. No, and I'm not sure how to attempt to reproduce the
problem either. It's conceivable that one or more of the nodes rebooted
but it's unlikely that two rebooted at the same time. But I'd expect
sheepdog to be able to deal with either of those situations without
corrupting the object store.

Struan