[sheepdog] consistency problem when replicating to multi-replica

[guping]

I find a interesting subject at
http://copilotco.com/mail-archives/drbd.2009/msg00750.html ,

The 'concurrent local write' in DRBD is very similar to the problem in
my previous mail.

Hope to be useful and we can come out a solution.

在 15/1/5 上午9:26, Hitoshi Mitake 写道:
> At Mon, 05 Jan 2015 09:12:27 +0800,
> guping wrote:
>>
>> Thanks Hitoshi,
>>
>> I just began to evaluate sheepdog for our cloud deployment. I will try
>> to reproduce the problem when I have enough time. If some clue, I will
>> let you know.
> 
> Thanks for your interest :) Please let us know if you find problems.
> 
>>
>> A question, does the IO stack components higher than sheepdog, such as
>> IO queue in VM, qemu block driver, etc, provide the necessary serialization?
> 
> I think these components doesn't have to provide the serialization
> functionality. sheepdog (including QEMU driver) must mock the
> sematntics of hard drives. If sheepdog allows the case of divergent
> objects, it means sheepdog cannot preserve the semantics (two or more
> continuous read requests to same location can return different
> data). So the serialization should be provided by sheepdog.
> 
> Thanks,
> Hitoshi
> 
>>
>> 在 15/1/5 上午8:52, Hitoshi Mitake 写道:
>>> At Sun, 04 Jan 2015 23:27:15 +0800,
>>> guping wrote:
>>>>
>>>> Sheepdog send write request to all replica simultaneously to do the
>>>> replication. Which I think could lead to divergent content in replicas.
>>>>
>>>> Imagine a scenario, there are two write requests A & B to update the
>>>> same offset in a 2-replicas object. If a qemu client send A and B to a
>>>> gateway concurrently, the gateway will send A and B to the same set of
>>>> replicas simultaneously. Which could not guarantee the same sequence to
>>>> apply A and B, you can apply A then B in a replica and apply B then A in
>>>> another replica, then you get two request return successfully but not
>>>> the same data in the replicas.
>>>>
>>>> Is this a consistency problem in sheepdog? How can we deal with it?
>>>
>>> Hi Gu,
>>>
>>> Currently, the above problem is prevented for a case of copy on write
>>> requests (first write to an object). Look at
>>> check_simultaneous_create() of sheepdog driver in qemu. Multiple COW
>>> requests for a single object is serialized for not losing written
>>> information illegally.
>>>
>>> But, the driver doesn't do the serialization for ordinal write
>>> request. As you say, there might be a possibility of divergent
>>> objects. Could you produce the problem with artificial workload?
>>>
>>> Thanks,
>>> Hitoshi
>>>
>>>>
>>>> --
>>>> Thanks,
>>>> Gu Ping
>>>> --
>>>> sheepdog mailing list
>>>> sheepdog at lists.wpkg.org
>>>> https://lists.wpkg.org/mailman/listinfo/sheepdog
>>>
>>
>> -- 
>> Thanks,
>> Gu Ping
>>
>>
>> -- 
>> sheepdog mailing list
>> sheepdog at lists.wpkg.org
>> https://lists.wpkg.org/mailman/listinfo/sheepdog
> 

-- 
Thanks,
Gu Ping