[sheepdog] [PATCH v4 0/6] create onode before uploading object completed
Liu Yuan
namei.unix at gmail.com
Wed Feb 26 05:03:42 CET 2014
On Wed, Feb 26, 2014 at 11:00:15AM +0800, Robin Dong wrote:
> From: Robin Dong <sanbai at taobao.com>
>
> At present, the implemention of swift interface for creating object in sheepdog
> is:
>
> 1. lock container
> 2. check whether the onode with same object name is exists.
> 3. unlock container
> 4. upload object
> 5. create onode
>
> this sequence have a problem: if two clients uploading same objects
> concurrently, it will create two objects with same names in container.
> To avoid duplicated names, we must put "create onode" operation in container
> lock regions.
>
> Therefore we need to change the processes of creating object to:
>
> 1. lock container
> 2. check whether the onode is exists.
> 3. allocate data space for object, and create onode, then write it done
> 4. unlock container
> 5. upload object
>
> this routine will avoid uploading duplicated objects.
>
> There is an exception on the new routine: if the client halt the uploading
> progress, we will have a "uploading incompleted" onode.
>
> The solution is: we can add code for onode to identify its status.
> A new onode will be set to "INIT", and after uploading completed, the onode will
> be set to "COMPLETED". So, when users try to use swift interface to GET a
> "incompleted" object, sheep will find out the onode is "INIT" which means
> "not completed", so sheep will return "partial content"for http request, and
> user could DELETE the object and upload it again.
>
> Signed-off-by: Robin Dong <sanbai at taobao.com>
> ---
> v3-->v4:
> 1. change 'Forced: true' to "Force: true'
> 2. fix compile warning for gcc-4.6
>
> v2-->v3:
> 1. add explain of design on commit log and source file
> 2. use sd_mutex_xxx() instead of pthread_mutex_xxx()
> 3. add HTTP Header 'Forced: true' to delete forcely,
> instead of using FORCE_DELETE
>
> v1-->v2:
> 1. remove some redundant function like xxx_nolock()
> 2. use the mmap-memory to store phtread_mutex_t
> 3. add flag SD_RES_INCOMPLETE
>
> Robin Dong (6):
> sheep/http: add nolock version for onode_lookup()
> sheep/http: change routine of kv_create_object()
> sheep/cluster: let local_lock() support nested
> sheep/http: add flag for onode
> sheep/http: allow 'Force: true' http header to delete 'incompleted'
> object forcely
> tests: add test case for new kv_create_object() implemention
>
> include/sheepdog_proto.h | 1 +
> include/util.h | 14 +++
> sheep/cluster/local.c | 109 ++++++++++++++---
> sheep/http/http.c | 5 +
> sheep/http/http.h | 4 +-
> sheep/http/kv.c | 286 +++++++++++++++++++++++++++++++++++---------
> sheep/http/s3.c | 2 +-
> sheep/http/swift.c | 12 +-
> tests/functional/087 | 102 ++++++++++++++++
> tests/functional/087.out | 32 +++++
> tests/functional/group | 1 +
> tests/functional/nginx.conf | 6 +
> 12 files changed, 495 insertions(+), 79 deletions(-)
> create mode 100755 tests/functional/087
> create mode 100644 tests/functional/087.out
>
> --
> 1.7.12.4
>
> --
> sheepdog mailing list
> sheepdog at lists.wpkg.org
> http://lists.wpkg.org/mailman/listinfo/sheepdog
Applied thanks
Yuan
More information about the sheepdog
mailing list