[sheepdog] [PATCH v4 0/6] create onode before uploading object completed
robin.k.dong at gmail.com
Wed Feb 26 04:00:15 CET 2014
From: Robin Dong <sanbai at taobao.com>
At present, the implemention of swift interface for creating object in sheepdog
1. lock container
2. check whether the onode with same object name is exists.
3. unlock container
4. upload object
5. create onode
this sequence have a problem: if two clients uploading same objects
concurrently, it will create two objects with same names in container.
To avoid duplicated names, we must put "create onode" operation in container
Therefore we need to change the processes of creating object to:
1. lock container
2. check whether the onode is exists.
3. allocate data space for object, and create onode, then write it done
4. unlock container
5. upload object
this routine will avoid uploading duplicated objects.
There is an exception on the new routine: if the client halt the uploading
progress, we will have a "uploading incompleted" onode.
The solution is: we can add code for onode to identify its status.
A new onode will be set to "INIT", and after uploading completed, the onode will
be set to "COMPLETED". So, when users try to use swift interface to GET a
"incompleted" object, sheep will find out the onode is "INIT" which means
"not completed", so sheep will return "partial content"for http request, and
user could DELETE the object and upload it again.
Signed-off-by: Robin Dong <sanbai at taobao.com>
1. change 'Forced: true' to "Force: true'
2. fix compile warning for gcc-4.6
1. add explain of design on commit log and source file
2. use sd_mutex_xxx() instead of pthread_mutex_xxx()
3. add HTTP Header 'Forced: true' to delete forcely,
instead of using FORCE_DELETE
1. remove some redundant function like xxx_nolock()
2. use the mmap-memory to store phtread_mutex_t
3. add flag SD_RES_INCOMPLETE
Robin Dong (6):
sheep/http: add nolock version for onode_lookup()
sheep/http: change routine of kv_create_object()
sheep/cluster: let local_lock() support nested
sheep/http: add flag for onode
sheep/http: allow 'Force: true' http header to delete 'incompleted'
tests: add test case for new kv_create_object() implemention
include/sheepdog_proto.h | 1 +
include/util.h | 14 +++
sheep/cluster/local.c | 109 ++++++++++++++---
sheep/http/http.c | 5 +
sheep/http/http.h | 4 +-
sheep/http/kv.c | 286 +++++++++++++++++++++++++++++++++++---------
sheep/http/s3.c | 2 +-
sheep/http/swift.c | 12 +-
tests/functional/087 | 102 ++++++++++++++++
tests/functional/087.out | 32 +++++
tests/functional/group | 1 +
tests/functional/nginx.conf | 6 +
12 files changed, 495 insertions(+), 79 deletions(-)
create mode 100755 tests/functional/087
create mode 100644 tests/functional/087.out
More information about the sheepdog