[sheepdog-users] about sheepdog object http storage

[郑晓川]

ok, i have added a github issue :
https://github.com/sheepdog/sheepdog/issues/240

and i already kown that , user can force DELETE a file by add an http header "force:true" , and indeed it works fine .

/*
 * Imaging a scenario:
 *
 * 1. Client A is uploading a large object which names 'elephant', it
 *    only allocate meta-data in oalloc and is creating SD_OBJ_DATA files.
 * 2. Client B send a DELETE request to remove object 'elephant', it
 *    will remove all the backend files for 'elephant'.
 *    At the same time, Client A doesn't know what happened because uploading
 *    progress don't need to lock any vdi.
 * 3. Client A return Create-object-success, but the real data have all been
 *    removed.
 *
 * To avoid this scenario, we let DELETE operation do nothing but only return
 * 'CONFLICT' when the object is 'incompleted'. And, users can send the DELETE
 * request with a new header 'Force: true' which will remove 'incompleted'
 * object forcely when users make sure that there isn't any uploading progress
 * for this object.
 */



________________________________
郑晓川

From: Hitoshi Mitake<mailto:mitake.hitoshi at gmail.com>
Date: 2016-05-10 21:30
To: 郑晓川<mailto:zhengxiaochuan at jd.com>
CC: sheepdog-users<mailto:sheepdog-users at lists.wpkg.org>
Subject: Re: [sheepdog-users] about sheepdog object http storage
Hi,

Thanks for reporting the problem. I cannot answer it quickly because I'm on business travel. Could you add a github issue?

Thanks,
Hitoshi

On Mon, May 9, 2016 at 11:26 AM, 郑晓川 <zhengxiaochuan at jd.com<mailto:zhengxiaochuan at jd.com>> wrote:

HI all:

I PUTed a film xxx.rmvb to sheepdog object http storage , failed 。

nginx error.log:

2016/05/09 10:23:16 [info] 8655#0: *42006 epoll_wait() reported that client prematurely closed connection, so upstream connection is closed too while sending request to upstream, client: 10.8.65.42, server: obs, request: "PUT /v1/admin/jls/xxx.rmvb HTTP/1.0", upstream: "fastcgi://10.8.65.42:8008<http://10.8.65.42:8008>", host: "10.8.65.42:8888<http://10.8.65.42:8888>"

my nginx conf :
        location / {
            fastcgi_pass    10.8.65.42:8008<http://10.8.65.42:8008>;

            fastcgi_connect_timeout 600s;
            fastcgi_read_timeout 600s;
            fastcgi_send_timeout 600s;


            fastcgi_param   SCRIPT_NAME             $fastcgi_script_name;
            fastcgi_param   REQUEST_METHOD          $request_method;
            fastcgi_param   CONTENT_LENGTH          $content_length;
            fastcgi_param   HTTP_RANGE              $http_range;
            fastcgi_param   DOCUMENT_URI            $document_uri;
            fastcgi_param   REQUEST_URI             $request_uri;
        }

sheep.log:

May 09 10:23:18 ERROR [http 8273] onode_populate_extents(838) Failed to do_vdi_write data_vid: eec946, offset: 7800000, total: 58b7d771, ret: I/O error
May 09 10:23:18 ERROR [http 8273] kv_create_object(1469) Failed to write data to onode xxx.rmvb
May 09 10:23:18 INFO [http 8273] swift_handle_request(352) /v1/admin/jls/xxx.rmvb PUT, status = 500 Internal Server Error, data_length = 1488443249



After this ,  GET or DELETE  all goes wrong , like this :

[root at node-65-42 hss_storage]# http DELETE "http://127.0.0.1:8888/v1/admin/jls/xxx.rmvb"
HTTP/1.1 409 Conflict
Connection: keep-alive
Content-Type: text/plain;
Date: Mon, 09 May 2016 14:36:26 GMT
Server: openresty/1.9.7.4<http://1.9.7.4>
Transfer-Encoding: chunked

[root at node-65-42 hss_storage]# http DELETE "http://127.0.0.1:8888/v1/admin/jls/xxx.rmvb"
HTTP/1.1 409 Conflict
Connection: keep-alive
Content-Type: text/plain;
Date: Mon, 09 May 2016 14:36:58 GMT
Server: openresty/1.9.7.4<http://1.9.7.4>
Transfer-Encoding: chunked

[root at node-65-42 hss_storage]# http GET "http://127.0.0.1:8888/v1/admin/jls/xxx.rmvb"
HTTP/1.1 206 Partial Content
Connection: keep-alive
Content-Length: 0
Content-Type: text/plain;
Date: Mon, 09 May 2016 14:39:38 GMT
Server: openresty/1.9.7.4<http://1.9.7.4>


my question is :
1、why did this happen , is the browser or  nginx or sheepdog problem ?
2、when it happend , how should i DELETE the file by force .

thanks a lot !

________________________________
郑晓川

--
sheepdog-users mailing lists
sheepdog-users at lists.wpkg.org<mailto:sheepdog-users at lists.wpkg.org>
https://lists.wpkg.org/mailman/listinfo/sheepdog-users


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.wpkg.org/pipermail/sheepdog-users/attachments/20160511/c948e957/attachment.html>