[Sheepdog] [PATCH v3] remove oids from object list cache when deleting a vdi

[星刻]

On 05/02/2012 03:03 PM, Li Wenpeng wrote:
> This patch set aims to clear the object list cache after
> a vdi is deleted, then we should remove the data objects
> which in form of an uint64_t inditifier from the object
> list cache.
>
> 1. Why we need the clear the object list cache ?
>
> The object list cache is used when the cluster is in recovery,
> to provide the object list for request_obj_list(), sheep uses
> the list to determine which objects exist in the cluster, and
> then try to recover them when cluster is changing.
>
> When the object has been deleted, certainly we should remove it
> from the object list cache, so that sheep would not try to recover
> the objects alreay deleted which may cause too much time.
>
> 2. How we do currently ?
>
> I try to remove the deleted object from the object list cache
> in store_remove_obj() after unlink() success.
>
> But there's a big problem:
>
> When the cluster changed one or more times (nodes join or leave) before,
> it causes some data objects to migrate from one node to another,
> let's talk them as the 'old node' and the 'new node',
> but neigher did we remove object id from the objlist cache in the
> old node, nor put the object id into the objlist cache in the new
> node, here's the problem,
>
> in store_remove_obj(), unlink() may success because the object does
> exist, but the object id may not exist in the object list cache in
> that node, it may exists in an other node.
>
> PS. Migrating object list cache is rather difficult in recovery, and
>      not so necessary if we didn't remove any object from the cluster.
>
> 3. What this patch set does ?
>
> After a vdi is deleted successfully, sheep nodifies all the data
> objects deleted to all the other node, every node who receives
> this message tries to remove the objects from object list cache.
>
> there's also a small problem, after an vdi is deleted, and before
> the notification message was received by all the other nodes,
> cluster recovery may happens in this time window, it may tries to
> recover the objects already deleted.
>
> This problem is difficult to avoid in current situation, but we
> can try our best to reduce the probability of recovering deleted
> objects by this patch set.
>
> 4. Summary about the four patches
>
> The first patch fixes a bug about nr_copies in delete_one, I notified
> that it hasn't been fixed in the master branch, with this bug, my
> patch doesn't work at all, so I fixed it.
>
> The second patch takes Liu Yuan's advise to change the name process_work
> and process_main to process_top and process_bottom.
>
> The third patch makes process_bottom running in worker thread for cluster
> request when given a flag SD_FLAG_CMD_WORKER for request header.
>
> The fourth patch do the clear work.
>
>
> Thanks,
> levin
>
This patch is really important, without it sheep may wastes too much
time recover objects already deleted when nodes change(join or leave).

Any comments?

thanks,
levin

________________________________

This email (including any attachments) is confidential and may be legally privileged. If you received this email in error, please delete it immediately and do not copy it or use it for any purpose or disclose its contents to any other person. Thank you.

本电邮(包括任何附件)可能含有机密资料并受法律保护。如您不是正确的收件人，请您立即删除本邮件。请不要将本电邮进行复制并用作任何其他用途、或透露本邮件之内容。谢谢。