From: levin li <xingke.lwp at taobao.com> v4 ----> v5: 1. use byte as the unit of cache size 2. use idx reserved bits for flags 3. removed some helper functions v3 ----> v4: 1. remove SD_RES_CACHE_REFERENCING, just return -1 if object can not be reclaimed in reclaim_object() 2. put the in_flush flags setting in object_cache_push() 3. when cache is in flush, do not stop reclaiming, but just skip the dirty entries. v2 ----> v3: 1. use uatomic_cmpxchg() in cache_in_reclaim() to check and set the reclaiming flags atomicly. 2. remove -W cmd arg, use -w instead to specify the max cache size 3. rename object_cache_access_begin{end} to get{put}_cache_entry() 4. move read{write}_cache_object() and push_cache_object() forward to avoid forwarding function declaration 5. use byte as the cache size unit, avoid recalucation of data_length 6. move the defination of list_for_each_entry_revert_safe_rcu() to list.h 7. rename some functions and refactor some code sequence. v1 ----> v2: 1. Merge the object tree and the dirty tree to make them share the cached entries. 2. remove check_cache_object_sanity() 3. remove the dirty flags using the bmap instead to check whether the entry is dirty. This patchset is to reclaim the cached object when the cache size reaches to max size specified by use in command line, the usage is like: #sheep/sheep -w 100 -d /home/levin/store/0 -p 7000 -z 0 As the example above, we specified the max cache size to 100M. I add a object tree for each VDI to store all the cached entries, which are shared with the dirty tree, and a lru list implemented by rculist for cache scheduling, when a cached object is accessed(read or write), it's moved to the head of the list, and when the cache size reaches the max cache size, we start to reclaim the cached object and the relative cache entry to 80% of the max size. Every time we create a new cached object, we create a new cache entry at the same time, so next time we check whether the cache is exist, we don't just goto disk using open(), but we find in the cached object tree to find whether the object is being cached. We didn't do reclaim work in the following case: 1. There's already an existing reclaim work. 2. The object to be reclaimed is being accessed. 3. The cache is flushing. levin li (8): sheep: use cmd argument -w to specify a max cache size object cache: add object cache tree for every VDI object cache: merge active and inactive dirt_tree/list object cache: use rwlock to replace mutex lock for per-vdi cache object cache: schedule the object cache in a lru list object cache: reclaim cached objects when cache reaches the max size object cache: refactor object_cache_remove() object cache: add a object_list for cache entry for cache deleting include/internal_proto.h | 1 + include/list.h | 10 + sheep/object_cache.c | 1001 ++++++++++++++++++++++++++++++++-------------- sheep/sheep.c | 20 +- sheep/sheep_priv.h | 3 + sheep/store.c | 8 + 6 files changed, 740 insertions(+), 303 deletions(-) |