From: Robin Dong <sanbai at taobao.com> After testing in large data environment, we find out that the performance of sheepdog will reduce raplidly when data is bigger than 2TB. The reason is in sd_inode_set_vid(), it only set one 'vid' at one time, and need to read ext-node out, set vid, then write ext-node back, which need read and write 4MB data. To promote the performance, we let sd_inode_set_vid() to set range of vids one time and introduce cache for B-tree code, therefore it only need read and write 4MB data at once after set many vids. Signed-off-by: Robin Dong <sanbai at taobao.com> --- v1-->v2: 1. add 'no rationale' for NUMBER_OF_CACHE 2. change function name from 'cache_xxx' to 'icache_xxx' 3. add comment to explain the cache can't run in concurrent env. 4. replace 'writer_c/reader_c' to 'cache_writer/cache_reader' Robin Dong (3): lib: create new function set_vid_for_btree() lib: change interface sd_inode_set_vid() lib: add cache version of writer and reader dog/dog.h | 7 +- include/sheepdog_proto.h | 4 +- lib/sd_inode.c | 200 ++++++++++++++++++++++++++++++++++++++--------- sheep/http/oalloc.c | 4 +- sheep/sheep_priv.h | 7 +- sheepfs/volume.c | 6 +- 6 files changed, 182 insertions(+), 46 deletions(-) -- 1.7.12.4 |