[sheepdog] [PATCH 3/4 v3] sheep : add new strage type "tree"

[Liu Yuan]

On Mon, Mar 23, 2015 at 02:18:24PM +0900, Saeki Masaki wrote:
> On 2015/03/21 21:02, Liu Yuan wrote:
> >On Fri, Mar 20, 2015 at 06:39:08PM +0900, Saeki Masaki wrote:
> >>Current sheepdog stores whole objects in single directory like "/var/lib/sheepdog/obj"
> >>This mechanism is difficult to handle massive files when increasing cluster volume.
> >>
> >>In particular, inode object having special informations about VDI,
> >>so it is preferable to divide.
> >>
> >>new storage type named "tree"
> >>It separates the inode object and data object.
> >>
> >>How to use ,
> >>specify the --store option at the time format
> >>dog cluster format --store tree
> >>
> >>v2: refactor using common functions for store driver
> >>     use check_store_type to identify tree store_driver
> >>
> >>v3: rename and little modify function
> >>
> >>Signed-off-by: Masaki Saeki <saeki.masaki at po.ntts.co.jp>
> >>---
> >>  sheep/Makefile.am         |    2 +-
> >>  sheep/sheep_priv.h        |   21 ++
> >>  sheep/store/common.c      |    5 +
> >>  sheep/store/md.c          |   14 +
> >>  sheep/store/plain_store.c |    1 +
> >>  sheep/store/tree_store.c  |  758 +++++++++++++++++++++++++++++++++++++++++++++
> >>  6 files changed, 800 insertions(+), 1 deletions(-)
> >>  create mode 100644 sheep/store/tree_store.c
> >>
> >>diff --git a/sheep/Makefile.am b/sheep/Makefile.am
> >>index 3ddd761..9dedb03 100644
> >>--- a/sheep/Makefile.am
> >>+++ b/sheep/Makefile.am
> >>@@ -28,7 +28,7 @@ sheep_SOURCES		= sheep.c group.c request.c gateway.c vdi.c \
> >>  			  journal.c ops.c recovery.c cluster/local.c \
> >>  			  object_cache.c object_list_cache.c \
> >>  			  store/common.c store/md.c \
> >>-			  store/plain_store.c \
> >>+			  store/plain_store.c store/tree_store.c \
> >>  			  config.c migrate.c
> >>  if BUILD_HTTP
> >>diff --git a/sheep/sheep_priv.h b/sheep/sheep_priv.h
> >>index e58901f..3f1bf6e 100644
> >>--- a/sheep/sheep_priv.h
> >>+++ b/sheep/sheep_priv.h
> >>@@ -84,6 +84,11 @@ enum REQUST_STATUS {
> >>  	REQUEST_DROPPED
> >>  };
> >>+enum store_id {
> >>+	PLAIN_STORE,
> >>+	TREE_STORE
> >>+};
> >>+
> >>  struct request_iocb {
> >>  	uint32_t count;
> >>  	int efd;
> >>@@ -235,6 +240,7 @@ struct vdi_info {
> >>  struct store_driver {
> >>  	struct list_node list;
> >>+	enum store_id id;
> >>  	const char *name;
> >>  	int (*init)(void);
> >>  	bool (*exist)(uint64_t oid, uint8_t ec_index);
> >>@@ -269,6 +275,20 @@ int default_format(void);
> >>  int default_remove_object(uint64_t oid, uint8_t ec_index);
> >>  int default_get_hash(uint64_t oid, uint32_t epoch, uint8_t *sha1);
> >>  int default_purge_obj(void);
> >>+
> >>+int tree_init(void);
> >>+bool tree_exist(uint64_t oid, uint8_t ec_index);
> >>+int tree_create_and_write(uint64_t oid, const struct siocb *iocb);
> >>+int tree_write(uint64_t oid, const struct siocb *iocb);
> >>+int tree_read(uint64_t oid, const struct siocb *iocb);
> >>+int tree_link(uint64_t oid, uint32_t tgt_epoch);
> >>+int tree_update_epoch(uint32_t epoch);
> >>+int tree_cleanup(void);
> >>+int tree_format(void);
> >>+int tree_remove_object(uint64_t oid, uint8_t ec_index);
> >>+int tree_get_hash(uint64_t oid, uint32_t epoch, uint8_t *sha1);
> >>+int tree_purge_obj(void);
> >>+
> >>  int for_each_object_in_wd(int (*func)(uint64_t, const char *, uint32_t,
> >>  				      uint8_t, struct vnode_info *, void *),
> >>  			  bool, void *);
> >>@@ -404,6 +424,7 @@ void queue_cluster_request(struct request *req);
> >>  int prepare_iocb(uint64_t oid, const struct siocb *iocb, bool create);
> >>  int err_to_sderr(const char *path, uint64_t oid, int err);
> >>  int discard(int fd, uint64_t start, uint32_t end);
> >>+bool store_id_match(enum store_id id);
> >>  int update_epoch_log(uint32_t epoch, struct sd_node *nodes, size_t nr_nodes);
> >>  int inc_and_log_epoch(void);
> >>diff --git a/sheep/store/common.c b/sheep/store/common.c
> >>index 8959392..22a915f 100644
> >>--- a/sheep/store/common.c
> >>+++ b/sheep/store/common.c
> >>@@ -102,6 +102,11 @@ int discard(int fd, uint64_t start, uint32_t end)
> >>  	return ret;
> >>  }
> >>+bool store_id_match(enum store_id id)
> >>+{
> >>+	return (sd_store->id == id);
> >>+}
> >>+
> >>  int update_epoch_log(uint32_t epoch, struct sd_node *nodes, size_t nr_nodes)
> >>  {
> >>  	int ret, len, nodes_len;
> >>diff --git a/sheep/store/md.c b/sheep/store/md.c
> >>index 87ab759..09438d8 100644
> >>--- a/sheep/store/md.c
> >>+++ b/sheep/store/md.c
> >>@@ -212,6 +212,20 @@ static int for_each_object_in_path(const char *path,
> >>  		if (unlikely(!strncmp(d->d_name, ".", 1)))
> >>  			continue;
> >>+		/* recursive call for tree store driver sub directories*/
> >>+		if (store_id_match(TREE_STORE)) {
> >
> >I just wonder, if store_id_match is necessary since we can check the store
> >driver by strcmp(name) as your first version. So what is motivation behind
> >to use store_id_match()?
> 
> In the future, I'm hoping that that store driver will increase.
> At that time, it is considered to be desirable keep abstracts
> 
> Thanks, Saeki.

keep abstracts? I don't get it. Well, I see name compare to distiguish store
driver as a temparory manner for fast development. Since we have already
abstract out store operstions, an ideal scenario is, 'dog format' will install
the right store and sheep only call store->xxx() to handle the store operation.
No name compare or store_id_match() is necessary because sheep doesn't need to
know about it. This is the ideal abstraction. With this considered, I don't
think store_id_match() is necessary because in the future, we don't this kind of
check code.

Thanks,
Yuan