[sheepdog] [PATCH v2 03/11] sheep: introduce generational reference counting for object reclaim

[MORITA Kazutaka]

At Fri, 28 Jun 2013 23:53:04 -0700,
Kai Zhang wrote:
> 
> MORITA Kazutaka <morita.kazutaka at gmail.com> writes:
> 
> > From: MORITA Kazutaka <morita.kazutaka at lab.ntt.co.jp>
> >
> > Generational reference counting is an algorithm to reclaim data
> > efficiently without race conditions on distributed system.  This
> > extends vdi objects structure to store generational reference counts,
> > and increments the counts when creating snapshots.
> >
> > Signed-off-by: MORITA Kazutaka <morita.kazutaka at lab.ntt.co.jp>
> > ---
> >  include/sheepdog_proto.h |    9 +++++++--
> >  sheep/vdi.c              |    8 +++++++-
> >  2 files changed, 14 insertions(+), 3 deletions(-)
> >
> > diff --git a/include/sheepdog_proto.h b/include/sheepdog_proto.h
> > index beef0a6..bb0f253 100644
> > --- a/include/sheepdog_proto.h
> > +++ b/include/sheepdog_proto.h
> > @@ -101,8 +101,7 @@
> >  #define SD_MAX_VDI_SIZE (SD_DATA_OBJ_SIZE * MAX_DATA_OBJS)
> >
> >  #define SD_INODE_SIZE (sizeof(struct sd_inode))
> > -#define SD_INODE_HEADER_SIZE (sizeof(struct sd_inode) - \
> > -			      sizeof(uint32_t) * MAX_DATA_OBJS)
> > +#define SD_INODE_HEADER_SIZE offsetof(struct sd_inode, data_vdi_id)
> >  #define SD_ATTR_OBJ_SIZE (sizeof(struct sheepdog_vdi_attr))
> >  #define CURRENT_VDI_ID 0
> >
> > @@ -193,6 +192,11 @@ struct sd_rsp {
> >	};
> >  };
> >
> > +struct generation_reference {
> > +	int32_t generation;
> > +	int32_t count;
> > +};
> > +
> >  struct sd_inode {
> >	char name[SD_MAX_VDI_LEN];
> >	char tag[SD_MAX_VDI_TAG_LEN];
> > @@ -209,6 +213,7 @@ struct sd_inode {
> >	uint32_t parent_vdi_id;
> >	uint32_t child_vdi_id[MAX_CHILDREN];
> >	uint32_t data_vdi_id[MAX_DATA_OBJS];
> > +	struct generation_reference data_ref[MAX_DATA_OBJS];
> >  };
> >
> 
> Does that mean it will incur an incompatable change?
> Does we need to clean up all data before we upgrade to a new version?

Basically the old format can be still usable, but the existing objects
can be reclaimed wrongly since the old inode doesn't contain correct
reference counts.  In the next version, I think of adding a check to
avoid this problem and a command to update the object reference counts
for the existing users.

Thanks,

Kazutaka