[sheepdog] [PATCH v10 06/19] sheep: introduce sparse objects

[Hitoshi Mitake]

At Tue, 17 Jun 2014 14:25:37 +0800,
Liu Yuan wrote:
> 
> On Tue, Jun 17, 2014 at 02:16:48PM +0800, Liu Yuan wrote:
> > On Tue, Jun 03, 2014 at 12:09:00AM +0900, Hitoshi Mitake wrote:
> > > We don't need to care about a fragmentation when
> > >  - the object is unlikely to be accessed sequentially, and
> 
> This is wrong assumption, many application do access volume sequentially like
> stream based APP(audio, vedio, etc.).

I don't say every object is not accessed sequentially. It is just a condition.

> 
> > >  - the object is read-only.
> > > 
> > > In that sense, we can make the objects sparse if they are not data
> > > objects and writable ones.
> > > 
> > > This fixes the problem that sheepdog consumes many disk spaces for
> > > deleted vdi objects if your filesystem supports FALLOC_FL_PUNCH_HOLE.
> > 
> > I don't understand this patch. If we want sparse objects, we can just stop
> > preallocation and no need to trim objects in the hottest path. Am I missing
> > anything in the sense that in one way, we preallocate the whole object, but in
> > the other we punch hole of objects.
> > 
> 
> Probably we should only make sparse of deleted inode objects, which is a
> placeholder and no pratical use. But for other objects, we should preallocate to
> avoid fragmentation and never try to trim objects in the hottest IO pathes. As

The new scheme of sparse object doesn't affect ordinal data
objects so it doesn't degrade performance of hot IO path. Its main
purpose is reducing disk size of ledger objects.

> far as I remember, trimming functions are problematic for erasure coding and
> cluster snapshot. Is this well-tested with EC and sha1 stuff?

At least tests/functional doesn't report problems.

Thanks,
Hitoshi