[Stgt-devel] stgt backing store & SSC

[Ming Zhang]

On Fri, 2007-08-03 at 10:53 +1000, Mark Harvey wrote:
> On 8/3/07, Ming Zhang <blackmagic02881 at gmail.com> wrote:
> > On Thu, 2007-08-02 at 14:58 -0400, Pete Wyckoff wrote:
> > > markh794 at gmail.com wrote on Tue, 31 Jul 2007 19:57 +1000:
> > > > For Variable block SSC device, the block size written needs to be
> > > > tracked.
> > > [..]
> > > > My current thoughts of a solution:
> > > > ========================
> > > > A block header describes each block written -> Analogy to the 'tar'
> > > > format where a header is written, followed by the 'data' followed by
> > > > another header, followed by more data...repeat...until blank header...
> > > [..]
> > > > However the current implementation for iSCSI -> bs_sync uses a
> > > > pread64()/pwrite64() and writes data based on information stored in
> > > > scsi_cmd -
> > > >  pwrite64(fd, cmd->uaddr, cmd->len, cmd->offset)
> > > >  pread64(fd, cmd->uaddr, cmd->len, cmd->offset)
> > > >
> > > >
> > > > Would it be OK to add a 'blk_header' structure to struct scsi_cmd and if
> > > > blk_header is set, write this blk header as well ?
> > > > I will attempt to put the above idea into code and submit for comment...
> > >
> > > I think rather than trying to add stuff to existing backing stores
> > > that you should consider writing your own.  You need to store both
> > > "metadata" (block descriptors) and data, and none of the BSes are
> > > set up for that.
> > >
> > > The complexity of trying to glue in the blk_header so that all BSes
> > > know how to tack that on top seems big.  Then you have to tell them
> > > to read the header, and consider fields in that when determining how
> > > much further data to read.  It gets messy fast.
> >
> > also considering tape can have compression and encryption, then each
> > block is variable size even in fixed size mode. so support variable size
> > is a must eventually.
> 
> Yep. The SSC metadata (header for each tape block) contains a
> structure with a 'block type' , size of original data & size of data
> stored.
> 
> All accounted for. Just no code for compression or encryption - as yet..
> 
> enum {
>         BLK_NOOP,
>         BLK_UNCOMPRESS_DATA,
>         BLK_COMPRESSED_DATA,
>         BLK_ENCRYPTED_DATA,
>         BLK_FILEMARK,
>         BLK_BOT,
>         BLK_EOD,
> };

are these bits? can i have multiple bits set like compressed before
encryption? also leave room for data deduplication for example storing a
token. ;)


> struct blk_header {
>         uint32_t blk_type;
>         uint32_t blk_size;
>         uint32_t disk_blk_size;
>         loff_t blk_number;
>         loff_t prev_blk;
>         loff_t curr_blk;
>         loff_t next_blk;
> };
> 
> 
> > > In the lu_init for the ssc device, you can change the BS.  Then in
> > > your READ/WRITE command handler, you can make calls into your own
> > > BS that knows how to store this information.  (Look out for the
> > > allocation size hack in target.c for TYPE_SPT; you may need
> > > something similar.)
> > >
> > > In general, the current separation of backing store from device only
> > > really works for block storage.  We need an OSD-only backing store
> > > for OSD too (not in tree).  May be cleaner just to rid the concept
> > > of backing store from everything except the device code.
> > >
> > >               -- Pete
> > > _______________________________________________
> > > Stgt-devel mailing list
> > > Stgt-devel at lists.berlios.de
> > > https://lists.berlios.de/mailman/listinfo/stgt-devel
> > --
> > Ming Zhang
> >
> >
> > @#$%^ purging memory... (*!%
> > http://blackmagic02881.wordpress.com/
> > http://www.linkedin.com/in/blackmagic02881
> > --------------------------------------------
> >
> >
-- 
Ming Zhang


@#$%^ purging memory... (*!%
http://blackmagic02881.wordpress.com/
http://www.linkedin.com/in/blackmagic02881
--------------------------------------------