[Stgt-devel] Dealing with filemarks in SSC ...

[Mark Harvey]

On Thu, Jul 31, 2008 at 6:22 AM, Richard Sharpe
<realrichardsharpe at gmail.com> wrote:
> Hi,
>
> I have moved closer to properly implementing filemarks (and setmarks)
> by thinking through the issues associated with the approach I want to
> take.
One other thing I feel needs to be kept in mind. How to support
variable block mode.
(or compressed data for that matter).

The current flat file backing store assumes every block is 512bytes.
If zlib was added and this block was compressed, the backing store
would have no way of determining the size of the data to read (when it
was time to read the data back).
e.g.
Write 512 bytes | zlib(compress) | write to file...

Now when we attempt to read this block back, how much data to we read
off disk ??

Same basic problem with variable block writes. The size of the
original data + size of data written to backing store (perhaps its
been compressed & encrypted by the ssc module) so the attempt to read
this data back will succeed - and/or return appropriate SENSE
under/over-run data.

How are the LOCATE / READ POSITION op codes to be implemented ?

The other fundamental problem I can see is how to detect when we hit a
filemark/setmark/<insert metadata type here>.
Each read, block locate, seek op code is going to have to search the
array to determine if the metadata matches.

Attempting to keep an array of meta-data with pointers seems to be a
difficult way of achieving this.

Sorry - I keep going back to a double-linked-list with the attached
data block as my preferred solution.

Each block header keeps the metadata for the block.
i.e. If the block is a filemark/setmark - there is no data just the metadata.
The block number is stored within the header. So LOCATE / READ
POSITION are simply implemented.
The original size of data + size of data stored is also stored in the
metadata header, making it relatively simple to handle variable block
/ compressed data.

My 2c worth. :)

Cheers
Mark
> That is to keep an array of marks (file and set) with offsets into the
> backing store file. Whether these are then stored as extended
> attributes or as a separate metadata file is an implementation
> decision that I am happy with either way.
>
> By the look of things, the cost of detecting a mark on each read op,
> at least for fixed block size devices, is a couple of additions and a
> comparison in the fast-path (although the code does lots of seeks that
> seem unnecessary at the moment as well). I suspect it will amount to
> the same for variable block-size devices as well.
>
> Once I have btape tests working correctly I will post a patch for consideration.
> _______________________________________________
> Stgt-devel mailing list
> Stgt-devel at lists.berlios.de
> https://lists.berlios.de/mailman/listinfo/stgt-devel
>