[stgt] persistent reservations through power loss

[Alexander Gordeev]

В Tue, 24 Sep 2013 10:13:42 -0700
Hi!

Lee Duncan <lduncan at suse.com> пишет:

> On 09/18/2013 08:41 AM, Alexander Gordeev wrote:
> > Hi!
> > 
> > I'd like to add correct handling of APTPL bit in PR OUT commands. I
> > have the following plan:
> > 1. create a new parameter ('pr_storage' ?) for target/LUN where tgtd
> > should store registrations and reservations according to the SCSI spec
> > 2. tgtd should read this storage LUN creation and populate lists of
> > registrations and reservations and save the fd in the struct scsi_lu
> > 3. adjust capabilities: return 1 for APTPL if the parameter is set for
> > a LUN
> > 4. when tgtd receives PR OUT command with APTPL set, it should
> > asynchronously write changes to the store
> 
> I'm worried about the obvious race condition potential if you write this
> information asynchronously. What if this asynchronous write fails? What
> if the storage containing this data is lost or goes offline?
> 
> I believe you need your backing store writes to be synchronous if you
> want persistence to work correctly.

By asynchronous here I meant that the writes should not block the main
thread. Of course, the registration/reservation requests should be
answered only when the data is written to disk.

I initially thought that I have to use AIO, but I discovered then that
there is no asynchronous fdatasync function (AFAIK io_fdsync blocks
too). :( So I'm going to use a separate thread now and synchronous
functions as you say.

> > Are you interested in these changes? What do you think about the plan?
> > Is there anything else I should do too?
> > At the very least, other possible problems aside, I think you should at least wait for the
> > I also have some questions regarding storage format and writing changes
> > asynchronously.
> > 
> > I'd like to do it in a way that minimizes fs metadata changes so it
> > should have some fixed size if possible. Are there any limits on the
> > number of sessions per LUN or registartions per LUN in tgtd?
> 
> I would have to look at the SCSI standard, but I'm sure there as check
> condition a target can return when it has no more memory (which is why
> you normally limit the list of registrants).

Yes, I'd like to limit the list of registrants if this ok. I didn't find
any limits in the current implementation.

I still have some concerns about which scope should each persistent
store file have. IMHO we have three options:

1. Have only one file for a single tgtd instance which will hold all
information for all targets and LUNS. Obviously this is not flexible:
one can't easily move a target/LUN to another tgtd instance.

2. Have one file per target. One can easily move a target to another
tgtd instance. Other pros are: less files then in per-LUN option, one
can setup a target easily before enabling any access to it - this
offers some flexibility too. Cons: one can't easily move a single LUN.

3. Have one file per LUN. There is a problem here because I have to
read persistent storage while the LUN is created or there will be a
race. AFAIK this is not quite simple with current implementation. Also
this means lots of files.

What do you think?

-- 
  Alexander
--
To unsubscribe from this list: send the line "unsubscribe stgt" in
the body of a message to majordomo at vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html