[Stgt-devel] [ofa-general] Re: [Ips] Calculating the VA in iSER header

[Pete Wyckoff]

dorfman.eli at gmail.com wrote on Thu, 17 Apr 2008 14:13 +0300:
> On Wed, Apr 16, 2008 at 6:46 PM, Roland Dreier <rdreier at cisco.com> wrote:
> >  > Agree with the interpretation of the spec, and it's probably a bit
> >   > clearer that way too.  But we have working initiators and targets
> >   > that do it the "wrong" way.
> >
> >  Yes... I guess the key question is whether there are any initiators that
> >  do things the "right" way.
> >
> >
> >   > 1. Flag day: all initiators and targets change at the same time.
> >   > Will see data corruption if someone unluckily runs one or the other
> >   > using old non-fixed code.
> >
> >  Seems unacceptable to me... it doesn't make sense at all to break every
> >  setup in the world just to be "right" according to the spec.
> 
> This will break only when both initiator and target will use
> InitialR2T=No, which means allow unsolicited data.
> As far as I know, STGT is not very common (and its version in RHEL5.1
> is considered experimental). Its default is also InitialR2T=Yes.
> Voltaire's iSCSI over iSER target also uses default InitialR2T=Yes.
> So it seems that nothing will break.

I finally got a chance to look at this just now.  I think you mean
default is InitialR2T=No above, which means no unsolicited data.
That is the default case, and true, the two different meanings
of the initiator-supplied VA coincide.

But you missed the impact of immediate data.  We run with the
defaults (I think) that say the first write request packet should be
filled with a bit of the coming data stream.  From iscsid.conf:

    # To enable immediate data (i.e., the initiator sends unsolicited data
    # with the iSCSI command packet), uncomment the following line:
    #
    # The default is Yes
    node.session.iscsi.ImmediateData = Yes

Looking at the offset printed out by your patch, it is indeed
non-zero for the first RDMA read.  Please correct me if I am
mistaken about this---you must have tested all four variations of
with and without the patches on initiator and target side, but I did
not.

Hence I am still a bit unhappy about having to deal with the
fallout, with no way to detect it.  For our local use, I'll keep an
older version of stgt in use until we switch to a new kernel, then
merge up the target side change.  It is a bother, but I can deal
with it.  For other institutions, this lockstep upgrade requirement
will not be obvious until they debug the resulting data corruption.

Still, I do understand why it would be nice to conform to the spec,
and it is maybe a bit cleaner that way too.  Maybe you can help with
the bug reports on stgt-devel during the transition, and maintain
and publish a patch to let it work with old kernels.

		-- Pete