[stgt] read-caching by tgtd

[Maurits van de Lande]

Hello Arne,

..
>The reason for this setup is to have HA if one target fails, the other takes over (after 20s timeout)

Then you do not need a dual primary setup.
Have you read the following post?
http://fghaas.wordpress.com/2011/11/29/dual-primary-drbd-iscsi-and-multipath-dont-do-that/

I'm using tgt and drbd in a active/passive corosync cluster setup and this works fine. Why do you use a dual primary multipath setup?

Regards,

Maurits


-----Oorspronkelijk bericht-----
Van: stgt-owner at vger.kernel.org [mailto:stgt-owner at vger.kernel.org] Namens Arne Van Theemsche
Verzonden: donderdag 16 mei 2013 11:19
Aan: stgt at vger.kernel.org
Onderwerp: read-caching by tgtd

Hi there

I have following setup

2 identical target-machines consisting of

HW raid controller
LVM on top of it
DRBD on top of LV (dual primary, don't panic yet) exported-as-target



both target are imported by 1 initiator, making a multipath device of it (in Active/Backup), so no load balancing

on that device I make an xfs FS

The reason for this setup is to have HA if one target fails, the other takes over (after 20s timeout)

Everything keeps going fine, until the initiator machines fails/reboot and the multipath choose at boottime another path (in other words:
choose the other target as primary).

to make a long story short: It seems that the tgtd process does some form of read caching, because if I disable the multipathing (ending up with 2 accessable block devices) and mount them one by one (not together off course) the FS structure is not in sync. To exclude the drbd process as the guilty one, I mount the FS on both target devices (using the drbd block device) and both targets give the right FS structure. It's only one of the two exported targets that give the wrong info, so the only building block left over is the tgtd-process doing some form of caching. As a final test, if I reboot the initiator, the wrong info keeps comming back from the target

in the <target> section I put
   write-cache off
   read-cache off

the write-cache seems to have effect, but the read-cache seems silently ignored.

Any advice?

kind regards
--
To unsubscribe from this list: send the line "unsubscribe stgt" in the body of a message to majordomo at vger.kernel.org More majordomo info at  http://vger.kernel.org/majordomo-info.html


--
To unsubscribe from this list: send the line "unsubscribe stgt" in
the body of a message to majordomo at vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html