[Stgt-devel] 2.6.24 kernel and LIO Target memory mapping
Nicholas A. Bellinger
nab
Mon Feb 4 14:27:20 CET 2008
On Sat, 2008-02-02 at 05:26 -0800, Nicholas A. Bellinger wrote:
> Hi Bart,
>
> Ok, I have 2.6.24 running on ppc64 doing iSCSI/HD on PS3-Linux. The
> changes for struct scatterlist->page moving to struct
> scatterlist->page_link where pretty straightforward, considering the LIO
> storage engine does not depend on struct scatterlist. I am going to
> take another look at the diffs later today and make sure everything
> looks corerct, and will make the commits then.
>
> If you could test these on your setup with 2.6.24 in the non IPoIB case
> (that from my previous emails I am guessing will be fine on your setup)
> I would really appreciate it. I will put 2.6.24 in VM on the
> Linux-iSCSI.org fabric in the upcoming days, and do some additional
> testing. Getting the CentOS v5u1 x86_64 builds release are a bit higher
> priority than 2.6.24, but I think that the 2.6.24 changes are reasonable
> and do not cause concern with the LIO v2.9 codebase.
>
> Have you made any futher progress on debugging the issue LIO Target with
> IPoIB..?
>
> Many thanks for your most valuable of time,
>
> --nab
>
> On Fri, 2008-02-01 at 06:50 -0800, Nicholas A. Bellinger wrote:
> > Hi There,
> >
> > I will doing a 2.6.24 build for LIO on PS3-Linux this weekend now that
> > ps3rom.c is reporting the proper struct scsi_host_template->max_sectors.
> > Also in the queue are releasing updated builds for CentOS v5.1 x86_64.
> > I have recently upgraded one pair of core nodes on the Linux-iSCSI.org
> > fabric to the newest CentOS release, and minus a few minor issues with
> > the upgrade process, everything is looking very stable. The few issues
> > that I had (an lvremove issue on v4.5, and qemu-dm performance with HVM
> > on v5.0) have been resolved and the fabric is running much smoother for
> > the VMs that are providing OCFS2 cluster storage to the LIO debian and
> > ubuntu repositories.
> >
> > I will let you know when I get 2.6.24 building. Until then, feel free
> > to post the build failure log here.
> >
> > Many thanks for your most valuable of time,
> >
> > --nab
> >
> > On Fri, 2008-02-01 at 15:09 +0100, Bart Van Assche wrote:
> > > Hello,
> > >
> > > The Linux-iSCSI target (kernel module) does not compile on the 2.6.24
> > > kernel because of changes in the scatterlist API. Has a target date
> > > for a version that compiles with the 2.6.24 kernel headers already
> > > been set ?
> > >
> > > Bart Van Assche.
> > >
>
Here are the changes to transport_memcpy_[write,read]_[contig,sg]()
respectively. This functions are legacy within v2.9 LIO SE, and are
currently unused in kernel mode because the SE core does not rely on
struct scatterlist.
Index: target/iscsi_target_transport.c
===================================================================
--- target/iscsi_target_transport.c (revision 205)
+++ target/iscsi_target_transport.c (revision 206)
@@ -4181,7 +4181,7 @@
if (length > total_length)
length = total_length;
- src = GET_ADDR_SG(sg_s, i);
+ src = GET_ADDR_SG(&sg_s[i]);
memcpy(dst, src, length);
@@ -4211,12 +4211,12 @@
if (length > total_length)
length = total_length;
- dst = GET_ADDR_SG(sg_d, i) + dst_offset;
+ dst = GET_ADDR_SG(&sg_d[i]) + dst_offset;
if (!dst)
BUG();
i++;
- src = GET_ADDR_SG(sg_s, j) + src_offset;
+ src = GET_ADDR_SG(&sg_s[j]) + src_offset;
if (!src)
BUG();
@@ -4228,7 +4228,7 @@
if (length > total_length)
length = total_length;
- dst = GET_ADDR_SG(sg_d, i) + dst_offset;
+ dst = GET_ADDR_SG(&sg_d[i]) + dst_offset;
if (!dst)
BUG();
@@ -4238,7 +4238,7 @@
} else
dst_offset = length;
- src = GET_ADDR_SG(sg_s, j) + src_offset;
+ src = GET_ADDR_SG(&sg_s[j]) + src_offset;
if (!src)
BUG();
j++;
@@ -4269,7 +4269,7 @@
if (length > total_length)
length = total_length;
- dst = GET_ADDR_SG(sg_d, i);
+ dst = GET_ADDR_SG(&sg_d[i]);
memcpy(dst, src, length);
These changes are followed up by transport_map_sg_to_mem()
and transport_map_mem_to_sg().. The latter is the default path for
LIO SE v2.9 to v2.6 Linux storage subsystems along with the other completely
virtual subsystem drivers. Note the former does reverse contigious
scatterlist array mapping to SE linked list memory which is then handed to
the SCSI transport, in the LIO case, traditional iSCSI. This is currently
unused, but may be useful in some future cases.
@@ -4299,12 +4299,12 @@
if (length > total_length)
length = total_length;
- src = GET_ADDR_SG(sg_s, i) + src_offset;
+ src = GET_ADDR_SG(&sg_s[i]) + src_offset;
if (!src)
BUG();
i++;
- dst = GET_ADDR_SG(sg_d, j) + dst_offset;
+ dst = GET_ADDR_SG(&sg_d[j]) + dst_offset;
if (!dst)
BUG();
@@ -4316,7 +4316,7 @@
if (length > total_length)
length = total_length;
- src = GET_ADDR_SG(sg_s, i) + src_offset;
+ src = GET_ADDR_SG(&sg_s[i]) + src_offset;
if (!src)
BUG();
@@ -4326,7 +4326,7 @@
} else
src_offset = length;
- dst = GET_ADDR_SG(sg_d, j) + dst_offset;
+ dst = GET_ADDR_SG(&sg_d[j]) + dst_offset;
if (!dst)
BUG();
j++;
@@ -5191,7 +5191,7 @@
INIT_LIST_HEAD(&se_mem->se_list);
if (*task_offset == 0) {
- se_mem->se_page = sg_s[j].page;
+ se_mem->se_page = GET_PAGE_SG(&sg_s[j]);
se_mem->se_off = sg_s[j].offset;
if (task_size >= sg_s[j].length)
@@ -5208,7 +5208,7 @@
if (saved_task_offset)
*task_offset = saved_task_offset;
} else {
- se_mem->se_page = sg_s[j].page;
+ se_mem->se_page = GET_PAGE_SG(&sg_s[j]);
se_mem->se_off = (*task_offset + sg_s[j].offset);
if ((sg_s[j].length - *task_offset) > task_size) {
@@ -5348,7 +5348,7 @@
while (task_size) {
if (*task_offset == 0) {
- sg[sg_no].page = se_mem->se_page;
+ SET_PAGE_SG(&sg[sg_no], se_mem->se_page);
sg[sg_no].offset = se_mem->se_off;
if (task_size >= se_mem->se_len) {
@@ -5375,7 +5375,7 @@
if (saved_task_offset)
*task_offset = saved_task_offset;
} else {
- sg[sg_no].page = se_mem->se_page;
+ SET_PAGE_SG(&sg[sg_no], se_mem->se_page);
sg[sg_no].offset = (*task_offset + se_mem->se_off);
if ((se_mem->se_len - *task_offset) > task_size) {
The SE algoritims can also potentially accept preallocated memory for
RDMA operations and then map said memory to contiguous struct
scatterlist buffers that are then mapped to the SCSI, BIO, and
[read,write]v() for struct file.
Here are the macro changes for new and legacy operation:
Index: include/iscsi_linux_defs.h
===================================================================
--- include/iscsi_linux_defs.h (revision 207)
+++ include/iscsi_linux_defs.h (revision 208)
@@ -44,15 +44,29 @@
#define inline
#endif
-/* added to address 64bit addressing issues. */
+/*
+ * 2.6.24 provides an updated struct scatterlist API. Use macros for
the new
+ * code, and use inline functions for legacy operation.
+ */
#if LINUX_VERSION_CODE >= KERNEL_VERSION(2,6,24)
# define GET_ADDR_SG(sg) sg_virt(sg)
# define GET_PAGE_SG(sg) sg_page(sg)
# define SET_PAGE_SG(sg, page) sg_assign_page(sg, page)
#else
-# define GET_ADDR_SG(sg) page_address(sg->page) + sg->offset
-# define GET_PAGE_SG(sg) sg->page
-# define SET_PAGE_SG(sg, page) sg->page = page
+#include <linux/scatterlist.h>
+static inline void *GET_ADDR_SG(struct scatterlist *sg)
+{
+ return(page_address(sg->page) + sg->offset);
+}
+static inline struct page *GET_PAGE_SG(struct scatterlist *sg)
+{
+ return(sg->page);
+}
+static inline void SET_PAGE_SG(struct scatterlist *sg, struct page
*page)
+{
+ sg->page = page;
+ return;
+}
#endif
/*
--nab
>
> --~--~---------~--~----~------------~-------~--~----~
> You received this message because you are subscribed to the Google Groups "Linux-iSCSI.org Target Development" group.
> To post to this group, send email to linux-iscsi-target-dev at googlegroups.com
> To unsubscribe from this group, send email to linux-iscsi-target-dev-unsubscribe at googlegroups.com
> For more options, visit this group at http://groups.google.com/group/linux-iscsi-target-dev?hl=en
> -~----------~----~----~----~------~----~------~--~---
>
More information about the stgt
mailing list