[Stgt-devel] sg_turs on stgt iscsi drive is very slow

[FUJITA Tomonori]

From: Ming Zhang <blackmagic02881 at gmail.com>
Subject: Re: [Stgt-devel] sg_turs on stgt iscsi drive is very slow
Date: Mon, 11 Dec 2006 18:47:25 -0500

> On Tue, 2006-12-12 at 08:21 +0900, FUJITA Tomonori wrote:
> > From: Ming Zhang <blackmagic02881 at gmail.com>
> > Subject: Re: [Stgt-devel] sg_turs on stgt iscsi drive is very slow
> > Date: Mon, 11 Dec 2006 18:18:06 -0500
> > 
> > > On Tue, 2006-12-12 at 08:02 +0900, FUJITA Tomonori wrote:
> > > > From: Ming Zhang <blackmagic02881 at gmail.com>
> > > > Subject: [Stgt-devel] sg_turs on stgt iscsi drive is very slow
> > > > Date: Mon, 11 Dec 2006 11:58:27 -0500
> > > > 
> > > > > Pulled out the linux-2.6-target git tree, compiled stgt and iet under
> > > > > same kernel. Export a 4 disk raid0 device to ini. Ini is open-iscsi
> > > > > r598.
> > > > > 
> > > > > Want to know how efficient stgt handle context switch. 
> > > > > 
> > > > > run  sg_turs -t -n=10000 /dev/sg0 on IET drive. results is 2521/sec.
> > > > 
> > > > With IET, several threads handle TUR. With stgt, the single thread
> > > > handles TUR (synchronously). stgt asynchronously handles only I/O
> > > > commands (READ, WRITE, etc), which are expected to be performed
> > > > efficiently.
> > > 
> > > if i set Wthreads to 1 and "ps axu" to double check there is only 1
> > > worker thread, i still get 5470 op/sec.
> > > 
> > > also when run stgt, the top show almost no cpu activity. 
> > 
> > With that configuration, IET still uses three threads (two network
> > threads and worker thread).
> 
> could u roughly explain a code flow of stgt for a TUR? why I see no cpu
> utilization during stgt run? for iet, i saw 4-5% cpu utilization so i
> know it is working. but for stgt i can not see any from top or vmstat.

One process reads an iSCSI pdu, and builds and sends response. Then
the process reads another iSCSI pdu...


> > > > So I don't think the results are related with context switch at all.
> > > 
> > > ok, it is not related to CS. but then why stgt is so slow?
> > > 
> > > it should be easy to reproduce this at u side. if u can not reproduce
> > > this slowness, i will check my environment again. i clone u git tress
> > > just yesterday.
> > 
> > Please do real workload tests. If stgt is slow with such workloads,
> > I'll dig into it. Thanks.
> 
> i tried simple dd read from iscsi drive and only give me 30MB/s. but i
> forgot if i disable the debug or not. i will run it again and report.

I used disktest months ago and got the comparable results. After that,
stgt switch to CMD_EPOLL_WAIT from AIO fd for event notification. That
might effect the performance, but probably stgt will switch to kevent
in the future.