On Fri, Jul 27, 2012 at 10:59:25AM -0500, Robert Nesius wrote: > On Fri, Jul 27, 2012 at 7:20 AM, Florin Iucha <florin at iucha.net> wrote: > > > On Thu, Jul 26, 2012 at 11:44:05PM -0500, Robert Nesius wrote: > > > With iSCSI, where is the filesystem management overhead?Is the > > > filesystem overhead is on the client side with the server just receiving > > > low-level I/O operations that go straight to disk, whereas with CIFS the > > > server is having to handle mapping the I/O from the filesystem layer > > > through to hardware layer, causing it to be slower on it's responses > > > (ACKS)? I've never worked with it myself... just curious. > > > > Yes, that's how it works. However, I am measuring the performance of > > the system composed of the two machines (plus the switch) and iSCSI > > shows twice the performance of CIFS. Somebody has to do the > > filesystem dirty work, be it on the client or on the server. > > > > I could see where you have a workload that you spread across two > > sub-systems and if one of them reaches capacity, that limits the > > throughput of the entire system (some variation of Amdahl's law). > > > I think there is more post-processing on the server side The server load is 5% in both cases. > and that's going > to slow down server responses and that's going to push back upstream and > cause writes to wait. On the contrary, with CIFS, more data gets sent through the network, because the client keeps track of cluster allocation, reads and updates the filesystem metadata. With CIFS is "here's the file, kthxby". > > But both boxes are very powerful (3.3GHz 6-core for workstation, > > 2.8GHz 4-core for server) and completely idle that neither is the > > bottleneck. I'm writing a 11GB file, and the server has 16GB of RAM. > > > If they are idle then that suggests to me they are blocking for I/O > (waiting). Of course they are waiting for network IO. The question is why is CIFS waiting for network IO and what can I do to reduce that wait. > Your drive lights are lit up pretty solidly yeah? I'm looking at dstat output which is more reliable than eyeballing two LEDs that face different ways, separated by 10 feet of cables and boxes. > Are you writing to a RAM DISK? No, I'm writing to a SATA disk that can write sequentially more than 100MBytes/second. > > The CIFS results were so bad, I was concerned there was a problem with > > the hardware. It just occurred me two days ago that I could flip things > > around, use iSCSI and test the system performance in a different way. > > > > I'm thinking of ways to test my hypothesis. I'm not sure changing the size > of the TCP window on the cifs server will have an impact if the CIFS client > is blocking and waiting for a response to a write. > > Maybe watch each of your tests with wireshark and watch latencies? How would that help me? What would that tell me that I don't know already? CIFS and iSCSI use different ports, different protocols, different services... Cheers, florin -- Beware of software written by optimists! -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: Digital signature URL: <http://mailman.mn-linux.org/pipermail/tclug-list/attachments/20120727/4804a974/attachment.pgp>