Re: [squid-users] Squid network read()'s only 2k long?

On Mon, 1 Nov 2010 23:20:52 +0000, Declan White <declanw_at_is.bbc.co.uk>
wrote:
> On Mon, Nov 01, 2010 at 10:55:12PM +0000, Declan White wrote:
>> On Mon, Nov 01, 2010 at 09:36:53PM +0000, Amos Jeffries wrote:
>> > On Mon, 1 Nov 2010 15:00:21 +0000, declanw_at_is.bbc.co.uk wrote:
>> > > I went for a rummage in the code for the buffer size decisions, but
>> > > got
>> > > very very lost in the OO abstractions very quickly. Can anyone
point
>> > > me at
>> > > anything I can tweak to fix this?
>> >
>> > It's a global macro defined by auto-probing your operating systems
TCP
>> > receiving buffer when building. Default is 16KB and max is 64KB.
There
>> > may
>> > also be auto-probing done at run time.
>> >
>> > It is tunable at run-time with
>> > http://www.squid-cache.org/Doc/config/tcp_recv_bufsize/
>>
>> Oh thank God! Thanks :) (and annoyed with myself that I missed that)
>
> Nuts.. actually, that didn't do anything :(
>
> 17314: write(16, " G E T / c f g m a n .".., 639) = 639
> 17314: ioctl(6, DP_POLL, 0x100459B90) = 1
> 17314: write(6, "\0\0\010\b\0\0\0\0\0\010".., 16) = 16
> 17314: ioctl(6, DP_POLL, 0x100459B90) = 1
> 17314: read(11, " H T T P / 1 . 1 2 0 0".., 2046) = 2046
> 17314: write(6, "\0\0\0\n\004\0\0", 8) = 8
> 17314: ioctl(6, DP_POLL, 0x100459B90) = 2
> 17314: write(10, " H T T P / 1 . 0 2 0 0".., 2180) = 2180
> 17314: read(11, " f o n t - s i z e : 3".., 2046) = 834
> 17314: ioctl(6, DP_POLL, 0x100459B90) = 2
> 17314: write(10, " f o n t - s i z e : 3".., 834) = 834
> 17314: read(11, " n o n e ;\n }\n\n # m".., 2046) = 1066
> 17314: ioctl(6, DP_POLL, 0x100459B90) = 1
> 17314: write(10, " n o n e ;\n }\n\n # m".., 1066) = 1066
> 17314: write(8, " [ 0 1 / N o v / 2 0 1 0".., 403) = 403
>
> It's still reading from the remote server in 2046 byte lumps, which
meant
> three trips round the event loop where it might only have needed one.
>
> I'm guessing that setting is for the kernel level TCP receive buffer,
and
> not the application read-from-that-buffer size.
>
> I even doubled HTTP_REQBUF_SZ in defines.h for fun, and that did nothing
> either. I can't find where the read() size might be decided in the code.

Hmm. I think I've tracked this down. It appears to be MemBuf::init()
defaulting to 2KB of buffer. The default is defined in src/Membuf.cc.
Though I'm unsure why an exponential grow() is not triggered to make that
third read 4K.

If you can confirm that I'll work on making that config setting affect the
buffer initial values for the future.

Amos
Received on Tue Nov 02 2010 - 01:15:00 MDT