Re: 1.2 Cache Size (again)

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Fri, 29 May 1998, Duane Wessels wrote:

> >> Mark Cooke writes:
> >>
> >> >1998/05/29 11:20:39| store_swap_size = 205609k
> >> >1998/05/29 11:20:49| WARNING: Disk space over limit: 199595 KB > 153600 KB
> >>

> It sounds like you are using the async-io code (since you have more
> than one squid process). In that case, you should not have an unlinkd
> process.
>
> Try this patch to src/tools.c. It should enable more debugging
> and may shed some light.

Okay, I have recompiled with the patch, and in the cache.log file I
am seeing a very large number of lines of the form:

1998/05/29 18:18:10| storeGetNextFile: /usr/squid/cache/00/25/000025A7: (24) Too many open files

One for seemingly every cache directory and file I have in the cache.

1998/05/29 18:16:16| Starting Squid Cache version 1.2.beta20 for
i586-pc-linux-g
nu...
1998/05/29 18:16:16| Process ID 20231
1998/05/29 18:16:16| With 256 file descriptors available
1998/05/29 18:16:16| Performing DNS Tests...
1998/05/29 18:16:16| Successful DNS name lookup tests...
1998/05/29 18:16:16| Started 8 'dnsserver' processes
1998/05/29 18:16:16| redirectOpenServers: Starting 5 '/usr/squirm/bin/squirm' processes
1998/05/29 18:16:16| Swap maxSize 153600 KB, estimated 11815 objects
1998/05/29 18:16:16| Target number of buckets: 236
1998/05/29 18:16:16| Using 8192 Store buckets, replacement runs every 10 seconds
1998/05/29 18:16:16| Max Mem size: 16384 KB
1998/05/29 18:16:16| Max Swap size: 153600 KB
1998/05/29 18:16:16| Local cache digest is 'off'
1998/05/29 18:16:16| Rebuilding storage in Cache Dir #0 (DIRTY)
1998/05/29 18:16:16| aio_queue_request: WARNING - Async request queue growing: Length = 17
1998/05/29 18:16:16| aio_queue_request: Perhaps you should increase NUMTHREADS in aiops.c
1998/05/29 18:16:16| aio_queue_request: First 16 items on request
queue
1998/05/29 18:16:16| aio_queue_request: 1 : close -> FD = 20
1998/05/29 18:16:16| aio_queue_request: 2 : close -> FD = 21
1998/05/29 18:16:16| aio_queue_request: 3 : close -> FD = 22
1998/05/29 18:16:16| aio_queue_request: 4 : close -> FD = 23
1998/05/29 18:16:16| aio_queue_request: 5 : close -> FD = 24
1998/05/29 18:16:16| aio_queue_request: 6 : close -> FD = 25
1998/05/29 18:16:16| aio_queue_request: 7 : close -> FD = 26
1998/05/29 18:16:16| aio_queue_request: 8 : close -> FD = 27
1998/05/29 18:16:16| aio_queue_request: 9 : close -> FD = 28
1998/05/29 18:16:16| aio_queue_request: 10 : close -> FD = 29
1998/05/29 18:16:16| aio_queue_request: 11 : close -> FD = 32
1998/05/29 18:16:16| aio_queue_request: 12 : unlink -> /usr/squid/cache/swap.state.last-clean
1998/05/29 18:16:16| aio_queue_request: 13 : close -> FD = 34
1998/05/29 18:16:16| aio_queue_request: 14 : open -> /usr/squid/cache/00/00/00000000
1998/05/29 18:16:16| aio_queue_request: 15 : close -> FD = 35
1998/05/29 18:16:16| aio_queue_request: 16 : open -> /usr/squid/cache/00/00/00000001
1998/05/29 18:16:16| Loaded Icons.
1998/05/29 18:16:16| Accepting HTTP connections on port 3128, FD 55.
1998/05/29 18:16:16| Accepting ICP messages on port 3130, FD 56.
1998/05/29 18:16:16| Pinger socket opened on FD 58
1998/05/29 18:16:18| NETDB state reloaded; 0 entries, 1783 msec
1998/05/29 18:16:18| Ready to serve requests.
1998/05/29 18:16:18| safeunlink: Couldn't delete /usr/squid/cache/swap.state.last-clean. (2) No such file or directory

It configures the parents, then:

1998/05/29 18:16:25| storeGetNextFile: /usr/squid/cache/00/02/000002AA: (24) Too many open files
.
.
.
1998/05/29 18:16:25| storeRebuildFromDirectory: DIR #0 done!
1998/05/29 18:16:25| /usr/squid/cache/swap.state: (24) Too many open files
FATAL: storeDirCloseTmpSwapLog: Failed to open swap log.
Squid Cache (Version 1.2.beta20): Terminated abnormally.
CPU Usage: 7.210 seconds
Maximum Resident Size: 0 KB
Page faults with physical i/o: 981
Memory usage for squid via mallinfo():
        total space in arena: 1552 KB
        Ordinary blocks: 1550 KB 23 blks
        Small blocks: 0 KB 0 blks
 KB 0 blks
        Free Small blocks: 0 KB
        Free Ordinary blocks: 2 KB
        Total in use: 1550 KB 100%
        Total free: 2 KB 0%

<repeat>

Cheers,

Mark

+-------------------------------------------------------------------------+
Mark Cooke The views expressed above are mine and are not
Systems Programmer necessarily representative of university policy
University Of Birmingham URL: http://www.sr.bham.ac.uk/~mpc/
+-------------------------------------------------------------------------+

-----BEGIN PGP SIGNATURE-----
Version: PGPfreeware 5.0i for non-commercial use
Charset: noconv

iQA/AwUBNW7vu3lbx4ZElNuLEQJm7ACcCbfE7a2J4QXKjn3hR5AirKB6LSUAnjZh
QuvQ2XqdmOAsVsiu4eH6hp07
=4wfM
-----END PGP SIGNATURE-----
Received on Fri May 29 1998 - 10:37:48 MDT