RE: [squid-users] Oom-killer and Squid

Colin,
Thanks for your reply. I checked into hugemem and it looks like about
16GB is hugemem for 64 bit systems.

After looking at pmap, it looks like the Squid heap is growing without
end. It's up to about 1.6GB resident at the moment and I suspect that
it crashes when the heap is larger than swap (2GB) and Squid decides to
swap or it just keeps growing until it consumes all the memory.

Any idea what would make the heap keep growing?
Thanks,
Dave

-----Original Message-----
From: Colin Campbell [mailto:sgcccdc@citec.qld.gov.au]
Sent: Tuesday, March 06, 2007 7:43 PM
To: Dave Rhodes
Cc: Henrik Nordstrom; squid-users@squid-cache.org
Subject: RE: [squid-users] Oom-killer and Squid

Hi,

I've been bitten by oom-killer in the past, not just on Squid boxes. The
problem comes from Linux' memory model which splits RAM into three
parts, the most imprtant of which are what's called "Low" and "High".
Essentially Low mem = 0..892 MBytes and High mem is the rest. If you run
"free -l" you can see how much of each is in use. You'll probably that
most of your Low mem is gone and little or none of your High mem is in
use.

Red Hat ship two kernel types, a "normal" one a "hugemem" one which is
for machines with > 4GB of RAM. On my 32 bit systems changing from the
normal to "hugemem" changed things. Here's two boxes both running squid:

HOST1 /root # uname -r
2.6.9-42.0.3.ELhugemem

HOST1 /root # free -l
             total used free shared buffers
cached
Mem: 4146972 4025500 121472 0 521828
2838672
Low: 3360540 3240092 120448
High: 786432 785408 1024
-/+ buffers/cache: 665000 3481972
Swap: 3967976 192 3967784

HOST2 /root # uname -r
2.6.9-42.0.8.ELsmp

HOST2 /root # free -l
             total used free shared buffers
cached
Mem: 4149240 4024556 124684 0 468904
2911876
Low: 872696 857516 15180
High: 3276544 3167040 109504
-/+ buffers/cache: 643776 3505464
Swap: 3967976 192 3967784

You might want to look at Suse to see if they do something similar
although you might find you need to rebuild your kernel.

Colin

On Tue, 2007-03-06 at 16:05 -0500, Dave Rhodes wrote:
> Thanks for the reply Henrik,
> The settings:
>
> cache_mem 1 GB
> cache_dir ufs /cache/normal 60000 9600 256
>
> I'm not sure about the cache_dir stuff, didn't know if it was better
> to have a lot of small dirs or a few large ones, I think I pulled this

> setting from someone setting up a cache about the same as mine in the
> archives.
>
> I think 60000 is 60000MB or 60GB?
> Dave
>
> -----Original Message-----
> From: Henrik Nordstrom [mailto:henrik@henriknordstrom.net]
> Sent: Tuesday, March 06, 2007 3:54 PM
> To: Dave Rhodes
> Cc: squid-users@squid-cache.org
> Subject: Re: [squid-users] Oom-killer and Squid
>
>
> tis 2007-03-06 klockan 14:47 -0500 skrev Dave Rhodes:
>
> > Squid 2.6 Stable5 on an HP DL390 w/6GB RAM, 60GB cache, 2GB swap
> > w/SuSE 10.1. As a rule, thanks to some help from Henrik, everything

> > runs well. Twice now though, I've had oom-killer jump in and kill
> > Squid and only Squid. I am running a very small test group of about

> > 30 users so it takes awhile (about 3 weeks) to run out of memory.
>
> You should not run out of memory unless you configured something very
> wrong...
>
> Whats your cache_mem and cache_dir settings?
>
> Regards
> Henrik
>

-- 
Colin Campbell
Unix Support/Postmaster/Hostmaster
Citec
+61 7 3227 6334