Re: [squid-users] limit squid memory ram use - squid becomes slow when ram full

From: <rpereyra_at_lavabit.com>
Date: Tue, 26 Apr 2011 07:06:41 -0400 (EDT)

> On Mon, 11 Apr 2011 07:52:41 -0400 (EDT), rpereyra_at_lavabit.com wrote:
>> Hi
>>
>> How I can limit the ram memory use in my squid/tproxy box ?
>>
>> I have a fast server with 16Gb ram. The average bandwidth is about
>> 60-70
>> Mb/s.
>>
>> The bridge works well but when the cache and memory becomes full its
>> goes
>> slow and becomes unusable.
>>
>> The cache is 10G size.
>>
>> I see that a few hours to be working and have used the 16 GB of RAM
>> starts to run slow.
>>
>> Any help ?. I have configured some memory optimization options but
>> looks
>> don't help for me.
>>
>> Thanks in advance
>>
>> roberto
>>
>> This is my config:
>>
>> -------------------------------------
>>
>> cache_mem 10 MB
>> memory_pools off
>> cache_swap_low 94
>> cache_swap_high 95
>>
> <snip>
>> # Example rule allowing access from your local networks.
>> # Adapt to list your (internal) IP networks from where browsing
>> # should be allowed
>> acl localnet src 10.0.0.0/8 # RFC1918 possible internal network
>> acl localnet src 172.16.0.0/12 # RFC1918 possible internal network
>> acl localnet src 192.168.0.0/16 # RFC1918 possible internal network
>> acl localnet src fc00::/7 # RFC 4193 local private network range
>> acl localnet src fe80::/10 # RFC 4291 link-local (directly plugged)
>> machines
>>
>> acl net-g1 src 200.117.xxx.xxx/24
>> acl net-g2 src 200.xxx.xxx.xxx/24
>> acl net-g3 src 190.xxx.xxx.xxx/24
>>
> <snip>
>>
>> http_access allow net-g1
>> http_access allow net-g2
>> http_access allow net-g3
>
> Being "allow" you can grab some extra speed by combining these all
> under one ACL name. "localnet" being the standard one for local network
> ranges.
>
> <snip>
>>
>> # Uncomment and adjust the following to add a disk cache directory.
>> cache_dir ufs /var/spool/squid 10000 64 256
>>
> <snip>
>
>
> Given those cache sizes, your Squid box should be using around 110 MB
> of RAM for index plus a little. Even assuming a worst-case of a minutes
> traffic accumulated in transit buffers comes nowhere close to filling 16
> GB up.
>
> Some questions that may help narrow down where the slow is coming from:
> What version of Squid is this?
> What are your avg object size?
> How many concurrent client connections?
> "slow" and "normal" response speeds?
> Do you notice any change in the Squid->Internet request types during
> the slowdown? (ie a move to extra MISS/HIT/IMS/REFRESH)
>
> And like Michael said the disk IO stats are important to look at. When
> the cache_dir gets to 94% full it will start spending CPU and disk
> cycles on erasing objects. If it reaches 95% a larger portion of cycles
> get used until it drops down below 94% again.
>
>
> Amos
>

Hi all !

These are two test samples when the proxy becomes slow.

Any clue ? Maybe an I/O disk issue ?

roberto

First sample:

[jorge_at_proxy ~]$ squidclient mgr:info
HTTP/1.0 200 OK
Server: squid/3.1.1
Mime-Version: 1.0
Date: Wed, 20 Apr 2011 17:38:35 GMT
Content-Type: text/plain
Expires: Wed, 20 Apr 2011 17:38:35 GMT
Last-Modified: Wed, 20 Apr 2011 17:38:35 GMT
X-Cache: MISS from proxy.net-g.com.ar
X-Cache-Lookup: MISS from proxy.net-g.com.ar:3128
Via: 1.0 proxy.net-g.com.ar (squid/3.1.1)
Proxy-Connection: close

Squid Object Cache: Version 3.1.1
Start Time: Wed, 20 Apr 2011 17:34:35 GMT
Current Time: Wed, 20 Apr 2011 17:38:35 GMT
Connection information for squid:
        Number of clients accessing cache: 171
        Number of HTTP requests received: 99367
        Number of ICP messages received: 0
        Number of ICP messages sent: 0
        Number of queued ICP replies: 0
        Number of HTCP messages received: 0
        Number of HTCP messages sent: 0
        Request failure ratio: 0.00
        Average HTTP requests per minute since start: 24781.1
        Average ICP messages per minute since start: 0.0
        Select loop called: 588320 times, 0.409 ms avg
Cache information for squid:
        Hits as % of all requests: 5min: 27.8%, 60min: 27.8%
        Hits as % of bytes sent: 5min: 16.9%, 60min: 16.9%
        Memory hits as % of hit requests: 5min: 6.3%, 60min: 6.3%
        Disk hits as % of hit requests: 5min: 45.5%, 60min: 45.5%
        Storage Swap size: 6029640 KB
        Storage Swap capacity: 98.1% used, 1.9% free
        Storage Mem size: 319540 KB
        Storage Mem capacity: 15.4% used, 84.6% free
        Mean Object Size: 14.94 KB
        Requests given to unlinkd: 0
Median Service Times (seconds) 5 min 60 min:
        HTTP Requests (All): 0.30459 0.30459
        Cache Misses: 0.42149 0.42149
        Cache Hits: 0.00179 0.00179
        Near Hits: 0.18699 0.18699
        Not-Modified Replies: 0.00091 0.00091
        DNS Lookups: 0.03374 0.03374
        ICP Queries: 0.00000 0.00000
Resource usage for squid:
        UP Time: 240.588 seconds
        CPU Time: 165.222 seconds
        CPU Usage: 68.67%
        CPU Usage, 5 minute avg: 67.92%
        CPU Usage, 60 minute avg: 67.92%
        Process Data Segment Size via sbrk(): 259836 KB
        Maximum Resident Size: 0 KB
        Page faults with physical i/o: 0
Memory usage for squid via mallinfo():
        Total space in arena: 260112 KB
        Ordinary blocks: 259914 KB 946 blks
        Small blocks: 0 KB 0 blks
        Holding blocks: 313188 KB 2048 blks
        Free Small blocks: 0 KB
        Free Ordinary blocks: 197 KB
        Total in use: 573102 KB 100%
        Total free: 197 KB 0%
        Total size: 573300 KB
Memory accounted for:
        Total accounted: 517250 KB 90%
        memPool accounted: 517250 KB 90%
        memPool unaccounted: 56049 KB 10%
        memPoolAlloc calls: 42349480
        memPoolFree calls: 40063448
File descriptor usage for squid:
        Maximum number of file descriptors: 16384
        Largest file desc currently in use: 13418
        Number of file desc currently in use: 13197
        Files queued for open: 6
        Available number of file descriptors: 3181
        Reserved number of file descriptors: 100
        Store Disk files open: 171
Internal Data Structures:
        404884 StoreEntries
         24853 StoreEntries with MemObjects
         23380 Hot Object Cache Items
        403505 on-disk objects

--------------------------

Second sample:

[jorge_at_proxy ~]$ squidclient mgr:info
HTTP/1.0 200 OK
Server: squid/3.1.1
Mime-Version: 1.0
Date: Thu, 21 Apr 2011 17:53:35 GMT
Content-Type: text/plain
Expires: Thu, 21 Apr 2011 17:53:35 GMT
Last-Modified: Thu, 21 Apr 2011 17:53:35 GMT
X-Cache: MISS from proxy.net-g.com.ar
X-Cache-Lookup: MISS from proxy.net-g.com.ar:3128
Via: 1.0 proxy.net-g.com.ar (squid/3.1.1)
Proxy-Connection: close

Squid Object Cache: Version 3.1.1
Start Time: Thu, 21 Apr 2011 17:44:32 GMT
Current Time: Thu, 21 Apr 2011 17:53:35 GMT
Connection information for squid:
        Number of clients accessing cache: 201
        Number of HTTP requests received: 237647
        Number of ICP messages received: 0
        Number of ICP messages sent: 0
        Number of queued ICP replies: 0
        Number of HTCP messages received: 0
        Number of HTCP messages sent: 0
        Request failure ratio: 0.00
        Average HTTP requests per minute since start: 26240.8
        Average ICP messages per minute since start: 0.0
        Select loop called: 1117825 times, 0.486 ms avg
Cache information for squid:
        Hits as % of all requests: 5min: 26.5%, 60min: 26.0%
        Hits as % of bytes sent: 5min: 13.4%, 60min: 13.8%
        Memory hits as % of hit requests: 5min: 18.9%, 60min: 15.6%
        Disk hits as % of hit requests: 5min: 33.5%, 60min: 36.9%
        Storage Swap size: 6028184 KB
        Storage Swap capacity: 98.1% used, 1.9% free
        Storage Mem size: 784264 KB
        Storage Mem capacity: 37.8% used, 62.2% free
        Mean Object Size: 15.61 KB
        Requests given to unlinkd: 0
Median Service Times (seconds) 5 min 60 min:
        HTTP Requests (All): 0.33943 0.33943
        Cache Misses: 0.46965 0.44492
        Cache Hits: 0.00286 0.00286
        Near Hits: 0.19742 0.20843
        Not-Modified Replies: 0.00179 0.00179
        DNS Lookups: 0.01331 0.03079
        ICP Queries: 0.00000 0.00000
Resource usage for squid:
        UP Time: 543.383 seconds
        CPU Time: 348.531 seconds
        CPU Usage: 64.14%
        CPU Usage, 5 minute avg: 65.91%
        CPU Usage, 60 minute avg: 64.14%
        Process Data Segment Size via sbrk(): 838644 KB
        Maximum Resident Size: 0 KB
        Page faults with physical i/o: 0
Memory usage for squid via mallinfo():
        Total space in arena: 838920 KB
        Ordinary blocks: 838698 KB 1764 blks
        Small blocks: 0 KB 0 blks
        Holding blocks: 311892 KB 2048 blks
        Free Small blocks: 0 KB
        Free Ordinary blocks: 221 KB
        Total in use: 1150590 KB 100%
        Total free: 221 KB 0%
        Total size: 1150812 KB
Memory accounted for:
        Total accounted: 1059722 KB 92%
        memPool accounted: 1059722 KB 92%
        memPool unaccounted: 91089 KB 8%
        memPoolAlloc calls: 76008750
        memPoolFree calls: 72446323
File descriptor usage for squid:
        Maximum number of file descriptors: 16384
        Largest file desc currently in use: 13386
        Number of file desc currently in use: 13078
        Files queued for open: 0
        Available number of file descriptors: 3306
        Reserved number of file descriptors: 100
        Store Disk files open: 143
Internal Data Structures:
        387582 StoreEntries
         58603 StoreEntries with MemObjects
         57148 Hot Object Cache Items
        386166 on-disk objects
[jorge_at_proxy ~]$
Received on Tue Apr 26 2011 - 11:06:42 MDT

This archive was generated by hypermail 2.2.0 : Tue Apr 26 2011 - 12:00:03 MDT