[squid-users] low squid throughput/scalability

Hi,

I'm using squid as an accelerator/reverse proxy server lots and lots of
small files (~20K each)

I used siege ( http://www.joedog.org/siege/ )to benchmark squid and got
some pretty disappointing results: only 2.5mb/sec throughput.

Here are some results:

/apps/siege/bin/siege -f urls.txt -c 100 -t 5M (100 concurrent
connections for 5 minutes)
Lifting the server
siege... done.
                                                    Transactions:
54751 hits
Availability: 100.00 %
Elapsed time: 300.27 secs
Data transferred: 726.25 MB
Response time: 0.01 secs
Transaction rate: 182.34 trans/sec
Throughput: 2.42 MB/sec
Concurrency: 2.05
Successful transactions: 54751
Failed transactions: 0
Longest transaction: 3.01
Shortest transaction: 0.00

/apps/siege/bin/siege -f urls.txt -c 150 -t 5M (150 concurrent
connections for 5 minutes)
ransactions: 55924 hits
Availability: 100.00 %
Elapsed time: 300.12 secs
Data transferred: 754.62 MB
Response time: 0.03 secs
Transaction rate: 186.34 trans/sec
Throughput: 2.51 MB/sec
Concurrency: 5.89
Successful transactions: 55924
Failed transactions: 0
Longest transaction: 0.15
Shortest transaction: 0.00

/apps/siege/bin/siege -f urls.txt -c 200 -b -t 5
ransactions: 54260 hits
Availability: 100.00 %
Elapsed time: 300.13 secs
Data transferred: 741.69 MB
Response time: 0.06 secs
Transaction rate: 180.79 trans/sec
Throughput: 2.47 MB/sec
Concurrency: 11.62
Successful transactions: 54307
Failed transactions: 0
Longest transaction: 0.37
Shortest transaction: 0.00

/apps/siege/bin/siege -f urls.txt -c 300 -b -t 5M
Transactions: 56664 hits
Availability: 100.00 %
Elapsed time: 300.43 secs
Data transferred: 814.88 MB
Response time: 0.07 secs
Transaction rate: 188.61 trans/sec
Throughput: 2.71 MB/sec
Concurrency: 13.46
Successful transactions: 56811
Failed transactions: 0
Longest transaction: 3.05
Shortest transaction: 0.00

The squid server and test machine is connected via 100mbps link. During
the test, the server cpu load is near zero. IO Wait is less than 10%. All
the requests were TCP_OFFLINE_HITs, according to top, swap is not
used. From teh benchmarks, it seems max throuput is 2.5mb/sec, kinda low
for such a power server.

Any comments/ideas?

mike

Here's my machine config:
dual processor, Xeon 3GHZ, HT enabled
4GB RAM
Linux FC4 on standalone drive
3ware 9500 SATA RAID, 128MB cache

RAID config:
12 SATA disks, 255GB each,
6 units, each has 2 disk, 64kb stripe RAID 0

Filesystem: each unit to 2 partitions
/dev/sda1 on /mnt/cache1a type reiserfs (rw,noatime,notail)
/dev/sda2 on /mnt/cache1b type reiserfs (rw,noatime,notail)
/dev/sdb1 on /mnt/cache2a type reiserfs (rw,noatime,notail)
/dev/sdb2 on /mnt/cache2b type reiserfs (rw,noatime,notail)
/dev/sdc1 on /mnt/cache3a type reiserfs (rw,noatime,notail)
/dev/sdc2 on /mnt/cache3b type reiserfs (rw,noatime,notail)
/dev/sdd1 on /mnt/cache4a type reiserfs (rw,noatime,notail)
/dev/sdd2 on /mnt/cache4b type reiserfs (rw,noatime,notail)
/dev/sde1 on /mnt/cache5a type reiserfs (rw,noatime,notail)
/dev/sde2 on /mnt/cache5b type reiserfs (rw,noatime,notail)
/dev/sdf1 on /mnt/cache6a type reiserfs (rw,noatime,notail)
/dev/sdf2 on /mnt/cache6b type reiserfs (rw,noatime,notail)

Squid configure flags:
--prefix=/apps/squid --enable-x-accelerator-vary --enable-linux-netfilter
--enable-cache-digests --enable-htcp --enable-snmp --enable-referer-log
--enable-useragent-log --enable-delay-pools --enable-icmp
--enable-async-io=100 --enable-carp --with-large-files --with-maxfd=10240
--enable-removal-policies=lru,heap

Squid Config:
cache_effective_user squid
cache_effective_user squid
visible_hostname f.d.q.n

http_port 80

httpd_accel_host a.b.c.e
#httpd_accel_host virtual
#httpd_accel_port 80

httpd_accel_with_proxy on
httpd_accel_uses_host_header on
redirect_rewrites_host_header off

#override the cache controls for urls that contain gexservelets
#objects w/o explict expirations date consider fresh for 2 weeks, w/ a max
of 1 month
refresh_pattern -i cachedpattern 20160 50 43200 override-expire ignore-reload

acl textfiles rep_mime_type -i ^text/plain$
acl htmlfiles rep_mime_type -i ^text/html$
acl jpegfiles rep_mime_type -i ^image/jpeg$
acl pngfiles rep_mime_type -i ^image/png$
acl giffiles rep_mime_type -i ^image/gif$

#don't cache text, html, gif, png
no_cache deny textfiles
no_cache deny htmlfiles
no_cache deny giffiles
no_cache deny pngfiles

emulate_httpd_log on
referer_log /apps/squid/var/logs/referer.log
pid_filename /var/run/squid.pid

#(((cache-size-KB/avg.-KB)/256)/256)*2= L1 dir. (L2 will always 256)
cache_dir aufs /mnt/cache1a/squid-cache 204800 320 256
cache_dir aufs /mnt/cache2a/squid-cache 204800 320 256
cache_dir aufs /mnt/cache3a/squid-cache 204800 320 256
cache_dir aufs /mnt/cache4a/squid-cache 204800 320 256
cache_dir aufs /mnt/cache5a/squid-cache 204800 320 256
cache_dir aufs /mnt/cache6a/squid-cache 204800 320 256
cache_dir aufs /mnt/cache1b/squid-cache 204800 320 256
cache_dir aufs /mnt/cache2b/squid-cache 204800 320 256
cache_dir aufs /mnt/cache3b/squid-cache 204800 320 256
cache_dir aufs /mnt/cache4b/squid-cache 204800 320 256
cache_dir aufs /mnt/cache5b/squid-cache 204800 320 256
cache_dir aufs /mnt/cache6b/squid-cache 204800 320 256

cache_mem 3072 MB

minimum_object_size 1 KB
maximum_object_size_in_memory 1 MB
store_avg_object_size 20 KB

#do not try to validate the objects
offline_mode on

#allow anyone to connect to our squid cache, but only connecting to the
accelerated hosts.
acl acceleratedHosts dst e.f.g.h
http_access allow acceleratedHosts

#allow the cache pool to connect to each other
acl mynet src a.b.c.d/24
http_access allow mynet
http_access allow localhost

#allow localhost to purge objects
acl PURGE method PURGE
http_access allow PURGE localhost
http_access deny PURGE

#deny everything else
http_access deny all

#allow snmp access
acl snmppublic snmp_community public
snmp_access allow snmppublic all
nonhierarchical_direct off
icp_query_timeout 7000
maximum_icp_query_timeout 10000
strip_query_terms off
log_icp_queries off

#-------------------------------------------------------------------
unique_hostname cache1
Received on Fri Mar 10 2006 - 17:22:41 MST