Re: [squid-users] Disk performance basics

Sounds like we have similar needs - all of our users are auth'd to an NT domain (5 auth
processes), and a pretty decent list of ACLs, including ~300,000 denial entries. How many
users do you support? What kind of peak/avg hits/sec do you see? What kind of hit ratios do
you see? (we get ~16-20%) Google Earth and Google video are starting to take a decent
chunk of bandwidth, even with 70kb/s delay pools.
I was wondering about the LB using DNS... this is something we've considered doing. We've
done it short term several times when transitioning proxies, and it seemed to work fine, but
we never left it in place for too long.

I'm noticing our CPU usage is climbing today, possibly due to the large log file, which is now
over 2Gb in size. I must plan for a larger log volume...

On 29 Mar 2006 at 21:37, Rodrigo A B Freire wrote:

> Shawn,
>
> We are a heavy user of ACLs. Besides, our traffic is authenticated
> (and
> authorized). This is a eager CPU eater.
>
> Our Dell has 2 GB RAM, is a SMP machine (non-HT; two real
> processors and
> disabled HT) and the load is quite balanced. One CPU gets a higher
> affinity with Squid's internal things (since Squid isn't
> multi-threaded to take full benefit of a SMP system) and the other
> deals with the interrupts, I/O, authenticators, diskd, and other banal
> (but cpu-consuming) stuffs.
>
> We also rely on delay pools to disencourage some traffics
> (streaming,
> webmails, etc). Altough not strictly prohibited, but the user willing
> to use it, will have to face the line! :-)
>
> I have two machines doing the proxy service; a quad-xeon P III 700
> with
> 3 GB RAM and 180 GB HDD (JBOD, 16 disks) and a Dell 3.2 SMP, 70 GB
> HDD.
>
> As weird as it may sound, my quad P III do the job better than the
> dual
> P IV. Wait, don't blame the memory: Even when had 2 MB performed
> better than the P IV.
>
> When we implemented a "load balancing" via dns (the name proxy has
> two
> IPs, a very primmary LB) with these two machines, it brought us a very
> pleasant result, with quicker response to the users. One of them (the
> Dell) I've configured to force the cache of Windows Update and Google
> Earth... It's just terrific!
>
> Well, that's it!
>
> Best regards,
>
> Rodrigo.
>
> ----- Original Message -----
> From: "Shawn Wright" <swright@sls.bc.ca>
> To: <squid-users@squid-cache.org>
> Sent: Wednesday, March 29, 2006 9:15 PM
> Subject: Re: [squid-users] Disk performance basics
>
>
> > Thanks for the info. I've been using XFS for years, ever since SGI
> > started patching RH for it, and have been very happy with it - fast,
> > and solid as a rock. I'll have to look into epoll. I've got some
> > time, as this won't happen for a few months.
> >
> > What are the current thoughts as to SMP with squid? I could buy more
> > disks or RAM for the cost of the 2nd CPU. We went from a single CPU
> > NT squid box to the current dual CPU linux squid box, so I can't
> > really say whether the dual CPU makes much of a difference. We
> > started out with async-io=24, which proved too much for the
> > hardware, and have settled at 16 threads, which seems about right
> > for the old box, and has proven to be very solid. How about
> > RAM/cache space? Currently we have 768Mb RAM, 27Gb cache. I would
> > probably go with around 2Gb RAM & 160Gb cache on the new box.
> >
> > We rely heavily on delay pools to keep bandwidth under control,
> > giving our students 70kb/s max on their own PCs. Even at that, with
> > the hours they are allowed to use it, we have some managing over
> > 2Gb/day of downloads. With fast connectivity on the campus, they can
> > really hammer the proxy during peak times, as our recent log-filling
> > experience can attest to.
> >
> > On 29 Mar 2006 at 20:34, Rodrigo A B Freire wrote:
> >
> >> Shawn,
> >>
> >> Undo the RAID5 and leave them as single disks. Format the disks
> >> with
> >> XFS. Use epoll.
> >>
> >> Be happy! ;-)
> >>
> >> ----- Original Message -----
> >> From: "Shawn Wright" <swright@sls.bc.ca>
> >> To: <squid-users@squid-cache.org>
> >> Sent: Wednesday, March 29, 2006 7:29 PM
> >> Subject: [squid-users] Disk performance basics
> >>
> >>
> >> > Our aging squid proxy is due for replacement, and I am looking to
> >> > maximize performance and stability on the new box. The current
> >> > platform is this:
> >> >
> >> > Dual P3/733, Mandrake 9.2
> >> > 768Mb RAM
> >> > 4x 9Gb 10k rpm scsi drives on adaptec 3940, 1 for O/S, 3 for
> >> > cache.
> >> >
> >> > The above has served well for a few years, handling a consistent
> >> > load of ~500 users, and about 350-450Gb/month of data. (with
> >> > delay pools on most users). Long-term MRTG graphing of load shows
> >> > the system is not CPU bound, with a steady avg CPU load of about
> >> > 20%, avg http & dns times of 75ms, and typical peak requests/sec
> >> > of ~60 each day.
> >> >
> >> > Our standard server is a Dell 2850 with 4 or more 15K drives and
> >> > RAID5. Obviously RAID5 isn't needed for the cache, so I'm
> >> > wondering what is the best disk option for cache performance in a
> >> > machine like this, which offers up to 6 drives in a scsi
> >> > backplane.
> >> >
> >> > Any tips appreciated.
> >> >
> >> > Thanks.
> >> > -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
> >> > Shawn Wright, I.T. Manager
> >> > Shawnigan Lake School
> >> > http://www.sls.bc.ca
> >> > swright@sls.bc.ca
> >> >
> >> >
> >> >
> >>
> >
> >
> > -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
> > Shawn Wright, I.T. Manager
> > Shawnigan Lake School
> > http://www.sls.bc.ca
> > swright@sls.bc.ca
> >
> >
> >
>

--
Shawn Wright
I.T. Manager
Shawnigan Lake School