Re: Squid development

On 12 Jun 2001, at 22:59, Henrik Nordstrom <hno@hem.passagen.se> wrote:

> Michael Y. Popov wrote:
>
> > 1. Squid uses MD5 algorithm quite frequently, so MD5 main (which took
> > most of CPU power) function may be rewritten in assembly for better
> > performance. I does it already, using Netwide Assembler (freeware).

 Why do you feel MD5 performance is a problem? My tests hint that on
 average MD5 calcs take about 4-5K cpu ticks, keeping total MD5 overhead
 well under 0.1%. During dirty startup, when lots of md5 is done, I saw
 16K md5 calcs/sec and cpu used was 3.9%. Thats imo very small overhead.
 Would it matter if we achieved 0.05% cpu instead of 0.1% with asm?

 Or did I miss something?

> > So code becomes little smaller and about 45% faster than code,
> > produces by C compiler (Microsoft Visual C++ 6.0, since I use SquidNT
> > ported by Romeo Anghelache). Only one disadvantage exists: this code
> > is only for 32-bit x86 architecture only. The code is highly optimized
> > for P6 architecture.
> > If you find this work useful for Squid, I'll send you code (asm file)
> > with 'MD5Transform' function.
>
> Squid-2.5 can be linked with OpenSSL which includes MD5 routines which
> is substantially faster than the code shipped with Squid (45% faster).

 Is 45% substantially faster? If squid was md5 bound mostly, that probably
 could matter, but as md5 only takes very small percentage of total req
 processing time, this performance increase is hardly noticable.

> Shipping assembly language versions with Squid is a bit messy,
> especially considering that there are other free packages available that
> Squid may use...

 If we used md5 on whole objects, then md5 performance could be a problem,
 but I'd agree that until then we are pretty cool.

> > 2. Second idea is about Squid's IP and FQDN caches. All Squid versions
> > (known to me) use this caches only during uptime, so Squid restarting
> > cause lost of both caches. I think it will be useful to write these
> > caches to files and reloads its on 'init' phase of Squid loading.
> > Modifications if quite little and all basic code will be unmodified.
> > Unfortunately, I implemented this idea only for Win32 platform for
> > now. But I think re-porting to standard file I/O functions will not be
> > difficult.
>
> Personally I like that the IP and FQDN caches in Squid are ditched on a
> restart, but I recognise that other may have other opinions on this.
>
> Fellow Squid developers: What is your opinions?

 IMO it only makes sense with short restarts. When squid crashes?
 It can be handled outside squid as well. Only if those caches did provide
 some long-term statistics, I would like that to persist. But as they don't
 keep anything that we could consider costly to loose, I wouldn't care.
 Startup Squid performance increase would be very small, if you think
 about it. On other hand, if some DNS zone changes and you want to reflect
 that to squid, you'd need to bother with deleting that cache file.

------------------------------------
 Andres Kroonmaa <andre@online.ee>
 CTO, Microlink Online
 Tel: 6501 731, Fax: 6501 708
 Pärnu mnt. 158, Tallinn,
 11317 Estonia
Received on Wed Jun 13 2001 - 04:15:08 MDT