Re: Features

On Mon, 24 Nov 1997, Dancer wrote:

> ZLIB is the obvious thing to use. At it's simplest, a cache item could be compressed
> before storage, and decompressed for transmission.
>
> Downsides: If in block mode: Considerable extra memory requirements (double the
> object, approximately), temp space to store decompressed objects that are 'in
> transit' (either coming in, ready to be compressed, or going out), CPU load.
>
> Stream compression: This is it's own bag of worms. I've _done_ this....and in fact,
> I have a library that does block/stream encryption via zlib. It was somewhat hellish
> to implement, but it works. It was intended for use in HTTP, and ties to an
> encryption library, but the encryption could be cut out of it. It encodes compressed
> data to 6-bit printable characters (not standard base-64), but to a quickish
> encoding that I cooked up at the time (we actually didn't want to be base-64
> compatible when it was written).
>
> I might look into how this might be patched into Squid....it might be useful for
> inter-cache transfers.
>
> D

I would also suggest you examine LZO... It has VERY low memory
requirements in most of it's modes for both compression and
decompression.. Decompression is always very fast, there are several
compression modes for every decomprssor that have differnt time/comprssion
tradeoff levels..

I would suggest squid provided both zlib and lzo..

Zlib would be suitable for slower links, and faster computers, while I
expect lzo to have more real world benifits.

With lzo squid could even change it's compression mode depending on system
load.. With some of the faster LZO compression modes a typical p100 could
easily saturate 100Mb/s ethernet..

How many of us have caches with more then %20 cpu in use? Not many I bet,
unless the cache is doing other things....

Why not harvest that cpu?
Received on Mon Nov 24 1997 - 06:08:19 MST