Re: Squid vs. NetCache

[fwded from Stephen Griffin]

From: Stephen Griffin <u-steve@ultra.net>
Subject: Re: Squid vs. NetCache
In-Reply-To: <199805281043.GAA11988@elektra.ultra.net> from Joe Provo - Network
Architect at "May 28, 98 06:43:09 am"
Date: Thu, 28 May 1998 09:13:20 -0400 (EDT)

In the referenced message, Joe Provo - Network Architect said:
>
[snip]
> We ran live, side-by-side tests of a netcache box against one of our squid
> servers (that was, incidentally, also handling other services) with both
> sharing the service behind a cisco localdirector. Log analysis via
> calamari (de facto cache log analysis standard, as far as I'm concerned)
> showed the netcache losing. I'll ask my colleagues who ran the test to
> comment further.
>
> Joe Provo

Here's some sample data in our squid vs. netcache bake-off.

Machine ligarius:
Squid running on Durango II (Alpha 164LX) 533MHz, 512MB ram, 4GB cache disk
also running other services such as ftp,nntp, etc...

Machine xpress24:
NetApp NetCache 630 533MHz Alpha, 512MB ram, 109GB cache disk providing
no other services

I'll leave the analysis to the reader, but pay special attention to the kB/sec
rates. Keep in mind that that NetCache is supposed to be VERY fast when
providing objects locally. Another interesting statistic is the distribution
of requests between the two boxes. Keep in mind that the Cisco LocalDirector
was set up to hand connections off to the box with the least number of
connections. After shutting off client-side persistent http connections,
it moved closer to 33%-66%, but still not anywhere near 50-50.

Preliminary heat:
xpress24 (netcache)
status request % kByte % sec kB/sec
--------------------------------- -------- ------ --------- ------ ---- -------
HIT 4423 8.98 16774 2.73 2 2.45
 TCP_HIT 2615 5.31 14705 2.40 2 2.80
 TCP_HIT_IFMS_NOTMOD 993 2.02 194 0.03 1 0.36
 TCP_HIT_VERIFY 815 1.65 1875 0.31 1 1.74

ligarius
status request % kByte % sec kB/sec
--------------------------------- -------- ------ --------- ------ ---- -------
HIT 50377 26.32 206614 15.15 1 5.22
 TCP_HIT 26985 14.10 155946 11.44 1 8.26
 TCP_IMS_HIT 13892 7.26 6755 0.50 0 1.39
 TCP_REFRESH_HIT 9453 4.94 43351 3.18 2 2.92
 TCP_REF_FAIL_HIT 47 0.02 562 0.04 21 0.56

First full day neck and neck:
xpress24
status request % kByte % sec kB/sec
--------------------------------- -------- ------ --------- ------ ---- -------
HIT 8746 11.26 31070 5.85 2 1.89
 TCP_HIT_VERIFY 3559 4.58 9485 1.78 2 1.33
 TCP_HIT 3556 4.58 21249 4.00 2 2.54
 TCP_HIT_IFMS_NOTMOD 1626 2.09 318 0.06 1 0.39
 TCP_HIT_VERIFY_FAIL 5 0.01 18 0.00 21 0.17

--------------------------------- -------- ------ --------- ------ ---- -------
Sum 77683 531510 4 1.58

ligarius
status request % kByte % sec kB/sec
--------------------------------- -------- ------ --------- ------ ---- -------
HIT 65916 28.40 315994 19.69 1 3.55
 TCP_HIT 37427 16.13 255239 15.90 1 4.72
 TCP_IMS_HIT 17846 7.69 6533 0.41 0 0.82
 TCP_REFRESH_HIT 10505 4.53 52732 3.29 1 3.40
 TCP_REF_FAIL_HIT 138 0.06 1490 0.09 84 0.13
--------------------------------- -------- ------ --------- ------ ---- -------
Sum 232097 1605191 3 2.01

Successive days results were comparable to the above, and I didn't log them.

Warning, the rest involves my opinions, and thus read at your own risk.

I also find the NetApp metric to be a bit interesting, they seem to base
everything on URL's per second. That appears to be more of a metric to show
the "breaking-point" of a box (ie when it can no-longer service new requests)
and not an indication on the latency of individual or aggregate requests.
Since I can get multiple machines such as I describe above for the cost
of a single NetCache, I can accept a box with a lower "breaking-point" if
it performs better in the things that matter to our consumers.

Also of note is that they use only one dns server, from my conversation I
believe it still blocks on dns lookups, and hence multiple requests with
uncached destinations will block serially instead of being handled in
parallel.

As in all things, YMMV,
Stephen

-- 
Stephen A. Griffin               UltraNet Communications, Inc., an RCN Company
Development Group                       Network Operations Center
u-steve@ultra.net                              noc@ultra.net