Re: [squid-users] Windows Updates, YouTube and WoW

On 8 November 2010 22:43, Amos Jeffries <squid3_at_treenet.co.nz> wrote:
> On Mon, 8 Nov 2010 18:32:52 -0500, Kevin Wilcox <kevin.wilcox_at_gmail.com>
> wrote:

<snip>

>> I understand that it could be related to the partial content reply for
>> the request and I understand that it could also be related to the
>> <URL>/<foo>? style request. Is the best approach to just automatically
>> pass anything for blizzard.com/worldofwarcraft.com straight through
>> and not attempt to cache the updates? I've seen some comments where
>> using
>>
>> acl QUERY urlpath_regex cgi-bin \?
>> cache deny QUERY
>>
>> will cause those requests to not be cached (and I understand why that
>> is) but I'm wondering if I should just ignore them altogether,
>> especially given the third item - YouTube.
>
> Yes, don't use that QUERY stuff. The dynamic URL which are cacheable will
> have expiry and control headers to make it happen. The others are caught
> and discarded properly by the new default refresh_pattern for cgi-bin and
> \?.

Excellent! I'm still seeing a *huge* performance difference between having

acl warcraft dstdomain blizzard.vo.llnwd.net
acl warcraft dstdomain attdist.blizzard.com
cache deny warcraft

in my config. As in, orders of magnitude (literally, and much more
reliable than forgetting to change my comments when I limited Squid to
2GB ;)) different. If I'm going to ignore them altogether, and the P2P
downloaded portion won't be cached anyway, that seems to me to be the
cleanest way to do it.

> Caching youtube still currently requires the storeurl feature of 2.7 which
> has not bee ported to 3.x.
> There are embeded visitor details and timestamps of when the video was
> requested in the YT URL which cause the cache to fill up with large videos
> at URL which will never be re-requested. This actively prevents any totally
> unrelated web objects from using the cache space.
>
> It is a good idea to prevent the YT videos from being stored at all unless
> you can de-duplicate them.

I didn't realise that about the URL. I was thinking the request came
for the URL shown in the address bar -
"http://youtube.com/watch?v=<static_string>", an address that can be
passed around and shared (how most of our folks share video). Good to
know that there is a unique request that's actually being sent for the
content.

Regarding storing - if put into production, the disk cache would be
sufficiently large enough to address storing a hundred gigs of videos
and regularly clearing the disk cache isn't a problem. I don't care if
we manually clear before the page actually expires, I'm just looking
to make *some* impact in the amount of bandwidth used by the target
audience.

>> # Cache Mem - ideal amount of RAM to use
>> cache_mem 2048 MB
>>
>> # Maximum object size - default is 4MB, not nearly enough to be useful
>> maximum_object_size 1024 MB
>>
>> # Maximum object size in memory - we have 4GB, we can handle larger
> objects
>> maximum_object_size_in_memory 512 MB
>
> Um, no you have 2GB (cache_mem) in which to store these objects. All 4+ of
> them.

Good point, I should have updated my comments as I played around with
the cache_mem and max object values.

Thanks Amos!

kmw
Received on Tue Nov 09 2010 - 13:50:28 MST