Re: [squid-users] Object becomes STALE: refresh_pattern min and max

Sorry it took a while to get back. Not sure how to interpre X-Cache and X-Cache-Lookup.

Here's the header info from Fiddler:

Request Header

GET /server1/websites/data/folder/myvideofile.vid HTTP/1.1

Client
    Accept: */*
Transport
    Host: ftp.mydomain.com
    Proxy-Connection: Keep-Alive

Response Header

HTTP/1.0 200 OK
Content-Length: 1775372
Content-Type: video/jpeg
Last-Modified: Tue, 02 Sep 2008 23:57:25 GMT
Accept-Ranges: none
ETag: "8020b2a557dc91:3ecc"
Server: Microsoft-IIS/6.0
X-Powered-By: ASP.NET
Date: Thu, 16 Oct 2008 22:52:49 GMT
X-Cache: MISS from squid.mydomain.com
X-Cache-Lookup: HIT from squid.mydomain.com:3128
Via: 1.0 squid.mydomain.com:3128 (squid/2.6.STABLE14)
Proxy-Connection: keep-alive

----- Original Message ----
From: Amos Jeffries <squid3_at_treenet.co.nz>
To: BUI18 <lbui18_at_yahoo.com>
Cc: squid-users_at_squid-cache.org; Itzcak Pechtalt <itzcak_at_gmail.com>
Sent: Wednesday, September 24, 2008 6:17:22 AM
Subject: Re: [squid-users] Object becomes STALE: refresh_pattern min and max

BUI18 wrote:
> Hi - Thanks for responding. URL for video file never changes.
>

What release of Squid?

Did you check the Expires header properly from the transfer rather than
from the (apparently untrustworthy) info in the store log?

> I did some more checking in the Squid logs and this is what I noticed:
>
> File Properties of video file (Pacific Daylight Time (PDT))
>
> Created On: Monday, September 22, 2008, 8:59:35 AM
>
> Modified On: Monday, September 22, 2008, 8:59:35 AM
>
> Accessed On: Today, September 24, 2008, 3:53:12 AM
>
> *******************************************************************
> Wget Grabs File (Time in India Standard Time (IST))
>
> --04:38:35-- http://ftp.mydomain.com/websites/data/myvideofile.vid
> => `/WGET/Temp/myvideofile.vid'
> 04:38:54 (93.91 KB/s) - `/WGET/Temp/myvideofile.vid' saved [1791244/1791244]
>
> The access.log confirms initial pre-fetch by wget.
>
> 1222124934.241 18968 192.168.200.4 TCP_MISS/200 1791684 GET http://ftp.mydomain.com/websites/data/myvideofile.vid - DIRECT/69.43.136.41 video/jpeg
>
> UTC = Mon, 22 Sep 2008 23:08:54 GMT
>
> The store.log shows a write from memory to disk:
>
> 1222124934.241 SWAPOUT 00 00057B65 1E18E35BDC9307C6BC3FBEFD5B4120A3 200 1222124765 1222099175 -1 video/jpeg 1791244/1791244 GET http://ftp.mydomain.com/websites/data/myvideofile.vid
>
> UTC = Mon, 22 Sep 2008 23:08:54 GMT
>
> *******************************************************************
>
> Then Store.log shows release or removal from cache:
>
> 1222253725.068 RELEASE 00 00057B65 605FAC36E93B0CDE81902BBC6C5EC71A 200 1222124765 1222099175 -1 video/jpeg 1791244/-279 GET http://ftp.mydomain.com/websites/data/myvideofile.vid
>
> UTC = Wed, 24 Sep 2008 10:55:25 GMT
>
> Notice the -1 for expiration header (I do not set one on the object). My min age is 5 days so I'm not sure why the object would be released from cache in less than 2 days.
>
> If the object was released from cache, when the user tried to access file, Squid reports TCP_REFRESH_MISS, which to me means that it was found in cache but when it sends a If-Modified-Since request, it thinks that the file has been modified (which it was not as seen by the lastmod date indicated in the store.log below.
>
> *******************************************************************
>
> User accessed file (access.log):
>
> 1222253742.005 17275 192.168.200.52 TCP_REFRESH_MISS/200 1791688 GET http://ftp.mydomain.com/websites/data/myvideofile.vid - DIRECT/69.43.136.41 video/jpeg
>
> UTC = Wed, 24 Sep 2008 10:55:42 GMT
>
> Then store.log shows a write to disk
>
> 1222253742.005 SWAPOUT 00 00088336 1E18E35BDC9307C6BC3FBEFD5B4120A3 200 1222253575 1222099175 -1 video/jpeg 1791244/1791244 GET http://ftp.mydomain.com/websites/data/myvideofile.vid
>
> UTC = Wed, 24 Sep 2008 10:55:42 GMT
> datehdr: Wed, 24 Sep 2008 10:55:55 GMT
> lastmod: Mon, 22 Sep 2008 15:59:35 GMT
>
> Anyone with ideas on why this behavior occurs?
>
> thanks
>
>
>
>
>
> ----- Original Message ----
> From: Itzcak Pechtalt <itzcak_at_gmail.com>
> To: Squid Users <squid-users_at_squid-cache.org>
> Sent: Wednesday, September 24, 2008 4:35:59 AM
> Subject: Re: [squid-users] Object becomes STALE: refresh_pattern min and max
>
> On Wed, Sep 24, 2008 at 1:39 PM, BUI18 <lbui18_at_yahoo.com> wrote:
>> Hi -
>>
>> I have squid box with tons of disk for the cache_dir
>> (hundreds of GB). I use wget to perform some pre-fetching of large
>> video files. I've set the min and max age to 5 days and 7 days (in
>> minutes). And although I have plenty of disk space available, I still
>> receive TCP_REFRESH_MISS for files that had been pre-fetched and later
>> accessed the same day. Does anyone know why Squid would consider it as
>> STALE? I thought that by setting the min value for refresh_pattern for
>> the video file would guarantee freshness. Not only does the cache
>> consider it STALE, it then goes and pre-fetches a new copy even though
>> I know that the video file has not changed. Any help would be greatly
>> appreciated. Thanks.
>>
>>
>>
>>
>
> Hi,
> Check if the video URL changes from request to request. In YouTube
> video even if the main URL is the same, there is request ID in URL who
> changes per request.
>
> Itzcak
>
>
>
>

-- 
Please use Squid 2.7.STABLE4 or 3.0.STABLE9
__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com