Re: Problem with squid interpretting gzipped content from Apache

Hi Amos,

I realise this could be a development related post. I will repost it there.
Sorry for the inconvenience.

I am raising this request internally in Squid code to fetch some urls that
is present in the already replied object.
Say I am using a client to request fetchcompress.html (without any encoding
set). The Squid fetches this fetchcompress.html from the origin
server(apache) and returns it to the client.
At the same time, it parses the fetchcompress.html to see if it has any
prefetchable urls.

In my case, fetchcompress.html has a prefetchable link compress.html.

To fetch this, I setup a fake request header with "Accept-Encoding: gzip" in
it. This is done internally by the squid code itself. I believe this is
successfully done as I can see it in the tcpdump (refer to "Request sent to
Apache (tcpdump)" section in my prev post).
When I retrieve this object using a StoreClientCopy(), it gives me an empty
object ie. Object length was 0.

a) Now did this happen because I simply retrieved the object based on the
url alone?
b) Why is the Content-Encoding tag absent from the reply header?

The code works for normal requests though without the "Accept Encoding"
set. I am able to retrieve objects from the cache.

Regards,
Anita

From: Amos Jeffries [mailto:squid3_at_treenet.co.nz]
Sent: 17 April 2013 14:41
To: squid-users_at_squid-cache.org
Subject: Re: [squid-users] Squid behaviour with gzip responses from origin
server

On 17/04/2013 8:19 p.m., anita wrote:
> Hi All,
>
> I am using Squid 3.1.16.
> I had modified the request header in Squid to include: "Accept-Encoding:
> gzip".
>
> The request is also sent to the Apache server. The server seems to
> even compress the object and return with proper headers.
>
> But the squid seems to drop the "content-encoding" header. It also
> seems to drop the content-length header.
>
> I am not able to understand why Squid replaces the Content-Encoding
> header with something else and why it was marking as internal?
>
> These are my tcpdumps for the request and response from the Apache:
>
> Request sent to Apache (tcpdump)
> ----------------------------------
> GET./compres
> s.html.HTTP/1.1.
> .User-Agent:.Wge
> t/1.12.(linux-gn
> u)..Accept:.*/*.
> .Host:.10.145.66
> .205..Referer:.h
> ttp://10.145.66.
> 205/fetchcompres
> s.html..Accept-E
> ncoding:.gzip..V
> ia:.1.1.prefetch
> .net.(squid/3.1.
> 16)..X-Forwarded
> -For:.unknown..C
> ache-Control:.ma
> x-age=259200..Co
> nnection:.keep-a
> live....
>
> Reply header as sent by Apache (tcpdump) HTTP/1.1.200
> .OK..Date:.Wed,.
> 17.Apr.2013.02:1
> 3:13.GMT..Server
> :.Apache/2.2.15.
> (Red.Hat)..Last-
> Modified:.Wed,.1
> 7.Apr.2013.01:21
> :00.GMT..ETag:."
> 1208df-282-4da84
> 4c80d89e"..Accep
> t-Ranges:.bytes.
> .Vary:.Accept-En
> coding..Content-
> Encoding:.gzip..
> Content-Length:.
> 125..Connection:
> .close..Content-
> Type:.text/html;
> .charset=UTF-8..
>
> Reply header (as given by Squid) after it receives data from Apache
> -----------------------------------------------------------------
> HTTP/1.0 200 Internal marker object
> Server: squid/3.1.16
> Mime-Version: 1.0
> Date: Wed, 17 Apr 2013 07:16:30 GMT
> Content-Type: x-squid-internal/vary
> Expires: Thu, 18 Apr 2013 11:03:10 GMT
> Vary: Accept-Encoding
>

This is an internal object used by Squid to manage variant caching. It is
telling Squid that future requests for this URL needs to be combined with
the clients Accept-Encoding header to determine what object to return. With
normal operation the Accept-Encoding sent to the server would be created by
the client and the server response would be stored at the location the
server request URL and Accept-Encoding headers.

I suspect that what is happening is that the client did not send any
Accept-Encoding header. Which makes the client URL plus client
Accept-Encoding header point back at this marker object instead of the
intended serve response. Please confirm that by looking at the client
request received by Squid.

Amos

--
View this message in context: http://squid-web-proxy-cache.1019090.n4.nabble.com/Problem-with-squid-interpretting-gzipped-content-from-Apache-tp4659542p4659545.html
Sent from the Squid - Development mailing list archive at Nabble.com.