[squid-users] Re: Obtaining the GET query string from logs?

As a followup.

I enabled the combined log format, I do get more information but I
think I still see incomplete GET query strings.

Here as an example... meebo and googles automatic search query suggestion.

67.199.62.74 - admin [26/Oct/2009:22:45:22 -0600] "GET
http://clients1.google.com/complete/search? HTTP/1.1" 200 509
"http://toolbarqueries.google.com/webhp" "Mozilla/5.0 (X11; U; Linux
i686; en-US; rv:1.9.0.14) Gecko/2009090216 Ubuntu/8.10 (intrepid)
Firefox/3.0.14" TCP_MISS:DIRECT
67.199.62.74 - admin [26/Oct/2009:22:45:23 -0600] "GET
http://clients1.google.com/complete/search? HTTP/1.1" 200 423
"http://toolbarqueries.google.com/webhp" "Mozilla/5.0 (X11; U; Linux
i686; en-US; rv:1.9.0.14) Gecko/2009090216 Ubuntu/8.10 (intrepid)
Firefox/3.0.14" TCP_MISS:DIRECT
67.199.62.74 - admin [26/Oct/2009:22:45:24 -0600] "GET
http://www.meebo.com/mcmd/events? HTTP/1.1" 200 467
"http://www.meebo.com/" "Mozilla/5.0 (X11; U; Linux i686; en-US;
rv:1.9.0.14) Gecko/2009090216 Ubuntu/8.10 (intrepid) Firefox/3.0.14"
TCP_MISS:DIRECT
67.199.62.74 - admin [26/Oct/2009:22:45:24 -0600] "GET
http://www.meebo.com/mcmd/events? HTTP/1.1" 200 461
"http://www.meebo.com/" "Mozilla/5.0 (X11; U; Linux i686; en-US;
rv:1.9.0.14) Gecko/2009090216 Ubuntu/8.10 (intrepid) Firefox/3.0.14"
TCP_MISS:DIRECT
67.199.62.74 - admin [26/Oct/2009:22:45:25 -0600] "GET
http://clients1.google.com/complete/search? HTTP/1.1" 200 424
"http://toolbarqueries.google.com/webhp" "Mozilla/5.0 (X11; U; Linux
i686; en-US; rv:1.9.0.14) Gecko/2009090216 Ubuntu/8.10 (intrepid)
Firefox/3.0.14" TCP_MISS:DIRECT
67.199.62.74 - admin [26/Oct/2009:22:45:27 -0600] "GET
http://clients1.google.com/complete/search? HTTP/1.1" 200 425
"http://toolbarqueries.google.com/webhp" "Mozilla/5.0 (X11; U; Linux
i686; en-US; rv:1.9.0.14) Gecko/2009090216 Ubuntu/8.10 (intrepid)
Firefox/3.0.14" TCP_MISS:DIRECT
67.199.62.74 - admin [26/Oct/2009:22:45:28 -0600] "GET
http://clients1.google.com/complete/search? HTTP/1.1" 200 426
"http://toolbarqueries.google.com/webhp" "Mozilla/5.0 (X11; U; Linux
i686; en-US; rv:1.9.0.14) Gecko/2009090216 Ubuntu/8.10 (intrepid)
Firefox/3.0.14" TCP_MISS:DIRECT

Why dont I see the full data after "google.com/complete/search?" ??

To reproduce this log one could go to : http://toolbarqueries.google.com/webhp

And start typing a search query, for every letter it will query google
so that it returns suggestions that one could use.. all of those
requests are not detailed!!

Same happens for other similar mechanisms.

Why?

Thanks

Andres

On Mon, Oct 26, 2009 at 10:09 PM, Andres Salazar <ndrsslzr80_at_gmail.com> wrote:
> Hello,
>
> I have entries like these in my log:
>
> 1256612777.111   1145 66.199.62.74 TCP_MISS/200 17337 GET
> http://www.bing.com/search? admin DIRECT/8.17.64.41 text/xml
> 1256612777.605    931 66.199.62.74 TCP_MISS/200 15785 GET
> http://www.bing.com/search? admin DIRECT/8.17.64.8 text/xml
> 1256612778.321    217 66.199.62.74 TCP_MISS/200 475 GET
> http://xml.alexa.com/data? admin DIRECT/75.101.162.204 text/xml
> 1256612778.423    228 66.199.62.74 TCP_MISS/200 475 GET
> http://xml.alexa.com/data? admin DIRECT/75.101.145.196 text/xml
>
>
> Id like to see if I can extract somehow the data after the ? (query string).
>
> Thanks..!
>
> Sincerely,
>  Andres S.
>
Received on Tue Oct 27 2009 - 04:49:48 MDT