Re: [squid-users] Any recommended squid log analyzer?

See inline.

Thanks,
kc

2010/9/7 Amos Jeffries <squid3_at_treenet.co.nz>:
> On Tue, 7 Sep 2010 01:45:46 +0800, Kaiwang Chen <kaiwang.chen_at_gmail.com>
> wrote:
>> Hello,
>>
>>
>> Looks like awstats.pl cannot recognize "squid" logformat from
>> squid3.1.6, as pointed out by LogFormat in awstats.mysite.conf
>> # Possible values: 1,2,3,4 or "your_own_personalized_log_format"
>> # 1 - Apache or Lotus Notes/Domino native combined log format (NCSA
>> combined/XLF/ELF log format)
>> # 2 - IIS or ISA format (IIS W3C log format). See FAQ-COM115 For ISA.
>> # 3 - Webstar native log format.
>> # 4 - Apache or Squid native common log format (NCSA common/CLF log
> format)
>>
>> Its capability for customization is also too limited to understand
>> logformat directives in squid.conf.documented, such as %tr.
>>
>> So any recommendation?
>
> Maybe an apache-format log just for awstats?
>  access_log /var/logs/squid-awstats.log common

So I have to keep redundant access log contents, 2 log entries per 1
access, one for awstats, ther other for other analysis.

>
> We used awstats.pl for the wiki.squid-cache.org website under 3.0 and IIRC
> early 3.1 releases before the log server sponsorship dropped. The native
> 'squid' format has not been changed since 2.6. Just the text held in some
> fields has been extended.

So you were using Common Log Format, rather than squid native format?
Looks like the logformat of 4 (CLF?) in awstats.pl differs from the
(native?) "squid" logformat...

-- awstats.pl --
$VERSION = "6.95 (build $REVISION)";

                elsif ( $LogFormat eq '4' ) { # Same than "%h %l %u
%t \"%r\" %>s %b"
                         # %u (user) is "(.+)" instead of "[^ ]+"
because can contain space (Lotus Notes).
                        $PerlParsingFormat =
"([^ ]+) [^ ]+ (.+) \\[([^ ]+) [^ ]+\\] \\\"([^ ]+) ([^ ]+)(?:
[^\\\"]+|)\\\" ([\\d|-]+) ([\\d|-]+)";
                        $pos_host = 0;
                        $pos_logname = 1;
                        $pos_date = 2;
                        $pos_method = 3;
                        $pos_url = 4;
                        $pos_code = 5;
                        $pos_size = 6;
                        @fieldlib =
                          ( 'host', 'logname', 'date', 'method',
'url', 'code', 'size' );
                }

-- /etc/squid/squid.conf.documented states(excerpted lines reordered)---
#Default:
# access_log /var/log/squid/access.log squid

# logformat squid %ts.%03tu %6tr %>a %Ss/%03>Hs %<st %rm %ru %un %Sh/%<A %mt
# ts Seconds since epoch
# tu subsecond time (milliseconds)
# tr Response time (milliseconds)
# >a Client source IP address
# [http::]Ss Squid request status (TCP_MISS etc)
# [http::]>Hs HTTP status code sent to the client
# [http::]<st Sent reply size including HTTP headers
# [http::]rm Request method (GET/POST etc)
# [http::]ru Request URL
# [http::]un User name
# [http::]Sh Squid hierarchy status (DEFAULT_PARENT etc)
# <A Server IP address or peer name
# [http::]mt MIME content type

>
> Amos
>
Received on Tue Sep 07 2010 - 12:29:56 MDT