Re: [squid-users] Any recommended squid log analyzer?

use the following command line to convert "squid" format to "LogFormat
4" in awstats:

perl -MPOSIX -anle '$F[3]=~s/[^0-9]//g; print "$F[2] - - [",
POSIX::strftime("%d/%b/%Y:%T %z",localtime($F[0])), "] \"$F[5] $F[6]\"
$F[3] $F[1]"'

Thanks,
kc

2010/9/7 Kaiwang Chen <kaiwang.chen_at_gmail.com>:
> See inline.
>
> Thanks,
> kc
>
> 2010/9/7 Amos Jeffries <squid3_at_treenet.co.nz>:
>> On Tue, 7 Sep 2010 01:45:46 +0800, Kaiwang Chen <kaiwang.chen_at_gmail.com>
>> wrote:
>>> Hello,
>>>
>>>
>>> Looks like awstats.pl cannot recognize "squid" logformat from
>>> squid3.1.6, as pointed out by LogFormat in awstats.mysite.conf
>>> # Possible values: 1,2,3,4 or "your_own_personalized_log_format"
>>> # 1 - Apache or Lotus Notes/Domino native combined log format (NCSA
>>> combined/XLF/ELF log format)
>>> # 2 - IIS or ISA format (IIS W3C log format). See FAQ-COM115 For ISA.
>>> # 3 - Webstar native log format.
>>> # 4 - Apache or Squid native common log format (NCSA common/CLF log
>> format)
>>>
>>> Its capability for customization is also too limited to understand
>>> logformat directives in squid.conf.documented, such as %tr.
>>>
>>> So any recommendation?
>>
>> Maybe an apache-format log just for awstats?
>>  access_log /var/logs/squid-awstats.log common
>
> So I have to keep redundant access log contents, 2 log entries per 1
> access, one for awstats, ther other for other analysis.
>
>>
>> We used awstats.pl for the wiki.squid-cache.org website under 3.0 and IIRC
>> early 3.1 releases before the log server sponsorship dropped. The native
>> 'squid' format has not been changed since 2.6. Just the text held in some
>> fields has been extended.
>
> So you were using Common Log Format, rather than squid native format?
> Looks like the logformat of 4 (CLF?) in awstats.pl differs from the
> (native?) "squid" logformat...
>
> -- awstats.pl --
> $VERSION  = "6.95 (build $REVISION)";
>
>                elsif ( $LogFormat eq '4' ) {    # Same than "%h %l %u
> %t \"%r\" %>s %b"
>                         # %u (user) is "(.+)" instead of "[^ ]+"
> because can contain space (Lotus Notes).
>                        $PerlParsingFormat =
> "([^ ]+) [^ ]+ (.+) \\[([^ ]+) [^ ]+\\] \\\"([^ ]+) ([^ ]+)(?:
> [^\\\"]+|)\\\" ([\\d|-]+) ([\\d|-]+)";
>                        $pos_host    = 0;
>                        $pos_logname = 1;
>                        $pos_date    = 2;
>                        $pos_method  = 3;
>                        $pos_url     = 4;
>                        $pos_code    = 5;
>                        $pos_size    = 6;
>                        @fieldlib    =
>                          ( 'host', 'logname', 'date', 'method',
> 'url', 'code', 'size' );
>                }
>
> -- /etc/squid/squid.conf.documented states(excerpted lines reordered)---
> #Default:
> # access_log /var/log/squid/access.log squid
>
> # logformat squid %ts.%03tu %6tr %>a %Ss/%03>Hs %<st %rm %ru %un %Sh/%<A %mt
> #               ts      Seconds since epoch
> #               tu      subsecond time (milliseconds)
> #               tr      Response time (milliseconds)
> #               >a      Client source IP address
> #               [http::]Ss      Squid request status (TCP_MISS etc)
> #               [http::]>Hs     HTTP status code sent to the client
> #               [http::]<st     Sent reply size including HTTP headers
> #               [http::]rm      Request method (GET/POST etc)
> #               [http::]ru      Request URL
> #               [http::]un      User name
> #               [http::]Sh      Squid hierarchy status (DEFAULT_PARENT etc)
> #               <A      Server IP address or peer name
> #               [http::]mt      MIME content type
>
>
>
>
>>
>> Amos
>>
>
Received on Wed Sep 08 2010 - 05:01:35 MDT