Re: [squid-users] WCCP

> From: Amos Jeffries <squid3_at_treenet.co.nz>
> Date: Fri, 30 Oct 2009 14:08:23 +1300
> Cc: "squid-users_at_squid-cache.org" <squid-users_at_squid-cache.org>
> Subject: Re: [squid-users] WCCP
>
> Ross Kovelman wrote:
>>> From: Amos Jeffries <squid3_at_treenet.co.nz>
>>> Date: Tue, 27 Oct 2009 12:17:12 +1300
>>> To: Ross Kovelman <rkovelman_at_gruskingroup.com>
>>> Cc: "squid-users_at_squid-cache.org" <squid-users_at_squid-cache.org>
>>> Subject: Re: [squid-users] WCCP
>>>
>>> On Wed, 21 Oct 2009 12:20:00 -0400, Ross Kovelman
>>> <rkovelman_at_gruskingroup.com> wrote:
>>>>> From: Ross Kovelman <rkovelman_at_gruskingroup.com>
>>>>> Date: Mon, 19 Oct 2009 22:35:36 -0400
>>>>> To: Amos Jeffries <squid3_at_treenet.co.nz>
>>>>> Cc: "squid-users_at_squid-cache.org" <squid-users_at_squid-cache.org>
>>>>> Subject: Re: [squid-users] WCCP
>>>>>
>>>>>> From: Amos Jeffries <squid3_at_treenet.co.nz>
>>>>>> Date: Tue, 20 Oct 2009 13:20:27 +1300
>>>>>> To: Ross Kovelman <rkovelman_at_gruskingroup.com>
>>>>>> Cc: "squid-users_at_squid-cache.org" <squid-users_at_squid-cache.org>
>>>>>> Subject: Re: [squid-users] WCCP
>>>>>>
>>>>>> On Mon, 19 Oct 2009 20:06:55 -0400, Ross Kovelman
>>>>>> <rkovelman_at_gruskingroup.com> wrote:
>>>>>>>> From: Amos Jeffries <squid3_at_treenet.co.nz>
>>>>>>>> Date: Tue, 20 Oct 2009 12:40:02 +1300
>>>>>>>> To: Ross Kovelman <rkovelman_at_gruskingroup.com>
>>>>>>>> Cc: "squid-users_at_squid-cache.org" <squid-users_at_squid-cache.org>
>>>>>>>> Subject: Re: [squid-users] WCCP
>>>>>>>>
>>>>>>>> On Mon, 19 Oct 2009 18:26:18 -0400, Ross Kovelman
>>>>>>>> <rkovelman_at_gruskingroup.com> wrote:
>>>>>>>>>> From: Amos Jeffries <squid3_at_treenet.co.nz>
>>>>>>>>>> Date: Tue, 20 Oct 2009 11:04:42 +1300
>>>>>>>>>> To: Ross Kovelman <rkovelman_at_gruskingroup.com>
>>>>>>>>>> Cc: "squid-users_at_squid-cache.org" <squid-users_at_squid-cache.org>
>>>>>>>>>> Subject: Re: [squid-users] WCCP
>>>>>>>>>>
>>>>>>>>>> On Mon, 19 Oct 2009 14:21:44 -0400, Ross Kovelman wrote:
>>>>>>>>>>>> From: Amos Jeffries
>>>>>>>>>>>>
>>>>>>>>>>>> Ross Kovelman wrote:
>>>>>>>>>>>> From: Amos Jeffries:
>>>>>>>>>>>>
>>>>>>>>>>>> Ross Kovelman wrote:
>>>>>>>>>>>> I am going to be using WCCP. I did another reconfigure with
>>>>>>>>>>>> the
>>>>>>>>>>>> --enable
>>>>>>>>>>>> WCCP option. How can I check that it is on and running? The
>>>>>> next
>>>>>>>>>>>> step I
>>>>>>>>>>>> need to do is upgrade to version 2 since the Cisco only
>>>>>>>> communicates
>>>>>>>>>>>> on
>>>>>>>>>>>> version 2. I tried to do the patch < upgrade patch but then
>>> I
>>>>>> get
>>>>>>>> a
>>>>>>>>>>>> response with path to upgrade and I am not sure where the
>>> file
>>>>>> is
>>>>>>>> I
>>>>>>>>>>>> need
>>>>>>>>>>>> patch.
>>>>>>>>>>>> There is zero need to patch for support WCCPv2. It's been
>>> built
>>>>>>>> into
>>>>>>>>>>>> Squid for many years now.
>>>>>>>>>>>>
>>>>>>>>>>>> Run "./configure --help".
>>>>>>>>>>>> * If it lists "--disable-wccpv2" there is no need to do
>>>>>> anything.
>>>>>>>>>>>> * If it lists "--enable-wccpv2" , add that to your build
>>>>>> options.
>>>>>>>>>>>> * If it does not mention "wccpv2" at all upgrade your Squid
>>>>>>>>>> version.
>>>>>>>>>>>> Then setup squid.conf with the relevant wccp2_* options.
>>>>>>>>>>>>
>>>>>>>>>>>> http://www.squid-cache.org/Doc/config/ or the wiki example
>>>>>> configs
>>>>>>>>>> have
>>>>>>>>>>>> details on those.
>>>>>>>>>>>> Thanks again.
>>>>>>>>>>>> Running the ./configure --help only says this:
>>>>>>>>>>>> --disable-wccp Disable Web Cache Coordination V1
>>>>>> Protocol
>>>>>>>>>>>> --disable-wccpv2 Disable Web Cache Coordination V2
>>>>>> Protocol
>>>>>>>>>>>> When I did the install I ran the ./configure --enable wccp
>>>>>>>>>>>> option.
>>>>>> I
>>>>>>>>>>>> didn't
>>>>>>>>>>>> say --enable-wccpv2, does this matter? I also have this in the
>>>>>>>>>> config:
>>>>>>>>>>>> wccp2_router 192.168.16.1
>>>>>>>>>>>> wccp2_forwarding_method 1
>>>>>>>>>>>> wccp2_return_method 1
>>>>>>>>>>>>
>>>>>>>>>>>> I am running Squid Web Proxy 2.7.STABLE5.
>>>>>>>>>>>> Okay. Thats fine.
>>>>>>>>>>>>
>>>>>>>>>>>> The ./configure results mean that both WCCP versions are built
>>>>>>>>>>>> into
>>>>>>>>>>>> Squid by default unless you explicitly say --disable. Nothing
>>>>>>>>>>>> extra
>>>>>>>>>>>> needed to build them.
>>>>>>>>>>>>
>>>>>>>>>>>> The config options you have there are already WCCPv2-only
>>> options
>>>>>> for
>>>>>>>>>>>> Cisco. Nothing new needed there either.
>>>>>>>>>>>>
>>>>>>>>>>>> If thats not working its a config error somewhere.
>>>>>>>>>>>>
>>>>>>>>>>> I am getting this in my cache log:
>>>>>>>>>>>
>>>>>>>>>>> Accepting proxy HTTP connections at 0.0.0.0, port 3128, FD 20.
>>>>>>>>>>> commBind: Cannot bind socket FD 21 to *:3128: (48) Address
>>> already
>>>>>> in
>>>>>>>>>> use
>>>>>>>>>>> Accepting proxy HTTP connections at 0.0.0.0, port 80, FD 21.
>>>>>>>>>>> commBind: Cannot bind socket FD 22 to *:80: (48) Address already
>>> in
>>>>>>>> use
>>>>>>>>>>
>> http://wiki.squid-cache.org/SquidFaq/TroubleShooting#Cannot_bind_socket_FD_NN
>> >>
>>>> _
>>>>>>>>>> to_.2A:8080_.28125.29_Address_already_in_use
>>>>>>>>>>
>>>>>>>>>> I would suspect this as part of the problem. The WCCP router will
>>> be
>>>>>>>>>> trying to contact whatever software is already running on port
>>> 3128,
>>>>>>>> not
>>>>>>>>>> the Squid you are starting with WCCP config.
>>>>>>>>>>
>>>>>>>>>>> Accepting ICP messages at 0.0.0.0, port 3130, FD 22.
>>>>>>>>>>> WCCP Disabled.
>>>>>>>>>>> Accepting WCCPv2 messages on port 2048, FD 23.
>>>>>>>> To answer your earlier question:
>>>>>>>> the above two lines means WCCPv1 is disabled, WCCPv2 is being
>>> used.
>>>>>>>>>>> Initialising all WCCPv2 lists
>>>>>>>>>>>
>>>>>>>>>>> As from my other posting I need WCCP enabled but it is showing
>>>>>>>> disabled.
>>>>>>>>>>> Any reason why? How can I resolve this. Below is my lines in
>>>>>> config
>>>>>>>>>>> wccp2_router 192.168.16.1
>>>>>>>>>>> wccp2_forwarding_method 1
>>>>>>>>>>> wccp2_return_method 1
>>>>>>>>>> The above are only the config of how squid sends packets to the
>>>>>> Cisco.
>>>>>>>>>> WCCP requires configuration Cisco, the squid box OS and firewall,
>>>>>>>>>> and
>>>>>>>>>> routing tables. Any one of which could be the problem.
>>>>>>>>>> The tutorials and troubleshooting info we have at present is a
>>>>>>>>>> little
>>>>>>>>>> spread out and disjointed. What how-to are you working from?
>>>>>>>>>>
>>>>>>>>>> Amos
>>>>>>>>> Amos,
>>>>>>>>> I just did a TCP dump and I think my problem is the GRE packet. It
>>>>>>>>> is
>>>>>>>>> being
>>>>>>>>> listed I think as unknown. Shouldn't squid be able to pick the
>>>>>>>>> packet
>>>>>>>> up
>>>>>>>>> and open it? The Cisco sees squid and relays the information good
>>>>>>>>> but
>>>>>>>> it
>>>>>>>>> is
>>>>>>>>> stopping at the squid box. Any ideas? I am just google'ing around
>>> no
>>>>>>>> set
>>>>>>>>> how to.
>>>>>>>> Okay. I've polished up our exemplar configs a little:
>>>>>>>> http://wiki.squid-cache.org/Features/Wccp2
>>>>>>>> (some way to go though).
>>>>>>>>
>>>>>>>> There are four parts to WCCP systems:
>>>>>>>>
>>>>>>>> 1) WCCP capture and redirect
>>>>>>>>
>>>>>>>> 2) gre tunnel between the Cisco and Squid boxes
>>>>>>>>
>>>>>>>> 3) squid box firewall settings and NAT capture of received gre
>>>>>>>> packets
>>>>>>>>
>>>>>>>>
>> http://wiki.squid-cache.org/ConfigExamples/Intercept#Traffic_Interception_cap
>> >>
>>>> t
>>>>>>>> ure_into_Squid
>>>>>>>>
>>>>>>>> 4) squid.conf settings to make Squid contact the cisco router
>>>>>>>>
>>>>>>>> Amos
>>>>>>>>
>>>>>>> From what I have read and what you show only for the PIX and ASA
>>> should
>>>>>> be
>>>>>>> the same. The Pix is actually correct for the ASA, although that is
>>>>>> what
>>>>>>> Cisco told me to do.
>>> Hmm, I was worried a bit by this. Then realized what the problem was.
>>> The difference appears to have been only a security ACL added to the ASA
>>> config and the screwy wrapping.
>>>
>>> Thanks for that hint.
>>>
>>>>>>> As far as:
>>>>>>> wccp2_router - My cisco router address
>>>>>>> wccp2_forwarding_method - I took this out of my config as GRE is
>>>>>>> default
>>>>>>> wccp2_return_method - same as forward
>>>>>>> wccp2_assignment_method - nothing in config
>>>>>>> wccp2_service - nothing in config
>>>>>>>
>>>>>>> Am I missing something? If I have my cisco config turned on for WCCP
>>>>>> and
>>>>>>> squid running no one can browse the web. If I turn squid off and
>>> leave
>>>>>>> wccp
>>>>>>> running on the Cisco browsing web is perfect. No issues. Anything
>>> else
>>>>>> to
>>>>>>> check?
>>>>>> ... rp_filter settings on the Squid box are turned off.
>>>>>>
>>>>>> ... iptables does REDIRECT or DNAT capture of the packets to the Squid
>>>>>> http_port marked with "transparent"
>>>>>>
>>>>>>> bert:~ administrator$ sudo tcpdump -n -i en1 ip proto gre
>>>>>>> tcpdump: verbose output suppressed, use -v or -vv for full protocol
>>>>>> decode
>>>>>>> listening on en1, link-type EN10MB (Ethernet), capture size 96 bytes
>>>>>>> 15:00:33.599161 IP 192.168.xx.1 > 192.168.xx.xxx: GREv0, length 60:
>>>>>>> gre-proto-0x883e
>>>>>>> 15:00:34.715585 IP 192.168.xx.1 > 192.168.xx.xxx: GREv0, length 60:
>>>>>>> gre-proto-0x883e
>>>>>>> 15:00:34.805734 IP 192.168.xx.1 > 192.168.xx.xxx: GREv0, length 56:
>>>>>>> gre-proto-0x883e
>>>>>>> 15:00:34.808181 IP 192.168.xx.1 > 192.168.xx.xxx: GREv0, length 56:
>>>>>>> gre-proto-0x883e gre-proto-0x883e
>>>>>>> 15:00:34.805734 IP 192.168.xx.1 > 192.168.xx.xxx: GREv0, length 56:
>>>>>>> gre-proto-0x883e
>>>>>>> 15:00:34.808181 IP 192.168.xx.1 > 192.168.xx.xxx: GREv0, length 56:
>>>>>>> gre-proto-0x883e
>>>>>>>
>>>>>>> Does that help? Let me know what you need from me so we can resolve
>>>>>> this.
>>>>>>> I did mask off my IP but the IP prior to the > is the ASA and the
>>>>>> numbers
>>>>>>> after is the squid server
>>>>>>>
>>>>>>> Thanks
>>>>> Amos,
>>>>>
>>>>> I have this in my sysctl config:
>>>>> net.ipv4.ip_forward =1
>>>>> net.ipv4.conf.all.rp_filter = 0
>>>>>
>>>>> That should take care of the rp_filter. Although how can I check that
>>> I
>>>>> don't know. I am also running transparent so I assume that iptables
>>>>> thing
>>>>> you wrote I do not need to do?
>>>>>
>>>>> Thanks
>>>>>
>>>>>
>>>> I am starting to look more into this and what I see is this on the
>>> firewall
>>>> log:
>>>> Oct 21 12:03:37 bert ipfw: 12313 Accept P:47 192.168.xx.1
>>> 192.168.xx.xxx
>>>> in
>>>> via en1
>>>>
>>>> P47 is GRE so I can see that the GRE packet from the ASA is passed and
>>>> accepted to the squid server. I do not think Squid knows how to either
>>>> decipher the GRE packet and or when it tries to send the information
>>> back
>>>> out its not going back to the client or ASA. How can I resolve this?
>>> Aha, you have an ASA. Somehow I missed that detail earlier. This is the
>>> specific ASA config details we have so far:
>>> http://wiki.squid-cache.org/ConfigExamples/Intercept/CiscoAsaWccp2
>>>
>>> Check that you have the squid bypass in the config. Thats one of the
>>> critical parts.
>>>
>>> Good tracking so far.
>>>
>>> It's the OS business to unwrap the GRE packet into a normal TCP packet
>>> before passing it to Squid. I'm not sure how ipfw ensures that. modprobe
>>> ip_gre?
>>>
>>> The next bit will be to see if Squid receives the packet at all. With
>>> debug_options ALL,6 or so cache.log should record a connection accepted
>>> from the client and show what happens to it.
>>>
>>> Amos
>>>
>> Amos,
>>
>> Got it working, but I am having some timeout issues when browsing all
>> websites. Do you know why or know what I can look for? I do see the ASA
>> and squid server communicating now.
>>
>> Thanks
>
> Not a clue. I'd guess some delays on the network. Perhaps during DNS
> lookups.
>
> Amos
> --
> Please be using
> Current Stable Squid 2.7.STABLE7 or 3.0.STABLE19
> Current Beta Squid 3.1.0.14

Amos,

I got cisco on the phone and did some packet debugging. What we found is
the ASA and squid communicate. The problem lies where for some reason the
clients browser will report a "connection time out". Do you know of
anything that should be done, IPTables or Firewall so the packet does go
correctly to the client. I was told WCCP communicates with the ASA and
Squid and then from there the request is between squid and the client. I
have done everything here:
http://www.sublime.com.au/squid-wccp/
Besides compile of the wccp module and kernal rebuilding. I know this is
not squid issue, or at least I do not think so, so an Fedora help you can
give would be appreciated. I have also followed the book from O'reilly
called squid. I am going to dig a lil deeper on the server side doing dumps
to see what else I find. The only other clue I see is the packet going from
the server to the ASA:
ICMP Destinatoon unreachable (Host administratively prohibited)

Thanks