Re: [squid-users] WCCP

> From: Amos Jeffries <squid3_at_treenet.co.nz>
> Date: Tue, 27 Oct 2009 12:17:12 +1300
> To: Ross Kovelman <rkovelman_at_gruskingroup.com>
> Cc: "squid-users_at_squid-cache.org" <squid-users_at_squid-cache.org>
> Subject: Re: [squid-users] WCCP
>
> On Wed, 21 Oct 2009 12:20:00 -0400, Ross Kovelman
> <rkovelman_at_gruskingroup.com> wrote:
>>> From: Ross Kovelman <rkovelman_at_gruskingroup.com>
>>> Date: Mon, 19 Oct 2009 22:35:36 -0400
>>> To: Amos Jeffries <squid3_at_treenet.co.nz>
>>> Cc: "squid-users_at_squid-cache.org" <squid-users_at_squid-cache.org>
>>> Subject: Re: [squid-users] WCCP
>>>
>>>> From: Amos Jeffries <squid3_at_treenet.co.nz>
>>>> Date: Tue, 20 Oct 2009 13:20:27 +1300
>>>> To: Ross Kovelman <rkovelman_at_gruskingroup.com>
>>>> Cc: "squid-users_at_squid-cache.org" <squid-users_at_squid-cache.org>
>>>> Subject: Re: [squid-users] WCCP
>>>>
>>>> On Mon, 19 Oct 2009 20:06:55 -0400, Ross Kovelman
>>>> <rkovelman_at_gruskingroup.com> wrote:
>>>>>> From: Amos Jeffries <squid3_at_treenet.co.nz>
>>>>>> Date: Tue, 20 Oct 2009 12:40:02 +1300
>>>>>> To: Ross Kovelman <rkovelman_at_gruskingroup.com>
>>>>>> Cc: "squid-users_at_squid-cache.org" <squid-users_at_squid-cache.org>
>>>>>> Subject: Re: [squid-users] WCCP
>>>>>>
>>>>>> On Mon, 19 Oct 2009 18:26:18 -0400, Ross Kovelman
>>>>>> <rkovelman_at_gruskingroup.com> wrote:
>>>>>>>> From: Amos Jeffries <squid3_at_treenet.co.nz>
>>>>>>>> Date: Tue, 20 Oct 2009 11:04:42 +1300
>>>>>>>> To: Ross Kovelman <rkovelman_at_gruskingroup.com>
>>>>>>>> Cc: "squid-users_at_squid-cache.org" <squid-users_at_squid-cache.org>
>>>>>>>> Subject: Re: [squid-users] WCCP
>>>>>>>>
>>>>>>>> On Mon, 19 Oct 2009 14:21:44 -0400, Ross Kovelman wrote:
>>>>>>>>>> From: Amos Jeffries
>>>>>>>>>>
>>>>>>>>>> Ross Kovelman wrote:
>>>>>>>>>>>> From: Amos Jeffries:
>>>>>>>>>>>>
>>>>>>>>>>>> Ross Kovelman wrote:
>>>>>>>>>>>> I am going to be using WCCP. I did another reconfigure with
>>>>>>>>>>>> the
>>>>>>>>>>>> --enable
>>>>>>>>>>>> WCCP option. How can I check that it is on and running? The
>>>> next
>>>>>>>>>>>> step I
>>>>>>>>>>>> need to do is upgrade to version 2 since the Cisco only
>>>>>> communicates
>>>>>>>>>>>> on
>>>>>>>>>>>> version 2. I tried to do the patch < upgrade patch but then
> I
>>>> get
>>>>>> a
>>>>>>>>>>>> response with path to upgrade and I am not sure where the
> file
>>>> is
>>>>>> I
>>>>>>>>>>>> need
>>>>>>>>>>>> patch.
>>>>>>>>>>>> There is zero need to patch for support WCCPv2. It's been
> built
>>>>>> into
>>>>>>>>>>>> Squid for many years now.
>>>>>>>>>>>>
>>>>>>>>>>>> Run "./configure --help".
>>>>>>>>>>>> * If it lists "--disable-wccpv2" there is no need to do
>>>> anything.
>>>>>>>>>>>> * If it lists "--enable-wccpv2" , add that to your build
>>>> options.
>>>>>>>>>>>> * If it does not mention "wccpv2" at all upgrade your Squid
>>>>>>>> version.
>>>>>>>>>>>>
>>>>>>>>>>>> Then setup squid.conf with the relevant wccp2_* options.
>>>>>>>>>>>>
>>>>>>>>>>>> http://www.squid-cache.org/Doc/config/ or the wiki example
>>>> configs
>>>>>>>> have
>>>>>>>>>>>> details on those.
>>>>>>>>>>>
>>>>>>>>>>> Thanks again.
>>>>>>>>>>> Running the ./configure --help only says this:
>>>>>>>>>>> --disable-wccp Disable Web Cache Coordination V1
>>>> Protocol
>>>>>>>>>>> --disable-wccpv2 Disable Web Cache Coordination V2
>>>> Protocol
>>>>>>>>>>>
>>>>>>>>>>> When I did the install I ran the ./configure --enable wccp
>>>>>>>>>>> option.
>>>> I
>>>>>>>>>>> didn't
>>>>>>>>>>> say --enable-wccpv2, does this matter? I also have this in the
>>>>>>>> config:
>>>>>>>>>>> wccp2_router 192.168.16.1
>>>>>>>>>>> wccp2_forwarding_method 1
>>>>>>>>>>> wccp2_return_method 1
>>>>>>>>>>>
>>>>>>>>>>> I am running Squid Web Proxy 2.7.STABLE5.
>>>>>>>>>>
>>>>>>>>>> Okay. Thats fine.
>>>>>>>>>>
>>>>>>>>>> The ./configure results mean that both WCCP versions are built
>>>>>>>>>> into
>>>>>>>>>> Squid by default unless you explicitly say --disable. Nothing
>>>>>>>>>> extra
>>>>>>>>>> needed to build them.
>>>>>>>>>>
>>>>>>>>>> The config options you have there are already WCCPv2-only
> options
>>>> for
>>>>>>>>>> Cisco. Nothing new needed there either.
>>>>>>>>>>
>>>>>>>>>> If thats not working its a config error somewhere.
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> I am getting this in my cache log:
>>>>>>>>>
>>>>>>>>> Accepting proxy HTTP connections at 0.0.0.0, port 3128, FD 20.
>>>>>>>>> commBind: Cannot bind socket FD 21 to *:3128: (48) Address
> already
>>>> in
>>>>>>>> use
>>>>>>>>> Accepting proxy HTTP connections at 0.0.0.0, port 80, FD 21.
>>>>>>>>> commBind: Cannot bind socket FD 22 to *:80: (48) Address already
> in
>>>>>> use
>>>>>>>>
>>>>>>>>
>>>>>>
>>>>
>>
>
http://wiki.squid-cache.org/SquidFaq/TroubleShooting#Cannot_bind_socket_FD_NN>>
>
>> _
>>>>>>>> to_.2A:8080_.28125.29_Address_already_in_use
>>>>>>>>
>>>>>>>> I would suspect this as part of the problem. The WCCP router will
> be
>>>>>>>> trying to contact whatever software is already running on port
> 3128,
>>>>>> not
>>>>>>>> the Squid you are starting with WCCP config.
>>>>>>>>
>>>>>>>>> Accepting ICP messages at 0.0.0.0, port 3130, FD 22.
>>>>>>>>> WCCP Disabled.
>>>>>>>>> Accepting WCCPv2 messages on port 2048, FD 23.
>>>>>>
>>>>>> To answer your earlier question:
>>>>>> the above two lines means WCCPv1 is disabled, WCCPv2 is being
> used.
>>>>>>
>>>>>>>>> Initialising all WCCPv2 lists
>>>>>>>>>
>>>>>>>>> As from my other posting I need WCCP enabled but it is showing
>>>>>> disabled.
>>>>>>>>> Any reason why? How can I resolve this. Below is my lines in
>>>> config
>>>>>>>>>
>>>>>>>>> wccp2_router 192.168.16.1
>>>>>>>>> wccp2_forwarding_method 1
>>>>>>>>> wccp2_return_method 1
>>>>>>>>
>>>>>>>> The above are only the config of how squid sends packets to the
>>>> Cisco.
>>>>>>>> WCCP requires configuration Cisco, the squid box OS and firewall,
>>>>>>>> and
>>>>>>>> routing tables. Any one of which could be the problem.
>>>>>>>> The tutorials and troubleshooting info we have at present is a
>>>>>>>> little
>>>>>>>> spread out and disjointed. What how-to are you working from?
>>>>>>>>
>>>>>>>> Amos
>>>>>>>
>>>>>>> Amos,
>>>>>>> I just did a TCP dump and I think my problem is the GRE packet. It
>>>>>>> is
>>>>>>> being
>>>>>>> listed I think as unknown. Shouldn't squid be able to pick the
>>>>>>> packet
>>>>>> up
>>>>>>> and open it? The Cisco sees squid and relays the information good
>>>>>>> but
>>>>>> it
>>>>>>> is
>>>>>>> stopping at the squid box. Any ideas? I am just google'ing around
> no
>>>>>> set
>>>>>>> how to.
>>>>>>
>>>>>> Okay. I've polished up our exemplar configs a little:
>>>>>> http://wiki.squid-cache.org/Features/Wccp2
>>>>>> (some way to go though).
>>>>>>
>>>>>> There are four parts to WCCP systems:
>>>>>>
>>>>>> 1) WCCP capture and redirect
>>>>>>
>>>>>> 2) gre tunnel between the Cisco and Squid boxes
>>>>>>
>>>>>> 3) squid box firewall settings and NAT capture of received gre
>>>>>> packets
>>>>>>
>>>>>>
>>>>
>>
>
http://wiki.squid-cache.org/ConfigExamples/Intercept#Traffic_Interception_cap>>
>
>> t
>>>>>> ure_into_Squid
>>>>>>
>>>>>> 4) squid.conf settings to make Squid contact the cisco router
>>>>>>
>>>>>> Amos
>>>>>>
>>>>> From what I have read and what you show only for the PIX and ASA
> should
>>>> be
>>>>> the same. The Pix is actually correct for the ASA, although that is
>>>> what
>>>>> Cisco told me to do.
>
> Hmm, I was worried a bit by this. Then realized what the problem was.
> The difference appears to have been only a security ACL added to the ASA
> config and the screwy wrapping.
>
> Thanks for that hint.
>
>>>>>
>>>>> As far as:
>>>>> wccp2_router - My cisco router address
>>>>> wccp2_forwarding_method - I took this out of my config as GRE is
>>>>> default
>>>>> wccp2_return_method - same as forward
>>>>> wccp2_assignment_method - nothing in config
>>>>> wccp2_service - nothing in config
>>>>>
>>>>> Am I missing something? If I have my cisco config turned on for WCCP
>>>> and
>>>>> squid running no one can browse the web. If I turn squid off and
> leave
>>>>> wccp
>>>>> running on the Cisco browsing web is perfect. No issues. Anything
> else
>>>> to
>>>>> check?
>>>>
>>>> ... rp_filter settings on the Squid box are turned off.
>>>>
>>>> ... iptables does REDIRECT or DNAT capture of the packets to the Squid
>>>> http_port marked with "transparent"
>>>>
>>>>>
>>>>> bert:~ administrator$ sudo tcpdump -n -i en1 ip proto gre
>>>>> tcpdump: verbose output suppressed, use -v or -vv for full protocol
>>>> decode
>>>>> listening on en1, link-type EN10MB (Ethernet), capture size 96 bytes
>>>>> 15:00:33.599161 IP 192.168.xx.1 > 192.168.xx.xxx: GREv0, length 60:
>>>>> gre-proto-0x883e
>>>>> 15:00:34.715585 IP 192.168.xx.1 > 192.168.xx.xxx: GREv0, length 60:
>>>>> gre-proto-0x883e
>>>>> 15:00:34.805734 IP 192.168.xx.1 > 192.168.xx.xxx: GREv0, length 56:
>>>>> gre-proto-0x883e
>>>>> 15:00:34.808181 IP 192.168.xx.1 > 192.168.xx.xxx: GREv0, length 56:
>>>>> gre-proto-0x883e gre-proto-0x883e
>>>>> 15:00:34.805734 IP 192.168.xx.1 > 192.168.xx.xxx: GREv0, length 56:
>>>>> gre-proto-0x883e
>>>>> 15:00:34.808181 IP 192.168.xx.1 > 192.168.xx.xxx: GREv0, length 56:
>>>>> gre-proto-0x883e
>>>>>
>>>>> Does that help? Let me know what you need from me so we can resolve
>>>> this.
>>>>> I did mask off my IP but the IP prior to the > is the ASA and the
>>>> numbers
>>>>> after is the squid server
>>>>>
>>>>> Thanks
>>>
>>> Amos,
>>>
>>> I have this in my sysctl config:
>>> net.ipv4.ip_forward =1
>>> net.ipv4.conf.all.rp_filter = 0
>>>
>>> That should take care of the rp_filter. Although how can I check that
> I
>>> don't know. I am also running transparent so I assume that iptables
>>> thing
>>> you wrote I do not need to do?
>>>
>>> Thanks
>>>
>>>
>>
>> I am starting to look more into this and what I see is this on the
> firewall
>> log:
>> Oct 21 12:03:37 bert ipfw: 12313 Accept P:47 192.168.xx.1
> 192.168.xx.xxx
>> in
>> via en1
>>
>> P47 is GRE so I can see that the GRE packet from the ASA is passed and
>> accepted to the squid server. I do not think Squid knows how to either
>> decipher the GRE packet and or when it tries to send the information
> back
>> out its not going back to the client or ASA. How can I resolve this?
>
> Aha, you have an ASA. Somehow I missed that detail earlier. This is the
> specific ASA config details we have so far:
> http://wiki.squid-cache.org/ConfigExamples/Intercept/CiscoAsaWccp2
>
> Check that you have the squid bypass in the config. Thats one of the
> critical parts.
>
> Good tracking so far.
>
> It's the OS business to unwrap the GRE packet into a normal TCP packet
> before passing it to Squid. I'm not sure how ipfw ensures that. modprobe
> ip_gre?
>
> The next bit will be to see if Squid receives the packet at all. With
> debug_options ALL,6 or so cache.log should record a connection accepted
> from the client and show what happens to it.
>
> Amos
>
Amos,

Got it working, but I am having some timeout issues when browsing all
websites. Do you know why or know what I can look for? I do see the ASA
and squid server communicating now.

Thanks