Re: [squid-users] possible SOAP problem with 3.1.4

Alan Lehman wrote:
>> From: Amos Jeffries [mailto:squid3_at_treenet.co.nz]
>> Sent: Tuesday, August 10, 2010 6:48 PM
>> To: Alan Lehman
>> Cc: squid-users_at_squid-cache.org
>> Subject: RE: [squid-users] possible SOAP problem with 3.1.4
>>
>> On Tue, 10 Aug 2010 09:14:05 -0500, "Alan Lehman" <alehman_at_gbateam.com>
>> wrote:
>>>>>> From: Amos Jeffries [mailto:squid3_at_treenet.co.nz]
>>>>>> Sent: Sunday, July 11, 2010 1:55 AM
>>>>>> To: squid-users_at_squid-cache.org
>>>>>> Subject: Re: [squid-users] possible SOAP problem with 3.1.4
>>>>>>
>>>>>> Alan Lehman wrote:
>>>>>>> We have particular application software license server for our
>>>> office
>>>>>>> that is located behind a Squid proxy. It stopped working after
>>>>>> upgrading
>>>>>>> Squid from 3.1.0.17 to 3.1.4. This server periodically goes to
>> the
>>>>>>> software company's web site to verify the license is valid and
>>>> upload
>>>>>>> user counts, etc. It appears to be some sort of SOAP
>> application.
>>>> The
>>>>>>> license server runs on a Windows server. From access.log:
>>>>>>>
>>>>>>> Running 3.1.0.17 (succeeds) -
>>>>>>> 1278609155.802 470 172.16.4.43 TCP_MISS/200 725 POST
>>>>>>> http://selectserver.bentley.com/bss/ws/Misc.asmx -
>>>>>> DIRECT/64.90.235.78
>>>>>>> text/xml
>>>>>>> 1278609157.482 1054 172.16.4.43 TCP_MISS/200 117679 POST
>>>>>>> http://selectserver.bentley.com/bss/ws/GatewayWS.asmx -
>>>>>>> DIRECT/64.90.235.78 text/xml
>>>>>>>
>>>>>>> Running 3.1.4 (fails) -
>>>>>>> 1278607986.223 1138 172.16.4.43 TCP_MISS/500 838 POST
>>>>>>> http://selectserver.bentley.com/bss/ws/Misc.asmx -
>>>>>> DIRECT/64.90.235.78
>>>>>>> application/soap+xml
>>>>>>> 1278607987.128 895 172.16.4.43 TCP_MISS/200 1178 POST
>>>>>>> http://selectserver.bentley.com/bss/ws/Misc.asmx -
>>>>>> DIRECT/64.90.235.78
>>>>>>> text/xml
>>>>>>>
>>>>>>> I verified the situation by going back to 3.1.0.17 with the same
>>>>>> config,
>>>>>>> whereupon it started working again. I tried adding cache deny
>> for
>>>>>> this
>>>>>>> domain but it didn't change anything.
>>>>>>>
>>>>>>> Any thoughts would be most appreciated.
>>>>>>> Thanks,
>>>>>>> Alan Lehman
>>>>>> Don't know the problem.
>>>>>> You are going to have to dig into the request/reply's a bit
>> further
>>>> to
>>>>>> see what the problems is.
>>>>>> The biggest difference between 3.1.0.17 and 3.1.4 is that
>> HTTP/1.1
>>>> is
>>>>>> sent to the server by 3.1.4. It may be doing some broken magic,
>> as
>>>>>> evidenced by the different response type given to Squid now.
>>>>>>
>>>>>> Amos
>>>>>> --
>>>>>> Please be using
>>>>>> Current Stable Squid 2.7.STABLE9 or 3.1.5
>>>>>
>>>>> So far I'm unable to determine a consistent pattern with
>> Wireshark.
>>>> Is there a way I can force 3.1.4 to use HTTP/1.0?
>>>>> Alan
>>>> You can reverse the 1.1 enabling patch found here:
>>>> http://www.squid-cache.org/Versions/v3/3.1/changesets/squid-3.1-
>>>> 9916.patch
>>>>
>>>> Amos
>>>> --
>>>> Please be using
>>>> Current Stable Squid 2.7.STABLE9 or 3.1.5
>>>
>>> Using Wireshark, I recorded the following conversation between the
>> license
>>> server and Squid-3.1.6. The capture with the patched version of squid
>> is
>>> very similar. It appears to me that the license server is not
>> responding
>>> correctly to Squid's 417, right? But why is Squid 3.1.6 (unpatched)
>>> issuing the 417?
>> Um, this is a little strange. The *server* is making these requests
>> through Squid?
>> The client-server model indicates the machine you are calling a server
>> here is in fact a client.
>>
>> So, the workaround is to turn on the ignore_expect100 directive in
>> Squid.
>> Which suppresses the 417 response going to clients.
>>
>>> POST http://selectserver.bentley.com/bss/ws/GatewayWS.asmx HTTP/1.1
>>> User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; MS Web Services Client
>>> Protocol 2.0.50727.3603)
>>> Content-Type: text/xml; charset=utf-8
>>> SOAPAction:
>> "http://bentley.com/selectserver/webservices/GetGatewayLicense"
>>> Host: selectserver.bentley.com
>>> Content-Length: 564
>>> Expect: 100-continue
>>> Proxy-Connection: Keep-Alive
>>>
>>> HTTP/1.0 417 Expectation Failed
>>> Server: squid/3.1.6
>>> Mime-Version: 1.0
>>> Date: Tue, 10 Aug 2010 13:40:31 GMT
>>> Content-Type: text/html
>>> Content-Length: 3944
>>> X-Squid-Error: ERR_INVALID_REQ 0
>>> Vary: Accept-Language
>>> Content-Language: en
>>> X-Cache: MISS from proxy2.gbateam.com
>>> Via: 1.0 proxy2.gbateam.com (squid/3.1.6)
>>> Proxy-Connection: close
>>>
>>> <!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01//EN"
>>> "http://www.w3.org/TR/html4/strict.dtd">
>>> <html><head>....
>>> </body></html>
>> So far so good.
>>
>>> <?xml version="1.0" encoding="utf-8"?><soap:Envelope
>>> xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/"
>>> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
>>>
>> xmlns:xsd="http://www.w3.org/2001/XMLSchema"><soap:Body><GetGatewayLice
>> nse
>> xmlns="http://bentley.com/selectserver/webservices/"><GatewayKey>062D04
>> 32571D1748859E50B1CD98B9DE</GatewayKey><GatewaySiteKeys><string>062D043
>> 2571D1748859E50B1CD98B9DE</string></GatewaySiteKeys><ComputerName>SUP1<
>> /ComputerName><SSHostName>selectserver.bentley.com</SSHostName></GetGat
>> ewayLicense></soap:Body></soap:Envelope>
>> Um, Where is that garbage coming from? The POST?
>>
>> Assuming so, that would make the client broken and maybe a bug in Squid
>> lettign that brokenness through.
>>
>> This type of behaviour is what the ignore_expect100 can help with.
>> Making
>> Squid suppress the 417 to the client and drop any 100 is receives from
>> the
>> server. Passing the request on as if it was a regular POST with body
>> directly after .
>>
>>
>>> POST http://selectserver.bentley.com/bss/ws/usagelogging.asmx
>> HTTP/1.0
>>> User-Agent: BSIlm/0.9.0.0
>>> Host: selectserver.bentley.com
>>> Content-Length: 0
>>> Proxy-Connection: Keep-Alive
>>> Pragma: no-cache
>>>
>>> HTTP/1.0 500 Internal Server Error
>>> Date: Tue, 10 Aug 2010 13:40:33 GMT
>>> Server: Microsoft-IIS/6.0
>>> X-Powered-By: ASP.NET
>>> X-AspNet-Version: 2.0.50727
>>> Cache-Control: private
>>> Content-Type: application/soap+xml; charset=utf-8
>>> Content-Length: 481
>>> X-Cache: MISS from proxy2.gbateam.com
>>> Via: 1.0 proxy2.gbateam.com (squid/3.1.6)
>>> Proxy-Connection: keep-alive
>> Weird.
>> <snip server reply body>
>>
>>> POST http://selectserver.bentley.com/bss/ws/usagelogging.asmx
>> HTTP/1.0
>>> User-Agent: BSIlm/0.9.0.0
>>> Host: selectserver.bentley.com
>>> Content-Type: text/xml; charset=UTF-8
>>> Proxy-Connection: Keep-Alive
>>> Pragma: no-cache
>>> Content-Length: 8153
>>> SOAPAction:
>>> "http://bentley.com/selectserver/webservices/ReportUsageEntries"
>>>
>>> <soap:Envelope.... ></soap:Envelope>
>>>
>>> HTTP/1.0 200 OK
>>> Date: Tue, 10 Aug 2010 13:40:43 GMT
>>> Server: Microsoft-IIS/6.0
>>> X-Powered-By: ASP.NET
>>> X-AspNet-Version: 2.0.50727
>>> Cache-Control: private, max-age=0
>>> Content-Type: text/xml; charset=utf-8
>>> Content-Length: 715
>>> X-Cache: MISS from proxy2.gbateam.com
>>> Via: 1.0 proxy2.gbateam.com (squid/3.1.6)
>>> Proxy-Connection: keep-alive
>>>
>>> <?xml version="1.0" encoding="utf-8"?><soap:Envelope
>>> xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/"
>>> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
>>>
>> xmlns:xsd="http://www.w3.org/2001/XMLSchema"><soap:Body><ReportUsageEnt
>> riesResponse
>> xmlns="http://bentley.com/selectserver/webservices/"><ReportUsageEntrie
>> sResult><Status>0</Status><LicenseAvaliable>1</LicenseAvaliable><Produc
>> tName>MicroStation</ProductName><Ack>MHd0oUWqXDp3tB89iGrAbXpR63A=</Ack>
>> <CompanyName>George
>>> Butler &
>>>
>> Associates</CompanyName><SiteID>4012052</SiteID><SelectServerSerialNumb
>> er>70000661800020</SelectServerSerialNumber><LicType>1</LicType></Repor
>> tUsageEntriesResult></ReportUsageEntriesResponse></soap:Body></soap:Env
>> elope>
>> Client then tries again without the Expect:. This is good behaviour
>> finally and it seems to work.
>>
>>> POST http://selectserver.bentley.com/bss/ws/usagelogging.asmx
>> HTTP/1.0
>>> User-Agent: BSIlm/0.9.0.0
>>> Host: selectserver.bentley.com
>>> Content-Length: 0
>>> Proxy-Connection: Keep-Alive
>>> Pragma: no-cache
>>>
>>> HTTP/1.0 500 Internal Server Error
>>> Date: Tue, 10 Aug 2010 13:40:44 GMT
>>> Server: Microsoft-IIS/6.0
>>> X-Powered-By: ASP.NET
>>> X-AspNet-Version: 2.0.50727
>>> Cache-Control: private
>>> Content-Type: application/soap+xml; charset=utf-8
>>> Content-Length: 481
>>> X-Cache: MISS from proxy2.gbateam.com
>>> Via: 1.0 proxy2.gbateam.com (squid/3.1.6)
>>> Proxy-Connection: keep-alive
>>>
>> Back to the weirdness with no apparent reason.
>>
>>
>> Amos
>
>
> Thanks for the help with this. Yes, the "license server" is the client. Sorry for that confusion. I added "ignore_expect_100 on". Now the license server client seems to be working and the conversation goes like this:
>
> POST http://selectserver.bentley.com/bss/ws/GatewayWS.asmx HTTP/1.1
> User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; MS Web Services Client Protocol 2.0.50727.3603)
> Content-Type: text/xml; charset=utf-8
> SOAPAction: "http://bentley.com/selectserver/webservices/GetGatewayLicense"
> Host: selectserver.bentley.com
> Content-Length: 564
> Expect: 100-continue
> Proxy-Connection: Keep-Alive
>
> <?xml version="1.0" encoding="utf-8"?><soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema"><soap:Body><GetGatewayLicense xmlns="http://bentley.com/selectserver/webservices/"><GatewayKey>062D0432571D1748859E50B1CD98B9DE</GatewayKey><GatewaySiteKeys><string>062D0432571D1748859E50B1CD98B9DE</string></GatewaySiteKeys><ComputerName>SUP1</ComputerName><SSHostName>selectserver.bentley.com</SSHostName></GetGatewayLicense></soap:Body></soap:Envelope>
>
> HTTP/1.0 200 OK
> Date: Thu, 12 Aug 2010 13:40:42 GMT
> Server: Microsoft-IIS/6.0
> X-Powered-By: ASP.NET
> X-AspNet-Version: 2.0.50727
> Cache-Control: private, max-age=0
> Content-Type: text/xml; charset=utf-8
> Content-Length: 118153
> X-Cache: MISS from proxy2.gbateam.com
> Via: 1.0 proxy2.gbateam.com (squid/3.1.6)
> Proxy-Connection: keep-alive
>
> <?xml version="1.0" encoding="utf-8"?><soap:Envelope
> [snip]
>
>
> So the "license server" is not understanding the 417?
> Do you see any problem with leaving ignore_expect_100 on ?

It's safe enough for use in production. Just the potential network lags
leading to annoyance. With the usual extra resource usage by the longer
request service times it allows.

So as not to be surprised by any client complaints of slowness you will
want to make yourself aware of the time delay doing the POST. For
comparison; with their small size they should really be just a few
milli-seconds when fully working with the 417. The workaround may see
this become whole seconds before the 200.

Amos

-- 
Please be using
   Current Stable Squid 2.7.STABLE9 or 3.1.6
   Beta testers wanted for 3.2.0.1