Re: [squid-users] HTTPS forward proxy?

Hi Amos, thanks for your reply.

On Jan 21, 2014, at 4:29 PM, Amos Jeffries <squid3_at_treenet.co.nz> wrote:

> On 2014-01-22 09:25, David Deller wrote:
>> Hello,
>> I’m trying to set up Squid as an HTTPS forward proxy, but I’m having
>> trouble getting it to work.
>> Here's some background about my problem:
>> * I have a web service running on Heroku, with a dynamic IP address.
>> Static IPs on Heroku are not an option.
>> * I need to connect to an external web service which is behind a
>> firewall. The people who operate the external web service will only
>> open their firewall to a specific static IP.
>> My attempted solution is to use Squid on a separate server with a
>> static IP to forward-proxy requests from Heroku to the external
>> service. That way, the external service always sees the proxy server's
>> static IP, instead of the Heroku service's dynamic IP.
>> Since my proxy server can't rely on an IP address for authentication
>
> You mean authorization, not authentication. A number (IP address) cannot own credentials, only an entity can have credentials and credentials are required for authentication.

Sorry - I really do know the difference between the two auths, but it sometimes comes out wrong.

>> (that's the problem to begin with!), it must rely on a username and
>> password. Further, the username and password cannot be transmitted in
>> clear text,
>
> Meaning: avoid Basic authentication type.
>
>> because if an attacker were to intercept that clear text,
>> then they could connect to my proxy pretending to be me, make outbound
>> requests using my proxy's static IP, and thus evade the external web
>> service's firewall.
>
> Squid is not limited to the Basic authentication protocol. I see you are already using Digest authentication, which already does what you want even in unencrypted HTTP.
>
>> Therefore, the Squid proxy must only accept connections over HTTPS,
>> not HTTP. (The connection to the external web service might be HTTP or
>> HTTPS.)
>
> No. Given you already use Digest auth the SSL layer is not necessary to prevent that attack type. It is just an extra layer to obscure the content being transferred. I think it is a good idea to use it of possible, but am aware there are some big troubles (namely that modern browsers still do not support connecting to a proxy over SSL/TLS).

I am distrustful of Digest because it uses MD5. The Wikipedia article about it seems to suggest that this may not in fact be a problem, despite MD5’s known weaknesses. Some quick Googling does not suggest a widespread consensus.

>> I'm running Squid 3.1.10 on CentOS 6.5.x,
>
> Please try an upgrade. You can find details about newer versions for CentOS at http://wiki.squid-cache.org/KnowledgeBase/CentOS

I will look into this. I have had problems with third-party repositories in the past.

> <snip>
>> Using this setup, HTTP proxying works fine, but HTTPS proxying does not.
>> Here's an HTTP proxy request from a local box:
>> $ curl --proxy http://my-proxy-server.example:3128 \
>> --proxy-anyauth --proxy-user redacted:redacted -w '\n' \
>> http://urlecho.appspot.com/echo?body=OK
>> OK
> <snip>
>> Here's another request, this time with HTTPS:
>> $ curl --proxy https://my-proxy-server.example:3129 \
>> --proxy-anyauth --proxy-user redacted:redacted -w '\n' \
>> http://urlecho.appspot.com/echo?body=OK
>> curl: (56) Recv failure: Connection reset by peer
>> Nothing in `access.log` after this one, but in `cache.log`:
>> 2014/01/20 20:46:15| clientNegotiateSSL: Error negotiating SSL
>> connection on FD 10: error:1407609C:SSL
>> routines:SSL23_GET_CLIENT_HELLO:http request (1/-1)
>
> See the serverfault response. curl is connecting to the proxy using plain-text instead of SSL.

I want to be able to connect to HTTP servers over the proxy as well, so changing the protocol of the URL I’m trying to access (per the serverfault comments) will not help me in cases where the remote web server does not support HTTPS. Are you saying that Squid does not support proxying plain HTTP requests over an HTTPS proxy? I could not find any documentation one way or the other about this.

In any case, I tried it with an HTTPS request to be sure, and it still failed with the same error (even though the remote web server in this case does support HTTPS).

>> Here's the above again, more verbosely:
>> $ curl -v --proxy https://my-proxy-server.example:3129 \
>> --proxy-anyauth --proxy-user redacted:redacted -w '\n' \
>> http://urlecho.appspot.com/echo?body=OK
>> * Adding handle: conn: 0x7f9a30804000
>> * Adding handle: send: 0
>> * Adding handle: recv: 0
>> * Curl_addHandleToPipeline: length: 1
>> * - Conn 0 (0x7f9a30804000) send_pipe: 1, recv_pipe: 0
>> * About to connect() to proxy my-proxy-server.example port 3129 (#0)
>> * Trying proxy.server.IP.redacted...
>> * Connected to my-proxy-server.example (proxy.server.IP.redacted)
>> port 3129 (#0)
>
> Note the absence of any mention about SSL/TLS here.

I did notice this and wondered if it might be a problem with curl itself. So I also tried similar tests with Google Chrome and a Ruby HTTP library called excon, both of which specifically mention support of HTTPS proxies. I also tried a few other HTTP libraries that have HTTP proxy support but don’t specifically mention HTTPS. Since I saw the same failing result with all of them, I went back to trying to troubleshoot Squid as the likely source of the problem.

>> > GET http://urlecho.appspot.com/echo?body=OK HTTP/1.1
>> > User-Agent: curl/7.30.0
>> > Host: urlecho.appspot.com
>> > Accept: */*
>> > Proxy-Connection: Keep-Alive
>> >
>> * Recv failure: Connection reset by peer
>> * Closing connection 0
>> curl: (56) Recv failure: Connection reset by peer
>> Looks like an SSL error.
>
> It is. Try using --ssl option for curl. You will also need --proxy-digest to get the Digest auth against the proxy working for curl.

--proxy-anyauth works fine with HTTP, as in my first example. It correctly detects that it needs to use Digest auth.

It seems like --ssl refers to the inner HTTP request rather than the proxy, but the docs are somewhat unclear.

Regardless, I tried with --ssl --proxy-digest just now, and it failed with the same error.

>> However, I'm reusing a subdomain-wildcard SSL
>> certificate, shown in the above config as `cert.pem` and `key.pem`,
>> that I've successfully deployed on other web servers. Moreover,
>> accessing the proxy server directly with curl works, or at least
>> establishes a connection past the SSL stage:
>> $ curl https://my-proxy-server.example:3129
>> <!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01//EN"
>> "http://www.w3.org/TR/html4/strict.dtd">
>> [--SNIP--]
>
> This tells us that the test machine can access the web serve. Please run the same test directly on the squid machine. That will tell whether or not connectivity for Squid is working properly.

There is no web server running on that machine other than Squid. The request in this example was to Squid directly. Note the (bowdlerized) hostname and port which match the proxy in the above examples. It was meant to show that there doesn’t seem to be a problem with the SSL certificate I have configured with Squid.

>> Any ideas what I'm doing wrong? Is what I'm attempting even possible?
>
> It is possible and being done already by some. Although to get the broken browsers to work properly a VPN/stunnel from the client machine to proxy port is often needed.

I am not particularly concerned about browsers since I am accessing the proxy programmatically from a server. I only need an HTTP library that can talk to Squid correctly.

I take it, then, that you don’t see any obvious problems with my Squid config for what I’m trying to do?

Thanks again,
David
Received on Tue Jan 21 2014 - 22:44:59 MST