Re: [squid-users] Reverse Proxy

From: Mario Remy Almeida <malmeida_at_isaaviation.ae>
Date: Sun, 17 May 2009 18:24:22 +0400

Thanks Amos,

Finally got it working.

Once again thanks for all the support.

Any idea where to start for scanning of https sites I mean documentation

//Remy

On Mon, 2009-05-18 at 02:04 +1200, Amos Jeffries wrote:
> Mario Remy Almeida wrote:
> > Hi Amos,
> >
> > Thanks for the configuration I managed to access http and https
> > (mail.airarabia.ae)
> >
> > webmail.airarabia.ae is discarded.
> >
> > now one more issue
> >
> > Any external sites http I can access but not https
> > example https://gmail.com not accessable
> >
> > access.log file I get
> > =======================================
> > 1242580515.608 0 10.200.2.172 TCP_DENIED/400 1570 CONNECT :0 -
> > NONE/- text/html
> > 1242580517.224 0 10.200.2.172 TCP_DENIED/400 1570 CONNECT :0 -
> > NONE/- text/html
> > 1242580536.539 0 10.200.2.172 TCP_DENIED/400 1570 CONNECT :0 -
> > NONE/- text/html
> > 1242580538.999 0 10.200.2.172 TCP_DENIED/400 1570 CONNECT :0 -
> > NONE/- text/html
> >
> >
> > browser I get
> > ==================================
> > While trying to process the request:
> > CONNECT www.google.com:443 HTTP/1.0
> > User-Agent: Opera/9.64 (X11; Linux i686; U; en) Presto/2.1.1
> > Host: www.google.com:443
> >
> >
> >
> > The following error was encountered:
> > Invalid Request
> >
> > Some aspect of the HTTP Request is invalid. Possible problems:
> > Missing or unknown request method
> > Missing URL
> > Missing HTTP Identifier (HTTP/1.0)
> > Request is too large
> > Content-Length missing for POST or PUT requests
> > Illegal character in hostname; underscores are not allowed
> >
>
> I think you are trying to use a reverse-proxy port (as configured below)
> as a forward-proxy (general web requests).
>
> The accel ports we setup below for OWA is not applicable for general web
> access. To use is for general access you need to setup a basic
> "http_port 3128" and configure that in the client browsers.
>
> Amos
>
> >
> > My squid.conf is as below
> > ========================================
> > acl all src all
> > acl manager proto cache_object
> > acl localhost src 127.0.0.1/32
> > acl to_localhost dst 127.0.0.0/8
> > acl SSL_ports port 443
> > acl Safe_ports port 80 # http
> > acl Safe_ports port 21 # ftp
> > acl Safe_ports port 443 # https
> > acl Safe_ports port 70 # gopher
> > acl Safe_ports port 210 # wais
> > acl Safe_ports port 1025-65535 # unregistered ports
> > acl Safe_ports port 280 # http-mgmt
> > acl Safe_ports port 488 # gss-http
> > acl Safe_ports port 591 # filemaker
> > acl Safe_ports port 777 # multiling http
> > acl CONNECT method CONNECT
> > acl localnet src 10.200.2.0/24
> > acl snmppublic snmp_community public
> > acl OWA dstdomain mail.airarabia.ae
> > http_access allow manager localhost
> > http_access deny manager
> > http_access deny !Safe_ports
> > http_access allow OWA all
> > http_access deny CONNECT !SSL_ports
> > http_access allow localnet
> > http_access allow localhost
> > http_access deny all
> > icp_access allow localnet
> > icp_access deny all
> > reply_body_max_size 52428800 allow all
> > follow_x_forwarded_for allow localnet
> > follow_x_forwarded_for allow localhost
> > follow_x_forwarded_for deny all
> > acl_uses_indirect_client on
> > delay_pool_uses_indirect_client on
> > log_uses_indirect_client on
> > ssl_unclean_shutdown on
> > http_port 10.200.22.49:80 accel defaultsite=mail.airarabia.ae vhost
> > https_port 10.200.22.49:443 accel cert=/etc/squid/keys/proxycert.pem
> > key=/etc/squid/keys/proxykey.pem defaultsite=mail.airarabia.ae
> > cache_peer 10.200.22.12 parent 80 0 no-query originserver login=PASS
> > front-end-https=on login=PASS name=owaServer
> > cache_peer proxy1.emirates.net.ae parent 8080 0 no-query default
> > cache_peer_access owaServer allow OWA
> > cache_peer_access proxy1.emirates.net.ae allow !OWA
> > hierarchy_stoplist cgi-bin ?
> > cache_mem 600 MB
> > maximum_object_size_in_memory 20 KB
> > memory_replacement_policy heap GDSF
> > cache_replacement_policy heap GDSF
> > cache_dir aufs /cache 29000 16 256
> > store_dir_select_algorithm least-load
> > max_open_disk_fds 0
> > minimum_object_size 0 KB
> > maximum_object_size 1096 MB
> > cache_swap_low 90
> > cache_swap_high 95
> > logformat squid %ts.%03tu %6tr %>a %Ss/%03Hs %<st %rm %ru %un %Sh/%<A %
> > mt
> > logformat mysql_columns %ts.%03tu %6tr %>a %Ss %03Hs %<st %rm %ru %un %
> > Sh %<A %mt
> > access_log /var/log/squid/access.log squid
> > access_log daemon:/usr/lib64/squid/db.cf mysql_columns
> > logfile_daemon /usr/lib64/squid/logmysqldb_daemon
> > cache_log /var/log/squid/cache.log
> > cache_store_log /var/log/squid/store.log
> > logfile_rotate 30
> > emulate_httpd_log on
> > log_ip_on_direct on
> > mime_table /etc/squid/mime.conf
> > log_mime_hdrs on
> > useragent_log /var/log/squid/useragent.lo
> > referer_log /var/log/squid/referer.log
> > pid_filename /var/run/squid.pid
> > debug_options ALL,1
> > log_fqdn off
> > strip_query_terms on
> > buffered_logs off
> > netdb_filename /var/log/squid/netdb.state
> > ftp_list_width 64
> > ftp_passive on
> > ftp_sanitycheck on
> > ftp_telnet_protocol on
> > diskd_program /usr/lib64/squid/diskd-daemon
> > unlinkd_program /usr/lib64/squid/unlinkd
> >
> > pinger_program /usr/lib64/squid/pinger
> > refresh_pattern ^ftp: 1440 20% 10080
> > refresh_pattern ^gopher: 1440 0% 1440
> > refresh_pattern -i (/cgi-bin/|\?) 0 0% 0
> > refresh_pattern . 0 20% 4320
> > read_ahead_gap 16 KB
> > negative_ttl 2 minutes
> > positive_dns_ttl 9 hours
> > negative_dns_ttl 1 minute
> > minimum_expiry_time 30 seconds
> > store_objects_per_bucket 15
> > request_header_max_size 20 KB
> > reply_header_max_size 25 KB
> > request_body_max_size 50 MB
> > acl shoutcast rep_header X-HTTP09-First-Line ^ICY.[0-9]
> > upgrade_http0.9 deny shoutcast
> > cache_vary on
> > acl apache rep_header Server ^Apache
> > broken_vary_encoding allow apache
> > collapsed_forwarding off
> > extension_methods RPC_IN_DATA RPC_OUT_DATA
> > shutdown_lifetime 30 seconds
> > cache_mgr Rusol <rskender_at_airarabia.com>
> > mail_from Rusol <rskender_at_airarabia.com>
> > mail_program mail
> > cache_effective_user squid
> > cache_effective_group squid
> > httpd_suppress_version_string on
> > visible_hostname vsquid-01-shj
> > umask 027
> > snmp_port 3401
> > snmp_access allow snmppublic localhost
> > snmp_access deny all
> > icon_directory /usr/share/squid/icons
> > global_internal_static on
> > short_icon_urls on
> > nonhierarchical_direct on
> > prefer_direct off
> > never_direct allow OWA
> > max_filedescriptors 0
> > check_hostnames off
> > allow_underscore on
> > dns_timeout 2 minutes
> > hosts_file /etc/hosts
> > ignore_unknown_nameservers on
> > ipcache_size 2048
> > ipcache_low 90
> > ipcache_high 95
> > fqdncache_size 1024
> > forwarded_for on
> > cachemgr_passwd disable all
> > client_db off
> > uri_whitespace strip
> > coredump_dir /var/spool/squid
> > windows_ipaddrchangemonitor off
> >
> >
> > Thanks for the help
> >
> > //Remy
> >
> > On Mon, 2009-05-18 at 00:57 +1200, Amos Jeffries wrote:
> >> Mario Remy Almeida wrote:
> >>> My squid.conf
> >>>
> >>> acl all src all
> >>> acl manager proto cache_object
> >>> acl localhost src 127.0.0.1/32
> >>> acl to_localhost dst 127.0.0.0/8
> >>> acl SSL_ports port 443
> >>> acl Safe_ports port 80 # http
> >>> acl Safe_ports port 21 # ftp
> >>> acl Safe_ports port 443 # https
> >>> acl Safe_ports port 70 # gopher
> >>> acl Safe_ports port 210 # wais
> >>> acl Safe_ports port 1025-65535 # unregistered ports
> >>> acl Safe_ports port 280 # http-mgmt
> >>> acl Safe_ports port 488 # gss-http
> >>> acl Safe_ports port 591 # filemaker
> >>> acl Safe_ports port 777 # multiling http
> >>> acl CONNECT method CONNECT
> >>> acl localnet src 10.200.2.0/24
> >>> acl OWA dstdomain webmail.airarabia.ae
> >>> http_access allow manager localhost
> >>> http_access deny manager
> >>> http_access deny !Safe_ports
> >>> http_access deny CONNECT !SSL_ports
> >>> http_access allow OWA all
> >>> http_access allow localnet
> >>> http_access allow localnet
> >>> http_access allow localhost
> >>> http_access deny all
> >>> icp_access allow localnet
> >>> icp_access deny all
> >>> miss_access allow OWA
> >>> miss_access deny all
> >>> http_port 10.200.22.49:80 defaultsite=webmail.airarabia.ae
> >>> https_port 10.200.22.49:443 defaultsite=webmail.airarabia.ae
> >>> cert=/etc/squid/keys/proxycert.pem key=/etc/squid/keys/proxykey.pem
> >>> cache_peer 10.200.22.12 parent 80 0 no-query originserver login=PASS
> >>> front-end-https=on login=PASS name=owaServer
> >>> cache_peer proxy1.emirates.net.ae parent 8080 0 no-query default
> >>> cache_peer_access owaServer allow OWA
> >>> hierarchy_stoplist cgi-bin ?
> >>> cache_dir aufs /cache 29000 16 256
> >>> logformat squid %ts.%03tu %6tr %>a %Ss/%03Hs %<st %rm %ru %un %Sh/%<A %mt
> >>> logformat mysql_columns %ts.%03tu %6tr %>a %Ss %03Hs %<st %rm %ru %un
> >>> %Sh %<A %mt
> >>> access_log /var/log/squid/access.log squid
> >>> access_log daemon:/usr/lib64/squid/db.cf mysql_columns
> >>> logfile_daemon /usr/lib64/squid/logmysqldb_daemon
> >>> pid_filename /var/run/squid.pid
> >>> refresh_pattern ^ftp: 1440 20% 10080
> >>> refresh_pattern ^gopher: 1440 0% 1440
> >>> refresh_pattern -i (/cgi-bin/|\?) 0 0% 0
> >>> refresh_pattern . 0 20% 4320
> >>> acl shoutcast rep_header X-HTTP09-First-Line ^ICY.[0-9]
> >>> upgrade_http0.9 deny shoutcast
> >>> acl apache rep_header Server ^Apache
> >>> broken_vary_encoding allow apache
> >>> prefer_direct off
> >>> never_direct allow OWA
> >>> coredump_dir /var/spool/squid
> >>>
> >>>
> >>> OUTPUT of "host webmail.airarabia.ae" taking from DNS
> >>> webmail.airarabia.ae has address 10.200.22.12
> >>>
> >>>
> >>> clients browser
> >>> proxy set to 10.200.22.49 port 80
> >>> NO by-pass
> >>>
> >>> Now confused with DNS what should be the DNS entires.
> >>>
> >>> the clients will not by-pass.
> >>>
> >>> should the DNS entry point to the OWA IP or to Squid Proxy?
> >>>
> >>>
> >>> Please help as I am confused.
> >>>
> >> Oh, I see...
> >>
> >> You need this:
> >>
> >> 10.200.22.49 -> SquidProxy
> >> 10.200.22.12 -> OWA
> >> 10.200.2.22 -> DNS Server
> >>
> >> DNS Entires,
> >> webmail.airarabia.com pointing to 10.200.22.49 (HTTP, HTTPS stuff)
> >> mail.airarabia.com pointing to 10.200.22.12 (SMTP stuff)
> >>
> >> On Squid Proxy Server,
> >>
> >> /etc/resolv.conf:
> >> nameserver 10.200.2.22
> >>
> >> /etc/hosts:
> >> 127.0.0.1 localhost
> >>
> >> squid.conf as above but:
> >>
> >> http_port 10.200.22.49:80 accel defaultsite=webmail.airarabia.ae
> >> https_port 10.200.22.49:443 accel defaultsite=webmail.airarabia.ae \
> >> cert=/etc/squid/keys/proxycert.pem key=/etc/squid/keys/proxykey.pem
> >>
> >> cache_peer 10.200.22.12 parent 80 0 no-query originserver login=PASS \
> >> front-end-https=on name=owaServer
> >> cache_peer_access owaServer allow OWA
> >>
> >> cache_peer proxy1.emirates.net.ae parent 8080 0 no-query default
> >> cache_peer_access proxy1.emirates.net.ae allow !OWA
> >>
> >>
> >>
> >> NOTE the 'accel' option on ports and "!OWA" on default parent peer access.
> >>
> >> Amos
> >>
> >>
> >>> //Remy
> >>>
> >>> On Sun, 2009-05-17 at 19:33 +1200, Amos Jeffries wrote:
> >>>> Mario Remy Almeida wrote:
> >>>>> Hi Amos,
> >>>>>
> >>>>> One thing I forgot to mentioned
> >>>>>
> >>>>> /etc/hosts has this entry
> >>>>> 10.200.22.12 mail.airarabia.ae
> >>>>>
> >>>>> Output of " host mail.airarabia.ae " from dns is ->
> >>>>> mail.airarabia.ae has address 10.200.9.20
> >>>>>
> >>>>>
> >>>>> User (browser) reads the host file from individual PCs
> >>>>> cat /etc/hosts | grep "mail.airarabia.ae"
> >>>>> 10.200.22.49 mail.airarabia.ae
> >>>>>
> >>>>>
> >>>>> 10.200.22.49 <- squid proxy ip
> >>>>> 10.200.22.12 <- OWA ip
> >>>> This could cause you some problems administering it.
> >>>>
> >>>> My advice on this is to setup DNS pointing at Squid for the HTTPS domain
> >>>> name, set squid.conf with the right OWA IP as a peer, and not have the
> >>>> individual hosts file overrides.
> >>>>
> >>>> The fact that the public IP for the domain is different to both the
> >>>> squid IP and the real OWA/Exchange IP is worrying. I trust that you know
> >>>> what destinations should be.
> >>>>
> >>>> Amos
> >>>>
> >>>>> Please find the answers below.
> >>>>>
> >>>>> //Remy
> >>>>>
> >>>>> On Sun, 2009-05-17 at 18:16 +1200, Amos Jeffries wrote:
> >>>>>> Mario Remy Almeida wrote:
> >>>>>>> Hi Amos,
> >>>>>>>
> >>>>>>> I followed the instruction as per
> >>>>>>> http://wiki.squid-cache.org/ConfigExamples/Reverse/OutlookWebAccess
> >>>>>>>
> >>>>>>> But I am some how failing to configure https.
> >>>>>>>
> >>>>>>> My squid.conf
> >>>>>>> ========================================================================
> >>>>>>> https_port 443 defaultsite=mail.airarabia.ae \
> >>>>>>> cert=/etc/squid/keys/cert.pem key=/etc/squid/keys/key.pem
> >>>>>> Okay two extra things about the port:
> >>>>>> 1) unless you have the wilcard cert its best to specify the IP:port
> >>>>>> combo and generate the cert for those IP:port. That way you can use
> >>>>>> other IP for other domains and be sure Squid is sending SSL on the right IP.
> >>>>> changed it to ->
> >>>>> https_port 10.200.22.49:443 defaultsite=mail.airarabia.ae \
> >>>>> cert=/etc/squid/keys/cert.pem key=/etc/squid/keys/key.pem
> >>>>>
> >>>>>> 2) check that the cert/key are correct for the IP:port squid is
> >>>>>> listening on.
> >>>>> use this command to generate the ssl certificate
> >>>>>
> >>>>> openssl req -x509 -days 365 -newkey rsa:1024 -keyout key.pem -nodes
> >>>>> \-out cert.pem
> >>>>>
> >>>> The keys do need to be signed in some way before they are valid for use.
> >>>> This looks like a key creation-only command, though with SSL certs I
> >>>> only know enough to follow the tutorials. Doing that (for all key steps)
> >>>> I've never had a problem.
> >>>>
> >>>> Amos
> >>>>
> >>>>>>> cache_peer 10.200.22.12 parent 80 0 no-query originserver login=PASS \
> >>>>>>> front-end-https=on login=PASS name=owaServer
> >>>>>> So OWA is listening on port 80?
> >>>>> yes on port 80 no issue
> >>>>>
> >>>>>>> cache_peer_access owaServer allow OWA
> >>>>>>> acl OWA dstdomain mail.airarabia.ae
> >>>>>>> http_access allow OWA
> >>>>>>> miss_access allow OWA
> >>>>>>> miss_access deny all
> >>>>>> Missing:
> >>>>>> never_direct allow OWA
> >>>>> Actually I forgot to mention it here
> >>>>> It is specified in squid.conf
> >>>>>
> >>>>>> that bit is important to prevent Squid even attempting to request a
> >>>>>> connection direct to OWA without the peerage settings.
> >>>>>>
> >>>>>> Amos
> >>>>>>
> >>>>>>> cache.log
> >>>>>>> ========================================================================
> >>>>>>> 2009/05/17 13:32:12| fwdNegotiateSSL: Error negotiating SSL connection \
> >>>>>>> on FD 24: error:00000000:lib(0):func(0):reason(0) (5/-1/104)
> >>>>>>> 2009/05/17 13:32:12| fwdNegotiateSSL: Error negotiating SSL connection \
> >>>>>>> on FD 24: error:00000000:lib(0):func(0):reason(0) (5/-1/104)
> >>>>>>> 2009/05/17 13:32:13| fwdNegotiateSSL: Error negotiating SSL connection \
> >>>>>>> on FD 24: error:00000000:lib(0):func(0):reason(0) (5/-1/104)
> >>>>>>>
> >>>>>>> Error on the browser
> >>>>>>> ========================================================================
> >>>>>>> While trying to retrieve the URL: https://mail.airarabia.ae/exchweb/
> >>>>>>>
> >>>>>>> The following error was encountered:
> >>>>>>>
> >>>>>>> * Connection to 10.200.22.12 Failed
> >>>>>>>
> >>>>>>> The system returned:
> >>>>>>>
> >>>>>>> (71) Protocol error
> >>>>>>>
> >>>>>>> The remote host or network may be down. Please try the request again.
> >>>>>>>
> >>>>>>>
> >>>>>>> Please help
> >>>>>>>
> >>>>>>> //Remy
> >>>>>>>
> >>>>>>>
> >>>>>>> On Fri, 2009-05-15 at 16:35 +1200, Amos Jeffries wrote:
> >>>>>>>> Mario Remy Almeida wrote:
> >>>>>>>>> Hi All,
> >>>>>>>>>
> >>>>>>>>> Need to setup Reverse proxy
> >>>>>>>>>
> >>>>>>>>> I have
> >>>>>>>>>
> >>>>>>>>> Squid 2.7STABLE6
> >>>>>>>>> OS Centos
> >>>>>>>>>
> >>>>>>>>> Web server= Microsoft Outlook Web Access
> >>>>>>>>> SSL enabled
> >>>>>>>>> port 443
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> My squid config is as below
> >>>>>>>>>
> >>>>>>>>> acl vhosts1_domains dstdomain mail.airarabiauae.com
> >>>>>>>>> http_port 443 accel defaultsite=mail.airarabiauae.com vhost
> >>>>>>>>> cache_peer 10.200.22.12 parent 443 0 no-query originserver name=vhost1 \
> >>>>>>>>> ssl
> >>>>>>>>> cache_peer_access vhost1 allow vhosts1_domains
> >>>>>>>>>
> >>>>>>>>> Please someone tell me it that is the right way to configure it.
> >>>>>>>>>
> >>>>>>>> No. Here is the tutorial:
> >>>>>>>>
> >>>>>>>> http://wiki.squid-cache.org/ConfigExamples/Reverse/OutlookWebAccess
> >>>>>>>>
> >>>>>>>> port 443 is often encrypted. It requires the https_port option instead
> >>>>>>>> of http_port, and the certificate as well.
> >>>>>>>>
> >>>>>>>> The peer part may be correct, or further ssl-related options may be
> >>>>>>>> needed. It depends on your peer so I can't say for certain unless you
> >>>>>>>> actually hit a problem.
> >>>>>>>>
> >>>>>>>>
> >> Amos
> >
>
> --
> Please be using
> Current Stable Squid 2.7.STABLE6 or 3.0.STABLE15
> Current Beta Squid 3.1.0.7

-- 
Mario Remy Almeida
Linux System Administrator
ISA
O: 06588817
M: 0508643912
E: malmeida_at_isaaviation.ae
------------------------------------------------------------------------------
Disclaimer and Confidentiality
This material has been checked for  computer viruses and although none has
been found, we cannot guarantee  that it is completely free from such problems
and do not accept any  liability for loss or damage which may be caused.
Please therefore  check any attachments for viruses before using them on your
own  equipment. If you do find a computer virus please inform us immediately
so that we may take appropriate action. This communication is intended  solely
for the addressee and is confidential. If you are not the intended recipient,
any disclosure, copying, distribution or any action  taken or omitted to be
taken in reliance on it, is prohibited and may be  unlawful. The views
expressed in this message are those of the  individual sender, and may not
necessarily be that of ISA.
Received on Sun May 17 2009 - 14:24:57 MDT

This archive was generated by hypermail 2.2.0 : Mon May 18 2009 - 12:00:02 MDT