Squid HTTP Proxy Henrik Nordström Open Source Consultant Squid developer

Todays topic ● Who is Henrik? ● What is Squid? ● Squid usage at ISPs ● Squid corporate usage ● Squid for Content Publishers ● The Squid Project

Who is Henrik? ● Independent Open Source consultant ● Active Squid developer ● Working on Squid since 1996 ● Have also been seen in – Linux Netfilter/iptables – User Mode Linux – And many other projects

What is Squid? ● HTTP Proxy Cache ● Supports HTTP, FTP, Gopher and whois. For HTTP clients. ● Internet proxy ● Reverse proxy/accelerator ● Access control

What Squid is not ● Squid is not a firewall ● Squid is not a filter ● Squid is not a generic proxy for non-HTTP protocols.

Benefits of HTTP caching ● Reduced bandwidth usage ● Improved latency ● Reduces cost ● Scales performance

Caching within HTTP ● Defined by the HTTP specicications (RFC2616) ● Browser caches ● Shared caches ● Controlled by the content publisher ● Heuristic model ● If-Modified-Since

Cache tuning ● Explicit expiry ● Heuristic model, last-modified ● Default 20% max 72 hours ● Tuned on URL patterns

Internet Proxy ● Provides web access ● Delegation by user/browser ● Used by ISPs ● Corporations ● And some home users

ISP Usage ● Reduce bandwidth usage ● Improve latency ● Internet quality/cost still problematic in many parts of the world ● 50% hit ratio, 30% byte

Corporate Usage ● Audit trail ● Access controls ● Filtering ● Virus scanning (third party) – Via scanner proxy – Offloaded using ICAP ● Authentication ● Active Directory integration

ICAP ● Internet Contend Adaptation Protocol (RFC3507) ● Generic ● HTTP ”only” ● Mostly virus scanners

Cache busting ● Content providers not playing well ● Everything uncachable/private ● Random URLs ● Etc. ● Education needed. ● Caching Tutorial for Web authors and Webmasters, mnot.net

Reverse proxy / accelerator ● Delegation by the webauthor/webmaster. ● Official web ”server” ● Easily scale performance ● Peaks removed, almost constant load ● SSL Offload ● ESI ● Persistent connections ● Typically 95%-99% hit ratio

Persistent connections ● Increases browsing performance ● Resource intensive for many web servers ● Easily maintained by Squid

SSL Offload ● Moves SSL encryption to the accelerator ● Easier scaling ● Less need for special crypto hardware ● Application awareness

ESI ● Edge Side Includes ● Delegate page composing ● Simple XML based language ● Akamai, Oracle, IBM and others ● Semi-dynamic content

Efficient use ● Design with caching in mind ● Static / dynamic ● Avoid server side dynamic composing ● Use client capabilities / DOM ● Unique URLs for unique content

Example: Wikipedia ● Very large web site ● 50K hits/s, 4Mbps traffic ● 97% hit ratio ● Soon 90 Squid servers (75 today) ● Intelligent request roting (2 level CARP) ● Automatic cache updates (HTCP)

The Squid Project ● Started with an NSF grant ● Run by volonteers ● Paid contracts ● About 6-7 active ● Several minor contributors

Current status ● Squid-2.6, maintenance ● Squid-3.0, this week

Why Squid-3.0 ● Major code restructuring ● Easier maintenance ● ICAP support ● ESI support

Ongoing projects ● Increased performance ● Tools to mitigate cache busting effects ● IPv6 ● HTTP/1.1 ● Inline ICAP (eCAP) ● SSL Interception ● Further internal cleanups

HTTP/1.1 ● Basic support for Squid-2 ● Full support planned for 3.1

SSL Interception ● Man-in-the-middle attack on SSL ● Corporate policy denying encrypted traffic ● Faked trust delegation ● Controlled environment

Mitigation of CDN cache busting ● Youtube and others ● Probably unintentional ● Same object, many URLs ● Custom rules per CDN ● Available in Squid-2.HEAD ● Squid 2.7 and 3.1.

Future goals ● Easier configuration ● Much better performance

Questions?