install squid with ecap gzip adapter

[l0ngb1t]

been playing around with squid also the ecap gzip adapter, for those who don't know, squid is proxy/cache server and ecap-gzip is an adapter for squid that does http compression.

most modern browsers support compression such as gzip and deflate, some webserver send the page compressed and the browser decompress it, if you look into the http request header you will see "Accept-Encoding: gzip,deflate" this is the browser telling the webserver that it support compression, send me the page compressed if possible.
in some http REPLY header you may see: "Content-Encoding: gzip"
it indicates that the page was sent from the web server compressed.

it is one of the wan optimization trends. caching along with compression can be very helpful for ISP's.

why use it?
there are still many web server that does not support compression so why not compress it, with this setup it will not reduce the isp international traffic and for sure the internal traffic. but caching will reduce the international traffic for sure (traffic between isp and the internet)

so here is a fast how-to, it cover installation of squid with the ecap-gzip adapter step by step.

OS: Centos
Make sure the following packages are installed :

Code: Select all

yum install gcc
yum install gcc-c++
yum install libcap-devel
yum install openssl-devel
yum install openldap-devel
yum install make 

install the libecap library:

Code: Select all

wget http://www.measurement-factory.com/tmp/ecap/libecap-0.0.3.tar.gz
tar xvf libecap-0.0.3.tar.gz
cd libecap-0.0.3/
./configure
make
make install

Download and install VIGOS eCAP GZIP Adapter:

Code: Select all

wget http://squid-ecap-gzip.googlecode.com/files/squid-ecap-gzip-1.3.0.tar.gz
tar xvf squid-ecap-gzip-1.3.0.tar.gz

eCAP GZIP only compress files with mime type text/html leaving all .js, .css , .xml… uncompressed, we can modify the adapter and enable it to compress all text/* files.
Open squid-ecap-gzip/src/adapter_gzip.cc with a text editor, at line 363 replace the following:

Code: Select all

if(contentType.size > 0) {
	std::string contentTypeString = contentType.toString(); // expensive

	if(strstr(contentTypeString.c_str(),"text/html")) {
		this->requirements.responseContentTypeOk = true;
	}
}

With the following:

Code: Select all

if(contentType.size > 0) {
    std::string contentTypeString = contentType.toString(); // expensive
    // extract the first 5 characters of the content type.
    std::string contentTypeText   = contentTypeString.substr(0,5);    
    // check if the content type is equal to text/, this should cover text/* 
    if(strstr(contentTypeText.c_str(),"text/")) { 
        this->requirements.responseContentTypeOk = true;
    }
}

Now proceed with the installation

Code: Select all

cd squid-ecap-gzip/
./configure
make
make install

Download and install SQUID proxy cache, version 3.1+

Code: Select all

wget http://www.squid-cache.org/Versions/v3/3.1/squid-3.1.11.tar.gz
tar xvfz squid-3.1.11.tar.gz
cd squid-3.1.11/
#with configure you only need --enable-ecap option to be able to add ecap-gzip but i need all the below
./configure '--prefix=/usr' '--exec-prefix=/usr' '--bindir=/usr/bin' '--sbindir=/usr/sbin' '--sysconfdir=/etc' '--datadir=/usr/share' '--includedir=/usr/include' '--libdir=/usr/lib64' '--libexecdir=/usr/libexec' '--sharedstatedir=/var/lib' '--mandir=/usr/share/man' '--infodir=/usr/share/info' '--exec_prefix=/usr' '--libexecdir=/usr/lib64/squid' '--localstatedir=/var' '--datadir=/usr/share/squid' '--sysconfdir=/etc/squid' '--with-logdir=/var/log/squid' '--with-pidfile=/var/run/squid.pid' '--disable-dependency-tracking' '--enable-arp-acl' '--enable-follow-x-forwarded-for' '--enable-auth=basic,digest,negotiate' '--enable-external-acl-helpers=ip_user,unix_group,wbinfo_group' '--enable-cache-digests' '--enable-cachemgr-hostname=localhost' '--enable-delay-pools' '--enable-epoll' '--enable-icap-client' '--enable-ident-lookups' '--enable-linux-netfilter' '--enable-referer-log' '--enable-snmp' '--enable-ssl' '--enable-storeio=aufs,ufs' '--enable-wccpv2' '--enable-esi' '--with-aio' '--with-default-user=proxy' '--with-filedescriptors=65536' '--with-dl' '--with-pthreads' '--with-libcap' '--with-netfilter-conntrack' '--with-openssl' '--enable-inline' '--enable-uselect' '--enable-disk-io' '--disable-htcp' '--with-gnu-ld' '--with-build-environment=default' '--enable-carp' '--enable-ecap' '--enable-linux-netfilter'

make

make install

Configure SQUID to use VIGOS eCAP GZIP Adapter
Add or edit the following lines in your /etc/squid/squid.conf:

Code: Select all

ecap_enable on
ecap_service gzip_service respmod_precache 0 ecap://www.vigos.com/ecap_gzip
loadable_modules /usr/local/lib/ecap_adapter_gzip.so
acl GZIP_HTTP_STATUS http_status 200
adaptation_access gzip_service allow GZIP_HTTP_STATUS

the following command should start squid with prompt mode:

Code: Select all

/usr/sbin/squid -NCd1

You may face some errors, the following indicate them along with the fix for each error.

Error:

squid: error while loading shared libraries: libecap.so.0: cannot open shared object file: No such file or directory

Fix:
Add the following: /usr/local/lib to: /etc/ld.so.conf then run the command “ldconfig”


Error:

2013/09/06 14:58:01| WARNING: Could not determine this machines public hostname. Please configure one or set 'visible_hostname'.

Fix:
Add the visible_hostname SERVER_HOST_NAME to /et c/squid/squid.conf

Error:

FATAL: getpwnam failed to find userid for effective user 'proxy'
Squid Cache (Version 3.1.11): Terminated abnormally.
CPU Usage: 0.015 seconds = 0.004 user + 0.011 sys
Maximum Resident Size: 18000 KB
Page faults with physical i/o: 0

Fix:
Create user called proxy: useradd proxy


Error:

Code: Select all

FATAL: Cannot open '/var/log/squid/access.log' for writing.
        The parent directory must be writeable by the
        user 'proxy', which is the cache_effective_user
        set in squid.conf.

Fix:
Create the file /var/log/squid/access.log, give owner ship to user proxy over it along with read/write permissions.

Code: Select all

> touch /var/log/squid/access.log
> chmod -R r+uX /var/log/squid/access.log
>  chmod -R u+rX /var/log/squid/access.log

And finally create a swap file for squid with following command:

Code: Select all

	touch /var/log/squid/cache.log 
        chown -R proxy /var/log/squid/cache.log 
	chmod -R u+rX /var/log/squid/cache.log
	/usr/sbin/squid –z

Allow your network to use the proxy by adding the following configuration lines to /etc/squid/squid.conf
Acl ACL_NAME src YOUR_NETWORK/SUBNET_PREFIX
http_access allow ACL_NAME
P.S there in an http_access deny all command in the configuration file, make sure to the above line before it.

now i am trying to configure squid to run as a TPROXY, the page requested by squid will have the client ip and not the squid ip as source, may come in handy in some situation

enjoy it

[bad_brain]

good work, stickyfied!