most modern browsers support compression such as gzip and deflate, some webserver send the page compressed and the browser decompress it, if you look into the http request header you will see "Accept-Encoding: gzip,deflate" this is the browser telling the webserver that it support compression, send me the page compressed if possible.
in some http REPLY header you may see: "Content-Encoding: gzip"
it indicates that the page was sent from the web server compressed.
it is one of the wan optimization trends. caching along with compression can be very helpful for ISP's.
why use it?
there are still many web server that does not support compression so why not compress it, with this setup it will not reduce the isp international traffic and for sure the internal traffic. but caching will reduce the international traffic for sure (traffic between isp and the internet)
so here is a fast how-to, it cover installation of squid with the ecap-gzip adapter step by step.
OS: Centos
Make sure the following packages are installed :
Code: Select all
yum install gcc
yum install gcc-c++
yum install libcap-devel
yum install openssl-devel
yum install openldap-devel
yum install make
Code: Select all
wget http://www.measurement-factory.com/tmp/ecap/libecap-0.0.3.tar.gz
tar xvf libecap-0.0.3.tar.gz
cd libecap-0.0.3/
./configure
make
make install
Download and install VIGOS eCAP GZIP Adapter:
Code: Select all
wget http://squid-ecap-gzip.googlecode.com/files/squid-ecap-gzip-1.3.0.tar.gz
tar xvf squid-ecap-gzip-1.3.0.tar.gz
Open squid-ecap-gzip/src/adapter_gzip.cc with a text editor, at line 363 replace the following:
Code: Select all
if(contentType.size > 0) {
std::string contentTypeString = contentType.toString(); // expensive
if(strstr(contentTypeString.c_str(),"text/html")) {
this->requirements.responseContentTypeOk = true;
}
}
Code: Select all
if(contentType.size > 0) {
std::string contentTypeString = contentType.toString(); // expensive
// extract the first 5 characters of the content type.
std::string contentTypeText = contentTypeString.substr(0,5);
// check if the content type is equal to text/, this should cover text/*
if(strstr(contentTypeText.c_str(),"text/")) {
this->requirements.responseContentTypeOk = true;
}
}
Now proceed with the installation
Code: Select all
cd squid-ecap-gzip/
./configure
make
make install
Download and install SQUID proxy cache, version 3.1+
Code: Select all
wget http://www.squid-cache.org/Versions/v3/3.1/squid-3.1.11.tar.gz
tar xvfz squid-3.1.11.tar.gz
cd squid-3.1.11/
#with configure you only need --enable-ecap option to be able to add ecap-gzip but i need all the below
./configure '--prefix=/usr' '--exec-prefix=/usr' '--bindir=/usr/bin' '--sbindir=/usr/sbin' '--sysconfdir=/etc' '--datadir=/usr/share' '--includedir=/usr/include' '--libdir=/usr/lib64' '--libexecdir=/usr/libexec' '--sharedstatedir=/var/lib' '--mandir=/usr/share/man' '--infodir=/usr/share/info' '--exec_prefix=/usr' '--libexecdir=/usr/lib64/squid' '--localstatedir=/var' '--datadir=/usr/share/squid' '--sysconfdir=/etc/squid' '--with-logdir=/var/log/squid' '--with-pidfile=/var/run/squid.pid' '--disable-dependency-tracking' '--enable-arp-acl' '--enable-follow-x-forwarded-for' '--enable-auth=basic,digest,negotiate' '--enable-external-acl-helpers=ip_user,unix_group,wbinfo_group' '--enable-cache-digests' '--enable-cachemgr-hostname=localhost' '--enable-delay-pools' '--enable-epoll' '--enable-icap-client' '--enable-ident-lookups' '--enable-linux-netfilter' '--enable-referer-log' '--enable-snmp' '--enable-ssl' '--enable-storeio=aufs,ufs' '--enable-wccpv2' '--enable-esi' '--with-aio' '--with-default-user=proxy' '--with-filedescriptors=65536' '--with-dl' '--with-pthreads' '--with-libcap' '--with-netfilter-conntrack' '--with-openssl' '--enable-inline' '--enable-uselect' '--enable-disk-io' '--disable-htcp' '--with-gnu-ld' '--with-build-environment=default' '--enable-carp' '--enable-ecap' '--enable-linux-netfilter'
make
make install
Add or edit the following lines in your /etc/squid/squid.conf:
Code: Select all
ecap_enable on
ecap_service gzip_service respmod_precache 0 ecap://www.vigos.com/ecap_gzip
loadable_modules /usr/local/lib/ecap_adapter_gzip.so
acl GZIP_HTTP_STATUS http_status 200
adaptation_access gzip_service allow GZIP_HTTP_STATUS
Code: Select all
/usr/sbin/squid -NCd1
Error:
Fix:squid: error while loading shared libraries: libecap.so.0: cannot open shared object file: No such file or directory
Add the following: /usr/local/lib to: /etc/ld.so.conf then run the command “ldconfig”
Error:
Fix:2013/09/06 14:58:01| WARNING: Could not determine this machines public hostname. Please configure one or set 'visible_hostname'.
Add the visible_hostname SERVER_HOST_NAME to /et c/squid/squid.conf
Error:
Fix:FATAL: getpwnam failed to find userid for effective user 'proxy'
Squid Cache (Version 3.1.11): Terminated abnormally.
CPU Usage: 0.015 seconds = 0.004 user + 0.011 sys
Maximum Resident Size: 18000 KB
Page faults with physical i/o: 0
Create user called proxy: useradd proxy
Error:
Code: Select all
FATAL: Cannot open '/var/log/squid/access.log' for writing.
The parent directory must be writeable by the
user 'proxy', which is the cache_effective_user
set in squid.conf.
Create the file /var/log/squid/access.log, give owner ship to user proxy over it along with read/write permissions.
Code: Select all
> touch /var/log/squid/access.log
> chmod -R r+uX /var/log/squid/access.log
> chmod -R u+rX /var/log/squid/access.log
Code: Select all
touch /var/log/squid/cache.log
chown -R proxy /var/log/squid/cache.log
chmod -R u+rX /var/log/squid/cache.log
/usr/sbin/squid –z
Acl ACL_NAME src YOUR_NETWORK/SUBNET_PREFIX
http_access allow ACL_NAME
P.S there in an http_access deny all command in the configuration file, make sure to the above line before it.
now i am trying to configure squid to run as a TPROXY, the page requested by squid will have the client ip and not the squid ip as source, may come in handy in some situation
enjoy it