Syed Jahanzaib Personal Blog to Share Knowledge !

August 13, 2012

Youtube caching with Squid + Nginx

Filed under: Linux Related — Tags: , , — Syed Jahanzaib / Pinochio~:) @ 11:52 AM

Updated Version of squid which cache youtube and many other contents. read following

https://aacable.wordpress.com/2012/01/19/youtube-caching-with-squid-2-7-using-storeurl-pl/

 

 

Advantages of Youtube Caching   !!!

In most part of the world, bandwidth is very expensive, therefore it is (in some scenarios) very useful to Cache Youtube videos or any other flash videos, so if one of user downloads video / flash file , why again the same user or other user can’t download the same file from the CACHE, why he sucking the internet pipe for same content again n again?
Peoples on same LAN ,sometimes watch similar videos. If I put some youtube video link on on FACEBOOK, TWITTER or likewise , and all my friend will  watch that video and that particular video gets viewed many times in few hours. Usually the videos are shared over facebook or other social networking sites so the chances are high for multiple hits per popular videos for my LAN users / friends. [syed.jahanzaib]

This is the reason why I wrote this article.

Disadvantages of Youtube Caching   !!!

The chances, that another user will watch the same video, is really slim. if I search for something specific on youtube, i get more then hundreds of search results for same video. What is the chance that another user will search for the same thing, and will click on the same link / result? Youtube hosts more than 10 million videos. Which is too much to cache anyway. You need lot of space to cache videos. Also accordingly you will be needing ultra modern fast hardware with tons of  SPACE to handle such kind of cache giant. anyhow Try it

AFAIK you are not supposed to cache youtube videos, youtube don’t like it. I don’t understand why. Probably because their ranking mechanism relies on views, and possibly completed views, which wouldn’t be measurable if the content was served from a local cache.

After unsuccessful struggling with storeurl.pl method , I was searching for alternate method to cache youtube videos. Finally I found ruby base method using Nginx to cache YT. Using this method I was able to cache all Youtube videos almost perfectly. (not 100%, but it works fine in most cases with some modification.I am sure there will be some improvement in near future).

Updated: 24thth August, 2012

Thanks to Mr. Eliezer Croitoru & Mr.Christian Loth & others for there kind guidance.

Following components were used in this guide.

Proxy Server Configuration:
Ubuntu Desktop 10.4
Nginix version: nginx/0.7.65
Squid Cache: Version 2.7.STABLE7

Client Configuration for testing videos:
Windows XP with Internet Explorer 6
Windows 7 with Internet Explorer 8

Lets start with the Proxy Server Configuration:

1) Update Ubuntu

First install Ubuntu, After installation, configure its networking components, then update it by following command
apt-get install update

2) Install SSH Server [Optional]

Now install SSH server so that you can manage your server remotely using PUTTY or any other ssh tool.

apt-get install openssh-server

3) Install Squid Server

Now install Squid Server by following command
apt-get install squid
[This will install squid 2.7 by default]

Now edit squid configuration files by using following command

nano /etc/squid/squid.conf

Remove all lines and paste the following data

# SQUID 2.7/ Nginx TEST CONFIG FILE
# Email: aacable@hotmail.com
# Web  : https://aacable.wordpress.com
# PORT and Transparent Option
http_port 8080 transparent
server_http11 on
icp_port 0

# Cache is set to 5GB in this example (zaib)
store_dir_select_algorithm round-robin
cache_dir aufs /cache1 5000 16 256
cache_replacement_policy heap LFUDA
memory_replacement_policy heap LFUDA

# If you want to enable DATE time n SQUID Logs,use following
emulate_httpd_log on
logformat squid %tl %6tr %>a %Ss/%03Hs %<st %rm %ru %un %Sh/%<A %mt
log_fqdn off

# How much days to keep users access web logs
# You need to rotate your log files with a cron job. For example:
# 0 0 * * * /usr/local/squid/bin/squid -k rotate
logfile_rotate 14
debug_options ALL,1
cache_access_log /var/log/squid/access.log
cache_log /var/log/squid/cache.log
cache_store_log /var/log/squid/store.log

#[zaib] I used DNSAMSQ service for fast dns resolving
#so install by using "apt-get install dnsmasq" first
dns_nameservers 127.0.0.1 221.132.112.8

#ACL Section
acl all src 0.0.0.0/0.0.0.0
acl manager proto cache_object
acl localhost src 127.0.0.1/255.255.255.255
acl to_localhost dst 127.0.0.0/8
acl SSL_ports port 443 563 # https, snews
acl SSL_ports port 873 # rsync
acl Safe_ports port 80 # http
acl Safe_ports port 21 # ftp
acl Safe_ports port 443 563 # https, snews
acl Safe_ports port 70 # gopher
acl Safe_ports port 210 # wais
acl Safe_ports port 1025-65535 # unregistered ports
acl Safe_ports port 280 # http-mgmt
acl Safe_ports port 488 # gss-http
acl Safe_ports port 591 # filemaker
acl Safe_ports port 777 # multiling http
acl Safe_ports port 631 # cups
acl Safe_ports port 873 # rsync
acl Safe_ports port 901 # SWAT
acl purge method PURGE
acl CONNECT method CONNECT
http_access allow manager localhost
http_access deny manager
http_access allow purge localhost
http_access deny purge
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
http_access allow localhost
http_access allow all
http_reply_access allow all
icp_access allow all

#[zaib]I used UBUNTU so user is proxy, in FEDORA you may use use squid
cache_effective_user proxy
cache_effective_group proxy
cache_mgr aacable@hotmail.com
visible_hostname proxy.aacable.net
unique_hostname aacable@hotmail.com

cache_mem 8 MB
minimum_object_size 0 bytes
maximum_object_size 100 MB
maximum_object_size_in_memory 128 KB

refresh_pattern ^ftp: 1440 20% 10080
refresh_pattern ^gopher: 1440 0% 1440
refresh_pattern -i (/cgi-bin/|\?) 0 0% 0
refresh_pattern (Release|Packages(.gz)*)$ 0 20% 2880
refresh_pattern . 0 50% 4320
acl apache rep_header Server ^Apache
broken_vary_encoding allow apache

# Youtube Cache Section [zaib]
url_rewrite_program /etc/nginx/nginx.rb
url_rewrite_host_header off
acl youtube_videos url_regex -i ^http://[^/]+\.youtube\.com/videoplayback\?
acl range_request req_header Range .
acl begin_param url_regex -i [?&]begin=
acl id_param url_regex -i [?&]id=
acl itag_param url_regex -i [?&]itag=
acl sver3_param url_regex -i [?&]sver=3
cache_peer 127.0.0.1 parent 8081 0 proxy-only no-query connect-timeout=10
cache_peer_access 127.0.0.1 allow youtube_videos id_param itag_param sver3_param !begin_param !range_request
cache_peer_access 127.0.0.1 deny all

Save & Exit.

4) Install Nginx

Now install Nginix by
apt-get install nginx

Now edit its config file by using following command
nano /etc/nginx/nginx.conf

Remove all lines and paste the following data

# This config file is not written by me, [syed.jahanzaib]
# My Email address is inserted Just for tracking purposes
# For more info, visit http://code.google.com/p/youtube-cache/
# Syed Jahanzaib / aacable [at] hotmail.com
user www-data;
worker_processes 4;
pid /var/run/nginx.pid;
events {
worker_connections 768;
}
http {
sendfile on;
tcp_nopush on;
tcp_nodelay on;
keepalive_timeout 65;
types_hash_max_size 2048;
include /etc/nginx/mime.types;
default_type application/octet-stream;
access_log /var/log/nginx/access.log;
error_log /var/log/nginx/error.log;
gzip on;
gzip_static on;
gzip_comp_level 6;
gzip_disable .msie6.;
gzip_vary on;
gzip_types text/plain text/css text/xml text/javascript application/json application/x-javascript application/xml application/xml+rss;
gzip_proxied expired no-cache no-store private auth;
gzip_buffers 16 8k;
gzip_http_version 1.1;
include /etc/nginx/conf.d/*.conf;
include /etc/nginx/sites-enabled/*;
# starting youtube section
server {
listen 127.0.0.1:8081;
location / {
root /usr/local/www/nginx_cache/files;
#try_files "/id=$arg_id.itag=$arg_itag" @proxy_youtube; # Old one
#try_files  "$uri" "/id=$arg_id.itag=$arg_itag.flv" "/id=$arg_id-range=$arg_range.itag=$arg_itag.flv" @proxy_youtube; #old2
try_files "/id=$arg_id.itag=$arg_itag.range=$arg_range.algo=$arg_algorithm" @proxy_youtube;
}
location @proxy_youtube {
resolver 221.132.112.8;
proxy_pass http://$host$request_uri;
proxy_temp_path "/usr/local/www/nginx_cache/tmp";
#proxy_store "/usr/local/www/nginx_cache/files/id=$arg_id.itag=$arg_itag"; # Old 1
proxy_store "/usr/local/www/nginx_cache/files/id=$arg_id.itag=$arg_itag.range=$arg_range.algo=$arg_algorithm";
proxy_ignore_client_abort off;
proxy_method GET;
proxy_set_header X-YouTube-Cache "aacable@hotmail.com";
proxy_set_header Accept "video/*";
proxy_set_header User-Agent "YouTube Cacher (nginx)";
proxy_set_header Accept-Encoding "";
proxy_set_header Accept-Language "";
proxy_set_header Accept-Charset "";
proxy_set_header Cache-Control "";}
}
}

Save & Exit.

Now Create directories to hold cache files

mkdir /usr/local/www
mkdir /usr/local/www/nginx_cache
mkdir /usr/local/www/nginx_cache/tmp
mkdir /usr/local/www/nginx_cache/files
chown www-data /usr/local/www/nginx_cache/files/ -Rf

Now create nginx .rb file

touch /etc/nginx/nginx.rb
chmod 755 /etc/nginx/nginx.rb
nano /etc/nginx/nginx.rb

Paste the following data in this newly created file

#!/usr/bin/env ruby1.8
# This script is not written by me,
# My Email address is inserted Just for tracking purposes
# For more info, visit http://code.google.com/p/youtube-cache/
# Syed Jahanzaib / aacable [at] hotmail.com
# url_rewrite_program <path>/nginx.rb
# url_rewrite_host_header off

require "syslog"
require "base64"

class SquidRequest
attr_accessor :url, :user
attr_reader :client_ip, :method

def method=(s)
@method = s.downcase
end

def client_ip=(s)
@client_ip = s.split('/').first
end
end

def read_requests
# URL <SP> client_ip "/" fqdn <SP> user <SP> method [<SP> kvpairs]<NL>
STDIN.each_line do |ln|
r = SquidRequest.new
r.url, r.client_ip, r.user, r.method, *dummy = ln.rstrip.split(' ')
(STDOUT << "#{yield r}\n").flush
end
end

def log(msg)
Syslog.log(Syslog::LOG_ERR, "%s", msg)
end

def main
Syslog.open('nginx.rb', Syslog::LOG_PID)
log("Started")

read_requests do |r|
if r.method == 'get' && r.url !~ /[?&]begin=/ && r.url =~ %r{\Ahttp://[^/]+\.youtube\.com/(videoplayback\?.*)\z}
log("YouTube Video [#{r.url}].")
"http://127.0.0.1:8081/#{$1}"
else
r.url
end
end
end
main

Save & Exit.

5) Install RUBY

What is RUBY?
Ruby is a dynamic, open source programming language with a focus on simplicity and productivity. It has an elegant syntax that is natural to read and easy to write. [syed.jahanzaib]

Now install RUBY by following command

apt-get install ruby

6) Configure Squid Cache DIR and Permissions

Now create cache dir and assign proper permission to proxy user

mkdir /cache1
chown proxy:proxy /cache1
chmod -R  777 /cache1

Now initialize squid cache directories by

squid -z

You should see Following message

Creating Swap Directories

7) Finally Start/restart SQUID & Nginx

service squid start
service nginx restart

Now from test pc, open youtube and play any video, after it download completely, delete the browser cache, and play the same video again, This time it will be served from the cache. You can verify it by monitoring your WAN link utilization while playing the cached file.

Look at the below WAN utilization graph, it was taken while watching the clip which is not in cache

WAN utilization of Proxy, While watching New Clip (Not in cache)

Now Look at the below WAN utilization graph, it was taken while watching the clip which is now in CACHE.

WAN utilization of Proxy, While watching already cached Clip

Playing Video, loaded from the cache chunk by chunk

It will load first chunk from the cache, if the user keep watching the clip, it will load next chunk at the end of first chunk, and will continue to do so.

Video cache files can be found in following locations.
/usr/local/www/nginx_cache/files

e.g:

ls -lh /usr/local/www/nginx_cache/files

The above file shows the clip is in 360p quality, and the length of the clip is 5:54 Seconds.
itag=34 shows the video quality is 360p.

Credits: Thanks to Mr. Eliezer Croitoru & Mr.Christian Loth & others for there kind guidance.

Find files that have not been accessed from x days. Useful to delete old cache files that have not been accessed since x days.

 find  /usr/local/www/nginx_cache/files  -atime  +30  -type f

Regard’s
Syed Jahanzaib

April 21, 2014

Howto Cache Youtube with SQUID / LUSCA and bypass Cached Videos from Mikrotik Queue [April, 2014 , zaib]


LAST UPDATED: 22nd April, 2014 / 0800 hours pkst

Youtube caching is working as of 20th June,  2014

[1st version > 11th January, 2011]

.

Following web sites are tested and working good 🙂

.

If this method helps you, please post your comment.

 

.

 What is LUSCA / SQUID ?

LUSCA is an advance version or Fork of  SQUID 2. The Lusca project aims to fix the shortcomings in the Squid-2. It also supports a variety of clustering protocols. By Using it, you can cache some dynamic contents that you previously can’t do with the squid.

For example [jz]
#  Video Cachingi.e Youtube / tube etc . . .
#  Windows / Linux Updates / Anti-virus , Anti-Malware i.e. Avira/ Avast / MBAM etc . . .
#  Well known sites i.e. facebook / google / yahoo etch. etch.
#  Download caching mp3’s/mpeg/avi etc . . .

Advantages of Youtube Caching   !!!

In most part of the world, bandwidth is very expensive, therefore it is (in some scenarios) very useful to Cache Youtube videos or any other flash videos, so if one of user downloads video / flash file , why again the same user or other user can’t download the same file from the CACHE, why he sucking the internet pipe for same content again n again?
Peoples on same LAN ,sometimes watch similar videos. If I put some youtube video link on on FACEBOOK, TWITTER or likewise , and all my friend will  watch that video and that particular video gets viewed many times in few hours. Usually the videos are shared over facebook or other social networking sites so the chances are high for multiple hits per popular videos for my lan users / friends / zaib.

This is the reason why I wrote this article. I have implemented Ubuntu with LUSCA/ Squid on it and its working great, but to achieve some results you need to have some TB of storage drives in your proxy machine.

Disadvantages of Youtube Caching   !!!

The chances, that another user will watch the same video, is really slim. if I search for something specific on youtube, i get more then hundreds of search results for same video. What is the chance that another user will search for the same thing, and will click on the same link / result? Youtube hosts more than 10 million videos. Which is too much to cache anyway. You need lot of space to cache videos. Also accordingly you will be needing ultra modern fast hardware with tons of RAM to handle such kind of cache giant. anyhow Try it

We will divide this article in following Sections

1#  Installing SQUID / LUSCA in UBUNTU
2#  Setting up SQUID / LUSCA Configuration files
3#  Performing some Tests, testing your Cache HIT
4# Using ZPH TOS to deliver cached contents to clients vai mikrotik at full LAN speed, Bypassing the User Queue for cached contents.

.

.

1#  Installing SQUID / LUSCA in UBUNTU

I assume your ubuntu box have 2 interfaces configured, one for LAN and second for WAN. You have internet sharing already configured. Now moving on to LUSCA / SQUID installation.

Here we go ….

Issue following command. Remember that its a one go command (which have multiple commands inside it so it may take a bit long to update, install and compile all required items)


apt-get update &&
apt-get install gcc -y &&
apt-get install build-essential -y &&
apt-get install libstdc++6 -y &&
apt-get install unzip -y &&
apt-get install bzip2 -y &&
apt-get install sharutils -y &&
apt-get install ccze -y &&
apt-get install libzip-dev -y &&
apt-get install automake1.9 -y &&
apt-get install acpid -y &&
apt-get install libfile-readbackwards-perl -y &&
apt-get install dnsmasq -y &&
cd /tmp &&
wget -c http://wifismartzone.com/files/linux_related/lusca/LUSCA_HEAD-r14942.tar.gz &&
tar -xvzf LUSCA_HEAD-r14942.tar.gz &&
cd /tmp/LUSCA_HEAD-r14942 &&
./configure \
--prefix=/usr \
--exec_prefix=/usr \
--bindir=/usr/sbin \
--sbindir=/usr/sbin \
--libexecdir=/usr/lib/squid \
--sysconfdir=/etc/squid \
--localstatedir=/var/spool/squid \
--datadir=/usr/share/squid \
--enable-async-io=24 \
--with-aufs-threads=24 \
--with-pthreads \
--enable-storeio=aufs \
--enable-linux-netfilter \
--enable-arp-acl \
--enable-epoll \
--enable-removal-policies=heap \
--with-aio \
--with-dl \
--enable-snmp \
--enable-delay-pools \
--enable-htcp \
--enable-cache-digests \
--disable-unlinkd \
--enable-large-cache-files \
--with-large-files \
--enable-err-languages=English \
--enable-default-err-language=English \
--enable-referer-log \
--with-maxfd=65536 &&
make &&
make install

EDIT SQUID.CONF FILE

Now edit SQUID.CONF file by using following command

nano /etc/squid/squid.conf

and Delete all previously lines , and paste the following lines.

Remember following squid.conf is not very neat and clean , you will find many un necessary junk entries in it, but as I didn’t had time to clean them all, so you may clean them as per your targets and goals.

Now Paste the following data … (in squid.conf)


#######################################################
## Squid_LUSCA configuration Starts from Here ...     #
## Thanks to some INDO friendsfor sharing Configs     #
## Syed.Jahanzaib / 22nd April, 2014                  #
## https://aacable.wordpress.com / aacable@hotmail.com #
#######################################################

# HTTP Port for SQUID Service
http_port 8080 transparent
server_http11 on

# Cache Pee, for parent proxy if you ahve any, or ignore it.
#cache_peer x.x.x.x parent 8080 0

# Various Logs/files location
pid_filename /var/run/squid.pid
coredump_dir /var/spool/squid/
error_directory /usr/share/squid/errors/English
icon_directory /usr/share/squid/icons
mime_table /etc/squid/mime.conf
access_log daemon:/var/log/squid/access.log squid
cache_log none
#debug_options ALL,1 22,3 11,2 #84,9
referer_log /var/log/squid/referer.log
cache_store_log none
store_dir_select_algorithm  round-robin
logfile_daemon /usr/lib/squid/logfile-daemon
logfile_rotate 1

# Cache Policy
cache_mem 6 MB
maximum_object_size_in_memory 0 KB
memory_replacement_policy heap GDSF
cache_replacement_policy heap LFUDA

minimum_object_size 0 KB
maximum_object_size 10 GB
cache_swap_low 98
cache_swap_high 99

# Cache Folder Path, using 5GB for test
cache_dir aufs /cache-1 5000 16 256

# ACL Section
acl all src all
acl manager proto cache_object
acl localhost src 127.0.0.1/32
acl to_localhost dst 127.0.0.0/8
acl localnet src 10.0.0.0/8            # RFC1918 possible internal network
acl localnet src 172.16.0.0/12        # RFC1918 possible internal network
acl localnet src 192.168.0.0/16        # RFC1918 possible internal network
acl localnet src 125.165.92.1        # RFC1918 possible internal network
acl SSL_ports port 443
acl Safe_ports port 80                # http
acl Safe_ports port 21                # ftp
acl Safe_ports port 443                # https
acl Safe_ports port 70                # gopher
acl Safe_ports port 210                # wais
acl Safe_ports port 1025-65535        # unregistered ports
acl Safe_ports port 280                # http-mgmt
acl Safe_ports port 488                # gss-http
acl Safe_ports port 591                # filemaker
acl Safe_ports port 777                # multiling http
acl CONNECT method CONNECT
acl purge method PURGE
acl snmppublic snmp_community public

acl range dstdomain .windowsupdate.com
range_offset_limit -1 KB range

#===========================================================================
#    Loading Patch
acl DENYCACHE urlpath_regex \.(ini|ui|lst|inf|pak|ver|patch|md5|cfg|lst|list|rsc|log|conf|dbd|db)$
acl DENYCACHE urlpath_regex (notice.html|afs.dat|dat.asp|patchinfo.xml|version.list|iepngfix.htc|updates.txt|patchlist.txt)
acl DENYCACHE urlpath_regex (pointblank.css|login_form.css|form.css|noupdate.ui|ahn.ui|3n.mh)$
acl DENYCACHE urlpath_regex (Loader|gamenotice|sources|captcha|notice|reset)
no_cache deny DENYCACHE

range_offset_limit 1 MB !DENYCACHE
uri_whitespace strip

#===========================================================================
#    Rules to block few Advertising sites
acl ads url_regex -i .youtube\.com\/ad_frame?
acl ads url_regex -i .(s|s[0-90-9])\.youtube\.com
acl ads url_regex -i .googlesyndication\.com
acl ads url_regex -i .doubleclick\.net
acl ads url_regex -i ^http:\/\/googleads\.*
acl ads url_regex -i ^http:\/\/(ad|ads|ads[0-90-9]|ads\d|kad|a[b|d]|ad\d|adserver|adsbox)\.[a-z0-9]*\.[a-z][a-z]*
acl ads url_regex -i ^http:\/\/openx\.[a-z0-9]*\.[a-z][a-z]*
acl ads url_regex -i ^http:\/\/[a-z0-9]*\.openx\.net\/
acl ads url_regex -i ^http:\/\/[a-z0-9]*\.u-ad\.info\/
acl ads url_regex -i ^http:\/\/adserver\.bs\/
acl ads url_regex -i !^http:\/\/adf\.ly
http_access deny ads
http_reply_access deny ads
#deny_info http://yoursite/yourad,htm ads
#==== End Rules: Advertising ====

strip_query_terms off

acl yutub url_regex -i .*youtube\.com\/.*$
acl yutub url_regex -i .*youtu\.be\/.*$
logformat squid1 %{Referer}>h %ru
access_log /var/log/squid/yt.log squid1 yutub

# ==== Custom Option REWRITE ====
acl store_rewrite_list urlpath_regex \/(get_video\?|videodownload\?|videoplayback.*id)

acl store_rewrite_list urlpath_regex \.(mp2|mp3|mid|midi|mp[234]|wav|ram|ra|rm|au|3gp|m4r|m4a)\?
acl store_rewrite_list urlpath_regex \.(mpg|mpeg|mp4|m4v|mov|avi|asf|wmv|wma|dat|flv|swf)\?
acl store_rewrite_list urlpath_regex \.(jpeg|jpg|jpe|jp2|gif|tiff?|pcx|png|bmp|pic|ico)\?
acl store_rewrite_list urlpath_regex \.(chm|dll|doc|docx|xls|xlsx|ppt|pptx|pps|ppsx|mdb|mdbx)\?
acl store_rewrite_list urlpath_regex \.(txt|conf|cfm|psd|wmf|emf|vsd|pdf|rtf|odt)\?
acl store_rewrite_list urlpath_regex \.(class|jar|exe|gz|bz|bz2|tar|tgz|zip|gzip|arj|ace|bin|cab|msi|rar)\?
acl store_rewrite_list urlpath_regex \.(htm|html|mhtml|css|js)\?

acl store_rewrite_list_web url_regex ^http:\/\/([A-Za-z-]+[0-9]+)*\.[A-Za-z]*\.[A-Za-z]*
acl store_rewrite_list_web_CDN url_regex ^http:\/\/[a-z]+[0-9]\.google\.com doubleclick\.net

acl store_rewrite_list_path urlpath_regex \.(mp2|mp3|mid|midi|mp[234]|wav|ram|ra|rm|au|3gp|m4r|m4a)$
acl store_rewrite_list_path urlpath_regex \.(mpg|mpeg|mp4|m4v|mov|avi|asf|wmv|wma|dat|flv|swf)$
acl store_rewrite_list_path urlpath_regex \.(jpeg|jpg|jpe|jp2|gif|tiff?|pcx|png|bmp|pic|ico)$
acl store_rewrite_list_path urlpath_regex \.(chm|dll|doc|docx|xls|xlsx|ppt|pptx|pps|ppsx|mdb|mdbx)$
acl store_rewrite_list_path urlpath_regex \.(txt|conf|cfm|psd|wmf|emf|vsd|pdf|rtf|odt)$
acl store_rewrite_list_path urlpath_regex \.(class|jar|exe|gz|bz|bz2|tar|tgz|zip|gzip|arj|ace|bin|cab|msi|rar)$
acl store_rewrite_list_path urlpath_regex \.(htm|html|mhtml|css|js)$

acl getmethod method GET

storeurl_access deny !getmethod
#this is not related to youtube video its only for CDN pictures
storeurl_access allow store_rewrite_list_web_CDN
storeurl_access allow store_rewrite_list_web store_rewrite_list_path
storeurl_access allow store_rewrite_list
storeurl_access deny all
storeurl_rewrite_program /etc/squid/storeurl.pl
storeurl_rewrite_children 10
storeurl_rewrite_concurrency 40
# ==== End Custom Option REWRITE ====

#===========================================================================
#    Custom Option REFRESH PATTERN
#===========================================================================
refresh_pattern (get_video\?|videoplayback\?|videodownload\?|\.flv\?|\.fid\?) 43200 99% 43200 override-expire ignore-reload ignore-must-revalidate ignore-private
refresh_pattern -i (get_video\?|videoplayback\?|videodownload\?) 5259487 999% 5259487 override-expire ignore-reload reload-into-ims ignore-no-cache ignore-private
# -- refresh pattern for specific sites -- #
refresh_pattern ^http://*.jobstreet.com.*/.* 720 100% 10080 override-expire override-lastmod ignore-no-cache
refresh_pattern ^http://*.indowebster.com.*/.* 720 100% 10080 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-auth
refresh_pattern ^http://*.21cineplex.*/.* 720 100% 10080 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-auth
refresh_pattern ^http://*.atmajaya.*/.* 720 100% 10080 override-expire ignore-no-cache ignore-auth
refresh_pattern ^http://*.kompas.*/.* 720 100% 10080 override-expire override-lastmod reload-into-ims ignore-no-cache ignore-auth
refresh_pattern ^http://*.theinquirer.*/.* 720 100% 10080 override-expire ignore-no-cache ignore-auth
refresh_pattern ^http://*.blogspot.com/.* 720 100% 10080 override-expire override-lastmod reload-into-ims ignore-no-cache ignore-auth
refresh_pattern ^http://*.wordpress.com/.* 720 100% 10080 override-expire override-lastmod reload-into-ims ignore-no-cache
refresh_pattern ^http://*.photobucket.com/.* 720 100% 10080 override-expire override-lastmod reload-into-ims ignore-no-cache ignore-auth
refresh_pattern ^http://*.tinypic.com/.* 720 100% 10080 override-expire override-lastmod reload-into-ims ignore-no-cache ignore-auth
refresh_pattern ^http://*.imageshack.us/.* 720 100% 10080 override-expire override-lastmod reload-into-ims ignore-no-cache ignore-auth
refresh_pattern ^http://*.kaskus.*/.* 720 100% 28800 override-expire override-lastmod reload-into-ims ignore-no-cache ignore-auth
refresh_pattern ^http://www.kaskus.com/.* 720 100% 28800 override-expire override-lastmod reload-into-ims ignore-no-cache ignore-auth
refresh_pattern ^http://*.detik.*/.* 720 50% 2880 override-expire override-lastmod reload-into-ims ignore-no-cache ignore-auth
refresh_pattern ^http://*.detiknews.*/*.* 720 50% 2880 override-expire override-lastmod reload-into-ims ignore-no-cache ignore-auth
refresh_pattern ^http://video.liputan6.com/.* 720 100% 10080 override-expire override-lastmod reload-into-ims ignore-no-cache ignore-auth
refresh_pattern ^http://static.liputan6.com/.* 720 100% 10080 override-expire override-lastmod reload-into-ims ignore-no-cache ignore-auth
refresh_pattern ^http://*.friendster.com/.* 720 100% 10080 override-expire override-lastmod ignore-no-cache ignore-auth
refresh_pattern ^http://*.facebook.com/.* 720 100% 10080 override-expire override-lastmod reload-into-ims ignore-no-cache ignore-auth
refresh_pattern ^http://apps.facebook.com/.* 720 100% 10080 override-expire override-lastmod reload-into-ims ignore-no-cache ignore-auth
refresh_pattern ^http://*.fbcdn.net/.* 720 100% 10080 override-expire override-lastmod reload-into-ims ignore-no-cache ignore-auth
refresh_pattern ^http://profile.ak.fbcdn.net/.* 720 100% 10080 override-expire override-lastmod reload-into-ims ignore-no-cache ignore-auth
refresh_pattern ^http://static.playspoon.com/.* 720 100% 10080 override-expire override-lastmod reload-into-ims ignore-no-cache ignore-auth
refresh_pattern ^http://cooking.game.playspoon.com/.* 720 100% 10080 override-expire override-lastmod reload-into-ims ignore-no-cache ignore-auth
refresh_pattern -i http://[^a-z\.]*onemanga\.com/? 720 80% 10080 override-expire override-lastmod reload-into-ims ignore-no-cache ignore-auth
refresh_pattern ^http://media?.onemanga.com/.* 720 80% 10080 override-expire override-lastmod reload-into-ims ignore-no-cache ignore-auth
refresh_pattern ^http://*.yahoo.com/.* 720 80% 10080 override-expire override-lastmod reload-into-ims ignore-no-cache ignore-auth
refresh_pattern ^http://*.google.com/.* 720 80% 10080 override-expire override-lastmod reload-into-ims ignore-no-cache ignore-auth
refresh_pattern ^http://*.forummikrotik.com/.* 720 80% 10080 override-expire override-lastmod reload-into-ims ignore-no-cache ignore-auth
refresh_pattern ^http://*.linux.or.id/.* 720 100% 10080 override-expire override-lastmod reload-into-ims ignore-no-cache ignore-auth
# -- refresh pattern for extension -- #
refresh_pattern -i \.(mp2|mp3|mid|midi|mp[234]|wav|ram|ra|rm|au|3gp|m4r|m4a)(\?.*|$) 5259487 999% 5259487 override-expire ignore-reload reload-into-ims ignore-no-cache ignore-private
refresh_pattern -i \.(mpg|mpeg|mp4|m4v|mov|avi|asf|wmv|wma|dat|flv|swf)(\?.*|$) 5259487 999% 5259487 override-expire ignore-reload reload-into-ims ignore-no-cache ignore-private
refresh_pattern -i \.(jpeg|jpg|jpe|jp2|gif|tiff?|pcx|png|bmp|pic|ico)(\?.*|$) 5259487 999% 5259487 override-expire ignore-reload reload-into-ims ignore-no-cache ignore-private
refresh_pattern -i \.(chm|dll|doc|docx|xls|xlsx|ppt|pptx|pps|ppsx|mdb|mdbx)(\?.*|$) 5259487 999% 5259487 override-expire ignore-reload reload-into-ims ignore-no-cache ignore-private
refresh_pattern -i \.(txt|conf|cfm|psd|wmf|emf|vsd|pdf|rtf|odt)(\?.*|$) 5259487 999% 5259487 override-expire ignore-reload reload-into-ims ignore-no-cache ignore-private
refresh_pattern -i \.(class|jar|exe|gz|bz|bz2|tar|tgz|zip|gzip|arj|ace|bin|cab|msi|rar)(\?.*|$) 5259487 999% 5259487 override-expire ignore-reload reload-into-ims ignore-no-cache ignore-private
refresh_pattern -i \.(htm|html|mhtml|css|js)(\?.*|$) 1440 90% 86400 override-expire ignore-reload reload-into-ims
#===========================================================================
refresh_pattern -i (/cgi-bin/|\?) 0 0% 0
refresh_pattern ^gopher: 1440 0% 1440
refresh_pattern ^ftp: 10080 95% 10080 override-lastmod reload-into-ims
refresh_pattern . 0 20% 10080 override-lastmod reload-into-ims

http_access allow manager localhost
http_access deny manager
http_access allow purge localhost
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports

http_access allow localnet
http_access allow all
http_access deny all

icp_access allow localnet
icp_access deny all
icp_port 0

buffered_logs on

acl shoutcast rep_header X-HTTP09-First-Line ^ICY.[0-9]
upgrade_http0.9 deny shoutcast

acl apache rep_header Server ^Apache
broken_vary_encoding allow apache

forwarded_for off
header_access From deny all
header_access Server deny all
header_access Link deny all
header_access Via deny all
header_access X-Forwarded-For deny all
httpd_suppress_version_string on

shutdown_lifetime 10 seconds

snmp_port 3401
snmp_access allow snmppublic all
dns_timeout 1 minutes

dns_nameservers 8.8.8.8 8.8.4.4

fqdncache_size 5000    #16384
ipcache_size 5000    #16384
ipcache_low 98
ipcache_high 99
log_fqdn off
log_icp_queries off
memory_pools off

maximum_single_addr_tries 2
retry_on_error on

icp_hit_stale on

strip_query_terms off

query_icmp on
reload_into_ims on
emulate_httpd_log off
negative_ttl 0 seconds
pipeline_prefetch on
vary_ignore_expire on
half_closed_clients off
high_page_fault_warning 2
nonhierarchical_direct on
prefer_direct off
cache_mgr aacable@hotmail.com
cache_effective_user proxy
cache_effective_group proxy
visible_hostname proxy.zaib
unique_hostname syed_jahanzaib
cachemgr_passwd none all
client_db on
max_filedescriptors 8192

# ZPH config Marking Cache Hit, so cached contents can be delivered at full lan speed via MT
zph_mode tos
zph_local 0x30
zph_parent 0
zph_option 136

.

.

SOTEURL.PL

Now We have to create an important file name storeurl.pl , which is very important and actually it does the
main job to redirect and pull video from cache.

touch /etc/squid/storeurl.pl
chmod +x /etc/squid/storeurl.pl
nano /etc/squid/storeurl.pl

Now paste the following lines, then Save and exit.

#!/usr/bin/perl
#######################################################
## Squid_LUSCA storeurl.pl starts from Here ...     #
## Thanks to Mr. Safatah [INDO] for sharing Configs  #
## Syed.Jahanzaib / 22nd April, 2014 #
## https://aacable.wordpress.com / aacable@hotmail.com #
#######################################################
$|=1;
while (<>) {
@X = split;
$x = $X[0] . " ";
##=================
## Encoding YOUTUBE
##=================
if ($X[1] =~ m/^http\:\/\/.*(youtube|google).*videoplayback.*/){
@itag = m/[&?](itag=[0-9]*)/;
@CPN = m/[&?]cpn\=([a-zA-Z0-9\-\_]*)/;
@IDS = m/[&?]id\=([a-zA-Z0-9\-\_]*)/;
$id = &GetID($CPN[0], $IDS[0]);
@range = m/[&?](range=[^\&\s]*)/;
print $x . "http://fathayu/" . $id . "&@itag@range\n";
} elsif ($X[1] =~ m/(youtube|google).*videoplayback\?/ ){
@itag = m/[&?](itag=[0-9]*)/;
@id = m/[&?](id=[^\&]*)/;
@redirect = m/[&?](redirect_counter=[^\&]*)/;
print $x . "http://fathayu/";
# ==========================================================================
#    VIMEO
# ==========================================================================
} elsif ($X[1] =~ m/^http:\/\/av\.vimeo\.com\/\d+\/\d+\/(.*)\?/) {
print $x . "http://fathayu/" . $1 . "\n";
} elsif ($X[1] =~ m/^http:\/\/pdl\.vimeocdn\.com\/\d+\/\d+\/(.*)\?/) {
print $x . "http://fathayu/" . $1 . "\n";
# ==========================================================================
#    DAILYMOTION
# ==========================================================================
} elsif ($X[1] =~ m/^http:\/\/proxy-[0-9]{1}\.dailymotion\.com\/(.*)\/(.*)\/video\/\d{3}\/\d{3}\/(.*.flv)/) {
print $x . "http://fathayu/" . $1 . "\n";
} elsif ($X[1] =~ m/^http:\/\/vid[0-9]\.ak\.dmcdn\.net\/(.*)\/(.*)\/video\/\d{3}\/\d{3}\/(.*.flv)/) {
print $x . "http://fathayu/" . $1 . "\n";
# ==========================================================================
#   YIMG
# ==========================================================================
} elsif ($X[1] =~ m/^http:\/\/(.*yimg.com)\/\/(.*)\/([^\/\?\&]*\/[^\/\?\&]*\.[^\/\?\&]{3,4})(\?.*)?$/) {
print $x . "http://fathayu/" . $3 . "\n";
# ==========================================================================
#   YIMG DOUBLE
# ==========================================================================
} elsif ($X[1] =~ m/^http:\/\/(.*?)\.yimg\.com\/(.*?)\.yimg\.com\/(.*?)\?(.*)/) {
print $x . "http://fathayu/" . $3 . "\n";
# ==========================================================================
#   YIMG WITH &sig=
# ==========================================================================
} elsif ($X[1] =~ m/^http:\/\/(.*?)\.yimg\.com\/(.*)/) {
@y = ($1,$2);
$y[0] =~ s/[a-z]+[0-9]+/cdn/;
$y[1] =~ s/&sig=.*//;
print $x . "http://fathayu/" . $y[0] . ".yimg.com/" . $y[1] . "\n";
# ==========================================================================
#    YTIMG
# ==========================================================================
} elsif ($X[1] =~ m/^http:\/\/i[1-4]\.ytimg\.com(.*)/) {
print $x . "http://fathayu/" . $1  . "\n";
# ==========================================================================
#   PORN Movies
# ==========================================================================
} elsif (($X[1] =~ /maxporn/) && (m/^http:\/\/([^\/]*?)\/(.*?)\/([^\/]*?)(\?.*)?$/)) {
print $x . "http://" . $1 . "/SQUIDINTERNAL/" . $3 . "\n";
#   Domain/path/.*/path/filename
} elsif (($X[1] =~ /fucktube/) && (m/^http:\/\/(.*?)(\.[^\.\-]*?[^\/]*\/[^\/]*)\/(.*)\/([^\/]*)\/([^\/\?\&]*)\.([^\/\?\&]{3,4})(\?.*?)$/)) {
@y = ($1,$2,$4,$5,$6);
$y[0] =~ s/(([a-zA-A]+[0-9]+(-[a-zA-Z])?$)|([^\.]*cdn[^\.]*)|([^\.]*cache[^\.]*))/cdn/;
print $x . "http://" . $y[0] . $y[1] . "/" . $y[2] . "/" . $y[3] . "." . $y[4] . "\n";
#   Like porn hub variables url and center part of the path, filename etention 3 or 4 with or without ? at the end
} elsif (($X[1] =~ /tube8|pornhub|xvideos/) && (m/^http:\/\/(([A-Za-z]+[0-9-.]+)*?(\.[a-z]*)?)\.([a-z]*[0-9]?\.[^\/]{3}\/[a-z]*)(.*?)((\/[a-z]*)?(\/[^\/]*){4}\.[^\/\?]{3,4})(\?.*)?$/)) {
print $x . "http://cdn." . $4 . $6 . "\n";
} elsif (($u =~ /tube8|redtube|hardcore-teen|pornhub|tubegalore|xvideos|hostedtube|pornotube|redtubefiles/) && (m/^http:\/\/(([A-Za-z]+[0-9-.]+)*?(\.[a-z]*)?)\.([a-z]*[0-9]?\.[^\/]{3}\/[a-z]*)(.*?)((\/[a-z]*)?(\/[^\/]*){4}\.[^\/\?]{3,4})(\?.*)?$/)) {
print $x . "http://cdn." . $4 . $6 . "\n";
#   acl store_rewrite_list url_regex -i \.xvideos\.com\/.*(3gp|mpg|flv|mp4)
#   refresh_pattern -i \.xvideos\.com\/.*(3gp|mpg|flv|mp4) 1440 99% 14400 override-expire override-lastmod ignore-no-cache ignore-private reload-into-ims ignore-must-revalidate ignore-reload store-stale
# ==========================================================================
} elsif ($X[1] =~ m/^http:\/\/.*\.xvideos\.com\/.*\/([\w\d\-\.\%]*\.(3gp|mpg|flv|mp4))\?.*/){
print $x . "http://fathayu/" . $1 . "\n";
} elsif ($X[1] =~ m/^http:\/\/[\d]+\.[\d]+\.[\d]+\.[\d]+\/.*\/xh.*\/([\w\d\-\.\%]*\.flv)/){
print $x . "http://fathayu/" . $1 . "\n";
} elsif ($X[1] =~ m/^http:\/\/[\d]+\.[\d]+\.[\d]+\.[\d]+.*\/([\w\d\-\.\%]*\.flv)\?start=0/){
print $x . "http://fathayu/" . $1 . "\n";
} elsif ($X[1] =~ m/^http:\/\/.*\.youjizz\.com.*\/([\w\d\-\.\%]*\.(mp4|flv|3gp))\?.*/){
print $x . "http://fathayu/" . $1 . "\n";
} elsif ($X[1] =~ m/^http:\/\/[\w\d\-\.\%]*\.keezmovies[\w\d\-\.\%]*\.com.*\/([\w\d\-\.\%]*\.(mp4|flv|3gp|mpg|wmv))\?.*/){
print $x . "http://fathayu/" . $1 . $2 . "\n";
} elsif ($X[1] =~ m/^http:\/\/[\w\d\-\.\%]*\.tube8[\w\d\-\.\%]*\.com.*\/([\w\d\-\.\%]*\.(mp4|flv|3gp|mpg|wmv))\?.*/) {
print $x . "http://fathayu/" . $1 . "\n";
} elsif ($X[1] =~ m/^http:\/\/[\w\d\-\.\%]*\.youporn[\w\d\-\.\%]*\.com.*\/([\w\d\-\.\%]*\.(mp4|flv|3gp|mpg|wmv))\?.*/){
print $x . "http://fathayu/" . $1 . "\n";
} elsif ($X[1] =~ m/^http:\/\/[\w\d\-\.\%]*\.spankwire[\w\d\-\.\%]*\.com.*\/([\w\d\-\.\%]*\.(mp4|flv|3gp|mpg|wmv))\?.*/) {
print $x . "http://fathayu/" . $1 . "\n";
} elsif ($X[1] =~ m/^http:\/\/[\w\d\-\.\%]*\.pornhub[\w\d\-\.\%]*\.com.*\/([[\w\d\-\.\%]*\.(mp4|flv|3gp|mpg|wmv))\?.*/){
print $x . "http://fathayu/" . $1 . "\n";
} elsif ($X[1] =~ m/^http:\/\/[\w\d\-\_\.\%\/]*.*\/([\w\d\-\_\.]+\.(flv|mp3|mp4|3gp|wmv))\?.*cdn\_hash.*/){
print $x . "http://fathayu/" . $1 . "\n";
} elsif (($X[1] =~ /maxporn/) && (m/^http:\/\/([^\/]*?)\/(.*?)\/([^\/]*?)(\?.*)?$/)) {
print $x . "http://fathayu/" . $1 . "/SQUIDINTERNAL/" . $3 . "\n";
} elsif (($X[1] =~ /fucktube/) && (m/^http:\/\/(.*?)(\.[^\.\-]*?[^\/]*\/[^\/]*)\/(.*)\/([^\/]*)\/([^\/\?\&]*)\.([^\/\?\&]{3,4})(\?.*?)$/)) {
@y = ($1,$2,$4,$5,$6);
$y[0] =~ s/(([a-zA-Z]+[0-9]+(-[a-zA-Z])?$)|([^\.]*cdn[^\.]*)|([^\.]*cache[^\.]*))/cdn/;
print $x . "http://fathayu/" . $y[0] . $y[1] . "/" . $y[2] . "/" . $y[3] . "." . $y[4] . "\n";
} elsif (($X[1] =~ /media[0-9]{1,5}\.youjizz/) && (m/^http:\/\/(.*?)(\.[^\.\-]*?\.[^\/]*)\/(.*)\/([^\/\?\&]*)\.([^\/\?\&]{3,4})(\?.*?)$/)) {
@y = ($1,$2,$4,$5);
$y[0] =~ s/(([a-zA-Z]+[0-9]+(-[a-zA-Z])?$)|([^\.]*cdn[^\.]*)|([^\.]*cache[^\.]*))/cdn/;
print $x . "http://fathayu/" . $y[0] . $y[1] . "/" . $y[2] . "." . $y[3] . "\n";
# ==========================================================================
#   Filehippo
# ==========================================================================
} elsif (($X[1] =~ /filehippo/) && (m/^http:\/\/(.*?)\.(.*?)\/(.*?)\/(.*)\.([a-z0-9]{3,4})(\?.*)?/)) {
@y = ($1,$2,$4,$5);
$y[0] =~ s/[a-z0-9]{2,5}/cdn./;
print $x . "http://fathayu/" . $y[0] . $y[1] . "/" . $y[2] . "." . $y[3] . "\n";
} elsif (($X[1] =~ /filehippo/) && (m/^http:\/\/(.*?)(\.[^\/]*?)\/(.*?)\/([^\?\&\=]*)\.([\w\d]{2,4})\??.*$/)) {
@y = ($1,$2,$4,$5);
$y[0] =~ s/([a-z][0-9][a-z]dlod[\d]{3})|((cache|cdn)[-\d]*)|([a-zA-Z]+-?[0-9]+(-[a-zA-Z]*)?)/cdn/;
print $x . "http://fathayu/" . $y[0] . $y[1] . "/" . $y[2] . "." . $y[3] . "\n";
} elsif ($X[1] =~ m/^http:\/\/.*filehippo\.com\/.*\/([\d\w\%\.\_\-]+\.(exe|zip|cab|msi|mru|mri|bz2|gzip|tgz|rar|pdf))/){
$y=$1;
for ($y) {
s/%20//g;
}
print $x . "http://fathayu//" . $y . "\n";
} elsif (($X[1] =~ /filehippo/) && (m/^http:\/\/(.*?)\.(.*?)\/(.*?)\/(.*)\.([a-z0-9]{3,4})(\?.*)?/)) {
@y = ($1,$2,$4,$5);
$y[0] =~ s/[a-z0-9]{2,5}/cdn./;
print $x . "http://fathayu/" . $y[0] . $y[1] . "/" . $y[2] . "." . $y[3] . "\n";
# ==========================================================================
#   4shared preview
# ==========================================================================
} elsif ($X[1] =~ m/^http:\/\/[a-z]{2}\d{3}\.4shared\.com\/img\/\d+\/\w+\/dlink__2Fdownload_2F.*_3Ftsid_(\w+)-\d+-\w+_26lgfp_3D1000_26sbsr_\w+\/preview.mp3/) {
print $x . "http://fathayu/" . $3 . "\n";

} else {
print $x . $X[1] . "\n";
}
}

sub GetID
{
$id = "";
use File::ReadBackwards;
my $lim = 200 ;
my $ref_log = File::ReadBackwards->new('/var/log/squid/referer.log');
while (defined($line = $ref_log->readline))
{
if ($line =~ m/.*youtube.*\/watch\?.*v=([a-zA-Z0-9\-\_]*).*\s.*id=$IDS[0].*/){
$id = $1;
last;
}
if ($line =~ m/.*youtube.*\/.*cpn=$CPN[0].*[&](video_id|docid|v)=([a-zA-Z0-9\-\_]*).*/){
$id = $2;
last;
}
if ($line =~ m/.*youtube.*\/.*[&?](video_id|docid|v)=([a-zA-Z0-9\-\_]*).*cpn=$CPN[0].*/){
$id = $2;
last;
}
last if --$lim <= 0;
}
if ($id eq ""){
$id = $IDS[0];
}
$ref_log->close();
return $id;
}
### STOREURL.PL ENDS HERE ###

INITIALIZE CACHE and LOG FOLDER …

Now create CACHE folder (here I used test in local drive)

# Log Folder and assign permissions
mkdir /var/log/squid
chown proxy:proxy /var/log/squid/

# Cache Folder
mkdir /cache-1
chown proxy:proxy /cache-1
#Now initialize cache dir by
squid -z

START SQUID SERVICE

Now start SQUID service by following command

squid 

and look for any error or termination it may get. If all ok, just press enter few times and you will be back to command prompt.
to verify if squid is running ok issue following command and look for squid instance, there should be 10+ instance for the squid process

ps aux |grep squid

 Something like below …

123

TIP:

To start SQUID Server in Debug mode, to check any erros, use following

squid -d1N

TEST time ….

It’s time to hit the ROAD and do some tests….

 

YOUTUBE TEST

Open Youtube and watch any Video. After complete download, Check the same video from another client. You will notice that it download very quickly (youtueb video is saved in chunks of 1.7mb each, so after completing first chunk, it will stop, if a user continue to watch the same video, it will serve second chunk and so on , you can watch the bar moving fast without using internet data.
As Shown in the example Below . . .

lusca_test.

YT cache HIT.

 

.

FILEHIPPO TEST [ZAIB]

FILHIPPO

As Shown in the example Below . . .

FILHIPPO

MUSIC DOWNLOAD TEST

Now test any music download. For example Go to
http://www.apniisp.com/songs/indian-movie-songs/ladies-vs-ricky-bahl/690/1.html
As Shown in the example Below . . .

and download any song , after its downloaded, goto 2nd client pc, and download the same song, and monitor the Squid access LOG. You will see cache hit TPC_HIT for this song.

As Shown in the example Below . . .

EXE / PROGRAM  DOWNLOAD TEST

Now test any .exe file download.
Goto http://www.rarlabs.com and download any package. After Download completes, goto 2nd client pc , and download the same file again. and monitor the Squid access LOG. You will see cache hit TPC_HIT for this file.

As Shown in the example Below . . .

SQUID LOGS

Other methods are as follow (I will update following (squid 2.7) article soon)

https://aacable.wordpress.com/2012/01/19/youtube-caching-with-squid-2-7-using-storeurl-pl/
https://aacable.wordpress.com/2012/08/13/youtube-caching-with-squid-nginx/

.

.

.

MIKROTIK with SQUID/ZPH: how to bypass Squid Cache HIT object with Queues Tree in RouterOS 5.x and 6.x

.

zph

.

Using Mikrotik, we can redirect HTTP traffic to SQUID proxy Server, We can also control user bandwidth, but its a good idea to deliver the already cached content to user at full lan speed, that’s why we setup cache server for, to save bandwidth and have fast browsing experience , right :p , So how can we do it in mikrotik that cache content should be delivered to users at unlimited speed, no queue on cache content. Here we go.

By using ZPH directives , we will mark cache content, so that it can later pick by Mikrotik.

Basic requirement is that Squid  must be running in transparent mode, can be done via iptables and squid.conf directives.
I am using UBUNTU squid 2.7 , (in ubuntu , apt-get install squid will install squid 2.7 by default which is gr8 for our work)
Add these lines in SQUID.CONF

#===============================================================================
#ZPH for SQUID 2.7 (Default in ubuntu 10.4) / Syed Jahanzaib aacable@hotmail.com
#===============================================================================
tcp_outgoing_tos 0x30 lanuser [lanuser is ACL for local network, change it to match your's]
zph_mode tos
zph_local 0x30
zph_parent 0
zph_option 136

Use following if you have squid 3.1.19


#======================================================
#ZPH for SQUID 3.1.19 (Default in ubuntu 12.4) / Syed Jahanzaib aacable@hotmail.com
#======================================================

# ZPH for Squid 3.1.19
qos_flows local-hit=0x30

That’s it for SQUID, Now moving on to Mikrotik box ,
Add following rules,

# Marking packets with DSCP (for MT 5.x) for cache hit content coming from SQUID Proxy

/ip firewall mangle add action=mark-packet chain=prerouting disabled=no dscp=12 new-packet-mark=proxy-hit passthrough=no comment="Mark Cache Hit Packets / aacable@hotmail.com"
/queue tree add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=0 max-limit=0 name=pmark packet-mark=proxy-hit parent=global-out priority=8 queue=default

# Marking packets with DSCP (for MT 6.x) for cache hit content coming from SQUID Proxy

/ip firewall mangle add action=mark-packet chain=prerouting comment="MARK_CACHE_HIT_FROM_PROXY_ZAIB" disabled=no dscp=12 new-packet-mark=proxy passthrough=no
/queue simple
add max-limit=100M/100M name="ZPH-Proxy Cache Hit Simple Queue / Syed Jahanzaib >aacable@hotmail.com" packet-marks=zph-hit priority=1/1 target="" total-priority=1

MAKE SURE YOU MOVE THE SIMPLE QUEUE ABOVE ALL OTHER QUEUES 😀
.

Now every packet which is marked by SQUID CACHE_HIT, will be delivered to user at Full lan speed, rest of traffic will be restricted by user Queue.

TROUBLESHOOTING:

the above config is fully tested with UBUNTU SQUID 2.7 and FEDORA 10 with LUSCA

Make sure your squid is marking TOS for cache hit packets. You can check it via TCPDUMP

tcpdump -vni eth0 | grep ‘tos 0×30′

(eht0 = LAN connected interface)

Can you see something like ???

tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 96 bytes
20:25:07.961722 IP (tos 0×30, ttl 64, id 45167, offset 0, flags [DF], proto TCP (6), length 409)
20:25:07.962059 IP (tos 0×30, ttl 64, id 45168, offset 0, flags [DF], proto TCP (6), length 1480)
192 packets captured
195 packets received by filter
0 packets dropped by kernel
_________________________________

Regard’s
SYED JAHANZAIB

 

 

Regard’s
SYED JAHANZAIB
https://aacable.wordpress.com

August 13, 2013

VIDEOCACHE and SQUID Configuration [Short Notes]

Filed under: Linux Related — Tags: , , , — Syed Jahanzaib / Pinochio~:) @ 9:13 AM

videocache-header-logo

WAN utilization / 60mb

WAN utilization @ 60 mbps / zaib

First of all , intro to what VIDEOCACHE is ?

” Videocache is a Squid URL plugin to cache videos from famous video sharing portals/websites like Youtube, Metacafe etc. It helps you save bandwidth when a particular video is requested more than once from the same network/machine.

Squid can not cache the videos served dynamically. Videocache fits into squid to help it cache the videos as well. The cached videos are stored on your proxy server’s local storage or on a storage server in your network. These cached videos can be served to your clients at a very fast speed saving your upstream bandwidth. “

Following are some working configuration samples for VIDEOCACHE with SQUID on UBUNTU. Installation of VC is fairly simple as mentioned here step by step

http://cachevideos.com/documentation/

I am not showing howto install VIDEOCACHE plugin as its very simple and briefly described at the author site, I am only posting the working configuration files. Also note that videocache is a paid plugin, which costs around 400$ per year.

If you are looking for something free, you may try the following NIGNX method.

https://aacable.wordpress.com/2012/08/13/youtube-caching-with-squid-nginx/

Also I would like to share that for VIDEOCACHE plugin, you *MUST* have some good hardware with *LOTS OF* memory and STORAGE. Just for an idea, I deployed it at a network of 300 Users which had Dual CPU with multi cores server for proxy  , 128GB RAM, and 24 TB of external HP Storage, CPU was not much utilized, but at full peak time, the memory usage was around 95%, and the storage usage was around 180 GB in just 1 day.

Also you should have some good amount of bandwidth, otherwise videocache will become bottleneck for your network 🙂 , OR schedule VIDEOCACHE to cache only in specific timings, its supported in /etc/videocache.conf

Following is an working Squid 2.7 on Ubuntu Server 10.4 with videocache support. Please modify necessary directives before using , for example cache_mem, hostname, ACL, Ip’s. Also make sure the CACHE_DIR have been changed to match your local directory structure with proper permissions assgined to user proxy or squid , depend on he nix flavor you are using

# SQUID 2.7 CONFIG FILE WITH VIDEOCACHE PLUGIN FOR SQUID/VIDEOS [ Zaib]

# SQUID 2.7 WORKING CONFIG FILE [August 2013] with Videocache Support
# By - Syed Jahanzaib
# Email: aacable@hotmail.com
# Web  : https://aacable.wordpress.com

cache_store_log /var/log/squid/store.log
acl this_machine src 127.0.0.1 192.168.100.1
http_access allow this_machine

# --BEGIN-- videocache config for squid
url_rewrite_program /usr/bin/python /usr/share/videocache/videocache.py
url_rewrite_children 20

acl vc_deny_url url_regex -i \.blip\.tv\/(.*)filename \.hardsextube\.com\/videothumbs \.xtube\.com\/(.*)(Thumb|videowall)

acl vc_url url_regex -i \/youku\/[0-9A-Z]+\/[0-9A-Z\-]+\.(flv|mp4|avi|mkv|mp3|rm|rmvb|m4v|mov|wmv|3gp|mpg|mpeg)
acl vc_url url_regex -i \/(.*)key=[a-z0-9]+(.*)\.flv
acl vc_url url_regex -i \.(youtube|youtube-nocookie)\.com\/feeds\/api\/videos\/[0-9a-zA-Z_-]{11}\/
acl vc_url url_regex -i \.(youtube|youtube-nocookie)\.com\/(videoplayback|get_video|watch|watch_popup|user_watch)\?
acl vc_url url_regex -i \.(youtube|youtube-nocookie)\.com\/(v|e|embed)\/[0-9a-zA-Z_-]{11}

acl vc_dom_r dstdom_regex -i msn\..*\.(com|net)
acl vc_dom_r dstdom_regex -i msnbc\..*\.(com|net)
acl vc_dom_r dstdom_regex -i video\..*\.fbcdn\.net
acl vc_dom_r dstdom_regex -i myspacecdn\..*\.footprint\.net

acl vc_dom dstdomain .stream.aol.com .videos.5min.com msn.com .blip.tv .vid.ec.dmcdn.net .break.com .vimeocdn.com
acl vc_dom dstdomain .cdn.turner.com .dailymotion.com .c.wrzuta.pl .v.imwx.com .mccont.com .myspacecdn.com
acl vc_dom dstdomain .hardsextube.com .public.extremetube.phncdn.com .redtubefiles.com .video.pornhub.phncdn.com
acl vc_dom dstdomain .public.keezmovies.com .public.keezmovies.phncdn.com .slutload-media.com .public.spankwire.com .xtube.com
acl vc_dom dstdomain .public.youporn.phncdn.com .xvideos.com .tube8.com .public.spankwire.phncdn.com .pornhub.com

refresh_pattern \.youtube\.com\/videoplayback\? 120    80%    180 ignore-no-cache override-expire override-lastmod ignore-private
refresh_pattern stream\.aol\.com\/(.*)/[a-zA-Z0-9]+\/(.*)\.(flv|mp4) 120    80%    180 ignore-no-cache override-expire override-lastmod ignore-private
refresh_pattern videos\.5min\.com\/(.*)/[0-9_]+\.(mp4|flv) 120    80%    180 ignore-no-cache override-expire override-lastmod ignore-private
refresh_pattern proxy[a-z0-9\-]?[a-z0-9]?[a-z0-9]?[a-z0-9]?\.dailymotion\.com\/(.*)\.(flv|on2|mp4|avi|mkv|mp3|rm|rmvb|m4v|mov|wmv|3gp|mpg|mpeg) 120    80%    180 ignore-no-cache override-expire override-lastmod ignore-private
refresh_pattern vid\.akm\.dailymotion\.com\/(.*)\.(flv|on2|mp4|avi|mkv|mp3|rm|rmvb|m4v|mov|wmv|3gp|mpg|mpeg) 120    80%    180 ignore-no-cache override-expire override-lastmod ignore-private
refresh_pattern vid\.ec\.dmcdn\.net\/(.*)\.(flv|on2|mp4|avi|mkv|mp3|rm|rmvb|m4v|mov|wmv|3gp|mpg|mpeg) 120    80%    180 ignore-no-cache override-expire override-lastmod ignore-private
refresh_pattern video\.(.*)\.fbcdn\.net\/(.*)/[0-9_]+\.(mp4|flv|avi|mkv|m4v|mov|wmv|3gp|mpg|mpeg) 120    80%    180 ignore-no-cache override-expire override-lastmod ignore-private
refresh_pattern (.*)\.myspacecdn\.com\/(.*)\/[a-zA-Z0-9]+\/vid\.(flv|mp4|avi|mkv|mp3|rm|rmvb|m4v|mov|wmv|3gp|mpg|mpeg) 120    80%    180 ignore-no-cache override-expire override-lastmod ignore-private
refresh_pattern (.*)\.myspacecdn\.(.*)\.footprint\.net\/(.*)\/[a-zA-Z0-9]+\/vid\.(flv|mp4|avi|mkv|mp3|rm|rmvb|m4v|mov|wmv|3gp|mpg|mpeg) 120    80%    180 ignore-no-cache override-expire override-lastmod ignore-private
refresh_pattern c\.wrzuta\.pl\/wv[0-9]+\/[a-z0-9]+/[0-9]+/ 120    80%    180 ignore-no-cache override-expire override-lastmod ignore-private
refresh_pattern c\.wrzuta\.pl\/wa[0-9]+\/[a-z0-9]+ 120    80%    180 ignore-no-cache override-expire override-lastmod ignore-private
refresh_pattern vs[a-z0-9]?[a-z0-9]?[a-z0-9]?\.hardsextube\.com\/(.*)\/(.*)\.(flv|mp4|avi|mkv|mp3|rm|rmvb|m4v|mov|wmv|3gp|mpg|mpeg) 120    80%    180 ignore-no-cache override-expire override-lastmod ignore-private

acl vc_deny_url url_regex -i crossdomain.xml
acl vc_method method GET
acl vc_header req_header X-Requested-With -i videocache
url_rewrite_access deny !vc_method
url_rewrite_access deny vc_header
url_rewrite_access deny vc_deny_url
url_rewrite_access allow vc_dom
url_rewrite_access allow vc_url
url_rewrite_access allow vc_dom_r
redirector_bypass on

#These lines are needed.
acl this_machine src 127.0.0.1
http_access allow this_machine
strip_query_terms off
maximum_object_size 1000 MB

# --END-- videocache config for squid

# PORT and Transparent Option
http_port 8080 transparent
store_dir_select_algorithm round-robin

# Cache folders, change them according to your need, its an example of 500 GB
cache_dir aufs /cache1 512000 16 256

memory_replacement_policy heap LFUDA
cache_replacement_policy heap GDSF

# If you want to enable DATE time n SQUID Logs,use following
emulate_httpd_log on
logformat squid %tl %6tr %>a %Ss/%03Hs %<st %rm %ru %un %Sh/%<A %mt
log_fqdn off

# How much days to keep users access web logs
# You need to rotate your log files with a cron job. For example:
# 0 0 * * * /usr/local/squid/bin/squid -k rotate
logfile_rotate 14
debug_options ALL,1
cache_access_log /var/log/squid/access.log
cache_log /var/log/squid/cache_log.log
cache_store_log /var/log/squid/store.log

# Block Ads [zaib]
#acl adsites dstdomain url_regex "/etc/squid/adslist.txt"
#http_access deny adsites
#deny_info http://192.168.6.1/psb.htm adsites

#I used DNSAMSQ service for fast dns resolving
#so install by using "apt-get install dnsmasq" first
dns_nameservers 127.0.0.1 8.8.8.8
ftp_user anonymous@
ftp_list_width 32
ftp_passive on
ftp_sanitycheck on

#ACL Section mylan myacl
acl all src 0.0.0.0/0.0.0.0
acl manager proto cache_object
acl localhost src 127.0.0.1/255.255.255.255
acl to_localhost dst 127.0.0.0/8
acl SSL_ports port 443 563 # https, snews
#acl SSL_ports port 873 # rsync
acl Safe_ports port 80 # http
acl Safe_ports port 21 # ftp
acl Safe_ports port 443 563 # https, snews
acl Safe_ports port 70 # gopher
acl Safe_ports port 210 # wais
acl Safe_ports port 1025-65535 # unregistered ports
acl Safe_ports port 280 # http-mgmt
acl Safe_ports port 488 # gss-http
acl Safe_ports port 591 # filemaker
acl Safe_ports port 777 # multiling http
acl Safe_ports port 631 # cups
acl Safe_ports port 873 # rsync
acl Safe_ports port 901 # SWAT
acl purge method PURGE
acl CONNECT method CONNECT
http_access allow manager all
http_access deny manager
http_access allow purge localhost
http_access deny purge
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
http_access allow localhost
http_access allow all
#http_access allow all2
http_reply_access allow all
#http_reply_access allow all2
icp_access allow all

#==========================
# Administrative Parameters
#==========================

#============================================================$
# SNMP , if you want to generate graphs for SQUID via MRTG / zaib
#============================================================$
acl snmppublic snmp_community zaib
snmp_port 3401
snmp_access allow snmppublic all
snmp_access allow all

# I used UBUNTU so user is proxy, in FEDORA you may use use squid
cache_effective_user proxy
cache_effective_group proxy
cache_mgr SYED_JAHAN_ZAIB
visible_hostname aacable.wordpress.com
unique_hostname aacable@hotmail.com

# Memory , If you have 4 GB of RAM in the server, then use 1 GB
cache_mem 1 GB
minimum_object_size 0 bytes
maximum_object_size_in_memory 1024 KB

# ZPH , to mark content which is in CACHE
tcp_outgoing_tos 0x30 all
zph_mode tos
zph_local 0x30
zph_parent 0
zph_option 136
acl store_rewrite_list urlpath_regex \/(get_video|videoplayback\?id|videoplayback.*id)
acl store_rewrite_list urlpath_regex \.(jp(e?g|e|2)|gif|png|tiff?|bmp|ico|flv|wmv|3gp|mp(4|3)|exe|msi|zip|on2|mar)\?
acl store_rewrite_list_domain url_regex ^http:\/\/([a-zA-Z-]+[0-9-]+)\.[A-Za-z]*\.[A-Za-z]*
acl store_rewrite_list_domain url_regex (([a-z]{1,2}[0-9]{1,3})|([0-9]{1,3}[a-z]{1,2}))\.[a-z]*[0-9]?\.[a-z]{3}
acl store_rewrite_list_path urlpath_regex \.(jp(e?g|e|2)|gif|png|tiff?|bmp|ico|flv|avc|zip|mp3|3gp|rar|on2|mar|exe)$
acl store_rewrite_list_domain_CDN url_regex \.rapidshare\.com.*\/[0-9]*\/.*\/[^\/]* ^http:\/\/(www\.ziddu\.com.*\.[^\/]{3,4})\/(.*) \.doubleclick\.net.*
acl store_rewrite_list_domain_CDN url_regex ^http:\/\/[.a-z0-9]*\.photobucket\.com.*\.[a-z]{3}$ quantserve\.com
acl store_rewrite_list_domain_CDN url_regex ^http:\/\/[a-z]+[0-9]\.google\.co(m|\.id)
acl store_rewrite_list_domain_CDN url_regex ^http:\/\/\.www[0-9][0-9]\.indowebster\.com\/(.*)(rar|zip|flv|wm(a|v)|3gp|mp(4|3)|exe|msi|avi|(mp(e?g|a|e|1|2|3|4))|cab|exe)
acl dontrewrite url_regex redbot\.org \.php
acl getmethod method GET
##
refresh_pattern -i \.htm 120 50% 10080 reload-into-ims
refresh_pattern -i \.html 120 50% 10080 reload-into-ims
refresh_pattern ^http://*.facebook.com/* 720 100% 4320
refresh_pattern ^http://mail.yahoo.com/.* 720 100% 4320
refresh_pattern ^http://*.yahoo.*/.* 720 100% 4320
refresh_pattern ^http://*.yimg.*/.* 720 100% 4320
refresh_pattern ^http://*.gmail.*/.* 720 100% 4320
refresh_pattern ^http://*.google.*/.* 720 100% 4320
refresh_pattern ^http://*.kaskus.*/.* 720 100% 4320
refresh_pattern ^http://*.googlesyndication.*/.* 720 100% 4320
refresh_pattern ^http://*.plasa.*/.* 720 100% 4320
refresh_pattern ^http://*.telkom.*/.* 720 100% 4320
##

# 1 year = 525600 mins, 1 month = 43800 mins
refresh_pattern imeem.*\.flv  0 0% 0     override-lastmod override-expire
refresh_pattern \.rapidshare.*\/[0-9]*\/.*\/[^\/]*   161280    90%    161280 ignore-reload

refresh_pattern (get_video\?|videoplayback\?|videodownload\?|\.flv?)    10800 80% 10800 ignore-no-cache  ignore-private override-expire override-lastmod reload-into-ims
refresh_pattern (get_video\?|videoplayback\?id|videoplayback.*id|videodownload\?|\.flv?)    10800 80% 10800 ignore-no-cache  ignore-private override-expire override-lastmod reload-into-ims
#refresh_pattern -i (get_video\?|videoplayback\?id|videoplayback.*id||videodownload\?|\.flv?)       10800 80% 10800 ignore-no-cache  ignore-private override-expire override-lastmod reload-into-ims
refresh_pattern \.(ico|video-stats) 10800 80% 10800    override-expire ignore-reload ignore-no-cache  ignore-private ignore-auth override-lastmod
refresh_pattern \.etology\?                       10800 80% 10800    override-expire ignore-reload ignore-no-cache
refresh_pattern galleries\.video(\?|sz)               10800 80% 10800    override-expire ignore-reload ignore-no-cache
refresh_pattern brazzers\?                       10800 80% 10800    override-expire ignore-reload ignore-no-cache
refresh_pattern \.adtology\?                      10800 80% 10800    override-expire ignore-reload ignore-no-cache
#refresh_pattern ^.*(utm\.gif|ads\?|rmxads\.com|ad\.z5x\.net|bh\.contextweb\.com|bstats\.adbrite\.com|a1\.interclick\.com|ad\.trafficmp\.com|ads\.cubics\.com|ad\.xtendmedia\.com|\.googlesyndication\.com|advertising\.com|yieldmanager|game-advertising\.com|pixel\.quantserve\.com|adperium\.com|doubleclick\.net|adserving\.cpxinteractive\.com|syndication\.com|media.fastclick.net).* 10800 20% 10800 ignore-no-cache  ignore-private override-expire ignore-reload ignore-auth   negative-ttl=40320 max-stale=10
refresh_pattern ^.*safebrowsing.*google  10800 80% 10800 override-expire ignore-reload ignore-no-cache ignore-private ignore-auth
refresh_pattern ^http://((cbk|mt|khm|mlt)[0-9]?)\.google\.co(m|\.uk) 10800 80% 10800 override-expire ignore-reload   ignore-private
refresh_pattern ytimg\.com.*\.jpg                   10800 80% 10800    override-expire ignore-reload
refresh_pattern images\.friendster\.com.*\.(png|gif)           10800 80% 10800    override-expire ignore-reload
refresh_pattern garena\.com                                   10800 80% 10800     override-expire reload-into-ims
refresh_pattern photobucket.*\.(jp(e?g|e|2)|tiff?|bmp|gif|png)  10800 80% 10800     override-expire ignore-reload
refresh_pattern vid\.akm\.dailymotion\.com.*\.on2\?           10800 80% 10800 ignore-no-cache override-expire override-lastmod
refresh_pattern mediafire.com\/images.*\.(jp(e?g|e|2)|tiff?|bmp|gif|png)    10800 80% 10800 reload-into-ims override-expire ignore-private
refresh_pattern ^http:\/\/images|pics|thumbs[0-9]\.      10800 80% 10800 reload-into-ims ignore-no-cache  ignore-reload override-expire
refresh_pattern ^http:\/\/www.onemanga.com.*\/           10800 80% 10800 reload-into-ims ignore-no-cache  ignore-reload override-expire

# ANTI VIRUS
refresh_pattern guru.avg.com/.*\.(bin)                      10800 80% 10800 ignore-no-cache  ignore-reload  reload-into-ims
refresh_pattern (avgate|avira).*(idx|gz)$                           10800 80% 10800 ignore-no-cache  ignore-reload  reload-into-ims
refresh_pattern kaspersky.*\.avc$                                   10800 80% 10800 ignore-no-cache  ignore-reload  reload-into-ims
refresh_pattern kaspersky                                           10800 80% 10800 ignore-no-cache  ignore-reload  reload-into-ims
refresh_pattern update.nai.com/.*\.(gem|zip|mcs)                    10800 80% 10800 ignore-no-cache  ignore-reload  reload-into-ims
refresh_pattern ^http:\/\/liveupdate.symantecliveupdate.com.*\(zip)     10800 80% 10800 ignore-no-cache  ignore-reload  reload-into-ims

refresh_pattern windowsupdate.com/.*\.(cab|exe)             10800  80%  10800 ignore-no-cache  ignore-reload  reload-into-ims
refresh_pattern update.microsoft.com/.*\.(cab|exe)             10800  80%  10800 ignore-no-cache  ignore-reload  reload-into-ims
refresh_pattern download.microsoft.com/.*\.(cab|exe)             10800  80%  10800 ignore-no-cache  ignore-reload  reload-into-ims

#images facebook
refresh_pattern ((facebook.com)|(85.131.151.39)).*\.(jpg|png|gif)      10800 80% 10800 ignore-reload  override-expire ignore-no-cache
refresh_pattern -i \.fbcdn.net.*\.(jpg|gif|png|swf|mp3)                  10800 80% 10800 ignore-reload  override-expire ignore-no-cache
refresh_pattern  static\.ak\.fbcdn\.net*\.(jpg|gif|png)                  10800 80% 10800 ignore-reload  override-expire ignore-no-cache
refresh_pattern ^http:\/\/profile\.ak\.fbcdn.net*\.(jpg|gif|png)      10800 80% 10800 ignore-reload  override-expire ignore-no-cache

#banner IIX
refresh_pattern ^http:\/\/openx.*\.(jp(e?g|e|2)|gif|pn[pg]|swf|ico|css|tiff?) 10800 99999% 10800 reload-into-ims  ignore-reload override-expire ignore-no-cache
refresh_pattern ^http:\/\/ads(1|2|3).kompas.com.*\/           10800 99999% 10800 reload-into-ims  ignore-reload override-expire ignore-no-cache
refresh_pattern ^http:\/\/img.ads.kompas.com.*\/           10800 99999% 10800 reload-into-ims  ignore-reload override-expire ignore-no-cache
refresh_pattern .kompasimages.com.*\.(jpg|gif|png|swf)       10800 99999% 10800 reload-into-ims  ignore-reload override-expire ignore-no-cache
refresh_pattern ^http:\/\/openx.kompas.com.*\/           10800 99999% 10800 reload-into-ims  ignore-reload override-expire ignore-no-cache
refresh_pattern kaskus.\us.*\.(jp(e?g|e|2)|gif|png|swf)        10800 99999% 10800 reload-into-ims  ignore-reload override-expire ignore-no-cache
refresh_pattern ^http:\/\/img.kaskus.us.*\.(jpg|gif|png|swf)       10800 99999% 10800 reload-into-ims  ignore-reload override-expire ignore-no-cache

#IIX DOWNLOAD
refresh_pattern ^http:\/\/\.www[0-9][0-9]\.indowebster\.com\/(.*)(mp3|rar|zip|flv|wmv|3gp|mp(4|3)|exe|msi|zip) 10800 99999% 10800 reload-into-ims  ignore-reload override-expire ignore-no-cache    ignore-auth

#All File
refresh_pattern -i \.(3gp|7z|ace|asx|avi|bin|cab|dat|deb|divx|dvr-ms)      10800 80% 10800 ignore-no-cache  ignore-private override-expire override-lastmod reload-into-ims
refresh_pattern -i \.(rar|jar|gz|tgz|bz2|iso|m1v|m2(v|p)|mo(d|v))          10800 80% 10800 ignore-no-cache  ignore-private override-expire override-lastmod reload-into-ims
refresh_pattern -i \.(jp(e?g|e|2)|gif|pn[pg]|bm?|tiff?|ico|swf|css|js)     10800 80% 10800 ignore-no-cache  ignore-private override-expire override-lastmod reload-into-ims
refresh_pattern -i \.(mp(e?g|a|e|1|2|3|4)|mk(a|v)|ms(i|u|p)|og(x|v|a|g)|rar|rm|r(a|p)m|snd|vob|wav) 10800 80% 10800 ignore-no-cache ignore-private override-expire override-lastmod reload-into-ims
refresh_pattern -i \.(pp(s|t)|wax|wm(a|v)|wmx|wpl|zip|cb(r|z|t))     10800 80% 10800 ignore-no-cache ignore-private override-expire override-lastmod reload-into-ims

#########################################################################

refresh_pattern (cgi-bin|\?)       0      0%      0
refresh_pattern ^gopher:    1440    0%    1440
refresh_pattern ^ftp:         10080     95%     10800 override-lastmod reload-into-ims
refresh_pattern         .     180     95% 10800 override-lastmod reload-into-ims

global_internal_static off
retry_on_error on
buffered_logs on
read_ahead_gap 32 KB

#header_access Accept-Encoding deny  all
client_persistent_connections off
server_persistent_connections on
half_closed_clients off
strip_query_terms off
quick_abort_min 0 KB
quick_abort_max 0 KB
quick_abort_pct 100
vary_ignore_expire on
reload_into_ims on
pipeline_prefetch on
read_timeout 30 minute
client_lifetime 6 hour
positive_dns_ttl 6 hour
pconn_timeout 15 second
request_timeout 1 minute
log_icp_queries off
ipcache_size 16384
ipcache_low 98
ipcache_high 99
log_fqdn off
fqdncache_size 16384
memory_pools off
forwarded_for on
client_db off
max_filedescriptors 8192

if you are using SQUID 3.1.19 (Ubuntu 12.4 default isntallation) then use following, pretty much same hmmm 🙂

SQUID 3.1.19 CONFIG FILE in /etc/squid3/squid.conf

# SQUID 3.1.19 CONFIG FILE WITH VIDEOCACHE PLUGIN FOR SQUID/VIDEOS [ Zaib]
# SQUID 3.1.19 WORKING CONFIG FILE [August 2013] with Videocache Support
# By - Syed Jahanzaib
# Email: aacable@hotmail.com
# Web  : https://aacable.wordpress.com

cache_store_log /var/log/squid3/store.log
acl this_machine src 127.0.0.1 192.168.100.1 # Change this ip to match your's
http_access allow this_machine

# --BEGIN-- videocache config for squid
#
# PASTE THIS FILE AT THE TOP OF YOUR SQUID CONFIGURATION FILE
#
# (C) Copyright White Magnet Software Private Limited
# Company Website : http://whitemagnet.com/
# Product Website : http://cachevideos.com/
#
# Videocache Version : Trial
#
# This configuration is for Squid versions 3.x ZABBO
url_rewrite_program /usr/bin/python /usr/share/videocache/videocache.py
url_rewrite_children 20

acl vc_deny_url url_regex -i \.blip\.tv\/(.*)filename \.hardsextube\.com\/videothumbs \.xtube\.com\/(.*)(Thumb|videowall)

acl vc_url url_regex -i \/youku\/[0-9A-Z]+\/[0-9A-Z\-]+\.(flv|mp4|avi|mkv|mp3|rm|rmvb|m4v|mov|wmv|3gp|mpg|mpeg)
acl vc_url url_regex -i \/(.*)key=[a-z0-9]+(.*)\.flv
acl vc_url url_regex -i \.(youtube|youtube-nocookie)\.com\/feeds\/api\/videos\/[0-9a-zA-Z_-]{11}\/
acl vc_url url_regex -i \.(youtube|youtube-nocookie)\.com\/(videoplayback|get_video|watch|watch_popup|user_watch)\?
acl vc_url url_regex -i \.(youtube|youtube-nocookie)\.com\/(v|e|embed)\/[0-9a-zA-Z_-]{11}

acl vc_dom_r dstdom_regex -i msn\..*\.(com|net)
acl vc_dom_r dstdom_regex -i msnbc\..*\.(com|net)
acl vc_dom_r dstdom_regex -i video\..*\.fbcdn\.net
acl vc_dom_r dstdom_regex -i myspacecdn\..*\.footprint\.net

acl vc_dom dstdomain .stream.aol.com .videos.5min.com msn.com .blip.tv .vid.ec.dmcdn.net .break.com .vimeocdn.com
acl vc_dom dstdomain .cdn.turner.com .dailymotion.com .c.wrzuta.pl .v.imwx.com .mccont.com .myspacecdn.com
acl vc_dom dstdomain .hardsextube.com .public.extremetube.phncdn.com .redtubefiles.com .video.pornhub.phncdn.com
acl vc_dom dstdomain .public.keezmovies.com .public.keezmovies.phncdn.com .slutload-media.com .public.spankwire.com .xtube.com
acl vc_dom dstdomain .public.youporn.phncdn.com .xvideos.com .tube8.com .public.spankwire.phncdn.com .pornhub.com

refresh_pattern \.youtube\.com\/videoplayback\? 120    80%    180 ignore-no-cache ignore-no-store override-expire override-lastmod ignore-private
refresh_pattern stream\.aol\.com\/(.*)/[a-zA-Z0-9]+\/(.*)\.(flv|mp4) 120    80%    180 ignore-no-cache ignore-no-store override-expire override-lastmod ignore-private
refresh_pattern videos\.5min\.com\/(.*)/[0-9_]+\.(mp4|flv) 120    80%    180 ignore-no-cache ignore-no-store override-expire override-lastmod ignore-private
refresh_pattern proxy[a-z0-9\-]?[a-z0-9]?[a-z0-9]?[a-z0-9]?\.dailymotion\.com\/(.*)\.(flv|on2|mp4|avi|mkv|mp3|rm|rmvb|m4v|mov|wmv|3gp|mpg|mpeg) 120    80%    180 ignore-no-cache ignore-no-store override-expire override-lastmod ignore-private
refresh_pattern vid\.akm\.dailymotion\.com\/(.*)\.(flv|on2|mp4|avi|mkv|mp3|rm|rmvb|m4v|mov|wmv|3gp|mpg|mpeg) 120    80%    180 ignore-no-cache ignore-no-store override-expire override-lastmod ignore-private
refresh_pattern vid\.ec\.dmcdn\.net\/(.*)\.(flv|on2|mp4|avi|mkv|mp3|rm|rmvb|m4v|mov|wmv|3gp|mpg|mpeg) 120    80%    180 ignore-no-cache ignore-no-store override-expire override-lastmod ignore-private
refresh_pattern video\.(.*)\.fbcdn\.net\/(.*)/[0-9_]+\.(mp4|flv|avi|mkv|m4v|mov|wmv|3gp|mpg|mpeg) 120    80%    180 ignore-no-cache ignore-no-store override-expire override-lastmod ignore-private
refresh_pattern (.*)\.myspacecdn\.com\/(.*)\/[a-zA-Z0-9]+\/vid\.(flv|mp4|avi|mkv|mp3|rm|rmvb|m4v|mov|wmv|3gp|mpg|mpeg) 120    80%    180 ignore-no-cache ignore-no-store override-expire override-lastmod ignore-private
refresh_pattern (.*)\.myspacecdn\.(.*)\.footprint\.net\/(.*)\/[a-zA-Z0-9]+\/vid\.(flv|mp4|avi|mkv|mp3|rm|rmvb|m4v|mov|wmv|3gp|mpg|mpeg) 120    80%    180 ignore-no-cache ignore-no-store override-expire override-lastmod ignore-private
refresh_pattern c\.wrzuta\.pl\/wv[0-9]+\/[a-z0-9]+/[0-9]+/ 120    80%    180 ignore-no-cache ignore-no-store override-expire override-lastmod ignore-private
refresh_pattern c\.wrzuta\.pl\/wa[0-9]+\/[a-z0-9]+ 120    80%    180 ignore-no-cache ignore-no-store override-expire override-lastmod ignore-private
refresh_pattern vs[a-z0-9]?[a-z0-9]?[a-z0-9]?\.hardsextube\.com\/(.*)\/(.*)\.(flv|mp4|avi|mkv|mp3|rm|rmvb|m4v|mov|wmv|3gp|mpg|mpeg) 120    80%    180 ignore-no-cache ignore-no-store override-expire override-lastmod ignore-private

acl vc_deny_url url_regex -i crossdomain.xml
acl vc_method method GET
acl vc_header req_header X-Requested-With -i videocache
url_rewrite_access deny !vc_method
url_rewrite_access deny vc_header
url_rewrite_access deny vc_deny_url
url_rewrite_access allow vc_dom
url_rewrite_access allow vc_url
url_rewrite_access allow vc_dom_r
url_rewrite_bypass on
#These lines are needed.
acl this_machine src 127.0.0.1
http_access allow this_machine
strip_query_terms off
maximum_object_size 10 GB
# --END-- videocache config for squid

# SQUID 3.x CONFIG FILE

# PORT and Transparent Option
#http_port 8080
http_port 8080 transparent
icp_port 0

store_dir_select_algorithm round-robin

# Cache Direcotries on external Storage, for sure you should change it
# Each folder is of 1.5 TB in space
cache_dir aufs /cache-1/squid-cache-1 1500000 16 256
cache_dir aufs /cache-2/squid-cache-2 15000000 16 256
cache_dir aufs /cache-3/squid-cache-3 15000000 16 256
cache_dir aufs /cache-4/squid-cache-4 15000000 16 256
cache_dir aufs /cache-5/squid-cache-5 15000000 16 256
cache_dir aufs /cache-6/squid-cache-6 15000000 16 256
cache_dir aufs /cache-7/squid-cache-7 15000000 16 256
cache_dir aufs /cache-8/squid-cache-8 15000000 16 256
cache_dir aufs /cache-9/squid-cache-9 15000000 16 256
cache_dir aufs /cache-10/squid-cache-10 15000000 16 256
cache_dir aufs /cache-11/squid-cache-11 15000000 16 256
cache_dir aufs /cache-12/squid-cache-12 1500000 16 256

#cache_dir ufs /mnt/hdd2/cache2 200000 16 256 # If you have secondary HDD
memory_replacement_policy heap LFUDA
cache_replacement_policy heap GDSF

# If you want to enable DATE time n SQUID Logs,use following
emulate_httpd_log on
logformat squid %tl %6tr %>a %Ss/%03Hs %<st %rm %ru %un %Sh/%<A %mt
log_fqdn off

# How much days to keep users access web logs
# You need to rotate your log files with a cron job. For example:
# 0 0 * * * /usr/local/squid/bin/squid -k rotate
logfile_rotate 14
debug_options ALL,1
cache_access_log /var/log/squid3/access.log
cache_log  /var/log/squid3/cache_log.log
cache_store_log /var/log/squid3/store.log

# Block Ads [zaib]
#acl adsites dstdomain url_regex "/etc/squid/adslist.txt"
#http_access deny adsites
#deny_info http://192.168.6.1/psb.htm adsites

#I used DNSAMSQ service for fast dns resolving
#so install by using "apt-get install dnsmasq" first
dns_nameservers 127.0.0.1 8.8.8.8
ftp_user anonymous@
ftp_list_width 32
ftp_passive on
ftp_sanitycheck on

#ACL Section mylan myacl
acl all src 0.0.0.0/0.0.0.0 # ALLOW ALL, you may want to change this /zaib
acl manager proto cache_object
acl localhost src 127.0.0.1/255.255.255.255
acl to_localhost dst 127.0.0.0/8
acl SSL_ports port 443 563 # https, snews
acl SSL_ports port 873 # rsync
acl Safe_ports port 80 # http
acl Safe_ports port 21 # ftp
acl Safe_ports port 443 563 # https, snews
acl Safe_ports port 70 # gopher
acl Safe_ports port 210 # wais
acl Safe_ports port 1025-65535 # unregistered ports
acl Safe_ports port 280 # http-mgmt
acl Safe_ports port 488 # gss-http
acl Safe_ports port 591 # filemaker
acl Safe_ports port 777 # multiling http
acl Safe_ports port 631 # cups
acl Safe_ports port 873 # rsync
acl Safe_ports port 901 # SWAT
acl purge method PURGE
acl CONNECT method CONNECT
http_access allow manager all
http_access deny manager
http_access allow purge localhost
http_access deny purge
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
http_access allow localhost
http_access allow all
#http_access allow all2
http_reply_access allow all
#http_reply_access allow all2
icp_access allow all

#==========================
# Administrative Parameters
#==========================

#============================================================$
# SNMP , if you want to generate graphs for SQUID via MRTG
#============================================================$
#acl snmppublic snmp_community zaib
#snmp_port 3401
#snmp_access allow snmppublic all
#snmp_access allow all

# I used UBUNTU so user is proxy, in FEDORA you may use use squid
cache_effective_user proxy
cache_effective_group proxy
cache_mgr SYED_JAHAN_ZAIB
visible_hostname aacable.wordpress.com
unique_hostname aacable@hotmail.com

# Memory, in this example, teh squid box have 32 B of RAM
cache_mem 8 GB # Change it as per your hardware config, you may want to lower it down
minimum_object_size 0 bytes
maximum_object_size_in_memory 5 MB # Change it as per your hardware config, you may want to lower it down if you have low ram

# 1 year = 525600 mins, 1 month = 43800 mins
refresh_pattern imeem.*\.flv  0 0% 0     override-lastmod override-expire
refresh_pattern \.rapidshare.*\/[0-9]*\/.*\/[^\/]*   161280    90%    161280 ignore-reload

refresh_pattern (get_video\?|videoplayback\?|videodownload\?|\.flv?)    10800 80% 10800 ignore-no-cache  ignore-private override-expire override-lastmod reload-into-ims
refresh_pattern (get_video\?|videoplayback\?id|videoplayback.*id|videodownload\?|\.flv?)    10800 80% 10800 ignore-no-cache  ignore-private override-expire override-lastmod reload-into-ims
#refresh_pattern -i (get_video\?|videoplayback\?id|videoplayback.*id||videodownload\?|\.flv?)       10800 80% 10800 ignore-no-cache  ignore-private override-expire override-lastmod reload-into-ims
refresh_pattern \.(ico|video-stats) 10800 80% 10800    override-expire ignore-reload ignore-no-cache  ignore-private ignore-auth override-lastmod
refresh_pattern \.etology\?                       10800 80% 10800    override-expire ignore-reload ignore-no-cache
refresh_pattern galleries\.video(\?|sz)               10800 80% 10800    override-expire ignore-reload ignore-no-cache
refresh_pattern brazzers\?                       10800 80% 10800    override-expire ignore-reload ignore-no-cache
refresh_pattern \.adtology\?                      10800 80% 10800    override-expire ignore-reload ignore-no-cache
#refresh_pattern ^.*(utm\.gif|ads\?|rmxads\.com|ad\.z5x\.net|bh\.contextweb\.com|bstats\.adbrite\.com|a1\.interclick\.com|ad\.trafficmp\.com|ads\.cubics\.com|ad\.xtendmedia\.com|\.googlesyndication\.com|advertising\.com|yieldmanager|game-advertising\.com|pixel\.quantserve\.com|adperium\.com|doubleclick\.net|adserving\.cpxinteractive\.com|syndication\.com|media.fastclick.net).* 10800 20% 10800 ignore-no-cache  ignore-private override-expire ignore-reload ignore-auth   negative-ttl=40320 max-stale=10
refresh_pattern ^.*safebrowsing.*google  10800 80% 10800 override-expire ignore-reload ignore-no-cache ignore-private ignore-auth
refresh_pattern ^http://((cbk|mt|khm|mlt)[0-9]?)\.google\.co(m|\.uk) 10800 80% 10800 override-expire ignore-reload   ignore-private
refresh_pattern ytimg\.com.*\.jpg                   10800 80% 10800    override-expire ignore-reload
refresh_pattern images\.friendster\.com.*\.(png|gif)           10800 80% 10800    override-expire ignore-reload
refresh_pattern garena\.com                                   10800 80% 10800     override-expire reload-into-ims
refresh_pattern photobucket.*\.(jp(e?g|e|2)|tiff?|bmp|gif|png)  10800 80% 10800     override-expire ignore-reload
refresh_pattern vid\.akm\.dailymotion\.com.*\.on2\?           10800 80% 10800 ignore-no-cache override-expire override-lastmod
refresh_pattern mediafire.com\/images.*\.(jp(e?g|e|2)|tiff?|bmp|gif|png)    10800 80% 10800 reload-into-ims override-expire ignore-private
refresh_pattern ^http:\/\/images|pics|thumbs[0-9]\.      10800 80% 10800 reload-into-ims ignore-no-cache  ignore-reload override-expire
refresh_pattern ^http:\/\/www.onemanga.com.*\/           10800 80% 10800 reload-into-ims ignore-no-cache  ignore-reload override-expire

# ANTI VIRUS
refresh_pattern guru.avg.com/.*\.(bin)                      10800 80% 10800 ignore-no-cache  ignore-reload  reload-into-ims
refresh_pattern (avgate|avira).*(idx|gz)$                           10800 80% 10800 ignore-no-cache  ignore-reload  reload-into-ims
refresh_pattern kaspersky.*\.avc$                                   10800 80% 10800 ignore-no-cache  ignore-reload  reload-into-ims
refresh_pattern kaspersky                                           10800 80% 10800 ignore-no-cache  ignore-reload  reload-into-ims
refresh_pattern update.nai.com/.*\.(gem|zip|mcs)                    10800 80% 10800 ignore-no-cache  ignore-reload  reload-into-ims
refresh_pattern ^http:\/\/liveupdate.symantecliveupdate.com.*\(zip)     10800 80% 10800 ignore-no-cache  ignore-reload  reload-into-ims

refresh_pattern windowsupdate.com/.*\.(cab|exe)             10800  80%  10800 ignore-no-cache  ignore-reload  reload-into-ims
refresh_pattern update.microsoft.com/.*\.(cab|exe)             10800  80%  10800 ignore-no-cache  ignore-reload  reload-into-ims
refresh_pattern download.microsoft.com/.*\.(cab|exe)             10800  80%  10800 ignore-no-cache  ignore-reload  reload-into-ims

#images facebook
refresh_pattern ((facebook.com)|(85.131.151.39)).*\.(jpg|png|gif)      10800 80% 10800 ignore-reload  override-expire ignore-no-cache
refresh_pattern -i \.fbcdn.net.*\.(jpg|gif|png|swf|mp3)                  10800 80% 10800 ignore-reload  override-expire ignore-no-cache
refresh_pattern  static\.ak\.fbcdn\.net*\.(jpg|gif|png)                  10800 80% 10800 ignore-reload  override-expire ignore-no-cache
refresh_pattern ^http:\/\/profile\.ak\.fbcdn.net*\.(jpg|gif|png)      10800 80% 10800 ignore-reload  override-expire ignore-no-cache

#banner IIX
refresh_pattern ^http:\/\/openx.*\.(jp(e?g|e|2)|gif|pn[pg]|swf|ico|css|tiff?) 10800 99999% 10800 reload-into-ims  ignore-reload override-expire ignore-no-cache
refresh_pattern ^http:\/\/ads(1|2|3).kompas.com.*\/           10800 99999% 10800 reload-into-ims  ignore-reload override-expire ignore-no-cache
refresh_pattern ^http:\/\/img.ads.kompas.com.*\/           10800 99999% 10800 reload-into-ims  ignore-reload override-expire ignore-no-cache
refresh_pattern .kompasimages.com.*\.(jpg|gif|png|swf)       10800 99999% 10800 reload-into-ims  ignore-reload override-expire ignore-no-cache
refresh_pattern ^http:\/\/openx.kompas.com.*\/           10800 99999% 10800 reload-into-ims  ignore-reload override-expire ignore-no-cache
refresh_pattern kaskus.\us.*\.(jp(e?g|e|2)|gif|png|swf)        10800 99999% 10800 reload-into-ims  ignore-reload override-expire ignore-no-cache
refresh_pattern ^http:\/\/img.kaskus.us.*\.(jpg|gif|png|swf)       10800 99999% 10800 reload-into-ims  ignore-reload override-expire ignore-no-cache

#IIX DOWNLOAD
refresh_pattern ^http:\/\/\.www[0-9][0-9]\.indowebster\.com\/(.*)(mp3|rar|zip|flv|wmv|3gp|mp(4|3)|exe|msi|zip) 10800 99999% 10800 reload-into-ims  ignore-reload override-expire ignore-no-cache    ignore-auth

#All File
refresh_pattern -i \.(3gp|7z|ace|asx|avi|bin|cab|dat|deb|divx|dvr-ms)      10800 80% 10800 ignore-no-cache  ignore-private override-expire override-lastmod reload-into-ims
refresh_pattern -i \.(rar|jar|gz|tgz|bz2|iso|m1v|m2(v|p)|mo(d|v))          10800 80% 10800 ignore-no-cache  ignore-private override-expire override-lastmod reload-into-ims
refresh_pattern -i \.(jp(e?g|e|2)|gif|pn[pg]|bm?|tiff?|ico|swf|css|js)     10800 80% 10800 ignore-no-cache  ignore-private override-expire override-lastmod reload-into-ims
refresh_pattern -i \.(mp(e?g|a|e|1|2|3|4)|mk(a|v)|ms(i|u|p)|og(x|v|a|g)|rar|rm|r(a|p)m|snd|vob|wav) 10800 80% 10800 ignore-no-cache ignore-private override-expire override-lastmod reload-into-ims
refresh_pattern -i \.(pp(s|t)|wax|wm(a|v)|wmx|wpl|zip|cb(r|z|t))     10800 80% 10800 ignore-no-cache ignore-private override-expire override-lastmod reload-into-ims

#########################################################################

refresh_pattern (cgi-bin|\?)       0      0%      0
refresh_pattern ^gopher:    1440    0%    1440
refresh_pattern ^ftp:         10080     95%     10800 override-lastmod reload-into-ims
refresh_pattern         .     180     95% 10800 override-lastmod reload-into-ims

global_internal_static off
retry_on_error on
buffered_logs on
read_ahead_gap 32 KB

#header_access Accept-Encoding deny  all
client_persistent_connections off
server_persistent_connections on
half_closed_clients off
strip_query_terms off
quick_abort_min 0 KB
quick_abort_max 0 KB
quick_abort_pct 100
vary_ignore_expire on
reload_into_ims on
pipeline_prefetch on
read_timeout 30 minute
client_lifetime 6 hour
positive_dns_ttl 6 hour
pconn_timeout 15 second
request_timeout 1 minute
log_icp_queries off
ipcache_size 16384
ipcache_low 98
ipcache_high 99
log_fqdn off
fqdncache_size 16384
memory_pools off
forwarded_for on
client_db off
max_filedescriptors 8192

# ZAIB
# ZPH for Squid 3.3.19
qos_flows local-hit=0x30

After videocache is installed, following is the fine tunned config file. Don’t forget to change the host ip (192.168.100.1) to match your own server ip

VIDEO CACHE CONFIG FILE in /etc/videocache.conf

[main]

# file : /etc/videocache.conf
# (C) Copyright White Magnet Software Private Limited
# Company Website : http://whitemagnet.com/
# Product Website : http://cachevideos.com/

# WORKING SAMPLE FILE [JULY 2013] - Syed Jahanzaib
# Email: aacable@hotmail.com
# Web  : https://aacable.wordpress.com

# version : cxxxxxxb

client_email = your_registered_emai_address@hotmail.com
cache_host = 192.168.100.1
videocache_user = proxy

# This option is used to define the cache directories where Videocache will store
# cached videos. We can specify multiple caching directories separated by ‘|’.
# Examples:
#   base_dir = /videocache/
#   base_dir = /cache/videocache2/ | /hdd2/videocache/ | /home/videocache/
base_dir = /cache1/vc/

disk_avail_threshold = 15000
disk_cleanup_strategy = 1
enable_videocache = 1
offline_mode = 0
base_dir_selection = 2
enable_store_log_monitoring = 1
squid_store_log = /var/log/squid3/store.log
this_proxy = 127.0.0.1:8080
hit_threshold = 1
max_cache_processes = 10
max_cache_speed = 0
max_cache_queue_size = 100000
cache_period =
proxy =
proxy_username =
proxy_password =
max_video_size = 0
min_video_size = 0
force_video_size = 1
scheduler_pidfile = /var/run/videocache.pid
logdir = /var/log/videocache/
logformat = %tl %p %s %i %w %c %v %m %d
scheduler_logformat = %tl %p %s %i %w %c %v %m %d
cleaner_logformat = %tl %p %s %w %c %v %m %d
timeformat = %d/%b/%Y:%H:%M:%S
enable_videocache_log = 1
enable_scheduler_log = 1
enable_cleaner_log = 1
enable_trace_log = 1
logfile = videocache.log
scheduler_logfile = scheduler.log
cleaner_logfile = cleaner.log
tracefile = trace.log
max_logfile_size = 10
max_scheduler_logfile_size = 10
max_cleaner_logfile_size = 10
max_tracefile_size = 10
max_logfile_backups = 10
max_scheduler_logfile_backups = 5
max_cleaner_logfile_backups = 1
max_tracefile_backups = 1

enable_youtube_cache = 1

# This options determines if Videocache will cache different YouTube video
# formats separately. Please select an appropriate algorithm from the listed below.
# Available strategies:
#   1 : (disabled) Don't check for YouTube video formats. Cache one of the formats
#       and serve it for requests for all kinds of formats.
#   2 : (strict) Strictly check for YouTube formats and cache all formats separately.
#       Consumes maximum bandwidth.
#   3 : (approximate) Check YouTube formats but with approximation. For example,
#       if a client asked for a video in 480p format and we already have 360p
#       format of the same video in cache, then serve 360p format and vice-versa.
# Default : 3
enable_youtube_format_support = 3
enable_youtube_html5_videos = 1
enable_youtube_3d_videos = 1
enable_youtube_partial_caching = 1
max_youtube_video_quality = 720p
min_youtube_views = 0
enable_aol_cache = 1
enable_bing_cache = 1
enable_bliptv_cache = 1
enable_breakcom_cache = 1
enable_cnn_cache = 1
enable_dailymotion_cache = 1
enable_facebook_cache = 1
enable_megavideo_cache = 1
enable_metacafe_cache = 1
enable_myspace_cache = 1
enable_vimeo_cache = 1
enable_weather_cache = 1
enable_wrzuta_cache = 1
enable_youku_cache = 1
enable_extremetube_cache = 1
enable_hardsextube_cache = 1
enable_keezmovies_cache = 1
enable_pornhub_cache = 1
enable_redtube_cache = 1
enable_slutload_cache = 1
enable_spankwire_cache = 1
enable_tube8_cache = 1
enable_xhamster_cache = 1
enable_xtube_cache = 1
enable_xvideos_cache = 1
enable_youporn_cache = 1

skip_apache_conf = 0
apache_conf_dir = /etc/apache2/conf.d/
hide_cache_dirs = 1
rpc_host = 127.0.0.1
rpc_port = 9100


To restart Videocache and related services, use following commands

vc-update

    service apache2 restart

    vc-scheduler -s restart

    service squid3 restart

To view Video cache Logs in live for info and troubleshooting

tail -f /var/log/videocache/videocache.log

tail -f /var/log/videocache/scheduler.log

tail -f /var/log/squid3/access.log

Sample image of logs at live server

VC-CACHE

VC-CACHE

☼☼☼☼☼☼☼☼

Regard’s
Syed Jahanzaib

%d bloggers like this: