Syed Jahanzaib Personal Blog to Share Knowledge !

January 30, 2012

Youtube Caching Problem : An error occured. Please try again later. [SOLVED] updated storeurl.pl


YOUTUBE CACHING WITH SQUID -by- Syed Jahanzaib

YOUTUBE CACHING WITH SQUID -by- Syed Jahanzaib

The last updated guide 22nd April, 2014 about youtube caching is as follows …

https://aacable.wordpress.com/2014/04/21/howto-cache-youtube-with-squid-lusca-and-bypass-cached-videos-from-mikrotik-queue/


 

=========================================================================================================

Last updated > 20th January, 2014

If you are caching YouTube using storeurl.pl method

EXAMPLE:

https://aacable.wordpress.com/2014/04/21/howto-cache-youtube-with-squid-lusca-and-bypass-cached-videos-from-mikrotik-queue/

https://aacable.wordpress.com/2012/01/19/youtube-caching-with-squid-2-7-using-storeurl-pl/ 

and you encounter following error while watching any cached video  “An error occured, Please try again later
As showed in the image below . . .

Then try using the following January, 2014 updated storeurl.pl

#!/usr/bin/perl
# This script is NOT written or modified by me, I only copy pasted it from the internet.
# It was First originally Written by chudy_fernandez@yahoo.com
# & Have been modified by various persons over the net to fix/add various functions.
# Like For Example modified by member of comstuff.net to satisfy common and dynamic content.
# th30nly @comstuff.net a.k.a invisible_theater , and possibly other people too.
# For more info, http://wiki.squid-cache.org/ConfigExamples/DynamicContent/YouTube
# Syed Jahanzaib / aacable@hotmail.com
# https://aacable.wordpress.com/2012/01/19/youtube-caching-with-squid-2-7-using-storeurl-pl/

#####################
#### REFERENCES #####  http://www2.fh-lausitz.de/launic/comp/misc/squid/projekt_youtube/
#####################
#####################

## storeurl.pl-130411-0.05
## s. 130411.howto.squid.youtube.html
## CHANGES
## 130411 hl+mf
## - fix youtube loop redir
## - rel. 0.05
## 130409 hl
## - add usleep
## 120726 hl
## - add printtimenow
## 120725 hl
## - remove html-tags
## - add debug
## - disable unused rules
## - rel. 0.04
## 120130 aacable
## - fix match variable.domain.com
##   https://aacable.wordpress.com/2012/01/30/youtube-caching-problem-an-error-occured-please-try-again-later-solved/
## 120111 aacable
##   https://aacable.wordpress.com/2012/01/11/howto-cache-youtube-with-squid-lusca-and-bypass-cached-videos-from-mikrotik-queue/

#### notes
## - config (s. 130411.howto.squid.youtube.html)
##   squid.conf:storeurl_rewrite_program /etc/squid/storeurl.pl
## - test
##   $ cat squid.diag.youtube.url | awk '{print "0",$7,"10.0.0.1/- - GET - myip=10.0.0.2 myport=3128"}' | ./storeurl.pl
#
#
################################################################################################
## UPDATED on 20 January, 2014 **** thanks to Mr. Safatah Purwonoto [Indonesia] / Syed Jahanzaib
################################################################################################

#### var

use IO::File;
$|=1;
STDOUT->autoflush(1);
$debug=1;        ## recommended:0
$bypassallrules=0;    ## recommended:0
$sucks="";        ## unused
$sucks="sucks" if ($debug>=1);
$timenow="";
$printtimenow=1;      ## print timenow: 0|1
my $logfile = '/tmp/storeurl.log';

open my $logfh, '>>', $logfile
or die "Couldn't open $logfile for appending: $!\n" if $debug;
$logfh->autoflush(1) if $debug;

#### main
while (<>) {
$timenow=time()." " if ($printtimenow);
print $logfh "$timenow"."in : $_" if ($debug>=1);
chop; ## strip eol
@X = split;
$x = $X[0]; ## 0
$u = $X[1]; ## url
$_ = $u; ## url

if ($bypassallrules){
$out="$u";    ## map 1:1

#youtube with range (YOUTUBE has split its videos into segments)
}elsif (m/(youtube|google).*videoplayback\?.*range/ ){
@itag = m/[&?](itag=[0-9]*)/;
@id = m/[&?](id=[^\&]*)/;
@range = m/[&?](range=[^\&\s]*)/;
@begin = m/[&?](begin=[^\&\s]*)/;
@redirect = m/[&?](redirect_counter=[^\&]*)/;
$out="http://video-srv.youtube.com.SQUIDINTERNAL/@itag&@id&@range&@redirect";
#sleep(1);    ## delay loop

#youtube without range
}elsif (m/(youtube|google).*videoplayback\?/ ){
@itag = m/[&?](itag=[0-9]*)/;
@id = m/[&?](id=[^\&]*)/;
@redirect = m/[&?](redirect_counter=[^\&]*)/;
$out="http://video-srv.youtube.com.SQUIDINTERNAL/@itag&@id&@redirect";
#sleep(1);    ## delay loop

#speedtest
}elsif (m/^http:\/\/(.*)\/speedtest\/(.*\.(jpg|txt))\?(.*)/) {
$out="http://www.speedtest.net.SQUIDINTERNAL/speedtest/" . $2 . "";

#mediafire
}elsif (m/^http:\/\/199\.91\.15\d\.\d*\/\w{12}\/(\w*)\/(.*)/) {
$out="http://www.mediafire.com.SQUIDINTERNAL/" . $1 ."/" . $2 . "";

#fileserve
}elsif (m/^http:\/\/fs\w*\.fileserve\.com\/file\/(\w*)\/[\w-]*\.\/(.*)/) {
$out="http://www.fileserve.com.SQUIDINTERNAL/" . $1 . "./" . $2 . "";

#filesonic
}elsif (m/^http:\/\/s[0-9]*\.filesonic\.com\/download\/([0-9]*)\/(.*)/) {
$out="http://www.filesonic.com.SQUIDINTERNAL/" . $1 . "";

#4shared
}elsif (m/^http:\/\/[a-zA-Z]{2}\d*\.4shared\.com(:8080|)\/download\/(.*)\/(.*\..*)\?.*/) {
$out="http://www.4shared.com.SQUIDINTERNAL/download/$2\/$3";

#4shared preview
}elsif (m/^http:\/\/[a-zA-Z]{2}\d*\.4shared\.com(:8080|)\/img\/(\d*)\/\w*\/dlink__2Fdownload_2F(\w*)_3Ftsid_3D[\w-]*\/preview\.mp3\?sId=\w*/) {
$out="http://www.4shared.com.SQUIDINTERNAL/$2";

#photos-X.ak.fbcdn.net where X a-z
}elsif (m/^http:\/\/photos-[a-z](\.ak\.fbcdn\.net)(\/.*\/)(.*\.jpg)/) {
$out="http://photos" . $1 . "/" . $2 . $3  . "";

#YX.sphotos.ak.fbcdn.net where X 1-9, Y a-z
} elsif (m/^http:\/\/[a-z][0-9]\.sphotos\.ak\.fbcdn\.net\/(.*)\/(.*)/) {
$out="http://photos.ak.fbcdn.net/" . $1  ."/". $2 . "";

#maps.google.com
} elsif (m/^http:\/\/(cbk|mt|khm|mlt|tbn)[0-9]?(.google\.co(m|\.uk|\.id).*)/) {
$out="http://" . $1  . $2 . "";

# compatibility for old cached get_video?video_id
} elsif (m/^http:\/\/([0-9.]{4}|.*\.youtube\.com|.*\.googlevideo\.com|.*\.video\.google\.com).*?(videoplayback\?id=.*?|video_id=.*?)\&(.*?)/) {
$z = $2; $z =~ s/video_id=/get_video?video_id=/;
$out="http://video-srv.youtube.com.SQUIDINTERNAL/" . $z . "";
#sleep(1);    ## delay loop

} elsif (m/^http:\/\/www\.google-analytics\.com\/__utm\.gif\?.*/) {
$out="http://www.google-analytics.com/__utm.gif";

#Cache High Latency Ads
} elsif (m/^http:\/\/([a-z0-9.]*)(\.doubleclick\.net|\.quantserve\.com|\.googlesyndication\.com|yieldmanager|cpxinteractive)(.*)/) {
$y = $3;$z = $2;
for ($y) {
s/pixel;.*/pixel/;
s/activity;.*/activity/;
s/(imgad[^&]*).*/\1/;
s/;ord=[?0-9]*//;
s/;&timestamp=[0-9]*//;
s/[&?]correlator=[0-9]*//;
s/&cookie=[^&]*//;
s/&ga_hid=[^&]*//;
s/&ga_vid=[^&]*//;
s/&ga_sid=[^&]*//;
# s/&prev_slotnames=[^&]*//
# s/&u_his=[^&]*//;
s/&dt=[^&]*//;
s/&dtd=[^&]*//;
s/&lmt=[^&]*//;
s/(&alternate_ad_url=http%3A%2F%2F[^(%2F)]*)[^&]*/\1/;
s/(&url=http%3A%2F%2F[^(%2F)]*)[^&]*/\1/;
s/(&ref=http%3A%2F%2F[^(%2F)]*)[^&]*/\1/;
s/(&cookie=http%3A%2F%2F[^(%2F)]*)[^&]*/\1/;
s/[;&?]ord=[?0-9]*//;
s/[;&]mpvid=[^&;]*//;
s/&xpc=[^&]*//;
# yieldmanager
s/\?clickTag=[^&]*//;
s/&u=[^&]*//;
s/&slotname=[^&]*//;
s/&page_slots=[^&]*//;
}
$out="http://" . $1 . $2 . $y . "";

#cache high latency ads
} elsif (m/^http:\/\/(.*?)\/(ads)\?(.*?)/) {
$out="http://" . $1 . "/" . $2  . "";

# spicific servers starts here....
} elsif (m/^http:\/\/(www\.ziddu\.com.*\.[^\/]{3,4})\/(.*?)/) {
$out="http://" . $1 . "";

#cdn, varialble 1st path
} elsif (($u =~ /filehippo/) && (m/^http:\/\/(.*?)\.(.*?)\/(.*?)\/(.*)\.([a-z0-9]{3,4})(\?.*)?/)) {
@y = ($1,$2,$4,$5);
$y[0] =~ s/[a-z0-9]{2,5}/cdn./;
$out="http://" . $y[0] . $y[1] . "/" . $y[2] . "." . $y[3] . "";

#rapidshare
} elsif (($u =~ /rapidshare/) && (m/^http:\/\/(([A-Za-z]+[0-9-.]+)*?)([a-z]*\.[^\/]{3}\/[a-z]*\/[0-9]*)\/(.*?)\/([^\/\?\&]{4,})$/)) {
$out="http://cdn." . $3 . "/SQUIDINTERNAL/" . $5 . "";

} elsif (($u =~ /maxporn/) && (m/^http:\/\/([^\/]*?)\/(.*?)\/([^\/]*?)(\?.*)?$/)) {
$out="http://" . $1 . "/SQUIDINTERNAL/" . $3 . "";

#like porn hub variables url and center part of the path, filename etention 3 or 4 with or without ? at the end
} elsif (($u =~ /tube8|pornhub|xvideos/) && (m/^http:\/\/(([A-Za-z]+[0-9-.]+)*?(\.[a-z]*)?)\.([a-z]*[0-9]?\.[^\/]{3}\/[a-z]*)(.*?)((\/[a-z]*)?(\/[^\/]*){4}\.[^\/\?]{3,4})(\?.*)?$/)) {
$out="http://cdn." . $4 . $6 . "";

#...spicific servers end here.

#photos-X.ak.fbcdn.net where X a-z
} elsif (m/^http:\/\/photos-[a-z].ak.fbcdn.net\/(.*)/) {
$out="http://photos.ak.fbcdn.net/" . $1  . "";

#for yimg.com video
} elsif (m/^http:\/\/(.*yimg.com)\/\/(.*)\/([^\/\?\&]*\/[^\/\?\&]*\.[^\/\?\&]{3,4})(\?.*)?$/) {
$out="http://cdn.yimg.com//" . $3 . "";

#for yimg.com doubled
} elsif (m/^http:\/\/(.*?)\.yimg\.com\/(.*?)\.yimg\.com\/(.*?)\?(.*)/) {
$out="http://cdn.yimg.com/"  . $3 . "";

#for yimg.com with &sig=
} elsif (m/^http:\/\/(.*?)\.yimg\.com\/(.*)/) {
@y = ($1,$2);
$y[0] =~ s/[a-z]+[0-9]+/cdn/;
$y[1] =~ s/&sig=.*//;
$out="http://" . $y[0] . ".yimg.com/"  . $y[1] . "";

#youjizz. We use only domain and filename
} elsif (($u =~ /media[0-9]{2,5}\.youjizz/) && (m/^http:\/\/(.*)(\.[^\.\-]*?\..*?)\/(.*)\/([^\/\?\&]*)\.([^\/\?\&]{3,4})((\?|\%).*)?$/)) {
@y = ($1,$2,$4,$5);
$y[0] =~ s/(([a-zA-A]+[0-9]+(-[a-zA-Z])?$)|(.*cdn.*)|(.*cache.*))/cdn/;
$out="http://" . $y[0] . $y[1] . "/" . $y[2] . "." . $y[3] . "";

#general purpose for cdn servers. add above your specific servers.
} elsif (m/^http:\/\/([0-9.]*?)\/\/(.*?)\.(.*)\?(.*?)/) {
$out="http://squid-cdn-url//" . $2  . "." . $3 . "";

#generic http://variable.domain.com/path/filename."ex" "ext" or "exte" with or withour "? or %"
} elsif (m/^http:\/\/(.*)(\.[^\.\-]*?\..*?)\/(.*)\.([^\/\?\&]{2,4})((\?|\%).*)?$/) {
@y = ($1,$2,$3,$4);
$y[0] =~ s/(([a-zA-Z]+[0-9]+(-[a-zA-Z])?$)|(.*cdn.*)|(.*cache.*))/cdn/;
$out="http://" . $y[0] . $y[1] . "/" . $y[2] . "." . $y[3] . "";

} else {
$out="$u"; ##$X[2]="$sucks";
}
print $logfh "$timenow"."out: $x $out $X[2] $X[3] $X[4] $X[5] $X[6] $X[7]\n" if ($debug>=1);
print "$x $out $X[2] $X[3] $X[4] $X[5] $X[6] $X[7]\n";
}
close $logfh if ($debug);

.

Please post your comments regarding updated storeurl.pl

Regard’s
SYED JAHANZAIB

Advertisements

January 29, 2012

Mikrotik / Linux Port Forwarding to Local Server on LAN

Filed under: Linux Related, Mikrotik Related — Tags: , — Syed Jahanzaib / Pinochio~:) @ 9:50 AM

Mikrotik Port Forwarding to Local Server

If you want to host any web server (or any other service like RDP or Game Server) behind mikrotik server and you want it to be publicly available for all internet users, you can use port forwarding and create one dstnat rule as below.Just make sure this rule comes above any masquerading rule.

Scenario:

DSL MODEM WAN IP = 221.xxx.xxx.xxx
DSL LAN IP = 192.168.1.1

MIKROTIK WAN IP = 192.168.1.2
MIKROTIK LAN IP = 192.168.0.1

WEB SERVER IP = 192.168.0.50

First setup port forwarding in your dsl modem to forward port 80 request to your mikrotik, I am not showing DSL modem config, as its very different for every mode, search for your modem confg page on howto do port forwarding. Just an example here for my Wi.Fi MODEM page.

Then in mikrotik , add an rule to forward port 80 request to your local web server, (one that is hosted behind your mikrotik server, on local user LAN)

MIKROTIK RULE :

/ip firewall nat
add action=dst-nat chain=dstnat disabled=no dst-port=80 in-interface=WAN2-QUBEE protocol=tcp to-addresses=192.168.0.50 \
to-ports=80

The above rule result would be something like below.

.

Linux Port Forwarding to Local Server

Linux WAN IP = 221.132.112.9 [Connected with WAN]
Linux LAN IP = 10.0.0.1 [Connected with User LAN]
Mikrotik LAN IP = 10.0.0.2 [Connected with User LAN or with SQUID]

sysctl net.ipv4.ip_forward=1

iptables -t nat -A PREROUTING -p tcp –dport 8291 -j DNAT –to-destination 10.0.0.2:8291

iptables -t nat -A POSTROUTING -j MASQUERADE

Regard’s
Syed Jahanzaib

January 28, 2012

Howto Disabe IPV6 in FEDORA 10

Filed under: Linux Related — Tags: , , — Syed Jahanzaib / Pinochio~:) @ 12:29 PM

Howto To disable IPV6 in FEDORA 10 :)~

Issue the following commands in terminal, (You must be logged in as root)

su -c 'echo "install ipv6 /bin/true
blacklist ipv6" > /etc/modprobe.d/blacklist-ipv6.conf'

su -c 'service ip6tables stop && chkconfig ip6tables off'

After making these changes reboot your system. Then check your interface card by

ifconfig

You will notice that you IPV6 lines have been gone 🙂

Regard’s
Syed Jahanzaib

January 23, 2012

SQUID: Howto Exclude some sites & Exntension from CACHING


Howto Exclude Few Sites from Caching

========================================

To exclude your local domain sites / range from cache, use following


acl local_server dst 192.168.0.0/24 192.168.1.0/24
cache deny local_server

========================================

If you are using SQUID proxy server and you don’t want to cache few sites , use the following directives,

SQUID 3.x

acl hotmail dstdomain .hotmail.com
always_direct allow hotmail

.

.

Another method

acl NO-CACHE-SITES dstdomain "/etc/squid/not-to-cache-sites.txt"
no_cache deny NO-CACHE-SITES
acl sharing_server dst 10.0.0.1
cache deny sharing_server

Now create the file which will contains our sites list which we don’t want to cache.

touch /etc/squid/not-to-cache-sites.txt

and add  your desired web sites name in /etc/squid/not-to-cache-sites.txt
For example

nano /etc/squid/not-to-cache-sites.txt


and add following or your entries

bankalhabib.com
aacable.wordpress.com
wordpress.com
nae.com.pk
jang.com.pk

Now we have to reload squid.conf so changes may apply by following command

squid -k rec
OR
squid -k reconfigure

OR if squid3 use

squid3 -k rec

.

.

.

Howto Exclude Specific Extension from Caching

Similarly if you don’t want to cache *.FLV files , use the following directives

hierarchy_stoplist cgi-bin ?
acl QUERY urlpath_regex cgi-bin \? \.flv
no_cache deny QUERY


Now we have to reload squid.conf so changes may apply by following command

squid -k rec
OR
squid -k reconfigure

.

.

.

.

Regard's
Syed Jahanzaib

January 21, 2012

UBUNTU rc.local not executing problem and its workaround

Filed under: Linux Related — Tags: , , , — Syed Jahanzaib / Pinochio~:) @ 9:03 PM

Sometimes in ubuntu, rc.local does not execute , specially in fresh installation.
So if you encounter this issue , you can solve it by using two methods.
I have used these methods several time with success. Also make sure you command entries are above ‘exit 0‘ line
Also adding 10 seconds delay helps in few cases.

 

Method # 1 , By Using WEBMIN

Method # 2 , By Removing -e from /etc/rc.loacl file [and add 10 seconds delay]

 

Method # 1 , By Using WEBMIN

Install webmin from http://www.webmin.com/deb.html

After installati0n is done, open webmin and goto
SYSTEM > BOOTUP & SHUTDOWN
Scroll down and and Click on rc.local
Select YES on Start at boot time?

As shown in the image below.

Method # 2 , By Removing -e from /etc/rc.loacl file

Edit rc.local file by

nano /etc/rc.local

at first line remove -e  for example

Change from

#!/bin/sh -e

to

#!/bin/sh

Save & Exit, then Reboot and see, hopefully rc.local will execute on the boot.

Also adding following sleep 10 directive before any script in rc.local will help. e.g:

Example of rc.local

 

root@ahmad-desktop:~# cat /etc/rc.local
#!/bin/sh -e
# rc.local
# This script is executed at the end of each multiuser runlevel.
# Make sure that the script will “exit 0” on success or any other
# value on error.
# In order to enable or disable this script just change the execution
# bits.
# By default this script does nothing.

# Add 10 seconds delay …

sleep 10

#Followign command will run my script for internet sharing etc / zaib

/etc/fw.sh
exit 0

 

 

Regard’s
Syed Jahanzaib

Ubuntu Memory Limitation Solution


If you install Ubuntu 32bit , and you have more then 4 GB of  RAM, you won’t be able to use it, The easiest solution is to grab latest 64 bit version and install it (Only if your hardware supports 64bit architecture) otherwise you have to use PAE enabled kernel.

For example if you have 8GB of ram, and you issue the command,

free -m

you will see only 3-4 GB of  total ram. You need to install Physical Address Extension (PAE) aware kernel under 32 bit Ubuntu Linux. It is a feature of x86 and x86-64 processors that allows more than 4 Gigabytes of physical memory to be used in 32-bit systems.

HOWTO Install PAE enabled kernel in UBUNTU

Open terminal and type the following command

If you are using Ubuntu v9.10 (Karmic Koala) and above, enter:

apt-get install linux-generic-pae linux-headers-generic-pae

Once kernel images installed, just reboot your workstation, type:

reboot

After reboot, login into your system and type the following command to verify memory usage:

free -m

if you are using Ubuntu version Ubuntu v9.04 and earlier:

apt-get update
sudo apt-get install linux-headers-server linux-image-server linux-server

Now reboot and check memory by free -m

January 19, 2012

Youtube caching with SQUID 2.7 [using storeurl.pl]

Filed under: Linux Related — Tags: , — Syed Jahanzaib / Pinochio~:) @ 11:34 AM

UPDATED:

21st April, 2014

YOUTUBE is caching now using LUSCA and storeurl.pl method. Tested and so far working Good, only dailymotion remains now.

https://aacable.wordpress.com/2014/04/21/howto-cache-youtube-with-squid-lusca-and-bypass-cached-videos-from-mikrotik-queue/

 

 

==========================================================================

You can also try this. (for automated installation of squid in Ubuntu).

https://aacable.wordpress.com/2014/01/05/automated-installation-script-for-squid-2-7-stable-9-with-video-caching-support/

As we all know that Mikrotik web proxy is a basic proxy package , suitable for basic caching , but its not possible to do the caching of Dynamic Contents, youtube videos and many other contents. To accomplish this task you have to add SQUID proxy server  , and route all http traffic from mikrotik to squid, then configure squid 2.7 stable9 with storeurl URL rewrite.

I wrote an easy guide regarding squid compilation via its source package and its configuration for caching video and other contents. It’s working good till date, and caching most of the videos including youtube, and many others. I have listed few web sites that are caching good.

Usually, this sort of caching is possible with commercial products only, like an addon of squid name CACHEVIDEO, or hardware products, But with some R&D , hit and trials, & some working configs examples, the caching worked. Please be aware that i have not reinvented the wheel, the method is out there since few years, but with some modifications and updates, its now working very good. I am still working on it to improve it. This config have few junk entries that are outdated or not required any more. You should do some research on it, like few directives on refresh patterns that are not supported in 2.7

This guide is actually its a collection of squid and storeurl configuration guides, picked up from multiple public and shared resources.Its not 100% perfect, but it does it job at some acceptable level :), and above all, ITS FREE 😉 and we all love Free items 😀 don’t we?

/ zaib

Blow is a quick reference guide for Squid 2.7 stable9 installation on Ubuntu ver 10.4 (or 12) with youtube & few other contents caching support. (or any linux flavor with Squid 2.7, because storeurl method is supported in squid 2.7 only ).

.

Following web sites are tested and working good 🙂

.

If this method helps you, please post your comment.

 

.

Ok here we start   . . .

Lets start.

First update UBUNTU and install some support tools for squid compilation by

apt-get update
apt-get install -y gcc build-essential sharutils ccze libzip-dev automake1.9

.
Now we have to download and compile SQUID 2.7 STABLE9

mkdir /temp
cd /temp
wget https://mikrotik-squid.googlecode.com/files/squid-2.7.STABLE9%2Bpatch.tar.gz
tar xvf squid-2.7.STABLE9+patch.tar.gz
cd squid-2.7.STABLE9

.

Now we have to compile SQUID , You can add/remove your required configure options.

./configure --prefix=/usr --exec_prefix=/usr --bindir=/usr/sbin --sbindir=/usr/sbin --libexecdir=/usr/lib/squid --sysconfdir=/etc/squid \
--localstatedir=/var/spool/squid --datadir=/usr/share/squid --enable-async-io=24 --with-aufs-threads=24 --with-pthreads --enable-storeio=aufs \
--enable-linux-netfilter --enable-arp-acl --enable-epoll --enable-removal-policies=heap,lru --with-aio --with-dl --enable-snmp \
--enable-delay-pools --enable-htcp --enable-cache-digests --disable-unlinkd --enable-large-cache-files --with-large-files \
--enable-err-languages=English --enable-default-err-language=English --with-maxfd=65536

Or if you want 64bit, try below


./configure \
--prefix=/usr \
--exec_prefix=/usr \
--bindir=/usr/sbin \
--sbindir=/usr/sbin \
--libexecdir=/usr/lib/squid \
--sysconfdir=/etc/squid \
--localstatedir=/var/spool/squid \
--datadir=/usr/share/squid \
--enable-async-io=24 \
--with-aufs-threads=24 \
--with-pthreads \
--enable-storeio=aufs \
--enable-linux-netfilter \
--enable-arp-acl \
--enable-epoll \
--enable-removal-policies=heap,lru \
--with-aio --with-dl \
--enable-snmp \
--enable-delay-pools \
--enable-htcp \
--enable-cache-digests \
--disable-unlinkd \
--enable-large-cache-files \
--with-large-files \
--enable-err-languages=English \
--enable-default-err-language=English --with-maxfd=65536 \
--enable-carp \
--enable-follow-x-forwarded-for \
--with-maxfd=65536  \
'amd64-debian-linux' 'build_alias=amd64-debian-linux' 'host_alias=amd64-debian-linux' 'target_alias=amd64-debian-linux' 'CFLAGS=-Wall -g -O2' 'LDFLAGS=-Wl,-Bsymbolic-functions' 'CPPFLAGS='

Now issue make and make install commands
[To understand what configure, make and make install does, read following,
http://www.codecoffee.com/tipsforlinux/articles/27.html]

make
make install

Create Log folders (if not exists) and assign write permissions to proxy user


mkdir /var/log/squid
chown proxy:proxy /var/log/squid


Now its time to Edit squid configuration files. open Squid Configuration file by


nano /etc/squid/squid.conf

.

Remove all previous lines , means empty the file, and paste all following lines . . .


# Last Updated : 09th FEBRAURY, 2014 / Syed Jahanzaib
# SQUID 2.7 Stable9 Configuration FILE with updated STOREURL.PL  [jz]
# Tested with Ubuntu 10.4 & 12.4 with compiled version of Squid 2.7 STABLE.9 [jz]
# Various contents copied from multiple public shared sources, personnel configs, hits and trial, VC etc
# It do have lot of junk / un-necessary entries, so remove them if not required.
# Syed Jahanzaib / https://aacable.wordpress.com
# Email: aacable@hotamil.com

# PORT and Transparent Option [jz]
http_port 8080 transparent
server_http11 on

# PID File location, we can use it for various functions later, like for squid status (JZ)
pid_filename /var/run/squid.pid

# Cache Directory , modify it according to your system. [jz]
# but first create directory in root by mkdir /cache1
# and then issue this command  chown proxy:proxy /cache1
# [for ubuntu user is proxy, in Fedora user is SQUID]
# I have set 200 GB for caching reserved just for caching ,
# adjust it according to your need.
# My recommendation is to have one cache_dir per drive. /zaib

# Using 10 GB in this example per drive
store_dir_select_algorithm round-robin

cache_dir aufs /cache-1 10240 16 256

# Cache Replacement Policies [jz]
cache_replacement_policy heap GDSF
memory_replacement_policy heap GDSF

# If you want to enable DATE time n SQUID Logs,use following [jz]
emulate_httpd_log on
logformat squid %tl %6tr %>a %Ss/%03Hs %<st %rm %ru %un %Sh/%<A %mt
log_fqdn off

# How much days to keep users access web logs [jz]
# You need to rotate your log files with a cron job. For example:
# 0 0 * * * /usr/local/squid/bin/squid -k rotate
logfile_rotate 14
debug_options ALL,1

# Squid Logs Section
# access_log none # To disable Squid access log, enable this option

cache_access_log /var/log/squid/access.log
cache_log /var/log/squid/cache.log
#referer_log /var/log/squid/referer.log
cache_store_log /var/log/squid/store.log
#mime_table /etc/squid/mime.conf
log_mime_hdrs off

# I used DNSAMSQ service for fast dns resolving
# so install by using "apt-get install dnsmasq" first / zaib
dns_nameservers 8.8.8.8

ftp_user anonymous@
ftp_list_width 32
ftp_passive on
ftp_sanitycheck on

#ACL Section
acl all src 0.0.0.0/0.0.0.0 # Allow All, you may want to change this to allow your ip series only
acl localhost src 127.0.0.1/255.255.255.255
acl to_localhost dst 127.0.0.0/8

###### cache manager section start, You can remote it if not required ####
# install following
# apt-get install squid-cgi
# add following entry in /etc/squid/cachemgr.conf
# localhost:8080
# then you can access it via http://squid_ip/cgi-bin/cachemgr.cgi

acl manager url_regex -i ^cache_object:// /squid-internal-mgr/
acl managerAdmin src 10.0.0.1 # Change it to your management pc ip
cache_mgr zaib@zaib.com
cachemgr_passwd zaib all
http_access allow manager localhost
http_access allow manager managerAdmin
http_access deny manager
#http_access allow localhost
####### CACHGEMGR END #########

acl SSL_ports port 443 563 # https, snews
acl SSL_ports port 873 # rsync
acl Safe_ports port 80 # http
acl Safe_ports port 21 # ftp
acl Safe_ports port 53 # dns
acl Safe_ports port 443 563 # https, snews
acl Safe_ports port 70 # gopher
acl Safe_ports port 210 # wais
acl Safe_ports port 1025-65535 # unregistered ports
acl Safe_ports port 280 # http-mgmt
acl Safe_ports port 488 # gss-http
acl Safe_ports port 591 # filemaker
acl Safe_ports port 777 # multiling http
acl Safe_ports port 631 # cups
acl Safe_ports port 873 # rsync
acl Safe_ports port 901 # SWAT
acl purge method PURGE
acl CONNECT method CONNECT
http_access allow purge localhost
http_access deny purge
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
http_access allow localhost
http_access allow all
http_reply_access allow all
icp_access allow all

#===============================
# Administrative Parameters [jz]
#===============================

# I used UBUNTU so user is proxy, in FEDORA you may use use squid [jz]
cache_effective_user proxy
cache_effective_group proxy
cache_mgr SYED_JAHANZAIB
visible_hostname aacable@hotmai.com
unique_hostname aacable@hotmai.com

#=================
# ACCELERATOR [jz]
#=================
half_closed_clients off
quick_abort_min 0 KB
quick_abort_max 0 KB
vary_ignore_expire on
reload_into_ims on
log_fqdn off
memory_pools off
cache_swap_low 90
cache_swap_high 95
max_filedescriptors 65536
fqdncache_size 16384
retry_on_error on
offline_mode off
pipeline_prefetch on
check_hostnames off
client_db on
#range_offset_limit 128 KB
max_stale 1 week
read_ahead_gap 1 KB
forwarded_for off
minimum_expiry_time 1960 seconds
collapsed_forwarding on
cache_vary on
update_headers off
vary_ignore_expire on
incoming_rate 9
ignore_ims_on_miss off

# If you want to hide your proxy machine from being detected at various site use following [jz]
via off

#==========================
# Squid Memory Tunning [jz]
#==========================
# If you have 4GB memory in Squid box, we will use formula of 1/3
# You can adjust it according to your need. IF squid is taking too much of RAM
# Then decrease it to 512 MB or even less.

cache_mem 1024 MB
minimum_object_size 0 bytes
maximum_object_size 1 GB

# Lower it down if your squid taking to much memory, e.g: 512 KB or even less
maximum_object_size_in_memory 512 KB

#============================================================$
# SNMP , if you want to generate graphs for SQUID via MRTG [jz]
#============================================================$
#acl snmppublic snmp_community gl
#snmp_port 3401
#snmp_access allow snmppublic all
#snmp_access allow all

#===========================================================================
# ZPH (for 2.7) , To enable cache content to be delivered at full lan speed,
# OR To bypass the queue at MT for cached contents / zaib
#===========================================================================
tcp_outgoing_tos 0x30 all
zph_mode tos
zph_local 0x30
zph_parent 0
zph_option 136

# ++++++++++++++++++++++++++++++++++++++++++++++++
# +++++++++++ REFRESH PATTERNS SECTION +++++++++++
# ++++++++++++++++++++++++++++++++++++++++++++++++

#===================================
# youtube Caching Configuration
#===================================

strip_query_terms off
acl yutub url_regex -i .*youtube\.com\/.*$
acl yutub url_regex -i .*youtu\.be\/.*$
logformat squid1 %{Referer}>h %ru
access_log /var/log/squid/yt.log squid1 yutub
acl redirec urlpath_regex -i .*&redirect_counter=1&cms_redirect=yes
acl redirec urlpath_regex -i .*&ir=1&rr=12
acl reddeny url_regex -i c\.youtube\.com\/videoplayback.*redirect_counter=1.*$
acl reddeny url_regex -i c\.youtube\.com\/videoplayback.*cms_redirect=yes.*$
acl reddeny url_regex -i c\.youtube\.com\/videoplayback.*\&ir=1.*$
acl reddeny url_regex -i c\.youtube\.com\/videoplayback.*\&rr=12.*$
storeurl_access deny reddeny

#--------------------------------------------------------#
# REFRESH PATTERN UPDATED: 27th September, 2013
#--------------------------------------------------------#
refresh_pattern ^http\:\/\/*\.facebook\.com\/ 10080 80% 43200 reload-into-ims
refresh_pattern ^http\:\/\/*\.kaskus\.us\/ 10080 80% 43200 reload-into-ims
refresh_pattern ^http\:\/\/*\.google\.co\*.*/ 10080 90% 43200 reload-into-ims
refresh_pattern ^http\:\/\/*\.yahoo\.co*\.*/ 10080 90% 43200 reload-into-ims
refresh_pattern ^http\:\/\/.*\.windowsupdate\.microsoft\.com\/ 10080 80% 43200 reload-into-ims
refresh_pattern ^http\:\/\/office\.microsoft\.com\/ 10080 80% 43200 reload-into-ims
refresh_pattern ^http\:\/\/windowsupdate\.microsoft\.com\/ 10080 80% 43200 reload-into-ims
refresh_pattern ^http\:\/\/w?xpsp[0-9]\.microsoft\.com\/ 10080 80% 43200 reload-into-ims
refresh_pattern ^http\:\/\/w2ksp[0-9]\.microsoft\.com\/ 10080 80% 43200 reload-into-ims
refresh_pattern ^http\:\/\/download\.microsoft\.com\/ 10080 80% 43200 reload-into-ims
refresh_pattern ^http\:\/\/download\.macromedia\.com\/ 10080 80% 43200 reload-into-ims
refresh_pattern ^ftp\:\/\/ftp\.nai\.com/ 10080 80% 43200 reload-into-ims
refresh_pattern ^http\:\/\/ftp\.software\.ibm\.com\/ 10080 80% 43200 reload-into-ims
refresh_pattern ^http\:\/\/*\.google\.co\*.*/ 10080 90% 43200 reload-into-ims
refresh_pattern ^http\:\/\/*\.yahoo\.co*\.*/ 10080 90% 43200 reload-into-ims
refresh_pattern ^http://*.apps.facebook.*/.* 720 80% 4320
refresh_pattern ^http://*.profile.ak.fbcdn.net/.* 720 80% 4320
refresh_pattern ^http://*.creative.ak.fbcdn.net/.* 720 80% 4320
refresh_pattern ^http://*.static.ak.fbcdn.net/.* 720 80% 4320
refresh_pattern ^http://*.facebook.poker.zynga.com/.* 720 80% 4320
refresh_pattern ^http://*.statics.poker.static.zynga.com/.* 720 80% 4320
refresh_pattern ^http://*.zynga.*/.* 720 80% 4320
refresh_pattern ^http://*.texas_holdem.*/.* 720 80% 4320
refresh_pattern ^http://*.google.*/.* 720 80% 4320
refresh_pattern ^http://*.indowebster.*/.* 720 80% 4320
refresh_pattern ^http://*.4shared.*/.* 720 80% 4320
refresh_pattern ^http://*.yahoo.com/.* 720 80% 4320
refresh_pattern ^http://*.yimg.*/.* 720 80% 4320
refresh_pattern ^http://*.boleh.*/.* 720 80% 4320
refresh_pattern ^http://*.kompas.*/.* 180 80% 4320
refresh_pattern ^http://*.google-analytics.*/.* 720 80% 4320

refresh_pattern ^http://(.*?)/get_video\? 10080 90% 999999 override-expire ignore-no-cache ignore-private
refresh_pattern ^http://(.*?)/videoplayback\? 10080 90% 999999 override-expire ignore-no-cache ignore-private
refresh_pattern -i (get_video\?|videoplayback\?id|videoplayback.*id) 161280 50000% 525948 override-expire ignore-reload

# compressed
refresh_pattern -i \.gz$ 10080 90% 999999 override-expire override-lastmod reload-into-ims ignore-reload
refresh_pattern -i \.cab$ 10080 90% 999999 override-expire override-lastmod reload-into-ims ignore-reload
refresh_pattern -i \.bzip2$ 10080 90% 999999 override-expire override-lastmod reload-into-ims ignore-reload
refresh_pattern -i \.bz2$ 10080 90% 999999 override-expire override-lastmod reload-into-ims ignore-reload
refresh_pattern -i \.gz2$ 10080 90% 999999 override-expire override-lastmod reload-into-ims ignore-reload
refresh_pattern -i \.tgz$ 10080 90% 999999 override-expire override-lastmod reload-into-ims ignore-reload
refresh_pattern -i \.tar.gz$ 10080 90% 999999 override-expire override-lastmod reload-into-ims ignore-reload
refresh_pattern -i \.zip$ 10080 90% 999999 override-expire override-lastmod reload-into-ims ignore-reload
refresh_pattern -i \.rar$ 1008000 90% 99999999 override-expire override-lastmod reload-into-ims ignore-reload
refresh_pattern -i \.tar$ 10080 90% 999999 override-expire override-lastmod reload-into-ims ignore-reload
refresh_pattern -i \.ace$ 10080 90% 999999 override-expire override-lastmod reload-into-ims ignore-reload
refresh_pattern -i \.7z$ 10080 90% 999999 override-expire override-lastmod reload-into-ims ignore-reload

# documents
refresh_pattern -i \.xls$ 10080 90% 999999 override-expire override-lastmod reload-into-ims ignore-reload
refresh_pattern -i \.doc$ 10080 90% 999999 override-expire override-lastmod reload-into-ims ignore-reload
refresh_pattern -i \.xlsx$ 10080 90% 999999 override-expire override-lastmod reload-into-ims ignore-reload
refresh_pattern -i \.docx$ 10080 90% 999999 override-expire override-lastmod reload-into-ims ignore-reload
refresh_pattern -i \.pdf$ 10080 90% 999999 override-expire override-lastmod reload-into-ims ignore-reload
refresh_pattern -i \.ppt$ 10080 90% 999999 override-expire override-lastmod reload-into-ims ignore-reload
refresh_pattern -i \.pptx$ 10080 90% 999999 override-expire override-lastmod reload-into-ims ignore-reload
refresh_pattern -i \.rtf\?$ 10080 90% 999999 override-expire override-lastmod reload-into-ims ignore-reload

# multimedia
refresh_pattern -i \.mid$ 10080 90% 999999 override-expire override-lastmod reload-into-ims ignore-reload
refresh_pattern -i \.wav$ 10080 90% 999999 override-expire override-lastmod reload-into-ims ignore-reload
refresh_pattern -i \.viv$ 10080 90% 999999 override-expire override-lastmod reload-into-ims ignore-reload
refresh_pattern -i \.mpg$ 10080 90% 999999 override-expire override-lastmod reload-into-ims ignore-reload
refresh_pattern -i \.mov$ 10080 90% 999999 override-expire override-lastmod reload-into-ims ignore-reload
refresh_pattern -i \.avi$ 10080 90% 999999 override-expire override-lastmod reload-into-ims ignore-reload
refresh_pattern -i \.asf$ 10080 90% 999999 override-expire override-lastmod reload-into-ims ignore-reload
refresh_pattern -i \.qt$ 10080 90% 999999 override-expire override-lastmod reload-into-ims ignore-reload
refresh_pattern -i \.rm$ 10080 90% 999999 override-expire override-lastmod reload-into-ims ignore-reload
refresh_pattern -i \.rmvb$ 10080 90% 999999 override-expire override-lastmod reload-into-ims ignore-reload
refresh_pattern -i \.mpeg$ 10080 90% 999999 override-expire override-lastmod reload-into-ims ignore-reload
refresh_pattern -i \.wmp$ 10080 90% 999999 override-expire override-lastmod reload-into-ims ignore-reload
refresh_pattern -i \.3gp$ 10080 90% 999999 override-expire override-lastmod reload-into-ims ignore-reload
refresh_pattern -i \.mp3$ 10080 90% 999999 override-expire override-lastmod reload-into-ims ignore-reload
refresh_pattern -i \.mp4$ 10080 90% 999999 override-expire override-lastmod reload-into-ims ignore-reload

# web content
refresh_pattern -i \.js$ 10080 90% 999999 override-expire override-lastmod reload-into-ims ignore-reload
refresh_pattern -i \.psf$ 10080 90% 999999 override-expire override-lastmod reload-into-ims ignore-reload
refresh_pattern -i \.html$ 10080 90% 999999 override-expire override-lastmod reload-into-ims ignore-reload
refresh_pattern -i \.htm$ 10080 90% 999999 override-expire override-lastmod reload-into-ims ignore-reload
refresh_pattern -i \.css$ 10080 90% 999999 override-expire override-lastmod reload-into-ims ignore-reload
refresh_pattern -i \.swf$ 10080 90% 999999 override-expire override-lastmod reload-into-ims ignore-reload
refresh_pattern -i \.js\?$ 10080 90% 999999 override-expire override-lastmod reload-into-ims ignore-reload
refresh_pattern -i \.css\?$ 10080 90% 999999 override-expire override-lastmod reload-into-ims ignore-reload
refresh_pattern -i \.xml$ 10080 90% 999999 override-expire override-lastmod reload-into-ims ignore-reload

# images
refresh_pattern -i \.gif$ 10080 90% 999999 override-expire override-lastmod reload-into-ims ignore-reload
refresh_pattern -i \.jpg$ 10080 90% 999999 override-expire override-lastmod reload-into-ims ignore-reload
refresh_pattern -i \.png$ 10080 90% 999999 override-expire override-lastmod reload-into-ims ignore-reload
refresh_pattern -i \.jpeg$ 10080 90% 999999 override-expire override-lastmod reload-into-ims ignore-reload
refresh_pattern -i \.bmp$ 10080 90% 999999 override-expire override-lastmod reload-into-ims ignore-reload
refresh_pattern -i \.psd$ 10080 90% 999999 override-expire override-lastmod reload-into-ims ignore-reload
refresh_pattern -i \.ad$ 10080 90% 999999 override-expire override-lastmod reload-into-ims ignore-reload
refresh_pattern -i \.gif\?$ 10080 90% 999999 override-expire override-lastmod reload-into-ims ignore-reload
refresh_pattern -i \.jpg\?$ 10080 90% 999999 override-expire override-lastmod reload-into-ims ignore-reload
refresh_pattern -i \.png\?$ 10080 90% 999999 override-expire override-lastmod reload-into-ims ignore-reload
refresh_pattern -i \.jpeg\?$ 10080 90% 999999 override-expire override-lastmod reload-into-ims ignore-reload
refresh_pattern -i \.psd\?$ 10080 90% 999999 override-expire override-lastmod reload-into-ims ignore-reload

# application
refresh_pattern -i \.deb$ 10080 90% 999999 override-expire override-lastmod reload-into-ims ignore-reload
refresh_pattern -i \.rpm$ 10080 90% 999999 override-expire override-lastmod reload-into-ims ignore-reload
refresh_pattern -i \.msi$ 10080 90% 999999 override-expire override-lastmod reload-into-ims ignore-reload
refresh_pattern -i \.exe$ 10080 90% 999999 override-expire override-lastmod reload-into-ims ignore-reload
refresh_pattern -i \.dmg$ 10080 90% 999999 override-expire override-lastmod reload-into-ims ignore-reload

# misc
refresh_pattern -i \.dat$ 10080 90% 999999 override-expire override-lastmod reload-into-ims ignore-reload
refresh_pattern -i \.qtm$ 10080 90% 999999 override-expire override-lastmod reload-into-ims ignore-reload

# itunes
refresh_pattern -i \.m4p$ 10080 90% 999999 override-expire override-lastmod reload-into-ims ignore-reload
refresh_pattern -i \.mpa$ 10080 90% 999999 override-expire override-lastmod reload-into-ims ignore-reload

# JUNK : O ~
refresh_pattern ^ftp: 1440 20% 10080
refresh_pattern -i \.(avi|wav|mid|mp?|mpeg|mov|3gp|wm?|swf|flv|x-flv|css|js|axd)$ 10080 95% 302400 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-private ignore-auth
refresh_pattern -i \.(gif|png|jp?g|ico|bmp)$ 4320 95% 10080 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-private ignore-auth
refresh_pattern -i \.(rpm|cab|exe|msi|msu|zip|tar|gz|tgz|rar|bin|7z|doc|xls|ppt|pdf)$ 4320 90% 10080 override-expire override-lastmod reload-into-ims
refresh_pattern -i (/cgi-bin/|\?) 0 0% 0
refresh_pattern . 360 90% 302400 override-lastmod reload-into-ims

########################################################################
## MORE REFRESH PATTERN SETTINGS (including video cache config too)
########################################################################

acl dontrewrite url_regex (get_video|video\?v=|videoplayback\?id|videoplayback.*id).*begin\=[1-9][0-9]* \.php\? \.asp\? \.aspx\? threadless.*\.jpg\?r=
acl store_rewrite_list urlpath_regex \/(get_video|videoplayback\?id|videoplayback.*id) \.(jp(e?g|e|2)|gif|png|tiff?|bmp|ico|flv|wmv|3gp|mp(4|3)|exe|msi|zip|on2|mar|swf)\?
acl store_rewrite_list urlpath_regex \/(get_video\?|videodownload\?|videoplayback.*id|watch\?)
acl store_rewrite_list urlpath_regex \.(3gp|mp(3|4)|flv|(m|f)4v|on2|fid|avi|mov|wm(a|v)|(mp(e?g|a|e|1|2))|mk(a|v)|jp(e?g|e|2)|gif|png|tiff?|bmp|tga|svg|ico|swf|exe|ms(i|u|p)|cab|psf|mar|bin|z(ip|[0-9]{2})|r(ar|[0-9]{2})|7z)\?
acl store_rewrite_list_domain url_regex ^http:\/\/([a-zA-Z-]+[0-9-]+)\.[A-Za-z]*\.[A-Za-z]*
acl store_rewrite_list_domain url_regex (([a-z]{1,2}[0-9]{1,3})|([0-9]{1,3}[a-z]{1,2}))\.[a-z]*[0-9]?\.[a-z]{3}
acl store_rewrite_list_path urlpath_regex \.(jp(e?g|e|2)|gif|png|tiff?|bmp|ico|flv|avc|zip|mp3|3gp|rar|on2|mar|exe)$
acl store_rewrite_list_domain_CDN url_regex streamate.doublepimp.com.*\.js\? .fbcdn.net \.rapidshare\.com.*\/[0-9]*\/.*\/[^\/]* ^http:\/\/(www\.ziddu\.com.*\.[^\/]{3,4})\/(.*) \.doubleclick\.net.* yield$
acl store_rewrite_list_domain_CDN url_regex (cbk|mt|khm|mlt|tbn)[0-9]?.google\.co(m|\.uk|\.id)
acl store_rewrite_list_domain_CDN url_regex ^http://(.*?)/windowsupdate\?
acl store_rewrite_list_domain_CDN url_regex photos-[a-z].ak.fbcdn.net
acl store_rewrite_list_domain_CDN url_regex ^http:\/\/([a-z])[0-9]?(\.gstatic\.com|\.wikimapia\.org)
acl store_rewrite_list_domain_CDN url_regex ^http:\/\/download[0-9]{3}.avast.com/iavs5x/
acl store_rewrite_list_domain_CDN url_regex ^http:\/\/dnl-[0-9]{2}.geo.kaspersky.com
acl store_rewrite_list_domain_CDN url_regex ^http:\/\/[1-4].bp.blogspot.com
acl store_rewrite_list_domain url_regex ^http:\/\/([a-zA-Z-]+[0-9-]+)\.[A-Za-z]*\.[A-Za-z]*
acl store_rewrite_list_domain url_regex (([a-z]{1,2}[0-9]{1,3})|([0-9]{1,3}[a-z]{1,2}))\.[a-z]*[0-9]?\.[a-z]{3}
acl store_rewrite_list_path urlpath_regex \.fid\?.*\&start= \.(jp(e?g|e|2)|gif|png|tiff?|bmp|ico|psf|flv|avc|zip|mp3|3gp|rar|on2|mar|exe)$
acl store_rewrite_list_domain_CDN url_regex \.rapidshare\.com.*\/[0-9]*\/.*\/[^\/]* ^http:\/\/(www\.ziddu\.com.*\.[^\/]{3,4})\/(.*) \.doubleclick\.net.*
acl store_rewrite_list_domain_CDN url_regex ^http:\/\/[.a-z0-9]*\.photobucket\.com.*\.[a-z]{3}$ quantserve\.com
acl store_rewrite_list_domain_CDN url_regex ^http:\/\/[a-z]+[0-9]\.google\.co(m|\.id)
acl store_rewrite_list_domain_CDN url_regex ^http:\/\/\.www[0-9][0-9]\.indowebster\.com\/(.*)(rar|zip|flv|wm(a|v)|3gp|psf|mp(4|3)|exe|msi|avi|(mp(e?g|a|e|1|2|3|4))|cab|exe)

# Videos Config / jz
acl videocache_allow_url url_regex -i \.googlevideo\.com\/videoplayback \.googlevideo\.com\/videoplay \.googlevideo\.com\/get_video\?
acl videocache_allow_url url_regex -i \.google\.com\/videoplayback \.google\.com\/videoplay \.google\.com\/get_video\?
acl videocache_allow_url url_regex -i \.google\.[a-z][a-z]\/videoplayback \.google\.[a-z][a-z]\/videoplay \.google\.[a-z][a-z]\/get_video\?
acl videocache_allow_url url_regex -i proxy[a-z0-9\-][a-z0-9][a-z0-9][a-z0-9]?\.dailymotion\.com\/
acl videocache_allow_url url_regex -i \.vimeo\.com\/(.*)\.(flv|mp4)
acl videocache_allow_url url_regex -i va\.wrzuta\.pl\/wa[0-9][0-9][0-9][0-9]?
acl videocache_allow_url url_regex -i \.youporn\.com\/(.*)\.flv
acl videocache_allow_url url_regex -i \.msn\.com\.edgesuite\.net\/(.*)\.flv
acl videocache_allow_url url_regex -i \.tube8\.com\/(.*)\.(flv|3gp)
acl videocache_allow_url url_regex -i \.mais\.uol\.com\.br\/(.*)\.flv
acl videocache_allow_url url_regex -i \.blip\.tv\/(.*)\.(flv|avi|mov|mp3|m4v|mp4|wmv|rm|ram|m4v)
acl videocache_allow_url url_regex -i \.apniisp\.com\/(.*)\.(flv|avi|mov|mp3|m4v|mp4|wmv|rm|ram|m4v)
acl videocache_allow_url url_regex -i \.break\.com\/(.*)\.(flv|mp4)
acl videocache_allow_url url_regex -i redtube\.com\/(.*)\.flv
acl videocache_allow_url url_regex -i vid\.akm\.dailymotion\.com\/
acl videocache_allow_url url_regex -i [a-z0-9][0-9a-z][0-9a-z]?[0-9a-z]?[0-9a-z]?\.xtube\.com\/(.*)flv
acl videocache_allow_url url_regex -i bitcast\.vimeo\.com\/vimeo\/videos\/
acl videocache_allow_url url_regex -i va\.wrzuta\.pl\/wa[0-9][0-9][0-9][0-9]?
acl videocache_allow_url url_regex -i \.files\.youporn\.com\/(.*)\/flv\/
acl videocache_allow_url url_regex -i \.msn\.com\.edgesuite\.net\/(.*)\.flv
acl videocache_allow_url url_regex -i media[a-z0-9]?[a-z0-9]?[a-z0-9]?\.tube8\.com\/ mobile[a-z0-9]?[a-z0-9]?[a-z0-9]?\.tube8\.com\/ www\.tube8\.com\/(.*)\/
acl videocache_allow_url url_regex -i \.mais\.uol\.com\.br\/(.*)\.flv
acl videocache_allow_url url_regex -i \.video[a-z0-9]?[a-z0-9]?\.blip\.tv\/(.*)\.(flv|avi|mov|mp3|m4v|mp4|wmv|rm|ram)
acl videocache_allow_url url_regex -i video\.break\.com\/(.*)\.(flv|mp4)
acl videocache_allow_url url_regex -i \.xvideos\.com\/videos\/flv\/(.*)\/(.*)\.(flv|mp4)
acl videocache_allow_url url_regex -i stream\.aol\.com\/(.*)/[a-zA-Z0-9]+\/(.*)\.(flv|mp4)
acl videocache_allow_url url_regex -i videos\.5min\.com\/(.*)/[0-9_]+\.(mp4|flv)
acl videocache_allow_url url_regex -i msn\.com\/(.*)\.(flv|mp4|avi|mkv|mp3|rm|rmvb|m4v|mov|wmv|3gp|mpg|mpeg)
acl videocache_allow_url url_regex -i msn\.(.*)\.(com|net)\/(.*)\.(flv|mp4|avi|mkv|mp3|rm|rmvb|m4v|mov|wmv|3gp|mpg|mpeg)
acl videocache_allow_url url_regex -i msnbc\.(.*)\.(com|net)\/(.*)\.(flv|mp4|avi|mkv|mp3|rm|rmvb|m4v|mov|wmv|3gp|mpg|mpeg)
acl videocache_allow_url url_regex -i \.blip\.tv\/(.*)\.(flv|mp4|avi|mkv|mp3|rm|rmvb|m4v|mov|wmv|3gp|mpg|mpeg)
acl videocache_deny_url  url_regex -i \.blip\.tv\/(.*)filename
acl videocache_allow_url url_regex -i \.break\.com\/(.*)\.(flv|mp4|avi|mkv|mp3|rm|rmvb|m4v|mov|wmv|3gp|mpg|mpeg)
acl videocache_allow_url url_regex -i cdn\.turner\.com\/(.*)/(.*)\.(flv)
acl videocache_allow_url url_regex -i \.dailymotion\.com\/video\/[a-z0-9]{5,9}_?(.*)
acl videocache_allow_url url_regex -i proxy[a-z0-9\-]?[a-z0-9]?[a-z0-9]?[a-z0-9]?\.dailymotion\.com\/(.*)\.(flv|on2|mp4|avi|mkv|mp3|rm|rmvb|m4v|mov|wmv|3gp|mpg|mpeg)
acl videocache_allow_url url_regex -i vid\.akm\.dailymotion\.com\/(.*)\.(flv|on2|mp4|avi|mkv|mp3|rm|rmvb|m4v|mov|wmv|3gp|mpg|mpeg)
acl videocache_allow_url url_regex -i vid\.ec\.dmcdn\.net\/(.*)\.(flv|on2|mp4|avi|mkv|mp3|rm|rmvb|m4v|mov|wmv|3gp|mpg|mpeg)
acl videocache_allow_url url_regex -i video\.(.*)\.fbcdn\.net\/(.*)/[0-9_]+\.(mp4|flv|avi|mkv|m4v|mov|wmv|3gp|mpg|mpeg)
acl videocache_allow_url url_regex -i \.mccont\.com\/ItemFiles\/(.*)?\.(flv|mp4|avi|mkv|mp3|rm|rmvb|m4v|mov|wmv|3gp|mpg|mpeg)
acl videocache_allow_url url_regex -i (.*)\.myspacecdn\.com\/(.*)\/[a-zA-Z0-9]+\/vid\.(flv|mp4|avi|mkv|mp3|rm|rmvb|m4v|mov|wmv|3gp|mpg|mpeg)
acl videocache_allow_url url_regex -i (.*)\.myspacecdn\.(.*)\.footprint\.net\/(.*)\/[a-zA-Z0-9]+\/vid\.(flv|mp4|avi|mkv|mp3|rm|rmvb|m4v|mov|wmv|3gp|mpg|mpeg)
acl videocache_allow_url url_regex -i \.vimeo\.com\/(.*)\.(flv|mp4|avi|mkv|mp3|rm|rmvb|m4v|mov|wmv|3gp|mpg|mpeg)
acl videocache_allow_url url_regex -i \.amazonaws\.com\/(.*)\.vimeo\.com(.*)\.(flv|mp4|avi|mkv|mp3|rm|rmvb|m4v|mov|wmv|3gp|mpg|mpeg)
acl videocache_allow_url url_regex -i v\.imwx\.com\/v\/wxcom\/[a-zA-Z0-9]+\.(flv|mp4|avi|mkv|mp3|rm|rmvb|m4v|mov|wmv|3gp|mpg|mpeg)\?(.*)videoId=[0-9]+&
acl videocache_allow_url url_regex -i c\.wrzuta\.pl\/wv[0-9]+\/[a-z0-9]+/[0-9]+/
acl videocache_allow_url url_regex -i c\.wrzuta\.pl\/wa[0-9]+\/[a-z0-9]+
acl videocache_allow_url url_regex -i cdn[a-z0-9]?[a-z0-9]?[a-z0-9]?\.public\.extremetube\.phncdn\.com\/(.*)\/[a-zA-Z0-9_-]+\.(flv|mp4|avi|mkv|mp3|rm|rmvb|m4v|mov|wmv|3gp|mpg|mpeg)
acl videocache_allow_url url_regex -i vs[a-z0-9]?[a-z0-9]?[a-z0-9]?\.hardsextube\.com\/(.*)\/(.*)\.(flv|mp4|avi|mkv|mp3|rm|rmvb|m4v|mov|wmv|3gp|mpg|mpeg)
acl videocache_deny_url  url_regex -i \.hardsextube\.com\/videothumbs
acl videocache_allow_url url_regex -i cdn[a-z0-9]?[a-z0-9]?[a-z0-9]?\.public\.keezmovies\.phncdn\.com\/(.*)\/[0-9a-zA-Z_\-]+\.(flv|mp4|avi|mkv|mp3|rm|rmvb|m4v|mov|wmv|3gp|mpg|mpeg)
acl videocache_allow_url url_regex -i cdn[a-z0-9]?[a-z0-9]?[a-z0-9]?\.public\.keezmovies\.com\/(.*)\/[0-9a-zA-Z_\-]+\.(flv|mp4|avi|mkv|mp3|rm|rmvb|m4v|mov|wmv|3gp|mpg|mpeg)
acl videocache_allow_url url_regex -i nyc-v[a-z0-9]?[a-z0-9]?[a-z0-9]?\.pornhub\.com\/(.*)/videos/[0-9]{3}/[0-9]{3}/[0-9]{3}/[0-9]+\.(flv|mp4|avi|mkv|mp3|rm|rmvb|m4v|mov|wmv|3gp|mpg|mpeg)
acl videocache_allow_url url_regex -i \.video\.pornhub\.phncdn\.com\/videos/(.*)/[0-9]+\.(flv|mp4|avi|mkv|mp3|rm|rmvb|m4v|mov|wmv|3gp|mpg|mpeg)
acl videocache_allow_url url_regex -i video(.*)\.redtubefiles\.com\/(.*)\.(flv|mp4|avi|mkv|mp3|rm|rmvb|m4v|mov|wmv|3gp|mpg|mpeg)
acl videocache_allow_url url_regex -i \.slutload-media\.com\/(.*)\/[a-zA-Z0-9_.-]+\.(flv|mp4|avi|mkv|mp3|rm|rmvb|m4v|mov|wmv|3gp|mpg|mpeg)
acl videocache_allow_url url_regex -i cdn[a-z0-9]?[a-z0-9]?[a-z0-9]?\.public\.spankwire\.com\/(.*)\/(.*)\.(flv|mp4|avi|mkv|mp3|rm|rmvb|m4v|mov|wmv|3gp|mpg|mpeg)
acl videocache_allow_url url_regex -i cdn[a-z0-9]?[a-z0-9]?[a-z0-9]?\.public\.spankwire\.phncdn\.com\/(.*)\/(.*)\.(flv|mp4|avi|mkv|mp3|rm|rmvb|m4v|mov|wmv|3gp|mpg|mpeg)
acl videocache_allow_url url_regex -i \.tube8\.com\/(.*)\.(flv|mp4|avi|mkv|mp3|rm|rmvb|m4v|mov|wmv|3gp|mpg|mpeg)
acl videocache_allow_url url_regex -i \.xtube\.com\/(.*)\.(flv|mp4|avi|mkv|mp3|rm|rmvb|m4v|mov|wmv|3gp|mpg|mpeg)
acl videocache_deny_url  url_regex -i \.xtube\.com\/(.*)(Thumb|videowall)
acl videocache_allow_url url_regex -i \.xvideos\.com\/videos\/flv\/(.*)\/(.*)\.(flv|mp4)
acl videocache_allow_url url_regex -i \.public\.youporn\.phncdn\.com\/(.*)\/[a-zA-Z0-9_-]+\/(.*)\.(flv|mp4|avi|mkv|mp3|rm|rmvb|m4v|mov|wmv|3gp|mpg|mpeg)
acl videocache_allow_dom dstdomain .mccont.com .metacafe.com .redtube.com .dailymotion.com .fbcdn.net
acl videocache_deny_dom  dstdomain .download.youporn.com .static.blip.tv
acl dontrewrite url_regex redbot\.org (get_video|videoplayback\?id|videoplayback.*id).*begin\=[1-9][0-9]*

acl getmethod method GET
storeurl_access deny !getmethod
storeurl_access deny dontrewrite
storeurl_access allow videocache_allow_url
storeurl_access allow videocache_allow_dom
storeurl_access allow store_rewrite_list_domain_CDN
storeurl_access allow store_rewrite_list
storeurl_access allow store_rewrite_list_domain store_rewrite_list_path
storeurl_access deny all

# Load SOTREURL.PL REWRITE PROGRAM
storeurl_rewrite_program /etc/squid/storeurl.pl
storeurl_rewrite_children 15
storeurl_rewrite_concurrency 999

acl store_rewrite_list urlpath_regex -i \/(get_video\?|videodownload\?|videoplayback.*id)
acl store_rewrite_list urlpath_regex -i \.flv$ \.mp3$ \.mov$ \.mp4$ \.swf$ \
storeurl_access allow store_rewrite_list
storeurl_access deny all

Now save squid.conf & Exit.

>

>

>

STOREURL.PL

Now create storeurl.pl which will be used to pull youtube video from thecache.


touch /etc/squid/storeurl.pl
chmod +x /etc/squid/storeurl.pl

Now edit this file

nano /etc/squid/storeurl.pl

and paste the following contents.


#!/usr/bin/perl
# This script is NOT written or modified by me, I only copy pasted it from the internet.
# It was First originally Written by chudy_fernandez@yahoo.com
# &amp; Have been modified by various persons over the net to fix/add various functions.
# Like For Example modified by member of comstuff.net to satisfy common and dynamic content.
# th30nly @comstuff.net a.k.a invisible_theater , Syaifudin JW , Ucok Karnadi and possibly other people too.
# For more info, http://wiki.squid-cache.org/ConfigExamples/DynamicContent/YouTube
# Syed Jahanzaib / aacable@hotmail.com
# https://aacable.wordpress.com/2012/01/19/youtube-caching-with-squid-2-7-using-storeurl-pl/
#######################
# Special thanks to some indonesian friends who provided some updates,
## UPDATED on 20 January, 2014 / Syed Jahanzaib

#####################
#### REFERENCES #####  http://www2.fh-lausitz.de/launic/comp/misc/squid/projekt_youtube/
#####################
#####################

use IO::File;
$|=1;
STDOUT->autoflush(1);
$debug=1;        ## recommended:0
$bypassallrules=0;    ## recommended:0
$sucks="";        ## unused
$sucks="sucks" if ($debug>=1);
$timenow="";
$printtimenow=1;      ## print timenow: 0|1
my $logfile = '/tmp/storeurl.log';

open my $logfh, '>>', $logfile
or die "Couldn't open $logfile for appending: $!\n" if $debug;
$logfh->autoflush(1) if $debug;

#### main
while (<>) {
$timenow=time()." " if ($printtimenow);
print $logfh "$timenow"."in : $_" if ($debug>=1);
chop; ## strip eol
@X = split;
$x = $X[0]; ## 0
$u = $X[1]; ## url
$_ = $u; ## url

if ($bypassallrules){
$out="$u";    ## map 1:1

#youtube with range (YOUTUBE has split its videos into segments)
}elsif (m/(youtube|google).*videoplayback\?.*range/ ){
@itag = m/[&?](itag=[0-9]*)/;
@id = m/[&?](id=[^\&]*)/;
@range = m/[&?](range=[^\&\s]*)/;
@begin = m/[&?](begin=[^\&\s]*)/;
@redirect = m/[&?](redirect_counter=[^\&]*)/;
$out="http://video-srv.youtube.com.SQUIDINTERNAL/@itag&@id&@range&@redirect";
#sleep(1);    ## delay loop

#youtube without range
}elsif (m/(youtube|google).*videoplayback\?/ ){
@itag = m/[&?](itag=[0-9]*)/;
@id = m/[&?](id=[^\&]*)/;
@redirect = m/[&?](redirect_counter=[^\&]*)/;
$out="http://video-srv.youtube.com.SQUIDINTERNAL/@itag&@id&@redirect";
#sleep(1);    ## delay loop

#speedtest
}elsif (m/^http:\/\/(.*)\/speedtest\/(.*\.(jpg|txt))\?(.*)/) {
$out="http://www.speedtest.net.SQUIDINTERNAL/speedtest/" . $2 . "";

#mediafire
}elsif (m/^http:\/\/199\.91\.15\d\.\d*\/\w{12}\/(\w*)\/(.*)/) {
$out="http://www.mediafire.com.SQUIDINTERNAL/" . $1 ."/" . $2 . "";

#fileserve
}elsif (m/^http:\/\/fs\w*\.fileserve\.com\/file\/(\w*)\/[\w-]*\.\/(.*)/) {
$out="http://www.fileserve.com.SQUIDINTERNAL/" . $1 . "./" . $2 . "";

#filesonic
}elsif (m/^http:\/\/s[0-9]*\.filesonic\.com\/download\/([0-9]*)\/(.*)/) {
$out="http://www.filesonic.com.SQUIDINTERNAL/" . $1 . "";

#4shared
}elsif (m/^http:\/\/[a-zA-Z]{2}\d*\.4shared\.com(:8080|)\/download\/(.*)\/(.*\..*)\?.*/) {
$out="http://www.4shared.com.SQUIDINTERNAL/download/$2\/$3";

#4shared preview
}elsif (m/^http:\/\/[a-zA-Z]{2}\d*\.4shared\.com(:8080|)\/img\/(\d*)\/\w*\/dlink__2Fdownload_2F(\w*)_3Ftsid_3D[\w-]*\/preview\.mp3\?sId=\w*/) {
$out="http://www.4shared.com.SQUIDINTERNAL/$2";

#photos-X.ak.fbcdn.net where X a-z
}elsif (m/^http:\/\/photos-[a-z](\.ak\.fbcdn\.net)(\/.*\/)(.*\.jpg)/) {
$out="http://photos" . $1 . "/" . $2 . $3  . "";

#YX.sphotos.ak.fbcdn.net where X 1-9, Y a-z
} elsif (m/^http:\/\/[a-z][0-9]\.sphotos\.ak\.fbcdn\.net\/(.*)\/(.*)/) {
$out="http://photos.ak.fbcdn.net/" . $1  ."/". $2 . "";

#maps.google.com
} elsif (m/^http:\/\/(cbk|mt|khm|mlt|tbn)[0-9]?(.google\.co(m|\.uk|\.id).*)/) {
$out="http://" . $1  . $2 . "";

# compatibility for old cached get_video?video_id
} elsif (m/^http:\/\/([0-9.]{4}|.*\.youtube\.com|.*\.googlevideo\.com|.*\.video\.google\.com).*?(videoplayback\?id=.*?|video_id=.*?)\&(.*?)/) {
$z = $2; $z =~ s/video_id=/get_video?video_id=/;
$out="http://video-srv.youtube.com.SQUIDINTERNAL/" . $z . "";
#sleep(1);    ## delay loop

} elsif (m/^http:\/\/www\.google-analytics\.com\/__utm\.gif\?.*/) {
$out="http://www.google-analytics.com/__utm.gif";

#Cache High Latency Ads
} elsif (m/^http:\/\/([a-z0-9.]*)(\.doubleclick\.net|\.quantserve\.com|\.googlesyndication\.com|yieldmanager|cpxinteractive)(.*)/) {
$y = $3;$z = $2;
for ($y) {
s/pixel;.*/pixel/;
s/activity;.*/activity/;
s/(imgad[^&]*).*/\1/;
s/;ord=[?0-9]*//;
s/;&timestamp=[0-9]*//;
s/[&?]correlator=[0-9]*//;
s/&cookie=[^&]*//;
s/&ga_hid=[^&]*//;
s/&ga_vid=[^&]*//;
s/&ga_sid=[^&]*//;
# s/&prev_slotnames=[^&]*//
# s/&u_his=[^&]*//;
s/&dt=[^&]*//;
s/&dtd=[^&]*//;
s/&lmt=[^&]*//;
s/(&alternate_ad_url=http%3A%2F%2F[^(%2F)]*)[^&]*/\1/;
s/(&url=http%3A%2F%2F[^(%2F)]*)[^&]*/\1/;
s/(&ref=http%3A%2F%2F[^(%2F)]*)[^&]*/\1/;
s/(&cookie=http%3A%2F%2F[^(%2F)]*)[^&]*/\1/;
s/[;&?]ord=[?0-9]*//;
s/[;&]mpvid=[^&;]*//;
s/&xpc=[^&]*//;
# yieldmanager
s/\?clickTag=[^&]*//;
s/&u=[^&]*//;
s/&slotname=[^&]*//;
s/&page_slots=[^&]*//;
}
$out="http://" . $1 . $2 . $y . "";

#cache high latency ads
} elsif (m/^http:\/\/(.*?)\/(ads)\?(.*?)/) {
$out="http://" . $1 . "/" . $2  . "";

# spicific servers starts here....
} elsif (m/^http:\/\/(www\.ziddu\.com.*\.[^\/]{3,4})\/(.*?)/) {
$out="http://" . $1 . "";

#cdn, varialble 1st path
} elsif (($u =~ /filehippo/) && (m/^http:\/\/(.*?)\.(.*?)\/(.*?)\/(.*)\.([a-z0-9]{3,4})(\?.*)?/)) {
@y = ($1,$2,$4,$5);
$y[0] =~ s/[a-z0-9]{2,5}/cdn./;
$out="http://" . $y[0] . $y[1] . "/" . $y[2] . "." . $y[3] . "";

#rapidshare
} elsif (($u =~ /rapidshare/) && (m/^http:\/\/(([A-Za-z]+[0-9-.]+)*?)([a-z]*\.[^\/]{3}\/[a-z]*\/[0-9]*)\/(.*?)\/([^\/\?\&]{4,})$/)) {
$out="http://cdn." . $3 . "/SQUIDINTERNAL/" . $5 . "";

} elsif (($u =~ /maxporn/) && (m/^http:\/\/([^\/]*?)\/(.*?)\/([^\/]*?)(\?.*)?$/)) {
$out="http://" . $1 . "/SQUIDINTERNAL/" . $3 . "";

#like porn hub variables url and center part of the path, filename etention 3 or 4 with or without ? at the end
} elsif (($u =~ /tube8|pornhub|xvideos/) && (m/^http:\/\/(([A-Za-z]+[0-9-.]+)*?(\.[a-z]*)?)\.([a-z]*[0-9]?\.[^\/]{3}\/[a-z]*)(.*?)((\/[a-z]*)?(\/[^\/]*){4}\.[^\/\?]{3,4})(\?.*)?$/)) {
$out="http://cdn." . $4 . $6 . "";

#...spicific servers end here.

#photos-X.ak.fbcdn.net where X a-z
} elsif (m/^http:\/\/photos-[a-z].ak.fbcdn.net\/(.*)/) {
$out="http://photos.ak.fbcdn.net/" . $1  . "";

#for yimg.com video
} elsif (m/^http:\/\/(.*yimg.com)\/\/(.*)\/([^\/\?\&]*\/[^\/\?\&]*\.[^\/\?\&]{3,4})(\?.*)?$/) {
$out="http://cdn.yimg.com//" . $3 . "";

#for yimg.com doubled
} elsif (m/^http:\/\/(.*?)\.yimg\.com\/(.*?)\.yimg\.com\/(.*?)\?(.*)/) {
$out="http://cdn.yimg.com/"  . $3 . "";

#for yimg.com with &sig=
} elsif (m/^http:\/\/(.*?)\.yimg\.com\/(.*)/) {
@y = ($1,$2);
$y[0] =~ s/[a-z]+[0-9]+/cdn/;
$y[1] =~ s/&sig=.*//;
$out="http://" . $y[0] . ".yimg.com/"  . $y[1] . "";

#youjizz. We use only domain and filename
} elsif (($u =~ /media[0-9]{2,5}\.youjizz/) && (m/^http:\/\/(.*)(\.[^\.\-]*?\..*?)\/(.*)\/([^\/\?\&]*)\.([^\/\?\&]{3,4})((\?|\%).*)?$/)) {
@y = ($1,$2,$4,$5);
$y[0] =~ s/(([a-zA-A]+[0-9]+(-[a-zA-Z])?$)|(.*cdn.*)|(.*cache.*))/cdn/;
$out="http://" . $y[0] . $y[1] . "/" . $y[2] . "." . $y[3] . "";

#general purpose for cdn servers. add above your specific servers.
} elsif (m/^http:\/\/([0-9.]*?)\/\/(.*?)\.(.*)\?(.*?)/) {
$out="http://squid-cdn-url//" . $2  . "." . $3 . "";

#generic http://variable.domain.com/path/filename."ex" "ext" or "exte" with or withour "? or %"
} elsif (m/^http:\/\/(.*)(\.[^\.\-]*?\..*?)\/(.*)\.([^\/\?\&]{2,4})((\?|\%).*)?$/) {
@y = ($1,$2,$3,$4);
$y[0] =~ s/(([a-zA-Z]+[0-9]+(-[a-zA-Z])?$)|(.*cdn.*)|(.*cache.*))/cdn/;
$out="http://" . $y[0] . $y[1] . "/" . $y[2] . "." . $y[3] . "";

} else {
$out="$u"; ##$X[2]="$sucks";
}
print $logfh "$timenow"."out: $x $out $X[2] $X[3] $X[4] $X[5] $X[6] $X[7]\n" if ($debug>=1);
print "$x $out $X[2] $X[3] $X[4] $X[5] $X[6] $X[7]\n";
}
close $logfh if ($debug);

Save & Exit.

Now create cache dir and assign proper permission to proxy user

>

mkdir /cache-1
chown proxy:proxy /cache-1
chmod -R  777 /cache-1

.

Now  initialize squid cache directories by

squid -z
chmod -R  777 /cache-1

You should see Following message

Creating Swap Directories

.

.

After this, start SQUID by


squid -d1N

Press Enttr , adn the issue this command to make sure squid is running

ps aux |grep squid

You will see few lines with squid name , if yes, congrats, your squid is up and running.

Also note that squid will not auto start by default when system reboots, you have to add an entry in

/etc/rc.local

just add following (before exit 0 command

squid

From Client end, point your browser to use your squid as proxy server and test any video.

.

.

TESTING YOUTUBE CACHING 🙂

Now from test pc, open youtube and play any video, after it download completely, delete the browser cache, and play the same video again, This time it will be served from the cache. You can verify it by monitoring your WAN link utilization while playing the cached file.

Look at the below WAN utilization graph, it was taken while watching the clip which is not in cache

WAN utilization of Proxy, While watching New Clip (Not in cache) ↑

.

.

.

.

.

.

Now Look at the below WAN utilization graph, it was taken while watching the clip which is now in CACHE.

zph

WAN utilization of Proxy, While watching already cached Clip

.

.

Playing Video, loaded from the cache chunk by chunk

It will load first chunk from the cache, if the user keep watching the clip, it will load next chunk , and will continue to do so.

More Cache HIT Example

FACEBOOK VIDEO Cache HIT Example:

1-fb hit

root@proxy:~# tail -f /var/log/squid/access.log |grep HIT

101.11.11.161 - - [18/Sep/2013:09:03:37 +0500] "GET http://video.ak.fbcdn.net/hvideo-ak-ash3/v/722993_237962626354676_1970647760_n.mp4?oh=82f3395ba830a587ae17f03b2e76847d&oe=523941CC&__gda__=1379485913_f5181f37ea7acb27b69b9fced76a380d HTTP/1.1" 200 2369427 TCP_MEM_HIT:NONE

.
.
.

Youtube Vidoes Cache Hit Example

2-yt-cache

You can monitor the CACHE TCP_HIT ENTRIES in squid logs, you can view them by

tail -f /var/log/squid/access.log | grep HIT

10.0.0.161 - - [18/Sep/2013:09:32:05 +0500] "GET http://r5---sn-gvnuxaxjvh-n8ve.c.youtube.com/videoplayback?algorithm=throttle-factor&burst=40&clen=2537620&cp=U0hWTlVLUV9NT0NONl9NRVVDOm9ReG8ybXFFU0hS&cpn=JOisEPFDiHzWwZDK&dur=159.730&expire=1379503285&factor=1.25&fexp=917000%2C912301%2C905611%2C934007%2C914098%2C916625%2C902533%2C924606%2C929117%2C929121%2C929906%2C929907%2C929922%2C929923%2C929127%2C929129%2C929131%2C929930%2C936403%2C925724%2C925726%2C936310%2C925720%2C925722%2C925718%2C925714%2C929917%2C906945%2C929933%2C929935%2C920302%2C906842%2C913428%2C919811%2C935020%2C935021%2C935704%2C932309%2C913563%2C919373%2C930803%2C908536%2C938701%2C931924%2C940501%2C936308%2C909549%2C901608%2C900816%2C912711%2C934507%2C907231%2C936312%2C906001&gir=yes&id=98d455f40d4132a5&ip=93.115.84.195&ipbits=8&itag=140&keepalive=yes&key=yt1&lmt=1370589022851995&ms=au&mt=1379478581&mv=m&range=2138112-2375679&ratebypass=yes&signature=C405B33844DEC9088DD546F2EDEC362737C776E1.5FDB10FD7B4F6C81F884F6FB2ABFDE067D2493A6&source=youtube&sparams=algorithm%2Cburst%2Cclen%2Ccp%2Cdur%2Cfactor%2Cgir%2Cid%2Cip%2Cipbits%2Citag%2Clmt%2Csource%2Cupn%2Cexpire&sver=3&upn=QZy7v7y0uxk HTTP/1.1" 302 1598 TCP_MEM_HIT:NONE
10.0.0.161 - - [18/Sep/2013:09:32:07 +0500] "GET http://r5---sn-gvnuxaxjvh-n8ve.c.youtube.com/videoplayback?algorithm=throttle-factor&burst=40&clen=5380615&cp=U0hWTlVLUV9NT0NONl9NRVVDOm9ReG8ybXFFU0hS&cpn=JOisEPFDiHzWwZDK&dur=159.059&expire=1379503285&factor=1.25&fexp=917000%2C912301%2C905611%2C934007%2C914098%2C916625%2C902533%2C924606%2C929117%2C929121%2C929906%2C929907%2C929922%2C929923%2C929127%2C929129%2C929131%2C929930%2C936403%2C925724%2C925726%2C936310%2C925720%2C925722%2C925718%2C925714%2C929917%2C906945%2C929933%2C929935%2C920302%2C906842%2C913428%2C919811%2C935020%2C935021%2C935704%2C932309%2C913563%2C919373%2C930803%2C908536%2C938701%2C931924%2C940501%2C936308%2C909549%2C901608%2C900816%2C912711%2C934507%2C907231%2C936312%2C906001&gir=yes&id=98d455f40d4132a5&ip=93.115.84.195&ipbits=8&itag=133&keepalive=yes&key=yt1&lmt=1370589028183073&ms=au&mt=1379478581&mv=m&range=4608000-5119999&ratebypass=yes&signature=8A1A558BF931AB3C8F58ADAF55B2488A88B9ADFD.108D982EB17E2F27C829F2521FF611808B4E8CAF&source=youtube&sparams=algorithm%2Cburst%2Cclen%2Ccp%2Cdur%2Cfactor%2Cgir%2Cid%2Cip%2Cipbits%2Citag%2Clmt%2Csource%2Cupn%2Cexpire&sver=3&upn=QZy7v7y0uxk HTTP/1.1" 302 1598 TCP_MEM_HIT:NONE
10.0.0.161 - - [18/Sep/2013:09:32:20 +0500] "GET http://r5---sn-gvnuxaxjvh-n8ve.c.youtube.com/videoplayback?algorithm=throttle-factor&burst=40&clen=2537620&cp=U0hWTlVLUV9NT0NONl9NRVVDOm9ReG8ybXFFU0hS&cpn=JOisEPFDiHzWwZDK&dur=159.730&expire=1379503285&factor=1.25&fexp=917000%2C912301%2C905611%2C934007%2C914098%2C916625%2C902533%2C924606%2C929117%2C929121%2C929906%2C929907%2C929922%2C929923%2C929127%2C929129%2C929131%2C929930%2C936403%2C925724%2C925726%2C936310%2C925720%2C925722%2C925718%2C925714%2C929917%2C906945%2C929933%2C929935%2C920302%2C906842%2C913428%2C919811%2C935020%2C935021%2C935704%2C932309%2C913563%2C919373%2C930803%2C908536%2C938701%2C931924%2C940501%2C936308%2C909549%2C901608%2C900816%2C912711%2C934507%2C907231%2C936312%2C906001&gir=yes&id=98d455f40d4132a5&ip=93.115.84.195&ipbits=8&itag=140&keepalive=yes&key=yt1&lmt=1370589022851995&ms=au&mt=1379478581&mv=m&range=2375680-2615295&ratebypass=yes&signature=C405B33844DEC9088DD546F2EDEC362737C776E1.5FDB10FD7B4F6C81F884F6FB2ABFDE067D2493A6&source=youtube&sparams=algorithm%2Cburst%2Cclen%2Ccp%2Cdur%2Cfactor%2Cgir%2Cid%2Cip%2Cipbits%2Citag%2Clmt%2Csource%2Cupn%2Cexpire&sver=3&upn=QZy7v7y0uxk HTTP/1.1" 302 1598 TCP_MEM_HIT:NONE
10.0.0.161 - - [18/Sep/2013:09:32:22 +0500] "GET http://r5---sn-gvnuxaxjvh-n8ve.c.youtube.com/videoplayback?algorithm=throttle-factor&burst=40&clen=5380615&cp=U0hWTlVLUV9NT0NONl9NRVVDOm9ReG8ybXFFU0hS&cpn=JOisEPFDiHzWwZDK&dur=159.059&expire=1379503285&factor=1.25&fexp=917000%2C912301%2C905611%2C934007%2C914098%2C916625%2C902533%2C924606%2C929117%2C929121%2C929906%2C929907%2C929922%2C929923%2C929127%2C929129%2C929131%2C929930%2C936403%2C925724%2C925726%2C936310%2C925720%2C925722%2C925718%2C925714%2C929917%2C906945%2C929933%2C929935%2C920302%2C906842%2C913428%2C919811%2C935020%2C935021%2C935704%2C932309%2C913563%2C919373%2C930803%2C908536%2C938701%2C931924%2C940501%2C936308%2C909549%2C901608%2C900816%2C912711%2C934507%2C907231%2C936312%2C906001&gir=yes&id=98d455f40d4132a5&ip=93.115.84.195&ipbits=8&itag=133&keepalive=yes&key=yt1&lmt=1370589028183073&ms=au&mt=1379478581&mv=m&range=5120000-5634047&ratebypass=yes&signature=8A1A558BF931AB3C8F58ADAF55B2488A88B9ADFD.108D982EB17E2F27C829F2521FF611808B4E8CAF&source=youtube&sparams=algorithm%2Cburst%2Cclen%2Ccp%2Cdur%2Cfactor%2Cgir%2Cid%2Cip%2Cipbits%2Citag%2Clmt%2Csource%2Cupn%2Cexpire&sver=3&upn=QZy7v7y0uxk HTTP/1.1" 302 1598 TCP_MEM_HIT:NONE

.

DAILYMOTION  Videos Cache Hit Example

dailymotion-cache-working-

Videos that are not in cache
↓
101.11.11.161 - - [30/Sep/2013:10:45:25 +0500] "GET http://proxy-62.dailymotion.com/sec(4f636a4f77762894959440a2a4bbc73f)/frag(1)/video/233/073/54370332_mp4_h264_aac.flv HTTP/1.1" 200 932336 TCP_MISS:DIRECT
101.11.11.161 - - [30/Sep/2013:10:45:31 +0500] "GET http://proxy-62.dailymotion.com/sec(4f636a4f77762894959440a2a4bbc73f)/frag(2)/video/233/073/54370332_mp4_h264_aac.flv HTTP/1.1" 200 580913 TCP_MISS:DIRECT
101.11.11.161 - - [30/Sep/2013:10:45:41 +0500] "GET http://proxy-62.dailymotion.com/sec(4f636a4f77762894959440a2a4bbc73f)/frag(3)/video/233/073/54370332_mp4_h264_aac.flv HTTP/1.1" 200 655602 TCP_MISS:DIRECT
101.11.11.161 - - [30/Sep/2013:10:45:51 +0500] "GET http://proxy-62.dailymotion.com/sec(4f636a4f77762894959440a2a4bbc73f)/frag(4)/video/233/073/54370332_mp4_h264_aac.flv HTTP/1.1" 200 545532 TCP_MISS:DIRECT
101.11.11.161 - - [30/Sep/2013:10:46:02 +0500] "GET http://proxy-62.dailymotion.com/sec(4f636a4f77762894959440a2a4bbc73f)/frag(5)/video/233/073/54370332_mp4_h264_aac.flv HTTP/1.1" 200 645288 TCP_MISS:DIRECT
↑


Videos CACHE_HIT that are in cache

101.11.11.161 - - [30/Sep/2013:11:07:26 +0500] "GET http://proxy-62.dailymotion.com/sec(4f636a4f77762894959440a2a4bbc73f)/frag(1)/video/233/073/54370332_mp4_h264_aac.flv HTTP/1.1" 200 932345 TCP_MEM_HIT:NONE
101.11.11.161 - - [30/Sep/2013:11:07:31 +0500] "GET http://proxy-62.dailymotion.com/sec(4f636a4f77762894959440a2a4bbc73f)/frag(2)/video/233/073/54370332_mp4_h264_aac.flv HTTP/1.1" 200 580922 TCP_MEM_HIT:NONE
101.11.11.161 - - [30/Sep/2013:11:07:43 +0500] "GET http://proxy-62.dailymotion.com/sec(4f636a4f77762894959440a2a4bbc73f)/frag(3)/video/233/073/54370332_mp4_h264_aac.flv HTTP/1.1" 200 655611 TCP_MEM_HIT:NONE
101.11.11.161 - - [30/Sep/2013:11:07:52 +0500] "GET http://proxy-62.dailymotion.com/sec(4f636a4f77762894959440a2a4bbc73f)/frag(4)/video/233/073/54370332_mp4_h264_aac.flv HTTP/1.1" 200 545541 TCP_MEM_HIT:NONE
101.11.11.161 - - [30/Sep/2013:11:08:03 +0500] "GET http://proxy-62.dailymotion.com/sec(4f636a4f77762894959440a2a4bbc73f)/frag(5)/video/233/073/54370332_mp4_h264_aac.flv HTTP/1.1" 200 645297 TCP_MEM_HIT:NONE
101.11.11.161 - - [30/Sep/2013:11:08:12 +0500] "GET http://proxy-62.dailymotion.com/sec(4f636a4f77762894959440a2a4bbc73f)/frag(6)/video/233/073/54370332_mp4_h264_aac.flv HTTP/1.1" 200 551354 TCP_MEM_HIT:NONE

and some more

101.11.11.161 - - [01/Oct/2013:12:05:45 +0500] "GET http://vid2.ak.dmcdn.net/sec(bb78b176d5d55fa2a74cc2b3a7d9fc1a)/frag(1)/video/235/784/69487532_mp4_h264_aac_hq.flv HTTP/1.1" 200 460619 TCP_MISS:DIRECT
101.11.11.161 - - [01/Oct/2013:12:05:45 +0500] "GET http://vid2.ak.dmcdn.net/sec(d1bc558a82841a2be2990fb944e0d603)/frag(2)/video/235/784/69487532_mp4_h264_aac_ld.flv HTTP/1.1" 200 242336 TCP_MEM_HIT:NONE
101.11.11.161 - - [01/Oct/2013:12:05:54 +0500] "GET http://vid2.ak.dmcdn.net/sec(09b97b67e9cdc1d4f4e41f2ddf6d027b)/frag(3)/video/235/784/69487532_mp4_h264_aac.flv HTTP/1.1" 200 361845 TCP_MEM_HIT:NONE
101.11.11.161 - - [01/Oct/2013:12:06:26 +0500] "GET http://vid2.ak.dmcdn.net/sec(09b97b67e9cdc1d4f4e41f2ddf6d027b)/frag(4)/video/235/784/69487532_mp4_h264_aac.flv HTTP/1.1" 200 384313 TCP_MISS:DIRECT

.

AOL Videos Cache Hit Example

aol

.

MSN Videos Cache Hit Example

msn-videos.

101.11.11.161 - - [27/Sep/2013:13:03:31 +0500] "GET http://content4.catalog.video.msn.com/e2/ds/6af0b936-2895-48dd-bbb7-c26803b957ab.mp4 HTTP/1.1" 200 9349059 TCP_HIT:NONE

.

TUNE.PK Videos Cache Hit Example

tune.pk

.

101.11.11.161 - - [19/Sep/2013:09:48:02 +0500] "GET http://storage4.tunefiles.com/files/videos/2013/06/26/1372274819407c1.flv HTTP/1.1" 200 5338729 TCP_HIT:NONE

.

BLIP.TV Videos Cache Hit Example

blip.tv.


101.11.11.161 - - [27/Sep/2013:12:45:27 +0500] "GET http://j46.video2.blip.tv/6640012033790/TornadoTitans-Season3Episode10Twins738.m4v?ir=12035&sr=1835 HTTP/1.1" 200 20540163 TCP_HIT:NONE

.
.

APNIISP.COM Audio & Videos Cache Hit Example

101.11.11.161 - - [27/Sep/2013:12:33:09 +0500] "GET http://songs.apniisp.com/videos/Qismat%20Apnay%20Haat%20Mein%20(Apniisp.Com).wmv HTTP/1.1" 200 94714 TCP_HIT:NONE
101.11.11.161 - - [27/Sep/2013:12:33:10 +0500] "GET http://songs.apniisp.com/videos/Qismat%20Apnay%20Haat%20Mein%20(Apniisp.Com).wmv HTTP/1.1" 304 333 TCP_IMS_HIT:NONE

VIMEO Videos Cache Hit Example

vimeo.


101.11.11.161 - - [27/Sep/2013:10:48:50 +0500] "GET http://pdl.vimeocdn.com/30816/658/190006311.mp4?aktimeoffset=0&aksessionid=308dc46bc6745f77ce229322a3b25d51&token=1380268125_e9b9f3afe81c729f378cae518631a643 HTTP/1.1" 200 90581259 TCP_HIT:NONE

 

.

.

Regard’s

Syed Jahanzaib

January 14, 2012

Howto Recover Mikrotik ADMIN account Forgotten Password


Last Updated: 18th April, 2014, 22:04 gmt+5

According to information on Mikrotik WIKI and forums, it is not possible to recover the passwords without resetting whole mikrotik box (resulting in loss of all configuration also). However following are few methods to recover the password.

0# Recover password from BACKUP file using a website https://www.mikrotikpasswordrecovery.net/

If you require it on urget basis, you can email me your config, and I will recover it for you, just in case if you dont have linux or urgency is required.

1# Recover password from BACKUP file using Ubuntu or Linux LIVE CD [updated April 2014]

2# Recover password by mounting Mikrotik Hard disk in Linux LIVE CD  and do recovery [not updated since last year, it was tested with 5 version)

 

 

2# Recover password from BACKUP file using Linux [working as of april 2014]

Login to your Ubuntu / Linux Box,
Download mikrotik password recovery tool and compile it by following commands,

sudo apt-get update
sudo apt-get install build-essential g++ libssl-dev libcurl4-gnutls-dev libexpat1-dev gettext libz-dev
mkdir /temp
cd /temp
wget http://manio.skyboo.net/mikrotik/mtpass-0.9.tar.bz2
tar jxvf mtpass-0.9.tar.bz2
cd mtpass-0.9
make

#If you receive *error* after issuing make command, then you can use following command to compile it in desi JUGAAR (workaround) way hehehehe
g++ mtpass.cpp -lgnutls-openssl -o mtpass

Now upload/copy your Mikrotik Backup File to /temp folder ( Either using WINSCP gui tool, WEBMIN File Manager,  via USB or any other method you like)

Now Issue the following command

 ./mtpass /temp/zaibmikrotik.backup

It will show you all account passwords in few seconds.
As shown in the image below . . .

mtpass.

.

.

 

3# Recover password of x86 PC Version

Boot from Ubuntu LIVE CD
(I used Ubuntu 9.1 Desktop CD in this example, you can download it from following link.

http://old-releases.ubuntu.com/releases/karmic/ubuntu-9.10-desktop-i386.iso

Select “Try Ubuntu
As Shown in the image below . . .

After you see Desktop, Open TERMINAL from APPLICATION > ACCESSORIES > TERMINAL

Now change to root user by
sudo su

Now check your partitions by issuing
fdisk -l

you will see something like below image.

You partition can be different, use your judgment to see what partition mt is in, either by mounting it one by one.

Now mount it any folder , for example

mkdir /temp
mount -t auto /dev/sda2 /temp

Now check its content by ls /temp and you will something like below image

*********************************************************

*********************************************************

Now Copy the ‘mikrotik password file’, (in newer ROS , it is located in /rw/store/user.dat) to USB flash drive, It will be used to decode the password.
(The USB flash drive should be ‘plug and play’ in Ubuntu Live. Just plug it in usb port and it will appear on desktop in few seconds , OR you can also copy it your network pc via configuring interface lan card accordingly)
(Note: In older ROS it is /nova/store/user.dat)

Now shutdown live cd.

then Boot from your Ubuntu box, and use the Method # 2 , described earlier in this article to decode password using mtpass tool  from this file.

Copy the user.dat file where you have decompressed mt password tool . e.g /temp , now use the mt pass tool to recover password from this file.

./mtpass user.dat

and you will see your admin password.

As Now you have seen the password now, you can login into mikrotik pc router  with your Old Password 🙂 🙂 🙂


If your architecture is ROUTERBOARD RB series, Please follow the below guide.

http://manio.skyboo.net/mikrotik/

Also it’s possible for you to send an SMS to your router, tell it to run a script (parameters supported) and the router can even respond, as it also supports SMS sending! More here:
http://wiki.mikrotik.com/wiki/Sms

You can setup an script just in case you forgot your password , and via sending sms, it can reset it to default. or send you return the new/original password.

Regard’s
SYED JAHANZAIB
https://aacable.wordpress.com

January 9, 2012

Using WEBMIN to create Forward/Reverse ZONE in BIND / UBUNTU


For some people , installing and configuring BIND for local network using CLI can be a typical task. However it can be done easily in few steps via using GUI base controller known as WEBMIN which is a very nice and powergul GUI to manage  almost every service of your Linux box via your browser.

In this article we will discuss following topics.

1# Install WEBMIN (Using 2 different methods)
2# Install BIND (Using WEBMIN)
3# Create Forward Zone for our domain
4# Create Reverse Zones for our domain
5# Test Zones via nslookup

1# Install WEBMIN Using DKPKG

First we need to install webmin. you can either download it from webmin site and install it by issuing following commands.

wget http://prdownloads.sourceforge.net/webadmin/webmin-1.570_all.deb
# then run the following command
dpkg --install webmin_1.570_all.deb

You should now be able to login to Webmin via your browser using the URL
http://192.168.2.1:10000/

OR
https://192.168.2.1:10000/

IF the above method fails to install WEBMIN on your system, Follow this alternate method to install it.

1# Install WEBMIN Using the Webmin APT repository [Alternate Way]

install and update Webmin via APT, edit the /etc/apt/sources.list file by using the following command

nano /etc/apt/sources.list
and add these lines at end.

deb http://download.webmin.com/download/repository sarge contrib
deb http://webmin.mirror.somersettechsolutions.co.uk/repository sarge contrib

Now Save the file and exit.

You should also fetch and install my GPG key with which the repository is signed, with the commands :

cd /root
wget http://www.webmin.com/jcameron-key.asc
apt-key add jcameron-key.asc

You will now be able to install with the commands :

apt-get update
apt-get install webmin

For more information on other distribution installation , please visit http://www.webmin.com

2# INSTALLING BIND via webmin.

After logging in to Webmin ,
Goto SERVERS > BIND DNS Server

It will inform you that bind is not installed, install it via click on “Click Here
As shown in the image below . . .


Now it will download BIND and install it. Make sure your Linux box is connected with the internet. After installation it will show you something
As shown in the image below . . .

Now that BIND is installed, We have to create Forward and Reverse zones for our new domain. In last we will check it by doing nslookup.

3# Adding FORWARD ZONE for our domain

Goto SERVERS > BIND DNS Server

Now you will see BIND DNS SERVER and various icons.
First we have to create forward zone,
Click on CREATE MASTER ZONE
As shown in the image below . . .

> in Zone Type : Select Forward
> in Domain name / Network , type the name you want to configure your domain with.
As shown in the image below . . .

4# Adding REVERSE ZONE for our domain

Goto SERVERS > BIND DNS Server
Now we have to create Reverse Zone,
Click on CREATE MASTER ZONE
As shown in the image below . . .

Now on your TOP RIGHT, click on APPLY CHANGES.

5# TESTING YOUR NEWLY CREATED ZONE using nslookup

Login to your client pc, open command prompt and issue the following command.
nslookup – 192.168.2.1

now type zaib.com and you should see your Linux box ip in response.
As shown in the image below . . .

All Done. Now you can careate hosts record / MX record etc using GUI base DNS MANAGEMENT CONSOLE 🙂

BIND TIP’S N TRICKS

*** Can’t find server name *** ERROR

When you do NSLOOKUP, you see an error something like.
*** Can’t find server name for address 192.168.2.1: Non-existent domain, Default ServeR: UnKnown

To solve this , jsut add reverse lookup entry for your domain.
Goto SERVERS > BIND DNS Server
Open your newly created zone in EXISTING DNS ZONES
As shown in the image below . . .

After opening zaib.com or your zone,
Click on ADDRESS
Here create an address
In Name: zaib.com
In Address: 192.168.2.1
Click on CREATE and then APPLY CHANGES.
Now from you client side, try to do nslookup, and this time you wont see any error.

Create MX record for MAIL Server 🙂

Creating an MX Record (Mail Exchange Record).

1) Open your zone , for example zaib.com ,
Click on Mail Server
2) For the name, type in the domain e.g zaib.com
3) For the mail server, type in zaib.com
4) Set the priority to what you like.. 10 is usually default.
5) Click on SAVE and aplpy changes.

Make sure your Linux box have your BIND Server IP at top.

Edit your resolv.conf and set dns name server search order.

# Generated by NetworkManager
search zaib.com
nameserver 127.0.0.1
nameserver 192.168.2.4

Adding Forwarding Entry (For resolving External Hosts)

Goto SERVERS > BIND DNS Server
Click on Forwarding and Transfers
In Servers to forward queries to, type your ISP’s DNS Server IP
As shown in the image below . . .

Now when your client will do query , BIND will first check in zaib.com, if it doesn’t fin any entry, it will forward this query to ISP’S DNS Server.

Have Fun 🙂

 SYED JAHANZAIB

January 5, 2012

QOS with Mikrotik [Reference Guide]



QOS With Mikrotik

Following are few scenarios for some examples : I will add more soon.

1# Limit user traffic using PCQ (also useful for Hotspot Bypassed MAC address)

To limit all users 192.168.1.0/24 to 512kb epr user, using PCQ, use following script.


/queue type

add kind=pcq name=download-512kb pcq-burst-rate=0 pcq-burst-threshold=0 pcq-burst-time=10s pcq-classifier=dst-address pcq-dst-address-mask=32 pcq-dst-address6-mask=64 pcq-limit=50 pcq-rate=524288 \
pcq-src-address-mask=32 pcq-src-address6-mask=64 pcq-total-limit=2000

add kind=pcq name=upload-512kb pcq-burst-rate=0 pcq-burst-threshold=0 pcq-burst-time=10s pcq-classifier=src-address pcq-dst-address-mask=32 pcq-dst-address6-mask=64 pcq-limit=50 pcq-rate=524288 \
pcq-src-address-mask=32 pcq-src-address6-mask=64 pcq-total-limit=2000

/queue simple
add burst-limit=0/0 burst-threshold=0/0 burst-time=0s/0s comment="Limit every Users at 512kb using PCQ. Can be used for hotspot BYPASSED macs too. Can be used for multi purpose  Syed Jahanzaib" \
direction=both disabled=no interface=all limit-at=0/0 max-limit=0/0 name=512k-limit packet-marks="" parent=none priority=8 queue=upload-512kb/download-512kb target-addresses=192.168.1.0/24 \
total-queue=default-small

Change the Bandwidth and Target IP addresses to meet your local requirements.
Bandwidth example:
512k = 524288
1mb = 1048576
2mb = 2097152
4mb = 4194304

Result as showed in the image below . . .

queue-pcq-per-user

2# Allowing specific extension Low Priority over other traffic

In this example we are marking traffic via mangle rules. .iso extension is marked as low priority traffic , and all other traffic is marked as hi priority traffic.

For example We have 256kb internet bandwidth in total. and we want that when users are downloading any .iso file, it gets low priority over other traffic i.e browsing etc. Use the following.

/ip firewall mangle

add action=mark-connection chain=postrouting comment="Mark Conn for .iso Ext" content=iso disabled=no new-connection-mark=iso-conn passthrough=yes

add action=mark-packet chain=postrouting comment="Mark Pkts for iso-conn Ext" connection-mark=iso-conn disabled=no new-packet-mark=lo-prio-traffic-pkts passthrough=no

add action=mark-connection chain=postrouting comment="Mark Conn for all other traffic" content=!iso disabled=no new-connection-mark=hi-prio-traffic-conn passthrough=yes

add action=mark-packet chain=postrouting comment="Mark Pkts for all other traffic" connection-mark=hi-prio-traffic-conn disabled=no new-packet-mark=hi-prio-traffic-pkts passthrough=no

/queue simple add name=wan_conn_limit interface=ether1 max-limit=256k/256k

/queue simple add name=hi-prio-traffic interface=ether1 parent=wan_conn_limit packet-marks=hi-prio-traffic-pkts priority=1

/queue simple add name=lo-prio-traffic packet-marks=lo-prio-traffic-pkts interface=ether1 parent=wan_conn_limit priority=8

3# Allowing Specific File Extensions High / Limited / Unlimited Bandwidth

For example, You have a network and every user have there bandwidth limited at 256kb. Now you want that if a user is downloading .FLV video file , He can view/download it at unlimited speed regardless of his allowed speed limit, i.e 256kb package, Use the following.

First mark all packets with .flv extention.

/ip firewall mangle
add action=mark-connection chain=postrouting comment="Mark Conn for .flv Ext" content=flv disabled=no new-connection-mark=flv-conn passthrough=yes

add action=mark-packet chain=postrouting comment="Mark Pkts for flv-conn Ext" connection-mark=flv-conn disabled=no new-packet-mark=flv passthrough=no

Now Create a Queue Tree and Set Unlimited or Limited Bandwidth (OR As per your requirements if you want to allot specific amount, set it) to Marked Packets

/queue tree
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=100M max-limit=100M name=Unlimited-Speed-4-Marked-Pkts packet-mark=flv parent=global-out priority=1 queue=defaul

You can use the same in reverse to limit specific extension type bandwidth usage.

4# Allowing Specific Ports High Priority over other traffic

First we need to mark protocols. (In this example we are using SMTP port 25 )

/ip firewall mangle add chain=prerouting protocol=tcp port=25 connection-state=new action=mark-connection new-connection-mark=hi_prio_conn
/ip firewall mangle add chain=prerouting connection-mark=hi_prio_conn action=mark-packet new-packet-mark=hi_prio_conn_pkts

Now we will create Simple Queue and give high priority to marked packets and other packets low priority. (In this example we have 2Mb WAN connection)

/queue simple add name=wan_conn_limit interface=ether1 max-limit=2M/2M
/queue simple add name=prio interface=ether1 parent=wan_conn_limit packet-marks=hi_prio_conn_pkts priority=1
/queue simple add name=other interface=ether1 parent=Internet priority=8

Now SMTP traffic will get higher priority over other traffic.

5# Equal Distribution of Bandwidth for a number of users using PCQ

If you have a 512 kbps WAN connection and you want to share this equally among your users  but if only one pc is active it should have the full 512 kbps, if 2 pcs are active 256 kbps each and vise versa.
As shown in the image below . . .


Use the following:

/queue type add name="PCQ_download" kind=pcq pcq-rate=512k pcq-classifier=dst-address
/queue type add name="PCQ_upload" kind=pcq pcq-rate=512k pcq-classifier=src-address

/queue simple add queue=PCQ_upload/PCQ_download target-addresses=192.168.2.0/24

6# Limiting Single User Bandwidth via Simple QUEUE (Lookout for order number)

/queue simple add name="Limiting Zaib to 256kb" target-addresses=192.168.2.6 max-limit=256k/256k

7# Simple Queue with BURST

Following simple queue with BURST

– Limit user at 64kb in general.
– When the user will download at full 64kbps speed, he will be able to burst upto 256kb for 5 seconds. after 5 seconds , user will fall to 64kb again for next 5 seconds.

In short 5 seconds on load 256kbps, and next 5 seconds, 64kbp.

/queue simple

add burst-limit=256k/256k burst-threshold=128k/128k burst-time=20s/20s direction=both disabled=no interface=all limit-at=0/0 max-limit=64k/64k name=queue1 \

packet-marks="" parent=none priority=8 queue=default-small/default-small target-addresses=172.16.0.10/32 total-queue=default-small

.

.

.

8# Give specific web site assigned Bandwidth on per user basis [updated: 14th April, 2014]

For example you want to limit bandwidth on per user basis for SPECIFIC WEBSITE ONLY. Let’s say 4mb per user for facebook.com
The logic is simple.
1- First create a script that adds the web site ip to an address list.
2- Add scheduler that runs above script after every 5 minutes so that even if the web site ip gets changes, it will update accordingly.
3- Now mark connection and packets for above created address list.
4- Add PCQ queue type (Bandwidth that will be distributed on per user basis using single simple queue)
5- Finally add a simple queue that will distribute bandwidth for marked packets using PCQ for per user basis.


/system script
add name=facebook-list policy=ftp,reboot,read,write,policy,test,winbox,password,sniff,sensitive,api source="# Script to add Facebook DNS IP addressess \r\
\n# Syed Jahanzaib / aacable@hotmail.com\r\
\n:log warning \"Script Started ... Adding Facebook DNS ip's to address list name   facebook_dns_ips\"\r\
\n:foreach i in=[/ip dns cache find] do={\r\
\n:local bNew \"true\";\r\
\n:local cacheName [/ip dns cache all get \$i name] ;\r\
\n:if ([:find \$cacheName \"facebook\"] != 0) do={\r\
\n:local tmpAddress [/ip dns cache get \$i address] ;\r\
\n:put \$tmpAddress;\r\
\n:if ( [/ip firewall address-list find ] = \"\") do={\r\
\n:log info (\"added entry: \$[/ip dns cache get \$i name] IP \$tmpAddress\");\r\
\n/ip firewall address-list add address=\$tmpAddress list=facebook_dns_ips comment=\$cacheName;\r\
\n} else={\r\
\n:foreach j in=[/ip firewall address-list find ] do={\r\
\n:if ( [/ip firewall address-list get \$j address] = \$tmpAddress ) do={\r\
\n:set bNew \"false\";\r\
\n}\r\
\n}\r\
\n:if ( \$bNew = \"true\" ) do={\r\
\n:log info (\"added entry: \$[/ip dns cache get \$i name] IP \$tmpAddress\");\r\
\n/ip firewall address-list add address=\$tmpAddress list=facebook_dns_ips comment=\$cacheName;\r\
\n}\r\
\n}\r\
\n}\r\
\n}\r\
\n# Script Ended..."


/system scheduler
add comment="Add Facebook IP's to address list name facebook-list after every 5 minutes / zaib" disabled=no interval=5m name=add_fb_to_list_scheduler_every_5mnts on-event=facebook-list policy=\
ftp,reboot,read,write,policy,test,winbox,password,sniff,sensitive,api start-date=feb/11/2014 start-time=00:00:00


/ip firewall mangle
add action=mark-connection chain=postrouting comment="Mark Conn for FB Site / zaib" disabled=no dst-address-list=facebook_dns_ips new-connection-mark=FB-conn passthrough=yes
add action=mark-packet chain=postrouting comment="Mark Packtes for FB-CONN / zaib" connection-mark=FB-conn disabled=no new-packet-mark=FB_Packets passthrough=no


/queue type
add kind=pcq name=Download-4mb pcq-burst-rate=0 pcq-burst-threshold=0 pcq-burst-time=10s pcq-classifier=dst-address pcq-dst-address-mask=32 pcq-dst-address6-mask=64 pcq-limit=50 pcq-rate=4194304 \
pcq-src-address-mask=32 pcq-src-address6-mask=64 pcq-total-limit=2000
add kind=pcq name=Upload-4mb pcq-burst-rate=0 pcq-burst-threshold=0 pcq-burst-time=10s pcq-classifier=src-address pcq-dst-address-mask=32 pcq-dst-address6-mask=64 pcq-limit=50 pcq-rate=4194304 \
pcq-src-address-mask=32 pcq-src-address6-mask=64 pcq-total-limit=2000

/queue simple
add burst-limit=0/0 burst-threshold=0/0 burst-time=0s/0s comment="Limit FB speed 4mb per user using PCQ  / Syed Jahanzaib" direction=both disabled=no interface=all limit-at=0/0 max-limit=0/0 name=\
4mb_Limit_For_FB_Per_User packet-marks=FB_Packets parent=none priority=8 queue=Upload-4mb/Download-4mb target-addresses="" total-queue=default-small

 

Do remember, its just an example to show you how you can twist things, You can modify it as per your requirements 🙂

Syed Jahanzaib

Older Posts »

%d bloggers like this: