Automated Installation Script for Squid 2.7 Stable 9 with Video Caching support

For updated video caching support witgh Youtube, you can use LUSCA as it is working fine (as of July 2014)

https://aacable.wordpress.com/2014/04/21/howto-cache-youtube-with-squid-lusca-and-bypass-cached-videos-from-mikrotik-queue/

 

---

 

---

---

---

---

---

 

Following is an automated script to install SQUID 2.7 Stable 9 for UBUNTU with aggressive contents & few video web sites video caching support as described in my other article @
https://aacable.wordpress.com/2012/01/19/youtube-caching-with-squid-2-7-using-storeurl-pl/

I will add more n more functions as soon as I get some free time, like configurable options via choice menu like cache size, mem, and other variables.

Revision History . . .

[ 6th January, 2014]    Removed few bugs including “unable to start squid” error in end, while actually squid is running
[ 7th January, 2014]    Download URL have been changed to http://aacable.rdo.pt/files/linux_related
[ 8th January, 2014]    O.S check for UBUNTU added
[ 8th January, 2014]    root user check added
[ 8th January, 2014]    Function added to remove previously downloaded files in /temp folder to avoid duplication
[ 8th January, 2014]    Internet and $URL availability check
[ 20th January, 2014]  Few refresh patterns + storeurl.pl updated for better YT HIT
[ 29th January, 2014]  aacable.rdo.pt was down dueto quota limit, now added secondary link URL2 = http://wifismartzone.com/files/linux_related/
[ 30th January, 2014]  64bit installation issue fixed.

▼

SCRIPT FUNCTIONS . . . 

This script will do the following

– First it will ask you if you want 32bit or 64 squid compile options
– Update Ubuntu
– Install some components required for Compilation of squid package
– Backup squid.conf if already in /etc/squid.conf with squid.conf.old, stop any running squid instance
– Download Squid 2.7 stable 9 source package to /temp folder and compile it
– Download squid.conf and storeurl.pl from the internet and place them in /etc/squid.conf
– Create cache directory like in /cache-1 and default cache size is 10 GB
– add squid in /etc/rc.local so it may start auto upon system reboot  [added: 5th January, 2014]

Note: You should modify all options in /etc/squid.conf after installation , like cache_dir, cache_mem and others as per your network and hardware specifications.

.

REQUIREMENTS . . .

1- Fresh Installation of UBUNTU OS and Configure Internet Access
2- root access to execute script
3- REMOVE ANY KIND OF PREVIOUSLY INSTALLED SQUID INSTALLATION IF ANY
4- Upload or create script in any folder of Ubuntu box,

or create new script with following commands

mkdir /temp
cd /temp
touch squid_2.7.9_install.sh
chmod +x squid_2.7.9_install.sh
nano squid_2.7.9_install.sh

and paste the following code . . .

#!/bin/bash
# Version 1.3 / 31st January, 2014
# SQUID 2.7 Stable9 Automated Installation Script for Ubuntu 10/12 32bit Edition
# Syed Jahanzaib / aacable @ hotmail.com  / https://aacable.wordpress.com

# Setting Variables . . . [JZ]
#URL=http://aacable.rdo.pt/files/linux_related/
URL=http://wifismartzone.com/files/linux_related/
SQUID_DIR="/etc/squid"
CACHE_DIR="/cache-1"
pid=`pidof squid`
osver=`cat /etc/issue |awk '{print $1}'`

# Colors Config  . . . [[ JZ . . . ]]
ESC_SEQ="\x1b["
COL_RESET=$ESC_SEQ"39;49;00m"
COL_RED=$ESC_SEQ"31;01m"
COL_GREEN=$ESC_SEQ"32;01m"

# OS checkup for UBUNTU
echo -e "$COL_GREEN Squid 2.7.9 Automated Installation Script ver 1.3 for Ubuntu . . .$COL_RESET"
echo -e "$COL_GREEN Checking OS version, as it must be Ubuntu in order to Continue . . .$COL_RESET"
if [[ $osver == Ubuntu ]]; then
echo
echo -e "$COL_GREEN Ubuntu is installed with following information fetched. $COL_RESET"
lsb_release -a
sleep 3
else
echo -e "$COL_RED Sorry, it seems your Linux Distribution is not UBUNTU . Exiting ...$COL_RESET"
exit 1
fi

# Make sure only root can run our script / Checking if user is root, otherwise exit with error [[Jz]]
echo
echo -e "$COL_GREEN Verifying if you are logged in with root privileges  . . .$COL_RESET" 1>&2
FILE="/tmp/out.$$"
GREP="/bin/grep"
if [ "$(id -u)" != "0" ]; then
echo
echo -e "$COL_RED This script must be run as root, switch to root now . . .$COL_RESET" 1>&2
exit 1
fi

# Clearing previous download if any in /temp folder
echo
echo -e "$COL_GREEN Clearing previous downloads if any in /temp folder to avoid duplication$COL_RESET"
sleep 3
rm -fr /temp/squid.conf
rm -fr /temp/storeurl.txt
rm -fr /temp/storeurl.pl
rm -fr /temp/squid-2.7.STABLE9*

# Checking IF $URL is accessible m if YES then continue further , otherwise EXIT the script with ERROR ! [[ JZ .. . .]]
echo
echo -e "$COL_GREEN Checking if $URL is accessible in order to proceed further. . .!! $COL_RESET"
cd /temp
wget -q $URL/squid.conf
{
if [ ! -f /temp/squid.conf ]; then
echo
echo -e "$COL_RED ERROR: Unable to contact $URL, or possibly itnernet is not working or your IP is in black list at destination server  !! $COL_RESET"
echo -e "$COL_RED ERROR: Please check manual if $URL is accessible or not or if it have required files, JZ  !! $COL_RESET"
exit 0
fi
}
rm -fr /temp/squid.conf
sleep 6
# Moving further . . .

clear
echo -e "$COL_GREEN You are logged in with root ID, Ok to proceed further . . .!! $COL_RESET"
echo
#################################################################
# Selection choice of SQUID 2.7  32bit or 64bit architecture [zaib]
echo -e "$COL_GREEN ********************************************** $COL_RESET"
echo -e "$COL_GREEN   Select the type of your OS Architecture JZ   $COL_RESET"
echo -e "$COL_GREEN ********************************************** $COL_RESET"
echo
echo
echo "1. 32bit Ububtu [If you have 32bit OS]"
echo "2. 64bit ubuntu [If you have 64bit OS]"
echo
echo -n "Choose an option: [1] "

read squidchoice
if [ -z "$squidchoice" ]; then
squidchoice="1"
fi
case $squidchoice in
1 ) echo "Selected Squid version is 32bit"
squidver="./configure --prefix=/usr --exec_prefix=/usr --bindir=/usr/sbin --sbindir=/usr/sbin --libexecdir=/usr/lib/squid --sysconfdir=/etc/squid --localstatedir=/var/spool/squid --datadir=/usr/share/squid --enable-async-io=24 --with-aufs-threads=24 --with-pthreads --enable-storeio=aufs --enable-linux-netfilter --enable-arp-acl --enable-epoll --enable-removal-policies=heap,lru --with-aio --with-dl --enable-snmp --enable-delay-pools --enable-htcp --enable-cache-digests --disable-unlinkd --enable-large-cache-files --with-large-files --enable-err-languages=English --enable-default-err-language=English --with-maxfd=65536"
squidlabel="Squid 32bit"
sleep 1
;;
2 ) echo "Selected Squid version is 64bit"
squidver="./configure --prefix=/usr --exec_prefix=/usr --bindir=/usr/sbin --sbindir=/usr/sbin --libexecdir=/usr/lib/squid --sysconfdir=/etc/squid --localstatedir=/var/spool/squid --datadir=/usr/share/squid --enable-async-io=24 --with-aufs-threads=24 --with-pthreads --enable-storeio=aufs --enable-linux-netfilter --enable-arp-acl --enable-epoll --enable-removal-policies=heap,lru --with-aio --with-dl --enable-snmp --enable-delay-pools --enable-htcp --enable-cache-digests --disable-unlinkd --enable-large-cache-files --with-large-files --enable-err-languages=English --enable-default-err-language=English --with-maxfd=65536 --enable-carp --enable-follow-x-forwarded-for --with-maxfd=65536 build_alias=amd64-debian-linux host_alias=amd64-debian-linux target_alias=amd64-debian-linux"
squidlabel="Squid 64bit"

echo .
sleep 3

# If typ anythign other then 1 adn 2, then give invalid error message and exit [zaib]
;;
* ) echo "Invalid selection. Installation aborted."
echo
exit
;;
esac

################################################################## [zaib]
echo
echo -e "$COL_GREEN Updating Ubuntu first . . . !! $COL_RESET"
apt-get update

echo
echo

echo -e "$COL_GREEN Installing required components . . . !! $COL_RESET"
sleep 3
apt-get -y install gcc build-essential sharutils ccze libzip-dev automake1.9

# Clearing OLD data files . . .
{
if [ -f $SQUID_DIR/squid.conf ]; then
echo
echo
echo -e "$COL_RED Previous SQUID configuration file found in $SQUID_DIR ! renaming it for backup purpose . . . $COL_RESET"
mv $SQUID_DIR/squid.conf $SQUID_DIR/squid.conf.old
else
echo
echo
echo -e "$COL_GREEN No Previous Squid configuration have been found in $SQUID_DIR. Proceeding further $COL_RESET"
fi
}

# Checking SQUID status if its already running - check by PID
if [ "$pid" == "" ]; then
echo
echo
echo -e "$COL_GREEN No SQUID instance found in memory , so it seems we are good to GO !!! $COL_RESET"
else
echo
echo -e "$COL_RED SQUID is already running, probably you have some previous copy of SQUID installation, Better to stop and remove all previous squid installation !! $COL_RESET"
echo
echo -e "$COL_RED KILLING PREVIOUS SQUID INSTANCE by killall -9 squid command  !! $COL_RESET"
killall -9 squid
sleep 3
fi

# Downloading Squid source package [zaib]
echo
echo
echo -e "$COL_GREEN Downloading SQUID source package in /temp folder. . . !! $COL_RESET"
sleep 3

# Checking if /temp folder is previously present or not . . .
{
if [ ! -d "/temp" ]; then
echo
echo
echo -e "$COL_RED /temp folder not found, Creating it so all downlaods will be placed here  . . . $COL_RESET"
mkdir /temp
else
echo
echo -e "$COL_GREEN /temp folder is already present , so no need to create it, Proceeding further . . . $COL_RESET"
fi
}

cd /temp

# Checking IF SQUID 2.7 Stable9 installation file have been ALREADY downloaded in /temp to avoid duplication! [[ JZ .. . .]]
{
if [ -f /temp/squid-2.7.STABLE9+patch.tar.gz ]; then
rm -fr /temp/squid-2.7.STABLE9+patch.tar.gz
fi
}

wget https://mikrotik-squid.googlecode.com/files/squid-2.7.STABLE9%2Bpatch.tar.gz

# Checking IF SQUID 2.7 Stable9 installation file have been downloaded properly. if YEs continue further , otherwise EXIT the script with ERROR ! [[ JZ .. . .]]
{
if [ ! -f /temp/squid-2.7.STABLE9+patch.tar.gz ]; then
echo
echo

echo -e "$COL_RED ERROR: SQUID source code package File could not be download or not found in /temp/ !! $COL_RESET"
exit 0
fi
}
echo
echo

echo -e "$COL_GREEN Extracting Squid from tar archive. . . !! $COL_RESET"
sleep 3
tar zxvf squid-2.7.STABLE9+patch.tar.gz
cd squid-2.7.STABLE9
mkdir /etc/squid

echo -e "$COL_GREEN Executing $squidlabel Compiler which you have selected architecture type in start menu . . . !! $COL_RESET"
$squidver
echo
echo
echo -e "$COL_GREEN Executing MAKE and MAKE INSTALL commands . . . !! $COL_RESET"
sleep 3
make
make install
echo
echo

echo -e "$COL_GREEN Creating SQUID LOGS folder and assiging permissions . . . !! $COL_RESET"
sleep 3

# Checking if log folder is previously present or not . . .
{
if [ -d "/var/log/squid" ]; then
echo
echo
echo -e "$COL_GREEN LOGS folder found. No ned to create, proceeding ruther . . . $COL_RESET"
else
echo
echo
echo -e "$COL_GREEN Creating LOG Folder in /var/log/squid and setting permissions accordingly (to user proxy) $COL_RESET"
mkdir /var/log/squid
fi
}

chown proxy:proxy /var/log/squid

## ** DOWNLOAD SQUID.CONF
echo
echo
echo -e "$COL_GREEN Downloading SQUID.CONF file from $URL and copy it to $SQUID_DIR. . . !! $COL_RESET"
sleep 3

# Checking IF SQUID.CONF File have been ALREADY downloaded in /temp to avoid duplication! [[ JZ .. . .]]
{
if [ -f /temp/squid.conf ]; then
rm -fr /temp/squid.conf
fi
}

cd /temp
wget $URL/squid.conf

# Checking IF SQUID.CONF file have been downloaded. if YEs continue further , otherwise EXIT the script with ERROR ! [[ JZ .. . .]]
{
if [ ! -f /temp/squid.conf ]; then
echo
echo
echo -e "$COL_RED ERROR: SQUID.CONF File could not be download or not found in /temp/ !! $COL_RESET"
exit 0
fi
}
cp -fr squid.conf $SQUID_DIR

## ** DOWNLOAD SQUID.CONF
echo
echo
echo -e "$COL_GREEN Downloading STOREURL.PL file from $URL and copy it to $SQUID_DIR. . . !! $COL_RESET"
sleep 3
cd /temp

{
if [ -f /temp/storeurl.txt ]; then
rm -fr /temp/storeurl.txt
fi
}

wget $URL/storeurl.txt

{
if [ -f /temp/storeurl.pl ]; then
rm -fr /temp/storeurl.pl
fi
}

mv storeurl.txt storeurl.pl

# Checking IF STOREURL.PL file have been downloaded. if YEs continue further , otherwise EXIT the script with ERROR ! [[ JZ .. . .]]
{
if [ ! -f /temp/storeurl.pl ]; then
echo
echo
echo -e "$COL_RED ERROR: STOREURL.PL File could not be download or not found in /temp/ !! $COL_RESET"
exit 0
fi
}
cp -fr storeurl.pl $SQUID_DIR

echo
echo
echo -e "$COL_GREEN Setting EXECUTE permission for storeurl.pl . . . !! $COL_RESET"
chmod +x $SQUID_DIR/storeurl.pl

# Creating CACHE folders
echo
echo
echo -e "$COL_GREEN Creating CACHE directory in $CACHE_DIR , in this example,I used 10GB for cache, adjust it accordingly  . . . !! $COL_RESET"
sleep 3

# Checking if /cache-1 folder exist  . . .
{
if [ ! -d "$CACHE_DIR" ]; then
echo
echo
echo -e "$COL_GREEN Creating cache folder in $CACHE_DIR , Default size is 10GB, you should set it accordingly to your requirements  . . . $COL_RESET"
mkdir $CACHE_DIR
chown proxy:proxy $CACHE_DIR
squid -z
chmod 777 -R $CACHE_DIR
else
echo
echo -e "$COL_RED $CACHE_DIR folder already exists , Clearing it before proceeding. . . $COL_RESET"
rm -fr $CACHE_DIR/*
chown proxy:proxy $CACHE_DIR
echo -e "$COL_GREEN $CACHE_DIR Initializing Cache Directories as per the config  . . . $COL_RESET"
echo
squid -z
chmod 777 -R $CACHE_DIR
fi
}

echo
echo
echo -e "$COL_GREEN Adding squid in /etc/rc.local for auto startup . . . !! $COL_RESET"
sed -i '/exit/d' /etc/rc.local
sed -i '/[/usr\/sbin\/squid]/d' /etc/rc.local
echo /usr/sbin/squid >> /etc/rc.local
echo exit 0 >> /etc/rc.local
echo
echo -e "$COL_GREEN Starting SQUID (and adding 10 seconds Pause for proper initialization). . . !! $COL_RESET"
squid
sleep 5

# Checking SQUID status via PID [zaib]
#if [ "$pid" == "" ]; then
#echo
#echo -e "$COL_RED ERROR: UNABLE to start SQUID, try to run with -d1N syntax and see where its showing error !! $COL_RESET"
#else
ps aux |grep squid
echo
echo -e "$COL_GREEN $squidlabel is Running OK with PID number "$pid", no further action required, EXITING  . . .$COL_RESET"
echo
echo To view squid web access activity log, use command
echo -e "$COL_GREEN tail -f /var/log/squid/access.log $COL_RESET"
echo OR
echo -e "$COL_GREEN tail -f /var/log/squid/access.log |ccze $COL_RESET"
echo
echo -e "$COL_GREEN Regard's / Syed Jahanzaib . . . !! $COL_RESET"
echo

☺♥

.

.

.

.

Regard’s
Syed Jahanzaib