VIDEOCACHE and SQUID Configuration [Short Notes]

August 13, 2013

VIDEOCACHE and SQUID Configuration [Short Notes]

Filed under: Linux Related — Tags: dailymotion video cache, squid3, videocache, youtube cache — Syed Jahanzaib / Pinochio~:) @ 9:13 AM

WAN utilization @ 60 mbps / zaib

First of all , intro to what VIDEOCACHE is ?

” Videocache is a Squid URL plugin to cache videos from famous video sharing portals/websites like Youtube, Metacafe etc. It helps you save bandwidth when a particular video is requested more than once from the same network/machine.

Squid can not cache the videos served dynamically. Videocache fits into squid to help it cache the videos as well. The cached videos are stored on your proxy server’s local storage or on a storage server in your network. These cached videos can be served to your clients at a very fast speed saving your upstream bandwidth. “

Following are some working configuration samples for VIDEOCACHE with SQUID on UBUNTU. Installation of VC is fairly simple as mentioned here step by step

http://cachevideos.com/documentation/

I am not showing howto install VIDEOCACHE plugin as its very simple and briefly described at the author site, I am only posting the working configuration files. Also note that videocache is a paid plugin, which costs around 400$ per year.

If you are looking for something free, you may try the following NIGNX method.

https://aacable.wordpress.com/2012/08/13/youtube-caching-with-squid-nginx/

Also I would like to share that for VIDEOCACHE plugin, you *MUST* have some good hardware with *LOTS OF* memory and STORAGE. Just for an idea, I deployed it at a network of 300 Users which had Dual CPU with multi cores server for proxy , 128GB RAM, and 24 TB of external HP Storage, CPU was not much utilized, but at full peak time, the memory usage was around 95%, and the storage usage was around 180 GB in just 1 day.

Also you should have some good amount of bandwidth, otherwise videocache will become bottleneck for your network 🙂 , OR schedule VIDEOCACHE to cache only in specific timings, its supported in /etc/videocache.conf

Following is an working Squid 2.7 on Ubuntu Server 10.4 with videocache support. Please modify necessary directives before using , for example cache_mem, hostname, ACL, Ip’s. Also make sure the CACHE_DIR have been changed to match your local directory structure with proper permissions assgined to user proxy or squid , depend on he nix flavor you are using

# SQUID 2.7 CONFIG FILE WITH VIDEOCACHE PLUGIN FOR SQUID/VIDEOS [ Zaib]

# SQUID 2.7 WORKING CONFIG FILE [August 2013] with Videocache Support
# By - Syed Jahanzaib
# Email: aacable@hotmail.com
# Web  : https://aacable.wordpress.com

cache_store_log /var/log/squid/store.log
acl this_machine src 127.0.0.1 192.168.100.1
http_access allow this_machine

# --BEGIN-- videocache config for squid
url_rewrite_program /usr/bin/python /usr/share/videocache/videocache.py
url_rewrite_children 20

acl vc_deny_url url_regex -i \.blip\.tv\/(.*)filename \.hardsextube\.com\/videothumbs \.xtube\.com\/(.*)(Thumb|videowall)

acl vc_url url_regex -i \/youku\/[0-9A-Z]+\/[0-9A-Z\-]+\.(flv|mp4|avi|mkv|mp3|rm|rmvb|m4v|mov|wmv|3gp|mpg|mpeg)
acl vc_url url_regex -i \/(.*)key=[a-z0-9]+(.*)\.flv
acl vc_url url_regex -i \.(youtube|youtube-nocookie)\.com\/feeds\/api\/videos\/[0-9a-zA-Z_-]{11}\/
acl vc_url url_regex -i \.(youtube|youtube-nocookie)\.com\/(videoplayback|get_video|watch|watch_popup|user_watch)\?
acl vc_url url_regex -i \.(youtube|youtube-nocookie)\.com\/(v|e|embed)\/[0-9a-zA-Z_-]{11}

acl vc_dom_r dstdom_regex -i msn\..*\.(com|net)
acl vc_dom_r dstdom_regex -i msnbc\..*\.(com|net)
acl vc_dom_r dstdom_regex -i video\..*\.fbcdn\.net
acl vc_dom_r dstdom_regex -i myspacecdn\..*\.footprint\.net

acl vc_dom dstdomain .stream.aol.com .videos.5min.com msn.com .blip.tv .vid.ec.dmcdn.net .break.com .vimeocdn.com
acl vc_dom dstdomain .cdn.turner.com .dailymotion.com .c.wrzuta.pl .v.imwx.com .mccont.com .myspacecdn.com
acl vc_dom dstdomain .hardsextube.com .public.extremetube.phncdn.com .redtubefiles.com .video.pornhub.phncdn.com
acl vc_dom dstdomain .public.keezmovies.com .public.keezmovies.phncdn.com .slutload-media.com .public.spankwire.com .xtube.com
acl vc_dom dstdomain .public.youporn.phncdn.com .xvideos.com .tube8.com .public.spankwire.phncdn.com .pornhub.com

refresh_pattern \.youtube\.com\/videoplayback\? 120    80%    180 ignore-no-cache override-expire override-lastmod ignore-private
refresh_pattern stream\.aol\.com\/(.*)/[a-zA-Z0-9]+\/(.*)\.(flv|mp4) 120    80%    180 ignore-no-cache override-expire override-lastmod ignore-private
refresh_pattern videos\.5min\.com\/(.*)/[0-9_]+\.(mp4|flv) 120    80%    180 ignore-no-cache override-expire override-lastmod ignore-private
refresh_pattern proxy[a-z0-9\-]?[a-z0-9]?[a-z0-9]?[a-z0-9]?\.dailymotion\.com\/(.*)\.(flv|on2|mp4|avi|mkv|mp3|rm|rmvb|m4v|mov|wmv|3gp|mpg|mpeg) 120    80%    180 ignore-no-cache override-expire override-lastmod ignore-private
refresh_pattern vid\.akm\.dailymotion\.com\/(.*)\.(flv|on2|mp4|avi|mkv|mp3|rm|rmvb|m4v|mov|wmv|3gp|mpg|mpeg) 120    80%    180 ignore-no-cache override-expire override-lastmod ignore-private
refresh_pattern vid\.ec\.dmcdn\.net\/(.*)\.(flv|on2|mp4|avi|mkv|mp3|rm|rmvb|m4v|mov|wmv|3gp|mpg|mpeg) 120    80%    180 ignore-no-cache override-expire override-lastmod ignore-private
refresh_pattern video\.(.*)\.fbcdn\.net\/(.*)/[0-9_]+\.(mp4|flv|avi|mkv|m4v|mov|wmv|3gp|mpg|mpeg) 120    80%    180 ignore-no-cache override-expire override-lastmod ignore-private
refresh_pattern (.*)\.myspacecdn\.com\/(.*)\/[a-zA-Z0-9]+\/vid\.(flv|mp4|avi|mkv|mp3|rm|rmvb|m4v|mov|wmv|3gp|mpg|mpeg) 120    80%    180 ignore-no-cache override-expire override-lastmod ignore-private
refresh_pattern (.*)\.myspacecdn\.(.*)\.footprint\.net\/(.*)\/[a-zA-Z0-9]+\/vid\.(flv|mp4|avi|mkv|mp3|rm|rmvb|m4v|mov|wmv|3gp|mpg|mpeg) 120    80%    180 ignore-no-cache override-expire override-lastmod ignore-private
refresh_pattern c\.wrzuta\.pl\/wv[0-9]+\/[a-z0-9]+/[0-9]+/ 120    80%    180 ignore-no-cache override-expire override-lastmod ignore-private
refresh_pattern c\.wrzuta\.pl\/wa[0-9]+\/[a-z0-9]+ 120    80%    180 ignore-no-cache override-expire override-lastmod ignore-private
refresh_pattern vs[a-z0-9]?[a-z0-9]?[a-z0-9]?\.hardsextube\.com\/(.*)\/(.*)\.(flv|mp4|avi|mkv|mp3|rm|rmvb|m4v|mov|wmv|3gp|mpg|mpeg) 120    80%    180 ignore-no-cache override-expire override-lastmod ignore-private

acl vc_deny_url url_regex -i crossdomain.xml
acl vc_method method GET
acl vc_header req_header X-Requested-With -i videocache
url_rewrite_access deny !vc_method
url_rewrite_access deny vc_header
url_rewrite_access deny vc_deny_url
url_rewrite_access allow vc_dom
url_rewrite_access allow vc_url
url_rewrite_access allow vc_dom_r
redirector_bypass on

#These lines are needed.
acl this_machine src 127.0.0.1
http_access allow this_machine
strip_query_terms off
maximum_object_size 1000 MB

# --END-- videocache config for squid

# PORT and Transparent Option
http_port 8080 transparent
store_dir_select_algorithm round-robin

# Cache folders, change them according to your need, its an example of 500 GB
cache_dir aufs /cache1 512000 16 256

memory_replacement_policy heap LFUDA
cache_replacement_policy heap GDSF

# If you want to enable DATE time n SQUID Logs,use following
emulate_httpd_log on
logformat squid %tl %6tr %>a %Ss/%03Hs %<st %rm %ru %un %Sh/%<A %mt
log_fqdn off

# How much days to keep users access web logs
# You need to rotate your log files with a cron job. For example:
# 0 0 * * * /usr/local/squid/bin/squid -k rotate
logfile_rotate 14
debug_options ALL,1
cache_access_log /var/log/squid/access.log
cache_log /var/log/squid/cache_log.log
cache_store_log /var/log/squid/store.log

# Block Ads [zaib]
#acl adsites dstdomain url_regex "/etc/squid/adslist.txt"
#http_access deny adsites
#deny_info http://192.168.6.1/psb.htm adsites

#I used DNSAMSQ service for fast dns resolving
#so install by using "apt-get install dnsmasq" first
dns_nameservers 127.0.0.1 8.8.8.8
ftp_user anonymous@
ftp_list_width 32
ftp_passive on
ftp_sanitycheck on

#ACL Section mylan myacl
acl all src 0.0.0.0/0.0.0.0
acl manager proto cache_object
acl localhost src 127.0.0.1/255.255.255.255
acl to_localhost dst 127.0.0.0/8
acl SSL_ports port 443 563 # https, snews
#acl SSL_ports port 873 # rsync
acl Safe_ports port 80 # http
acl Safe_ports port 21 # ftp
acl Safe_ports port 443 563 # https, snews
acl Safe_ports port 70 # gopher
acl Safe_ports port 210 # wais
acl Safe_ports port 1025-65535 # unregistered ports
acl Safe_ports port 280 # http-mgmt
acl Safe_ports port 488 # gss-http
acl Safe_ports port 591 # filemaker
acl Safe_ports port 777 # multiling http
acl Safe_ports port 631 # cups
acl Safe_ports port 873 # rsync
acl Safe_ports port 901 # SWAT
acl purge method PURGE
acl CONNECT method CONNECT
http_access allow manager all
http_access deny manager
http_access allow purge localhost
http_access deny purge
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
http_access allow localhost
http_access allow all
#http_access allow all2
http_reply_access allow all
#http_reply_access allow all2
icp_access allow all

#==========================
# Administrative Parameters
#==========================

#============================================================$
# SNMP , if you want to generate graphs for SQUID via MRTG / zaib
#============================================================$
acl snmppublic snmp_community zaib
snmp_port 3401
snmp_access allow snmppublic all
snmp_access allow all

# I used UBUNTU so user is proxy, in FEDORA you may use use squid
cache_effective_user proxy
cache_effective_group proxy
cache_mgr SYED_JAHAN_ZAIB
visible_hostname aacable.wordpress.com
unique_hostname aacable@hotmail.com

# Memory , If you have 4 GB of RAM in the server, then use 1 GB
cache_mem 1 GB
minimum_object_size 0 bytes
maximum_object_size_in_memory 1024 KB

# ZPH , to mark content which is in CACHE
tcp_outgoing_tos 0x30 all
zph_mode tos
zph_local 0x30
zph_parent 0
zph_option 136
acl store_rewrite_list urlpath_regex \/(get_video|videoplayback\?id|videoplayback.*id)
acl store_rewrite_list urlpath_regex \.(jp(e?g|e|2)|gif|png|tiff?|bmp|ico|flv|wmv|3gp|mp(4|3)|exe|msi|zip|on2|mar)\?
acl store_rewrite_list_domain url_regex ^http:\/\/([a-zA-Z-]+[0-9-]+)\.[A-Za-z]*\.[A-Za-z]*
acl store_rewrite_list_domain url_regex (([a-z]{1,2}[0-9]{1,3})|([0-9]{1,3}[a-z]{1,2}))\.[a-z]*[0-9]?\.[a-z]{3}
acl store_rewrite_list_path urlpath_regex \.(jp(e?g|e|2)|gif|png|tiff?|bmp|ico|flv|avc|zip|mp3|3gp|rar|on2|mar|exe)$
acl store_rewrite_list_domain_CDN url_regex \.rapidshare\.com.*\/[0-9]*\/.*\/[^\/]* ^http:\/\/(www\.ziddu\.com.*\.[^\/]{3,4})\/(.*) \.doubleclick\.net.*
acl store_rewrite_list_domain_CDN url_regex ^http:\/\/[.a-z0-9]*\.photobucket\.com.*\.[a-z]{3}$ quantserve\.com
acl store_rewrite_list_domain_CDN url_regex ^http:\/\/[a-z]+[0-9]\.google\.co(m|\.id)
acl store_rewrite_list_domain_CDN url_regex ^http:\/\/\.www[0-9][0-9]\.indowebster\.com\/(.*)(rar|zip|flv|wm(a|v)|3gp|mp(4|3)|exe|msi|avi|(mp(e?g|a|e|1|2|3|4))|cab|exe)
acl dontrewrite url_regex redbot\.org \.php
acl getmethod method GET
##
refresh_pattern -i \.htm 120 50% 10080 reload-into-ims
refresh_pattern -i \.html 120 50% 10080 reload-into-ims
refresh_pattern ^http://*.facebook.com/* 720 100% 4320
refresh_pattern ^http://mail.yahoo.com/.* 720 100% 4320
refresh_pattern ^http://*.yahoo.*/.* 720 100% 4320
refresh_pattern ^http://*.yimg.*/.* 720 100% 4320
refresh_pattern ^http://*.gmail.*/.* 720 100% 4320
refresh_pattern ^http://*.google.*/.* 720 100% 4320
refresh_pattern ^http://*.kaskus.*/.* 720 100% 4320
refresh_pattern ^http://*.googlesyndication.*/.* 720 100% 4320
refresh_pattern ^http://*.plasa.*/.* 720 100% 4320
refresh_pattern ^http://*.telkom.*/.* 720 100% 4320
##

# 1 year = 525600 mins, 1 month = 43800 mins
refresh_pattern imeem.*\.flv  0 0% 0     override-lastmod override-expire
refresh_pattern \.rapidshare.*\/[0-9]*\/.*\/[^\/]*   161280    90%    161280 ignore-reload

refresh_pattern (get_video\?|videoplayback\?|videodownload\?|\.flv?)    10800 80% 10800 ignore-no-cache  ignore-private override-expire override-lastmod reload-into-ims
refresh_pattern (get_video\?|videoplayback\?id|videoplayback.*id|videodownload\?|\.flv?)    10800 80% 10800 ignore-no-cache  ignore-private override-expire override-lastmod reload-into-ims
#refresh_pattern -i (get_video\?|videoplayback\?id|videoplayback.*id||videodownload\?|\.flv?)       10800 80% 10800 ignore-no-cache  ignore-private override-expire override-lastmod reload-into-ims
refresh_pattern \.(ico|video-stats) 10800 80% 10800    override-expire ignore-reload ignore-no-cache  ignore-private ignore-auth override-lastmod
refresh_pattern \.etology\?                       10800 80% 10800    override-expire ignore-reload ignore-no-cache
refresh_pattern galleries\.video(\?|sz)               10800 80% 10800    override-expire ignore-reload ignore-no-cache
refresh_pattern brazzers\?                       10800 80% 10800    override-expire ignore-reload ignore-no-cache
refresh_pattern \.adtology\?                      10800 80% 10800    override-expire ignore-reload ignore-no-cache
#refresh_pattern ^.*(utm\.gif|ads\?|rmxads\.com|ad\.z5x\.net|bh\.contextweb\.com|bstats\.adbrite\.com|a1\.interclick\.com|ad\.trafficmp\.com|ads\.cubics\.com|ad\.xtendmedia\.com|\.googlesyndication\.com|advertising\.com|yieldmanager|game-advertising\.com|pixel\.quantserve\.com|adperium\.com|doubleclick\.net|adserving\.cpxinteractive\.com|syndication\.com|media.fastclick.net).* 10800 20% 10800 ignore-no-cache  ignore-private override-expire ignore-reload ignore-auth   negative-ttl=40320 max-stale=10
refresh_pattern ^.*safebrowsing.*google  10800 80% 10800 override-expire ignore-reload ignore-no-cache ignore-private ignore-auth
refresh_pattern ^http://((cbk|mt|khm|mlt)[0-9]?)\.google\.co(m|\.uk) 10800 80% 10800 override-expire ignore-reload   ignore-private
refresh_pattern ytimg\.com.*\.jpg                   10800 80% 10800    override-expire ignore-reload
refresh_pattern images\.friendster\.com.*\.(png|gif)           10800 80% 10800    override-expire ignore-reload
refresh_pattern garena\.com                                   10800 80% 10800     override-expire reload-into-ims
refresh_pattern photobucket.*\.(jp(e?g|e|2)|tiff?|bmp|gif|png)  10800 80% 10800     override-expire ignore-reload
refresh_pattern vid\.akm\.dailymotion\.com.*\.on2\?           10800 80% 10800 ignore-no-cache override-expire override-lastmod
refresh_pattern mediafire.com\/images.*\.(jp(e?g|e|2)|tiff?|bmp|gif|png)    10800 80% 10800 reload-into-ims override-expire ignore-private
refresh_pattern ^http:\/\/images|pics|thumbs[0-9]\.      10800 80% 10800 reload-into-ims ignore-no-cache  ignore-reload override-expire
refresh_pattern ^http:\/\/www.onemanga.com.*\/           10800 80% 10800 reload-into-ims ignore-no-cache  ignore-reload override-expire

# ANTI VIRUS
refresh_pattern guru.avg.com/.*\.(bin)                      10800 80% 10800 ignore-no-cache  ignore-reload  reload-into-ims
refresh_pattern (avgate|avira).*(idx|gz)$                           10800 80% 10800 ignore-no-cache  ignore-reload  reload-into-ims
refresh_pattern kaspersky.*\.avc$                                   10800 80% 10800 ignore-no-cache  ignore-reload  reload-into-ims
refresh_pattern kaspersky                                           10800 80% 10800 ignore-no-cache  ignore-reload  reload-into-ims
refresh_pattern update.nai.com/.*\.(gem|zip|mcs)                    10800 80% 10800 ignore-no-cache  ignore-reload  reload-into-ims
refresh_pattern ^http:\/\/liveupdate.symantecliveupdate.com.*\(zip)     10800 80% 10800 ignore-no-cache  ignore-reload  reload-into-ims

refresh_pattern windowsupdate.com/.*\.(cab|exe)             10800  80%  10800 ignore-no-cache  ignore-reload  reload-into-ims
refresh_pattern update.microsoft.com/.*\.(cab|exe)             10800  80%  10800 ignore-no-cache  ignore-reload  reload-into-ims
refresh_pattern download.microsoft.com/.*\.(cab|exe)             10800  80%  10800 ignore-no-cache  ignore-reload  reload-into-ims

#images facebook
refresh_pattern ((facebook.com)|(85.131.151.39)).*\.(jpg|png|gif)      10800 80% 10800 ignore-reload  override-expire ignore-no-cache
refresh_pattern -i \.fbcdn.net.*\.(jpg|gif|png|swf|mp3)                  10800 80% 10800 ignore-reload  override-expire ignore-no-cache
refresh_pattern  static\.ak\.fbcdn\.net*\.(jpg|gif|png)                  10800 80% 10800 ignore-reload  override-expire ignore-no-cache
refresh_pattern ^http:\/\/profile\.ak\.fbcdn.net*\.(jpg|gif|png)      10800 80% 10800 ignore-reload  override-expire ignore-no-cache

#banner IIX
refresh_pattern ^http:\/\/openx.*\.(jp(e?g|e|2)|gif|pn[pg]|swf|ico|css|tiff?) 10800 99999% 10800 reload-into-ims  ignore-reload override-expire ignore-no-cache
refresh_pattern ^http:\/\/ads(1|2|3).kompas.com.*\/           10800 99999% 10800 reload-into-ims  ignore-reload override-expire ignore-no-cache
refresh_pattern ^http:\/\/img.ads.kompas.com.*\/           10800 99999% 10800 reload-into-ims  ignore-reload override-expire ignore-no-cache
refresh_pattern .kompasimages.com.*\.(jpg|gif|png|swf)       10800 99999% 10800 reload-into-ims  ignore-reload override-expire ignore-no-cache
refresh_pattern ^http:\/\/openx.kompas.com.*\/           10800 99999% 10800 reload-into-ims  ignore-reload override-expire ignore-no-cache
refresh_pattern kaskus.\us.*\.(jp(e?g|e|2)|gif|png|swf)        10800 99999% 10800 reload-into-ims  ignore-reload override-expire ignore-no-cache
refresh_pattern ^http:\/\/img.kaskus.us.*\.(jpg|gif|png|swf)       10800 99999% 10800 reload-into-ims  ignore-reload override-expire ignore-no-cache

#IIX DOWNLOAD
refresh_pattern ^http:\/\/\.www[0-9][0-9]\.indowebster\.com\/(.*)(mp3|rar|zip|flv|wmv|3gp|mp(4|3)|exe|msi|zip) 10800 99999% 10800 reload-into-ims  ignore-reload override-expire ignore-no-cache    ignore-auth

#All File
refresh_pattern -i \.(3gp|7z|ace|asx|avi|bin|cab|dat|deb|divx|dvr-ms)      10800 80% 10800 ignore-no-cache  ignore-private override-expire override-lastmod reload-into-ims
refresh_pattern -i \.(rar|jar|gz|tgz|bz2|iso|m1v|m2(v|p)|mo(d|v))          10800 80% 10800 ignore-no-cache  ignore-private override-expire override-lastmod reload-into-ims
refresh_pattern -i \.(jp(e?g|e|2)|gif|pn[pg]|bm?|tiff?|ico|swf|css|js)     10800 80% 10800 ignore-no-cache  ignore-private override-expire override-lastmod reload-into-ims
refresh_pattern -i \.(mp(e?g|a|e|1|2|3|4)|mk(a|v)|ms(i|u|p)|og(x|v|a|g)|rar|rm|r(a|p)m|snd|vob|wav) 10800 80% 10800 ignore-no-cache ignore-private override-expire override-lastmod reload-into-ims
refresh_pattern -i \.(pp(s|t)|wax|wm(a|v)|wmx|wpl|zip|cb(r|z|t))     10800 80% 10800 ignore-no-cache ignore-private override-expire override-lastmod reload-into-ims

#########################################################################

refresh_pattern (cgi-bin|\?)       0      0%      0
refresh_pattern ^gopher:    1440    0%    1440
refresh_pattern ^ftp:         10080     95%     10800 override-lastmod reload-into-ims
refresh_pattern         .     180     95% 10800 override-lastmod reload-into-ims

global_internal_static off
retry_on_error on
buffered_logs on
read_ahead_gap 32 KB

#header_access Accept-Encoding deny  all
client_persistent_connections off
server_persistent_connections on
half_closed_clients off
strip_query_terms off
quick_abort_min 0 KB
quick_abort_max 0 KB
quick_abort_pct 100
vary_ignore_expire on
reload_into_ims on
pipeline_prefetch on
read_timeout 30 minute
client_lifetime 6 hour
positive_dns_ttl 6 hour
pconn_timeout 15 second
request_timeout 1 minute
log_icp_queries off
ipcache_size 16384
ipcache_low 98
ipcache_high 99
log_fqdn off
fqdncache_size 16384
memory_pools off
forwarded_for on
client_db off
max_filedescriptors 8192

☻

☺

if you are using SQUID 3.1.19 (Ubuntu 12.4 default isntallation) then use following, pretty much same hmmm 🙂

SQUID 3.1.19 CONFIG FILE in /etc/squid3/squid.conf

# SQUID 3.1.19 CONFIG FILE WITH VIDEOCACHE PLUGIN FOR SQUID/VIDEOS [ Zaib]
# SQUID 3.1.19 WORKING CONFIG FILE [August 2013] with Videocache Support
# By - Syed Jahanzaib
# Email: aacable@hotmail.com
# Web  : https://aacable.wordpress.com

cache_store_log /var/log/squid3/store.log
acl this_machine src 127.0.0.1 192.168.100.1 # Change this ip to match your's
http_access allow this_machine

# --BEGIN-- videocache config for squid
#
# PASTE THIS FILE AT THE TOP OF YOUR SQUID CONFIGURATION FILE
#
# (C) Copyright White Magnet Software Private Limited
# Company Website : http://whitemagnet.com/
# Product Website : http://cachevideos.com/
#
# Videocache Version : Trial
#
# This configuration is for Squid versions 3.x ZABBO
url_rewrite_program /usr/bin/python /usr/share/videocache/videocache.py
url_rewrite_children 20

acl vc_deny_url url_regex -i \.blip\.tv\/(.*)filename \.hardsextube\.com\/videothumbs \.xtube\.com\/(.*)(Thumb|videowall)

acl vc_url url_regex -i \/youku\/[0-9A-Z]+\/[0-9A-Z\-]+\.(flv|mp4|avi|mkv|mp3|rm|rmvb|m4v|mov|wmv|3gp|mpg|mpeg)
acl vc_url url_regex -i \/(.*)key=[a-z0-9]+(.*)\.flv
acl vc_url url_regex -i \.(youtube|youtube-nocookie)\.com\/feeds\/api\/videos\/[0-9a-zA-Z_-]{11}\/
acl vc_url url_regex -i \.(youtube|youtube-nocookie)\.com\/(videoplayback|get_video|watch|watch_popup|user_watch)\?
acl vc_url url_regex -i \.(youtube|youtube-nocookie)\.com\/(v|e|embed)\/[0-9a-zA-Z_-]{11}

acl vc_dom_r dstdom_regex -i msn\..*\.(com|net)
acl vc_dom_r dstdom_regex -i msnbc\..*\.(com|net)
acl vc_dom_r dstdom_regex -i video\..*\.fbcdn\.net
acl vc_dom_r dstdom_regex -i myspacecdn\..*\.footprint\.net

acl vc_dom dstdomain .stream.aol.com .videos.5min.com msn.com .blip.tv .vid.ec.dmcdn.net .break.com .vimeocdn.com
acl vc_dom dstdomain .cdn.turner.com .dailymotion.com .c.wrzuta.pl .v.imwx.com .mccont.com .myspacecdn.com
acl vc_dom dstdomain .hardsextube.com .public.extremetube.phncdn.com .redtubefiles.com .video.pornhub.phncdn.com
acl vc_dom dstdomain .public.keezmovies.com .public.keezmovies.phncdn.com .slutload-media.com .public.spankwire.com .xtube.com
acl vc_dom dstdomain .public.youporn.phncdn.com .xvideos.com .tube8.com .public.spankwire.phncdn.com .pornhub.com

refresh_pattern \.youtube\.com\/videoplayback\? 120    80%    180 ignore-no-cache ignore-no-store override-expire override-lastmod ignore-private
refresh_pattern stream\.aol\.com\/(.*)/[a-zA-Z0-9]+\/(.*)\.(flv|mp4) 120    80%    180 ignore-no-cache ignore-no-store override-expire override-lastmod ignore-private
refresh_pattern videos\.5min\.com\/(.*)/[0-9_]+\.(mp4|flv) 120    80%    180 ignore-no-cache ignore-no-store override-expire override-lastmod ignore-private
refresh_pattern proxy[a-z0-9\-]?[a-z0-9]?[a-z0-9]?[a-z0-9]?\.dailymotion\.com\/(.*)\.(flv|on2|mp4|avi|mkv|mp3|rm|rmvb|m4v|mov|wmv|3gp|mpg|mpeg) 120    80%    180 ignore-no-cache ignore-no-store override-expire override-lastmod ignore-private
refresh_pattern vid\.akm\.dailymotion\.com\/(.*)\.(flv|on2|mp4|avi|mkv|mp3|rm|rmvb|m4v|mov|wmv|3gp|mpg|mpeg) 120    80%    180 ignore-no-cache ignore-no-store override-expire override-lastmod ignore-private
refresh_pattern vid\.ec\.dmcdn\.net\/(.*)\.(flv|on2|mp4|avi|mkv|mp3|rm|rmvb|m4v|mov|wmv|3gp|mpg|mpeg) 120    80%    180 ignore-no-cache ignore-no-store override-expire override-lastmod ignore-private
refresh_pattern video\.(.*)\.fbcdn\.net\/(.*)/[0-9_]+\.(mp4|flv|avi|mkv|m4v|mov|wmv|3gp|mpg|mpeg) 120    80%    180 ignore-no-cache ignore-no-store override-expire override-lastmod ignore-private
refresh_pattern (.*)\.myspacecdn\.com\/(.*)\/[a-zA-Z0-9]+\/vid\.(flv|mp4|avi|mkv|mp3|rm|rmvb|m4v|mov|wmv|3gp|mpg|mpeg) 120    80%    180 ignore-no-cache ignore-no-store override-expire override-lastmod ignore-private
refresh_pattern (.*)\.myspacecdn\.(.*)\.footprint\.net\/(.*)\/[a-zA-Z0-9]+\/vid\.(flv|mp4|avi|mkv|mp3|rm|rmvb|m4v|mov|wmv|3gp|mpg|mpeg) 120    80%    180 ignore-no-cache ignore-no-store override-expire override-lastmod ignore-private
refresh_pattern c\.wrzuta\.pl\/wv[0-9]+\/[a-z0-9]+/[0-9]+/ 120    80%    180 ignore-no-cache ignore-no-store override-expire override-lastmod ignore-private
refresh_pattern c\.wrzuta\.pl\/wa[0-9]+\/[a-z0-9]+ 120    80%    180 ignore-no-cache ignore-no-store override-expire override-lastmod ignore-private
refresh_pattern vs[a-z0-9]?[a-z0-9]?[a-z0-9]?\.hardsextube\.com\/(.*)\/(.*)\.(flv|mp4|avi|mkv|mp3|rm|rmvb|m4v|mov|wmv|3gp|mpg|mpeg) 120    80%    180 ignore-no-cache ignore-no-store override-expire override-lastmod ignore-private

acl vc_deny_url url_regex -i crossdomain.xml
acl vc_method method GET
acl vc_header req_header X-Requested-With -i videocache
url_rewrite_access deny !vc_method
url_rewrite_access deny vc_header
url_rewrite_access deny vc_deny_url
url_rewrite_access allow vc_dom
url_rewrite_access allow vc_url
url_rewrite_access allow vc_dom_r
url_rewrite_bypass on
#These lines are needed.
acl this_machine src 127.0.0.1
http_access allow this_machine
strip_query_terms off
maximum_object_size 10 GB
# --END-- videocache config for squid

# SQUID 3.x CONFIG FILE

# PORT and Transparent Option
#http_port 8080
http_port 8080 transparent
icp_port 0

store_dir_select_algorithm round-robin

# Cache Direcotries on external Storage, for sure you should change it
# Each folder is of 1.5 TB in space
cache_dir aufs /cache-1/squid-cache-1 1500000 16 256
cache_dir aufs /cache-2/squid-cache-2 15000000 16 256
cache_dir aufs /cache-3/squid-cache-3 15000000 16 256
cache_dir aufs /cache-4/squid-cache-4 15000000 16 256
cache_dir aufs /cache-5/squid-cache-5 15000000 16 256
cache_dir aufs /cache-6/squid-cache-6 15000000 16 256
cache_dir aufs /cache-7/squid-cache-7 15000000 16 256
cache_dir aufs /cache-8/squid-cache-8 15000000 16 256
cache_dir aufs /cache-9/squid-cache-9 15000000 16 256
cache_dir aufs /cache-10/squid-cache-10 15000000 16 256
cache_dir aufs /cache-11/squid-cache-11 15000000 16 256
cache_dir aufs /cache-12/squid-cache-12 1500000 16 256

#cache_dir ufs /mnt/hdd2/cache2 200000 16 256 # If you have secondary HDD
memory_replacement_policy heap LFUDA
cache_replacement_policy heap GDSF

# If you want to enable DATE time n SQUID Logs,use following
emulate_httpd_log on
logformat squid %tl %6tr %>a %Ss/%03Hs %<st %rm %ru %un %Sh/%<A %mt
log_fqdn off

# How much days to keep users access web logs
# You need to rotate your log files with a cron job. For example:
# 0 0 * * * /usr/local/squid/bin/squid -k rotate
logfile_rotate 14
debug_options ALL,1
cache_access_log /var/log/squid3/access.log
cache_log  /var/log/squid3/cache_log.log
cache_store_log /var/log/squid3/store.log

# Block Ads [zaib]
#acl adsites dstdomain url_regex "/etc/squid/adslist.txt"
#http_access deny adsites
#deny_info http://192.168.6.1/psb.htm adsites

#I used DNSAMSQ service for fast dns resolving
#so install by using "apt-get install dnsmasq" first
dns_nameservers 127.0.0.1 8.8.8.8
ftp_user anonymous@
ftp_list_width 32
ftp_passive on
ftp_sanitycheck on

#ACL Section mylan myacl
acl all src 0.0.0.0/0.0.0.0 # ALLOW ALL, you may want to change this /zaib
acl manager proto cache_object
acl localhost src 127.0.0.1/255.255.255.255
acl to_localhost dst 127.0.0.0/8
acl SSL_ports port 443 563 # https, snews
acl SSL_ports port 873 # rsync
acl Safe_ports port 80 # http
acl Safe_ports port 21 # ftp
acl Safe_ports port 443 563 # https, snews
acl Safe_ports port 70 # gopher
acl Safe_ports port 210 # wais
acl Safe_ports port 1025-65535 # unregistered ports
acl Safe_ports port 280 # http-mgmt
acl Safe_ports port 488 # gss-http
acl Safe_ports port 591 # filemaker
acl Safe_ports port 777 # multiling http
acl Safe_ports port 631 # cups
acl Safe_ports port 873 # rsync
acl Safe_ports port 901 # SWAT
acl purge method PURGE
acl CONNECT method CONNECT
http_access allow manager all
http_access deny manager
http_access allow purge localhost
http_access deny purge
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
http_access allow localhost
http_access allow all
#http_access allow all2
http_reply_access allow all
#http_reply_access allow all2
icp_access allow all

#==========================
# Administrative Parameters
#==========================

#============================================================$
# SNMP , if you want to generate graphs for SQUID via MRTG
#============================================================$
#acl snmppublic snmp_community zaib
#snmp_port 3401
#snmp_access allow snmppublic all
#snmp_access allow all

# I used UBUNTU so user is proxy, in FEDORA you may use use squid
cache_effective_user proxy
cache_effective_group proxy
cache_mgr SYED_JAHAN_ZAIB
visible_hostname aacable.wordpress.com
unique_hostname aacable@hotmail.com

# Memory, in this example, teh squid box have 32 B of RAM
cache_mem 8 GB # Change it as per your hardware config, you may want to lower it down
minimum_object_size 0 bytes
maximum_object_size_in_memory 5 MB # Change it as per your hardware config, you may want to lower it down if you have low ram

# 1 year = 525600 mins, 1 month = 43800 mins
refresh_pattern imeem.*\.flv  0 0% 0     override-lastmod override-expire
refresh_pattern \.rapidshare.*\/[0-9]*\/.*\/[^\/]*   161280    90%    161280 ignore-reload

refresh_pattern (get_video\?|videoplayback\?|videodownload\?|\.flv?)    10800 80% 10800 ignore-no-cache  ignore-private override-expire override-lastmod reload-into-ims
refresh_pattern (get_video\?|videoplayback\?id|videoplayback.*id|videodownload\?|\.flv?)    10800 80% 10800 ignore-no-cache  ignore-private override-expire override-lastmod reload-into-ims
#refresh_pattern -i (get_video\?|videoplayback\?id|videoplayback.*id||videodownload\?|\.flv?)       10800 80% 10800 ignore-no-cache  ignore-private override-expire override-lastmod reload-into-ims
refresh_pattern \.(ico|video-stats) 10800 80% 10800    override-expire ignore-reload ignore-no-cache  ignore-private ignore-auth override-lastmod
refresh_pattern \.etology\?                       10800 80% 10800    override-expire ignore-reload ignore-no-cache
refresh_pattern galleries\.video(\?|sz)               10800 80% 10800    override-expire ignore-reload ignore-no-cache
refresh_pattern brazzers\?                       10800 80% 10800    override-expire ignore-reload ignore-no-cache
refresh_pattern \.adtology\?                      10800 80% 10800    override-expire ignore-reload ignore-no-cache
#refresh_pattern ^.*(utm\.gif|ads\?|rmxads\.com|ad\.z5x\.net|bh\.contextweb\.com|bstats\.adbrite\.com|a1\.interclick\.com|ad\.trafficmp\.com|ads\.cubics\.com|ad\.xtendmedia\.com|\.googlesyndication\.com|advertising\.com|yieldmanager|game-advertising\.com|pixel\.quantserve\.com|adperium\.com|doubleclick\.net|adserving\.cpxinteractive\.com|syndication\.com|media.fastclick.net).* 10800 20% 10800 ignore-no-cache  ignore-private override-expire ignore-reload ignore-auth   negative-ttl=40320 max-stale=10
refresh_pattern ^.*safebrowsing.*google  10800 80% 10800 override-expire ignore-reload ignore-no-cache ignore-private ignore-auth
refresh_pattern ^http://((cbk|mt|khm|mlt)[0-9]?)\.google\.co(m|\.uk) 10800 80% 10800 override-expire ignore-reload   ignore-private
refresh_pattern ytimg\.com.*\.jpg                   10800 80% 10800    override-expire ignore-reload
refresh_pattern images\.friendster\.com.*\.(png|gif)           10800 80% 10800    override-expire ignore-reload
refresh_pattern garena\.com                                   10800 80% 10800     override-expire reload-into-ims
refresh_pattern photobucket.*\.(jp(e?g|e|2)|tiff?|bmp|gif|png)  10800 80% 10800     override-expire ignore-reload
refresh_pattern vid\.akm\.dailymotion\.com.*\.on2\?           10800 80% 10800 ignore-no-cache override-expire override-lastmod
refresh_pattern mediafire.com\/images.*\.(jp(e?g|e|2)|tiff?|bmp|gif|png)    10800 80% 10800 reload-into-ims override-expire ignore-private
refresh_pattern ^http:\/\/images|pics|thumbs[0-9]\.      10800 80% 10800 reload-into-ims ignore-no-cache  ignore-reload override-expire
refresh_pattern ^http:\/\/www.onemanga.com.*\/           10800 80% 10800 reload-into-ims ignore-no-cache  ignore-reload override-expire

# ANTI VIRUS
refresh_pattern guru.avg.com/.*\.(bin)                      10800 80% 10800 ignore-no-cache  ignore-reload  reload-into-ims
refresh_pattern (avgate|avira).*(idx|gz)$                           10800 80% 10800 ignore-no-cache  ignore-reload  reload-into-ims
refresh_pattern kaspersky.*\.avc$                                   10800 80% 10800 ignore-no-cache  ignore-reload  reload-into-ims
refresh_pattern kaspersky                                           10800 80% 10800 ignore-no-cache  ignore-reload  reload-into-ims
refresh_pattern update.nai.com/.*\.(gem|zip|mcs)                    10800 80% 10800 ignore-no-cache  ignore-reload  reload-into-ims
refresh_pattern ^http:\/\/liveupdate.symantecliveupdate.com.*\(zip)     10800 80% 10800 ignore-no-cache  ignore-reload  reload-into-ims

refresh_pattern windowsupdate.com/.*\.(cab|exe)             10800  80%  10800 ignore-no-cache  ignore-reload  reload-into-ims
refresh_pattern update.microsoft.com/.*\.(cab|exe)             10800  80%  10800 ignore-no-cache  ignore-reload  reload-into-ims
refresh_pattern download.microsoft.com/.*\.(cab|exe)             10800  80%  10800 ignore-no-cache  ignore-reload  reload-into-ims

#images facebook
refresh_pattern ((facebook.com)|(85.131.151.39)).*\.(jpg|png|gif)      10800 80% 10800 ignore-reload  override-expire ignore-no-cache
refresh_pattern -i \.fbcdn.net.*\.(jpg|gif|png|swf|mp3)                  10800 80% 10800 ignore-reload  override-expire ignore-no-cache
refresh_pattern  static\.ak\.fbcdn\.net*\.(jpg|gif|png)                  10800 80% 10800 ignore-reload  override-expire ignore-no-cache
refresh_pattern ^http:\/\/profile\.ak\.fbcdn.net*\.(jpg|gif|png)      10800 80% 10800 ignore-reload  override-expire ignore-no-cache

#banner IIX
refresh_pattern ^http:\/\/openx.*\.(jp(e?g|e|2)|gif|pn[pg]|swf|ico|css|tiff?) 10800 99999% 10800 reload-into-ims  ignore-reload override-expire ignore-no-cache
refresh_pattern ^http:\/\/ads(1|2|3).kompas.com.*\/           10800 99999% 10800 reload-into-ims  ignore-reload override-expire ignore-no-cache
refresh_pattern ^http:\/\/img.ads.kompas.com.*\/           10800 99999% 10800 reload-into-ims  ignore-reload override-expire ignore-no-cache
refresh_pattern .kompasimages.com.*\.(jpg|gif|png|swf)       10800 99999% 10800 reload-into-ims  ignore-reload override-expire ignore-no-cache
refresh_pattern ^http:\/\/openx.kompas.com.*\/           10800 99999% 10800 reload-into-ims  ignore-reload override-expire ignore-no-cache
refresh_pattern kaskus.\us.*\.(jp(e?g|e|2)|gif|png|swf)        10800 99999% 10800 reload-into-ims  ignore-reload override-expire ignore-no-cache
refresh_pattern ^http:\/\/img.kaskus.us.*\.(jpg|gif|png|swf)       10800 99999% 10800 reload-into-ims  ignore-reload override-expire ignore-no-cache

#IIX DOWNLOAD
refresh_pattern ^http:\/\/\.www[0-9][0-9]\.indowebster\.com\/(.*)(mp3|rar|zip|flv|wmv|3gp|mp(4|3)|exe|msi|zip) 10800 99999% 10800 reload-into-ims  ignore-reload override-expire ignore-no-cache    ignore-auth

#All File
refresh_pattern -i \.(3gp|7z|ace|asx|avi|bin|cab|dat|deb|divx|dvr-ms)      10800 80% 10800 ignore-no-cache  ignore-private override-expire override-lastmod reload-into-ims
refresh_pattern -i \.(rar|jar|gz|tgz|bz2|iso|m1v|m2(v|p)|mo(d|v))          10800 80% 10800 ignore-no-cache  ignore-private override-expire override-lastmod reload-into-ims
refresh_pattern -i \.(jp(e?g|e|2)|gif|pn[pg]|bm?|tiff?|ico|swf|css|js)     10800 80% 10800 ignore-no-cache  ignore-private override-expire override-lastmod reload-into-ims
refresh_pattern -i \.(mp(e?g|a|e|1|2|3|4)|mk(a|v)|ms(i|u|p)|og(x|v|a|g)|rar|rm|r(a|p)m|snd|vob|wav) 10800 80% 10800 ignore-no-cache ignore-private override-expire override-lastmod reload-into-ims
refresh_pattern -i \.(pp(s|t)|wax|wm(a|v)|wmx|wpl|zip|cb(r|z|t))     10800 80% 10800 ignore-no-cache ignore-private override-expire override-lastmod reload-into-ims

#########################################################################

refresh_pattern (cgi-bin|\?)       0      0%      0
refresh_pattern ^gopher:    1440    0%    1440
refresh_pattern ^ftp:         10080     95%     10800 override-lastmod reload-into-ims
refresh_pattern         .     180     95% 10800 override-lastmod reload-into-ims

global_internal_static off
retry_on_error on
buffered_logs on
read_ahead_gap 32 KB

#header_access Accept-Encoding deny  all
client_persistent_connections off
server_persistent_connections on
half_closed_clients off
strip_query_terms off
quick_abort_min 0 KB
quick_abort_max 0 KB
quick_abort_pct 100
vary_ignore_expire on
reload_into_ims on
pipeline_prefetch on
read_timeout 30 minute
client_lifetime 6 hour
positive_dns_ttl 6 hour
pconn_timeout 15 second
request_timeout 1 minute
log_icp_queries off
ipcache_size 16384
ipcache_low 98
ipcache_high 99
log_fqdn off
fqdncache_size 16384
memory_pools off
forwarded_for on
client_db off
max_filedescriptors 8192

# ZAIB
# ZPH for Squid 3.3.19
qos_flows local-hit=0x30

☺

↓

After videocache is installed, following is the fine tunned config file. Don’t forget to change the host ip (192.168.100.1) to match your own server ip

VIDEO CACHE CONFIG FILE in /etc/videocache.conf

[main]

# file : /etc/videocache.conf
# (C) Copyright White Magnet Software Private Limited
# Company Website : http://whitemagnet.com/
# Product Website : http://cachevideos.com/

# WORKING SAMPLE FILE [JULY 2013] - Syed Jahanzaib
# Email: aacable@hotmail.com
# Web  : https://aacable.wordpress.com

# version : cxxxxxxb

client_email = your_registered_emai_address@hotmail.com
cache_host = 192.168.100.1
videocache_user = proxy

# This option is used to define the cache directories where Videocache will store
# cached videos. We can specify multiple caching directories separated by â€˜|â€™.
# Examples:
#   base_dir = /videocache/
#   base_dir = /cache/videocache2/ | /hdd2/videocache/ | /home/videocache/
base_dir = /cache1/vc/

disk_avail_threshold = 15000
disk_cleanup_strategy = 1
enable_videocache = 1
offline_mode = 0
base_dir_selection = 2
enable_store_log_monitoring = 1
squid_store_log = /var/log/squid3/store.log
this_proxy = 127.0.0.1:8080
hit_threshold = 1
max_cache_processes = 10
max_cache_speed = 0
max_cache_queue_size = 100000
cache_period =
proxy =
proxy_username =
proxy_password =
max_video_size = 0
min_video_size = 0
force_video_size = 1
scheduler_pidfile = /var/run/videocache.pid
logdir = /var/log/videocache/
logformat = %tl %p %s %i %w %c %v %m %d
scheduler_logformat = %tl %p %s %i %w %c %v %m %d
cleaner_logformat = %tl %p %s %w %c %v %m %d
timeformat = %d/%b/%Y:%H:%M:%S
enable_videocache_log = 1
enable_scheduler_log = 1
enable_cleaner_log = 1
enable_trace_log = 1
logfile = videocache.log
scheduler_logfile = scheduler.log
cleaner_logfile = cleaner.log
tracefile = trace.log
max_logfile_size = 10
max_scheduler_logfile_size = 10
max_cleaner_logfile_size = 10
max_tracefile_size = 10
max_logfile_backups = 10
max_scheduler_logfile_backups = 5
max_cleaner_logfile_backups = 1
max_tracefile_backups = 1

enable_youtube_cache = 1

# This options determines if Videocache will cache different YouTube video
# formats separately. Please select an appropriate algorithm from the listed below.
# Available strategies:
#   1 : (disabled) Don't check for YouTube video formats. Cache one of the formats
#       and serve it for requests for all kinds of formats.
#   2 : (strict) Strictly check for YouTube formats and cache all formats separately.
#       Consumes maximum bandwidth.
#   3 : (approximate) Check YouTube formats but with approximation. For example,
#       if a client asked for a video in 480p format and we already have 360p
#       format of the same video in cache, then serve 360p format and vice-versa.
# Default : 3
enable_youtube_format_support = 3
enable_youtube_html5_videos = 1
enable_youtube_3d_videos = 1
enable_youtube_partial_caching = 1
max_youtube_video_quality = 720p
min_youtube_views = 0
enable_aol_cache = 1
enable_bing_cache = 1
enable_bliptv_cache = 1
enable_breakcom_cache = 1
enable_cnn_cache = 1
enable_dailymotion_cache = 1
enable_facebook_cache = 1
enable_megavideo_cache = 1
enable_metacafe_cache = 1
enable_myspace_cache = 1
enable_vimeo_cache = 1
enable_weather_cache = 1
enable_wrzuta_cache = 1
enable_youku_cache = 1
enable_extremetube_cache = 1
enable_hardsextube_cache = 1
enable_keezmovies_cache = 1
enable_pornhub_cache = 1
enable_redtube_cache = 1
enable_slutload_cache = 1
enable_spankwire_cache = 1
enable_tube8_cache = 1
enable_xhamster_cache = 1
enable_xtube_cache = 1
enable_xvideos_cache = 1
enable_youporn_cache = 1

skip_apache_conf = 0
apache_conf_dir = /etc/apache2/conf.d/
hide_cache_dirs = 1
rpc_host = 127.0.0.1
rpc_port = 9100

☺
To restart Videocache and related services, use following commands

vc-update

service apache2 restart

vc-scheduler -s restart

service squid3 restart

To view Video cache Logs in live for info and troubleshooting

tail -f /var/log/videocache/videocache.log

tail -f /var/log/videocache/scheduler.log

tail -f /var/log/squid3/access.log

Sample image of logs at live server

VC-CACHE

☺

☼☼☼☼☼☼☼☼

Regard’s
Syed Jahanzaib

Comments (29)

29 Comments »

Thank you Syed, I will try it today. I belive that the big problem is to configure well mikrotik router for full use of proxy. (that’s my problem !)

LikeLike

Comment by Ivan — August 13, 2013 @ 11:16 AM

Reply
- Its not the problem, You need to configure the mikrotik and squid zph
  
  LikeLike
  
  Comment by Syed Jahanzaib / Pinochio~:) — August 14, 2013 @ 6:33 PM
  
  Reply
  - My noc is configured like your virtualization.
    cisco router (fiber company)—> (wan public ip ) RB1100AHx2 —> (lan noc ip 10.1.1.0/24 ) (radius manager ecc.) —-> backbone OSPF based- and client with any different subnet any tower
    | |–> (local lan 192.168.1.0/24)) office lan
    |–> (proxy lan 172.16.1.0/30) — > proxy server
    
    I want a parallel proxy with only one interface and I had to log any connection.
    
    Any time I make a mod on routerboard, somethings happen and nobody can use internet, how configure miktorik with this configuration?
    
    Thanks
    Ivan
    
    LikeLike
    
    Comment by Ivan — August 17, 2013 @ 7:18 PM
  - you can use squid with single nic. its common. just plug the squid in mikrotik separate interface
    on squid server, create a route pointing to Mikrotik. this way squid will gets its data from the mikrotik, also it will get all the subnet info from the mikrotik too.
    
    LikeLike
    
    Comment by Syed Jahanzaib / Pinochio~:) — August 19, 2013 @ 9:04 AM
thank you Very much 🙂 I would like to ask you if you have tutorial for Varnish thank you again bro

LikeLike

Comment by alfanet1 — August 13, 2013 @ 12:17 PM

Reply
- I have not worked with the VARNISH yet, so no idea about it.
  
  LikeLike
  
  Comment by Syed Jahanzaib / Pinochio~:) — August 14, 2013 @ 6:32 PM
  
  Reply
need to download video cache 2.3 ?

LikeLike

Comment by nawal — August 14, 2013 @ 1:03 AM

Reply
- ??
  
  LikeLike
  
  Comment by Syed Jahanzaib / Pinochio~:) — August 14, 2013 @ 6:34 PM
  
  Reply
  - i confuse at install videocache ,” Untar the VideoCache software bundle you received in your email “
    
    LikeLike
    
    Comment by nawal — August 15, 2013 @ 12:16 AM
  - videocache is not available for download from there website.
    you have to request them for the trial or licensed version. Once you receive it, copy it your squid box, and unrar it in any folder and run its install script as mentioned in the guide.
    
    LikeLike
    
    Comment by Syed Jahanzaib / Pinochio~:) — August 15, 2013 @ 8:36 AM
  - i just confuse at your post ” After videocache is installed, following is the fine tunned config file. Don’t forget to change the host ip (192.168.100.1) to match your own server ip ” so as you know we need to buy videocache version first to get the installer to install
    
    LikeLike
    
    Comment by nawal — August 16, 2013 @ 5:02 PM
  - It is pre understood that you must have this software before installation. You can also get its trial version from the author.
    
    LikeLike
    
    Comment by Syed Jahanzaib / Pinochio~:) — August 16, 2013 @ 5:03 PM
  - i got cachevideos 2.3 trial version, so if im using your squid.conf and videocache.conf it will convert to unlimited version ?
    
    LikeLike
    
    Comment by nawal — August 24, 2013 @ 2:49 PM
  - I have posted only configuration files, they do work fine.
    but licensing is something else, you have to consult your videocache guy for it.
    
    Also there is another free method to do the caching of youtube
    
    Youtube caching with SQUID 2.7 [using storeurl.pl]
    
    LikeLike
    
    Comment by Syed Jahanzaib / Pinochio~:) — August 26, 2013 @ 9:20 AM
Syed, what is the difference between Videocache or https://aacable.wordpress.com/?s=squid+nginx. Squid+nginx is working for me. The good thing like you said is the perfomance that need the server and the line, because is can become a bottleneck

LikeLike

Comment by Rafael — August 14, 2013 @ 8:14 PM

Reply
- Videocache is an advance plugin for squid, it supports caching of many video websites with good stats / logging/ cleaner and some other features. read there web site for full list of supported web sites and its features.
  
  LikeLike
  
  Comment by Syed Jahanzaib / Pinochio~:) — August 15, 2013 @ 8:35 AM
  
  Reply
its hard to understand from videocache documentatiom , i hope you make step by step tutorial to install videocache 🙂 thank you sir great website

LikeLike

Comment by nawal — August 16, 2013 @ 2:23 PM

Reply
Thank you Syed Jahanzaib Bahi

LikeLike

Comment by Waseem — August 20, 2013 @ 11:56 PM

Reply
Can you give out your full hardware configuration with its model and also 24 TB of external HP Storage model and its cost.

LikeLike

Comment by tubeguy — September 2, 2013 @ 11:29 AM

Reply
- I dont remember exactly, but it was XEON dual processors with 16 cores / 128 GB RAM / HP External Storage, I dont remember the exact model but it had 24 Hard Drives of 1 TB each in RAID0 mode. later on it was converted to Raid10, so 12 TB was used.
  
  LikeLike
  
  Comment by Syed Jahanzaib / Pinochio~:) — September 3, 2013 @ 8:02 AM
  
  Reply
  - 44: refresh_pattern \.youtube\.com\/videoplayback\? 120 80% 180 ignore-no-cache ignore-no-store override-expire override-lastmod ignore-private
    Squid Cache (Version 3.1.19): Terminated abnormally.
    
    LikeLike
    
    Comment by using you script following error is coming zabib bahi. — September 7, 2013 @ 6:46 PM
hi,
i’m using 2 squid box,
the first one as videocache and the other as normal squid.

my topology network just like this,
mikrotik —- squid1
—— videocache

can squid1 bypass bandwidth from videocache as the parent squid?

and how can i set the qos rule for these?

thx

LikeLike

Comment by Synard Zoldic — September 19, 2013 @ 11:12 AM

Reply
i have the last version and cracked and test 100% if you needed i can send it to you
the problem of this script is that i eat the whole bandwidth you have
so in my opinion it didnt save bandwidth, it consume the whole bandwidth !!
tell me bro, squid3 better than squid2.7 ? i didnt test squid3 yet …
thank you …

LikeLike

Comment by Hussein — October 5, 2013 @ 7:06 PM

Reply
- in performance 3.x is far better 🙂
  but unfortunately it donot support storeurl method.
  
  LikeLike
  
  Comment by Syed Jahanzaib / Pinochio~:) — October 22, 2013 @ 3:27 PM
  
  Reply
- can get it plz
  
  LikeLike
  
  Comment by dh — December 12, 2013 @ 6:09 PM
  
  Reply
  - me 2 please? =) can’t find 2.3 anywhere but I have 2.1 if anyone wants. the problem is 2.1 does not seem to be caching youtube videos anymore. they appear in log but not in cache folder and then the log shows a MISS. The author is bloody mad to ask $400… there are many home users who want to use this and who don’t consider it worth more than $50 never mind $400.. It’s a linux-based package – an extension to SQUID which is 100% free, and it’s coded in python, and yet he’s asking so much??? Crazy. anyways
    
    LikeLike
    
    Comment by Dimitri Pappas (@fragtion) — December 26, 2013 @ 2:44 PM
  - 2.1 is common in market, but its not caching youtube properly, better to get 2.3
    
    Also I do agree that 400$ per year is very high price , just for python codes :p
    BTW, have you tried following?
    
    Youtube caching with SQUID 2.7 [using storeurl.pl]
    
    Its working much better then VC, as this method does not consume double bandwidth for view+download. The only downside is its compatible with 2.7 Only.
    
    LikeLike
    
    Comment by Syed Jahanzaib / Pinochio~:) — December 26, 2013 @ 2:58 PM
- Hussein plz can u sent it to me?
  
  LikeLike
  
  Comment by azu — May 15, 2016 @ 10:28 AM
  
  Reply
sayed what your email address plz

can you tallk to mee ?

LikeLike

Comment by khedor tahesh — February 18, 2015 @ 10:29 PM

Reply