Syed Jahanzaib Personnel Blog to Share Knowledge !

July 21, 2011

MIKROTIK with SQUID/ZPH: how to bypass Squid Cache HIT object with Queues Tree in RouterOS 5.x & 6.x


.

zph

.

Using Mikrotik, we can redirect HTTP traffic to SQUID proxy Server, We can also control user bandwidth, but its a good idea to deliver the already cached content to user at full lan speed, that’s why we setup cache server for, to save bandwidth and have fast browsing experience , right :p , So how can we do it in mikrotik that cache content should be delivered to users at unlimited speed, no queue on cache content. Here we go.

By using ZPH directives , we will mark cache content, so that it can later pick by Mikrotik.

Basic requirement is that Squid  must be running in transparent mode, can be done via iptables and squid.conf directives.
I am using UBUNTU squid 2.7 , (in ubuntu , apt-get install squid will install squid 2.7 by default which is gr8 for our work)
Add these lines in SQUID.CONF

#===============================================================================
#ZPH for SQUID 2.7 (Default in ubuntu 10.4) / Syed Jahanzaib aacable@hotmail.com
#===============================================================================
tcp_outgoing_tos 0x30 lanuser [lanuser is ACL for local network, change it to match your's]
zph_mode tos
zph_local 0x30
zph_parent 0
zph_option 136

Use following if you have squid 3.1.19


#======================================================
#ZPH for SQUID 3.1.19 (Default in ubuntu 12.4) / Syed Jahanzaib aacable@hotmail.com
#======================================================

# ZPH for Squid 3.1.19
qos_flows local-hit=0x30

That’s it for SQUID, Now moving on to Mikrotik box ,
Add following rules,

# Marking packets with DSCP (for MT 5.x) for cache hit content coming from SQUID Proxy

/ip firewall mangle add action=mark-packet chain=prerouting disabled=no dscp=12 new-packet-mark=proxy-hit passthrough=no comment="Mark Cache Hit Packets / aacable@hotmail.com"

/queue tree add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=0 max-limit=0 name=pmark packet-mark=proxy-hit parent=global-out priority=8 queue=default

# Marking packets with DSCP (for MT 6.x) for cache hit content coming from SQUID Proxy

/ip firewall mangle add action=mark-packet chain=prerouting comment="MARK_CACHE_HIT_FROM_PROXY_ZAIB" disabled=no dscp=12 new-packet-mark=proxy passthrough=no
/queue tree add comment="ZPH Cache Hit Queue : ) / Syed Jahanzaib" name=proxy packet-mark=zph-hit parent=global priority=1 queue=default

# If above queue tree donot work with ROS 6.x then try adding simple queue and move it on TOP in simple QUEUE section,
.


/queue simple
add max-limit=100M/100M name="ZPH-Proxy Cache Hit Simple Queue / Syed Jahanzaib >aacable@hotmail.com" packet-marks=zph-hit priority=1/1 target="" total-priority=1

Now every packet which is marked by SQUID CACHE_HIT, will be delivered to user at Full lan speed, rest of traffic will be restricted by user Queue.

TROUBLESHOOTING:

the above config is fully tested with UBUNTU SQUID 2.7 and FEDORA 10 with LUSCA

Make sure your squid is marking TOS for cache hit packets. You can check it via TCPDUMP

__________________________________________________________
tcpdump -vni eth0 | grep ‘tos 0×30′

(eht0 = LAN connected interface)

Can you see something like ???

tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 96 bytes
20:25:07.961722 IP (tos 0×30, ttl 64, id 45167, offset 0, flags [DF], proto TCP (6), length 409)
20:25:07.962059 IP (tos 0×30, ttl 64, id 45168, offset 0, flags [DF], proto TCP (6), length 1480)
192 packets captured
195 packets received by filter
0 packets dropped by kernel
_________________________________

Regard’s
SYED JAHANZAIB

MIKROTIK :Howto Redirect HTTP traffic to SQUID with Original Source Client IP


Howto connect SQUID box with Mikrotik and Log user’s original source ip in squid proxy log.

Consider the following Scenario.

Mikrotik + SQUID + Client Source IP Loggin

In this scenario Mikrotik is acting as PPPoE Server. All internet traffic is passed from Mikrotik, except the HTTP PORT 80 data, which is marked and route to SQUID proxy server for caching facility and some other filtering task. We want to preserve source client IP address in squid logs, so that user web activity can be tracked via squid access log, sometimes it can be require by law enforcing agencies or can be required by administration / reporting / management purposes. So how to do it ?

I assume you already have working SQUID and Mikrotik (pppoe server with pppoe user ip pool 172.16.0.0/16 pool in place. however you can change it according to your scenario)

We will divide this article in two sections

1# MIKROTIK RouterOS Configuration
2# SQUID Proxy Server Configuration


1# MIKROTIK  RouterOS CONFIGURATION !

In this example, Mikrotik have 3 LAN interfaces,  Details are as following . . .

[admin@MikroTik] > ip address print
#   ADDRESS            NETWORK         INTERFACE
 0   192.168.0.1/24     192.168.0.0     ether1       >> LAN INTERFACE
 1   192.168.2.1/24     192.168.2.0     ether2       >> PROXY INTERFACE
 2   192.168.1.2/24     192.168.1.0     ether3       >> WAN - DSL INTERFACE

Now we will start with the Mikrotik configuration:

# Add ip addresses for interface cards,

/ip address
add address=192.168.0.1/24 disabled=no interface=ether1 network=192.168.0.0
add address=192.168.2.1/24 disabled=no interface=ether2 network=192.168.2.0
add address=192.168.1.2/24 disabled=no interface=ether3 network=192.168.1.0

# Set DNS for Host Resolving

/ip dns set allow-remote-requests=yes cache-max-ttl=1w cache-size=8000KiB max-udp-packet-size=512 servers=192.168.1.1

# Mark All HTTP Port 80 Traffic, so that we can use these Marked Packets in Route section.

/ip firewall nat
add action=accept chain=srcnat disabled=no dst-port=80 protocol=tcp

/ip firewall mangle
add action=mark-routing chain=prerouting disabled=no dst-port=80 new-routing-mark=http passthrough=yes protocol=tcp

# Masquerade all traffic (Except http] on ether3 Only, which is connected with DSL Router. This is important to masquerade traffic on WAN Interface only, otherwise http packets will also be masqueraded with mikrotik ip.

add action=masquerade chain=srcnat disabled=no out-interface=ether3

# Define Route for HTTP Marked packets, and set default rule for all other traffic, This is called policy base or pre traffic base routing

/ip route
add disabled=no distance=1 dst-address=0.0.0.0/0 gateway=192.168.2.2 routing-mark=http scope=30 target-scope=10
add disabled=no distance=1 dst-address=0.0.0.0/0 gateway=192.168.1.1 scope=30 target-scope=10

(Where 192.168.2.2 is the Squid Proxy Server LAN IP , and 192.168.1.1 is our DSL Router IP)

That’s all for MIKROTIK, Now Mikrotik will Redirect HTTP Traffic to Squid Proxy via interface ether2. and all rest of traffic will be masqueraded/nat to WAN (ether3) which is connected with DSL.

Now moving on to SQUID section !

2# SQUID Proxy Server CONFGIURATION !

I assume you already have working squid in transparent mode [its necessary you set this via iptables and some directives in squid.conf

For example: (squid.conf)

#==============================
#Transparent Mode & Example ACL
#==============================
http_port 8080 transparent
acl mylocalnet src 0.0.0.0/0.0.0.0
http_access allow mylocalnet

and IPTABLES example:

# DNAT port 80 request comming from LAN systems to squid 8080 aka transparent proxy .zaib.
iptables -t nat -A PREROUTING -i eth0 -p tcp --dport 80 -j DNAT --to 192.168.2.2:8080

In this example, Squid proxy server have 2 Interface cards

eth0: LAN (connected with Mikrotik's PROXY INTERFACE ether2) = 192.168.2.2

eth1: WAN (connected with DSL Router) = 192.168.1.3 /  Default GW = 192.168.1.1  / DNS = 192.168.1.1 , 8.8.8.8

It is necessary that SQUID can directly communicate with the users by setting ROUTE to communicate user subnet via mikrotik, otherwise it won't be able to communicate with the user , Issue the following command ,

route add -net 172.16.0.0 netmask 255.255.0.0 gw 192.168.2.1 dev eth0

172.16.0.0/16 is pppoe user IP pool, After successfully execution, Squid will be able to see the Users ip. Also add the above route command in /etc/rc.local (u must add any command in rc.local before 'exit 0'  if using ubuntu)

Above command Explanation:  172.16.0.0 are pppoe users ip pool and 192.168.2.1 is Mikrotik ether2 which is directly connected to Squid via crossover cable , thus we are telling Squid to look after for users 172.16.0.0 via gateway 192.168.2.1 which is mikrotik, if we don't use this, squid and users wont be able to communicate with each other)

That's it. Now when user will try to use internet, his HTTP traffic will be marked by Mikrotik and then all HTTP marked traffic will be ROUTED to SQUID proxy [192.168.2.2]  with original client ip, instead of Mikrotik ip.

Some useful links for fine tuned working squid.conf and Linux internet sharing script.

http://aacable.wordpress.com/2011/06/01/linux-simple-internet-sharing-script/

http://aacable.wordpress.com/2011/06/01/working-squid-conf-example-fil/

Cheers,
:)

ALLAH  HAFIZ ,
Syed Jahanzaib !

The Silver is the New Black Theme Blog at WordPress.com.

Follow

Get every new post delivered to your Inbox.

Join 2,047 other followers