Syed Jahanzaib Personal Blog to Share Knowledge !

June 19, 2020

Mikrotik Queue Tree with Traffic Priority

Filed under: Mikrotik Related — Syed Jahanzaib / Pinochio~:) @ 3:17 PM


Mikrotik Queue tree with Traffic Priority

Example if you have assigned user with 1 mb profile, & if user is doing full download, his stream will get degrade, buffered because IDM will fetch using full available bandwidth using multiple connection.

Using Priority feature in TREE, we can configure mikrotik to assign user 1mb bandwidth limit, but it should always give 1mb priority to CDN traffic first, then to others. This way even if user is downloading at full speed liek using IDM, and if he plays video from CDN , then bandwidth priority will be given to CDN first (IDM download will degrade so that priority can be given to CDN.

This was done to avoid streaming buffer issue even if user is downloading at full speed .

– Youtube Link: https://www.youtube.com/watch?v=WxDzEonl-Bk

Queue Code: [dynamic queue for user is auto created upon user pppoe connection, on top of that we r using queue tree to prioritize]


/ip firewall address-list
add address=172.16.99.0/24 list=1mb
add address=58.27.130.0/24 list=cdn_list

/ip firewall mangle
add action=mark-connection chain=forward new-connection-mark=ICMP_Conn protocol=icmp
add action=mark-packet chain=forward connection-mark=ICMP_Conn new-packet-mark=ICMP_Pkts passthrough=no
add action=mark-packet chain=forward comment="MARK CDN UP" dst-address-list=1mb new-packet-mark=cdn_up passthrough=no \
src-address-list=cdn_list
add action=mark-packet chain=forward comment="MARK CDN DOWN" dst-address-list=cdn_list new-packet-mark=cdn_down \
passthrough=no src-address-list=1mb
add action=mark-packet chain=forward comment=MARK_1MB_UP_USER new-packet-mark=1mb_up passthrough=no src-address-list=1mb
add action=mark-packet chain=forward comment=MARK_1MB_DOWN_USER dst-address-list=1mb new-packet-mark=1mb_down \
passthrough=no

/queue simple
add max-limit=1M/1M name=ICMP packet-marks=ICMP_Pkts target=""

/queue tree
add max-limit=1G name=10G-ZAIB-WAN-Link parent=global
add name="icmp pkts Top Priority from Main Feed - Zaib" packet-mark=ICMP_Pkts parent=10G-ZAIB-WAN-Link priority=1
add name=1mb-overall-internet-up packet-mark=1mb_up parent=10G-ZAIB-WAN-Link queue=1mb-upload
add name=1mb-overall-internet-down packet-mark=1mb_down parent=10G-ZAIB-WAN-Link queue=1mb-download
add name=cdn-down-hi-priority packet-mark=cdn_down parent=1mb-overall-internet-down priority=2 queue=1mb-download
add name=cdn-up-hi-priority packet-mark=cdn_up parent=1mb-overall-internet-up priority=2 queue=1mb-upload

/queue simple
add max-limit=1M/1M name=ICMP packet-marks=ICMP_Pkts target=""

Regard’s
Syed Jahanzaib

May 15, 2020

Mikrotik to Mikrotik/Cisco Bonding – Reference Notes

Filed under: Cisco Related, Mikrotik Related — Tags: , , , , — Syed Jahanzaib / Pinochio~:) @ 2:40 PM

aggregated

cisco bonding mikrotik to cisco

2 ports bonding

4ports_bonding

2020-05-13 13.19.04

This post was made for self reference purposes, so that I can find the configs easily from this page when needed again.


This post describes possible methods of creating ethernet interfaces bonding between Mikrotik to Cisco (etherchannel) or Mikrotik to Mikrotik to achieve load balancing & failover to achieve higher speed from multiple ethernet interfaces. Bonding is a technology that allows aggregation of multiple Ethernet-like interfaces into a single virtual link, thus getting Maximum Throughput and providing failover. You can use this technique to create bonding between WAN Cisco switch/R to User Mikrotik, then from User mikrotik to User distribution vlan switch. lots of combinations you can create in this regard.

Possible Scenarios:

OP have Mikrotik CCR1036 routerboard with SFP+. ISP have given OP with 10G of SFP+ connectivity on OP WAN Mikrotik router. but on user side router OP have simple CCR1036 with 1G ports only. So the wan link from 1036-wan to 1036-LAN is choking at  1G. Proper solution is to have back to back 10G connectivity between all routers, but since it requires cash investment, therefore the OP sometimes chooses workaround to fulfill the requirements for the time being. BONDING is one of that workaround that can be chosen. although I try to avoid using it as much as possible & always look for 10/25G solutions.


Hardware Used in this post

  • Mikrotik: RB2011
  • Cisco 36450 24 Ports Switch
  • In general you can use upto 8 ports Bonding, should be of same type/speed.
  • All ports should be enabled before adding them to bond.

Option-1# Mikrotik to Cisco Bonding (using 1Gx2 interfaces to achieve failover / load balancing / higher speed)

Using any two interfaces, we are creating 2 port Bonding interface. Example from Mikrotik Port 9 & 10 we will connect two cables connected to Cisco Switch port 23 & 24.

#Mikrotik Example Code , using port 9 & 10

/interface bonding
add link-monitoring=none mode=802.3ad name=bonding1-lan slaves=ether9,ether10 transmit-hash-policy=layer-2-and-3

# Cisco Switch Example Code, suing port 23 & 24

configure terminal
inter range gigabitEthernet 1/0/23-24
channel-group 1 mode active
channel-protocol lacp
#on newer IOS you dont need following CMD
#switchport trunk encapsulation dot1q
switchport mode trunk

Note: If you are doing Cisco to Cisco Port Bonding , then you need to repeat above code on 2nd switch too

Option-2# Mikrotik to Mikrotik Bonding

Its very simple. Just issue below command on both mikrotik and connect the cables from designated interfaces, assign IP, and you are good to go …

/interface bonding
add name=bonding1-to-ppp slaves=ether1,ether2

Some Helpful CMD’s related to etherchannel on Cisco switch …

show etherchannel summary
show etherchannel 1 port-channel
show interfaces etherchannel

Note: Before you connect both cable, make sure to add the config on Mikrotik & Cisco side, Else cisco port will get shut dueto STP protection to avoid loop.

*May 13 04:58:33.710: %ETC-5-L3DONTBNDL2: Gi1/0/23 suspended: LACP currently not enabled on the remote port.

Configuring EtherChannel Load Balancing

 

To view etherchannel load balancing setting, use the show command …

enable
show etherchannel load-balance

To configure load balancing, use the following commands:

enable
configure terminal
port-channel load-balance dst-mac
do wr

Regard’s
Syed Jahanzaib

January 8, 2020

Syslog-ng – Part 3: Minimized logging to mysql with dynamic tables & trimming

Filed under: Linux Related, Mikrotik Related — Tags: , — Syed Jahanzaib / Pinochio~:) @ 1:27 PM

syslog cgnat

Revision: 7th-JAN-2020


In continuation to existing posts related to syslog-ng, Following post illustrates on how you can log only particular messages with pattern matching and let syslog-ng creates dynamic table based on the dates so that searching/querying becomes easy.

This task was required in relation to CGNAT logging. you may want to read it here

https://aacable.wordpress.com/2020/01/01/mikrotik-cgnat/

Hardware Software used in this post:

  • Mikrotik Routerboard – firmware 6.46.1
  • Ubuntu 18.4 Server x64 along with syslog-ng version 3.25.1 on some decent hardware

Requirements:

I assume that you already have working setup for Syslog-ng& your remote devices are already sending logs and they are storing in the mysql already. See Part 1 & 2


Ref: Installing latest version of syslog-ng

#Make sure to change the version, I have used this CMD on Ubuntu 16.04 , for version 18, you may change this to 18.04

wget -qO - http://download.opensuse.org/repositories/home:/laszlo_budai:/syslog-ng/xUbuntu_16.04/Release.key | sudo apt-key add -
touch /etc/apt/sources.list.d/syslog-ng-obs.list
echo "deb http://download.opensuse.org/repositories/home:/laszlo_budai:/syslog-ng/xUbuntu_16.04 ./" > /etc/apt/sources.list.d/syslog-ng-obs.list
apt-get update
apt-get install syslog-ng

at my system I got this

syslog-ng 3 (3.25.1)
Config version: 3.25
Installer-Version: 3.25.1
Revision: 3.25.1-1
Compile-Date: Dec 12 2019 12:00:29
Module-Directory: /usr/lib/syslog-ng/3.25
Module-Path: /usr/lib/syslog-ng/3.25
Include-Path: /usr/share/syslog-ng/include
Error opening plugin module; module='mod-java', error='libjvm.so: cannot open shared object file: No such file or directory'
Available-Modules: add-contextual-data,afsmtp,tfgetent,afsql,cryptofuncs,http,confgen,sdjournal,system-source,cef,syslogformat,json-plugin,afprog,riemann,csvparser,affile,afsocket,afamqp,redis,examples,disk-buffer,xml,linux-kmsg-format,map-value-pairs,hook-commands,kafka,tags-parser,dbparser,graphite,appmodel,afstomp,pacctformat,afmongodb,pseudofile,basicfuncs,geoip2-plugin,kvformat,stardate,timestamp,mod-python,afuser,snmptrapd-parser
Enable-Debug: off
Enable-GProf: off
Enable-Memtrace: off
Enable-IPv6: on
Enable-Spoof-Source: on
Enable-TCP-Wrapper: on
Enable-Linux-Caps: on
Enable-Systemd: on

Status:

root@agpis-linux-test:/var/log/zlogs# service syslog-ng status
â syslog-ng.service - System Logger Daemon
Loaded: loaded (/lib/systemd/system/syslog-ng.service; enabled; vendor preset: enabled)
Active: active (running) since Wed 2020-01-08 12:46:52 PKT; 27min ago
Docs: man:syslog-ng(8)
Main PID: 7086 (syslog-ng)
Tasks: 2 (limit: 2290)
CGroup: /system.slice/syslog-ng.service
ââ7086 /usr/sbin/syslog-ng -F

Jan 08 12:46:52 agpis-linux-test systemd[1]: Starting System Logger Daemon...
Jan 08 12:46:52 agpis-linux-test syslog-ng[7086]: [2020-01-08T12:46:52.362728] Macro escaping can only be specified for inline templates;
Jan 08 12:46:52 agpis-linux-test syslog-ng[7086]: [2020-01-08T12:46:52.364052] WARNING: With use-dns(no), dns-cache() will be forced to 'no' too!;
Jan 08 12:46:52 agpis-linux-test systemd[1]: Started System Logger Daemon.

Create Database in mySQL to store dynamic tables

Create Base Database for storing dynamically created date wise tables

mysql -uroot -pXXX -e "create database syslog;"

Now edit the syslog-ng file

nano /etc/syslog-ng/syslog-ng.conf

& use following as sample. I would recommend that you should add only relevant part, just dont do blind copy paste. This is just sample for demonstration purposes only …

 

Syslog-ng Sample File

@version: 3.25
@include "scl.conf"
# Syslog-ng CUSTOMIZED configuration  file
# Syed Jahanzaib / aacable at hotmail dot com /https://aacable.wordpress.com
# First, set some global options.
options { chain_hostnames(off); flush_lines(0); use_dns(no); use_fqdn(no);
owner("root"); group("adm"); perm(0640); stats_freq(0);
bad_hostname("^gconfd$");
};

######## Zaib Section Starts here
# Accept connection on UDP
source s_net { udp (); };

# Adding filter for our Mikrotik Routerboard, store logs in FILE as primary
# MIKROTIK ###########

# This entry will LOG all information coming from this IP
filter f_mikrotik_252 { host("101.11.11.252"); };
# This entry will LOG ONLY particular message that contains word NAT, useful to minimize CGNAT logging, Enable one entry at a time # ZAIB
#filter f_mikrotik_252 { host("101.11.11.252") and match("NAT" value("MESSAGE")) };
log { source ( s_net ); filter( f_mikrotik_252 ); destination ( df_mikrotik_252 ); };
# add info in LOG (Part1)
destination df_mikrotik_252 {
file("/var/log/zlogs/${HOST}.${YEAR}.${MONTH}.${DAY}.log"
template-escape(no));
};
source s_mysql {
udp(port(514));
tcp(port(514));
};

# Store Logs in MYSQL DB as secondary # add info in MYSQL (Part2)
destination d_mysql {
sql(type(mysql)
host("localhost")
# MAKE SURE TO CHANGE CREDENTIALS
username("root")
password("XXXXXXXX")
database("syslog")
table("${R_YEAR}_${R_MONTH}_${R_DAY}")
columns( "id int(11) unsigned not null auto_increment primary key", "host varchar(40) not null", "date datetime", "message text not null")
values("0", "$FULLHOST", "$R_YEAR-$R_MONTH-$R_DAY $R_HOUR:$R_MIN:$R_SEC", "$MSG")
indexes("id"));
};
log {
source(s_net);
filter(f_mikrotik_252);
destination(d_mysql);
};
########################
# Sources
########################
# This is the default behavior of sysklogd package
# Logs may come from unix stream, but not from another machine.
#
source s_src {
system();
internal();
};

IMPORTANT:

Create ‘zlogs‘ folder in /var/log , so that mikrotik logs will be saved in separate file.

mkdir /var/log/zlogs

Mikrotik rule to LOG Forward chain

/system logging action

set 1 disk-file-count=50 disk-lines-per-file=5000
set 3 remote=101.11.11.254

/system logging add action=remote topics=info

/system logging action set 3 remote=101.11.11.254

Restart Syslog-ng server

Now restart syslog-ng service

service syslog-ng restart

and you will see the dynamic tables created as follows

mysql -uroot -pXXXXX
mysql: [Warning] Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 411
Server version: 5.7.28-0ubuntu0.18.04.4-log (Ubuntu)
Copyright (c) 2000, 2019, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> use syslog;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Database changed
mysql> show tables;
+------------------+
| Tables_in_syslog |
+------------------+
| 2020_01_08 |
+------------------+
1 row in set (0.00 sec)

mysql> describe 2020_01_08;
+---------+------------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+---------+------------------+------+-----+---------+----------------+
| id | int(11) unsigned | NO | PRI | NULL | auto_increment |
| host | varchar(40) | NO | | NULL | |
| date | datetime | YES | | NULL | |
| message | text | NO | | NULL | |
+---------+------------------+------+-----+---------+----------------+
4 rows in set (0.00 sec)

& you can then see data insertion into the table as soon LOG is received from remote devices

2020-01-08T07:49:43.020811Z 430 Query INSERT INTO 2020_01_08 (id, host, date, message) VALUES ('0', '101.11.11.252', '2020-01-08 12:49:28', 'forward: in: out:ether1-agp-wan, src-mac d0:bf:9c:f7:88:76, proto TCP (ACK,PSH), 172.16.0.2:57193->172.217.19.174:443, NAT (172.16.0.2:57193->101.11.11.252:2244)->172.217.19.174:443, len 79')
2020-01-08T07:49:43.031281Z 430 Query INSERT INTO 2020_01_08 (id, host, date, message) VALUES ('0', '101.11.11.252', '2020-01-08 12:49:28', 'forward: in: out:ether1-agp-wan, src-mac d0:bf:9c:f7:88:76, proto TCP (ACK,FIN), 172.16.0.2:57096->3.228.94.102:443, NAT (172.16.0.2:57096->101.11.11.252:2219)->3.228.94.102:443, len 40')
2020-01-08T07:49:43.041420Z 430 Query INSERT INTO 2020_01_08 (id, host, date, message) VALUES ('0', '101.11.11.252', '2020-01-08 12:49:38', 'forward: in: out:ether1-agp-wan, src-mac d0:bf:9c:f7:88:76, proto UDP, 172.16.0.2:49247->216.58.208.234:443, NAT (172.16.0.2:49247->101.11.11.252:2202)->216.58.208.234:443, len 1378')
2020-01-08T07:49:43.051112Z 430 Query INSERT INTO 2020_01_08 (id, host, date, message) VALUES ('0', '101.11.11.252', '2020-01-08 12:49:38', 'forward: in: out:ether1-agp-wan, src-mac d0:bf:9c:f7:88:76, proto UDP, 172.16.0.2:49247->216.58.208.234:443, NAT (172.16.0.2:49247->101.11.11.252:2202)->216.58.208.234:443, len 1378')
2020-01-08T07:49:43.061280Z 430 Query INSERT INTO 2020_01_08 (id, host, date, message) VALUES ('0', '101.11.11.252', '2020-01-08 12:49:39', 'forward: in: out:ether1-agp-wan, src-mac d0:bf:9c:f7:88:76, proto UDP, 172.16.0.2:49760->172.217.19.1:443, NAT (172.16.0.2:49760->101.11.11.252:2202)->172.217.19.1:443, len 1378')
2020-01-08T07:49:43.071449Z 430 Query INSERT INTO 2020_01_08 (id, host, date, message) VALUES ('0', '101.11.11.252', '2020-01-08 12:49:39', 'forward: in: out:ether1-agp-wan, src-mac d0:bf:9c:f7:88:76, proto UDP, 172.16.0.2:49760->172.217.19.1:443, NAT (172.16.0.2:49760->101.11.11.252:2202)->172.217.19.1:443, len 1378')
2020-01-08T07:49:44.828993Z 430 Query INSERT INTO 2020_01_08 (id, host, date, message) VALUES ('0', '101.11.11.252', '2020-01-08 12:49:44', 'forward: in: out:ether1-agp-wan, src-mac d0:bf:9c:f7:88:76, proto UDP, 172.16.0.2:53503->216.58.208.234:443, NAT (172.16.0.2:53503->101.11.11.252:2203)->216.58.208.234:443, len 827')
2020-01-08T07:49:44.851034Z 430 Query INSERT INTO 2020_01_08 (id, host, date, message) VALUES ('0', '101.11.11.252', '2020-01-08 12:49:44', 'forward: in: out:ether1-agp-wan, src-mac d0:bf:9c:f7:88:76, proto UDP, 172.16.0.2:53503->216.58.208.234:443, NAT (172.16.0.2:53503->101.11.11.252:2203)->216.58.208.234:443, len 827')
2020-01-08T07:51:37.518276Z 430 Query INSERT INTO 2020_01_08 (id, host, date, message) VALUES ('0', '101.11.11.252', '2020-01-08 12:51:37', 'forward: in: out:ether1-agp-wan, src-mac d0:bf:9c:f7:88:76, proto TCP (ACK), 172.16.0.2:57202->91.195.240.126:80, NAT (172.16.0.2:57202->101.11.11.252:2260)->91.195.240.126:80, len 41')
2020-01-08T07:51:37.522015Z 430 Query INSERT INTO 2020_01_08 (id, host, date, message) VALUES ('0', '101.11.11.252', '2020-01-08 12:51:37', 'forward: in: out:ether1-agp-wan, src-mac d0:bf:9c:f7:88:76, proto TCP (ACK), 172.16.0.2:57202->91.195.240.126:80, NAT (172.16.0.2:57202->101.11.11.252:2260)->91.195.240.126:80, len 41')

syslog-ng dynamic table data from phpmyadmin.PNG


Regard’s
Syed Jahanzaib

January 1, 2020

CGNAT Deployment using Mikrotik RouterOS

Filed under: Mikrotik Related — Tags: , , , , , , , — Syed Jahanzaib / Pinochio~:) @ 5:34 PM

mdnet cgnat conept.PNG

Note: This is In-complete Post. It contains src-nat method part only. Second method of NETMAP will be added later (if time will allow)  which is I feel far more simple & efficient as compared to the src-nat method. But this method is ok too to comply with the Law using little resources.

My humble request, Kindly donot consider me as an expert on this stuff, I am NOT certified in anything Mikrotik/Cisco/Linux or Windows. However I have worked with some networks and I read , research & try stuff all of the time. When you are enslaved by private job & working as one man army, you have to perform many task in which you are not formally trained for. So I am not speaking/posting about stuff I am formerly trained in, I pretty much go with experience and what I have learned on my own. And , If I don’t know something then I read & learn all about it.

So , please don’t hold me/my-postings to be always 100 percent correct. I make mistakes just like everybody else. However – I do my best, learn from my mistakes and try to help others

Some references used in this post


*CG-NAT* as Workaround:

CGNAT concept is used to share one or preferably more public IP addresses with large number of private ip addresses on ratio basis.CGNAT/NAT444 is a conception, not a function. In terms of RouterOS functionality it’s simple SRC NAT rule.

To combat with this IPV4 exhausting issue, we can use CGNAT as a workaround. This is by no means a solution, & the OP should get public IP space (either ipv4 or ipv6) to comply with the LAW.

Note: Please note that CGNAT concept is mostly for UDP/TCP and its generally not meant for other protocols.

Some possible disadvantages of using CGNAT concept:

  • CGNAT is not sustainable in the long term, hectic to manage the private/public pools especially if you have multiple NASes doing same job
  • ISP deploying IP address sharing techniques should also deploy a corresponding logging architecture to maintain records of the relation between a customer’s identity and IP/port resources utilized
  • You should deploy additional SYSLOG server (either windows or linux base) to store logs. I would prefer linux base SYSLOG-NG). Tracking of users for legal reasons means searching hundreds GB’s of logging would be required, as multiple end users go behind one (or more) public IP address(es). Tracking Logs is not an easy task particularly when you have tongs of Logging (in a DB).Logging every NAT translation is resource consuming. Some super fast computing resources (including preferably RAID10 or SSD based storage) and fine tune DB would be required
  • A CG-NAT device must use the same external IP address mapping for all sessions associated with the same internal IP address
  • Most Applications do not behave well with TCP resets
  • Many operators are still not familiar with CG-NAT complexities. There is a lot of trial and error on the part of ISP’s

 

in my personal experience , Deployment is somewhat hectic, & tracking any request is daunting task ! z@ib


Hardware/Software Used in this post:


CGNAT logging to remote syslog server with some customization

https://aacable.wordpress.com/2020/01/08/syslog-ng-part-3-minimized-logging-to-mysql-with-dynamic-tables-trimming/


Scenario#1

OP is running mini ISP with around 200 active subscribers. Mikrotik Router is being used as PPPoE Server along with Freeradius as AAA. On Mikrotik, one public IP is configured for WAN and additional /24 routed pool (256 public IP addresses) is provided to the OP via ISP so that he can provide public IP to each user. After the network upgrades , OP have reached 700 users in total, and since he have only 256 public ip’s , he is now using natting for half of his users.

We all know that IPV4 shortage is on peak , getting ipv4 is expensive for 3rd world countries & small ISP’s as well.

This NATTING workaround is creating hurdles in tracking illegal activity performed by any NATTED users because hundreds of NATTED user will have same public ip (Mikrotik WAN IP). nowadays law sometimes provide only the public ip along with source port and ask for the user credentials details for investigation purposes.

with single public IP and hundreds of natted hosts behind it. tracking is nearly impossible.


IP scheme example used in this Scenario#1:

Public IP range: (/24 public IP’s routed pool)

  • 1.1.1-1.1.1.255
  • Total Public IP useable: 255

Private IP range for PPPoE users:

  • 172.16.1.1-172.16.1.255
  • 172.16.2.1-172.16.2.255
  • 172.16.3.1-172.16.3.255
  • Total Private IP useable: 765

For 765 Users, we will be using 1:5 Ratio, thus 153 public ips will be used for 765 users. (on a ratio of 1:5).

  • per private IP, we will reserve 10,000 ports, which should be more than enough for each user.
  • per private IP, we will be creating 3 rules, one for TCP, second for UDP, 3rd for non ports range [Use 3rd this rule with caution, it will nat every non tcp/udp traffic, some firewalling may be put, ALSO YOU MAY NOT BE NEEDING 3rd rule which can eliminate 1/3 rules]

in my personal expeirence, CGNAT configuration on RouterOS is very much similar to regular source NAT configuration.


To add multiple Public IP addresses on WAN interface in bulk using single CMD on Terminal

You may need to add all of your public IP addresses (which will be used for CGNAT) on WAN interface(required for troubleshooting purposes as well).

To add ips in bulk using single CMD, you can use Mikrotik FOR X script function for ease / ZAIB

:for x from 1 to 153 do={ /ip address add address="1.1.1.$x/32" comment="1.1.1.$x - Routed IP for ppp CGNAT - zaib" interface="ether1-wan"}

 


Adding FUNCTION in Mikrotik for later Automation

Paste this in Mikrotik RouterOS terminal:

# CGNAT Customized minimalistic Script to add function.
# Disclaimer: This particular function is not made by ME, I only trimmed/modified it to suite my local requirements
# Syed Jahanzaib / aacable at hotmail dot com
:global sqrt
:global sqrt do={
:for i from=0 to=$1 do={
:if (i * i > $1) do={ :return ($i - 1) }
}
}
:global addNatRules do={
:local x [$sqrt $count]
:local y $x
:if ($x * $x = $count) do={ :set y ($x + 1) }
:for i from=0 to=($count - 1) do={
:local prange "$($portStart + ($i * $portsPerAddr))-$($portStart + (($i + 1) * $portsPerAddr) - 1)"
# src-nat TCP traffic
/ip firewall nat add chain=srcnat action=src-nat protocol=tcp src-address=($srcStart + $i) to-address=$toAddr to-ports=$prange
# src-nat UDP traffic
/ip firewall nat add chain=srcnat action=src-nat protocol=udp src-address=($srcStart + $i) to-address=$toAddr to-ports=$prange
# This below 3rd rule is created to allow protocols other then tcp/udp, example ICMP ? , use it with caution , zaib
/ip firewall nat add chain=srcnat action=src-nat src-address=($srcStart + $i) to-address=$toAddr
}
}

Now we have function inserted with the help of above code, and using this function, we can create rules in bulk using following CMD to add rules in NAT section

# per private IP, we will reserve 10000 ports, which should be more than enough for each user.
# per private IP, we will be creating 3 rules, one for TCP, second for UDP, 3rd for non ports range

$addNatRules count=5 srcStart=172.16.1.1 toAddr=1.1.1.1 portStart=10000 portsPerAddr=9999
$addNatRules count=5 srcStart=172.16.1.6 toAddr=1.1.1.2 portStart=10000 portsPerAddr=9999
$addNatRules count=5 srcStart=172.16.1.11 toAddr=1.1.1.3 portStart=10000 portsPerAddr=9999
$addNatRules count=5 srcStart=172.16.1.16 toAddr=1.1.1.4 portStart=10000 portsPerAddr=9999
$addNatRules count=5 srcStart=172.16.1.21 toAddr=1.1.1.5 portStart=10000 portsPerAddr=9999
$addNatRules count=5 srcStart=172.16.1.26 toAddr=1.1.1.6 portStart=10000 portsPerAddr=9999
$addNatRules count=5 srcStart=172.16.1.31 toAddr=1.1.1.7 portStart=10000 portsPerAddr=9999
$addNatRules count=5 srcStart=172.16.1.36 toAddr=1.1.1.8 portStart=10000 portsPerAddr=9999
$addNatRules count=5 srcStart=172.16.1.41 toAddr=1.1.1.9 portStart=10000 portsPerAddr=9999
$addNatRules count=5 srcStart=172.16.1.46 toAddr=1.1.1.10 portStart=10000 portsPerAddr=9999

# & so on for rest of the pool, you can further automate this by using additional functions & scripting

Enable Logging of CG-NAT Output:

# To log user IP/NAT information on LOG window / you can configure remote log to syslog-server too
/ip firewall filter
add action=accept chain=forward log=yes log-prefix="NAT_INFO_FW> " src-address=172.16.0.0/16

Log Result (from different servers , so ip scheme may be changed in these logs, For example purposes)

In this log you can clearly see the src-dst address, and on which public ip request was natted along with ports. This is useful

Rules from LAB Router:

Mikrotik WAN IP’s (2 for test purposes):

  • 101.11.11.255/32
  • 101.11.11.253/32

PPPoE Users (2 for test)

  • 172.16.0.1
  • 172.16.0.2

REMOTE WEB SERVER (considering it’s a web server on internet which our user is accessing or doing illegal stuff)

  • 101.11.11.255

SRC-NAT Rules on MIKROTIK:

/ip firewall nat
add action=src-nat chain=srcnat protocol=tcp src-address=172.16.0.1 to-addresses=101.11.11.255 to-ports=10000-19999
add action=src-nat chain=srcnat protocol=udp src-address=172.16.0.1 to-addresses=101.11.11.255 to-ports=10000-19999
add action=src-nat chain=srcnat src-address=172.16.0.1 to-addresses=101.11.11.255
add action=src-nat chain=srcnat protocol=tcp src-address=172.16.0.2 to-addresses=101.11.11.253 to-ports=20000-29999
add action=src-nat chain=srcnat protocol=udp src-address=172.16.0.2 to-addresses=101.11.11.253 to-ports=20000-29999
add action=src-nat chain=srcnat src-address=172.16.0.2 to-addresses=101.11.11.253

cgnat-log-1.PNG

Result:

On internet web server, we see following

[101.11.11.255]:10133 - - [02/Jan/2020:15:44:37 +0500] "GET /? HTTP/1.1" 200 3138 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.88 Safari/537.36"

so the law enforcement agency come to us, and tell us that this is your public IP+Port  101.11.11.255:10133, now give us his details. And as we know that we are doing CGNAT, so we have to do little tracking.

On Mikrotik LOG we see following [after enabling LOGS,

cgnat-log-2.PNG

You can now see that our public IP having port 10133 was natted for our local user IP 172.16.0.1. with PPPoE it will show you the user name as well, so you can catch it right from here, or else if RADIUS is being used, you can track the IP via freeradius DB in radacct.


Scenario#2

OP have single public IP (e.g: 101.11.11.252) configured on Mikrotik WAN interface. End user subscriber is connected to mikrotik pppoe server using pppoe dialer. In this example we will be using 172.16.0.0/24 (256 users) and each user IP will be allowed to use 200 ports (200 ports per private IP).

This way when LAW will ask to provide details for 101.11.11.252:41636 , we can look into our LOGS (usually SYSLOG server either in linux, or using windows based SYSLOG like solarwinds syslog serveR) we can look into the 101.11.11.252:41636 & we can see the pppoe username or its private ip and search the ip in radius radacct table if radius is being used)

$addNatRules count=255 srcStart=172.16.0.1 toAddr=101.11.11.252 portStart=2000 portsPerAddr=200

Above CMD will create 765 rules (for 256 users) in IP / Firewall / NAT section. (make sure you have pasted the addNatRules function in the terminal before using above command.

– Enable mikrotik logs in Mikrotik LOG window

To enable LOGS in mikrotik LOG window , use

/ip firewall filter
add action=accept chain=forward log=yes log-prefix="NAT_INFO_FW> " src-address=172.16.0.0/16

– Enable mikrotik built in DISK base logging

To enable DISK base LOGGING in Mikrotik itself, (avoid this, it will OVERLOAD your routerboard which is not designed to handle such massive load of LOGS)

/system logging action
set 1 disk-file-count=25 disk-lines-per-file=5000
/system logging
add action=disk prefix=NAT_INFO_FW topics=info

– Enable remote SYSLOG logging in mikrotik

To ENABLE remote SYSLOG (I used Solarwind SYSLOG server on Windows in this example.

/system logging action
set 3 bsd-syslog=yes remote=10.0.0.2
/system logging
add action=remote prefix=NAT_INFO_FW topics=info

Now we can see in the LOG window (just an example, in actual you have to use some SYSLOG server) to search for 101.11.11.252:41636

Jan/03/2020 10:48:43 firewall,info NAT_INFO_FW> forward: in: out:ether1-wan, src-mac d0:bf:9c:f7:88:76, proto TCP (ACK), 172.16.0.199:54326->179.60.194.35:443, NAT (172.16.0.199:54326->101.11.11.252:41636)->179.60.194.35:443, len 40
Jan/03/2020 10:48:43 firewall,info NAT_INFO_FW> forward: in: out:ether1-wan, src-mac d0:bf:9c:f7:88:76, proto TCP (ACK), 172.16.0.199:54326->179.60.194.35:443, NAT (172.16.0.199:54326->101.11.11.252:41636)->179.60.194.35:443, len 52

& as you can see that 101.11.11.252:41636 was used private IP 172.16.0.199 & it will also show the <pppoe-zaib> . This way you can pull the user details & provide it to law enforcement agencies.

on windows base REMOTE syslog we can see the results, and can search easily as well.

solarwind syslog.png


To Delete older logs from syslog mysql DB

 mysql -uroot -pSQLPASSWORD -s -e "use syslog; DELETE FROM logs WHERE date(datetime) < (CURDATE() - INTERVAL 3 MONTH);"

TIPS for Linux base SYSLOG-NG trimming

I am using SYSLOG-NG to store all logs , to log only the NAT related queries (which actually shows the entries of public:port vs private ip:port use following in syslog ng configuration (before SOURCE section

######## Zaib Section Starts here
# Accept connection on UDP
source s_net { udp (); };
# MIKROTIK ########### add logs into files & in mysql dB as well.zaib
# Add Filter to add our mikrotik
filter f_mikrotik_1 { host("10.0.0.1") and match("NAT" value("MESSAGE")) };
#filter f_mikrotik_1 { host( "10.0.0.1" ); };
log { source ( s_net ); filter( f_mikrotik_1 ); destination ( df_mikrotik_1 ); };
destination df_mikrotik_1 {
file("/var/log/zlogs/${HOST}.${YEAR}.${MONTH}.${DAY}.log"
template-escape(no));
};

source s_mysql {
udp(port(514));
tcp(port(514));
};
# Play with below, some confusion here
destination d_mysql { pipe("/var/log/mysql.pipe" template("INSERT INTO
logs (host,facility,priority,level,tag,datetime,program,msg) VALUES
('$HOST','','','','','$YEAR-$MONTH-$DAY
$HOUR:$MIN:$SEC','','$MSG');\n") template-escape(yes)); };

log {
source(s_net);
filter(f_mikrotik_1);
destination(d_mysql);
};
####### #Zaib Section ends here

Note: For 500 active subscribers , the average log size on the syslog DB was 500 MB per day. This was after the controlled syslog entries (logging of requests that contains word NAT only).


Regard’s
~ Syed Jahanzaib ~

 

zaib_scattered

December 13, 2019

Mikrotik – Packet Chain Topology

Filed under: Mikrotik Related — Syed Jahanzaib / Pinochio~:) @ 9:36 AM

 


Firewall Mangle

Firewall mangle is used to mark IP packet. These marks are used by other router facilities like routing, firewall filter and bandwidth management to identified the packets. Moreover it also used to modify some fields in the IP header, like TOS (DSCP) and TTL fields. There are 5 default chain in firewall mangle.

INPUT

Packet that come into router will check with input chain. It is used to process packets entering the router. For example If we want to filter packet that telnet or ssh to router we need to use input chain in firewall filter.– Used to process packets entering the router through one of the interfaces with the destination IP address which is one of the router addresses. Chain input is useful for limiting the configuration access to Mikrotik Router.
or
– The connection that occurs from local to router and ends in router
example: use of the internal proxy (internal proxy right in the router)

PREROUTING

Prerouting is looked at before the router makes a routing decision. It’s happen before input chain. The packet that forward passthrough the router will match prerouting first.Pre-routing means that the connection will enter the router (no matter from where / depends on the settings mangle in interface later) . this connection will be in the process inside the router, can the process of bending to the external proxy, can filtering port, can anything, anyway there is a process, prerouting it marks the connection before the process occurs.

FORWARD

After prerouting, packet that passthrough router will process by forward chain. Used to process packets passing through the router. Example we want to block users to open facebook. We will use firewall forward chain to do it.– Used to process data packets through routers, connections that occur from the public to local
or
– The connection that occurs from local to public
with the provision that there is no process inside the router, it means that the connection is direct and only passing in the router does not happen a process inside the router.

POSTROUTING

Postrouting is a connection that will exit the router after a process occurs inside the router. It’s happen after forward. Postrouting is looked at after the router makes a routing decision.

OUTPUT

OUTPUT is used to process packets that originated from the router. Normally we rarely use this chain. Example we ping from router to Internet that’s output traffic.

or to process data packets originating from the router and left through one of the interfaces, connections that occur from the router to the public. It is used to process packets that originated from the router

============================================================================

Firewall Filter

Most of the time we use it for filter traffic simply say to protect our network from unauthorized user or bad guy. There are three default chain in firewall filter. It’s input, output and forward.

 

Does it matter where you mark?

Yes, because sometimes you might overmark and sometimes you might undermark. Choose the right place for the right classification/marking

 

December 5, 2019

Intervlan Routing with Mikrotik DHCP Option 121 & 249

Filed under: Mikrotik Related — Syed Jahanzaib / Pinochio~:) @ 12:18 PM

network.jpg

Disclaimer! This is important!

My humble request is that kindly donot consider me as an expert on this stuff, I am NOT certified in anything Mikrotik/Cisco/Linux or Windows. However I have worked with some core networks and I read , research & try stuff all of the time. When you are enslaved by private job & working as one man army, you have to perform many task in which you are not formally trained for. So I am not speaking/posting about stuff I am formerly trained in, I pretty much go with experience and what I have learned on my own. And , If I don’t know something then I read & learn all about it.

So , please don’t hold me/my-postings to be always 100 percent correct. I make mistakes just like everybody else. However – I do my best, learn from my mistakes and try to help others


This post is a sequel of Prevent Mikrotik from Chocking with Cisco Inter-Vlan Routing

If you are running mikrotik DHCP along with VLAN, & you want to get benefits of intervlan routing to avoid local traffic (same network but different subnets/vlans) hitting mikrotik by routing local resource requests through switch level directly. There are 2 options to sort this issue. Either provide the static routes for each vlan which becomes useless if you are using pppoe because as soon as client connected with pppoe, it will get preference over your existing default gateway thus all traffic even the local one will be passing via mikrotik resulting in excessive usage of mikrotik resources. This was discussed in details here.

Far better solution is to avoid providing any default routes to DHCP users but assign them CLASSLESS ROUTES for local resources via your DHCP options, either via Cisco or Mikrotik DHCP server. A classless route adds specified route in clients routing table

In this post we will discuss only about Mikrotik DHCP option 121/249.

  • 121 – Generally used for devices / Linux / Mikrotik etc. Option 121 is a RFC3442 *standard*.However, note that DHCP Option 121 is ignored by DHCP clients prior to Windows Vista. O
  • 249 – It is the pre-standard Microsoft implemented option for Classless Static Routes (CSRs). It was eventually standardized as option 121. I think 249 is for microsoft only.

Personnel Note: Don’t use both.  I think this causes routes to be ignored. If you need to support both older and newer OS, use the option 249.

Scenario:

We assume you have working DHCP for each vlan under Mikrotik.

  • Mikrotik interface VLAN-x IP : 192.168.100.1/24
  • Cisco switch Vlan-x IP : 192.168.100.2/24
  • Mikrotik DHCP assignment for VLAN-x: 192.168.100.10 – 192.168.100.255 [no dns, no gateway]
  • Local sharing servers: We have some sharing servers on different subnet on local switch port: 101.0.0.0/8

Now what we want is that our DHCP LAN client 192.168.100.0/24 can access 101.0.0.0/8 via switch intervlan routing directly [via switch vlan port 192.168.100.2]. To achieve this we will assign stateless routes using DHCP options.

Side Note: It strikes me that you have to use classful routes in the Classless Static Routes DHCP Option for older version of windows like XP/2000.


Step # 1

To transfer classless routes to the Mikrotik DHCP server, the option with code 121/249 is used First we need to convert IP to hexadecimal HEX code. If you are a beginner you can simply get the ready made code using this site.

https://ip-pro.eu/en/mikrotik_dhcp_option_121_generator

Enter the details as per your local network scheme

For single Subnet:

dhcp option hex code via web site.PNG

For multiple Subnet:

You can add ADD NEW ROW to add multiple subnet and gateways to get combined HEX value . Z

multiple subnet routes in single line.PNG

Or try

https://billing-beta.galaxy.net.pk/public/option121/


Step # 2

Now we got the HEX value & we can use it in mikrotik dhcp option 121-249 .

  • Goto IP / DHCP / OPTIONS / + Add new

dhcp options for 121-249.PNG

Note: Make two entries, one for 121 & second for 249. As shown above !

Step # 3

Now we will add the above option in OPTION SETS , we can include multiple options in the OPTION SETS window

  • Goto IP / DHCP / OPTION SETS / + Add new

dhcp options set 2.PNG

Step # 4

Now goto IP / DHCP / Double Click on required DHCP & under DHCP OPTION SETS, select the option set we created in Step # 4


Testing …

At client end, renew the IP and you will get all the routes you configured in above steps.

For single Subnet entry:

===========================================================================

IPv4 Route Table
===========================================================================
Active Routes:
Network Destination Netmask Gateway Interface Metric
101.0.0.0 255.0.0.0 192.168.100.2 192.168.100.254 11 ***************
127.0.0.0 255.0.0.0 On-link 127.0.0.1 306
127.0.0.1 255.255.255.255 On-link 127.0.0.1 306
127.255.255.255 255.255.255.255 On-link 127.0.0.1 306
192.168.100.0 255.255.255.0 On-link 192.168.100.254 266
192.168.100.254 255.255.255.255 On-link 192.168.100.254 266
192.168.100.255 255.255.255.255 On-link 192.168.100.254 266
224.0.0.0 240.0.0.0 On-link 127.0.0.1 306
224.0.0.0 240.0.0.0 On-link 192.168.100.254 266
255.255.255.255 255.255.255.255 On-link 127.0.0.1 306
255.255.255.255 255.255.255.255 On-link 192.168.100.254 266
===========================================================================
Persistent Routes:
None

IPv6 Route Table
===========================================================================
Active Routes:
If Metric Network Destination Gateway
1 306 ::1/128 On-link
1 306 ff00::/8 On-link
===========================================================================
Persistent Routes:
None

For Multiple Subnet entry:

===========================================================================

IPv4 Route Table
===========================================================================
Active Routes:
Network Destination Netmask Gateway Interface Metric
10.0.0.0 255.0.0.0 192.168.100.1 192.168.100.254 11 ***************
127.0.0.0 255.0.0.0 On-link 127.0.0.1 306
127.0.0.1 255.255.255.255 On-link 127.0.0.1 306
127.255.255.255 255.255.255.255 On-link 127.0.0.1 306
172.16.10.0 255.255.255.0 192.168.100.1 192.168.100.254 11 ***************
192.168.100.0 255.255.255.0 On-link 192.168.100.254 266
192.168.100.254 255.255.255.255 On-link 192.168.100.254 266
192.168.100.255 255.255.255.255 On-link 192.168.100.254 266
221.132.112.8 255.255.255.255 192.168.100.1 192.168.100.254 11 ***************
224.0.0.0 240.0.0.0 On-link 127.0.0.1 306
224.0.0.0 240.0.0.0 On-link 192.168.100.254 266
255.255.255.255 255.255.255.255 On-link 127.0.0.1 306
255.255.255.255 255.255.255.255 On-link 192.168.100.254 266
===========================================================================
Persistent Routes:
None

IPv6 Route Table
===========================================================================
Active Routes:
If Metric Network Destination Gateway
1 306 ::1/128 On-link
1 306 ff00::/8 On-link
===========================================================================
Persistent Routes:
None

Regard’s
Syed Jahanzaib

November 23, 2019

DMASOFTLAB Radius Manager – Adding custom attribute to facilitate Dynamic address list on Mikrotik

Filed under: Mikrotik Related, Radius Manager — Tags: , , — Syed Jahanzaib / Pinochio~:) @ 3:04 PM

Quick Recipe: If you are using DMA Radius Manager & want to assign Dynamic Address list to a service so that user can automatically be added to dynamic address list under NAS, you can do so by using custom RADIUS attributesunder services section

  • Login to Admin Panel
  • Goto Services
  • Click on your desired service example 4mb
  • Under `Custom RADIUS attributes` , add below attribute
Mikrotik-Address-List := 4mb

adding attribute in radius manager service

Save , & test any user authentication by CMD,

rmauth 127.0.0.1 test 1

freeradius attribute for dma radius manager test via cmd

 

on NAS dynamic address list will be created for each user of this service group. Late ryou can use this address list to mark connections / packets/ routing/queue etc.

4mb pppoe.PNG

Happy Attributing 😉


Regard’s
Jz.

August 7, 2019

Exploiting Mikrotik for Good ?

Filed under: Mikrotik Related — Tags: , , — Syed Jahanzaib / Pinochio~:) @ 11:30 AM

mikrotik exploit logo.png

Last updated: 7-AUGUST-2019 / 1400 hours

Note: Lot have been written on this vulnerability & this is not something NEW, but this vulnerability helped us in accessing one of our remote site old router with forgotten credentials.


In our remote branch network , we had installed one Mikrotik small box RB750 for branch to HO connectivity. This small unit was installed few years back & we never looked into it again. Recently we needed to make some configuration changes but following some bad practices we didn’t added this particular mikrotik in our centralized automated backup system & we didn’t noted down the credentials & IP configurations of its VPN settings after its deployment considering it less important.

Luckily it was running old firmware which was exploitable dueto to its critical WinBox vulnerability (CVE-2018-14847) which allows for arbitrary file read of plain text passwords.


Index of this post

  1. Vulnerable Versions
  2. Requirements
  3. Executing scripts in linux
  4. Results
  5. Securing older version with firewalling
  6. Securing the Mikrotik Router at basics

Vulnerable Versions

Versions affected:

  • Affected all bugfix releases from 6.30.1 to 6.40.7, fixed in 6.40.8 on 2018-Apr-23
  • Affected all current releases from 6.29 to 6.42, fixed in 6.42.1 on 2018-Apr-23
  • Affected all RC releases from 6.29rc1 to 6.43rc3, fixed in 6.43rc4 on on 2018-Apr-23

For more information see: https://blog.mikrotik.com/security/winbox-vulnerability.html

Using this exploit we were able to recover the password and after changes we upgraded it immediately.

We can use Windows or Linux to remotely exploit the older mikrotik firmware to query for all user accounts.


Requirements:

The scripts can be run using PYTHON version 3+ & I have uploaded the scripts @ my Google Drive.


Driving in Linux !

  • I have tested it with Ubuntu ver 12 & 16
sudo apt-get update
sudo apt-get install python3

Now extract scripts in any temp folder.

Executing the scripts …

Extract users details using the Remote Mikrotik IP address [default 8291 port]

python3 WinboxExploit.py 10.0.0.1

Extract users details using the Remote Mikrotik IP address [custom port]

python3 WinboxExploit.py 10.0.0.1 1122

Discover Mikrotik on the network

(it will scan the network for Mikrotik, may take some time, or you can press CTRL+C to exit)

python3 MACServerDiscover.py

 

Extract users details using the Remote Mikrotik MAC Address

python3 MACServerExploit.py e4:8d:8c:9a:ed:11

Results:

mikrotik winbox exploit results.PNG

 

If the firmware is latest or not exploitable, it will give error “Exploit failed


# Securing older version with firewalling

If you dont want to upgrade, than at least use firewall filter to secure older versions……

/ip firewall filter
add action=reject chain=input comment="block CVE-2018-14847 exploit by z@ib" content=user.dat
add action=drop chain=input content="user.dat"

# Securing the Mikrotik Router at basics

  • TOP OF THE LINE THING TO DO : apply port scanning filtering !
  • Remotely Accessible Router Services should be limited to few addresses/interfaces
  • Never use default ports for Winbox / SSH & other services
  • Change there ports number to preferably higher unused ports like 50000 or above or likewise
  • If not in use, Disable all services like FTP / SSH & others
  • Never use default usernames like ADMIN , disable or delete them, and make alternate admin accounts with difficult passwords

Disable following

  • MAC-telnet services
    /tool mac-server set allowed-interface-list=none
  • MAC-Winbox
    /tool mac-server mac-winbox set allowed-interface-list=none
  • MAC-Ping service
    /tool mac-server ping set enabled=no
  • MikroTik Neighbor discovery protocol
    /ip neighbor discovery-settings set discover-interface-list=none
    /ipv6 nd set [find] disabled=yes
  • DNS cache
    /ip dns set allow-remote-requests=no
  • Socks proxy
    /ip proxy set enabled=no
    /ip socks set enabled=no
  • UPNP service
    /ip upnp set enabled=no
  • MikroTik dynamic name service or ip cloud
    /ip cloud set ddns-enabled=no update-time=no
  • Enable More Secure SSH access
    /ip ssh set strong-crypto=yes

Regard’s
Syed Jahanzaib

May 12, 2019

Manipulate CDN Traffic with Mikrotik Mangle & Queue Tree

Filed under: Mikrotik Related — Tags: , , , — Syed Jahanzaib / Pinochio~:) @ 12:49 PM

ncdn_-_cdn.png

Control / Facilitate CDN traffic with

~ Mikrotik Router ~

 

Result

First some DRY theory !

CDNs replicate content in multiple places. There’s a better chance of content being closer to the user, with fewer hops, and content will run over a more friendly network. The general idea of a CDN is to deliver content as fast as possible to the user without compromising the user’s experience. Usually, a CDN have global location servers, called Point of Presence. These PoPs store data as cache. When a user requests for a website, the nearest PoP will handle the request using stored cache.

The BIG players such as Google in order to enhance user experience have tried to get as close to the user as possible by direct peering with the regional service providers and provide contents using CDN (Content delivery network) providers. Google is having its own CDN network branded as a service called Google Global Cache (GGC)

Nowadays all the major ISPs have CDN facility , which tremendously helps them to reduce burden on there internet feed. Without CDN, cost of real internet bandwidth will be a heavy burden for any OP. With CDN user will get better video streaming experience.

I know few ISP’s here in Karachi (& one particularly originated from Gulshan Area) which totally relies on CDN (more than 50-60% of there internet data is routing via CDN) , I have used one of them, there real internet speed is pathetic but if you browse YT/FB they works excellent.


Scenario:

Our upstream ISP have CDN server installed in there data center & traffic going to CDN have no limit. But we want to control the traffic as following

1 Mb package Users break up for bandwidth controlling …

  • 1mb internet bandwidth
  • 2mb CDN bandwidth

So if a user is surfing the internet he will get full 1mb internet speed, & if he uses the traffic going to YOUTUBE CDN ,  He will get additional 2 mb.

Using Mikrotik, we can achieve this task by using Firewall Mangle & Queues Tree. Same can be done with Mangle & PCQ base simple queues too. It’s a debatable topic on what to use, & Depends on the selection, mangle marking method would also be changed.

Every network is different so one configuration cannot fit all. Number of users & traffic volume plays vital role in selection of marking / queue type to use.

Choose the marking/queue type wisely to save your Mikrotik CPU from becoming Mr. SPIKY 🙂 YKWIM 😀

Disclaimer: This is just an example for sharing purposes ONLY & yes there are many other methods and tuning techniques you can adopt to make this process much more efficient.


Script !

/ip pool
add name=pppoe_pool ranges=172.16.99.1-172.16.99.255

/ip firewall address-list
add address=172.16.99.0/24 list=1mb
add address=58.27.130.0/24 list=cdn_list
# 58.27.130.0/24 is WATEEN telecom CDN server range. It may change over time or different for each ISP/Network. Pay attention to this part.

# MANGLE Rules orderwise placement is very important. Make sure to mark CDN (or requireD) traffic before other traffic

/ip firewall mangle
# Marking traffic going to CDN (or required sites/ips)
add action=mark-packet chain=forward src-address-list=1mb dst-address-list=cdn_list new-packet-mark=cdn_1mb_up passthrough=no
add action=mark-packet chain=forward src-address-list=cdn_list dst-address-list=1mb new-packet-mark=cdn_1mb_down passthrough=no
# Marking other traffic , normal traffic other then CDN
add action=mark-packet chain=forward src-address-list=1mb new-packet-mark=1mb_up passthrough=no
add action=mark-packet chain=forward dst-address-list=1mb new-packet-mark=1mb_down passthrough=no

# Adding PCQ base queue type for per user distribution

/queue type
add kind=pcq name=2mb-cdn-download pcq-classifier=dst-address pcq-dst-address6-mask=64 pcq-rate=2048k pcq-src-address6-mask=64
add kind=pcq name=2mb-cdn-upload pcq-classifier=src-address pcq-dst-address6-mask=64 pcq-rate=2048k pcq-src-address6-mask=64
add kind=pcq name=1mb-upload pcq-classifier=src-address pcq-dst-address6-mask=64 pcq-limit=50KiB pcq-rate=1100k pcq-src-address6-mask=64 pcq-total-limit=2000KiB
add kind=pcq name=1mb-download pcq-classifier=dst-address pcq-dst-address6-mask=64 pcq-limit=50KiB pcq-rate=1100k pcq-src-address6-mask=64 pcq-total-limit=2000KiB

# Adding QUEUE&nbsp; TREE to control bandwidth on per user basis policy using PCQ

/queue tree
# Adding 2mb QUEUE TREE for CDN traffic
add name="CDN - 1mb - upload" packet-mark=cdn_1mb_up parent=global priority=1 queue=2mb-cdn-upload
add name="CDN - 1mb - download" packet-mark=cdn_1mb_down parent=global priority=1 queue=2mb-cdn-download
# Adding 1mb QUEUE TREE for Normal traffic (Other then CDN)
add name=1mb-internet-up packet-mark=1mb_up parent=global queue=1mb-upload
add name=1mb-internet-down packet-mark=1mb_down parent=global queue=1mb-download

 

cdn and net queue tree

 

As shown in above example image, 1mb user is download contents from the internet at full 1 Mb speed (infact I add some extra liberty that is 1100 kb) and on the same side Youtube video (via CDN) is getting 2mb of bandwidth.

With the help of CDN, Real internet bandwidth is saved a lot, and available for other tasks/users,  thus providing relief to the real internet bandwidth pipe.


Personal experience regarding PCQ base queue – Simple Queue vs Tree !

In my personal experience, if you have hundreds of users, then Dynamic simple queues are better as there load spreads over multiple cores.

But for CDN we have to create separate queues for marked packets, therefore we have to use PCQ base queues , either in SIMPLE or TREE. In my experience I have seen that PCQ base queue with Queue Tree puts less load on the CPU.

 


Regard’s
Syed Jahanzaib

durood

April 5, 2019

Mikrotik with Freeradius/mySQL # Part-22 – Create Dynamic Address List using Mikrotik-Address-List Attribute

Filed under: freeradius, Mikrotik Related — Tags: — Syed Jahanzaib / Pinochio~:) @ 12:28 PM

fre



Disclaimer! This is important!

Every Network is different , so one solution cannot be applied to all. Therefore try to understand logic & create your own solution as per your network scenario. Just dont follow copy paste.

If anybody here thinks I am an expert on this stuff, I am NOT certified in anything Mikrotik/Cisco/Linux or Windows. However I have worked with some core networks and I read , research & try stuff all of the time. So I am not speaking/posting about stuff I am formerly trained in, I pretty much go with experience and what I have learned on my own. And , If I don’t know something then I read & learn all about it.

So , please don’t hold me/my-postings to be always 100 percent correct. I make mistakes just like everybody else. However – I do my best, learn from my mistakes and always try to help others.

Regard’s
Syed Jahanzaib~


Scenario:

  • We have FREERADIUS installed as a AAA system in Ubuntu 16.04 server
  • Mikrotik version 6.44 is acting as PPPoE NAS connected with radius for AAA

Requirement:

When any user connects with our NAS, he should be added to mikrotik dynamic address list under IP > firewall > address list, so that we can manipulate this address list for different tasks, example mark connections/packets/routing and use them in Queues / Routes section or perform different sort of filtering as required.

In this particular task we are dynamically adding user in particular address list using radius attributes, then using this address list packet marking is being made, and then in Queues we are using these marked packets for different sort of bandwidth policies, example for normal internet we will limit 1mb per user , and for CDN traffic we will add addition 2mb for YT & FB. [and vice versa for different packages accordingly]

 


Solution:

We will use Mikrotik-Address-List attribute in radgroupreply section. as shown here.

1# Adding User entry in RADCHECK table so user can authenticate …

mysql> select * from radcheck;
+----+----------+--------------------+----+-------------------+
| id | username | attribute | op | value |
+----+----------+--------------------+----+-------------------+
| 1 | zaib | Cleartext-Password | := | zaib |
+----+----------+--------------------+----+-------------------+
1 rows in set (0.01 sec)

2# Adding Radius Group Reply for 1mb Group, Example 1mb group user will get 1mb dynamic queue plus they will be added dynamically in address list name 1mb

mysql> select * from radgroupreply;
+----+-----------+-----------------------+----+--------------+
| id | groupname | attribute | op | value |
+----+-----------+-----------------------+----+--------------+
| 21 | 1mb | Mikrotik-Rate-Limit | == | 1024k/1024k |
| 22 | 1mb | Mikrotik-Address-List | := | 1mb |
+----+-----------+-----------------------+----+--------------+
2 rows in set (0.00 sec)

2# Adding username ZAIB in the Radius user group & assign him 1mb Group.

 

mysql> select * from radusergroup;
+----+----------+-----------+----------+
| id | username | groupname | priority |
+----+----------+-----------+----------+
| 5 | zaib | 1mb | 1 |
+----+----------+-----------+----------+
1 row in set (0.00 sec)

RADTEST:

Now we will test user via RADTEST cmd …


radtest zaib zaib localhost 1812 testing123

Result:

Sending Access-Request of id 130 to 127.0.0.1 port 1812
User-Name = "zaib"
User-Password = "zaib"
NAS-IP-Address = 101.11.11.254
NAS-Port = 1812
Message-Authenticator = 0x00000000000000000000000000000000
rad_recv: Access-Accept packet from host 127.0.0.1 port 1812, id=130, length=50

Mikrotik-Rate-Limit = "1024k/1024k"
Mikrotik-Address-List = "1mb"

Freeradius Debug Result:

Sending Access-Accept of id 156 to 127.0.0.1 port 34563
Mikrotik-Rate-Limit == "1024k/1024k"
Mikrotik-Address-List := "1mb"
Finished request 32.

Now try to connect user from your user device, & upon connection you will see new address list entry for this user IP ..

& its 1mb queues have been created as well


# Mikrotik Mangling & Queueing Section !

Now we will move towards Mikrotik related configuration for mangling & queue. in above steps we added DYNAMIC queue for test purposes, & as we will be using simple queues therefore we need to remove the dynamic queue, Do so , then we will move further …

  • Marking upload & download separately for 1mb user address list …

/ip firewall mangle
add action=mark-packet chain=forward comment="1mb users UPLOAD" new-packet-mark=1mb_users_up src-address-list=1mb passthrough=no
add action=mark-packet chain=forward comment="1mb users DOWNLOAD" dst-address-list=1mb new-packet-mark=1mb_users_down passthrough=no
  • Creating PCQ base 1mb download/upload limit variable …
/queue type
add kind=pcq name=download-1mb pcq-classifier=dst-address pcq-dst-address6-mask=64 pcq-rate=1024k pcq-src-address6-mask=64
add kind=pcq name=upload-1mb pcq-classifier=src-address pcq-dst-address6-mask=64 pcq-rate=1024k pcq-src-address6-mask=64
  • Creating PCQ base simple Queues to actual limit each user with 1mb download/upload …
/queue simple
add name="1mb user DOWN - PCQ" packet-marks=1mb_users_down queue=upload-1mb/download-1mb target=""
add name="1mb user UP - PCQ" packet-marks=1mb_users_up queue=128k-per-user/128k-per-user target=""

 

PC#1

1st- user - 1mb user test

PC#2

2nd pc 128K 1mb


TIPS:

How to remove all dynamic queues [can be used in script login section]

dynamic queue removal.PNG

/queue simple remove [find where dynamic]

Conclusion:

As we can see that address list have been created successfully, now we can manipulate it for our different tasks using marked packets for customized PCQ base queues for policy base queueing.

I will write more on it later if manage to get some spare time.


 

Regard’s
Syed Jahanzaib

 

Older Posts »

%d bloggers like this: