Syed Jahanzaib Personal Blog to Share Knowledge !

January 8, 2020

Syslog-ng – Part 3: Minimized logging to mysql with dynamic tables & trimming

Filed under: Linux Related, Mikrotik Related — Tags: , — Syed Jahanzaib / Pinochio~:) @ 1:27 PM

syslog cgnat

Revision: 7th-JAN-2020


In continuation to existing posts related to syslog-ng, Following post illustrates on how you can log only particular messages with pattern matching and let syslog-ng creates dynamic table based on the dates so that searching/querying becomes easy.

This task was required in relation to CGNAT logging. you may want to read it here

https://aacable.wordpress.com/2020/01/01/mikrotik-cgnat/

Hardware Software used in this post:

  • Mikrotik Routerboard – firmware 6.46.1
  • Ubuntu 18.4 Server x64 along with syslog-ng version 3.25.1 on some decent hardware

Requirements:

I assume that you already have working setup for Syslog-ng& your remote devices are already sending logs and they are storing in the mysql already. See Part 1 & 2


Ref: Installing latest version of syslog-ng

#Make sure to change the version, I have used this CMD on Ubuntu 16.04 , for version 18, you may change this to 18.04

wget -qO - http://download.opensuse.org/repositories/home:/laszlo_budai:/syslog-ng/xUbuntu_16.04/Release.key | sudo apt-key add -
touch /etc/apt/sources.list.d/syslog-ng-obs.list
echo "deb http://download.opensuse.org/repositories/home:/laszlo_budai:/syslog-ng/xUbuntu_16.04 ./" > /etc/apt/sources.list.d/syslog-ng-obs.list
apt-get update
apt-get install syslog-ng

at my system I got this

syslog-ng 3 (3.25.1)
Config version: 3.25
Installer-Version: 3.25.1
Revision: 3.25.1-1
Compile-Date: Dec 12 2019 12:00:29
Module-Directory: /usr/lib/syslog-ng/3.25
Module-Path: /usr/lib/syslog-ng/3.25
Include-Path: /usr/share/syslog-ng/include
Error opening plugin module; module='mod-java', error='libjvm.so: cannot open shared object file: No such file or directory'
Available-Modules: add-contextual-data,afsmtp,tfgetent,afsql,cryptofuncs,http,confgen,sdjournal,system-source,cef,syslogformat,json-plugin,afprog,riemann,csvparser,affile,afsocket,afamqp,redis,examples,disk-buffer,xml,linux-kmsg-format,map-value-pairs,hook-commands,kafka,tags-parser,dbparser,graphite,appmodel,afstomp,pacctformat,afmongodb,pseudofile,basicfuncs,geoip2-plugin,kvformat,stardate,timestamp,mod-python,afuser,snmptrapd-parser
Enable-Debug: off
Enable-GProf: off
Enable-Memtrace: off
Enable-IPv6: on
Enable-Spoof-Source: on
Enable-TCP-Wrapper: on
Enable-Linux-Caps: on
Enable-Systemd: on

Status:

root@agpis-linux-test:/var/log/zlogs# service syslog-ng status
â syslog-ng.service - System Logger Daemon
Loaded: loaded (/lib/systemd/system/syslog-ng.service; enabled; vendor preset: enabled)
Active: active (running) since Wed 2020-01-08 12:46:52 PKT; 27min ago
Docs: man:syslog-ng(8)
Main PID: 7086 (syslog-ng)
Tasks: 2 (limit: 2290)
CGroup: /system.slice/syslog-ng.service
ââ7086 /usr/sbin/syslog-ng -F

Jan 08 12:46:52 agpis-linux-test systemd[1]: Starting System Logger Daemon...
Jan 08 12:46:52 agpis-linux-test syslog-ng[7086]: [2020-01-08T12:46:52.362728] Macro escaping can only be specified for inline templates;
Jan 08 12:46:52 agpis-linux-test syslog-ng[7086]: [2020-01-08T12:46:52.364052] WARNING: With use-dns(no), dns-cache() will be forced to 'no' too!;
Jan 08 12:46:52 agpis-linux-test systemd[1]: Started System Logger Daemon.

Create Database in mySQL to store dynamic tables

Create Base Database for storing dynamically created date wise tables

mysql -uroot -pXXX -e "create database syslog;"

Now edit the syslog-ng file

nano /etc/syslog-ng/syslog-ng.conf

& use following as sample. I would recommend that you should add only relevant part, just dont do blind copy paste. This is just sample for demonstration purposes only …

 

Syslog-ng Sample File

@version: 3.25
@include "scl.conf"
# Syslog-ng CUSTOMIZED configuration  file
# Syed Jahanzaib / aacable at hotmail dot com /https://aacable.wordpress.com
# First, set some global options.
options { chain_hostnames(off); flush_lines(0); use_dns(no); use_fqdn(no);
owner("root"); group("adm"); perm(0640); stats_freq(0);
bad_hostname("^gconfd$");
};

######## Zaib Section Starts here
# Accept connection on UDP
source s_net { udp (); };

# Adding filter for our Mikrotik Routerboard, store logs in FILE as primary
# MIKROTIK ###########

# This entry will LOG all information coming from this IP
filter f_mikrotik_252 { host("101.11.11.252"); };
# This entry will LOG ONLY particular message that contains word NAT, useful to minimize CGNAT logging, Enable one entry at a time # ZAIB
#filter f_mikrotik_252 { host("101.11.11.252") and match("NAT" value("MESSAGE")) };
log { source ( s_net ); filter( f_mikrotik_252 ); destination ( df_mikrotik_252 ); };
# add info in LOG (Part1)
destination df_mikrotik_252 {
file("/var/log/zlogs/${HOST}.${YEAR}.${MONTH}.${DAY}.log"
template-escape(no));
};
source s_mysql {
udp(port(514));
tcp(port(514));
};

# Store Logs in MYSQL DB as secondary # add info in MYSQL (Part2)
destination d_mysql {
sql(type(mysql)
host("localhost")
# MAKE SURE TO CHANGE CREDENTIALS
username("root")
password("XXXXXXXX")
database("syslog")
table("${R_YEAR}_${R_MONTH}_${R_DAY}")
columns( "id int(11) unsigned not null auto_increment primary key", "host varchar(40) not null", "date datetime", "message text not null")
values("0", "$FULLHOST", "$R_YEAR-$R_MONTH-$R_DAY $R_HOUR:$R_MIN:$R_SEC", "$MSG")
indexes("id"));
};
log {
source(s_net);
filter(f_mikrotik_252);
destination(d_mysql);
};
########################
# Sources
########################
# This is the default behavior of sysklogd package
# Logs may come from unix stream, but not from another machine.
#
source s_src {
system();
internal();
};

IMPORTANT:

Create ‘zlogs‘ folder in /var/log , so that mikrotik logs will be saved in separate file.

mkdir /var/log/zlogs

Mikrotik rule to LOG Forward chain

/system logging action

set 1 disk-file-count=50 disk-lines-per-file=5000
set 3 remote=101.11.11.254

/system logging add action=remote topics=info

/system logging action set 3 remote=101.11.11.254

Restart Syslog-ng server

Now restart syslog-ng service

service syslog-ng restart

and you will see the dynamic tables created as follows

mysql -uroot -pXXXXX
mysql: [Warning] Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 411
Server version: 5.7.28-0ubuntu0.18.04.4-log (Ubuntu)
Copyright (c) 2000, 2019, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> use syslog;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Database changed
mysql> show tables;
+------------------+
| Tables_in_syslog |
+------------------+
| 2020_01_08 |
+------------------+
1 row in set (0.00 sec)

mysql> describe 2020_01_08;
+---------+------------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+---------+------------------+------+-----+---------+----------------+
| id | int(11) unsigned | NO | PRI | NULL | auto_increment |
| host | varchar(40) | NO | | NULL | |
| date | datetime | YES | | NULL | |
| message | text | NO | | NULL | |
+---------+------------------+------+-----+---------+----------------+
4 rows in set (0.00 sec)

& you can then see data insertion into the table as soon LOG is received from remote devices

2020-01-08T07:49:43.020811Z 430 Query INSERT INTO 2020_01_08 (id, host, date, message) VALUES ('0', '101.11.11.252', '2020-01-08 12:49:28', 'forward: in: out:ether1-agp-wan, src-mac d0:bf:9c:f7:88:76, proto TCP (ACK,PSH), 172.16.0.2:57193->172.217.19.174:443, NAT (172.16.0.2:57193->101.11.11.252:2244)->172.217.19.174:443, len 79')
2020-01-08T07:49:43.031281Z 430 Query INSERT INTO 2020_01_08 (id, host, date, message) VALUES ('0', '101.11.11.252', '2020-01-08 12:49:28', 'forward: in: out:ether1-agp-wan, src-mac d0:bf:9c:f7:88:76, proto TCP (ACK,FIN), 172.16.0.2:57096->3.228.94.102:443, NAT (172.16.0.2:57096->101.11.11.252:2219)->3.228.94.102:443, len 40')
2020-01-08T07:49:43.041420Z 430 Query INSERT INTO 2020_01_08 (id, host, date, message) VALUES ('0', '101.11.11.252', '2020-01-08 12:49:38', 'forward: in: out:ether1-agp-wan, src-mac d0:bf:9c:f7:88:76, proto UDP, 172.16.0.2:49247->216.58.208.234:443, NAT (172.16.0.2:49247->101.11.11.252:2202)->216.58.208.234:443, len 1378')
2020-01-08T07:49:43.051112Z 430 Query INSERT INTO 2020_01_08 (id, host, date, message) VALUES ('0', '101.11.11.252', '2020-01-08 12:49:38', 'forward: in: out:ether1-agp-wan, src-mac d0:bf:9c:f7:88:76, proto UDP, 172.16.0.2:49247->216.58.208.234:443, NAT (172.16.0.2:49247->101.11.11.252:2202)->216.58.208.234:443, len 1378')
2020-01-08T07:49:43.061280Z 430 Query INSERT INTO 2020_01_08 (id, host, date, message) VALUES ('0', '101.11.11.252', '2020-01-08 12:49:39', 'forward: in: out:ether1-agp-wan, src-mac d0:bf:9c:f7:88:76, proto UDP, 172.16.0.2:49760->172.217.19.1:443, NAT (172.16.0.2:49760->101.11.11.252:2202)->172.217.19.1:443, len 1378')
2020-01-08T07:49:43.071449Z 430 Query INSERT INTO 2020_01_08 (id, host, date, message) VALUES ('0', '101.11.11.252', '2020-01-08 12:49:39', 'forward: in: out:ether1-agp-wan, src-mac d0:bf:9c:f7:88:76, proto UDP, 172.16.0.2:49760->172.217.19.1:443, NAT (172.16.0.2:49760->101.11.11.252:2202)->172.217.19.1:443, len 1378')
2020-01-08T07:49:44.828993Z 430 Query INSERT INTO 2020_01_08 (id, host, date, message) VALUES ('0', '101.11.11.252', '2020-01-08 12:49:44', 'forward: in: out:ether1-agp-wan, src-mac d0:bf:9c:f7:88:76, proto UDP, 172.16.0.2:53503->216.58.208.234:443, NAT (172.16.0.2:53503->101.11.11.252:2203)->216.58.208.234:443, len 827')
2020-01-08T07:49:44.851034Z 430 Query INSERT INTO 2020_01_08 (id, host, date, message) VALUES ('0', '101.11.11.252', '2020-01-08 12:49:44', 'forward: in: out:ether1-agp-wan, src-mac d0:bf:9c:f7:88:76, proto UDP, 172.16.0.2:53503->216.58.208.234:443, NAT (172.16.0.2:53503->101.11.11.252:2203)->216.58.208.234:443, len 827')
2020-01-08T07:51:37.518276Z 430 Query INSERT INTO 2020_01_08 (id, host, date, message) VALUES ('0', '101.11.11.252', '2020-01-08 12:51:37', 'forward: in: out:ether1-agp-wan, src-mac d0:bf:9c:f7:88:76, proto TCP (ACK), 172.16.0.2:57202->91.195.240.126:80, NAT (172.16.0.2:57202->101.11.11.252:2260)->91.195.240.126:80, len 41')
2020-01-08T07:51:37.522015Z 430 Query INSERT INTO 2020_01_08 (id, host, date, message) VALUES ('0', '101.11.11.252', '2020-01-08 12:51:37', 'forward: in: out:ether1-agp-wan, src-mac d0:bf:9c:f7:88:76, proto TCP (ACK), 172.16.0.2:57202->91.195.240.126:80, NAT (172.16.0.2:57202->101.11.11.252:2260)->91.195.240.126:80, len 41')

syslog-ng dynamic table data from phpmyadmin.PNG


Regard’s
Syed Jahanzaib

January 1, 2020

CGNAT Deployment using Mikrotik RouterOS

Filed under: Mikrotik Related — Tags: , , , , , , , — Syed Jahanzaib / Pinochio~:) @ 5:34 PM

mdnet cgnat conept.PNG

Incomplete Post, this contains only src-nat part, second method of NETMAP will  be added soon which is far more simple & efficient as compared to the src-nat method

My humble request, Kindly donot consider me as an expert on this stuff, I am NOT certified in anything Mikrotik/Cisco/Linux or Windows. However I have worked with some networks and I read , research & try stuff all of the time. When you are enslaved by private job & working as one man army, you have to perform many task in which you are not formally trained for. So I am not speaking/posting about stuff I am formerly trained in, I pretty much go with experience and what I have learned on my own. And , If I don’t know something then I read & learn all about it.

So , please don’t hold me/my-postings to be always 100 percent correct. I make mistakes just like everybody else. However – I do my best, learn from my mistakes and try to help others

Some references used in this post


*CG-NAT* as Workaround:

CGNAT concept is used to share one or preferably more public IP addresses with large number of private ip addresses on ratio basis.CGNAT/NAT444 is a conception, not a function. In terms of RouterOS functionality it’s simple SRC NAT rule.

To combat with this IPV4 exhausting issue, we can use CGNAT as a workaround. This is by no means a solution, & the OP should get public IP space (either ipv4 or ipv6) to comply with the LAW.

Some possible disadvantages of using CGNAT concept:

  • CGNAT is not sustainable in the long term, hectic to manage the private/public pools especially if you have multiple NASes doing same job
  • ISP deploying IP address sharing techniques should also deploy a corresponding logging architecture to maintain records of the relation between a customer’s identity and IP/port resources utilized
  • You should deploy additional SYSLOG server (either windows or linux base) to store logs. I would prefer linux base SYSLOG-NG). Tracking of users for legal reasons means searching hundreds GB’s of logging would be required, as multiple end users go behind one (or more) public IP address(es)
  • Tracking Logs is not an easy task particularly when you have tongs of Logging (in a DB).Logging every NAT translation is resource consuming. Some super fast computing resources (including preferably RAID10 or SSD based storage) and fine tune DB would be required
  • A CG-NAT device must use the same external IP address mapping
    for all sessions associated with the same internal IP address
  • Most Applications do not behave well with TCP resets
  • Many operators are still not familiar with CG-NAT complexities. There is a lot of trial and error on the part of ISP’s

 

in my personal experience , Deployment is somewhat hectic, & tracking any request is daunting task ! z@ib


Hardware/Software Used in this post:


CGNAT logging to remote syslog server with some customization

https://aacable.wordpress.com/2020/01/08/syslog-ng-part-3-minimized-logging-to-mysql-with-dynamic-tables-trimming/


Scenario#1

OP is running mini ISP with around 200 active subscribers. Mikrotik Router is being used as PPPoE Server along with Freeradius as AAA. On Mikrotik, one public IP is configured for WAN and additional /24 routed pool (256 public IP addresses) is provided to the OP via ISP so that he can provide public IP to each user. After the network upgrades , OP have reached 700 users in total, and since he have only 256 public ip’s , he is now using natting for half of his users.

We all know that IPV4 shortage is on peak , getting ipv4 is expensive for 3rd world countries & small ISP’s as well.

This NATTING workaround is creating hurdles in tracking illegal activity performed by any NATTED users because hundreds of NATTED user will have same public ip (Mikrotik WAN IP). nowadays law sometimes provide only the public ip along with source port and ask for the user credentials details for investigation purposes.

with single public IP and hundreds of natted hosts behind it. tracking is nearly impossible.


IP scheme example used in this Scenario#1:

Public IP range: (/24 public IP’s routed pool)

  • 1.1.1-1.1.1.255
  • Total Public IP useable: 255

Private IP range for PPPoE users:

  • 172.16.1.1-172.16.1.255
  • 172.16.2.1-172.16.2.255
  • 172.16.3.1-172.16.3.255
  • Total Private IP useable: 765

For 765 Users, we will be using 1:5 Ratio, thus 153 public ips will be used for 765 users. (on a ratio of 1:5).

  • per private IP, we will reserve 10,000 ports, which should be more than enough for each user.
  • per private IP, we will be creating 3 rules, one for TCP, second for UDP, 3rd for non ports range

CGNAT configuration on RouterOS is very much similar to regular source NAT configuration.


To add multiple Public IP addresses on WAN interface in bulk using single CMD on Terminal

You may need to add all of your public IP addresses (which will be used for CGNAT) on WAN interface(required fro troubleshooting purposes as well).

To add ips in bulk using single CMD, you can use Mikrotik FOR X script function for ease / ZAIB

:for x from 1 to 153 do={ /ip address add address="1.1.1.$x/32" comment="1.1.1.$x - Routed IP for ppp CGNAT - zaib" interface="ether1-wan"}

 


Adding FUNCTION in Mikrotik for later Automation

Paste this in Mikrotik RouterOS terminal:

# CGNAT Customized minimalistic Script to add function.
# Syed Jahanzaib / aacable at hotmail dot com
:global sqrt
:global sqrt do={
:for i from=0 to=$1 do={
:if (i * i > $1) do={ :return ($i - 1) }
}
}
:global addNatRules do={
:local x [$sqrt $count]
:local y $x
:if ($x * $x = $count) do={ :set y ($x + 1) }
:for i from=0 to=($count - 1) do={
:local prange "$($portStart + ($i * $portsPerAddr))-$($portStart + (($i + 1) * $portsPerAddr) - 1)"
/ip firewall nat add chain=srcnat action=src-nat protocol=tcp src-address=($srcStart + $i) to-address=$toAddr to-ports=$prange
/ip firewall nat add chain=srcnat action=src-nat protocol=udp src-address=($srcStart + $i) to-address=$toAddr to-ports=$prange
/ip firewall nat add chain=srcnat action=src-nat src-address=($srcStart + $i) to-address=$toAddr
}
}

Now we have function inserted with the help of above code, and using this function, we can create rules in bulk using following CMD to add rules in NAT section

# per private IP, we will reserve 10000 ports, which should be more than enough for each user.
# per private IP, we will be creating 3 rules, one for TCP, second for UDP, 3rd for non ports range

$addNatRules count=5 srcStart=172.16.1.1 toAddr=1.1.1.1 portStart=10000 portsPerAddr=9999
$addNatRules count=5 srcStart=172.16.1.6 toAddr=1.1.1.2 portStart=10000 portsPerAddr=9999
$addNatRules count=5 srcStart=172.16.1.11 toAddr=1.1.1.3 portStart=10000 portsPerAddr=9999
$addNatRules count=5 srcStart=172.16.1.16 toAddr=1.1.1.4 portStart=10000 portsPerAddr=9999
$addNatRules count=5 srcStart=172.16.1.21 toAddr=1.1.1.5 portStart=10000 portsPerAddr=9999
$addNatRules count=5 srcStart=172.16.1.26 toAddr=1.1.1.6 portStart=10000 portsPerAddr=9999
$addNatRules count=5 srcStart=172.16.1.31 toAddr=1.1.1.7 portStart=10000 portsPerAddr=9999
$addNatRules count=5 srcStart=172.16.1.36 toAddr=1.1.1.8 portStart=10000 portsPerAddr=9999
$addNatRules count=5 srcStart=172.16.1.41 toAddr=1.1.1.9 portStart=10000 portsPerAddr=9999
$addNatRules count=5 srcStart=172.16.1.46 toAddr=1.1.1.10 portStart=10000 portsPerAddr=9999

# & so on for rest of the pool, you can further automate this by using additional functions & scripting

Enable Logging of CG-NAT Output:

# To log user IP/NAT information on LOG window / you can configure remote log to syslog-server too
/ip firewall filter
add action=accept chain=forward log=yes log-prefix="NAT_INFO_FW> " src-address=172.16.0.0/16

Log Result (from different servers , so ip scheme may be changed in these logs, For example purposes)

In this log you can clearly see the src-dst address, and on which public ip request was natted along with ports. This is useful

Rules from LAB Router:

Mikrotik WAN IP’s (2 for test purposes):

  • 101.11.11.255/32
  • 101.11.11.253/32

PPPoE Users (2 for test)

  • 172.16.0.1
  • 172.16.0.2

REMOTE WEB SERVER (considering it’s a web server on internet which our user is accessing or doing illegal stuff)

  • 101.11.11.255

SRC-NAT Rules on MIKROTIK:

/ip firewall nat
add action=src-nat chain=srcnat protocol=tcp src-address=172.16.0.1 to-addresses=101.11.11.255 to-ports=10000-19999
add action=src-nat chain=srcnat protocol=udp src-address=172.16.0.1 to-addresses=101.11.11.255 to-ports=10000-19999
add action=src-nat chain=srcnat src-address=172.16.0.1 to-addresses=101.11.11.255
add action=src-nat chain=srcnat protocol=tcp src-address=172.16.0.2 to-addresses=101.11.11.253 to-ports=20000-29999
add action=src-nat chain=srcnat protocol=udp src-address=172.16.0.2 to-addresses=101.11.11.253 to-ports=20000-29999
add action=src-nat chain=srcnat src-address=172.16.0.2 to-addresses=101.11.11.253

cgnat-log-1.PNG

Result:

On internet web server, we see following

[101.11.11.255]:10133 - - [02/Jan/2020:15:44:37 +0500] "GET /? HTTP/1.1" 200 3138 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.88 Safari/537.36"

so the law enforcement agency come to us, and tell us that this is your public IP+Port  101.11.11.255:10133, now give us his details. And as we know that we are doing CGNAT, so we have to do little tracking.

On Mikrotik LOG we see following [after enabling LOGS,

cgnat-log-2.PNG

You can now see that our public IP having port 10133 was natted for our local user IP 172.16.0.1. with PPPoE it will show you the user name as well, so you can catch it right from here, or else if RADIUS is being used, you can track the IP via freeradius DB in radacct.


Scenario#2

OP have single public IP (e.g: 101.11.11.252) configured on Mikrotik WAN interface. End user subscriber is connected to mikrotik pppoe server using pppoe dialer. In this example we will be using 172.16.0.0/24 (256 users) and each user IP will be allowed to use 200 ports (200 ports per private IP).

This way when LAW will ask to provide details for 101.11.11.252:41636 , we can look into our LOGS (usually SYSLOG server either in linux, or using windows based SYSLOG like solarwinds syslog serveR) we can look into the 101.11.11.252:41636 & we can see the pppoe username or its private ip and search the ip in radius radacct table if radius is being used)

$addNatRules count=255 srcStart=172.16.0.1 toAddr=101.11.11.252 portStart=2000 portsPerAddr=200

Above CMD will create 765 rules (for 256 users) in IP / Firewall / NAT section. (make sure you have pasted the addNatRules function in the terminal before using above command.

– Enable mikrotik logs in Mikrotik LOG window

To enable LOGS in mikrotik LOG window , use

/ip firewall filter
add action=accept chain=forward log=yes log-prefix="NAT_INFO_FW> " src-address=172.16.0.0/16

– Enable mikrotik built in DISK base logging

To enable DISK base LOGGING in Mikrotik itself, (avoid this, it will OVERLOAD your routerboard which is not designed to handle such massive load of LOGS)

/system logging action
set 1 disk-file-count=25 disk-lines-per-file=5000
/system logging
add action=disk prefix=NAT_INFO_FW topics=info

– Enable remote SYSLOG logging in mikrotik

To ENABLE remote SYSLOG (I used Solarwind SYSLOG server on Windows in this example.

/system logging action
set 3 bsd-syslog=yes remote=10.0.0.2
/system logging
add action=remote prefix=NAT_INFO_FW topics=info

Now we can see in the LOG window (just an example, in actual you have to use some SYSLOG server) to search for 101.11.11.252:41636

Jan/03/2020 10:48:43 firewall,info NAT_INFO_FW> forward: in: out:ether1-wan, src-mac d0:bf:9c:f7:88:76, proto TCP (ACK), 172.16.0.199:54326->179.60.194.35:443, NAT (172.16.0.199:54326->101.11.11.252:41636)->179.60.194.35:443, len 40
Jan/03/2020 10:48:43 firewall,info NAT_INFO_FW> forward: in: out:ether1-wan, src-mac d0:bf:9c:f7:88:76, proto TCP (ACK), 172.16.0.199:54326->179.60.194.35:443, NAT (172.16.0.199:54326->101.11.11.252:41636)->179.60.194.35:443, len 52

& as you can see that 101.11.11.252:41636 was used private IP 172.16.0.199 & it will also show the <pppoe-zaib> . This way you can pull the user details & provide it to law enforcement agencies.

on windows base REMOTE syslog we can see the results, and can search easily as well.

solarwind syslog.png


To Delete older logs from syslog mysql DB

 mysql -uroot -pSQLPASSWORD -s -e "use syslog; DELETE FROM logs WHERE date(datetime) < (CURDATE() - INTERVAL 3 MONTH);"

TIPS for Linux base SYSLOG-NG trimming

I am using SYSLOG-NG to store all logs , to log only the NAT related queries (which actually shows the entries of public:port vs private ip:port use following in syslog ng configuration (before SOURCE section

######## Zaib Section Starts here
# Accept connection on UDP
source s_net { udp (); };
# MIKROTIK ########### add logs into files & in mysql dB as well.zaib
# Add Filter to add our mikrotik
filter f_mikrotik_1 { host("10.0.0.1") and match("NAT" value("MESSAGE")) };
#filter f_mikrotik_1 { host( "10.0.0.1" ); };
log { source ( s_net ); filter( f_mikrotik_1 ); destination ( df_mikrotik_1 ); };
destination df_mikrotik_1 {
file("/var/log/zlogs/${HOST}.${YEAR}.${MONTH}.${DAY}.log"
template-escape(no));
};

source s_mysql {
udp(port(514));
tcp(port(514));
};
# Play with below, some confusion here
destination d_mysql { pipe("/var/log/mysql.pipe" template("INSERT INTO
logs (host,facility,priority,level,tag,datetime,program,msg) VALUES
('$HOST','','','','','$YEAR-$MONTH-$DAY
$HOUR:$MIN:$SEC','','$MSG');\n") template-escape(yes)); };

log {
source(s_net);
filter(f_mikrotik_1);
destination(d_mysql);
};
####### #Zaib Section ends here

Note: For 500 active subscribers , the avg logs size on the syslog DB was 240 MB. This was after the controlled syslog entries (logging of requests that contains word NAT only).


Regard’s
~ Syed Jahanzaib ~

 

.

December 16, 2019

TACACS+ Cisco centralized authentication server

Filed under: Cisco Related — Tags: , , — Syed Jahanzaib / Pinochio~:) @ 3:29 PM

tacacs plus server.png

Disclaimer:

My humble request, Kindly donot consider me as an expert on this stuff, I am NOT certified in anything Mikrotik/Cisco/Linux or Windows. However I have worked with some core networks and I read , research & try stuff all of the time. When you are enslaved by private job & working as one man army, you have to perform many task in which you are not formally trained for. So I am not speaking/posting about stuff I am formerly trained in, I pretty much go with experience and what I have learned on my own. And , If I don’t know something then I read & learn all about it.

So , please don’t hold me/my-postings to be always 100 percent correct. I make mistakes just like everybody else. However – I do my best, learn from my mistakes and try to help others


TACACS+

Terminal Access Controller Access-Control System (TACACS, usually pronounced like tack-axe) refers to a authentication / authorization related services for cisco switches/routers/firewalls access control through a centralized server. With the help of Tacacs+ you can set up a much more granular level access for the users, groups, subnets or device type etc. Example which user can issue which commands on switches etc.

 

central auth server.jpeg

 

Hardware Software Components used in this guide:

In this post I have used

  • Ubuntu 18 server edition for TACACS+ deployment / IP: 101.11.11.254
  • Cisco WS-C3850-24T switch / IOS Version 16.3.9 [Denali]

Quick Notes:

TACACS Server installation

apt-get -y install tacacs+

Once the installation is done , we will modify or add the tacacs+ server default config file to to suite our needs. On default installation, the configuration file is found here /etc/tacacs+/tac_plus.conf

nano /etc/tacacs+/tac_plus.conf

Remove existing configuration, and use below sample config, make sure to change the KEY, id pass as required

# Key is like password or shared secret, make sure to make it strong

key = testing123
accounting file = /var/log/tacplus.log
#default authentication = file /etc/passwd
group = admins {
default service = permit
service = exec {
priv-lvl = 15
}
}

# For support group, we are allowing only specific sets of CMD only
group = support {
default service = deny
service = shell {
priv-lvl = 15
}
cmd = show {
permit version.*
permit clock.*
permit interface.*
permit running-config.*
permit logging.*
}
cmd = configure {
permit .*
}
cmd = interface {
permit .*
}
cmd = vlan {
permit .*
}
cmd = switchport {
permit .*
}
cmd = write {
permit .*
}
}

#Create local user here
user = admin {
login = cleartext admin123
name = "Admin Group"
member = admins
}
user = support {
login = cleartext support123
name = "Network Support"
member = support
}<span style="color:var(--color-text);">


 

& if all configuration is OK , you should get something like below …

 * Checking TACACS+ authentication daemon configuration files successful tacacs+

Restart tacacs+ service

/etc/init.d/tacacs_plus restart

Next up we will make changes to the Cisco switch ,
in this example am using a Cisco switch WS-C3850-24T and the one working configuration look like this:

Note: This is just basic example. It may be not well tuned insecure too but for test it will work fine.


Switch configuration

enable
conf t

aaa new-model
aaa authentication login default group tacacs+ local
aaa authentication enable default enable
aaa authorization config-commands
aaa authorization commands 1 support group tacacs+ local
aaa authorization commands 15 admins group tacacs+ local
aaa accounting commands 1 support-act1 start-stop group tacacs+
aaa accounting commands 15 admins-act15 start-stop group tacacs+
login on-success log

# change tacacs IP address / KEY as per your local network
tacacs-server host 101.11.11.254
tacacs-server key 0 testing123

!
line con 0
exec-timeout 35791 0
privilege level 15
stopbits 1
line aux 0
no exec
stopbits 1
line vty 0 3
privilege level 15
authorization commands 1 support
authorization commands 15 admins
accounting commands 1 support-act1
accounting commands 15 admins-act15
length 0
transport input ssh
line vty 4
exec-timeout 35791 0
privilege level 15
authorization commands 1 support
authorization commands 15 admins
length 0
transport input ssh
line vty 5 15
length 0
!

do wr

Done.

Now try to login to switch with support account & execute try to permitted / non-permitted commands.

Result for SUPPORT ACCOUNT

login as: support
Using keyboard-interactive authentication.
Password:

spare-sw#ping 101.11.11.254
Command authorization failed.

spare-sw#show clock
*10:24:07.527 UTC Mon Dec 16 2019

spare-sw#sh inter
spare-sw#sh interfaces status

Port Name Status Vlan Duplex Speed Type
Gi1/0/1 notconnect 1 auto auto 10/100/1000BaseTX
Gi1/0/2 notconnect 1 auto auto 10/100/1000BaseTX
Gi1/0/3 notconnect 1 auto auto 10/100/1000BaseTX
Gi1/0/4 notconnect 1 auto auto 10/100/1000BaseTX
Gi1/0/5 notconnect 1 auto auto 10/100/1000BaseTX
Gi1/0/6 notconnect 1 auto auto 10/100/1000BaseTX
Gi1/0/7 notconnect 1 auto auto 10/100/1000BaseTX
Gi1/0/8 notconnect 1 auto auto 10/100/1000BaseTX
Gi1/0/9 notconnect 1 auto auto 10/100/1000BaseTX
Gi1/0/10 notconnect 1 auto auto 10/100/1000BaseTX
Gi1/0/11 notconnect 1 auto auto 10/100/1000BaseTX

spare-sw#

 



Regard’s
Syed Jahanzaib

December 13, 2019

Mikrotik – Packet Chain Topology

Filed under: Mikrotik Related — Syed Jahanzaib / Pinochio~:) @ 9:36 AM

 


Firewall Mangle

Firewall mangle is used to mark IP packet. These marks are used by other router facilities like routing, firewall filter and bandwidth management to identified the packets. Moreover it also used to modify some fields in the IP header, like TOS (DSCP) and TTL fields. There are 5 default chain in firewall mangle.

INPUT

Packet that come into router will check with input chain. It is used to process packets entering the router. For example If we want to filter packet that telnet or ssh to router we need to use input chain in firewall filter.– Used to process packets entering the router through one of the interfaces with the destination IP address which is one of the router addresses. Chain input is useful for limiting the configuration access to Mikrotik Router.
or
– The connection that occurs from local to router and ends in router
example: use of the internal proxy (internal proxy right in the router)

PREROUTING

Prerouting is looked at before the router makes a routing decision. It’s happen before input chain. The packet that forward passthrough the router will match prerouting first.Pre-routing means that the connection will enter the router (no matter from where / depends on the settings mangle in interface later) . this connection will be in the process inside the router, can the process of bending to the external proxy, can filtering port, can anything, anyway there is a process, prerouting it marks the connection before the process occurs.

FORWARD

After prerouting, packet that passthrough router will process by forward chain. Used to process packets passing through the router. Example we want to block users to open facebook. We will use firewall forward chain to do it.– Used to process data packets through routers, connections that occur from the public to local
or
– The connection that occurs from local to public
with the provision that there is no process inside the router, it means that the connection is direct and only passing in the router does not happen a process inside the router.

POSTROUTING

Postrouting is a connection that will exit the router after a process occurs inside the router. It’s happen after forward. Postrouting is looked at after the router makes a routing decision.

OUTPUT

OUTPUT is used to process packets that originated from the router. Normally we rarely use this chain. Example we ping from router to Internet that’s output traffic.

or to process data packets originating from the router and left through one of the interfaces, connections that occur from the router to the public. It is used to process packets that originated from the router

============================================================================

Firewall Filter

Most of the time we use it for filter traffic simply say to protect our network from unauthorized user or bad guy. There are three default chain in firewall filter. It’s input, output and forward.

 

Does it matter where you mark?

Yes, because sometimes you might overmark and sometimes you might undermark. Choose the right place for the right classification/marking

 

December 10, 2019

Short notes for UNBOUND Caching DNS Server under Ubuntu 18

Filed under: Linux Related — Tags: , , , — Syed Jahanzaib / Pinochio~:) @ 12:05 PM

unbound.PNG

Installation of UNBOUND dns server for local network is fairly simple but I encountered some hurdles setting it up with Ubuntu 18 therefore I took notes on how I resolved it in this post for reference purposes.

After fresh installation of Ubuntu 18, It’a a good idea to keep your system TIME with any NTP source.

apt-get -y install ntp ntpdate
# Change timezone as per your local
cp /usr/share/zoneinfo/Asia/Karachi /etc/localtime
sudo /etc/init.d/ntp restart

Install UNBOUND DNS Server

Step#1

apt-get install -y unbound

Step#2

#Additional notes for Ubuntu 18 version

The problem with Ubuntu 18.04 is the systemd-resolved service which is listening on port 53 and therefore conflicts with unbound service

Edit the file /etc/systemd/resolved.conf

nano /etc/systemd/resolved.conf 

& modify this

DNSStubListener=no

Now reboot

shutdown -r now

You can now confirm if 53 port is now free up

netstat -tulpn | grep :53

Step#3

Some housekeeping stuff

sudo service systemd-resolved stop
sudo rm -f /etc/resolv.conf
sudo ln -s /run/systemd/resolve/resolv.conf /etc/resolv.conf
sudo service systemd-resolved start

Step#4

Edit the existing UNBOUND configuration file for customization…

nano /etc/unbound/unbound.conf

Example of unbound.conf

# Unbound configuration file for Debian.
server:
# Use the root servers key for DNSSEC
#auto-trust-anchor-file: "/var/lib/unbound/root.key"
# Enable logs
chroot: ""
#verbosity (log level from 0 to 4, 4 is debug)
#verbosity: 1
#logfile: /var/log/unbound/unbound.log
#log-queries: yes
#use-syslog: (do not write logs in syslog file in ubuntu /var/log/syslog -zaib)
use-syslog: no
#interface (interfaces on which Unbound will be launched and requests will be listened to)
# Respond to DNS requests on all interfaces
interface: 0.0.0.0
# DNS request port, IP and protocol
port: 53
do-ip4: yes
do-ip6: no
do-udp: yes
do-tcp: yes

# Authorized IPs to access the DNS Server / access-control (determines whose requests are allowed to be processed)
access-control: 127.0.0.0/8 allow
access-control: 10.0.0.0/8 allow
access-control: 172.16.0.0/16 allow
access-control: 192.168.0.0/16 allow
access-control: 101.0.0.0/8 allow

# Root servers information (To download here: ftp://ftp.internic.net/domain/named.cache)
#root-hints: "/var/lib/unbound/root.hints"

# Hide DNS Server info
hide-identity: yes
hide-version: yes

# Improve the security of your DNS Server (Limit DNS Fraud and use DNSSEC)
harden-glue: yes
harden-dnssec-stripped: yes

# Rewrite URLs written in CAPS
use-caps-for-id: yes

# TTL Min (Seconds, I set it to 7 days)
cache-min-ttl: 604800
# TTL Max (Seconds, I set it to 14 days)
cache-max-ttl: 1209600
# Enable the prefetch
prefetch: yes

# Number of maximum threads CORES to use / zaib
num-threads: 4

### Tweaks and optimizations
# Number of slabs to use (Must be a multiple of num-threads value)
msg-cache-slabs: 8
rrset-cache-slabs: 8
infra-cache-slabs: 8
key-cache-slabs: 8
# Cache and buffer size (in mb)
rrset-cache-size: 51m
msg-cache-size: 25m
so-rcvbuf: 1m

# Make sure your DNS Server treat your local network requests
#private-address: 101.0.0.0/8

# Add an unwanted reply threshold to clean the cache and avoid when possible a DNS Poisoning
unwanted-reply-threshold: 10000

# Authorize or not the localhost requests
do-not-query-localhost: no

# Use the root.key file for DNSSEC
#auto-trust-anchor-file: "/var/lib/unbound/root.key"
val-clean-additional: yes
include: "/etc/unbound/unbound.conf.d/*.conf"

Example of /etc/unbound/myrecords.conf

You can use this file to add your custom records as well.

Create new file at

nano /etc/unbound/myrecords.conf
local-zone: "doubleclick.net" redirect
local-data: "doubleclick.net A 127.0.0.1"
local-zone: "googlesyndication.com" redirect
local-data: "googlesyndication.com A 127.0.0.1"
local-zone: "googleadservices.com" redirect
local-data: "googleadservices.com A 127.0.0.1"
local-zone: "google-analytics.com" redirect
local-data: "google-analytics.com A 127.0.0.1"
local-zone: "ads.youtube.com" redirect
local-data: "ads.youtube.com A 127.0.0.1"
local-zone: "adserver.yahoo.com" redirect
local-data: "adserver.yahoo.com A 127.0.0.1"
local-zone: "1.com" redirect
local-data: "1.com A 0.0.0.0"
local-data: "zaib.com A 1.2.3.4"
local-data: "zaib2.com A 1.2.3.4"

Once all done, restart the unbound service by

service unbound restart
OR
service unbound reload

Test if UNBOUND service is started successfully.

service unbound status

Result:

â unbound.service - Unbound DNS server
Loaded: loaded (/lib/systemd/system/unbound.service; enabled; vendor preset: enabled)
Active: active (running) since Tue 2019-12-10 12:28:59 PKT; 2s ago
Docs: man:unbound(8)
Process: 1588 ExecStartPre=/usr/lib/unbound/package-helper root_trust_anchor_update (code=exited, status=0/SUCCESS)
Process: 1576 ExecStartPre=/usr/lib/unbound/package-helper chroot_setup (code=exited, status=0/SUCCESS)
Main PID: 1610 (unbound)
Tasks: 4 (limit: 2290)
CGroup: /system.slice/unbound.service
ââ1610 /usr/sbin/unbound -d

Dec 10 12:28:58 u18 systemd[1]: Starting Unbound DNS server...
Dec 10 12:28:59 u18 package-helper[1588]: /var/lib/unbound/root.key has content
Dec 10 12:28:59 u18 package-helper[1588]: success: the anchor is ok
Dec 10 12:28:59 u18 unbound[1610]: [1575962939] unbound[1610:0] warning: so-rcvbuf 1048576 was not granted. Got 425984. To fix: start with root permissions(linux) or sysctl bigger net.core.rmem_max
Dec 10 12:28:59 u18 unbound[1610]: [1575962939] unbound[1610:0] notice: init module 0: subnet
Dec 10 12:28:59 u18 unbound[1610]: [1575962939] unbound[1610:0] notice: init module 1: validator
Dec 10 12:28:59 u18 unbound[1610]: [1575962939] unbound[1610:0] notice: init module 2: iterator
Dec 10 12:28:59 u18 unbound[1610]: [1575962939] unbound[1610:0] info: start of service (unbound 1.6.7).
Dec 10 12:28:59 u18 systemd[1]: Started Unbound DNS server.

Test if DNS server is responding to DNS queries

dig @127.0.0.1 bbc.com

1st Result: [check the Query time]

;  DiG 9.11.3-1ubuntu1.11-Ubuntu  @127.0.0.1 bbc.com
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 16313
;; flags: qr rd ra; QUERY: 1, ANSWER: 4, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;bbc.com. IN A

;; ANSWER SECTION:
bbc.com. 86400 IN A 151.101.192.81
bbc.com. 86400 IN A 151.101.128.81
bbc.com. 86400 IN A 151.101.0.81
bbc.com. 86400 IN A 151.101.64.81

;; Query time: 971 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Tue Dec 10 07:04:21 UTC 2019
;; MSG SIZE rcvd: 100

2nd Result: [check the Query time]

root@u18:/etc/unbound/unbound.conf.d# dig @127.0.0.1 bbc.com

;  DiG 9.11.3-1ubuntu1.11-Ubuntu  @127.0.0.1 bbc.com
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 14171
;; flags: qr rd ra; QUERY: 1, ANSWER: 4, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;bbc.com. IN A

;; ANSWER SECTION:
bbc.com. 86398 IN A 151.101.192.81
bbc.com. 86398 IN A 151.101.128.81
bbc.com. 86398 IN A 151.101.0.81
bbc.com. 86398 IN A 151.101.64.81

;; Query time: 0 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Tue Dec 10 07:04:23 UTC 2019
;; MSG SIZE rcvd: 100

See the difference between 1st & second response which shows that cache is working

 


Enabling LOG File [recommended for troubleshoot purposes only]

Create a Log file and assign rights to write logs:

mkdir /var/log/unbound
touch /var/log/unbound/unbound.log
chmod -R 777  /var/log/unbound/

Now enable it in the unbound config file. I have commented it in the configuration file.

An example of viewing logs:

sudo tail -f /var/log/unbound/unbound.log
sudo tail -f /var/log/syslog

UNBOUND.LOG

[1575963664] unbound[1962:3] info: 101.11.11.161 bbc.com.agp1. A IN
[1575963664] unbound[1962:3] info: resolving bbc.com.agp1. A IN
[1575963664] unbound[1962:3] info: response for bbc.com.agp1. A IN
[1575963664] unbound[1962:3] info: reply from  193.0.14.129#53
[1575963664] unbound[1962:3] info: query response was NXDOMAIN ANSWER
[1575963664] unbound[1962:3] info: validate(nxdomain): sec_status_secure
[1575963664] unbound[1962:3] info: validation success bbc.com.agp1. A IN
[1575963664] unbound[1962:3] info: 101.11.11.161 bbc.com.agp1. AAAA IN
[1575963664] unbound[1962:3] info: resolving bbc.com.agp1. AAAA IN
[1575963664] unbound[1962:3] info: response for bbc.com.agp1. AAAA IN
[1575963664] unbound[1962:3] info: reply from  199.7.83.42#53
[1575963664] unbound[1962:3] info: query response was NXDOMAIN ANSWER
[1575963664] unbound[1962:3] info: validate(nxdomain): sec_status_secure
[1575963664] unbound[1962:3] info: validation success bbc.com.agp1. AAAA IN
[1575963664] unbound[1962:1] info: 101.11.11.161 bbc.com. A IN
[1575963664] unbound[1962:1] info: resolving bbc.com. A IN
[1575963664] unbound[1962:1] info: resolving bbc.com. DS IN
[1575963664] unbound[1962:1] info: NSEC3s for the referral proved no DS.
[1575963664] unbound[1962:1] info: Verified that unsigned response is INSECURE
[1575963672] unbound[1962:0] info: 101.11.11.161 bbc.com. AAAA IN

Example of cache export and import:

unbound-control dump_cache > backup
unbound-control load_cache < backup

#Clear one site from cache

unbound-control flush_zone google.com

# View cached DNS contents or count

unbound-control dump_cache
unbound-control dump_cache | wc -l

Regard’s
Syed Jahanzaib

December 5, 2019

Intervlan Routing with Mikrotik DHCP Option 121 & 249

Filed under: Mikrotik Related — Syed Jahanzaib / Pinochio~:) @ 12:18 PM

network.jpg

Disclaimer! This is important!

My humble request is that kindly donot consider me as an expert on this stuff, I am NOT certified in anything Mikrotik/Cisco/Linux or Windows. However I have worked with some core networks and I read , research & try stuff all of the time. When you are enslaved by private job & working as one man army, you have to perform many task in which you are not formally trained for. So I am not speaking/posting about stuff I am formerly trained in, I pretty much go with experience and what I have learned on my own. And , If I don’t know something then I read & learn all about it.

So , please don’t hold me/my-postings to be always 100 percent correct. I make mistakes just like everybody else. However – I do my best, learn from my mistakes and try to help others


This post is a sequel of Prevent Mikrotik from Chocking with Cisco Inter-Vlan Routing

If you are running mikrotik DHCP along with VLAN, & you want to get benefits of intervlan routing to avoid local traffic (same network but different subnets/vlans) hitting mikrotik by routing local resource requests through switch level directly. There are 2 options to sort this issue. Either provide the static routes for each vlan which becomes useless if you are using pppoe because as soon as client connected with pppoe, it will get preference over your existing default gateway thus all traffic even the local one will be passing via mikrotik resulting in excessive usage of mikrotik resources. This was discussed in details here.

Far better solution is to avoid providing any default routes to DHCP users but assign them CLASSLESS ROUTES for local resources via your DHCP options, either via Cisco or Mikrotik DHCP server. A classless route adds specified route in clients routing table

In this post we will discuss only about Mikrotik DHCP option 121/249.

  • 121 – Generally used for devices / Linux / Mikrotik etc. Option 121 is a RFC3442 *standard*.However, note that DHCP Option 121 is ignored by DHCP clients prior to Windows Vista. O
  • 249 – It is the pre-standard Microsoft implemented option for Classless Static Routes (CSRs). It was eventually standardized as option 121. I think 249 is for microsoft only.

Personnel Note: Don’t use both.  I think this causes routes to be ignored. If you need to support both older and newer OS, use the option 249.

Scenario:

We assume you have working DHCP for each vlan under Mikrotik.

  • Mikrotik interface VLAN-x IP : 192.168.100.1/24
  • Cisco switch Vlan-x IP : 192.168.100.2/24
  • Mikrotik DHCP assignment for VLAN-x: 192.168.100.10 – 192.168.100.255 [no dns, no gateway]
  • Local sharing servers: We have some sharing servers on different subnet on local switch port: 101.0.0.0/8

Now what we want is that our DHCP LAN client 192.168.100.0/24 can access 101.0.0.0/8 via switch intervlan routing directly [via switch vlan port 192.168.100.2]. To achieve this we will assign stateless routes using DHCP options.

Side Note: It strikes me that you have to use classful routes in the Classless Static Routes DHCP Option for older version of windows like XP/2000.


Step # 1

To transfer classless routes to the Mikrotik DHCP server, the option with code 121/249 is used First we need to convert IP to hexadecimal HEX code. If you are a beginner you can simply get the ready made code using this site.

https://ip-pro.eu/en/mikrotik_dhcp_option_121_generator

Enter the details as per your local network scheme

For single Subnet:

dhcp option hex code via web site.PNG

For multiple Subnet:

You can add ADD NEW ROW to add multiple subnet and gateways to get combined HEX value . Z

multiple subnet routes in single line.PNG

 

Step # 2

Now we got the HEX value & we can use it in mikrotik dhcp option 121-249 .

  • Goto IP / DHCP / OPTIONS / + Add new

dhcp options for 121-249.PNG

Note: Make two entries, one for 121 & second for 249. As shown above !

Step # 3

Now we will add the above option in OPTION SETS , we can include multiple options in the OPTION SETS window

  • Goto IP / DHCP / OPTION SETS / + Add new

dhcp options set 2.PNG

Step # 4

Now goto IP / DHCP / Double Click on required DHCP & under DHCP OPTION SETS, select the option set we created in Step # 4


Testing …

At client end, renew the IP and you will get all the routes you configured in above steps.

For single Subnet entry:

===========================================================================

IPv4 Route Table
===========================================================================
Active Routes:
Network Destination Netmask Gateway Interface Metric
101.0.0.0 255.0.0.0 192.168.100.2 192.168.100.254 11 ***************
127.0.0.0 255.0.0.0 On-link 127.0.0.1 306
127.0.0.1 255.255.255.255 On-link 127.0.0.1 306
127.255.255.255 255.255.255.255 On-link 127.0.0.1 306
192.168.100.0 255.255.255.0 On-link 192.168.100.254 266
192.168.100.254 255.255.255.255 On-link 192.168.100.254 266
192.168.100.255 255.255.255.255 On-link 192.168.100.254 266
224.0.0.0 240.0.0.0 On-link 127.0.0.1 306
224.0.0.0 240.0.0.0 On-link 192.168.100.254 266
255.255.255.255 255.255.255.255 On-link 127.0.0.1 306
255.255.255.255 255.255.255.255 On-link 192.168.100.254 266
===========================================================================
Persistent Routes:
None

IPv6 Route Table
===========================================================================
Active Routes:
If Metric Network Destination Gateway
1 306 ::1/128 On-link
1 306 ff00::/8 On-link
===========================================================================
Persistent Routes:
None

For Multiple Subnet entry:

===========================================================================

IPv4 Route Table
===========================================================================
Active Routes:
Network Destination Netmask Gateway Interface Metric
10.0.0.0 255.0.0.0 192.168.100.1 192.168.100.254 11 ***************
127.0.0.0 255.0.0.0 On-link 127.0.0.1 306
127.0.0.1 255.255.255.255 On-link 127.0.0.1 306
127.255.255.255 255.255.255.255 On-link 127.0.0.1 306
172.16.10.0 255.255.255.0 192.168.100.1 192.168.100.254 11 ***************
192.168.100.0 255.255.255.0 On-link 192.168.100.254 266
192.168.100.254 255.255.255.255 On-link 192.168.100.254 266
192.168.100.255 255.255.255.255 On-link 192.168.100.254 266
221.132.112.8 255.255.255.255 192.168.100.1 192.168.100.254 11 ***************
224.0.0.0 240.0.0.0 On-link 127.0.0.1 306
224.0.0.0 240.0.0.0 On-link 192.168.100.254 266
255.255.255.255 255.255.255.255 On-link 127.0.0.1 306
255.255.255.255 255.255.255.255 On-link 192.168.100.254 266
===========================================================================
Persistent Routes:
None

IPv6 Route Table
===========================================================================
Active Routes:
If Metric Network Destination Gateway
1 306 ::1/128 On-link
1 306 ff00::/8 On-link
===========================================================================
Persistent Routes:
None

Regard’s
Syed Jahanzaib

 

 

 

 

December 3, 2019

Cisco Switch: Upgrading Firmwares & Recovering from Failed ones !

Filed under: Cisco Related — Tags: , , , , , , , , — Syed Jahanzaib / Pinochio~:) @ 9:56 AM

If it ain’t broke, don’t fix it ! So true 🙂 Z@ib


advise on upgrade of switch fw.PNG


Quick notes for myself: Disclaimer! This is important!

My humble request is that kindly donot consider me as an expert on this stuff, I am NOT certified in anything Mikrotik/Cisco/Linux or Windows. However I have worked with some core networks and I read , research & try stuff all of the time. When you are enslaved by private job & working as one man army, you have to perform many task in which you are not formally trained for. So I am not speaking/posting about stuff I am formerly trained in, I pretty much go with experience and what I have learned on my own. And , If I don’t know something then I read & learn all about it.

So , please don’t hold me/my-postings to be always 100 percent correct. I make mistakes just like everybody else. However – I do my best, learn from my mistakes and try to help others

Maybe this post will help some other novice like me

Regard’s
Syed Jahanzaib


WS-C3650-48PD

Fortunately this model have nice GUI which supports easy upgrade of firmwares , therefore I used it to upgrade to latest stable build Fuji-16.9.4

365048pdl fw upgrade.PNG


SG300-28PP/SG500-52PP

This model also have nice GUI which supports easy upgrade of firmwares , therefore I used it to upgrade to latest stable build

https://www.cisco.com/c/en/us/support/switches/sg500-52-52-port-gigabit-stackable-managed-switch/model.html#~tab-downloads

sg300-500.PNG


3750X-48T-S

.

WS-C3750E-24TD

OLD version:

WS-C3750E-24TD     12.2(40)SE

3750 fw old.PNG

Steps for Upgrade:

First setup new or use existing TFTP server, download the new firmware from

https://software.cisco.com/download/home/280831016/type/280805680/release/15.0.2-SE11?i=!pp

& copy the firmware file (.BIN format) in TFTP root folder. Now from switch console , copy the file from TFTp into switch FLASH

copy tftp: flash:

It will ask the tftp server IP, and source/destination file name, fill them up as per your local scenario

>Address or name of remote host? <IP_ADDRESS_OF_TFTP_SERVER>
>source filename? <IMAGE_NAME.bin>
>Destination filename? <IMAGE_NAME.bin>
conf t
boot system flash:<IMAGE_NAME.bin>
exit
wr
reload

after this switch will reboot , and new firmware will be overwritten.

New version:

Release 15.0.2-SE11 MD

3750 fw new.PNG

 


WS-C3850-24T

Notes:

With this model, I encountered few issues in upgrading 3850 switch.

  • Gibralter 16.12.1 ED : 3850 / Switch was doing reboot in loop with following error

Kernel panic – not syncing: VFS: Unable to mount root fs on unknown-block(1,0)

  • Fuji 16.9.4 MD : 3850 / Switch Port Orange Light issue

With this upgrade , switch booted but all ports lights runed to amber.

  • Denali 16.3.9 MD : 3850 / Well tested , worked OK

Therefore I reverted back to to Denali-16.3.9 which worked fine & stable.

Steps for Upgrade:

First setup new or use existing TFTP server, download the new firmware from

https://www.cisco.com/c/en/us/support/switches/catalyst-3850-24t-s-switch/model.html#~tab-downloads

>Address or name of remote host? <IP_ADDRESS_OF_TFTP_SERVER>
>source filename? <IMAGE_NAME.bin>
>Destination filename? <IMAGE_NAME.bin>
conf t
software install file flash:cat3k_caa-universalk9.16.03.09.SPA.bin new force verbose

after this switch will ask to reload , do so to apply changes

New version:

16.3.9

3850 new ver.PNG


Recovering from IOS FAILED upgrade on 3850 Switch

after the Gibralter firmware upgrade , 3850 switch wen into reboot loop.

Kernel panic – not syncing: VFS: Unable to mount root fs on unknown-block(1,0)

More infor on this issue mentioned here @ https://community.cisco.com/t5/switching/catalyst-c3850-gibraltar-16-12-1-ed/td-p/3907723

After entering recovery mode , I made the situation worsen by deleting some flash files. This is how I recovered from this situation.

from the switch Management port, connect a cable directly to your laptop/desktop & assign any private IP on the system like 192.168.99.1/24. Install any free TFTP server like SOLARWINDS TFTP Server. copy the stable firmware like DENALI 16.0.3.09 in the tftp root folder.

cat3k_caa-universalk9.16.03.09.SPA.bin

Now using any terminal tool like putty, connected to switch via CONSOLE port, press MODE button while booting & entered RECOVERY mode. & issued following CMD’s

  • flash_init
  • mgmt_init
  • set IP_ADDR 192.168.99.2/255.255.255.0
  • set default_router 192.168.99.1
  • emergency-install tftp://192.168.99.1/cat3k_caa-universalk9.16.03.09.SPA.bin

There was another easy method by connecting USB into usb port of switch, but since switch may not recognize most modern usb’s , therefore I had to take a long route of TFTP.

To read in more details , refere following post

http://blog.unolution.com/networking/how-to-recover-a-cisco-switch-3560x-3750x-from-boot-loader/


Regard’s
Syed Jahanzaib

 

November 28, 2019

Virtualization: Quick Notes for myself

Filed under: VMware Related — Tags: , , — Syed Jahanzaib / Pinochio~:) @ 10:20 AM

vmware esxi



ESXI Server: ‘SEL Fullness’ message

Recently after upgrading ESXI from 6.5 to 6.7u3 on Lenovo (IBM) x3650 M5 machine, I received following error on vsphere web client

System Management Software 1 SEL Fullness: Log almost full

In vSphere 6.7 , Navigate to following and reset event log

Monitor > Tasks and Events > Hardware Health > SYSTEM EVENT LOG > RESET EVENT LOG


Updating ESXI 6.5 to 6.7u3 from the CLI

Recently I did some major updates in our virtual infrastructure including ESXi (ver 6.5.0, 10719125) ,  Vcenter & Veeam (9.5.4.2866) . Things were updated in following order

  • Veeam B&R upgraded to ver 9.5.4.2866
  • Vcenter upgraded to ver 6.7.0.40000
  • ESXI hosts upgraded to ver 6.5.0, 10719125

For ESXI update from 6.5 to 6.7u3. In the past I always use Installer CD/USB to upgrade from older esxi to new version, but for this approach, I have to compromise on my holidays or sit very late in office. This time time I took another approach and upgraded all the esxi hosts one by one on sundays remotely from the home using offline bundle installer & esxi CLI method. This is how I accomplished it.

I first downloaded the 6.7 update offline bundle from the Lenovo site (since we have all the IBM/Lenovo brand servers therefore I selected this option to avoid any hardware driver issue).

https://vmware.lenovo.com/content/custom_iso/6.7/6.7u3/

Afterwards I uploaded this offline bundle zip file to Esxi datastore, then logged in to esxi host via SSH, and issued

esxcli software vib install -d /vmfs/volumes/5d0cf64f-a83e7c86-6a4d-40f2e922c64a/Lenovo_Offline_Bundle_VMware_ESXi_6.7.0.update03_14320388_LNV_20190920.zip

Note: make sure to change datastore and filename as required.

It took few minutes, once I saw SUCCESS message, I completed the process by simply rebooted the ESXI host by cmd

reboot

Better approach is to update rather than install

esxcli software vib update -d /vmfs/volumes/5d0cf64f-a83e7c86-6a4d-40f2e922c64a/Lenovo_Offline_Bundle_VMware_ESXi_6.7.0.update03_14320388_LNV_20190920.zip

Difference between VIB update and VIB install

Excerpt from “https://communities.vmware.com/thread/435959&#8221;

To install or update a .zip file, use the -d option. To install or update a .vib file use the -v option.

Using the update command is the recommended method for patch application. Using this command applies all of the newer contents in a patch, including all security fixes. Contents of the patch that are a lower revision than the existing packages on the system are not applied.

Using the install command overwrites the existing packages in the system with contents of the patch you are installing, including installing new packages and removing old packages. The install command may downgrade packages on the system and should be used with caution. If required, the install command can be used to downgrade a system (only for image profiles) when the –allow-downgrade flag is set.

The install method has the possibility of overwriting existing drivers. If you are using 3rd party ESXi images, VMware recommends using the update method to prevent an unbootable state.

Check esxi version from CLI

esxcli system version get
Product : VMware ESXi
Version : 6.7.0
Build : Releasebuild-14320388
Update : 3
Patch : 73<span style="color:var(--color-text);">


Will update more

Regards
Syed Jahanzaib

November 26, 2019

Assigning friendly/fix name to USB device

Filed under: Linux Related — Tags: , , , , , , — Syed Jahanzaib / Pinochio~:) @ 9:34 AM

depositphotos_86584980-stock-photo-select-confusion-sign

Teltonika Usb Modem

Teltonika Usb Modem


Scenario:

We have Teltonika USB modem connected to our Linux box (Ubuntu) & Kannel is configured as SMS gateway for our network devices so that they can send Info/Alert SMS incase of any failure via HTTP method.

/dev/ttyACM0 is default port name on which this modem is detected by default BUT it happens Often that when the Linux box reboots, modem is detected on different port name like ttyACM1 which result in failure of kannel modem detection as the name is hardcoded in /etc/kannel/kannel.conf

To settle this issue, we fixed the USB device with our customized name , and in kannel we used this fixed name which sorted the modem port name change issue.

Step#1

Run following CMD

udevadm info -a -p $(udevadm info -q path -n /dev/ttyACM1)

& look for following attributes & note down both

  • idVendor
  • idProduct

Step#2

now Edit or create new file in /etc/udev by

nano /etc/udev/rules.d/99-usb-serial.rules

& paste following

SUBSYSTEM=="tty", ATTRS{idVendor}=="1d12", ATTRS{idProduct}=="3e11", SYMLINK+="gsm"

Make sure to change the idVendor & idProduct numbers that you noted in step #1.

Save & Exit.

Now issue below CMD to reload udev rules

sudo udevadm trigger

if all goes well, then look for your new device name gsm in /dev folder

ls -l /dev/gsm

Ok result would be similar to this

lrwxrwxrwx 1 root root 7 Nov 26 09:04 /dev/gsm -> ttyACM1

Regard’s
Syed Jahanzaib

 

November 23, 2019

DMASOFTLAB Radius Manager – Adding custom attribute to facilitate Dynamic address list on Mikrotik

Filed under: Mikrotik Related, Radius Manager — Tags: , , — Syed Jahanzaib / Pinochio~:) @ 3:04 PM

Quick Recipe: If you are using DMA Radius Manager & want to assign Dynamic Address list to a service so that user can automatically be added to dynamic address list under NAS, you can do so by using custom RADIUS attributesunder services section

  • Login to Admin Panel
  • Goto Services
  • Click on your desired service example 4mb
  • Under `Custom RADIUS attributes` , add below attribute
Mikrotik-Address-List := 4mb

adding attribute in radius manager service

Save , & test any user authentication by CMD,

rmauth 127.0.0.1 test 1

freeradius attribute for dma radius manager test via cmd

 

on NAS dynamic address list will be created for each user of this service group. Late ryou can use this address list to mark connections / packets/ routing/queue etc.

4mb pppoe.PNG

Happy Attributing 😉


Regard’s
Jz.

Older Posts »

%d bloggers like this: