Syed Jahanzaib Personal Blog to Share Knowledge !

January 1, 2020

CGNAT Deployment using Mikrotik RouterOS

Filed under: Mikrotik Related — Tags: , , , , , , , — Syed Jahanzaib / Pinochio~:) @ 5:34 PM

mdnet cgnat conept.PNG

Incomplete Post, this contains only src-nat part, second method of NETMAP will  be added soon which is far more simple & efficient as compared to the src-nat method

My humble request, Kindly donot consider me as an expert on this stuff, I am NOT certified in anything Mikrotik/Cisco/Linux or Windows. However I have worked with some networks and I read , research & try stuff all of the time. When you are enslaved by private job & working as one man army, you have to perform many task in which you are not formally trained for. So I am not speaking/posting about stuff I am formerly trained in, I pretty much go with experience and what I have learned on my own. And , If I don’t know something then I read & learn all about it.

So , please don’t hold me/my-postings to be always 100 percent correct. I make mistakes just like everybody else. However – I do my best, learn from my mistakes and try to help others

Some references used in this post


*CG-NAT* as Workaround:

CGNAT concept is used to share one or preferably more public IP addresses with large number of private ip addresses on ratio basis.CGNAT/NAT444 is a conception, not a function. In terms of RouterOS functionality it’s simple SRC NAT rule.

To combat with this IPV4 exhausting issue, we can use CGNAT as a workaround. This is by no means a solution, & the OP should get public IP space (either ipv4 or ipv6) to comply with the LAW.

Some possible disadvantages of using CGNAT concept:

  • CGNAT is not sustainable in the long term, hectic to manage the private/public pools especially if you have multiple NASes doing same job
  • ISP deploying IP address sharing techniques should also deploy a corresponding logging architecture to maintain records of the relation between a customer’s identity and IP/port resources utilized
  • You should deploy additional SYSLOG server (either windows or linux base) to store logs. I would prefer linux base SYSLOG-NG). Tracking of users for legal reasons means searching hundreds GB’s of logging would be required, as multiple end users go behind one (or more) public IP address(es)
  • Tracking Logs is not an easy task particularly when you have tongs of Logging (in a DB).Logging every NAT translation is resource consuming. Some super fast computing resources (including preferably RAID10 or SSD based storage) and fine tune DB would be required
  • A CG-NAT device must use the same external IP address mapping
    for all sessions associated with the same internal IP address
  • Most Applications do not behave well with TCP resets
  • Many operators are still not familiar with CG-NAT complexities. There is a lot of trial and error on the part of ISP’s

 

in my personal experience , Deployment is somewhat hectic, & tracking any request is daunting task ! z@ib


Hardware/Software Used in this post:


CGNAT logging to remote syslog server with some customization

https://aacable.wordpress.com/2020/01/08/syslog-ng-part-3-minimized-logging-to-mysql-with-dynamic-tables-trimming/


Scenario#1

OP is running mini ISP with around 200 active subscribers. Mikrotik Router is being used as PPPoE Server along with Freeradius as AAA. On Mikrotik, one public IP is configured for WAN and additional /24 routed pool (256 public IP addresses) is provided to the OP via ISP so that he can provide public IP to each user. After the network upgrades , OP have reached 700 users in total, and since he have only 256 public ip’s , he is now using natting for half of his users.

We all know that IPV4 shortage is on peak , getting ipv4 is expensive for 3rd world countries & small ISP’s as well.

This NATTING workaround is creating hurdles in tracking illegal activity performed by any NATTED users because hundreds of NATTED user will have same public ip (Mikrotik WAN IP). nowadays law sometimes provide only the public ip along with source port and ask for the user credentials details for investigation purposes.

with single public IP and hundreds of natted hosts behind it. tracking is nearly impossible.


IP scheme example used in this Scenario#1:

Public IP range: (/24 public IP’s routed pool)

  • 1.1.1-1.1.1.255
  • Total Public IP useable: 255

Private IP range for PPPoE users:

  • 172.16.1.1-172.16.1.255
  • 172.16.2.1-172.16.2.255
  • 172.16.3.1-172.16.3.255
  • Total Private IP useable: 765

For 765 Users, we will be using 1:5 Ratio, thus 153 public ips will be used for 765 users. (on a ratio of 1:5).

  • per private IP, we will reserve 10,000 ports, which should be more than enough for each user.
  • per private IP, we will be creating 3 rules, one for TCP, second for UDP, 3rd for non ports range

CGNAT configuration on RouterOS is very much similar to regular source NAT configuration.


To add multiple Public IP addresses on WAN interface in bulk using single CMD on Terminal

You may need to add all of your public IP addresses (which will be used for CGNAT) on WAN interface(required fro troubleshooting purposes as well).

To add ips in bulk using single CMD, you can use Mikrotik FOR X script function for ease / ZAIB

:for x from 1 to 153 do={ /ip address add address="1.1.1.$x/32" comment="1.1.1.$x - Routed IP for ppp CGNAT - zaib" interface="ether1-wan"}

 


Adding FUNCTION in Mikrotik for later Automation

Paste this in Mikrotik RouterOS terminal:

# CGNAT Customized minimalistic Script to add function.
# Syed Jahanzaib / aacable at hotmail dot com
:global sqrt
:global sqrt do={
:for i from=0 to=$1 do={
:if (i * i > $1) do={ :return ($i - 1) }
}
}
:global addNatRules do={
:local x [$sqrt $count]
:local y $x
:if ($x * $x = $count) do={ :set y ($x + 1) }
:for i from=0 to=($count - 1) do={
:local prange "$($portStart + ($i * $portsPerAddr))-$($portStart + (($i + 1) * $portsPerAddr) - 1)"
/ip firewall nat add chain=srcnat action=src-nat protocol=tcp src-address=($srcStart + $i) to-address=$toAddr to-ports=$prange
/ip firewall nat add chain=srcnat action=src-nat protocol=udp src-address=($srcStart + $i) to-address=$toAddr to-ports=$prange
/ip firewall nat add chain=srcnat action=src-nat src-address=($srcStart + $i) to-address=$toAddr
}
}

Now we have function inserted with the help of above code, and using this function, we can create rules in bulk using following CMD to add rules in NAT section

# per private IP, we will reserve 10000 ports, which should be more than enough for each user.
# per private IP, we will be creating 3 rules, one for TCP, second for UDP, 3rd for non ports range

$addNatRules count=5 srcStart=172.16.1.1 toAddr=1.1.1.1 portStart=10000 portsPerAddr=9999
$addNatRules count=5 srcStart=172.16.1.6 toAddr=1.1.1.2 portStart=10000 portsPerAddr=9999
$addNatRules count=5 srcStart=172.16.1.11 toAddr=1.1.1.3 portStart=10000 portsPerAddr=9999
$addNatRules count=5 srcStart=172.16.1.16 toAddr=1.1.1.4 portStart=10000 portsPerAddr=9999
$addNatRules count=5 srcStart=172.16.1.21 toAddr=1.1.1.5 portStart=10000 portsPerAddr=9999
$addNatRules count=5 srcStart=172.16.1.26 toAddr=1.1.1.6 portStart=10000 portsPerAddr=9999
$addNatRules count=5 srcStart=172.16.1.31 toAddr=1.1.1.7 portStart=10000 portsPerAddr=9999
$addNatRules count=5 srcStart=172.16.1.36 toAddr=1.1.1.8 portStart=10000 portsPerAddr=9999
$addNatRules count=5 srcStart=172.16.1.41 toAddr=1.1.1.9 portStart=10000 portsPerAddr=9999
$addNatRules count=5 srcStart=172.16.1.46 toAddr=1.1.1.10 portStart=10000 portsPerAddr=9999

# & so on for rest of the pool, you can further automate this by using additional functions & scripting

Enable Logging of CG-NAT Output:

# To log user IP/NAT information on LOG window / you can configure remote log to syslog-server too
/ip firewall filter
add action=accept chain=forward log=yes log-prefix="NAT_INFO_FW> " src-address=172.16.0.0/16

Log Result (from different servers , so ip scheme may be changed in these logs, For example purposes)

In this log you can clearly see the src-dst address, and on which public ip request was natted along with ports. This is useful

Rules from LAB Router:

Mikrotik WAN IP’s (2 for test purposes):

  • 101.11.11.255/32
  • 101.11.11.253/32

PPPoE Users (2 for test)

  • 172.16.0.1
  • 172.16.0.2

REMOTE WEB SERVER (considering it’s a web server on internet which our user is accessing or doing illegal stuff)

  • 101.11.11.255

SRC-NAT Rules on MIKROTIK:

/ip firewall nat
add action=src-nat chain=srcnat protocol=tcp src-address=172.16.0.1 to-addresses=101.11.11.255 to-ports=10000-19999
add action=src-nat chain=srcnat protocol=udp src-address=172.16.0.1 to-addresses=101.11.11.255 to-ports=10000-19999
add action=src-nat chain=srcnat src-address=172.16.0.1 to-addresses=101.11.11.255
add action=src-nat chain=srcnat protocol=tcp src-address=172.16.0.2 to-addresses=101.11.11.253 to-ports=20000-29999
add action=src-nat chain=srcnat protocol=udp src-address=172.16.0.2 to-addresses=101.11.11.253 to-ports=20000-29999
add action=src-nat chain=srcnat src-address=172.16.0.2 to-addresses=101.11.11.253

cgnat-log-1.PNG

Result:

On internet web server, we see following

[101.11.11.255]:10133 - - [02/Jan/2020:15:44:37 +0500] "GET /? HTTP/1.1" 200 3138 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.88 Safari/537.36"

so the law enforcement agency come to us, and tell us that this is your public IP+Port  101.11.11.255:10133, now give us his details. And as we know that we are doing CGNAT, so we have to do little tracking.

On Mikrotik LOG we see following [after enabling LOGS,

cgnat-log-2.PNG

You can now see that our public IP having port 10133 was natted for our local user IP 172.16.0.1. with PPPoE it will show you the user name as well, so you can catch it right from here, or else if RADIUS is being used, you can track the IP via freeradius DB in radacct.


Scenario#2

OP have single public IP (e.g: 101.11.11.252) configured on Mikrotik WAN interface. End user subscriber is connected to mikrotik pppoe server using pppoe dialer. In this example we will be using 172.16.0.0/24 (256 users) and each user IP will be allowed to use 200 ports (200 ports per private IP).

This way when LAW will ask to provide details for 101.11.11.252:41636 , we can look into our LOGS (usually SYSLOG server either in linux, or using windows based SYSLOG like solarwinds syslog serveR) we can look into the 101.11.11.252:41636 & we can see the pppoe username or its private ip and search the ip in radius radacct table if radius is being used)

$addNatRules count=255 srcStart=172.16.0.1 toAddr=101.11.11.252 portStart=2000 portsPerAddr=200

Above CMD will create 765 rules (for 256 users) in IP / Firewall / NAT section. (make sure you have pasted the addNatRules function in the terminal before using above command.

– Enable mikrotik logs in Mikrotik LOG window

To enable LOGS in mikrotik LOG window , use

/ip firewall filter
add action=accept chain=forward log=yes log-prefix="NAT_INFO_FW> " src-address=172.16.0.0/16

– Enable mikrotik built in DISK base logging

To enable DISK base LOGGING in Mikrotik itself, (avoid this, it will OVERLOAD your routerboard which is not designed to handle such massive load of LOGS)

/system logging action
set 1 disk-file-count=25 disk-lines-per-file=5000
/system logging
add action=disk prefix=NAT_INFO_FW topics=info

– Enable remote SYSLOG logging in mikrotik

To ENABLE remote SYSLOG (I used Solarwind SYSLOG server on Windows in this example.

/system logging action
set 3 bsd-syslog=yes remote=10.0.0.2
/system logging
add action=remote prefix=NAT_INFO_FW topics=info

Now we can see in the LOG window (just an example, in actual you have to use some SYSLOG server) to search for 101.11.11.252:41636

Jan/03/2020 10:48:43 firewall,info NAT_INFO_FW> forward: in: out:ether1-wan, src-mac d0:bf:9c:f7:88:76, proto TCP (ACK), 172.16.0.199:54326->179.60.194.35:443, NAT (172.16.0.199:54326->101.11.11.252:41636)->179.60.194.35:443, len 40
Jan/03/2020 10:48:43 firewall,info NAT_INFO_FW> forward: in: out:ether1-wan, src-mac d0:bf:9c:f7:88:76, proto TCP (ACK), 172.16.0.199:54326->179.60.194.35:443, NAT (172.16.0.199:54326->101.11.11.252:41636)->179.60.194.35:443, len 52

& as you can see that 101.11.11.252:41636 was used private IP 172.16.0.199 & it will also show the <pppoe-zaib> . This way you can pull the user details & provide it to law enforcement agencies.

on windows base REMOTE syslog we can see the results, and can search easily as well.

solarwind syslog.png


To Delete older logs from syslog mysql DB

 mysql -uroot -pSQLPASSWORD -s -e "use syslog; DELETE FROM logs WHERE date(datetime) < (CURDATE() - INTERVAL 3 MONTH);"

TIPS for Linux base SYSLOG-NG trimming

I am using SYSLOG-NG to store all logs , to log only the NAT related queries (which actually shows the entries of public:port vs private ip:port use following in syslog ng configuration (before SOURCE section

######## Zaib Section Starts here
# Accept connection on UDP
source s_net { udp (); };
# MIKROTIK ########### add logs into files & in mysql dB as well.zaib
# Add Filter to add our mikrotik
filter f_mikrotik_1 { host("10.0.0.1") and match("NAT" value("MESSAGE")) };
#filter f_mikrotik_1 { host( "10.0.0.1" ); };
log { source ( s_net ); filter( f_mikrotik_1 ); destination ( df_mikrotik_1 ); };
destination df_mikrotik_1 {
file("/var/log/zlogs/${HOST}.${YEAR}.${MONTH}.${DAY}.log"
template-escape(no));
};

source s_mysql {
udp(port(514));
tcp(port(514));
};
# Play with below, some confusion here
destination d_mysql { pipe("/var/log/mysql.pipe" template("INSERT INTO
logs (host,facility,priority,level,tag,datetime,program,msg) VALUES
('$HOST','','','','','$YEAR-$MONTH-$DAY
$HOUR:$MIN:$SEC','','$MSG');\n") template-escape(yes)); };

log {
source(s_net);
filter(f_mikrotik_1);
destination(d_mysql);
};
####### #Zaib Section ends here

Note: For 500 active subscribers , the avg logs size on the syslog DB was 240 MB. This was after the controlled syslog entries (logging of requests that contains word NAT only).


Regard’s
~ Syed Jahanzaib ~

 

.

2 Comments »

  1. […] This task was required in relation to CGNAT logging. you may want to read it here […]

    Like

    Pingback by Syslog-ng – Part 3: Minimized logging to mysql with dynamic tables & trimming | Syed Jahanzaib Personal Blog to Share Knowledge ! — January 8, 2020 @ 1:28 PM


RSS feed for comments on this post. TrackBack URI

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: