Syed Jahanzaib – Personal Blog to Share Knowledge !

August 24, 2016

Radius Manager Connection Tracking System for Mikrotik

Filed under: Mikrotik Related, Radius Manager — Tags: , , — Syed Jahanzaib / Pinochio~:) @ 10:35 AM

CGNAT is another method to track users if you are NATTING private IP addresses in your network. Read following

https://aacable.wordpress.com/2020/01/01/mikrotik-cgnat/

trackme

PTA LETTER for log recording


As requested by few OP, following is a short technical reference guide on how you can enable TCP/UDP connections in Mikrotik to be stored in Radius manager connection tracking database so that you can view user base connection tracking report. Recently PTA also issues letter to all operators to comply with the order as showed in above scanned image …

In my personal view, it’s not much useful and at low end hardware it creates bottleneck, but if you have good resources with some fast storage (like SSD or RAID-10) it is better to set it up.

CTS is best described in the manual by dmasoftlab manual as mentioned below

Radius Manager has a special feature: the Connection Tracking System. It is available only in Radius Manager CTS version or higher. With the help of it the system can track and log all the TCP and UDP connections for all registered (online) users.

By default when You install the CTS enabled version of Radius Manager, it will use the default CTS database (CONNTRACK). It is strongly recommended to use a separate database host for the CONNTRACK database, due to the enormous amount of data stored daily. It can be even a 100-500 MegaBytes (and in my personal experience it can grow as much as 3-5 GB on busy network, ZAIB) per day. Fast disks (like SSD in RAID 10 mode,  zaib) are also recommended to be able to seek and store the data in real time. Radius Manager periodically stores the traffic data to CONNTRACK database (typically in
every 5–60 seconds).


Mikrotik (6.x) Configuration to enable Firewall Logging to remote server (RM)

If you have already configured the radius manager, then the conntrack database also get configured via the installation script. Next step is to enable the firewall logging in the mikrotik router so that mikrotik can send the categorized TCP/UDP data to radius manager conntrack database.

In mikrotik, open terminal and issue following commands …

In below example, we have following IP scheme.

PPPoE users ip pool = 172.16.0.1-172.16.0.255
Radius Manager IP  = 101.11.11.254

# Better to use address list rather then defining ip range
/ip firewall filter add chain=forward src-address=172.16.0.1-172.16.0.255 protocol=tcp connection-state=new action=log
# You may want to EXCLUDE UDP packets to avoid excessive logging
/ip firewall filter add chain=forward src-address=172.16.0.1-172.16.0.255 protocol=udp connection-state=new action=log
#Change the Radius server IP
/system logging action
add name=rmctszaib remote=192.168.100.1 target=remote remote-port=4950
add topics=firewall action=rmctszaib
# Following line will avoid printing of logs on LOG window of TIK
set 0 action=memory disabled=no prefix="" topics=info,!firewall

If you don’t see any errors , you are good to Go.


RADIUS MANAGER SECTION:

1. Radius Manager should be licensed with CTS level. Basic license wont gonna work ..

2. Make sure that  RMCONNTRACK service is UP and running.To make sure it’s running , get its process by

ps aux |grep rmconntrack

and you should get result something like following

root@radius:/# ps aux |grep rmconntrack
root xxxx 0.0 0.0 xxxxxx xxx ? Ssl 13:22 0:00 /usr/local/bin/rmconntrack

if you see its running, proceed further , otherwise you may start it manually by

rmconntrack –x

Now login to RM ACP  >  Reports  >  Connection Report

Here you can get report for all or individual user.

As showed in the image below …

c1

c2


Regard’s
Syed Jahanzaib

1

46 Comments »

  1. Asalam O Alaikum Bhai, i am big fan of your posts, i always read it even i dont need it, recently got a project of Radius Manager after a long time CTS was the main requirement as govt do not allow open internet in public. wasted 2 days but above configuration was not working for me, its all my mistake that i tried to send logs without even a little research, i didnt start rmconntrack service on the RM side, didnt know that rmconntrack service needs to be started manually ..
    Please add below recommendations for careless guys like me ..

    1. Radius Manager should be licensed with CTS .. basic license wont work ..
    2. rmconntrack –x on server to start the service
    4. ps –aux | grep 4950 to check if its listening on port or not.

    Liked by 1 person

    Comment by Malik — August 24, 2016 @ 12:41 PM

    • RM CONNTRACK service do get starts automatically , there is no need to start it manually as you mentioned.
      However yes its a good point that one must ensure that rmconntrack service is UP and running.
      I have Updated the article accordingly.

      Liked by 1 person

      Comment by Syed Jahanzaib / Pinochio~:) — August 24, 2016 @ 1:18 PM

      • Thank you sir, i thought the same but mine was not started automatically maybe during the setup i did select soemething else, after trying for 2 days i went back to RM user manual and check CTS and got it started manually.
        shukria sir for your guidance you are our inspiration…. please check if we can achieve the same task with linux syslog-ng or rsyslog..

        Like

        Comment by Malik — August 25, 2016 @ 12:05 AM

  2. Sir one question, what if somewhere we dont have Radius Manager, can we do the same setup with a linux machine installed with syslog-ng and mikrotik sends all connection logs to it ? and syslog server give us all the usage logs the same way RM. ?

    Like

    Comment by Malik — August 24, 2016 @ 12:51 PM

  3. Hi Syed,

    Great article. But still one thing is missing which one I need most. On my Radius manager I configured CTS and its working well. The issue is On my server the space is very less and as per the radius manager setup we need to configure separate database server if the there is more client.

    So can you please do some research and help us to make separate database server for CTS.

    Like

    Comment by Anupam Pradhan — August 24, 2016 @ 2:24 PM

    • when you install radius manager, it asks for conntrack database host, you can configure it there.
      or if radius is already installed, then change the conntrack database name password and host in the radius manager CFG file, adn then on remote host, create the conntrack DB and table and assign proper rights. You need to read the manual in depth.

      Like

      Comment by Syed Jahanzaib / Pinochio~:) — August 24, 2016 @ 3:24 PM

  4. how does it work on cable docsis users. Do we see this logs correctly and is there a way that we can set it up without radius manager, something with linux.

    Liked by 1 person

    Comment by Blead Demt — August 24, 2016 @ 10:39 PM

  5. sir it’s working on Mikrotik 5.20v also

    Like

    Comment by ramanji neyuluk — August 29, 2016 @ 9:02 PM

  6. Dear Sir,

    Please share configuration for Authentication Log in radius manager in tools menu.

    dilip

    On Wed, Aug 24, 2016 at 11:05 AM, Syed Jahanzaib Personal Blog to Share Knowledge ! wrote:

    > Syed Jahanzaib / Pinochio~:) posted: ” As requested by few OP, following > is a short technical reference guide on how you can enable logging of > TCP/UDP connection tracking in Mikrotik to be sent to Radius manager > tracking database to acquire user base connection tracking report. In my > perso” >

    Like

    Comment by Dilip Saini — September 1, 2016 @ 8:20 AM

  7. Sir, a simple question, do you know if there a way to run a hotspot without internet?
    Best regards.

    Like

    Comment by int21 — September 8, 2016 @ 3:19 AM

    • hotspot requires DNS resolving in order to display the login page to user.
      so you should have at lease dns resolving access on mikrotik.

      Like

      Comment by Syed Jahanzaib / Pinochio~:) — September 18, 2016 @ 4:11 PM

  8. Good day Sir. Can Mikrotik Local Auth be done with Radius Manager as it can be done with Free Radius? If so how do I go about? I dont particularly want to run a seperate Free Radius for this. Your help will be much appreciated…

    Like

    Comment by Riaan Griesel — September 26, 2016 @ 10:55 PM

  9. Hello Sir.. According to your steps i am successfully installed Radius manager and Connection Tracking…. when user logged in connection tracking store data in Database Properly but i am not able retrieve in Admin Panel… its gives me System error… When i check my DB entries are there… Please help me

    Like

    Comment by Dev — October 14, 2016 @ 3:39 PM

    • Have you manage to solve this problem? I am stuck, please help 🙂

      Like

      Comment by Aleksandar Jezdic — August 20, 2019 @ 5:03 AM

  10. Hi Syed
    Great job , I have succefully installed Connection tracing , working very well, just that what you say on top this article , Hard drive is very quick filled full with data from tracing , can you let me know maybe how to add another HDD under system, and how to change locations to save Connections Tracing to new partitions, or maybe will be posible to save this Connections Tracing data on some external NAS drive.

    Like

    Comment by Tom — October 28, 2016 @ 3:18 PM

  11. Aoa Sir Jahanzaib, I want to configure Dynamic VLAN assignment for my LAN users. Please guide me that:
    1 – Can I configure Dynamic VLAN on RADIUS Manager 4?
    2 – Can I integrate RADIUS Manager 4 with Active Directory?

    Like

    Comment by Muhammad Adil — November 5, 2016 @ 2:34 PM

  12. Assalaam o Aalaikum sir
    i m asif or me mikrotik k bary sekhna chata hn plz…

    Like

    Comment by Rana Asif — November 7, 2016 @ 9:44 AM

  13. sir
    Incorrect key file for table ‘./conntrack/2016@002d12@002d16.MYI’; try to repair it pls help me for solve this error

    Like

    Comment by hardik — December 16, 2016 @ 9:59 PM

    • @hardik
      First check if you are OUT OF SPACE.
      If space is plenty, then goto the directory

      cd /var/lib/mysql/conntrack/
      service mysql stop
      myisamchk -r -v -f 2016@002d12@002d16.MYI

      once repair is done, start mysql and let me know if it solves the issue.

      Like

      Comment by Syed Jahanzaib / Pinochio~:) — December 19, 2016 @ 12:49 PM

  14. hello sir , If i delete a user from the admin panel of radius manager , then is his record deleted from the database also or not?
    i mean his connection report , ip details will also be deleted ???

    Like

    Comment by ricky — February 19, 2017 @ 1:35 PM

  15. Jazakallah khairan katsiran for all post

    Like

    Comment by arulrizal — May 30, 2017 @ 11:45 AM

  16. Sir my conntrack database size is getting bigger.I have taken backup now i want to make it empty. Can you pls help me

    Like

    Comment by Rahul — June 11, 2017 @ 4:29 PM

  17. […] radius manager we have option of CONNECTION TRACKING. Using Connection Tracking requires lot of disk space for local ISP. Recently I recommend many OP […]

    Like

    Pingback by Quick Notes on moving MySQL database(s) to new partition | Syed Jahanzaib Personal Blog to Share Knowledge ! — June 16, 2017 @ 5:00 PM

  18. Dear Sir,

    this is atik. i have MIkrotik Router. is there any way to maintain log which site client visited with Src MAC address,Src IP Address and destination URL without using webproxy. if possible please share configuration.

    Like

    Comment by Atikur Rahman — August 25, 2017 @ 9:12 PM

  19. Sir,
    1.I want to know that is there any option to view visited URL instead of destination IP.
    2.Conntrack doesn’t show mac of user. how can we save that.

    Thanks in advance.

    Like

    Comment by Sushil kumar — August 27, 2017 @ 4:54 PM

  20. Hi Syed,

    In my RM, conntrack is not showing any data. I ran rmconntrack –x and checked that data is received from router but its not getting stored on Mysql. Please help.

    Like

    Comment by Anupam Pradhan — August 31, 2017 @ 4:29 PM

  21. Sir I have completed syslog-ng logging system with the help of your blog. But syslog-ng only store limited users and admin activities. Then I have tried web proxy technique you have mentioned in your post and I successfully got result in Mikrotik/Winbox panel. But the problem is that , that I can’t send these logs to remote server database. I tried my best and work day and night but all in vain. Is it possible to attach these logs with syslog-ng? Or is it possible to store all logs in remote database? If yes, then HOW.

    One thing more sir, can I store conntrack data without RM?
    جزاک اللہ

    Like

    Comment by Muhammad Faryad — January 12, 2018 @ 6:07 AM

  22. sir plz help me how see client whos open website

    Like

    Comment by Aman Farooqi — April 24, 2018 @ 7:37 PM

  23. with radius manger

    Like

    Comment by Aman Farooqi — April 24, 2018 @ 7:37 PM

  24. Do we need to put firewall rule in every router that do pppoe or only the router that is doing nat to the internet?

    Thanks!!
    Pat

    Like

    Comment by patrickcasavant — May 1, 2018 @ 5:01 PM

    • all nases should have firewall rule

      Like

      Comment by Syed Jahanzaib / Pinochio~:) — May 2, 2018 @ 8:30 AM

      • If all my pppoe ip subnet are within 10.0.0.0/8 in multiple routers, dont this make it double logging to rmconntrack database, local router + internet router that do Nat translation?
        /ip firewall filter add chain=forward src-address=10.0.0.0/8 protocol=tcp connection-state=new action=log
        /ip firewall filter add chain=forward src-address=10.0.0.0/8 protocol=udp connection-state=new action=log
        I would like to enable FastTrack on inside network router…
        Thank’s

        Like

        Comment by Patrick — May 11, 2019 @ 8:02 PM

  25. Many thank’s for your helpful post, My issue is, is there any way to clear that log where created in radius manager database ?

    Like

    Comment by katem07 — June 24, 2019 @ 4:31 AM

    • what particular logs you want to clear?
      you can perform function by
      1) manually login to mysql and clear particular table logs
      2) automate the process using bash script

      Like

      Comment by Syed Jahanzaib / Pinochio~:) — August 23, 2019 @ 10:48 AM

  26. Dear sir,where is the log file generally located?
    Regards

    Like

    Comment by jihad — June 26, 2019 @ 7:26 PM

  27. I get no data but in database is stored log… Dont know what is problem

    Like

    Comment by Alec — July 27, 2019 @ 4:28 PM

  28. Sir, the log is stored in database but not showing in connection report, no data found in my case.

    Like

    Comment by Aleksandar Jezdic — July 27, 2019 @ 8:14 PM

    • please check your radius license type.

      Like

      Comment by Syed Jahanzaib / Pinochio~:) — August 23, 2019 @ 10:47 AM

      • License is ok.. Database is storing data. But manager list only date and then it not get mysql data about connections..

        Like

        Comment by Aleksandar Jezdic — September 16, 2019 @ 9:36 PM

      • I don’t know is it something with some permission or what, not get it.

        Like

        Comment by Aleksandar Jezdic — September 16, 2019 @ 9:44 PM

  29. Jazak Allah,
    1). its running perfectly, my problem is below command is not working, please share alternate command for avoid printing.
    # Following line will avoid printing of logs on LOG window of TIK
    set 0 action=memory disabled=no prefix=”” topics=info,!firewall
    2). i wand to keep only 3 month data of customer please share settings

    Like

    Comment by Usman Haider — May 13, 2020 @ 1:41 PM

    • 1) use “/system logging action” branch tree.
      2) you have to make custom script that should run on daily basis & delete record/table older then X DAYS !

      Like

      Comment by Syed Jahanzaib / Pinochio~:) — October 1, 2020 @ 11:52 AM

  30. Hi, I have problem with Radius CTS I do not no how to change connection tracking to another hdd. I Have 8 TB hdd and I want to save connection tracking that. Do you have some tutorial? TNX

    Like

    Comment by Milos — February 27, 2021 @ 4:59 PM

  31. […] can delete single or multiple table older than X time from the mysql DB. It was pretty useful for DMASOFTLAB RADIUS MANAGER CONNTRACK table OR customized  SYSLOG-NG logging system, where table is daily created automagically in […]

    Like

    Pingback by MySQL: DROP tables older than X Period using BASH Script | Syed Jahanzaib Personal Blog to Share Knowledge ! — September 10, 2021 @ 1:33 PM


RSS feed for comments on this post. TrackBack URI

Leave a comment