Part # 1 – Howto Save Mikrotik/Cisco Logs to Remote SYSLOG Server >> You are here
Part # 2 – Centralized Syslog-ng logging to MySql DB
Part # 3 Minimized logging to mysql with dynamic tables & trimming
First Published Date: Nov 29, 2011 @ 11:58
Last Modified : Nov 9, 2017
In some situations, you may want to save your mikrotik router or CISCO switches logs (or web proxy logs) for record / tracking purpose regarding Mikrotik activity. In most countries it is required by the law as well to keep record of users public IP assignment, like when you will apply for LICENSE, it is required to have such record at your disposal. Its much better from management point of view to intercept mikrotik info using external Linux base logs server.
This post demonstrate how to send Mikrotik logs to remote Ubuntu/Linux base syslog server. We will use SYSLOG-NG package in this example.
SYSLOG Server = 192.168.100.1 [OS > Ubuntu 12.4 32 bit]
Mikrotik Server = 192.168.100.2
First We will configure Mikrotik section
# MIKROTIK CONFIGURATION
In Mikrotik, Open Terminal & paste the following.
Currently we are storing this type of information only from the mikrotik to syslog
- pppoe info like login/logout/peer not responding/
- winbox login
- sysinfo , like rules disabled/enabled
/system logging action set 3 remote=192.168.100.1 add name=ZLogServer remote=192.168.100.1 target=remote /system logging add action=ZLogServer topics=pppoe,ppp,info add action=ZLogServer topics=system,info
[Note: 192.168.100.1 is Linux syslog server ip, Change this ip to match your remote syslog server ip. You can modify the topics as per your requirement, just an example below]
That’s it for Mikrotik 🙂 Now moving to Linux Section, in this example I used Ubuntu 12.4 You can use any other flavor of your choice.
# UBUNTU 12.4 CONFIGURATION
First we have to install the syslog server. In this example we are using syslog-ng log server.
Install various packages including syslog-ng server, phpmyadmin, mysql server to store logs in DB[if required], supporting libraries etc.
apt-get -y install apache2 mc wget make gcc mysql-server mysql-client curl phpmyadmin libdbd-pgsql aptitude libboost-system-dev libboost-thread-dev libboost-regex-dev syslog-ng libmongo-client0 libesmtp6 syslog-ng-mod-sql libdbd-mysql
During installation of above packages it may ask you to enter mysql/phpmyadmin password, do so as required. Once all the packages are installed, edit the syslog-ng file by
After installation, edit its configuration file available in /etc/syslog-ng.conf
Use the following command to edit config file.
nano /etc/syslog-ng/syslog-ng.conf
Now paste following lines before SOURCES section.
# Accept connection on UDP source s_net { udp (); }; # MIKROTIK ########### # Add Filter to add our mikrotik filter f_mikrotik { host( "192.168.100.22" ); }; log { source ( s_net ); filter( f_mikrotik ); destination ( df_mikrotik ); }; destination df_mikrotik { file("/var/log/zlogs/${HOST}.${YEAR}.${MONTH}.${DAY}.log" template-escape(no)); };
As shown below …
Now Save & Exit.
IMPORTANT:
- Create ‘zlogs‘ folder in /var/log and file also, so that mikrotik logs will be saved in separate file.
mkdir /var/log/zlogs
Restart the syslog-ng service to apply changes
service syslog-ng restart
Monitoring the LOGS
Now check the file name in /var/log/mikrotik and monitor it by tail command
tail -f /var/log/zlogs/#HOST.YYMMDD.log
At mikrotik , perform any action, for example open ‘New Terminal‘ OR try to add any new rule, you will see its logs in the tail output.
For example.
DONE !
LOG ROTATE !
As we have successfully managed to add the new log file to the system, it is crucial that we must configure log rotation to move / delete older logs otherwise it may fill the disk quickly if its heavily used production system.
To add log rotation edit the syslog-ng configuration file.
nano /etc/logrotate.d/syslog-ng
and add following in the start or before end . . . .
[This will rotate log files on daily basis, it will compress the last day log file. useful if you have receive some heavy logs from the devices.
Note: change folder names as required
/var/log/mikrotik/*.log { daily rotate 90 missingok compress notifempty missingok sharedscripts /etc/init.d/syslog-ng restart endscript # invoke-rc.d syslog-ng reload > /dev/null }
Save & Exit. and reload the syslog-ng service
service syslog-ng restart
Explanation of above code.
- daily the logrotation for mikrotik log in /var/log/mikrotik/mikrotik.log file will be don eon daily basis. this value describes the interval of rotation
- rotate 90 means syslog will keep 90 log file. [number of files]
- compress log file will be compressed using the gzip format
- missingok avoids halting on any error
- notifempty will not rotate log file if its empty
‘size‘ parameter is also very important setting if you want to control the sizing of the logs for heavy production server.
A configuration setting of around 50 MB would look like:
size 50M
Note that If both size and rotation interval are set, then size will override rotation parameter
Change SYSLOG Log Rotation Time
By default log.rotate starts at 6:47am in the morning. To change it to run in midnight, edit file
/etc/crontab
and change the cron.daily line to following
0 0 * * * root test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.daily )
This will run it in mid night. 🙂
DELETE LOG FILES with ZERO SIZE
You may need this ; )
find /var/log/mikrotik/ -name 'mikrotik*' -size 0 -print0 | xargs -0 rm
DELETE LOG FILES OLDER THEN 90 DAYS
This one too ; )
find /var/log/mikrotik/* -daystart -mtime +90-type f -exec rm {} \;
Or better to use complete script as defined here.
Centralized log server to store multiple devices logs
Updated: 9-NOV-2017
In a situation where you want to have centralized log server to log multiple devices logs with separate files, you can use following..
# MIKROTIK ########### # Accept connection on UDP source s_net { udp (); }; # Add Filter to add our mikroti filter f_mikrotik { host( "101.11.11.1" ); }; filter f_mikrotik2 { host( "101.11.11.2" ); }; filter f_ciscoswnoc { host( "101.11.11.3" ); }; filter f_ciscosw2 { host( "101.11.11.4" ); }; # Add destination file where logs will be stored, for each host destination d_mikrotik { file("/var/log/mikrotik/$HOST.mikrotik.${YEAR}.${MONTH}.${DAY}.log"); }; destination d_mikrotik2 { file("/var/log/mikrotik/$HOST.mikrotik.${YEAR}.${MONTH}.${DAY}.log"); }; destination d_ciscoswnoc { file("/var/log/mikrotik/$HOST.ciscosw.${YEAR}.${MONTH}.${DAY}.log"); }; destination d_ciscosw2 { file("/var/log/mikrotik/$HOST.ciscosw.${YEAR}.${MONTH}.${DAY}.log"); }; log { source(s_net); filter(f_mikrotik); destination(d_mikrotik); }; log { source(s_net); filter(f_mikrotik2); destination(d_mikrotik2); }; log { source(s_net); filter(f_ciscoswnoc); destination(d_ciscoswnoc); }; log { source(s_net); filter(f_ciscosw2); destination(d_ciscosw2); };
Make sure to restart syslog-ng server
service syslog-ng restart
Check SYSLOG-NG error
/usr/sbin/syslog-ng -F -p /var/run/syslogd.pid
Salam Alykum
Regard’s
Syed Jahanzaib
i do that but get error
root@Squid:~# apt-get install syslogd
Reading package lists… Done
Building dependency tree
Reading state information… Done
Package syslogd is a virtual package provided by:
sysklogd 1.5-6ubuntu1
busybox-syslogd 1:1.18.4-2ubuntu2
You should explicitly select one to install.
E: Package ‘syslogd’ has no installation candidate
??????
LikeLike
Comment by Ma7mod — April 18, 2012 @ 10:37 AM
Which version of Ubuntu you are using ?
Try
sysklogd 1.5-6ubuntu1
LikeLike
Comment by Syed Jahanzaib / Pinochio~:) — April 18, 2012 @ 12:42 PM
Assalam O Aalikum Sir tabiat kaise ha apki sir mujhe Mikrotik 3.3 ki keygen.npk chaye sir mane bohat dhonda par mila ni ap mujhe sent kar den(arsalanmalick2010@hotmail.com)
LikeLike
Comment by Arsalan Malick — July 6, 2012 @ 7:48 PM
Sent!
LikeLike
Comment by Syed Jahanzaib / Pinochio~:) — July 9, 2012 @ 11:29 AM
i cant get log from mkv5.5.My syslog server installed on centos 5.5.
Syslog server configuration info:-
#yum install syslog* -y.
#vim /etc/syslog.conf
###########syslog################
!*
+192.168.1.1 ### MK IP ###
local0.* /var/log/mt.log
#vim /var/log/mt.log
#chmod 700 /var/log/mt.log
#/etc/init.d/syslog restart
#chkconfig syslog on
but get no result on /var/log/mt.log
LikeLike
Comment by imran — September 27, 2012 @ 10:51 AM
hmmm i have no idea about centos, but in ubuntu, it works for sure.
LikeLike
Comment by Syed Jahanzaib / Pinochio~:) — September 28, 2012 @ 9:08 AM
i configure syslog server in buntu 12.04.1 LTS. but still now i cant get any log in syslog server. what parameter should i check
LikeLike
Comment by imran — October 14, 2012 @ 2:11 PM
Nice write up. How can I set up a central Ubuntu syslog server that will accept incoming ‘logs’ from remote Mikrotiks that could potentially be at any IP address? I’m good on the Mikrotik side but am lost at how to set up syslog.conf to accept the incoming logs. Any tips?
LikeLiked by 1 person
Comment by jimstolz76 (@jimstolz76) — November 30, 2012 @ 4:51 AM
COOL! Thank a lot!!
LikeLike
Comment by Jenk Za — December 4, 2012 @ 2:01 PM
Seep Terimaksih om
http://mr-ekoapriadi.blogspot.com/2013/05/tutorial-clear-log-disk-mikrotik.html
LikeLike
Comment by adam setiawan — May 12, 2013 @ 10:09 AM
Thanks for sharing.
LikeLike
Comment by Syed Jahanzaib / Pinochio~:) — May 13, 2013 @ 8:38 AM
someone make a facebook fakei Id on my network, How can I recognize the ip address where from login/created ?? Fake ID make a greate troble for our family.
LikeLike
Comment by Qamar — July 23, 2013 @ 10:50 AM
Its really hard to track such activity specially if your clients are using private ip scheming. How will you track that what facebook ID is created at which dates? it would be like searching for a needle in a sea. go through all the logs you have and see what clients have logged into facebook registration URL, and do blank guesses, but still its very very hard.
LikeLike
Comment by Syed Jahanzaib / Pinochio~:) — July 25, 2013 @ 7:54 AM
Dear Sir,
I tried a lot but all in vain …be low is my configuration of Mikrotik and Ubuntu
system logging action print
Flags: * – default
# NAME TARGET REMOTE
0 * memory memory
1 * disk disk
2 * echo echo
3 * remote remote 27.116.59.60
[bb@MikroTik] > system logging print
Flags: X – disabled, I – invalid, * – default
# TOPICS ACTION PREFIX
0 * info memory
1 * error memory
2 * warning memory
3 * critical echo
4 !async remote
========================================
I have add lines at the end
!*
+180.94.86.214
local0.* /var/log/mt.log
LikeLike
Comment by Muammad Kazim — January 4, 2014 @ 4:07 PM
Thanks for nice article.
Could you please give us idea how to log user’s activity (url visited) with username (hotspot user with radius accounting) along with IP address?
Best regards
LikeLike
Comment by Inder P. MEEL — February 7, 2014 @ 6:36 PM
Dear Sir,
I found your articles extremely useful.
I want to know.. Is it possible to log mikrotik hotspot (without proxy) user’s url visit activity (like sarg) and data transfferd?
I created syslog as per your article, but it don’t log username, (only log IP address).
Best regards.
Inder P. MEEL
LikeLike
Comment by Inder P. MEEL — February 11, 2014 @ 9:34 AM
Thank you..
LikeLike
Comment by chris — June 10, 2014 @ 4:00 PM
[…] source:https://aacable.wordpress.com/2011/11/29/howto-save-mikrotik-logs-to-remote-syslog-server/ […]
LikeLike
Pingback by Howto Save Mikrotik Logs to Remote SYSLOG Server | Supriyono pages — June 26, 2015 @ 2:10 PM
Dear Friends,
witch syslog server is best????????
LikeLike
Comment by bharat — August 7, 2015 @ 8:20 AM
[…] https://aacable.wordpress.com/2011/11/29/howto-save-mikrotik-logs-to-remote-syslog-server/(GFWed注意) […]
LikeLike
Pingback by windows上的log服务器 | Ferrets家的WordPress — August 16, 2015 @ 9:04 PM
hey sir, you r missed “k” at “apt-get install syslogd”. so u can can change it be “apt-get install sysklogd”
LikeLike
Comment by Joko Fani Andrinto — December 21, 2015 @ 11:28 AM
thx mate
LikeLike
Comment by Syed Jahanzaib / Pinochio~:) — December 21, 2015 @ 7:11 PM
Hi, Thanks for the tutorial, can you advise any way to segregate logs like accesslog, browsing log all in different file name?
LikeLiked by 1 person
Comment by Sankar — August 15, 2016 @ 1:19 PM
Hi Sir,
is there any way to record the connections (TCP/UDP) established by Broadband users to linux server for ISP record
LikeLike
Comment by Pawandeep Singh Behgal — September 5, 2016 @ 9:52 PM
Yes there can be by using syslog.
You also need to make rules at mikrotik to send tcp/udp connection details.
just for info purposes , see this post.
LikeLike
Comment by Syed Jahanzaib / Pinochio~:) — September 18, 2016 @ 4:14 PM
Can you share the working of connection tracking system radius manager to create our own? Just need some knowledge about that so that I can create my own cts like radius manager? I think it uses IPtables commands in shell script and create idle threads for mysql connection and execute queries. Please share if know something about that!
LikeLike
Comment by Anuj Upadhyay — March 23, 2019 @ 10:05 PM
Dear Sir i want To view Internet users complete history of web site search downloading and uploading log view and reporting pls help me
LikeLike
Comment by Imran Ali — March 9, 2017 @ 12:09 PM
Dear Sir i am use Mikrotik 5.20 in pc and i want To view Internet users complete history of web site search downloading
and uploading log view and reporting pls help me
LikeLike
Comment by Imran Ali — March 9, 2017 @ 12:11 PM
you can use squid proxy as addon to route http traffic to squid which can keep the user logs it self.
OR enable web proxy in mikrotik. then send its log to remote syslog server.
but there is no simple way of search in such scenarios,
LikeLike
Comment by Syed Jahanzaib / Pinochio~:) — March 10, 2017 @ 8:19 AM
I need log with website name, how i can do it ?
LikeLike
Comment by Forhad — April 24, 2017 @ 11:18 PM
[…] assigning users with public ip, then OP should record users public ip assignment only like showed here, but as private natted IP are still being used therefore OP should save users traffic as per law […]
LikeLike
Pingback by Quick Notes on moving MySQL database(s) to new partition | Syed Jahanzaib Personal Blog to Share Knowledge ! — June 16, 2017 @ 5:00 PM
assalamualaiykum, jazak Allah khair, for your posts, i have been used your posts for several year now… above example show a single remote host to add to the syslog.conf.. how to add several more remote mikrotik
LikeLike
Comment by Abdul Razak Ameen — September 27, 2017 @ 1:58 PM
Article updated for multiple devices as well.
LikeLike
Comment by Syed Jahanzaib / Pinochio~:) — November 9, 2017 @ 10:20 AM
How to see logs via browser as following video link ?
I have Ubuntu Server ubuntu-16.04.1 64-bit on (1tb Hard Drive)
& I Have about 35 Mikrotik Devices with different public IP’s.
Currently using DMA Radius Manager’s CTS module.
But problem is the Radius Manager is tracking visited IP addresses.
syslog-ng can track URL.
Please give detailed configuration for this scenario.
Thanks in advance.
my e-mail id is: j@fuzn.in
LikeLike
Comment by Momin Jafar Ali S. — October 29, 2017 @ 1:28 PM
[…] continuation to existing post related to syslog-ng, Following post describes on how you can push syslog logs entries to mysql DB for easy access and […]
LikeLike
Pingback by Centralized Syslog-ng logging to MySql DB | Syed Jahanzaib Personal Blog to Share Knowledge ! — November 10, 2017 @ 11:59 AM
Hi sir . I want to log about 500 mikrotik router logs on syslog and i can’t write all ips in syslog config . how can i configure syslog for 500 router without write one by one?
and somthing else . i can’t recive any log even for one mikrotik . I think port 514 is close . how can i open it in ubuntu 14.04 ?
LikeLiked by 1 person
Comment by sepidar2010 — December 18, 2017 @ 10:48 AM
you can make a simple bash script that can take ips from simple text file and feed it in the syslog config file. pretty easy to make such script.
LikeLike
Comment by Syed Jahanzaib / Pinochio~:) — December 23, 2017 @ 10:44 PM
Dear; I want to store logs to secondary hard disk. What should I do ? It would be pleasure if you reply.
LikeLike
Comment by Kazi Sala Uddin — December 25, 2017 @ 7:03 PM
you want to save Syslog-NG logs to local files or in Mysql ?
if you want to save logs in mysql & want them to be on secondary disk, see following …
Quick Notes on moving MySQL database(s) directory to New Partition
LikeLike
Comment by Syed Jahanzaib / Pinochio~:) — December 26, 2017 @ 4:37 PM
[…] Part # 1 – Howto Save Mikrotik/Cisco Logs to Remote SYSLOG Server […]
LikeLike
Pingback by Syslog-ng – Part 3: Minimized logging to mysql with dynamic tables & trimming | Syed Jahanzaib Personal Blog to Share Knowledge ! — January 8, 2020 @ 1:27 PM
Thanks Surajit!
LikeLike
Comment by jembatan baja — June 14, 2020 @ 6:01 PM
Dear Mr.syed,
Thanks for the post,, my config based on your tutorial works good for a single device, but for multiple device when i restart the service , it is giving error, i created the corresponding path aslo as per the config for multiple device, kindly advice
Job for syslog-ng.service failed because the control process exited with error code.
See “systemctl status syslog-ng.service” and “journalctl -xe” for details.
LikeLike
Comment by Abdul Razak Ameen — November 26, 2020 @ 6:27 PM