Syed Jahanzaib Personal Blog to Share Knowledge !

November 29, 2011

Howto Save Mikrotik Logs to Remote SYSLOG Server


log-title

First Published Date: Nov 29, 2011 @ 11:58
Revision Date : May 15, 2016 @ 1300 hours


In some situations, you may want to save your mikrotik router logs (or web proxy logs) for record / tracking purpose regarding Mikrotik activity. In most countries it is required by the law as well to keep record of users public IP assignment, like when you will apply for LICENSE, it is required to have such record at your disposal. Its much better from management point of view to intercept mikrotik info using external Linux base logs server.

This post demonstrate how to send Mikrotik logs to remote Ubuntu/Linux base syslog server. We will use SYSLOG-NG package in this example.

SYSLOG Server  =  192.168.100.1   [OS > Ubuntu 12.4 32 bit] 
Mikrotik Server = 192.168.100.2

First We will configure Mikrotik section


# MIKROTIK CONFIGURATION

In Mikrotik, Open Terminal & paste the following.

/system logging action
set 0 memory-lines=100
set 1 disk-file-count=30 disk-file-name=MT-log-zaib disk-lines-per-file=500
set 3 remote=192.168.100.1
# 192.168.100.1 is the remote syslog-ng server we will configure in second step.

# Now we will add few topics that we want to be stored in syslog server.zaib
/system logging
add action=remote topics=critical
add action=remote topics=error
add action=remote topics=info
add action=remote topics=warning

[Note:  192.168.100.1 is Linux syslog server ip, Change this ip to match your remote syslog server ip. You can modify the topics as per your requirement, just an example below]

 

log1
log2

That’s it for Mikrotik 🙂 Now moving to Linux Section, in this example I used Ubuntu 12.4 You can use any other flavor of your choice.


# UBUNTU 12.4 CONFIGURATION

First we have to install the syslog server. In this example we are using syslog-ng log server.

Install syslog-ng package by

apt-get install syslog-ng

After installation, edit its configuration file available in /etc/syslog-ng.conf

Use the following command to edit config file.

nano /etc/syslog-ng/syslog-ng.conf

Now paste following lines before SOURCES section.

# Accept connection on UDP
source s_net { udp (); };

# MIKROTIK ###########
# Add Filter to add our mikroti
filter f_mikrotik { host( "192.168.100.2" ); };
# Add destination file where logs will be stored
#destination df_mikrotik { file("/var/log/mikrotik.log"); };
log { source ( s_net ); filter( f_mikrotik ); destination ( df_mikrotik ); };
destination df_mikrotik {
 file("/var/log/mikrotik/mikrotik.${YEAR}.${MONTH}.${DAY}.log"
# template("${HOUR}:${MIN}:${SEC} ${HOST} ${MSG} ${MSG}\n")
 template-escape(no));
};

As shown below …

syslog

Now Save & Exit.


IMPORTANT:

  • Create ‘mikrotik‘ folder in /var/log and file also, so that mikrotik logs will be saved in separate file.
mkdir /var/log/mikrotik

Restart the syslog-ng service to apply changes

service syslog-ng restart

Monitoring the LOGS

Now check the file name in /var/log/mikrotik and monitor it by tail command

tail -f /var/log/mikrotik/mikrotik.log

At mikrotik , perform any action, for example open ‘New Terminal‘ OR try to add any new rule, you will see its logs in the tail output.

For example.


log

DONE !


LOG ROTATE !

As we have successfully managed to add the new log file to the system, it is crucial that we must configure log rotation to move / delete older logs otherwise it may fill the disk quickly if its heavily used production system.

To add log rotation edit the syslog-ng configuration file.

nano /etc/logrotate.d/syslog-ng

and add following in the start or before end . . . .

[This will rotate log files on daily basis, it will compress the last day log file. useful if you have receive some heavy logs from the devices.

/var/log//mikrotik/*.log {
 daily
 rotate 90
 missingok
 compress
 notifempty
 missingok
 sharedscripts
 /etc/init.d/syslog-ng restart
 endscript
# invoke-rc.d syslog-ng reload > /dev/null
}

Save & Exit. and reload the syslog-ng service

service syslog-ng restart

Explanation of above code.

  • daily the logrotation for mikrotik log in /var/log/mikrotik/mikrotik.log file will be don eon daily basis. this value describes the interval of rotation
  • rotate 90 means syslog will keep 90 log file. [number of files]
  • compress log file will be compressed using the gzip format
  • missingok avoids halting on any error
  • notifempty will not rotate log file if its empty

size‘ parameter is  also very important setting if you want to control the sizing of the logs for heavy production server.

A configuration setting of around 50 MB would look like:

size 50M

Note that If both size and rotation interval are set, then size will override rotation parameter


TIP: Log file name with Year-Date

If we want syslog to store mikrotik file in daily date year format file, then use

Example Config

# MIKROTIK ###########
# Add Filter to add our mikrotik
filter f_mikrotik { host( "192.168.100.2" ); };
# Add destination file where logs will be stored
#destination df_mikrotik { file("/var/log/mikrotik.log"); };
log { source ( s_net ); filter( f_mikrotik ); destination ( df_mikrotik ); };
destination df_mikrotik {
file("/var/log/mikrotik/mikrotik.${YEAR}.${MONTH}.${DAY}.log"
# template("${HOUR}:${MIN}:${SEC} ${HOST} ${MSG} ${MSG}\n")
template-escape(no));
};

log


Change SYSLOG Log Rotation Time

By default log.rotate starts at 6:47am in the morning. To change it to run in midnight, edit file

/etc/crontab

and change the cron.daily line to following

0 0     * * *   root    test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.daily )

This will run it in mid night. 🙂


DELETE LOG FILES with ZERO SIZE

You may need this ; )

find /var/log/mikrotik/ -name 'mikrotik*' -size 0 -print0 | xargs -0 rm

DELETE LOG FILES OLDER THEN 90 DAYS

This one too ; )

find /var/log/mikrotik/* -daystart -mtime +90-type f -exec rm {} \;

Or better to use complete script as defined here.


Take Care

Regard’s
Syed Jahanzaib

November 28, 2011

November 25, 2011

Howto Login on Remote Mikrotik & Linux without PASSWORD to execute commands using DSA key


~ Article by Syed Jahanzaib ~

By Following this guide , You will be able to Execute Scripts from a Remote Linux machine to Mikrotik RouterOS [OR Linux] without requiring password.


> SCENARIO# 1

Login From Linux to Mikrotik to execute commands via ssh without Password !!!

[STEP # 1]

First you need to generate public dsa key on your linux bx {which you will upload to mikrotik in later stage}.
At your Linux box, issue the following command.

ssh-keygen -t dsa

This will create a DSA key pair that is compatible with Mikrotik/Linux

 ssh-keygen -t dsa

It will ask you few questions, just press enter , as showed below…

root@zaib-desktop:~# ssh-keygen -t dsa
Generating public/private dsa key pair.
Enter file in which to save the key (/root/.ssh/id_dsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /root/.ssh/id_dsa.
Your public key has been saved in /root/.ssh/id_dsa.pub.
The key fingerprint is:
ed:da:88:da:d1:b1:f0:b5:f2:39:04:85:9d:d0:19:f1 root@zaib-desktop
The key's randomart image is:
+--[ DSA 1024]----+
|        .=o=     |
|        . *.     |
|         .  E    |
|        ..       |
|      . S.o      |
|       + =..     |
|      . =.o      |
|     . o *..     |
|    ..o o +.     |
+-----------------+
root@zaib-desktop:~#

Note: Make sure to leave the passphrase blank if you are going to be using this key in automated scripts. You do not want to be prompted for a password. zaib

 

[STEP # 2]
Now the key have been generated, It’s time to upload it to MIKROTIK using FTP. Make sure FTP service is enabled on mikrotik. Upload this id_dsa.pub key via ftp using below commands.

root@zaib-desktop:~# cd /root/.ssh/
root@zaib-desktop:~/.ssh# ftp 192.168.2.2
Connected to 192.168.2.2.

220 MikroTik FTP server (MikroTik 3.3) ready
Name (192.168.2.2:root): admin
331 Password required for admin
Password:
230 User admin logged in
Remote system type is UNIX.

ftp> put id_dsa.pub
local: id_dsa.pub remote: id_dsa.pub
200 PORT command successful
150 Opening ASCII mode data connection for '/id_dsa.pub'
226 ASCII transfer complete
608 bytes sent in 0.00 secs (2207.2 kB/s)

ftp> exit
221 Closing
root@zaib-desktop:~/.ssh#

OR

[STEP # 3] – MIKROTIK SECTION
Now login to Mikrotik via Winbox, and open Terminal , you need to import the key. to import key, use the below command.

user ssh-keys import file=id_dsa.pub
user: admin

The user field above determines which user account will be logged in when you pass the key, In this example , I am using default admin id.

All Done. You’ve created a key pair and imported the public key into Mikrotik ,

Now you can start running commands from your remote machine without using the password.

Some examples are below, from your Linux box, try the following . . .

(For the first time Login, It will ask you “Are you sure you want to continue connecting (yes/no)?” Type yes to continue)

ssh admin@192.168.2.2  /system resource print
The authenticity of host '192.168.2.2 (192.168.2.2)' can't be established.
DSA key fingerprint is 5f:d5:ee:51:8b:1c:c3:df:4d:3c:29:d8:af:48:35:a5.
Are you sure you want to continue connecting (yes/no)? yes

Again try to execute command and this time it will execute smoothly without asking any thing.

root@zaib-desktop:~# ssh admin@192.168.2.2  /system resource print
uptime: 40m37s
version: "3.3"
free-memory: 40512kB
total-memory: 62276kB
cpu: "Intel(R)"
cpu-count: 1
cpu-frequency: 3200MHz
cpu-load: 1
free-hdd-space: 956832kB
total-hdd-space: 1021408kB
write-sect-since-reboot: 2373
write-sect-total: 2373

OR

You can do so many interesting things using this method, you can link scripts with php or webmin and control your mikrotik / linux box with webmin as Frontend.


SCNEARIO # 2

Login From Linux to Linux to execute commands via ssh without Password !!!

Assumption:
[LINUX]  ADMIN PC IP  = 192.168.2.1
[LINUX]  REMOTE SERVER IP = 192.168.2.9

Suppose, We want to login from ADMIN PC to REMOTE SERVER without password , or we want to execute command from ADMIN PC to REMOTE SERVER.

[STEP # 1]

You have to first generate DSA public key on ADMIN PC.
You can create it by following [STEP # 1]  in Scenario # 1 of this post.
If you have already generated it, then skip this Step#1

[STEP # 2]

From Admin PC , issue the following command to upload id_dsa.pub to Remote Server.

scp id_dsa.pub root@192.168.2.9:.ssh/authorized_keys

[It will ask Remote Server Password, type password and hit enter.

Now try to Login to REMOTE SERVER using following command

ssh 192.168.2.9

root@zaib-desktop:~/.ssh# ssh 192.168.2.9
Linux test2-proxy 2.6.31-14-generic #48-Ubuntu SMP Fri Oct 16 14:04:26 UTC 2009 i686

To access official Ubuntu documentation, please visit:
http://help.ubuntu.com/

353 packages can be updated.
202 updates are security updates.

Last login: Fri Nov 25 03:01:45 2011 from 192.168.2.1
root@test2-proxy:~#

SUCCESS ! You are now able to Login to remote server without password.

You can Execute any command on remote server from admin pc, For example, you can shutdown / restart or whatever you like . . .

root@zaib-desktop:~/.ssh# ssh 192.168.2.9 df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/sda1             3.8G  2.1G  1.5G  59% /
udev                  186M  224K  186M   1% /dev
none                  186M  164K  186M   1% /dev/shm
none                  186M   88K  186M   1% /var/run
none                  186M     0  186M   0% /var/lock
none                  186M     0  186M   0% /lib/init/rw
root@zaib-desktop:~/.ssh#

November 16, 2011

DMASOFTLAB Radius Manager Backup Script !


LAST UPDATED:  3rd-Feb-2017

Following is an customized backup script for DMASOFTLAB Radius Manager database (radius) and other radius manager related files. It creates FULL Backup of all configuration files related to RM & mysql database (radius) in local storage, you can connect your USB or secondary HDD. You can also configure cloud like DROPBOX so that copy of backup should be copied to CLOUD as well, MOST RECOMMENDED.

NOTE: Using S.S.D disk (or raid) are highly recommended as there are less chances of failure for SSD disks, and above all there read/write rates are good plus if you use RAID system, then it will provide redundancy as well.

Modify the script as per your requirements …

Revision History:

[10th January, 2014]    Added various functions like script execution time Logs
[10th January, 2014]    Added LOG file to record all script activity in /var/log/fullbackup.log
[10th January, 2014]    Added Description for various tasks in the scripts
[10th January, 2014]    Howto section added on scheduling to run script on daily basis in night at 00:00 hours


Let’s Start …


 

First create temp folder and create script in it.
mkdir /temp
cd /temp
touch fullbackup.sh
chmod +x fullbackup.sh
touch /var/log/fullbackup.log

Now open the file

nano /temp/fullbackup.sh

and paste the following code.


#!/bin/bash
# Version 2.1 / 3-Feb-2017
# Syed Jahanzaib / Web: https://aacable.wordpress.com / Email: aacable@hotmail.com
# This script will create FULL Backup of MySQL DB (Radius) and RADIUS MANAGER related DATA files.
# We can modify it to do incremental basis backup too, but based on my personal experiences, I prefer to have FULL backup instead of incremental,
# Because you never know what you will going to need in case of disaster recovery
# Adjust below DATA fields accordingly. remove / add desired folders.

# Settings various VARIABLES for the script
# Colors Config . . . [[ JZ . . . ]]
ESC_SEQ="\x1b["
COL_RESET=$ESC_SEQ"39;49;00m"
COL_RED=$ESC_SEQ"31;01m"
COL_GREEN=$ESC_SEQ"32;01m"

# IF YOU HAVE FEDORA or CENTOS, Change the /var/www to /var/www/html/

# Following is FULL
#TARGET="/var/www/radiusmanager /sql_backup /etc /usr/local /var/lib/mysql"

# Following is only RADIUS HTML FILES, mysqldb is separate from this, will come later in the script with mysqldump
TARGET="/var/www/radiusmanager /etc/radiusmanager.cfg /sql_backup /temp /cfg"
SAVEDIR_FULL="/backup"

# For Dropbox , you must activate DROPBOX first. I post guide on it another post.
DROPBOX="/root/Dropbox"
sql_dir="/sql_backup"
radpwd="PASSWORD"

#MYSQL DETAILS
SQLUSER="root"
SQLPASS="PASSWORD"

#LOG FILE
LOG="/var/log/fullbackup.log"

#SET DATE TIME
set $(date)
time=`date |awk '{print $4}'`
YESTERDAY=`date --date='yesterday' +%Y-%m-%d`

rm -fr /var/www/radiusmanager/tmpimages/*
clear

GMAILID="GMAIL-ID@gmail.com"
GMAILPASS="GMAIL-PASSWORD"
ADMINMAIL1="ADMIN1@HOTMAIL.COM"
COMPANY="ZAIB (Pvt) Ltd."

# Set KANNEL Details if you want to send SMS
KHOST="127.0.0.1"
KID="kannel"
KPASS="KANNEL-PASSWORD"
CELL1="03333021909"
RMBACKUPSMSFILE="/tmp/rmbackupsms.txt"
> $RMBACKUPSMSFILE

#############################################
######## START the BACKUP PROCESS ... #######
#############################################

# Start counting start time
start_time=`date +%s`
echo -e "$COL_RED Welcome ! This is Radius Manager Backup Script, It will take backup of RADIUS Database and Folders Following ...
$TARGET
Backup started at $6-$2-$3 Time $time . . .

Powered by Syed.Jahanzaib $COL_RESET"

# Checking if $SAVEDIR_FULL folder is previously present or not . . .
{
if [ ! -d "$SAVEDIR_FULL" ]; then
echo
echo
echo -e "$COL_RED $SAVEDIR_FULL folder not found, Creating it so all backup's should be placed there . . . $COL_RESET"
mkdir $SAVEDIR_FULL
else
echo
echo -e "$COL_GREEN ********* $SAVEDIR_FULL folder is already present , so no need to create it, Proceeding further . . . $COL_RESET"
fi
}

# Checking if $sql_dir folder is previously present or not . . .
{
if [ ! -d "$sql_dir" ]; then
echo
echo
echo -e "$COL_RED $sql_dir folder not found, Creating it MSQL EXPORT/DUMP backup should be placed there . . . $COL_RESET"
mkdir $sql_dir
else
echo
echo -e "$COL_GREEN ********* $sql_dir folder is already present , so no need to create it, Proceeding further . . . $COL_RESET"
fi
}

# Creating MYSQL dump of databases
echo "++++++++++++++++" > $LOG
echo "Main Backup started at $6-$2-$3 Time $time" >> $LOG
echo " " >> $LOG
echo " " >> $LOG
echo
echo "******************************** MYSQL DUMP LOG *************" >> $LOG
echo -e "$COL_GREEN ********* Exporting MYSQL DUMP to $sql_dir ... $COL_RESET"
echo "++++++++++++++++" >> $LOG
echo
echo "Mysql SQL export started at $6-$2-$3 Time $time" >> $LOG
mysqldump -u$SQLUSER -p$SQLPASS radius > $sql_dir/mysql_db_full_$6-$2-$3.sql
echo "++++++++++++++++" >> $LOG
echo "Mysql SQL export ended at $6-$2-$3 Time $time" >> $LOG
echo "++++++++++++++++" >> $LOG
echo " " >> $LOG
echo " " >> $LOG

# GZIP MYSQL D.B
echo "******************************* GZIP LOG Section *************" >> $LOG
echo "GZIP command started at $6-$2-$3 Time $time" >> $LOG
gzip -fv $sql_dir/mysql_db_full_$6-$2-$3.sql >> $LOG
echo "++++++++++++++++" >> $LOG
echo "GZIP command ended at $6-$2-$3 Time $time" >> $LOG
echo "++++++++++++++++" >> $LOG
echo " " >> $LOG
echo " " >> $LOG
echo

# TAR GZIP (mysql)
echo -e "$COL_GREEN ********* TAR - Compressing all Backup Folders to $SAVDIR_FULL ... $COL_RESET"
echo
echo "******************************** TAR All DATA Section *************" >> $LOG
echo "TAR command strted at $6-$2-$3 Time $time" >> $LOG
FILENAME="dma_backup_by_zaib_$6-$2-$3"
tar cfzv $SAVEDIR_FULL/$FILENAME.tgz $TARGET >> $LOG

# COPY BACKUP TO 'DROPBOX' FOLDER, I have disabled it as not every1 may use it.
# cp $SAVEDIR_FULL/$FILENAME.tgz $DROPBOX

echo " " >> $LOG
echo "++++++++++++++++" >> $LOG
echo "TAR command ended at $6-$2-$3 Time $time" >> $LOG
echo "++++++++++++++++" >> $LOG

# Deleting sql db from $sql_dir because its zipped with the above command already and now all data available in single file : )
rm -fr $sql_dir/*
echo

# Print END time
echo "MAIN Backup ended at $6-$2-$3 Time $time"
echo "MAIN Backup ended at $6-$2-$3 Time $time" >> $LOG
echo
echo -e "$COL_GREEN ********* Backup completed to $SAVEDIR_FULL ... $COL_RESET"
echo
echo -e "$COL_RED Backup ended at $6-$2-$3 Time $time . . . $COL_RESET"
end_time=`date +%s`
echo
echo
echo " " >> $LOG
echo " " >> $LOG
echo " " >> $LOG

# Delete files older than 30 days, to prevent disk fillup
echo "Deleting Older files then 30 days, to save disk space . . ."

# echo Deleting Older files then 30 days, to save disk space >> $LOG
find $SAVEDIR_FULL/* -mtime +30 -exec rm {} \;

# Also delete files older than 340 day from dropbox folder
find $DROPBOX/* -mtime +20 -exec rm {} \;

# Print Complete Timings
echo MAIN Backup Completion Time was `expr $end_time - $start_time` s.
echo MAIN Backup Completion Time was `expr $end_time - $start_time` s. >> $LOG
echo

#STORE LAST FILE FILE NAME IN TEMP HOLDER
LASTFILE=`find $SAVEDIR_FULL -type f -mmin -10 > /tmp/lastbfile`
LASTFILESMS=`cat /tmp/lastbfile | sed -e 's/\/temp\///'`

# CHECK FILE SIZE AND COMPARE, IF ITS LESS , THEN ALERT
SIZE=`ls -lh $LASTFILESMS | awk '{print $5}'`
SIZEB=`ls -l $LASTFILESMS | awk '{print $5}'`
if [ $SIZEB -lt 1 ]
then
echo "FAILED FAILED FAILED"
RESULT="FAILED FAILED FAILED!!"
else
echo "SUCCESSFULL!"
RESULT="SUCCESSFULL"
fi
######## PRINT INFO SECTION #########
# Print Fetched Information on Screen , for info to see
echo "Last Backup File is $LASTFILESMS"
echo "GT RM INFO: Backup for $YESTERDAY is completed successfully.
File Name = $LASTFILESMS
Size = $SIZE

Backup Completion Time was `expr $end_time - $start_time` s.
$COMPANY
Powered by Syed Jahanzaib"

############## SMS SECTION ##############

# Send SMS
MSG="GT RM INFO: Backup OF Radius for $YESTERDAY is $RESULT.
File Name = $LASTFILESMS
Size = $SIZE

Backup Completion Time was `expr $end_time - $start_time` s.

$COMPANY
Powered by Syed Jahanzaib"

echo "GT RM INFO: Backup OF Radius for $YESTERDAY is $RESULT.
File Name = $LASTFILESMS
Size = $SIZE

Backup Completion Time was `expr $end_time - $start_time` s.

$COMPANY
Powered by Syed Jahanzaib" > $RMBACKUPSMSFILE

# Send SMS Alert via KANNEL
#echo "Sending SMS ALERT to $CELL1 & $CELL2 ..."
#echo ""
#curl "http://$KHOST:13013/cgi-bin/sendsms?username=$KID&password=$KPASS&to=$CELL1+$CELL2" -G --data-urlencode text@$RMBACKUPSMSFILE
#curl -G "http://localhost/test.php?sender=$SENDER&to=$MOBILE" --data-urlencode "message=$MSG"
############## EMAIL SECTION ##############

# Make sure you install sendEMAIL tool and test it properly before using email section.
#SEND EMAIL Alert As well using sendEMAIL tool using GMAIL ADDRESS.
# If you want to send email , use below ...
echo ""
echo "Sending EMAIL ALERT to $ADMINMAIL1 ..."
/temp/sendEmail-v1.56/sendEmail -t $email -u "GT RM Backup Info for $YESTERDAY / Size=$SIZE" -o tls=yes -s smtp.gmail.com:587 -t $ADMINMAIL1 -xu $GMAILID -xp $GMAILPASS -f $GMAILID -o message-file=$RMBACKUPSMSFILE -o message-content-type=text

# Print Credits : )
echo -e "$COL_GREEN ********* Syed Jahanzaib / aacable@hotmail.com / https://aacable.wordpress.com $COL_RESET"
echo Syed Jahanzaib / aacable@hotmail.com / https://aacable.wordpress.com >> $LOG

🙂

 


Now execute this script ,

After completion see the logs files

nano /var/log/fullbackup.log
OR
cat /var/log/fullbackup.log

Also Visit…

https://aacable.wordpress.com/2012/11/30/dmasoftlab-radius-manager-install-backup-restore-short-reference-guide/


https://aacable.wordpress.com/2014/01/04/automated-installation-script-for-radius-manager-v-4-0-44-1-4-with-latest-patch/


HOWTO SCHEDULE THE SCRIPT TO RUN ON DAILY BASIS

To run this file on daily basis (at 00:00 hours),
Open terminal, (make sure you are logged in with the root user.
type

crontab -e

(if it asks for text editor, select nano ,)

Now paste following code …

@daily  /temp/fullbackup.sh          # Run Daily in night at 00:00 hours

Save & Exit:

Now, based on above selection, cron job will run this command at selected scheduled timings and clear any memory cache


TIP: Remove files older than xx days

When you will implement this script, it will backup every day, and continue to do so, and its a good idea to remove backup files older then 1 month, to save disk space otherwise one day it will chew whole disk space 😀
You can add following in the same backup script at end, so that whenever it will execute backup script, it will delete older files too

echo Deleting Older files then 30 days, to save disk space
# echo Deleting Older files then 30 days, to save disk space  >> /var/log/fullbackup.log
find /backup/* -mtime +30 -exec rm {} \;


> SCRIPT   EXECUTION   RESULT …

 

b2

> EMAIL   ALERT   EXECUTION …

 

b1

> SMS   ALERT   EXECUTION …

 

2016-05-29 05.32.34


Regard’s
Syed Jahanzaib

November 12, 2011

Smokeping : Best tool to plot ping / latency graphs


Recently I was troubleshooting a network where concerned Admin complained that they frequently lost connectivity with multiple local server’s and also sometimes with the Internet. Sometimes pings replies works okay but latency gets high or timeout / breaks occurs. So I decided to setup mrtg base ping graph to monitor ping latency. The custom made mrtg ping probe worked fine and can provide an overview on target ping / rtt and Downtime in a nice manner, its sample can be viewed at http://www.billing.onmypc.net/mrtg/ . . .



BUT . . . . . . . . . . . . . . . . . . .

I was thinking far ahead , I was thinking for much more advanced latency and quality graphs which can show ping latency / rtt / loss in much more detailed way. I recalled my memory from old days when I used to monitor my old network with variety of tools and scripts and suddenly a name popped in my mind ” SMOKEPING ” , yes this was the tool I was looking for.

SmokePing generates graphs that can reveal the quality (packet loss and latency variability) & reachability of your IP address from several distributed locations. SmokePing is a network latency monitor. It measures network latency to a configurable set of destinations on the network, and displays its findings in easy-to-read Web pages. It uses RRDtool as its logging and graphing back-end, making the system very efficient. The presentation of the data on the Web is done through a CGI with some AJAX capabilities for interactive graph exploration.

In this article I will show you howto install smokeping on FEDORA 10.

HOWTO INSTALL SMOKEPING on FEDORA 10

Installing Smokeping is relatively simple. Just issue this command as root.

yum -y install smokeping

and It will install smokeping along with all other tools and dependencies it requires. It may download for about 18-20 Mb of data from internet, It may take few minutes depending on your internet connection and system speed.

NOTE:  selinux is the Problem. By default its enabled in Fedora installation. Disable it by editing

nano /etc/selinux/config

and change the enable to disable as following.

SELINUX=disable

(Restart your nix box.)

After installation completes, You can access it via

http://yourip/cgi-bin/sm.cgi

You will see something similar as shown in the following screenshot.


NOTE: If you get “Access Denied: You don’t have permission to access this Folder” error, edit the following line.

nano /etc/httpd/conf.d/smokeping.conf

and remove all lines in this file , and paste the following.

<Directory "/usr/share/smokeping">
order deny,allow
allow from 127.0.0.1
allow from all
</Directory>

<Directory "/var/lib/smokeping">
order deny,allow
allow from 127.0.0.1
allow from all
</Directory>

ScriptAlias /smokeping/sm.cgi  /usr/share/smokeping/cgi/smokeping.cgi
ScriptAlias /smokeping/tr.cgi  /usr/share/smokeping/cgi/tr.cgi
Alias       /smokeping/images  /var/lib/smokeping/images
Alias       /smokeping         /usr/share/smokeping/htdocs

Save & exit. Now restart apache web server by

service httpd restart

Now try to access the smokeping cgi, it will open properly.

http://yourip/cgi-bin/sm.cgi

Now It’s time to add your target in Smokeping for monitoring. The default location for the Smokeping config file is /etc/smokeping/config, Now just for example, we will edit this file.

 nano /etc/smokeping/config

Remove all the lines in it and paste the following lines

*** General ***
owner    = Syed Jahanzaib
contact  = aacable@hotmail.com
mailhost = smtp.ptcl.com
sendmail = /usr/sbin/sendmail
imgcache = /var/lib/smokeping/images
imgurl   = /smokeping/images
datadir  = /var/lib/smokeping/rrd
piddir   = /var/run/smokeping
cgiurl   = http://localhost/smokeping/smokeping.cgi
smokemail = /etc/smokeping/smokemail
tmail     = /etc/smokeping/tmail
syslogfacility = local0
*** Alerts ***
to = root@localhost
from = root@localhost
+someloss
type = loss
pattern = >0%,*12*,>0%,*12*,>0%
comment = loss 3 times  in a row
*** Database ***
step     = 300
pings    = 20
# consfn mrhb steps total
AVERAGE  0.5   1  1008
AVERAGE  0.5  12  4320
MIN  0.5  12  4320
MAX  0.5  12  4320
AVERAGE  0.5 144   720
MAX  0.5 144   720
MIN  0.5 144   720
*** Presentation ***

template = /etc/smokeping/basepage.html

+ charts

menu = Charts
title = The most interesting destinations

++ stddev
sorter = StdDev(entries=>4)
title = Top Standard Deviation
menu = Std Deviation
format = Standard Deviation %f

++ max
sorter = Max(entries=>5)
title = Top Max Roundtrip Time
menu = by Max
format = Max Roundtrip Time %f seconds

++ loss
sorter = Loss(entries=>5)
title = Top Packet Loss
menu = Loss
format = Packets Lost %f

++ median
sorter = Median(entries=>5)
title = Top Median Roundtrip Time
menu = by Median
format = Median RTT %f seconds

+ overview

width = 600
height = 50
range = 10h

+ detail

width = 600
height = 200
unison_tolerance = 2

"Last 3 Hours"    3h
"Last 30 Hours"   30h
"Last 10 Days"    10d
"Last 400 Days"   400d

*** Probes ***
+ FPing

binary = /usr/sbin/fping

*** Slaves ***
secrets=/etc/smokeping/smokeping_secrets
+boomer
display_name=boomer
color=0000ff

+slave2
display_name=another
color=00ff00

*** Targets ***
probe = FPing

menu = Top
title = Network Latency Grapher
remark = Welcome to the SmokePing website of <b>GLASSLINE (Pvt) Ltd.</b> <br> Here you will learn all about the latency of our network.<br><br><br><br><br> This page is maintained by Glassline. (Pvt) ltd . <br><br>Support Email: aacable@hotmail.com<br>Web: https://aacable.wordpress.com

### YOU CAN CHANGE THE FOLLOWING ACCORDING TO YOUR NETWORK ###

+ Ping

menu = WAN Connectivity
title = WAS Side Network

++ yahoo

menu = yahoo
title = yahoo ping report
host = yahoo.com

++ google

menu = google
title = Google ping report
host = google.com

### YOU CAN CHANGE FOLLOWING ACCORDING TO YOUR NETWORK ###
+ Ping2

menu = LAN Connectivity
title = LAN Side Network

++ Mikrotik

menu = Mikrotik
title = Mikrotik PPP ping report
host = 10.10.0.1

++ Proxy

menu = Proxy
title = Proxy Server ping report
host = 10.10.0.2

++ Billing

menu = Billing
title = Radius billing Server ping report
host = 10.10.0.2

Now restart smokeping process by

/etc/init.d/smokeping restart

Default update time for all ping probes are 5 Minutes. Try accessing the main smokeping page after 5-10 minutes so it can create necessary RRD for targets and update its graphs.

http://yourip/cgi-bin/sm.cgi

Now click on LAN connectivity to open graphs. You will see something similar as shown in the following screenshot.


Now, Click on WAN Connectivity

Now click on  the Yahoo Ping Graph showing on Right to get more detailed graph report.

To get more detailed view, click on Last 3 hours graphs, It will open this graph in NAVIGATOR mode. Now here you can select any specific area to zoom to troubleshoot at exact which time the latency problems occurs. For example in the following screenshot you can see the blue dots which shows that ping timeout occurs at specific time.

PULL DATA FROM SPECIFIC TIME

A client complained that the internet worked very slow in the morning at 10:30am. So you can pull data from smokeping graph to validate the complain or to troubleshoot what exactly happened at 10:30am 🙂

As you can see the blue dots showed that the ping breaks occured in the specific timings.

There are so many interesting things you can do with SMOKE-PING.

For some more examples , please visit the author web site @

http://oss.oetiker.ch/smokeping/index.en.html

Take Care, n ALLAH HAFIZ

Regard’s

Syed Jahanzaib
aacable@hotmail.com

November 5, 2011

Howto Protect a web folder on APACHE with Password

Filed under: Linux Related — Tags: , , , , , — Syed Jahanzaib / Pinochio~:) @ 12:19 PM

Protecting your private web folders is important to keep your file privates from unauthorized users. There are many ways you can password protect directories under Apache web server.

In order to create apache password protected directories you need to set follwing:

  • Password file (must be placed in a folder which must not be accessible via oute rworld.
  • And Directory name which you would like to password protect (/var/www/html/mrtg) (Example)

One of the simplest is to add the following to your Apache config file:

<Directory "/var/www/html/mrtg">
AuthType Basic
AuthName "My Personnel Folder - Not for General Public"
AuthUserFile "/var/www/htpasswd"
Require valid-user
</Directory>

Then create the htpasswdfile, and add at least one user to it (as root):

htpasswd -c /var/www/htpasswd username
It will ask for password, just add your desired one. This file should be outside of the directories available from the web.
More info can be found at http://www.cyberciti.biz/faq/howto-setup-apache-password-protect-directory-with-htaccess-file/

November 3, 2011

Server’s Monitoring Status Page a.k.a “SERVER STATUS”


I was looking for a web base server status tool to monitor my connectivity with the ISP and local servers , I did this in the past using simple Javascript which shows ON/OFF status for specific pc’s, But it was not fancy, I was looking for more simple n visually appealing tool/page, After some googling , I came across with “Server Status”  / http://rushland.net/projects.htm

Its a very good tool made in PHP/HTML along with MYSQL support. I used it to monitor WAN connectivity / Local Servers Live Status. It can be use to monitor services status of local or remote servers too. I modified it according to my requirements and the final result is as follows 🙂

Create a free website or blog at WordPress.com.

%d bloggers like this: