Syed Jahanzaib Personal Blog to Share Knowledge !

November 26, 2015

Linux bash script for IP monitoring status / SMS alerting via KANNEL sms Gateway

Filed under: Linux Related — Tags: , — Syed Jahanzaib / Pinochio~:) @ 12:10 PM

net_sms_details

Bash script to check any HOST IP status status & send alert via kannel sms, while preventing repeated alert for same status.

#!/bin/bash
# Linux BASH Script to check any HOST/Internet Status & send SMS alert #(trigger one time for Per status changed)
# Syed Jahanzaib
# Created: 26-NOV-2015
#set -x
HOSTNAME=`hostname`
COMPANY="zaib"
DATE=`date`
TEMP="temp"
HOST1="$1"
# Temporary file holder for host status
HOST1_STATUS="/$TEMP/$1.ping.status"
# Create temp file if not already present, usually for 1st time execution
touch $HOST1_STATUS
# SMS RELATED and KANNEL INFO
# KANNEL SMS Gateway Info
# KANNEL SECTION
KURL="127.0.0.1:13013"
KID="kannel"
KPASS="KANNEL_PASS"
CELL1="03333021909"
FOOTER="Powered by $COMPANY"
MNT="10"
# How many PING attempts
PING_ATTEMPTS="10"
# SMS Messages for UP / DOWN
SMS_DOWN="$COMPANY ALERT: $DATE
$HOST1 not responding to $PING_ATTEMPTS ping request. Check connectivity
$FOOTER"
SMS_UP="$COMPANY INFO: $DATE
$HOSTNAME: $HOST1 is reachable now. OK
$FOOTER"
SMSDOWNHOLDER="/$TEMP/$HOST1.ping.down.sms"
SMSUPHOLDER="/$TEMP/$HOST1.ping.up.sms"

###########################################################
# CHECKING IF SYSTEM IS JUST RESTARTED, TO AVOID SENDING SMS
upSeconds="$(cat /proc/uptime | grep -o '^[0-9]\+')"
upMins=$((${upSeconds} / 60))
if [ "${upMins}" -lt "$MNT" ]
then
echo "System BOOT time is less then $MNT Minutes, so nothing to do, ignoring and exiting the SCRIPT"
exit 1
fi
if [ ! -d "/$TEMP" ]; then
echo
echo
echo "/$TEMP folder not found, Creating it so all ping results should be saved there . . ."
mkdir /$TEMP
fi

# Run the script
echo -e "Trying to ping - $HOST1 / $PING_ATTEMPTS times"
for HOST in $HOST1
do
count=$(ping -c $PING_ATTEMPTS $HOST | awk -F, '/received/{print $2*1}')
if [ $count -eq 0 ]; then
echo "$HOST is down."

if [ $(grep -c "$HOST" "$HOST1_STATUS") -eq 1 ]; then
echo "SMS alert have already been sent ..."
fi

if [ $(grep -c "$HOST" "$HOST1_STATUS") -eq 0 ]; then
echo "$HOST is down (ping failed) at $(date) .. SENDING DOWN SMS ..."
echo "$SMS_DOWN" > /$TEMP/$HOST1.down.sms
echo "$HOST" >> $HOST1_STATUS
# Sending DOWN SMS via KANNEL
cat /$TEMP/$HOST1.down.sms| curl "http://$KURL/cgi-bin/sendsms?username=$KID&password=$KPASS&to=$CELL1" -G --data-urlencode text@-
fi
else
echo "$HOST is alive"
if [ $(grep -c "$HOST" "$HOST1_STATUS") -eq 1 ]; then
DATE=`date`
echo "$HOST is now up (ping ok) at $(date)... SENDING UP SMS ..."
echo "$SMS_UP" > /$TEMP/$HOST1.up.sms
# Sending UP SMS via KANNEL
cat /$TEMP/$HOST1.up.sms| curl "http://$KURL/cgi-bin/sendsms?username=$KID&password=$KPASS&to=$CELL1" -G --data-urlencode text@-
sed -i "/$HOST/d" "$HOST1_STATUS"
fi
fi
done

Schedule it to run after every x minutes in cron , example:


*/1 * * * * /temp/net_mon.sh 8.8.8.8 >/dev/null 2>&1

 

Regard’s
Syed Jahanzaib

BASH: Scheduled script to check linux service status and sms/email while preventing repeated alerts

Filed under: Linux Related — Tags: , — Syed Jahanzaib / Pinochio~:) @ 12:00 PM

BOYSERVICECHECK

sms

MAIL1

Following is linux base bash script which can be scheduled to run every X minutes and it can do following …

Personnel Note:

This script was made for an operator who was facing strange problem that in random timings , the mysql or some times radiusd service stopped automatically, unfortunately the error was not sorted out as it occurs in awkward timings when no one is at NOC, so I made an workaround in the form of this script so that we can diagnose the issue later in details and in the meanwhile manually startup of services should not be required. The script was scheduled to run after every 10 minutes.

Note: This is not a solution, Just a workaround for small period of time.

This script will check one or more services status, if found stopped, it will try to re-start them, and then check there status again and output there status in email.

In this example I took MYSQL and RADIUS services.

Its much easier to use some centralized network monitoring system like Mikrotik DUDE or NAGIOS  or likewise , but as we know that every system have its known limitations, and sometime it is un necessary to setup a network monitoring system just to monitor single PC or service.
In such situation where resources are limited OR you want some thing very customized solution of your own choices with your required bells and whistles ,  its recommended to do it with simple bash without needing any 3rd party tool.

 

PSEUDO CODE:

  • Check for Service status example mysqld
  • If it found it running, then do nothing. Exit
  • If it found it STOPPED, it will send you SMS alert for down status (one time only until next status change) via KANNEL sms gateway. It will also gonna try to start the service one time.
  • If it found the service running on next run, it will send service UP info via sms. (one time only until next status change)
  • Email Alerts for UP n DOWN added on 4th Dec, 2015

SCRIPT:

mkdir /temp
touch /temp/checksrv.sh
chmod +x /temp/checksrv.sh
nano /temp/checksrv.sh

Now paste the following



#!/bin/bash
# Scheduled Script to check linux service status after every 5 minutes.
# If found stopped, send sms or email Alerts, but donot repeat it untill next status change.
# Script Designed by Syed Jahanzaib
# aacable at hotmail dot com
# https://aacable.wordpress.com
# Created : 25-NOV-2015
# Last Modified = 4th-NOV-2015 1600 hours
# Pakistan !!!
#set -x

#SRV="$1"
SRV1="$1"
DATE=`date`

STATUS_HOLDER="/tmp/$SRV1.txt"
touch $STATUS_HOLDER

# COMPANY NAME
COMPANY="ZAIB"

# Hostname
HOSTNAME=`hostname`

# KANNEL SMS Gateway Info
KANNELURL="KANNELIP:13013"
KANNELID="kannel"
KANNELPASS="kannelpass"
CELL1="03333021909"

# GMAIL ACCOUNT INFORMATION AND RELATED ....
MAILER="/temp/sendEmail-v1.56/sendEmail"
TO1="aacable at hotmail dot com"
GMAILID="YOURGMAILID@gmail.com"
GMAILPASS="YOURGMAILPASSWORD"
MAILBODYDOWN="$SRV1 DOWN"
MAILBODYUP="$SRV1 UP"
MAILSUBJECTDOWN="$COMPANY ALERT: $HOSTNAME - $SRV1 is now DOWN @ $DATE"
MAILSUBJECTUP="$COMPANY INFO: $HOSTNAME - $SRV1 is now UP @ $DATE"


# Colors Config  . . . [[ JZ . . . ]]
ESC_SEQ="\x1b["
COL_RESET=$ESC_SEQ"39;49;00m"
COL_RED=$ESC_SEQ"31;01m"
COL_GREEN=$ESC_SEQ"32;01m"
OS="1"
osver=`cat /etc/issue |awk '{print $1}'`

# OS checkup for UBUNTU
if [[ $osver == Ubuntu ]]; then
echo -e "$COL_GREEN OS = Ubuntu $COL_RESET"
set OS="Ubuntu"
OSPKG="apt-get install -y chkconfig"

else

echo -e "$COL_GREEN OS = CENTOS $COL_RESET"
set OS="CENTOS"
OSPKG="yum -y install chkconfig"
fi

########################################

# Check if no service name is given
if [ "$SRV1" == "" ]; then
echo -e "$COL_RED No service name have been provided. $COL_RESET"
echo "Usage exmaple:"
echo -e "/temp/checksrv.sh mysqld"
echo "OR"
echo -e "/temp/checksrv.sh (Depend on your OS deployment)"
echo
exit 0
else

# Check if CHKCONFIG command is installed or not.
CHK=`which chkconfig`
if [ "$CHK" == "" ]; then
echo -e "$COL_RED CHKCONFIG command is not installed.
Please install it with following command

$OSPKG $COL_RESET"
exit 0
else

# Check if service is installed / valid or not
CHKSRV=`chkconfig | grep -w $SRV1`
if [ "$CHKSRV" == "" ]; then
echo -e "$COL_RED NO SERVICE is INSTALLED WITH $SRV1 NAME. Exiting ...$COL_RESET"
exit 0
else

#########################################
############## IF ALL OK, PROCEED FURTHER
#########################################

# Query Service UPTIME
QUERYSRVUPTIME=`ps -eo "%U %c %t" |grep "$SRV1" |grep -v grep| grep root`
uptime=`echo $QUERYSRVUPTIME |awk '{ print $NF }'`

SERVICE1="$SRV1"
SUBJECT="ALERT: $SRV1 is Down..."

# SMS Msgs test for up n down
MSG_UP="$COMPANY Info:

$HOSTNAME - $SERVICE1 is now UP @ $DATE.
$SRV1 UPTIME is $uptime.
Powered by Syed.Jahanzaib"
fi

MSG_DOWN="$COMPANY Alert:

$HOSTNAME - $SERVICE1 is now DOWN @ $DATE. Trying to restarting it. wait 1 mnt for next result.

Powered by Jz"

##### CHECK FOR SERVICE STATUS

for SRVCHK in $SERVICE1
do
PID=$(pgrep $SERVICE1)
if [ "$PID" == "" ]; then
echo -e "$COL_RED $SRVCHK is down $COL_RESET "
if  [ $(grep -c "$SRVCHK" "$STATUS_HOLDER") -eq 0 ]; then
echo -e "$COL_RED ALERT: $SERVICE1 is down at $(date) / trying to restart and SENDING SMS and EMAIL....$COL_RESET"
echo "$MSG_DOWN" > /tmp/$SERVICE1_down.sms

# Sending DOWN SMS via KANNEL
cat /tmp/$SERVICE1_down.sms | curl "http://$KANNELURL/cgi-bin/sendsms?username=$KANNELID&password=$KANNELPASS&to=$CELL1" -G --data-urlencode text@-

# Sending DOWN alert EMAIL via GMAIL
$MAILER -t $TO1 -u "$MAILSUBJECTDOWN" -s smtp.gmail.com:587 -xu $GMAILID -xp $GMAILPASS -f $GMAILID -o tls=yes -o message-file=/tmp/$SERVICE1_up.sms

# Start $SERVICE1 service if found down
service $SERVICE1 start
echo "$SRVCHK" >> $STATUS_HOLDER
fi

else
echo -e "$COL_GREEN $SRVCHK is alive with Uptime of $uptime
and its PID are as follows... $COL_RESET  \n$PID"

# IF Service found UP send SMS ONE TIME
if  [ $(grep -c "$SRVCHK" "$STATUS_HOLDER") -eq 1 ]; then
echo -e "$COL_GREEN INFO ALERT : $SERVICE1 is UP at $(date)
$SRV1 Uptime is $uptime
SENDING SMS and EMAIL.... $COL_RESET"
echo "$MSG_UP" > /tmp/$SERVICE1_up.sms

# Sending UP SMS via KANNEL
cat /tmp/$SERVICE1_up.sms | curl "http://$KANNELURL/cgi-bin/sendsms?username=$KANNELID&password=$KANNELPASS&to=$CELL1" -G --data-urlencode text@-
sed -i "/$SRVCHK/d" "$STATUS_HOLDER"

# Sending UP alert EMAIL via GMAIL
$MAILER -t $TO1 -u "$MAILSUBJECTUP" -s smtp.gmail.com:587 -xu $GMAILID -xp $GMAILPASS -f $GMAILID -o tls=yes -o message-file=/tmp/$SERVICE1_up.sms


fi
fi
done
fi
fi

USAGE:

/temp/checksrv.sh mysql
#OR
/temp/checksrv.sh squid

As showed in the image below …

SMS RESULT


cmd_details

EMAIL RESULTS:

FOR EMAIL ALERTS,

2- Download sendEmail tool to send email alerts

Install sendEmail Tool

tar zxvf sendEmail-v1.56.tar.gz
cd sendEmail-v1.56/

ADD SUPPORTING LIBRARY

for ubuntu

apt-get -y install libio-socket-ssl-perl libnet-ssleay-perl perl

for centos

yum -y install perl perl-Crypt-SSLeay perl-IO-Socket-SSL

 

EMAIL CONFIGURATION AND SAMPLE RESULTS OF RECEIVING

Try to send email using command line , make sure you fill the variables with your original values like mail address password etc: Example


/temp/sendEmail-v1.56/sendEmail -t $TO1 -u "Test Email" -s smtp.gmail.com:587 -xu $GMAILID -xp $GMAILPASS -f $GMAILID -o tls=yes -m "hi"

 

MAIL1

MAIL2

SCHEDULE the SCRIPT in CRON:

Open crontab in editor

crontab -e

Now add following

# Run CHECK SERVICE script after very 5 minutes
*/5 * * * * /temp/checksrv.sh mysqld
OR
*/5 * * * * /temp/checksrv.sh mysql

SAVE and Exit!


 

Regard’s

Syed Jahanzaib

%d bloggers like this: