Syed Jahanzaib Personal Blog to Share Knowledge !

March 22, 2021

Modifying Expiration Time in Dmasoftlab Radius Manager

Filed under: Mikrotik Related — Tags: , — Syed Jahanzaib / Pinochio~:) @ 8:38 AM

This post is published as a personal reference, It describes a method via which you can modify the user account expiration default time of 00:00:00 to some other time, so that middle night disconnection can be avoided & user can get some time in official hours to recharge there account. with traditional FREERADIUS we can modify the disconnection in RADCHECK table, but since DMA doesn’t defines the expiration there & uses its own authentication module which checks for the user expiration date in RM_USERS table, therefore we have to make little modification to facilitate this option.

Another option is to create TRIGGERS, so that whenever a new user is created, it should modify the time instantly, or add the user name in separate table, and using predefined bash script which can monitor the table and perform action accordingly. lots of options can be opted.


As asked by few network OP’s who are using dmasoftlab radius manager as there billing system & performs manual recharge after getting payment from the users by door to door collection, One of the most annoying issue is the expiration time at which users get disconnected dueto expiration limit expires. By default when a user is created his time is entered in following format

2021-03-20 00:00:00

As a result, as soon as date changes at 00:00:00 hours, the user will be disconnected from the system. which means in the middle of night. Users starts calling OP help desk & most of the time, its difficult to recharge accounts in mid night.

Therefore its better to change the expiration time to happen in official working hours so that both user & OP can have some time to recharge the accounts within office hours.

To automate this process, create a bash script & schedule it to run at 11:50 pm daily. This script will change all users expiration HOURS to your modified time.

SCRIPT !

Create TEMP folder / script file along with executable permission

mkdir /temp
touch /temp/expmod.sh
chmod +x /temp/expmod.sh
nano /temp/expmod.sh

& paste following contents, make sure to change MYSQL user/password & required expiration hours

#!/bin/sh
# set -x
# BASH base script to change EXPIRATION hours in DMA RADIUS Manager RM_USERS table
# to modify users expiration disconnection time so that middle night disconnection can be avoided
# You can schedule this script to run every XX minutes/hours
# example : in cron use below line , means run at 11:50pm
# 50 23 * * * * /temp/expmod.sh
################################################
# By Syed Jahanzaib / aacable at hotmail dot com
# CREATED on : 20th-March-2021
################################################

# MYSQL related. MAKE SURE TO CHANGE THESE to MATCH YOUR LOCAL's
SQLUSER="root"
SQLPASS="zaib1234"
export MYSQL_PWD=$SQLPASS
CMD="mysql -u$SQLUSER --skip-column-names -s -e"
DB="radius"
# DMA related, below is 8pm. change timings as per your requirements
DEFAULT_TIME="00:00:00"
NEW_EXP_TIME="20:00:00"
COLUMN_NAME="expiration"
# R.M Table which contain users expiration information (in freeradius, we use Expiration attribute in radcheck,
# But DMA uses its own authentication module to validate users details from the rm_users table
USER_TABLE="rm_users"
# Date Related
DATE=`date`
TODAY=$(date +"%Y-%m-%d")

# Start execution
# Modify the 00:00:00 hours to suite yours local time, I have used 8pm timings as an example
$CMD "use $DB; UPDATE $USER_TABLE SET $COLUMN_NAME = DATE_FORMAT(expiration, '%Y-%m-%d $NEW_EXP_TIME');"

# or you can use single line code here in mysql directly or by $CMD
#UPDATE rm_users SET expiration = DATE_FORMAT(expiration, '%Y-%m-%d $NEW_EXP_TIME');"

# ECHO on screen and also LOG in /var/log/syslog (for ubuntu)
echo "DMASOFTLAB RADIUS MANAGER - User expiration HOURS now changed from $COLUMN_NAME to $NEW_EXP_TIME - Script executed successfully @ $DATE"
logger "DMASOFTLAB RADIUS MANAGER - User expiration HOURS now changed from $COLUMN_NAME to $NEW_EXP_TIME - Script executed successfully @ $DATE"
#Script Ends here

CRON Scheduler:

You can schedule it to run at 23:50 hours daily (this is the default time when dma expires account program runs, so we will modify the cron schedule to modify expiration time just before DMA program runs …

50 23 * * * /temp/expmod.sh

Regard’s
Syed Jahanzaib

March 5, 2021

Ubuntu Default 200GB Partition & it’s extension

Filed under: Linux Related — Tags: , , , — Syed Jahanzaib / Pinochio~:) @ 8:53 AM

Thanks Mr. GerardBeekmans for detailed guidance.

Scenario:

I have create one VM guest (on esxi) & 900GB disk is assigned to it. Ubuntu 16 server is installed with default installation options. But when I see disk report, it shows only 200 GB of disk space

as shown below …

root@XXX-log:/temp# df -h
Filesystem Size Used Avail Use% Mounted on
udev 7.8G 0 7.8G 0% /dev
tmpfs 1.6G 900K 1.6G 1% /run
/dev/mapper/ubuntu--vg-ubuntu--lv 196G 92G 94G 50% /
tmpfs 7.9G 8.0K 7.9G 1% /dev/shm
tmpfs 5.0M 0 5.0M 0% /run/lock
tmpfs 7.9G 0 7.9G 0% /sys/fs/cgroup
/dev/sda2 976M 146M 764M 16% /boot
tmpfs 1.6G 0 1.6G 0% /run/user/0
root@XXX-log:/temp#

But when I run FDISK or other tools, it shows below

root@XXX-log:/temp# fdisk -l
Disk /dev/sda: 920 GiB, 987842478080 bytes, 1929379840 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 2C824CE0-94E5-4515-B28D-8FA40983CFF5

Device Start End Sectors Size Type
/dev/sda1 2048 4095 2048 1M BIOS boot
/dev/sda2 4096 2101247 2097152 1G Linux filesystem
/dev/sda3 2101248 1929377791 1927276544 919G Linux filesystem

Disk /dev/mapper/ubuntu--vg-ubuntu--lv: 200 GiB, 214748364800 bytes, 419430400 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

Ubuntu DEFAULT regular installer does it by default. It’ll provision about a 200 GB LVM based Logical Volume (LV) for use. The rest of the space is not used until you decide what to do with it (assign to the existing root and extend it, or create additional volumes later).

If you use the alternative installer you get more advanced abilities regarding partitioning and can configure this right at the start vs. making changes after installation.

It looks like your /dev/sda3 is our LVM’s physical volume. Some commands to run to check for size details:

pvdisplay
vgdisplay

The first command will show you the physical partition details and the volume group (vg) that is attached to it. The second command will show you the volume group details. Check for “VG Size” and “Free PE / Size”.

If the VG itself already spans the entire LVM (ie around that 900 GB size of your actual disk) then you can simply expand the Logical Volume named /dev/mapper/ubuntu–vg-ubuntu–lv and afterwards expand the filesystem on top of it.

If you simply want that single LV to take up the entire volume group’s space without needing to create more volumes for now, this should get the trick done:

lvresize -l +100%FREE /dev/mapper/ubuntu--vg-ubuntu--lv
resize2fs /dev/mapper/ubuntu--vg-ubuntu--lv

& afterwards DF showed correct space.

root@bbi-log:~# df -h
Filesystem Size Used Avail Use% Mounted on
udev 7.8G 0 7.8G 0% /dev
tmpfs 1.6G 900K 1.6G 1% /run
/dev/mapper/ubuntu--vg-ubuntu--lv 904G 94G 771G 11% /
tmpfs 7.9G 8.0K 7.9G 1% /dev/shm
tmpfs 5.0M 0 5.0M 0% /run/lock
tmpfs 7.9G 0 7.9G 0% /sys/fs/cgroup
/dev/sda2 976M 146M 764M 16% /boot
tmpfs 1.6G 0 1.6G 0% /run/user/0
root@bbi-log:~#

Thanks Mr. GerardBeekmans for detailed guidance.


These steps were based on default Ubuntu behaviour but your setup may be different.

Regard’s
Syed Jahanzaib

March 2, 2021

Bash Script for General Customizable Report via Email

Filed under: Linux Related — Tags: , — Syed Jahanzaib / Pinochio~:) @ 12:37 PM

Note for MySelf:

This post contains bash script sample , which upon executed, can query various system components & send the report via email. Useful to monitor remote server. Further functions can be added or existing can be customized according to the requirements. I opted for LOOP Formula to show mysql DB sizes in MB/GB using IF ELSE statements & some other fun stuff for myself as well.

The script is bit messy & scrambled in terms of proper organized display, but it works fine. You may customized or trim as per your taste

Feel free to use as you like …

Regard’s
Syed Jahanzaib


#!/bin/sh
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/etc
#set -x
# Version 1.1 / 10th January, 2014
# Last Modified / 5th-MARCH-2021
# Syed Jahanzaib / Web: https://aacable.wordpress.com / Email: aacabl AT hotmail DOT com
# This script generalized & customized DISK reports and email to admin
# Adjust below DATA fields accordingly. remove / add desired tasks.
# Settings various VARIABLES for the script
clear
# Colors Config ... [[ JZ ... ]]
COMPANY="ZAIB_LTD"
CREDITS="Powered by Syed Jahanzaib / 0333.3021.909 / aacable at hotmail dot com / https:// aacable . wordpress .com"
#MYSQL DETAILS
SRV="mysql"
SQLUSER="root"
SQLPASS="XXXXXX"
export MYSQL_PWD=$SQLPASS
CMD="mysql -uroot --skip-column-names -e"
ALL_DB_TEMP_LIST="/tmp/mysq_all_dbs.txt"
DB1="radius"
DB2="conntrack"
DB3="syslog"
SQL_ACCOUNTING_TABLE="radacct"
CMD="mysql -u$SQLUSER --skip-column-names -s -e"
EMAILMSG="/tmp/report1.log"
DB_HOLDER="/temp/temp_db_size_holder.log"
> $EMAILMSG
> $DB_HOLDER
HOSTNAME=`hostname`
INT_IP1=`hostname -I`
INT_IP2=`ip route get 1 | awk '{print $NF;exit}'`
EXT_IP=`dig +short myip.opendns.com @resolver1.opendns.com`
URL="google.com"
DNS=$(cat /etc/resolv.conf | sed '1 d' | awk '{print $2}')
# Check OS Type
os=$(uname -o)
###################################
# Check OS Release Version and Name
###################################
OS=`uname -s`
REV=`uname -r`
MACH=`uname -m`
GetVersionFromFile()
{
VERSION=`cat $1 | tr "\n" ' ' | sed s/.*VERSION.*=\ // `
}
if [ "${OS}" = "SunOS" ] ; then
OS=Solaris
ARCH=`uname -p`
OSSTR="${OS} ${REV}(${ARCH} `uname -v`)"
elif [ "${OS}" = "AIX" ] ; then
OSSTR="${OS} `oslevel` (`oslevel -r`)"
elif [ "${OS}" = "Linux" ] ; then
KERNEL=`uname -r`
if [ -f /etc/redhat-release ] ; then
DIST='RedHat'
PSUEDONAME=`cat /etc/redhat-release | sed s/.*\(// | sed s/\)//`
REV=`cat /etc/redhat-release | sed s/.*release\ // | sed s/\ .*//`
elif [ -f /etc/SuSE-release ] ; then
DIST=`cat /etc/SuSE-release | tr "\n" ' '| sed s/VERSION.*//`
REV=`cat /etc/SuSE-release | tr "\n" ' ' | sed s/.*=\ //`
elif [ -f /etc/mandrake-release ] ; then
DIST='Mandrake'
PSUEDONAME=`cat /etc/mandrake-release | sed s/.*\(// | sed s/\)//`
REV=`cat /etc/mandrake-release | sed s/.*release\ // | sed s/\ .*//`
elif [ -f /etc/os-release ]; then
DIST=`awk -F "PRETTY_NAME=" '{print $2}' /etc/os-release | tr -d '\n"'`
elif [ -f /etc/debian_version ] ; then
DIST="Debian `cat /etc/debian_version`"
REV=""
fi
if ${OSSTR} [ -f /etc/UnitedLinux-release ] ; then
DIST="${DIST}[`cat /etc/UnitedLinux-release | tr "\n" ' ' | sed s/VERSION.*//`]"
fi
OSSTR="${OS} ${DIST} ${REV}(${PSUEDONAME} ${KERNEL} ${MACH})"
fi
# Check Architecture
architecture=$(uname -m)
# Check Kernel Release
kernelrelease=$(uname -r)
#SET DATE TIME
set $(date)
time=`date |awk '{print $4}'`
DT=`date +%d.%b.%Y_time_%H.%M`
DATE=$(date +%Y-%m-%d)
DT_HMS=$(date +'%H:%M:%S')
FULL_DATE=`date`
TODAY=$(date +"%Y-%m-%d")
TODAYYMD=`date +"%d-%b-%Y"`
#Get ip which have default route
logger General report has been started @ $DATE / $DT_HMS
# Check FREERADIUS online sessions
#SESSIONS=`$CMD "use radius; SELECT username FROM $SQL_ACCOUNTING_TABLE WHERE acctstoptime IS NULL;" |wc -l`
# Adding OS level Details in email message
# modify below disk name we want to monitor, make sure to change this
DISK="/dev/sda2"
DISKTOT=`df -h $DISK |awk '{print $2}'| sed -n 2p`
DISKUSED=`df -h $DISK |awk '{print $3}'| sed -n 2p`
DISKAVA=`df -h $DISK |awk '{print $4}'| sed -n 2p`
DISKUSEPER=`df -h $DISK |awk '{print $5}'| sed -n 2p`
MEMTOT=`free -m |awk '{print $2}'| sed -n 2p`
MEMUSED=`free -m |awk '{print $3}'| sed -n 2p`
MEMAVA=`free -m |awk '{print $4}'| sed -n 2p`
MEMUSEDPER=`free -m | grep Mem | awk '{print $3/$2 * 100.0}'`
MEMAVAPER=`free -m | grep Mem | awk '{print $4/$2 * 100.0}'`
#GMAIL Details
GMAILID="XXXX@gmail.com"
GMAILPASS="XXXXX"
TO1="aacableXhotmail.com"
SMTP="smtp.gmail.com:587"
#Collect all data in file
echo "
General Report for $HOSTNAME - $INT_IP - $EXT_IP

===============
NETWORK DETAILS:
===============

HOST: $HOSTNAME
Operating System Type $os
$OSSTR
Architecture : $architecture
Kernel Release : $kernelrelease
INT_IP1: $INT_IP1
INT_IP2: $INT_IP2
EXT_IP: $EXT_IP
DNS: $DNS

================
MYSQL DB REPORT:
================
" >> $EMAILMSG
# Fetch ALL DB's & calculate there sizes and convert sizes in MB/GB
MYSQLALLDB=`$CMD "show databases;" > $ALL_DB_TEMP_LIST`
num=0
cat $ALL_DB_TEMP_LIST | while read database
do
num=$[$num+1]
DB=`echo $database | awk '{print $1}'`
MYSQLDBSIZE=`$CMD "SELECT table_schema '$DB', sum(data_length + index_length)/1024/1024 FROM information_schema.TABLES WHERE table_schema='$DB' GROUP BY table_schema;" | cut -f1 -d"." | sed 's/[^0-9]*//g'`
if [ "$MYSQLDBSIZE" -ge 1024 ]; then
MYSQLDBSIZE_FINAL=`echo "scale=2; $MYSQLDBSIZE/1024" |bc -l`
echo "$DB / $MYSQLDBSIZE_FINAL GB" | column -t >> $DB_HOLDER
fi
if [ "$MYSQLDBSIZE" -le 1024 ]; then
MYSQLDBSIZE_FINAL=`echo "scale=2; $MYSQLDBSIZE" |bc -l`
echo "$DB / $MYSQLDBSIZE_FINAL MB" | column -t >> $DB_HOLDER
fi
done
cat $DB_HOLDER | column -t >> $EMAILMSG
echo "
=============
Disk Details:
=============
" >> $EMAILMSG
df -h |grep sda2 | column -t >> $EMAILMSG
echo "

==============
MEMORY_REPORT:
==============
Total_RAM = $MEMTOT MB
Total_RAM_Used = $MEMUSED MB
Total_RAM_Available = $MEMAVA MB
Total_RAM_Used_Percent = $MEMUSEDPER %
Total_RAM_Available_Percent = $MEMAVAPER %
" > /tmp/temp_memory_report.log

cat /tmp/temp_memory_report.log | column -t >> $EMAILMSG

echo "
$CREDITS
" >> $EMAILMSG
# PRINT INFO SECTION #########
# Print Fetched Information on Screen , for info to see
cat $EMAILMSG
# EMAIL SECTION ##############
# Make sure you install sendEMAIL tool and test it properly before using email section.
#SEND EMAIL Alert As well using sendEMAIL tool using GMAIL ADDRESS.
# If you want to send email , use below ...
echo " - Sending SMS/EMAIL info ..."
#curl "http://$KHOST/cgi-bin/sendsms?username=$KID&password=$KPASS&to=$CELL2+$CELL3+$CELL4" -G --data-urlencode text@$SMSMSG
sendemail -u "$HOSTNAME - $EXT_IP - General Report- $DATE " -o tls=yes -s $SMTP -t $TO1 -xu $GMAILID -xp $GMAILPASS -f $GMAILID -o message-file=$EMAILMSG -o message-content-type=text
# log entry in /var/log/syslog
logger General Report have been end @ $DATE / $DT_HMS

Make sure to install BC to calculate size

apt-get -y install bc

Sample snapshot for Email Reporting !


Howto install ‘sendemail’ tool to send email via Gmail ID.

Very Well with Tested For UBUNTU 12.x , may work on other ubuntu versions too

Quick copy paste …

1
2
apt-get -y install libio-socket-ssl-perl libnet-ssleay-perl perl
apt-get -y install sendemail

 

February 28, 2021

netElastic vBNG


Pending Post …

For Mikrotik base ISP’s

As most small scale ISP’s are using mikrotik which itslef is very user friendly, & easy to manage even for a beginner admin , it also provides greater level of control/visibility/tracing, But it maxes out on 2000-3000 users (CCR series). Probably Mikrotik is not aimed for enterprise/large level market in terms of features & scalability. I have read that BNG is written from scratch, therefore its scalability is far better. Maybe in coming time, principle will add good visibility in upcoming BNG versions because visibility & controlling is not very good at vBNG. So far as per customer reviews, BNG have outclass mikrotik routers in terms of scalability/CPU resource control . In general ,Mikrotik routers can support 2000-3000 ppp sessions with hurdles, whereas vBNG can support upto 128k sessions (depends on the model & hardware/clustering). Best part of BNG is user modular base pricing.

VBNG is costlier then Mikrotik But cheaper than other big name brands ! It’s worth trying …

What is vBNG

(vBNG) is a high-performance (3rd party proprietary) software (or hardware appliance) router that can run on any server system from any x86 vendor. vBNG supports all common features such as PPPoE and IPoE, subscriber traffic policing and shaping, and CG-NAT. Full routing protocol support includes MPLS, OSPF, BGP, and others.

netElastic vBNG has two main components, the Control Plane (CP) and the Data Plane (DP).
The Data Plane moves packets in and out, applying QoS and other policies along the way. It’s sizing is based on how much bandwidth is needed and uses CPU Cores, network interfaces, and RAM.
The Control Plane communicates with management tools, manages policies, establishes and updates routing tables for the Data Plane, handles AAA and most other functions that aren’t involved with actual packet forwarding. The Control Plane sizing is generally based on the complexity of the use case and number of subscribers expected on the vBNG.

Deployment Options

vBNG can be deployed in several different manners described below.

Host Mode on Bare Metal – Entire vBNG running directly on a server. This is the most common option and uses the least amount of resources, avoiding virtualization layer overhead.
Host Mode in a Virtual Machine – Entire vBNG running in one VM. This is a good option for smaller vBNGs going into environments with existing virtualization capabilities and eliminates the need for a dedicated server

 

Click on following document link for further elaboration

VBNG Server and VM Sizing for vBNG


vBNG Single Box Test Results:

Server Hardware specs:

  • Dell R730
    64 GB RAM
    12 Cores processor x 2 @ approx 2.4 GHz
    500 GB x 2 HDD (Raid 1)
    2 ports x 2 Ten Gigabit Fibre Network Cards (Bonding 2 Port WAN, 2 Port LAN)

Server Software specs:

  • Centos 7.x on bare metal
    On TOP of centos, virtual machine is created for vBNG

On 2500+ active pppoe users , CPU utilization ratio was under 5%.

 

 

CPU:


vBNG Models/Packages: Redirecting …

Look for following URL to see the feature comparison,

BNG Packages

Case Study:

Harbour Isp Case Study:

Click to access Harbour-ISP-Case-Study.pdf


Adding vBNG DICTIONARY in Freeradius

To add additional/3rd party dictionaries in freeradius, first copy the dictionary file in /usr/share/freeradius folder.

then edit the file DICTIONARY file in /usr/share/freeradius/dictionary

nano /usr/share/freeradius/dictionary

& add the dictionary file location in the end of this file

Example File:

ZAIB #### 15-FEB-2021
#### Add VBNG NETELASTIC support in Freeradius as well
$INCLUDE dictionary.netelastic-2019q3

Save the file & exit the editor. Now Reload the freeradius service

service freeradius reload
#OR
service freeradius restart

Freeradius Attributes for vBNG

You can use the following attribute to assign profile on vBNG. this profile example 1Mbprofile must be configured on the vBNG first.

FR3 attribute:

Attribute: NetElastic-Qos-Profile-Name  | Op := | Value 1Mbprofile

Disconnect user on vBNG via Freeradius RADCLIENT

# Send DISCONNECT REQUEST TO NAS FOR SPECIFIC USERS, Modify the parameters as per your local config
# 
echo user-name=USERNAME | radclient -x 192.168.0.1:3799 disconnect SECRET

& in return you should get

Received Disconnect-ACK Id 168 from 192.168.x.x:3799 to 192.168.x.x:43014 length 20



vBNG manuals

vBNG manuals / infor & dictionary files are available at my gdrive , @ following link

https://drive.google.com/drive/folders/1DhJ1FhEaOaF9ok-zqcLoN96AHzBEKPnX

 


SALES Inquiries …

Fro Global:

sales@netelastic.com

For Pakistan region, following reseller can be contact @ [you may refer this blog]

 

 

 


Regards

Syed Jahanzaib

January 21, 2021

Possibilities: Mikrotik PPP Disconnection/Yellow Sign Problems

Filed under: Mikrotik Related — Tags: , , , — Syed Jahanzaib / Pinochio~:) @ 9:58 AM


Disclaimer! This is important!

Every Network is different , so one solution cannot fit/applied to all. Therefore try to understand logics & create/modify the solution as per your network scenario. Do Not follow copy paste blindly.

My humble request is that kindly donot consider me as an expert on this stuff, I am NOT certified in anything Mikrotik/Cisco/Linux or Windows. However I have worked with some core networks and I read , research & try stuff all of the time. So I am not speaking/posting about stuff I am formerly trained in, I pretty much go with experience and what I have learned on my own. And , If I don’t know something then I read & learn all about it.

So , please don’t hold me/my-postings to be always 100 percent correct. I make mistakes just like everybody else. However – I do my best, learn from my mistakes and try to share tips that worked for me.

Tips posted here are based on personal experiences which I faced/sorted at various networks locally/internationally. It is requested to kindly contribute your valuable experience & any tips to help others.
Sharing is Caring …

Regard’s
Syed Jahanzaib~


PPP Common Problems

From some time we were getting following complains from few ISP’s regarding

  • Few websites (like banking) not opening if user is connected via pppoe only
  • User pppoe dial stuck , not able to reach to mikrotik pppoe server
  • User pppoe connectivity frequent/intermittent disconnection/termination
  • User pppoe dialer is connected but yellow mark at user device/workstation , No internet

 

Try to diagnose the issue one by one by below tips

  1. For few websites not working on pppoe clients only issue , try to add following rule & test
    /ip firewall mangle
    add action=change-mss chain=forward new-mss=clamp-to-pmtu passthrough=yes protocol=tcp tcp-flags=syn
  2. Mikrotik RouterOS Firmwares plays very important roles in the stability in various segments, Try STABLE or LONG-TERM release. Sometimes upgrading/downgrading rectifies issues without modifying any configuration. Read Mikrotik Forums to see if other users are having similar issues on particular version.
  3. Cheap wifi routers at client end example TPLINK/TENDA are headache to manage. Most of the older models have BUGS from security & stability issues. Always make sure that you dont use buggy routers brands, Always upgrade the Firmwares to latest. This mostly rectifies many issues. For test, under WiFi Routers dNS setting, try to make DNS static, primary to your local DNS (like linux base unbound), and secondary to google.
  4. Pay attention to mikrotik CPU, if you have high number of users on single Tik, OR if you have CONNTRACK/NATTING enabled, then disconnection of pppoe users can cause CPU spikes resulting in Tik freezing for a minute or it can cause other users disconnection dueto cpu not responding timely, resulting in looping as well. Use separate router for natting. If you have high number of PPP users along with some NATTING rules, Stop using Masquarade on same router that have a lot of dynamic interfaces. DO NOT use NAT on any router that have high number of connecting/disconnecting interfaces , like pppoe/vpn. Place an additional router connected with your PPPoE NAS, and route NAT traffic there. Make sure to disable CONNECTION TRACKING on PPPoE NAS router. As a rule of thumb, to divide load (& as a failover) , if you are using ccr1036 , add another ccr1036 after every 1200-1500 users.
  5. Adding your local DNS & assign it to user profile as a primary DNS sorted the yellow sign problems in some users WiFi Routers.
  6. PPP is sensitive to high delays and network timeouts, Make sure you dont have layer 2 level broadcast/delays
  7. If you Cisco switch with VLANs , set STP/RSTP to none on switch TRUNK  [*** This sorted the ppp disconnection at few networks]
  8. If you have Cisco switches with VLANs, Do Not allow all VLANS on TRUNK ports, Allow only limited/designated vlans on TRUNK port [*** This sorted dialup stuck issues at few networks]
  9. Changing the MTU [sometimes it sorts websites & few apps related issues , examples whatsAPP , Telegram, etc]
  10. Try to disable Encryption/Compression on the profile of the pppoe. Choosing only (pap) for pppoe server [This sorts some old freeradius related issues]
  11. Disable RSTP on all ports/VLANS [Test with caution, for temporary basis only just to confirm if its related issue]
  12. Disable LOOP protection in mikrotik ports settings [Test with caution, for temporary basis only]
  13. Do Not disable ICMP Some user end routers checks for icmp reachability to detect internet access. It’s quite worse when there are operators that think that ICMP is dangerous and it has to be blocked. Make sure you are not blocking all ICMP traffic, just fine tune it to allow at least certain type of icmp packets, however, when someone further upstream does that, you will have problems
  14. Do Not disable NTP protocol, [it is being used by many devices like android devices like android TV’s, Gaming devices etc]

Part 3/4 Annexure Example: [Test it with caution or preferably in LAB tests]

no spanning-tree vlan 1-1014
interface GigabitEthernet2/0/1
description Trunk-LAN-2-Mikrotik
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 2-16,99
switchport mode trunk

Personnel Opinion!

Well TBH, Mikrotik is a cheap/affordable solution & overall Mikrotik is excellent for core routing too BUT its not made for large scale ppp NATTING. Mikrotik is not an enterprise grade solution with reference to pppoe concentrator. It have it’s architecture’s limitations. As a rule of thumb/In general , We suggest that after crossing 1200-1400 ppp users (& max 2Gb of traffic), just add another mikrotik (ccr1036 or likewise) & so on. I knows few ISp’s locally who are using mikrotik who have used Mikrotik routers just start up their journey in the SP business but later they moves to more mature products like cisco/juniper/vBNG. One ISP in particular using 10-12 Mikrotiks to cater 15k users load (in routing mode only, no natting). With natting situation gets worse when ppp users disconnects in large quantity resulting in CPU hiking/freezing creating nightmares for admins)

If you have thousands of users , then you are in serious business, go with *Huawei/Juniper/Cisco* (which are much mature but comparatively costly products ) & as an alternate, you may look for *VBNG* which have pay as per you go modules.

Syed Jahanzaib

January 19, 2021

January 11, 2021

Cisco 10G Switch & Lenovo SFP Module Compatibility issue

Filed under: Cisco Related — Tags: , , , , — Syed Jahanzaib / Pinochio~:) @ 11:46 AM

Recently we acquired cisco 10g SFP+ switch to be added in existing stack. While trying to connect Lenovo ThinkSystem SR650 (P.No: 7X06CTO1WW ) server along with lenovo provide SFP+ modules (P.No 46C3447) with 10g Cisco switch (WS-C3850-24XS-S) via MM Fiber cable. Upon SFP+ module insertion, at both end (server to switch) then the port gets shuts with err-disabled with following error on switch logs

010834: Jan 4 09:43:44: %GBIC_SECURITY_CRYPT-4-VN_DATA_CRC_ERROR: GBIC in port Te1/0/7 has bad crc
010836: Jan 4 09:43:44: %PM-4-ERR_DISABLE: gbic-invalid error detected on Te1/0/7, putting Te1/0/7 in err-disable state

& on vmware esxi  , it showed *DISCONNECTED*

Following were technical details:

SEVER END:

  • ThinkSystem SR650 (P.No: 7X06CTO1WW )
  • 10g NIC: Emulex VFA5.2 2×10 GbE SFP+ PCIe Adapter (P.No: AT7S )
  • 10g SFP+ Module: Lenovo SFP 10gbase-sr Fiber Optic Transceiver Module (P.No 46C3447 / )

SWITCH END:

  • SWITCH MODEL : Cisco 10g SFP+ switch (P.No: WS-C3850-24XS-S )
  • Cisco Switch 10GBASE Fiber Optic SFP 10G Transceiver Module: Cisco SFP-10G-SR * Part No: 10-2415-03)
  • Vivanco Optical Fiber Patch Cord: LC-LC MM DUPLEX OM3 10M

Solution:

After searching here & there, I found that we have to disable SFP compatibility check in the switch using below CMD’s

Add these two commands (blue highlighted) to the switch:

Switch(config)# service unsupported-transceiver
— you will get a warning message here—
Switch(config)# no errdisable detect cause gbic-invalid

Afterwards , shut/no shut the switch interface then plugged in the Lenovo cable back in.  & the connectivity got OK. (make sure WRITE the config on switch so that it stays permanent.

Note: Any time non-Cisco optics are going to be plugged in to a Cisco switch it’s worth adding these commands.


Regard’s
Syed Jahanzaib

Vmware VCenter inaccessible Datastore

Filed under: VMware Related — Tags: , , — Syed Jahanzaib / Pinochio~:) @ 11:03 AM

Recently one of our VCenter 6.7 got crashed & services were not accessible, I spent hours but couldn’t restore it. To save time in further troubleshooting we removed the VC from the esxi server . Few Esxi servers were managed by this Vcenter. I logged in to each ESXI  server & in Actions I selected “Disconnect it from Vcenter“. Afterwards when new Vcenter (VCSA-7) got installed , all esxi were added successfully , But one of the ESXI server was showing some errors, therefore I removed it from the Vcenter, & when I tried to add it again to VC , following error appeared

Datastore ‘M5-11.10–8TB-raid10’ conflicts with an existing datastore in the datacenter that has the same URL (ds:///vmfs/volumes/5d810e33-e56c55bf-71be-0894ef440178/), but is backed by different physical storage.

At VCenter I was seeing below

When I right clicked on this data store, DELETE/MOUNT/UNMOUNT option were greyed out as well. How can I remove this inaccessible data store?

From the Vcenter, I Browsed that inaccessible datastore  , & clicked on VMs tab, it was showing one VM which was moved to another esxi host in the past, I edited that old VM on that esxi server , and removed the mounted ISO (which was pointing to the affected vm esxi server)  , afterwards, the inaccessible datastore disappeared automagically, and the Esxi got re-added in VC again smoothly.


Sharing is caring !

Regard’s
Syed Jahanzaib

November 18, 2020

mySQL Master-Slave Replication Notes

Filed under: mysql — Tags: , , , — Syed Jahanzaib / Pinochio~:) @ 11:44 AM

 


This post contains short notes on how we can create mySQL master to slave replication. It is generally required to fulfill multi stage backup topology. Example using the SLAVE server, we can have always uptodate live replica partner which can be used in case of any master server failure.

Very useful for replicate the DB in real time , or to be used in DR sites scenarios as well! I deployed it at few local networks, & results were good when it comes to DR/DB recovery.

 

Pros:
Master-Slave is very fast as in general it doesn’t impose any restrictions on performance.We can split read and write requests to different servers. For example, all analytics queries can be made on Slave nodes.

Cons:
Write requests can hardly be scaled. The only option to scale write requests is to increase compute capacity (RAM and CPU) of the Master node. Failover process is manual in a general case. You should take care of promotion replica node to master one.

Components used in this guide,

  • Two VM’s
  • OS: Ubuntu 16.04.3 LTS (Xenial Xerus)
  • mySQL Version: mysql Ver 14.14 Distrib 5.7.32, for Linux (x86_64) using EditLine wrapper
  • Name/IP: mySQL MASTER server: master-254 / 101.11.11.254
  • Name/IP: mySQL SLAVE server : slave-255 / 101.11.11.255
  • Credentials: mysql root password: zaib1234

Assumptions:

Mysql is installed on both servers.

MASTER Configuration

mysql -uroot -pzaib1234
create database radius;
grant all on radius.* to radius@localhost identified by "zaib1234";
# Create separate ID for replication , it will be used on SLAVE
create user 'zaib'@'%' identified by 'zaib1234';
grant replication slave on *.* to 'zaib'@'%';
exit

Edit mysql configuration to make it MASTER

nano /etc/mysql/mysql.conf.d/mysqld.cnf

Add the below entries in [mysqld] section

log-bin=mysql-bin
#(comment below line if you want all DB's to be replicated, in below e.g we are doing only radius DB replication)
binlog-do-db=radius
server-id=1
log_bin = /var/log/mysql/mysql-bin.log
bind-address = 0.0.0.0 # Search this and change it manually , donot copy paste

Restart mySQL service so changes can take effect

service mysql restart

Now we need to lock table & note down the file/position number which will be later use in SLAVE configuration

mysql -uroot -pzaib1234
FLUSH TABLES WITH READ LOCK;
show master status;
# Note down the File name and Position number, this is important, we will use it in SLAVE server
exit

Export the required DB (This will be imported on SLAVE server later)

mysqldump -u root -pzaib1234 radius --master-data > master.sql

once the export is done, unlock the tables

mysql -uroot -pzaib1234
UNLOCK TABLES;
exit

SLAVE Configuration

using any tool like winscp, copy the master.sql from the master server into this slave server.& import it.

mysql -uroot -pzaib1234 radius < master.sql

Once done, Edit mySQL configuration to make it SLAVE

nano /etc/mysql/mysql.conf.d/mysqld.cnf
log-bin=mysql-bin
#(comment below line if you want all DB's to be replicated, in below e.g we are doing only radius DB replication)
binlog-do-db=radius
# This is slave, it will have server ID number 2, on master server we have id=1
server-id=2
log_bin = /var/log/mysql/mysql-bin.log
bind-address = 0.0.0.0 # Search this and change it manually , donot copy paste

SAVE & EXIT

Restart mySQL service so that changes can take effect

service mysql restart

Now login to mySQL & config the change master parameter

mysql -uroot -pzaib1234
CHANGE MASTER TO MASTER_HOST='101.11.11.254',MASTER_USER='zaib', MASTER_PASSWORD='zaib1234', MASTER_LOG_FILE='mysql-bin.000001', MASTER_LOG_POS=154;
START SLAVE;

[Note: In above code, we used MASTER_LOG_FILE & MASTER_LOG_POS parameters when we issued show master status in master configuration section]


TEST

Login to master mySQL & create any tables or entries


root@MASTER-254:~# mysql -uroot -pzaib1234
mysql: [Warning] Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 98
Server version: 5.7.32-0ubuntu0.16.04.1-log (Ubuntu)

Copyright (c) 2000, 2020, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> use radius;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Database changed
mysql> show tables;
+----------------------+
| Tables_in_radius |
+----------------------+
| masterdb_mast_table1 |
+----------------------+
1 row in set (0.00 sec)

mysql>

now login to SLAVE mySQL & look for the tables status


root@SLAVE-255:/tmp# mysql -uroot -pzaib1234
mysql: [Warning] Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 49
Server version: 5.7.32-0ubuntu0.16.04.1-log (Ubuntu)

Copyright (c) 2000, 2020, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> use radius;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Database changed
mysql> show tables;
+----------------------+
| Tables_in_radius |
+----------------------+
| masterdb_mast_table1 |
+----------------------+
1 row in set (0.00 sec)

& we can see that tables got replicated fine.


Tips:

If you want to exclude one table from exporting (master-data) (example any large table within DB which is not important)

mysqldump -u root -pSQLPASS  DBNAME  --master-data --ignore-table=DBNAME.TABLE > export_file_name.sql

 

June 19, 2020

Mikrotik Queue Tree with Traffic Priority

Filed under: Mikrotik Related — Syed Jahanzaib / Pinochio~:) @ 3:17 PM


Mikrotik Queue tree with Traffic Priority

Example if you have assigned user with 1 mb profile, & if user is doing full download, his stream will get degrade, buffered because IDM will fetch using full available bandwidth using multiple connection.

Using Priority feature in TREE, we can configure mikrotik to assign user 1mb bandwidth limit, but it should always give 1mb priority to CDN traffic first, then to others. This way even if user is downloading at full speed liek using IDM, and if he plays video from CDN , then bandwidth priority will be given to CDN first (IDM download will degrade so that priority can be given to CDN.

This was done to avoid streaming buffer issue even if user is downloading at full speed .

– Youtube Link: https://www.youtube.com/watch?v=WxDzEonl-Bk

Queue Code: [dynamic queue for user is auto created upon user pppoe connection, on top of that we r using queue tree to prioritize]


/ip firewall address-list
add address=172.16.99.0/24 list=1mb
add address=58.27.130.0/24 list=cdn_list

/ip firewall mangle
add action=mark-connection chain=forward new-connection-mark=ICMP_Conn protocol=icmp
add action=mark-packet chain=forward connection-mark=ICMP_Conn new-packet-mark=ICMP_Pkts passthrough=no
add action=mark-packet chain=forward comment="MARK CDN UP" dst-address-list=1mb new-packet-mark=cdn_up passthrough=no \
src-address-list=cdn_list
add action=mark-packet chain=forward comment="MARK CDN DOWN" dst-address-list=cdn_list new-packet-mark=cdn_down \
passthrough=no src-address-list=1mb
add action=mark-packet chain=forward comment=MARK_1MB_UP_USER new-packet-mark=1mb_up passthrough=no src-address-list=1mb
add action=mark-packet chain=forward comment=MARK_1MB_DOWN_USER dst-address-list=1mb new-packet-mark=1mb_down \
passthrough=no

/queue simple
add max-limit=1M/1M name=ICMP packet-marks=ICMP_Pkts target=""

/queue tree
add max-limit=1G name=10G-ZAIB-WAN-Link parent=global
add name="icmp pkts Top Priority from Main Feed - Zaib" packet-mark=ICMP_Pkts parent=10G-ZAIB-WAN-Link priority=1
add name=1mb-overall-internet-up packet-mark=1mb_up parent=10G-ZAIB-WAN-Link queue=1mb-upload
add name=1mb-overall-internet-down packet-mark=1mb_down parent=10G-ZAIB-WAN-Link queue=1mb-download
add name=cdn-down-hi-priority packet-mark=cdn_down parent=1mb-overall-internet-down priority=2 queue=1mb-download
add name=cdn-up-hi-priority packet-mark=cdn_up parent=1mb-overall-internet-up priority=2 queue=1mb-upload

/queue simple
add max-limit=1M/1M name=ICMP packet-marks=ICMP_Pkts target=""

Regard’s
Syed Jahanzaib

Older Posts »

<span>%d</span> bloggers like this: