Syed Jahanzaib Personal Blog to Share Knowledge !

April 11, 2018

Short Notes for Cisco 3850-24T IOS XE Upgrade & Stack Configuration

Filed under: Cisco Related — Tags: , — Syed Jahanzaib / Pinochio~:) @ 3:49 PM

ZAIB_ Cisco 3850- 24-t.jpg

 


Upgrade Firmware of Cisco 3850 24T-S (INSTALL MODE) by copying FW from TFTP to Switch Flash Directory

Read these first on BUNDLE vs INSTALL mode.

http://blog.qsolved.com/2014/02/what-are-methods-to-boot-and-run-ios-xe.html

http://wannabelab.blogspot.com/2015/09/cisco-catalyst-3650-ios-recovery-via-usb.html

Last week we received new Cisco switches shipment having model WS-C3850-24T which will be replacing existing 3750 stack series in our data center. These switches have 2 methods of booting and running IOS XE software (in 3850).  By default, the switches comes with INSTALL mode. 

Before Upgrading Firmware ,we need to check for current mode in which the switch is currently booted in. [It should be INSTALL mode]

show version | begin Switch Port
Switch Ports Model SW Version SW Image Mode
------ ----- ----- ---------- ---------- ----
* 1 32 WS-C3850-24T 16.3.5b CAT3K_CAA-UNIVERSALK9 INSTALL
Configuration register is 0x102

We can use local USB as well to copy the firmware but since I already have TFTP for centralized backup for all Cisco switches therefore I am using TFTP method in this post.

Having TFTP server will give more leverage of doing various luxuries. copy the latest firmware to TFTP and then from switch console download the switch upgraded ISO and upload it to TFTP root folder. At the time of writing latest IOS XE version for 385024-T-S is cat3k_caa-universalk9.SPA.03.06.08.E.152-2.E8.bin

Connect to switch by ssh/telnet or by console and issue following command

copy tftp: flash:
Address or name of remote host []? 192.168.0.1
Source file name []? cat3k_caa-universalk9.SPA.03.06.08.E.152-2.E8.bin
Destination file name []? cat3k_caa-universalk9.SPA.03.06.08.E.152-2.E8.bin

After some minutes the new IOS will be downloaded to flash directory. in my case it took around 90 seconds to copy 291 MB IOS EX file.

# Switch to enable mode
en
#Issue command to upgrade to new firmware we just downloaded in above steps
request platform software package install switch all file flash:cat3k_caa-universalk9.SPA.03.06.08.E.152-2.E8.bin

Switch will now start the upgrade processing …

--- Starting install local lock acquisition on switch 1 ---
Finished install local lock acquisition on switch 1
Expanding image file: flash:cat3k_caa-universalk9.SPA.03.06.08.E.152-2.E8.bin
[1]: Expanding file
[1]: Finished expanding all-in-one software package in switch 1
SUCCESS: Finished expanding all-in-one software package.
[1]: Performing install
 SUCCESS: install Finished
[1]: install package(s) on switch 1
--- Starting list of software package changes ---
Old files list:
 Removed cat3k_caa-guestshell.16.03.05b.SPA.pkg
 Removed cat3k_caa-rpbase.16.03.05b.SPA.pkg
 Removed cat3k_caa-rpcore.16.03.05b.SPA.pkg
 Removed cat3k_caa-srdriver.16.03.05b.SPA.pkg
 Removed cat3k_caa-wcm.16.03.05b.SPA.pkg
 Removed cat3k_caa-webui.16.03.05b.SPA.pkg
New files list:
 Added cat3k_caa-base.SPA.03.06.08E.pkg
 Added cat3k_caa-drivers.SPA.03.06.08E.pkg
 Added cat3k_caa-infra.SPA.03.06.08E.pkg
 Added cat3k_caa-iosd-universalk9.SPA.152-2.E8.pkg
 Added cat3k_caa-platform.SPA.03.06.08E.pkg
 Added cat3k_caa-wcm.SPA.10.2.180.0.pkg
Finished list of software package changes
SUCCESS: Software provisioned. New software will load on reboot.
[1]: Finished install successful on switch 1
Checking status of install on [1]
[1]: Finished install in switch 1
SUCCESS: Finished install: Success on [1]

Now issue reload command to restart/reload the new config.

reload

Continue to reboot.

After the upgrade We will see new version in action !

show version | b SW
Switch Ports Model SW Version SW Image Mode
------ ----- ----- ---------- ---------- ----
* 1 32 WS-C3850-24T 03.06.08E cat3k_caa-universalk9 INSTALLConfiguration
register is 0x102


Short Notes on STACK Configuration!

cisco 3850 stack cable.jpg

Stacking is simple ! Example we want to connect 2 switches in stack mode.

1.  Power up the switch master only, let it boot completely,

2.   Attach the stacking cable from 1st switch to second one,

3.  Power up the second switch.

By default there will be no extra config required. and if both switches matches they will be added in STACK automatically.

JUST MAKE SURE THAT …

1.  All switches are running the same IOS version, same Feature Set (example IP Base);

Setting Switch Priority for MASTER

We can set switch priority to decide which switch should be MASTER. To do this, Power ON the Master switch only and issue following command

sh switch

Switch/Stack Mac Address : 009a.d29c.daXX - Local Mac Address
Mac persistency wait time: Indefinite
H/W Current
Switch# Role Mac Address Priority Version State
------------------------------------------------------------
*1 Active 009a.d29c.daXX 15 V07 Ready

Note down your switch number.

# switch to ENABLE mode
en
# Set Switch Priority
switch 1 priority 15
# WRITE CONFIG
wr
# RELOAD the configuration / reboot
reload

now boot second switch.

Once all Done. Issue following command to confirm the STACKING status.

Switch#sh switch
Switch/Stack Mac Address : 009a.d29c.da00 - Local Mac Address
Mac persistency wait time: Indefinite
H/W Current
Switch# Role Mac Address Priority Version State
------------------------------------------------------------
*1 Active 009a.d29c.da00 15 V07 Ready
2 Standby 682c.7b3c.4180 1 V07 Ready

Check Stack Ports status

Switch#show switch stack-ports
Switch# Port1 Port2
----------------------------
1 OK DOWN
2 OK DOWN

Some Useful STACK commands

show license right-to-use
switch stack-bandwidth
show switch detail

Following is one good Link for 3750 stack configuration …

https://www.petenetlive.com/KB/Article/0001205



Regard’s
Syed Jahanzaib ~

Advertisements

April 10, 2018

Symantec SEPM 12.x Migration to 14.x

Filed under: Symentec Related — Tags: , — Syed Jahanzaib / Pinochio~:) @ 10:04 AM

sepm

Today was a hectic day. We received new series of HP G5/8th Gen laptop which supports Windows 10 only  , & when we tried to installed SEPM 12.x client, it stated that the App is not compatible with this version of windows. That was really an hectic news for us because our SEPM server was based on windows 2003 32bit & there is no straightforward method for in-place upgrade to SEPM 14. It requires minimum 64bit of server window 2008 or above server . Any way we managed to migrate SEPM 12.1.6 to SEPM 14.x on Windows 2012 R2 64bit at a cost of whole day brainstorming.

Following are steps I followed for the migration of SEPM v12 W2003 32bit to SEPM v14 on Windows 2012 R2 64bit.


First Take Backup of current SEPM 12.x DB

Step 1: Back up the Database

  1. Click Start Programs Symantec Endpoint Protection Manager > Database Backup and Restore.
  2. Click Back Up. The database backup file name is date_timestamp.zip and is located in the following directory:

\Program Files\Symantec\Symantec Endpoint Protection Manager\data\backup

Copy above folder at some centralized file server folder or USB.

Step 2: Back up the Disaster Recovery File

Copy the following folder at some centralized file server folder or USB.

\Program Files\Symantec\Symantec Endpoint Protection Manager\Server Private Key Backup\

Now shutdown current SEPM Server machine & proceed to install SEPM on new machine


# Install SEPM on new Windows 2012 R2 server

on New Windows 2012 R2 , configure same Computer name & IP Address which was setup on old SEPM server.

Install the Same SEPM version, in my case it was 12.1.6 MP6.

Once installation is done, & When the Management Server Configuration Wizard runs, select Custom configuration & choose Use a recovery file. Point to the backup folder we took in Step 1/2.

As showed in the image below …

1- sav restore from backup.JPG

Once its done, Start the SEPM console one time to confirm if all is running ok.

Restore the database:

To restore DB, Stop the following services,

  • Symantec Endpoint Protection Manager
  • Symantec Endpoint Protection Manager Webserver
  1. Click Start Programs Symantec Endpoint Protection Manager > Database Backup and Restore.
  2. Click Restore.

Once All done, reboot server one time and make sure all services are started properly in the SERVICES console.


User Rights Assignment in Group Policy.

If your SEPM is an member of Active Directory then services will not start dueto lack of LOGON AS SERVICE rights.

As a workaround I installed GROUP POLICY MANAGEMENT on the new SEPM server, and edit Group Policy to add following users in LOGON AS SERVICE section.

  • NT SERVICE\semsrv
  • NT SERVICE\semwebsrv
  • NT SERVICE\SQLANYs_sem5
  • NT SERVICE\semapisrv

Since I was editing Domain Group Policy from the SEPM server itself, that is why I changed Location to local PC and then above accounts was added successfully. As a workaround we can add user SID as well. See following command to get SID of accounts and SID accordingly.

sc showsid semsrv
sc showsid semwebsrv

Client SEPM 14.x Server Console

sepm server.JPG

Client SEPM 14.x Client

client sepm.JPG

April 6, 2018

Veeam B&R 9.5 Update 3 Error: This Veeam Backup & Replication installation can not be updated automatically

Filed under: Microsoft Related, VMware Related — Tags: — Syed Jahanzaib / Pinochio~:) @ 2:32 PM

We encountered following error were tried to apply Veeam B&R 9.5 Upgrade Patch 3 .

This Veeam Backup & Replication installation can not be updated automatically. Please contact Veeam customer support for assistance with manual update.

vbr95up3 error.JPG

After some investigation it found that it can occur dueto either you are running trial or if you have problem with the license files. Therefore as a workaround to enforce Update Pack 3 which was required dueto requirement of addition of Esxi 6.5 Vcenter, I followed below steps …

Rename following files

VeeamLicense.dll
[ available in C:\Program Files\Common Files\Veeam\ ]

veComLic.dll
[ available in C:\Program Files\Common Files\Veeam\Backup And Replication ]

Now run the Update Pack 3 and it will run smoothly.

v95up3

 


 

Done.

v br 95 up3 final.JPG

March 27, 2018

Separating NATTING from ROUTING in Mikrotik

Filed under: Mikrotik Related — Tags: — Syed Jahanzaib / Pinochio~:) @ 10:57 AM

nattinv and routing brother.jpg

mikrotik natting and routing

Disclaimer:

Every Network is different , so one solution cannot be applied to all. Therefore try to understand logic & create a solution that can match with your network scenario. Do not follow copy paste blindly.

I am NOT certified in anything Mikrotik/Cisco/Linux or Windows. However I have worked with some core networks and I read & research & try stuff all of the time. So I am not speaking/posting about stuff I am formerly trained in, I pretty much go with experience and what I have learned on my own. And , If I don’t know something then I read & learn all about it.

So , please don’t hold me/my-postings to be always 100 percent correct. I make mistakes just like everybody else. However – I do my best, learn from my mistakes and always try to help others

Regard’s
Syed Jahanzaib


Scenario:

We are using Mikrotik CCR as PPPOE NAS. Its a mix match router where large number of users received private ip (via pppoe connection) and other large portion receives IP from routed Public pool as well.

 


Problem:

When we have network outages like light failure in any particular area , in LOG we can see many PPPoE sessions disconnects with ‘peer not responding‘ messages. Exactly at this moments, our NAS CPU usage reaches to almost 100% , which results in router stops passing any kind of traffic. This can continue for a minute or so on.

As showed in the image below …

pppoe high cpu usage

If you are using Masquarade /NAT on the router, that is the problem. When using Masquarade, RouterOS has to do full connection tracking recalculation on EACH interface connect/disconnect. So if you have lots of PPP session connecting/disconnecting, connection tracking will constantly be recalculated which can cause high CPU usage. When interfaces connect/disconnect, in combination with NAT, it gives you high CPU usage.


Solution OR Possible Workaround :

  • If you have lots of PPP users along with some NATTING rules, Stop using Masquarade on same router that have a lot of dynamic interfaces. DO NOT use NAT on any router that have high number of connecting/disconnecting interfaces , like pppoe/vpn. Place an additional router connected with your PPPoE NAS, and route NAT there.
    Example: Add another router & perform all natting on that router by sending marked traffic from private ip series to that nat router. Setup routing between the PPPoE NAS and the NAT router.

Following is an working example.

1# Main CCR as PPPOE NAS

Interface Details:

  • ETHER1-LAN-: 192.168.88.1/24 < User facing interface where pppoe connections establishes
  • PUBLIC-WAN: 101.11.11.254 < WAN Interface for public IP routing
  • 2-NAT-ROUTER: 192.168.60.2/24  < interface connected with another CCR for natting

PPPoE User IP Pool > 172.16.0.1-172.16.0.255
UPSTREAM ISP Core Router Gateway IP >  101.11.11.36

2# Second CCR as NATTING Router

Interface Details:

  • 2-CCR-LAN: 192.168.60.1/24 < interface connected with main CCR [pppoe]
  • NATTING-WAN: 101.11.11.253 < Wan interface for natting users [traffic coming from main CCR for natting]
  • UPSTREAM ISP Core Router Gateway IP >  101.11.11.36

1# Main CCR Configuration for marking traffic

First we will mark traffic for private/public ip and will create routes for them as well.

/ip firewall mangle
add action=mark-routing chain=prerouting comment="Mark routing for Private IP users - zaib" disabled=no new-routing-mark=nat_routing passthrough=yes src-address=172.16.0.1-172.16.0.255
# We really dont need to mark traffic for public ip's because they will simply pass from our default route , but just for the sake of demonstration we are doing it.
add action=mark-routing chain=prerouting comment="Mark routing for PUBLIC IP users - zaib" disabled=no new-routing-mark=public_routing passthrough=yes src-address=1.1.1.1-1.1.1.255

Make sure you dont have any NAT rule in place. [in NAT section]

Now add Routes for marked traffic

/ip route
add comment="Route private ip traffic via second NAT router" disabled=no distance=1 dst-address=0.0.0.0/0 gateway=192.168.100.1 routing-mark=nat_routing scope=30 target-scope=10
add comment="Route public ip via this router default Gateway" disabled=no distance=1 dst-address=0.0.0.0/0 gateway=101.11.11.36 routing-mark=public_routing scope=30 target-scope=10
# DEFAULT Gateway for router's own traffic - zaib
add check-gateway=ping disabled=no distance=1 dst-address=0.0.0.0/0 gateway=101.11.11.36 scope=30 target-scope=10

Main CCR configuration part is done. Now moving towards second router where all NATTING will be done.

2# NATTING CCR Configuration for Masquerade

First create Default NAT rule [you may want to add ip series for security purposes.

/ip route
add comment="Default Router for NATTING router " disabled=no distance=1 dst-address=0.0.0.0/0 gateway=101.11.11.36 scope=30 target-scope=10
# Add reverse Route so that NATTING router can see the pppoe user directly
add disabled=no distance=1 dst-address=172.16.0.0/16 gateway=192.168.100.2 scope=30 target-scope=10

Testing !

  • Create TEST user in main CCR pppoe NAS,
  • Assign him private ip series profile,
  • Connect this TEST id from test PC & run TRACEROUTE

As showed in the image below …

ccr pppoe active private.JPG

.

nat-vs-route.JPG

RUN Torch on NATTING Router… as we can see that NATTING router is seeing pppoe users directly dueto reverse route in it.

NATTING CCR torch

.


 

March 21, 2018

FUN with Mikrotik BRIDGE Series# Redirecting Traffic with Mikrotik Bridge – Part#2

Filed under: Mikrotik Related — Tags: , , , — Syed Jahanzaib / Pinochio~:) @ 3:19 PM

link-redirection

  1. FUN with Mikrotik BRIDGE Series#1. Filter PPPoE Requests – Part#1
  2. FUN with Mikrotik BRIDGE Series# Redirecting Traffic with Mikrotik Bridge – Part#2 < You are Here

Disclaimer! This is important!

This post is related to a solution designed specific to cater some local manipulation requirement therefore you may continue to read it as an reference purpose only !

Every Network is different , so one solution cannot be applied to all. Therefore try to understand logic & create your own solution as per your network scenario. Just dont follow copy paste.

Please donot think that I am an expert on this stuff, I am NOT certified in anything including Mikrotik/Cisco/Linux or Windows. However I have worked with some core networks and I read , research & try stuff all of the time. So I am not speaking/posting about stuff I am formerly trained in, I pretty much go with experience and what I have learned on my own. And , If I don’t know something then I read & learn all about it.

So , please don’t hold me/my-postings to be always 100 percent correct. I am human being , I do make mistakes just like everybody else. However – I do my best, learn from my mistakes and always try to help others

Regard's
Syed Jahanzaib~

Scenario & Requirements:

We want to connect Network A & B using Mikrotik Bridge so that we can transparently intercept some traffic for control & redirection purposes. Example we want to make sure that any dns traffic that is traveling from A to B or B to A should be redirected to Mikrotik DNS for manipulation purposes. Also we would like to Block ICMP traffic travelling between both networks.

Solution:

We are using Mikrotik 2011UiAS-2HnD model.

Port-1 is connected with Network A and Port-2 is connected with Network B.

# BRIDGE Configuration

First we will do Bridge configuration & add ports in it,

/interface bridge
add name=bridge1

/interface bridge port
add bridge=bridge1 interface=ether1
add bridge=bridge1 interface=ether2
/interface bridge settings
set use-ip-firewall=yes

As showed in image below …

bridge rules

# DNS Configuration

Now setup Local DNS server

/ip dns
set allow-remote-requests=yes servers=8.8.8.8

# Now we will add static DNS entry for our requirements
/ip dns static
add address=1.2.3.4 name=aacable.wordpress.com

As showed in image below …

bridge dns add static.JPG

# DNS Redirection

Firewall NAT configuration to redirect DNS traffic travelling via BRIDGE interface to Mikrotik local DNS for manipulation purposes

/ip firewall nat
add action=redirect chain=dstnat comment="Redirect DNS Traffic via BRIDGE to local DNS - Zaib" dst-port=53 in-interface=bridge1 protocol=udp to-ports=53

# ICMP Filteration

Firewall Filter configuration to block ICMP protocol

/ip firewall filter
add action=reject chain=forward comment="Block ICMP Rule in BRIDGE - Zaib" in-interface=bridge1 protocol=icmp reject-with=icmp-network-unreachable

Client Testing

Result of testing NSLOOKUP from user PC. [Before vs After]

bridge - dns resolve nslookup result

Result of testing ICMP & PING from user PC.

bridge - icmp block result result


Linux is amazing 🙂 however Mikrotik is handy most of the times 🙂

March 8, 2018

Short Notes on LibreNMS !

Filed under: Linux Related — Tags: , , — Syed Jahanzaib / Pinochio~:) @ 3:46 PM

libre dashboard

This post contains short notes & step by step guide on open source network omonitoring system called LIBRENMS , it is based on Php/MySQL/Snmp.


STEP BY STEP iNSTALLATION FOR LIBRENMS

OS: Ubuntu 16 (Server Edition/64bit)

First install required support packages

apt-get update
# Now install all the required components in a single GO ...
apt install -y apache2 composer fping git graphviz imagemagick libapache2-mod-php7.0 mariadb-client mariadb-server mtr-tiny nmap php7.0-cli php7.0-curl php7.0-gd php7.0-json php7.0-mcrypt php7.0-mysql php7.0-snmp php7.0-xml php7.0-zip python-memcache python-mysqldb rrdtool snmp snmpd whois curl acl

Add local user for librenms

useradd librenms -d /opt/librenms -M -r
usermod -a -G librenms www-data
systemctl restart mysql

Create user & DB

mysql -uroot -pzaib1234
CREATE DATABASE librenms CHARACTER SET utf8 COLLATE utf8_unicode_ci;
CREATE USER 'librenms'@'localhost' IDENTIFIED BY 'zaib1234';
GRANT ALL PRIVILEGES ON librenms.* TO 'librenms'@'localhost';
FLUSH PRIVILEGES;
exit

Now edit maria-db config file

nano /etc/mysql/mariadb.conf.d/50-server.cnf

& add following

[Within the [mysqld] section please add]

innodb_file_per_table=1
sql-mode=""
lower_case_table_names=0
systemctl restart mysql

Save & Exit.

Web Server Config

Edit PHP files & setup TIMEZONEs

nano /etc/php/7.0/apache2/php.ini
nano /etc/php/7.0/cli/php.ini

Search for ‘date.timezone‘ & change it to

date.timezone = Asia/Karachi

Save & EXit.

Configure Apache

nano /etc/apache2/sites-available/librenms.conf

Add the following config,



DocumentRoot /opt/librenms/html/
ServerName librenms.localhost
AllowEncodedSlashes NoDecode

Require all granted
AllowOverride All
Options FollowSymLinks MultiViews



Save & Exit

a2dissite 000-default
a2ensite librenms.conf
a2enmod rewrite
systemctl restart apache2

Configure snmpd

cp /opt/librenms/snmpd.conf.example /etc/snmp/snmpd.conf

Now edit SNMPD config file and add our SNMP community string like public or other

nano /etc/snmp/snmpd.conf

Edit the text which says RANDOMSTRINGGOESHERE and set your own community string.
save and exit.

curl -o /usr/bin/distro https://raw.githubusercontent.com/librenms/librenms-agent/master/snmp/distro
chmod +x /usr/bin/distro
systemctl restart snmpd

CRONJOBS for librenms

cp /opt/librenms/librenms.nonroot.cron /etc/cron.d/librenms
cp /opt/librenms/misc/librenms.logrotate /etc/logrotate.d/librenms
chown -R librenms:librenms /opt/librenms
setfacl -d -m g::rwx /opt/librenms/rrd /opt/librenms/logs
setfacl -R -m g::rwx /opt/librenms/rrd /opt/librenms/logs

Install LIBRE-NMS via composer command

cd /opt
composer create-project --no-dev --keep-vcs librenms/librenms librenms dev-master

Example:

1- install libre via cmd

Now access libre-nms via browser …

http://yourip/install.php

& follow onscreen steps…

2- libre first step of gui

Make sure to add id password which is authorized to change DB , as required …

 

 

 

 

AT one point, You have to create the config.php manually as showed here.

nano /opt/librenms/config.php

& paste the data as showed in the config window of gui.

<?php
## Have a look in defaults.inc.php for examples of settings you can set here. DO NOT EDIT defaults.inc.php!

### Database config
$config['db_host'] = 'localhost';
$config['db_port'] = '3306';
$config['db_user'] = 'librenms';
$config['db_pass'] = 'zaib1234';
$config['db_name'] = 'librenms';
$config['db_socket'] = '';

// This is the user LibreNMS will run as
//Please ensure this user is created and has the correct permissions to your install
$config['user'] = 'librenms';

### Locations - it is recommended to keep the default
#$config['install_dir'] = "/opt/librenms";

### This should *only* be set if you want to *force* a particular hostname/port
### It will prevent the web interface being usable form any other hostname
#$config['base_url'] = "http://librenms.company.com";

### Enable this to use rrdcached. Be sure rrd_dir is within the rrdcached dir
### and that your web server has permission to talk to rrdcached.
#$config['rrdcached'] = "unix:/var/run/rrdcached.sock";

### Default community
$config['snmp']['community'] = array("public");

### Authentication Model
$config['auth_mechanism'] = "mysql"; # default, other options: ldap, http-auth
#$config['http_auth_guest'] = "guest"; # remember to configure this user if you use http-auth

### List of RFC1918 networks to allow scanning-based discovery
#$config['nets'][] = "10.0.0.0/8";
#$config['nets'][] = "172.16.0.0/12";
#$config['nets'][] = "192.168.0.0/16";

# Update configuration
#$config['update_channel'] = 'release'; # uncomment to follow the monthly release channel
#$config['update'] = 0; # uncomment to completely disable updates 

Save & Exit.

and click on FINISH on GUI.


librenms install done

 

Make sure to remove COMPOSER & set permissions when all is OK. Else it will give error in VALIDATION part.

chown -R librenms:librenms /opt/librenms
apt-get remove composer

 


Some more snaps

  • Add Device

 

add device

librenms add device with snmp

Devices Overview

device view

libre dashboard.JPG

February 26, 2018

Power of Open Source / Traffic Controlling with TC

Filed under: Linux Related — Syed Jahanzaib / Pinochio~:) @ 9:49 AM

speed meter

 

pwoerful pawn

Living in the Dark & Playing  & with the Open Source is Fun & Amazingly Powerful ! z@iB


Scenario:

We have few Linux base FTP/HTTP base media file sharing server setup. User can access these server by bypassing the router as we donot want to put load on the router itself & user should access these servers from switch level by intervlan routing setup.

Problem:

Since the FTP data is in many terabytes and hundreds of users are accessing it, this is creating bottleneck for server’s network/storage.

Solution:

On Linux we can use TRAFFIC CONTROLLER (TC) to limit bandwidth on per user IP basis.  We will create a simple bash script which will do the following in sequence …

  • FTP Server IP: 101.11.11.254
  • Clear any existing shaping rules
  • Create 4 bandwidth packages as following
    – 1024kbps  [for General Users subnet 101.11.11.0/24]
    – 2048kbps  [for Support Staff  subnet 101.11.50.0/24]
    – 512kbps     [for ICMP, so that ping should work fine , Protocol base]
    – 128kbps    [for any unknown IP series, that is no define above, for ALL 0.0.0.0/0.0.0.0]
  • Assign ip subnet to these packages (/24 and it will create per ip base rules via LOOP statement)
  • In this example, we are limiting Bandwidth in one way only, means packets sending from server to client will be applied limitation .. For both ways just duplicate rules and use src as well.

Create , the script!

mkdir /temp
touch /temp/shaper.sh
chmod +x /temp/shaper.sh

Now Edit

nano /temp/shaper.sh

& paste following,

#!/bin/bash
# Traffic Shaping Using `TC` in Linux / This was made for specific requirements.
# ITs customzied designed to fulfill some specific requirements & it went well.
# Following Script will limit outgoing bandwidth for specific ip range series.
# It will also allow ICMP more bandwidth so that ping to local server may not delay or imtout.
# It will then limit all other ips that are not listed to minimum bandwidth that is 128kbps.
# Adjust is as per your requirements.
# Syed Jahanzaib / aacable @ hotmail . com / https://aacable.wordpress.com :~)
# 23-Febraury-2018

# Enable following to see debug
# set -x

# Local Network Interface name
INT="ens33"
INT_SPEED="100Mbit"

# First IP Range & its package with class
IP1="101.11.11"
IP1_PKG="1mbit"
IP1_CLASSID="1:10"

# Second IP Range & its package with class
IP2="101.11.50"
IP2_PKG="2mbit"
IP2_CLASSID="1:20"

# Separate Bandwidth for ICMP protocol to minimze delay in PING/ICMP packets
ICMP_PKG="512kbit"
ICMP_CLASSID="1:200"

# Other Default Package Line, means all other unmarked IP's or traffic
UNKNOWN_PKG="128kbit"
UNKNOWN_CLASSID="1:201"

if [[ "$1" != "start" && "$1" != "stop" && "$1" != "status" ]]; then
echo "Usage:

./shaper.sh start
./shaper stop
./shaper status

by Jz!"
exit 1
fi

if [ $1 == "stop" ]; then
echo "Clearing all existing Queues on user request ...

Done!"
tc qdisc del dev $INT root 2> /dev/null > /dev/null
exit 1
fi

if [ $1 == "status" ]; then
echo "Showing Current TC Class and Filter Status
=========================================="
tc class show dev $INT
echo "

"
tc -s -d class show dev $INT
echo "=========================================="

exit 1
fi

if [ $1 == "start" ]; then
echo "Starting Shaper ...
Clearing all existing Queues for re-implementation ...
=========================================="
"

# clean existing down- and uplink qdiscs, hide errors
tc qdisc del dev $INT root 2> /dev/null > /dev/null

echo "
Adding Queues ...."
# Create egress shaping for all required classes
tc qdisc add dev $INT root handle 1: htb default 20 r2q 50
tc class add dev $INT parent 1: classid 1:1 htb rate $INT_SPEED ceil $INT_SPEED
tc class add dev $INT parent 1:1 classid $IP2_CLASSID htb rate $IP1_PKG ceil $IP1_PKG prio 100
tc class add dev $INT parent 1:1 classid $IP1_CLASSID htb rate $IP2_PKG ceil $IP2_PKG prio 100
tc class add dev $INT parent 1:1 classid $ICMP_CLASSID htb rate $ICMP_PKG ceil $ICMP_PKG prio 100
tc class add dev $INT parent 1:1 classid $UNKNOWN_CLASSID htb rate $UNKNOWN_PKG ceil $UNKNOWN_PKG prio 100

# ICMP reply Adjustment so ping may not timeout or high delay may not occur at client end for this server local IP
tc filter add dev $INT parent 1: protocol ip prio 100 u32 match ip protocol 1 0xff flowid $ICMP_CLASSID

# IP-1 series limitation applies here ...
for (( i = 1 ; i <= 255; i++ ))
do
b=$i
tc filter add dev $INT parent 1: protocol ip prio 100 u32 match ip dst $IP1.$i flowid $IP1_CLASSID
done

# IP-2 series limitation applies here ...
for (( i = 1 ; i <= 255; i++ ))
do
b=$i
tc filter add dev $INT parent 1: protocol ip prio 100 u32 match ip dst $IP2.$i flowid $IP2_CLASSID
done

# Send every unknown IP to classid UNKNOWN_CLASS that is lowest. so that any IP that is not listed above will get htis lowest bandwidth
tc filter add dev $INT parent 1: protocol ip prio 100 u32 match ip dst 0.0.0.0/0.0.0.0 flowid $UNKNOWN_CLASSID

echo "
Done!"
fi

Result:

linux TC down-result.jpg

 

Renew DHCP lease if Gateway not responding

Filed under: Mikrotik Related — Tags: , , — Syed Jahanzaib / Pinochio~:) @ 8:49 AM

automation

Nothing Fancy , just short notes for script command reference purposes!

Following is a very simple 2 minutes instant cooked noddle type script. It will simply check the gateway status acquired by the dhcp-client using  ARP ping, , if it fails, then it will simply try to renew the ip by release/renew. Its very basic level, but its interesting to see how Mikrotik can really help you in day to day task by facilitated with the Tik level scripting !

Regard’s
Syed Jahanzaib

# Mikrotik Script - Tested with 5.x
# Script to check default gateway acquired by dhcp client on specific interface,
# Lot of room for improvements and modification but following was enough for some particular task,
# You can add BOUND status as well too, but i wanted this particular checking, you can add whatever you like
# Syed Jahanzaib == aacable AT hotmail DOT com - https:// aacable DOT wordpress DOT com
# Feb,2018
# Setting Variables

# Set Interface name which will get DHCP ip , This is the only option you may need to modify
:local INTERFACE "wan1"

# Number of Ping Count, how many times mikrotik should ping the target device
:local PINGCOUNT "5"

# Ping threshold, how many values should set alert, like if 5 out of 5 goes out
:local PINGTS "5"

:local i 0;
#:local i value=0;
:local F 0;
:local date;
:local time;

:log info "Checking default gateway for $INTERFACE interfaces."
:local DHCPGW [ /ip dhcp-client get [/ip dhcp-client find where interface=$INTERFACE ] gateway ]

# IF there is no default gateway on dhcpclient interface or if interface is disabled, then error
:if ([:len $DHCPGW] = 0) do={
:log error "No DEFAULT gateway found on $INTERFACE interface @ $date $time ..."
# Try to renew ip
/ip dhcp-client release $INTERFACE
/ip dhcp-client renew $INTERFACE
# Exit the script without further process ... I found this recently because in mikrotik there is no EXIT 1
error :error
}

# PING host $PINGCOUNT times
:for i from=1 to=$PINGCOUNT do={
:if ([/ping arp-ping=yes interface=$INTERFACE $DHCPGW count=1]=0) do={:set F ($F + 1)}
:delay 1;
};

# If no ping found then give error and do action
:if (($F=$PINGTS)) do={
:log error "PING to $DHCPGW via $INTERFACE is DOWN! @ $date $time "

# Take action if unable to ping gateway
/ip dhcp-client release $INTERFACE
/ip dhcp-client renew $INTERFACE
} else={
:log warning "PING to $DHCPGW via $INTERFACE is UP! @ $date $time "
}

February 20, 2018

Bursting with Mikrotik Burst ^o^

Filed under: Mikrotik Related — Tags: , , — Syed Jahanzaib / Pinochio~:) @ 3:42 PM

Disclaimer! This is important!

Every Network is different , so one solution cannot be applied to all. Therefore try to understand logic & create your own solution as per your network scenario. Just dont follow copy paste.

If anybody here thinks I am an expert on this stuff, I am NOT certified in anything Mikrotik/Cisco/Linux or Windows. However I have worked with some core networks and I read , research & try stuff all of the time. So I am not speaking/posting about stuff I am formerly trained in, I pretty much go with experience and what I have learned on my own. And , If I don’t know something then I read & learn all about it.

So , please don’t hold me/my-postings to be always 100 percent correct. I make mistakes just like everybody else. However – I do my best, learn from my mistakes and always try to help others

Regard's
Syed Jahanzaib~

Mikrotik Burst feature provides predefined extra bandwidth for a limited period of time IF the user remains under the burst threshold limit, or else He will be limited to his max–limit package.

It is best explained here

https://wiki.mikrotik.com/wiki/Manual:Queues_-_Burst

 

Real life Example:


Explanation !

  • User IP (1-TARGET) on which this queue will be implemented
  • When this IP will start downloading he can reach download rate of 512 kbps (3-BURST_LIMIT)
  • Until he continue to do so for a minute (5-BURST_TIME) (period of time, in seconds, over which the average data rate is calculated. (This is NOT the time of actual burst, so on avg it will become 30s)
  • That is on an average basis his download remains 256 kbps (4-BURST_THRESHOLD) for a minute, (average of 30 seconds)
  • Then he will be get back limited to his max-limit (2-MAX_LIMIT)
  • When a user doesn’t use the traffic at all and 30 second average goes to 0 so the next time traffic is requested then it will be at the Burst speed (3-BURST_LIMIT).

 

Small BURST TIME may not give you correct results. So use reasonable time. in this example I used shorter time for demonstration purposes. A large burst isn’t a problem technically, it’s more of a business decision.

To calculate burst and relates values, download this excel sheet named “MikroTik burst simulator.xlsx” from my google drive & try it yourself … you will get clear picture


Another Example by joshaven! so that you can better understand —-


Now look at its Demo ,

 

 

FREERADIUS WITH MIKROTIK – Part #13 – Detecting user device vendor based on MAC address

Filed under: freeradius — Tags: , , , , , — Syed Jahanzaib / Pinochio~:) @ 11:52 AM

fre2

mac-address

1- identify vendor from mac

 

FREERADIUS WITH MIKROTIK – Part #1 – General Tip’s Click here to read more on FR tutorials …

Disclaimer! This is important!

Every Network is different , so one solution cannot be applied to all. Therefore try to understand logic & create your own solution as per your network scenario. Just dont follow copy paste.

If anybody here thinks I am an expert on this stuff, I am NOT certified in anything Mikrotik/Cisco/Linux or Windows. However I have worked with some core networks and I read , research & try stuff all of the time. So I am not speaking/posting about stuff I am formerly trained in, I pretty much go with experience and what I have learned on my own. And , If I don’t know something then I read & learn all about it.

So , please don’t hold me/my-postings to be always 100 percent correct. I make mistakes just like everybody else. However – I do my best, learn from my mistakes and always try to help others

Regard's
Syed Jahanzaib~

* Scenario:

We have a generic freeradius based billing system in place. in RADACCT table, we have a trigger that fire-up after successfull connection made from user device & it inserts user device MAC address in radcheck table for MAC validation & restriction.

* Requirement:

For better management , control & reporting purposes we want to add more checks and controls by adding user calling device VENDOR name in an additional column so that we can have idea what devices are most common in our network, or to detect any device that is prohibited by the company SOP or policy example commercial grade routers to minimize misuse of our residential services. Based on this reporting  we can prohibit connecting these devices as well if required.

To fulfill this we will do following

  1. Create a TRIGGER on RADACCT that will executes after INSERT record (like when user will connect to system successfully)
  2. Create DB and upload vendors mac address data so that we can query for vendor name locally (alternate approach is to simple use any Perl or bash script to look up vendor name dynamically on the fly form the internet)

 


OK before creating TRIGGER we must learn or understand our USER table where mac related will be added or update. One example for such table is showed below …

root@radius:/temp# mysql -uroot -pSQLPASS -e "use radius; describe users;"

+-------------------+--------------+------+-----+-------------------+-----------------------------+
| Field | Type | Null | Key | Default | Extra |
+-------------------+--------------+------+-----+-------------------+-----------------------------+
| id | int(10) | NO | PRI | NULL | auto_increment |
| username | varchar(128) | NO | UNI | NULL | |
| password | varchar(32) | NO | | NULL | |
| firstname | text | NO | | NULL | |
| lastname | text | NO | | NULL | |
| email | text | NO | | NULL | |
| mobile | text | NO | | NULL | |
| cnic | text | NO | | NULL | |
| srvname | text | NO | | NULL | |
| srvid | int(3) | NO | | NULL | |
| expiration | date | YES | | NULL | |
| mac | varchar(30) | NO | | NULL | |
| macvendor | varchar(128) | NO | | NULL | |
| bwpkg | varchar(256) | NO | | NULL | |
| pool | varchar(128) | YES | | other | |
| is_enabled | int(1) | NO | | NULL | |
| is_days_expired | int(1) | NO | | NULL | |
| is_qt_expired | int(1) | NO | | NULL | |
| is_uptime_expired | int(1) | NO | | NULL | |
| qt_total | varchar(32) | NO | | NULL | |
| qt_used | varchar(20) | NO | | NULL | |
| uptime_limit | varchar(20) | NO | | NULL | |
| uptime_used | varchar(32) | NO | | NULL | |
| owner | text | NO | | NULL | |
| vlanid | varchar(32) | NO | | NULL | |
| createdon | timestamp | NO | | CURRENT_TIMESTAMP | on update CURRENT_TIMESTAMP |
+-------------------+--------------+------+-----+-------------------+-----------------------------+

In this post we have interest in two columns named mac and macvendor


Now we will create TRIGGER with our so called magical code 😉

1- TRIGGER for radacct table

CREATE TRIGGER `chk_mac_after_insert` AFTER INSERT ON `radacct`
 FOR EACH ROW BEGIN
# Check if user mac is already added in radcheck table,
SET @mac = (SELECT count(*) from radcheck where username=New.username and attribute='Calling-Station-ID');
# If there is no entry for mac, then update mac in radcheck table, so that in future only this mac will be able to connect with that username
IF (@mac = 0) THEN 
INSERT into radcheck (username,attribute,op,value) values (NEW.username,'Calling-Station-ID',':=',NEW.callingstationid); 
# add mac in users table for general record purpose
UPDATE users SET mac = NEW.callingstationid where username = NEW.username;
# trim mac for first 3 strings to detect vendor company
SET @mactrim = (select LEFT(mac, 8) from users where username=New.username);
# get vendor name from mac db table
SET @macvendor1 = (select vendor from macdb where oui=@mactrim);
# Update vendor name in user table
UPDATE users SET macvendor=@macvendor1 where username = NEW.username;
END IF;
END
#SYED Jahanzaib - 
2- triger

2- Create MAC Address Database for VENDOR Lookup

This is a bit debatable part, I prefer to have local database for vendor mac addresses so that all lookup should be done locally rather then using any API for mac lookup. But if you want to prefer using internet base lookup , then you can use perl, bash or any other method to do lookup one example for internet lookup is as follows …

curl http://api.macvendors.com/70-54-D2-16-A5-D9
#OUTPUT

PEGATRON CORPORATION

OK lets move forward to create local DB for mac and vendor lookup.

  • Install required modules
apt-get install python-mysqldb python-minimal
  • Download the MAC address OUI from the IEEE site.
cd /tmp
wget http://standards.ieee.org/develop/regauth/oui/oui.txt
  • Once download is done, Create script that will insert mac records into separate table named ​mactable
touch mactosql.sh
chmod +x touch mactosql.sh

Now edit file

nano mactosql.sh

Paste following

# 09-07-2016
# ouiUpdate, grabs the iee oui list and adds it to a table in mysql database.
# Provides no syntax checking of the input file, use with caution
# based on load_iou_to_pgsql.py https://gist.github.com/pwldp/dee6d5f9868a1dd1d076
# Usage:
# python ouiUpdateMysql.py

import MySQLdb
import re
import urllib
import sys

# Globals
# Database Handle, again, no need to change them; Only Declarations
DB = ''
DB_CURSOR = ''
OUI_URL = "http://standards.ieee.org/develop/regauth/oui/oui.txt"
OUI_FILE = "oui.txt"

# Database Related Globals
DB_NAME='radius'
DB_USERNAME='root'
DB_PASSWORD='SQL_PASSWORD'
DB_TABLE='macdb'
DB_HOST='localhost'

def main():
global DB
global DB_CURSOR
global DB_TABLE
# Download oui.txt Comment out the two lines below if you do not want to download the file and use a offline copy
#####
# print "Downloading ",OUI_URL
# urllib.urlretrieve(OUI_URL, OUI_FILE)
#####
##connect to db
try:
## Opening connection to Database with pre-configured Database, Host, User and Password
DB = MySQLdb.connect(host=DB_HOST, user=DB_USERNAME, passwd=DB_PASSWORD, db=DB_NAME)
DB_CURSOR = DB.cursor()
except:
sys.exit('Unable to Open Connection to Database')

# Drop table and start from new
DB_CURSOR.execute("DROP TABLE IF EXISTS ouiList")
# Create table as per requirement
sql ="""CREATE TABLE IF NOT EXISTS ouiList (
id int(6) NOT NULL AUTO_INCREMENT,
oui varchar(8) DEFAULT NULL,
vendor varchar(100) DEFAULT NULL,
PRIMARY KEY (id)
) ENGINE=InnoDB"""

# make the new table
DB_CURSOR.execute(sql)

# parsing oui.txt data and adding to table
print "Parsing data..."
with open(OUI_FILE) as infile:
print "Inserting data into table, please wait..."
for line in infile:
#do_something_with(line)
if re.search("(hex)", line):
try:
mac,vendor = line.strip().split("(hex)")
except:
mac = vendor = ''
#print line.strip().split("(hex)")
#print mac.strip().replace("-",":").lower(), vendor.strip()
if mac!='' and vendor!='':
sql = "INSERT INTO ouiList "
sql+= "(oui,vendor) "
sql+= "VALUES ("
sql+= "'%s'," % mac.strip().replace("-",":").lower()
sql+= "'%s'" % vendor.strip().replace("'","`")
sql+= ")"
#print sql
try:
DB_CURSOR.execute(sql)
DB.commit()
except Exception, e:
DB.rollback()
print "Not inserted because error: "

# count the number of lines in the table
sql = ("SELECT count(id) FROM ouiList WHERE 1")
DB_CURSOR.execute(sql)
result = DB_CURSOR.fetchone()
if result:
print 'Completed: Total lines in table is: {0}'.format(result[0])
else:
print "Unable to count lines in table"

infile.close()
DB_CURSOR.close()
DB.close()
#

if __name__=="__main__":
main()

SAVE & EXIT. Now execute it.

python mactosql.sh

If no error occurs you will be seeing a new table named macdb with all vendors names as well 🙂

db import successfull.JPG

 

to get trimmed OUI, use following

sourcecode]mysql -uroot -pSQLPASS –skip-column-names -s -e “use radius; select LEFT(mac, 8) from users;”/sourcecode]


Now connect any user as normal, and see the mysql.log file

2018-02-20T06:41:04.593739Z 24 Query INSERT INTO radacct (acctsessionid, acctuniqueid, username, realm, nasipaddress, nasportid, nasporttype, acctstarttime, acctstoptime, acctsessiontime, acctauthentic, connectinfo_start, connectinfo_stop, acctinputoctets, acctoutputoctets, calledstationid, callingstationid, acctterminatecause, servicetype, framedprotocol, framedipaddress, acctstartdelay, acctstopdelay, xascendsessionsvrkey) VALUES ('81100003', '702d22ac0a080f57', 'zaib', '', '101.11.11.253', '9', 'Ethernet', '2018-02-20 11:41:04', NULL, '0', 'RADIUS', '', '', '0', '0', 'service1', '00:0C:29:B9:D8:A0', '', 'Framed-User', 'PPP', '192.168.50.255', '0', '0', '')

# As soon we receive entry for INSERT, TRIGGER will fire-up, see below log

2018-02-20T06:41:04.594676Z 24 Query SET @mac = (SELECT count(*) from radcheck where username=New.username and attribute='Calling-Station-ID')

2018-02-20T06:41:04.594871Z 24 Query INSERT into radcheck (username,attribute,op,value) values (NEW.username,'Calling-Station-ID',':=',NEW.callingstationid)
2018-02-20T06:41:04.595020Z 24 Query UPDATE users SET mac = NEW.callingstationid where username = NEW.username

2018-02-20T06:41:04.595151Z 24 Query SET @mactrim = (select LEFT(mac, 8) from users where username=New.username)
2018-02-20T06:41:04.595256Z 24 Query SET @macvendor1 = (select vendor from macdb where oui=@mactrim)

2018-02-20T06:41:04.607786Z 24 Query UPDATE users SET macvendor=@macvendor1 where username = NEW.username

& FINALLY we will see records in USERS table that will be displayed on FRONTEND 🙂

1- identify vendor from mac

Older Posts »

%d bloggers like this: