Syed Jahanzaib Personal Blog to Share Knowledge !

March 9, 2015

Mikrotik: WAN Data Monitoring via Scripting

Filed under: Mikrotik Related — Tags: , , , , — Syed Jahanzaib / Pinochio~:) @ 12:08 PM

DUETO WORDPRESS CODING MESSED UP, THIS POST MAY NOT HELP MUCH, IF YOU NEED INFO ON IT, EMAIL ME AND I WILL PROVIDE NEW SCRIPT.

As requested by few , specially from PK who are using PTCL DSL links and on few packages, PTCL have embossed Quota limit of 300GB (or likewise) and if the user exceed this limit, some penalty or extra charges are added in the monthly bill which is in some cases creates extra financial burden for DCN. Following are few scripts which can help you in this particular case that if the link crosses specific percentage of Quota limit, the link can be disabled, or second link (if available) can be activated or whatever operator chooses can be done by modifying the script action.

[The script idea was taken from the Mikrotik forum but those scripts didn’t performed well, therefore I modified and re-created few sections in some parts ]

Hope it will help you. Leave your comments …

Regard’s
Syed Jahanzaib



 

Scenario:

Two DSL Links , Primary DSL (ether1) have 4mb link with 50 GB data limit, where as secondary (or backup) link (ether2) have 2mbps with no data limit. Our requirement is to use Primary DSL Link which have higher bandwidth capacity and if the 50GB quota is reached to 90%, it should give warning (or email or disable Primary Link interface which have distance value of 1 and ENABLE Secondary Link which have distance value of 2 to auto take over)

Primary DSL = ether1
Secondary DSL = ether2

 


Scripts Description: [Tested with Mikrotik 6.27 Only]

In this example I have created 3 Scripts.

Benefit of using scripting method is that it can save values in a text file, so even if a route reboots, the script can retrieve last recorded data form the text file , that’s the biggest advantage for DCN 😉 YKWIM

1- monitor_data

This script will get gather data from the interface stats and add it to text file which will be later read by second script which does the required action based on the counters in this file

2- check_useage

This script will read values from the text file created by above script, (monitor_data) and act accordingly to either ignore if the data usage is below specific value, and to perform what action if the data usage is above specific value.

3- check_date

This script will check in which the data value should be set to ZERO, usually start of month. So that counters can be reset.

 


 

1- MONITOR_DATA    [To get interface usage data]

Set this script in scheduler to run every hour. (collecting data every hour is reasonable IMHO]

# MONITOR_DATA Script (to get interface usage in file)
# First Part of WAN Data Quota Monitoring System
# Syed Jahanzaib / aacable@hotmail.com
# https://aacable.wordpress.com

# Set WAN interface you want to monitor
:local INT ether1

# Gather Interface RX bytes
:local counter [/interface get $INT rx-byte]

# Set Counter DATA value gathered by Interface RX bytes and add  TX bytes to get combined value
:set $counter ($counter + [/interface get $INT tx-byte])

:local traffic

# If previous data file not found, then create new one
:if ([:len [/file find where name=wandatauseage.txt]] < 1 ) do={
/file print file=wandatauseage.txt where name=wandatauseage.txt;
# Add some delay, for slow or high load routers
/delay delay-time=2;

# Adding Traffic Value in the file

/file set wandatauseage.txt contents="0";
};
:local before value=[/file get wandatauseage.txt contents]

:if ($counter > $before) do={
/file set wandatauseage.txt contents=$counter
} else= {
:set $traffic ($counter+$before)
/file set wandatauseage.txt contents=$traffic
};

 


2- check_usage    [To read data usage from file, and act accordingly]

Set this script in scheduler to run every hour or two.

# CHECK_USAGE > CHECK DATA USAGE Script (to read from file and act accordingly)
# Second Part of WAN Data Quota Monitoring System
# Syed Jahanzaib / aacable@hotmail.com
# https://aacable.wordpress.com</pre>
local traffic ([/file get wandatauseage.txt contents] / 1024 / 1024 / 1024)

# Set the WAN Interface Quota limit in GB
:local limit 50
:local percent ($traffic*100 / $limit)

# Set the percentage as required
:if ($percent >= 90) do={
:log warning "WAN INTERFACE Quota Exceeded 90% of $limit GB / zaib"

# OR Take specific action, either disable interface, change route, email , punch the fun-bags ; ) YKWIM ; ) etc etc
# Example Email like , but first make sure you have configured your email client properly
# /tool e-mail send to=YOUR_EMAIL_ADDRESS subject="WAN Traffic Quota warning: $percent% reached at $[/system clock get date], $[/system clock get time]" body="The traffic amount is $traffic GB \r\nThis is $percent% of the monthly limit\r\nQuota Monitor by Z"

}

Example if the quota reaches to its limit.

 

log_warning.

Example of EMAIL if the quota reaches to its limit.

 

email-war


 

3- check-date    [To reset counters in text file, if its 1st of the month]

Set this script in scheduler to run every daily in night at 1:00am, and if it found date to 1st of the month, it will reset the data counter file to zero.

# CHECK_DATE > CHECK DATE and RESET Counters on specific date ...
# Third Part of WAN Data Quota Monitoring System
# Syed Jahanzaib / aacable@hotmail.com
# https://aacable.wordpress.com

if  ([:pick [/system clock get date] ([:find [/system clock get date] "/" ] + 1) 6 ] = "01") do={/file set wandatauseage.txt contents="0"}

 


 

 

 

ME

Advertisements

26 Comments »

  1. Nice to see this script .. but problem is that we already have electrical issues … so several time server will reboot or i reboot server manually for any reasons … so its safe our last usages ?.

    Like

    Comment by Abubaker SIddiq Lasania — March 9, 2015 @ 1:23 PM

    • read the SCRIPT DESCRIPTION” section, which states
      “Benefit of using scripting method is that it can save values in a text file, so even if a route reboots, the script can retrieve last recorded data form the text file , that’s the biggest advantage for DCN 😉 YKWIM”

      Monitoring via queue or interface stats is the easiest way to go, but i selected scripting way , because we can put last used values of interface stats in the text file (by addition) and even if the router reboots, no issue, last valued will be read from last recorded time. 🙂

      Like

      Comment by Syed Jahanzaib / Pinochio~:) — March 9, 2015 @ 1:34 PM

  2. sir kia ham ye file wandatauseage.txt open kar sakty hain ?

    Like

    Comment by qamarzaman — March 9, 2015 @ 9:13 PM

  3. Nice to see this script Syed Jahanzaib bhai

    Like

    Comment by faisal — March 10, 2015 @ 2:51 AM

    • mera matlb hy k ham file ko open kar k apni usage chek kar lia kary

      Like

      Comment by qamarzaman — March 11, 2015 @ 2:51 PM

      • Goto files and you will see this file (but first run the MONITOR_DATA SCRIPT one or two times, to create the file auto. then download it your desktop by dragging and open it in note pad
        it will show you value in bytes, use the formula to convert it to your need.

        Like

        Comment by Syed Jahanzaib / Pinochio~:) — March 11, 2015 @ 3:02 PM

  4. this script is not working for me. is there any problem, I have changed ether1 to WAN1 as per my configuration. file is then generated but contents are different from desired. what might be the problem.
    File data is as under

    # mar/16/2015 16:47:32 by RouterOS 6.27
    # software id = 702M-5YAD
    #
    # NAME TYPE SIZE CREATION-TIME

    Like

    Comment by qdsl — March 16, 2015 @ 5:07 PM

  5. do you can edit to used in mikrotik v5

    Like

    Comment by ابراهيم الشيمى — June 8, 2015 @ 3:44 AM

  6. I tried making sense of the queue examples for ages. This is not as complicated. It was easy to implement and it works.Thank you.

    Like

    Comment by Ronald de Villiers — September 1, 2015 @ 2:07 AM

  7. Hi,

    I think your logic is flawed in a number of ways, either that or I have misinterpreted the scripts.

    a) if the router is rebooted mid month, wandatauseage.txt contents will increase faster than usage on the interface
    b) it will not reset properly once a month as expected

    Lets start with this section, I think it is going to compound increase the data amount each time the script runs

    :local before value=[/file get wandatauseage.txt contents]

    :if ($counter > $before) do={
    /file set wandatauseage.txt contents=$counter
    } else= {
    :set $traffic ($counter+$before)
    /file set wandatauseage.txt contents=$traffic
    };

    Take the simple scenario that after a router reboot
    the contents of wandatauseage.txt is 10GB
    after an hour of use the counter on the interface is 1GB of traffic

    therefore
    $before =10GB
    $counter=1GB

    which executes as follows
    :if (1 > 10) do={
    /file set wandatauseage.txt contents=$counter
    } else= {
    :set $traffic (10+1)
    /file set wandatauseage.txt contents=11
    };

    wandatausage.txt now reads 11GB which is fine

    BUT when it executes an hour later to collect some more data we have a problem. Lets say that there has been no more usage on the interface so tx-byte and rx-byte still equal 1GB. wandatausage.txt will still increase

    $before=11GB
    $counter=1GB

    which executes as follows
    :local before value=11
    :if (1 > 11) do={
    /file set wandatauseage.txt contents=$counter
    } else= {
    :set $traffic (11+1)
    /file set wandatauseage.txt contents=11
    };

    The wandatausage.txt file now contains a 13GB counter instead of a 12GB counter.

    This error will keep compounding every time the scripts runs. Even if there was no further usage, each time the script runs; $counter will be less than $before and they will get added together and put in wandatauseage.txt which will keep increasing

    Also once a month this runs

    if ([:pick [/system clock get date] ([:find [/system clock get date] “/” ] + 1) 6 ] = “01”) do={/file set wandatauseage.txt contents=”0″}

    It resets the wandatausage.txt to 0 on the first of the month but as it does not reset the interface counters at the same time the first time the collector runs it returns the file to the point it was at at the end of last month. And unfortunately tx-byte and rx-byte are read-only so they cannot be reset. So this script will only work for the first month, unless you reboot the router each month to reset the interface counters to 0.

    Maybe I do not understand properly but that is the way it looks to me?

    Andrew

    Like

    Comment by Andrew — September 5, 2015 @ 1:33 PM

    • I have used this script on one device for testing. Initially it seemed to work until I noticed the same issue after a while.

      MONITOR_DATA Script updates the contents of wandatausage.txt every hour, but the tx/rx bytes are still increasing in stead of being reset and gets added again.

      The tx/rx bytes are read only and will only reset if you reboot device and it is unacceptable to reboot every hour.

      Can any one post a solution? Please.

      Like

      Comment by digiron — December 1, 2015 @ 10:37 AM

  8. the first script not working for me .
    I have RB750GL, can you help me ???

    Like

    Comment by akha666 — September 12, 2015 @ 2:14 AM

  9. I used this script on one device for testing. I initially thought it is working, until I discovered a problem.

    MONITOR_DATA Script is adding rx/tx bytes to wandatausage.txt content every hour, but does not reset rx/tx bytes.
    So after the next hour the rx/tx bytes that gets added is more than it is supposed to be.

    The rx/tx bytes is read only and resets only after a reboot, but it is unacceptable to reboot every hour.

    Can any one post a solution? Please.

    Like

    Comment by Ronald de Villiers — December 1, 2015 @ 10:51 AM

    • I have played around with this and have the following solution.
      Create 1st script called traffic.
      ################################################################
      #
      # Script to create items for traffic monitoring.
      #
      ################################################################
      # If previous data file not found, then create new one
      #
      :if ([:len [/file find where name=useage.txt]] =$lastCount) do={
      #
      # Else
      # 1) get traffic value from file
      # 2) add lastcount session to traffic
      # 3) update traffic value in usage.txt file
      # 4) update traffic comment with total interface usage
      #
      } else={
      :local traffic [/file get useage.txt contents]
      :set $traffic ($traffic+$lastCount)
      /file set useage.txt contents=$traffic
      :local megs ($traffic / 1024 / 1024)
      :local gigs ($traffic / 1024 / 1024 / 1024)
      /system script set traffic comment=”Tot rxtx MB:$megs or GB:$gigs”
      }
      ####################################################################
      # Update note with current interface usage
      #
      /system note set note=$session
      #
      ####################################################################
      # update monitor comment with current interface usage
      #
      :local megs ($session / 1024 / 1024)
      :local gigs ($session / 1024 / 1024 / 1024)
      /system script set monitor comment=”Cur rxtx MB:$megs or GB:$gigs”
      #
      ####################################################################

      Now create scheduler to run monitor script every 10 seconds

      What does this do?

      if you open System > Scripts
      1) The monitor script keeps a running count of current rx + tx in comment
      2) Any time router reboots the running count gets added to traffic script comment
      3) Todo if you want to email daily usage reboot at midnight and email traffic value

      Like

      Comment by Ronald de Villiers — December 20, 2015 @ 2:32 PM

      • Sorry my bad here is monitor script

        ####################################################################
        # Set WAN interface you want to monitor
        #
        :local INT ether1
        #
        ####################################################################
        # RXTX Part 1 – Add Interface RX bytes to session
        #
        :local session [/interface get $INT rx-byte]
        #
        ####################################################################
        # RXTX Part 2 – Add Interface TX bytes to session
        #
        :set $session ($session + [/interface get $INT tx-byte])
        #
        ####################################################################
        # Get lastcount value from note to compare
        #
        :local lastCount [/system note get note]
        #
        ####################################################################
        # If session traffic is increasing do nothing
        #
        :if ($session>=$lastCount) do={
        #
        # Else
        # 1) get traffic value from file
        # 2) add lastcount session to traffic
        # 3) update traffic value in usage.txt file
        # 4) update traffic comment with total interface usage
        #
        } else={
        :local traffic [/file get useage.txt contents]
        :set $traffic ($traffic+$lastCount)
        /file set useage.txt contents=$traffic
        :local megs ($traffic / 1024 / 1024)
        :local gigs ($traffic / 1024 / 1024 / 1024)
        /system script set traffic comment=”Tot rxtx MB:$megs or GB:$gigs”
        }
        ####################################################################
        # Update note with current interface usage
        #
        /system note set note=$session
        #
        ####################################################################
        # update monitor comment with current interface usage
        #
        :local megs ($session / 1024 / 1024)
        :local gigs ($session / 1024 / 1024 / 1024)
        /system script set monitor comment=”Cur rxtx MB:$megs or GB:$gigs”
        #
        ####################################################################

        Like

        Comment by Ronald de Villiers — December 20, 2015 @ 2:36 PM

      • gr8. i will test it.

        Like

        Comment by Syed Jahanzaib / Pinochio~:) — December 21, 2015 @ 7:23 PM

  10. Selamun Aleykum Zaib, Thanks for the script. But script dont create any file in ROS 6.34. Any advice would be appreciated. Thank you.

    Like

    Comment by Ayhan — March 30, 2016 @ 2:37 PM

    • Updated scripts.

      Review of script function
      A. initTrafficFiles
      Create or recreate relevant files for monitoring usage on start of new monthly schedule.
      B. Monitor
      Update text files to accumulate all rx-bytes and tx-bytes even if router reboots.
      I scheduled this script to run every minute.
      C. curUsageViewLog
      Wait awhile (2minutes) before running this script to give first two scripts time to update.
      Then run this script to view log of current usage totals.
      You can then use this data to email clients usage as desired.

      Scripts

      A. initTrafficFiles
      ################################################################
      #
      # Script to create items for traffic monitoring.
      #
      ################################################################
      #
      # If previous data files found, remove them
      #
      ################################################################
      /file remove rx.txt
      /file remove rxCur.txt
      /file remove tx.txt
      /file remove txCur.txt
      ################################################################
      #
      # If previous data files not found, then create new ones
      #
      ################################################################
      # Create files for data totals
      # to acumelate all traffic recieved
      # and all traffic sent on ether1 interface
      #
      :if ([:len [/file find where name=rx.txt]] =$rxCount) do={
      #
      # Else
      # rx-byte is smaller than saved rxCur value and router rebooted
      # add 1 to didReboot flag
      #
      } else={
      :set $didReboot ($didReboot+1);
      }
      ####################################################################
      #
      # 4.2 Check tx-byte
      # Compare saved counter value from file and active tx-byte value
      # as long as tx-byte is increasing router did not reboot
      :if ($txCur>=$txCount) do={
      #
      # Else
      # tx-byte is smaller than saved txCur value and router rebooted
      # add 1 to didReboot flag
      #
      } else={
      :set $didReboot ($didReboot+1);
      }
      ####################################################################
      #
      # 5. Check flag if router rebooted
      #
      # 5.1 If router rebooted
      #
      :if ($didReboot>=1) do={
      :set $rxIn ($rxIn+$rxCount)
      /file set rx.txt contents=$rxIn
      :set $txOut ($txOut+$txCount)
      /file set tx.txt contents=$txOut
      } else={
      }
      ####################################################################
      #
      # 6.1 Update recieved counter value file
      #
      :set rxCount ($rxCur);
      /file set rxCur.txt contents=$rxCount
      #
      ####################################################################
      #
      # 6.2 Update sent counter value file
      #
      :set txCount ($txCur);
      /file set txCur.txt contents=$txCount
      #
      ####################################################################
      };
      ####################################################################

      C. curUsageViewLog
      ####################################################################
      #
      # Display Current Usage in Log Script
      #
      ####################################################################
      #
      # 1. Set interface you want to monitor
      #
      :local INT ether1
      #
      ####################################################################
      #
      :local rxCur [/interface get $INT rx-byte]
      #
      :local rxIn [/file get rx.txt contents]
      #
      ####################################################################
      #
      :local txCur [/interface get $INT tx-byte]
      #
      :local txOut [/file get tx.txt contents]
      #
      ####################################################################
      #
      :local rxTot ($rxIn+$rxCur)
      :local megs ($rxTot / 1024 / 1024)
      :local gigs ($rxTot / 1024 / 1024 / 1024)
      :log info “MB:$megs or GB:$gigs rxTot:$rxTot Uploaded”
      ####################################################################
      #
      :local txTot ($txOut+$txCur)
      :local megs ($txTot / 1024 / 1024)
      :local gigs ($txTot / 1024 / 1024 / 1024)
      :log info “MB:$megs or GB:$gigs txTot:$txTot Downloaded”
      ####################################################################
      };
      ####################################################################

      Like

      Comment by Ronald de Villiers — July 23, 2016 @ 7:52 PM

  11. A. initTrafficFiles
    ################################################################
    #
    # Script to create items for traffic monitoring.
    #
    ################################################################
    #
    # If previous data files found, remove them
    #
    ################################################################
    /file remove rx.txt
    /file remove rxCur.txt
    /file remove tx.txt
    /file remove txCur.txt
    ################################################################
    #
    # If previous data files not found, then create new ones
    #
    ################################################################
    # Create files for data totals
    # to acumelate all traffic recieved
    # and all traffic sent on ether1 interface
    #
    :if ([:len [/file find where name=rx.txt]] < 1 ) do={
    /file print file=rx.txt where name=rx.txt;
    /file print file=tx.txt where name=tx.txt;
    #
    # Create files for storing current session value (counter)
    # Sent and recieved data values zeros everytime device restarts.
    # Every minute data will be updated in these files until device
    # restart and last values will be added to files above to
    # acumelate total traffic.
    #
    /file print file=rxCur.txt where name=rxCur.txt;
    /file print file=txCur.txt where name=txCur.txt;
    #
    # Add some delay, for slow or high load routers
    #
    /delay delay-time=2;
    #
    # Initialise
    #
    :local INT ether1
    #
    :local rxIn 1
    /file set rx.txt contents=$rxIn
    /file set rxCur.txt contents=$rxIn
    #
    :local txOut 1
    /file set tx.txt contents=$txOut
    /file set txCur.txt contents=$txOut
    #
    ################################################################
    };
    ################################################################

    Like

    Comment by Ronald de Villiers — July 23, 2016 @ 8:07 PM

  12. B. Monitor
    ####################################################################
    #
    # Data Usage Monitoring Script
    #
    ####################################################################
    #
    # 1. Set interface you want to monitor
    #
    :local INT ether1
    #
    ####################################################################
    #
    # 2. Interface counters reset to zero everytime router reboot.
    # So we need a way to save the last known value before reboot.
    # Here is one solution.
    #
    # 2.1 Retrieve current session bytes received from interface ether1
    #
    :local rxCur [/interface get $INT rx-byte]
    #
    # 2.2 As long as rx-byte keeps increasing we want to save the value
    # periodically to a file and read it after an interval for
    # comparrison to current rx-byte value.
    # So now we will retrieve last accumulated bytes saved from
    # rxCur.txt file.
    #
    :local rxCount [/file get rxCur.txt contents];
    #
    # 2.3 We also want to save this usage to a file 1st time router
    # reboots or add consecutive sessions accumelated bytes to the
    # saved contents of the file.
    # So now we will retrieve last accumulated bytes saved from
    # rx.txt file.
    #
    :local rxIn [/file get rx.txt contents]
    #
    ####################################################################
    #
    # 3. Now follow the same procedure for transmitted bytes.
    #
    # 3.1 Retrieve session bytes transmitted from interface ether1
    #
    :local txCur [/interface get $INT tx-byte]
    #
    # 3.2 Retrieve accumulated bytes transmitted from txCur.txt
    #
    :local txCount [/file get txCur.txt contents];
    #
    # 3.3 Retrieve last saved total of transmitted traffic from tx.text
    #
    :local txOut [/file get tx.txt contents]
    #
    ####################################################################
    #
    # 4. We need to know when router rebooted
    # So create a flag didReboot and initialize it
    #
    :local didReboot 0
    #
    ####################################################################
    #
    # 4.1 Check rx-byte
    # Compare saved counter value from file and active rx-byte value
    # as long as rx-byte is increasing router did not reboot
    :if ($rxCur>=$rxCount) do={
    #
    # Else
    # rx-byte is smaller than saved rxCur value and router rebooted
    # add 1 to didReboot flag
    #
    } else={
    :set $didReboot ($didReboot+1);
    }
    ####################################################################
    #
    # 4.2 Check tx-byte
    # Compare saved counter value from file and active tx-byte value
    # as long as tx-byte is increasing router did not reboot
    :if ($txCur>=$txCount) do={
    #
    # Else
    # tx-byte is smaller than saved txCur value and router rebooted
    # add 1 to didReboot flag
    #
    } else={
    :set $didReboot ($didReboot+1);
    }
    ####################################################################
    #
    # 5. Check flag if router rebooted
    #
    # 5.1 If router rebooted
    #
    :if ($didReboot>=1) do={
    :set $rxIn ($rxIn+$rxCount)
    /file set rx.txt contents=$rxIn
    :set $txOut ($txOut+$txCount)
    /file set tx.txt contents=$txOut
    } else={
    }
    ####################################################################
    #
    # 6.1 Update recieved counter value file
    #
    :set rxCount ($rxCur);
    /file set rxCur.txt contents=$rxCount
    #
    ####################################################################
    #
    # 6.2 Update sent counter value file
    #
    :set txCount ($txCur);
    /file set txCur.txt contents=$txCount
    #
    ####################################################################
    };
    ####################################################################

    Like

    Comment by Ronald de Villiers — July 23, 2016 @ 8:08 PM

  13. C. curUsageViewLog
    ####################################################################
    #
    # Display Current Usage in Log Script
    #
    ####################################################################
    #
    # 1. Set interface you want to monitor
    #
    :local INT ether1
    #
    ####################################################################
    #
    :local rxCur [/interface get $INT rx-byte]
    #
    :local rxIn [/file get rx.txt contents]
    #
    ####################################################################
    #
    :local txCur [/interface get $INT tx-byte]
    #
    :local txOut [/file get tx.txt contents]
    #
    ####################################################################
    #
    :local rxTot ($rxIn+$rxCur)
    :local megs ($rxTot / 1024 / 1024)
    :local gigs ($rxTot / 1024 / 1024 / 1024)
    :log info “MB:$megs or GB:$gigs rxTot:$rxTot Uploaded”
    ####################################################################
    #
    :local txTot ($txOut+$txCur)
    :local megs ($txTot / 1024 / 1024)
    :local gigs ($txTot / 1024 / 1024 / 1024)
    :log info “MB:$megs or GB:$gigs txTot:$txTot Downloaded”
    ####################################################################
    };
    ####################################################################

    Like

    Comment by Ronald de Villiers — July 23, 2016 @ 8:09 PM


RSS feed for comments on this post. TrackBack URI

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: