Syed Jahanzaib Personal Blog to Share Knowledge !

September 26, 2022

BIND – Quick Reference Notes for De Beast!



 


Some DRY theory FOR BIND DNS: [copy pasted text]

BIND can be used to run a caching DNS server or an authoritative name server, and provides features like load balancing, notify, dynamic update, split DNS, DNSSEC, IPv6, and more. Berkeley Internet Name Domain (BIND) is the most popular Domain Name System (DNS) server in use today.

Common BIND Operations and Features

BIND provides the following main features and capabilities:

  • Authoritative DNS – publishes DNS records under the server’s authoritative control
  • Cache-Only DNS – provides DNS name resolution for applications by relaying requests to an authoritative server, or acting as a secondary DNS server that holds a read-only copy of the authoritative zone file
  • Basic DNS load balancing – can be achieved using multiple A records for one name
  • DNS notify – allows primary DNS servers to notify secondary servers of changes to zone data
  • Dynamic update – a method for adding, replacing or deleting records in a primary server by sending a special type of DNS message (defined in RFC 2136)
  • Incremental zone transfer (IXFR) – allows secondary servers to transfer only modified data, instead of the entire DNS zone (defined in RFC 1995)
  • Split DNS – allows different views of the DNS space to internal and external resolvers – for example to hide internal DNS data from external clients

Advantages of BIND

BIND enjoys several important advantages, which make it by far the most popular DNS server on the Internet:

  • Broad usage and strong community BIND is a De-Facto standard for DNS in Linux systems, and is actively supported by a large open source community.
  • Stable – BIND is used in millions of production DNS servers and is known for stable and predictable operation.
  • Good platform support – BIND supports Linux, NetBSD, FreeBSD, OpenBSD, macOS and Windows.
    Comprehensive feature set – BIND is one of the only DNS servers that covers all basic DNS functionality – see Wikipedia’s detailed comparison of BIND with other DNS servers.
  • BIND enjoys several important advantages, which make it by far the most popular DNS server on the Internet:
  • Broad usage and strong community – BIND is a de facto standard for DNS in Linux systems, and is actively supported by a large open source community.
  • Stable – BIND is used in millions of production DNS servers and is known for stable and predictable operation.
  • Good platform support – BIND supports Linux, NetBSD, FreeBSD, OpenBSD, macOS and Windows.
  • Comprehensive feature set – BIND is one of the only DNS servers that covers all basic DNS functionality – see Wikipedia’s detailed comparison of BIND with other DNS servers.

MY NOTES:

Following post contains quick reference notes on howto deploy BIND DNS server as MASTER/SLAVE for LAN/WAN. This post also contains some tweaks / tips to make life little easier for my self. At some of my clients, I deployed BIND (at some placed UNBOUND) & created few custom commands menu so that client uses the webmin gui module of webmin to modify records , service reload , service restart, dns test etc. Webmin panel example for operator.

Webmin for BIND Management

Usermin Module for BIND Management (Under WEBMIN)


Scenario:

We have a public domain name example.com & we want to host our own authoritative public dns server so that dependency on host provider reduced & we can manage all types of records at our site. All Internal clients can use our DNS to resolve dns related queries & other hosts on  internet can resolve our domain related hosts via our DNS server. In this scenario our clients will use our dns as primary for resolving, and other internet zones resolving will be forwarded to other public DNS.

Assumptions before proceeding …

  • Domain Name: exmaple.com
  • We will have two servers which will be designated as our DNS name servers. This guide will refer to these as ns1 and ns2
  • We will have two additional client pc’s records that will be using the DNS infrastructure you create, referred to as
  • We will configure MASTER/SLAVE configuration so that in case primary goes down, secondary will still be available to fulfill resolving.
    Primary DNS: ns1.example.com – 10.0.0.1
    Secondary DNS : ns1.example.com – 10.0.0.2
  • Using your hosting provider CPANEL, change the ns pointing to your local DNS (local DNs must have static public ip address & must be accessible/allowed via any firewall u might have)
  • Every time you modify zone records, make sure to reload the bind9 service or use the bash script which can change the records as per date/ss & reload bind service which will trigger SLAVE server to replicate from Master.

When configured as a recursive server, BIND will first use the zones for which it is authoritative before walking down the entire DNS tree from the root/forwarder. for other ZONES , we have configured forwarders, (and root servers can be used in absence of forwarder). We can even create multiple zones, the only limitation is that you will need to maintain all the records in zones you create and ensure they are up to date.

Once we configure & test our DNS, next step is to change NS records on your hosting provider cpanel so that internet can resolve your domain name using your hosted DNS server.


Housekeeping Stuff !

Before installing Bind, we need to perform some housekeeping Stuff first,

NTP:

Make sure you have correct date time syncing with any NTP server with appropriate Time zone as well.

apt-get -y install ntp ntpdate
# Following is for KARCAHI/PK zone, u may select yours accordingly
cp /usr/share/zoneinfo/Asia/Karachi /etc/localtime

BIND installation on Ubuntu 16.4:

apt-get update
apt-get install -y bind9 bind9utils bind9-doc dnsutils

Force IPV4 Only

  • OS Level:

Copy Paste below to disable ipv6

echo "net.ipv6.conf.all.disable_ipv6 = 1" >> /etc/sysctl.conf
echo "net.ipv6.conf.default.disable_ipv6 = 1" >> /etc/sysctl.conf
echo "net.ipv6.conf.lo.disable_ipv6 = 1" >> /etc/sysctl.conf
sysctl -p
  • BIND Level ipv4 config:
nano /etc/default/bind9

Example:

# run resolvconf?
RESOLVCONF=no
# startup options for the server
OPTIONS="-4 -u bind"
  • RESOLV.CONF:

Once BIND is installed, you may want to modify the resolv.conf to look something like below
(For permanent changes, edit  appropriate settings in /etc/network/interfaces file)

search example.com # example.com is your domain name
nameserver 10.0.0.1 # your local host ip

** Configuration files for BIND9 – MASTER **

Default installation location folder is /etc/bind9. We will create subfolder named ZONES to hold all zones files separately. Also we will add forwarder/reverse records files location so that BIND should know where our ZONES files are located.

BIND9 DNS Zones Location & Master/Slave Config File


  • named.con.local [It will contain ZONE Files & its parameters]

Edit below file to enter zone name, type, file location etc

nano /etc/bind/named.conf.options
//The following code defines the forwarder lookup zone.
zone "example.com" {
type master;

allow-update { none; }; //Since this is the primary DNS, it should be none.
allow-transfer { 10.0.0.2; }; //Allow Transfer of zone from the master server
also-notify { 10.0.0.2; }; //Notify slave for zone changes

file "/etc/bind/zones/forwarder.example.com";
};

//The following code defines the reverse lookup zone.
zone "11.11.101.in-addr.arpa" {
type master;
allow-update { none; }; //Since this is the primary DNS, it should be none.
allow-transfer { 10.0.0.2; }; //Allow Zone Transfer from master server to ns2 slave server
also-notify { 10.0.0.2; }; //Notify slave for zone changes

file "/etc/bind/zones/reverse.example.com";
};

  • named.conf.options [It will contain ACL , Forwarders & related parameters]
nano /etc/bind/named.conf.options
# First we will Define IP pool ACL so that only these ips will be allowed to resolve internal/exteral dns resolving
acl "trusted" {
127.0.0.0/8;
10.0.0.0/8;
# or any other pool/x;
};

options {
directory "/var/cache/bind";
recursion yes; # enables recursive queries
allow-recursion { trusted; }; # allows recursive queries from "trusted" clients ONLY, external clients will be able to resolve records related to exmaple.com only, internal clients mentioned in ACL can query all records lan/wan
listen-on { 10.0.0.1; }; # ns1 private IP address - listen on private network only, ns1 local ip
listen-on { 127.0.0.1; }; # ns1 private IP address - listen on private network only
allow-transfer { 10.0.0.2; }; # Allow Transfer of zone to secondary dns ns2.example.com
# Forward all other zones queries to following public dns (Only Trusted ACL will be able to resolve external queries
# This can help to make the responses to these queries faster by reducing the load on the local network.
forwarders {
1.1.1.1;
8.8.8.8;
};
dnssec-validation auto;
auth-nxdomain no; # conform to RFC1035
listen-on-v6 { any; };
};

  • ZONE FILES Location and FORWARDER Configuration File:

It will contain zone details , all dns related records

First Create a folder named ZONES which will contain our zones files (for/rev)

mkdir /etc/bind/zones
### Now create a new file for forwarders records ###
nano /etc/bind/zones/forwarder.exmaple.com

FORWARDER Configuration File:

; BIND FORWARDER A RECORDS
; example.com- LOCLA HOSTED DNS FOR LAN/WAN (RESOLVER/RECURSIVE/CACHING/AUTHORITATIVE)
; 24-SEPTEMBER-202
; BIND DNS Configuration - by Syed Jahanzaib
; aacableAThotmailDOTcom / https://aacableDOTwordpressDOTcom

$ORIGIN example.com.
$TTL 86400 ; (1 day)
@ IN SOA ns1.example.com. zaib.example.com. (
2022092420 ; serial
14400 ; refresh (4 hours)
1800 ; retry (30 minutes)
1209600 ; expire (2 weeks)
3600 ; minimum (1 hour)
)

@ IN NS ns1.example.com.
@ IN NS ns2.example.com.
@ IN A 10.0.0.1
example.com. IN A 10.0.0.3 ; website
example.com. IN MX 10 example.com. ; email record
mail.example.com. IN CNAME example.com.
www.example.com. IN CNAME example.com.
www IN CNAME example.com.
ns1 IN A 10.0.0.1
ns2 IN A 10.0.0.2
host1 IN A 10.0.0.10
host2 IN A 10.0.0.11

Save & Exit.


REVERSE Configuration File:

It will contain reverse PTR records & also$GENERATE statement for auto generation of unused ips ptr records for 

nano /etc/bind/zones/reverse.exmaple.com

Example File:

; BIND REVERSE PTR RECORDS
; example.com - LOCLA HOSTED DNS FOR LAN/WAN (RESOLVER/RECURSIVE/CACHING/AUTHORITATIVE)
; 24-SEPTEMBER-202
; BIND DNS Configuration - by Syed Jahanzaib
; aacableDOThotmail.com / https://aacableDOTwordpressDOTcom

$TTL 43200
@ IN SOA example.com. ns1 (
2022092416 ;
14400 ;
1800 ;
1209600 ;
3600 ;
)

@ IN NS ns1.example.com.
1 IN PTR ns1.example.com.
2 IN PTR ns2.example.com.
10 IN PTR host1.example.com.
11 IN PTR host1.example.com.

;Generate 100 hosts PTR record automatically
$GENERATE 1-100 $ IN PTR 10-0-0-$.example.com.

Save & Exit.


BIND ZONE FILES CHECK CMD’s

Every time you edit the zone files, its better to run a checkzone cmd to ensure there are no syntax mistakes.

  • a) Main File Syntax Check
named-checkconf

If your named configuration files have no syntax errors, there won’t be any error messages and you will return to your shell prompt. If there are problems with your configuration files, review the error message , troubleshoot & fix it, then try named-checkconf again

  • b) Zone File Syntax Check
named-checkzone example.com /etc/bind/zones/forwarder.example.com

named-checkzone example.com /etc/bind/zones/reverse.example.com

** Configuration files for BIND9 – SLAVE **

Configuring SLAVE is quite simple. Install the BIND on 2nd server (ns2.example.com). set its resolve to its own ip. Edit below files

nano /etc/bind/named.conf.local

Example File:

//The following code defines the forwarder lookup zone.
zone "example.com" {
type slave;
masters { 10.0.0.1; };
masterfile-format text;
file "/var/cache/bind/forwarder.example.com";
serial-update-method unixtime;
};

//The following code defines the reverse lookup zone.
zone "0.0.10.in-addr.arpa" {
type slave;
masters { 10.0.0.1; };
masterfile-format text;
file "/var/cache/bind/reverse.example.com";
serial-update-method unixtime;
};

Save & Exit.

Now edit the /etc/bind/named.conf.options & copy paste all data from master ns named.conf.options file. You have to modify just one parameter listen-on to point it to ns2 ip example

  • listen-on { 10.0.0.2; };

Save & Exit. & restart BIND

service bind9 restart

now look at syslog file to check for any errors

tail -f /var/log/syslog

Sep 26 06:48:06 ns2 named[8496]: client 10.0.0.1#55064: received notify for zone 'example.com'
Sep 26 06:48:06 ns2 named[8496]: zone example.com/IN: notify from 10.0.0.1#55064: zone is up to date
Sep 26 06:48:07 ns2 named[8496]: client 10.0.0.1#44867: received notify for zone '0.0.0.10.in-addr.arpa'
Sep 26 06:48:07 ns2 named[8496]: zone 0.0.0.10.in-addr.arpa/IN: notify from 10.0.0.1#44867: zone is up to date


BIND Service Related CMD’s

service bind9 stop
service bind9 start
service bind9 restart
service bind9 status

NSLOOKUP Test: (Windows)

nslookup -query=any example.com
nslookup -query=ns example.com
nslookup -query=soa example.com
nslookup -query=mx example.com
nslookup -10.0.0.1 example.com
nslookup -10.0.0.1 yahoo.com

DIG Test: (Linux)

Perform nslookup/dig on dns server

dig @127.0.0.1 ns1.example.com
dig @127.0.0.1 ns2.example.com
dig -x 10.0.0.1
dig example.com @10.0.0.1 | grep -e "^host" -e ";; flags"
dig +norec example.com @10.0.0.1 | grep -e "^host" -e ";; flags"
dig +norec example.com @10.0.0.1 | sed -n -e '/;; flags/p' -e '/^;; AUTH/,/^$/p'
dig +noall +answer SOA +multi example.com

TIPS:


  • SERIAL of SOA

The serial number of the SOA record is very important to the correct operation of slave servers. Each time you edit your zone, the serial number must be increased so that the slave server can detect that the zone has been updated. Whenever the slave server notices that the serial number has changed, it performs a zone transfer and updates its cached zone file.

Every time, you make any changes in MASTER NS zone file, Ensure following

  • Change Serial Number , else it will not be replicated to Slave.
  • Reload bind by
service bind9 reload

  • Auto Generation of PTR Records pointing to fixed generated IP-Hostname

If we want to reply the reverse PTR records automatically for our unused IP’s, Edit the REVERSE zone & add following line in the end

$GENERATE 1-255 $ IN PTR 10-11-11-$.example.com.

Save & Exit, & Reload BIND9 service by

service bind9 reload

now if you do nslookup for any unused ip/record from your user pool, you will get results as below

#10.0.0.1 is our NS server, and 10.0.0.50,100 are host whose record is not manually added,

C:\>nslookup 10.0.0.50 10.0.0.1
Server: ns1.example.com
Address: 10.0.0.1

Name: 101-11-11-50.example.com
Address: 10.0.0.50

C:\>nslookup 10.0.0.100 10.0.0.1
Server: ns1.example.com
Address: 10.0.0.1

Name: 101-11-11-100.example.com
Address: 10.0.0.100

  • BASH Script to change ZONE SERIAL number with YYMMDDnn format 

If you have Master/Slave Setup, then you need to update the serial number in zones every time you make any modifications, else it will not replicate to SLAVE. A bash script is handy to run when you update any record, it will update serial to YYMMDDSS (SS=sequence Serial) number, and will increment the SS every time u run it. so total of 99 changes u can make in single date.

Old Serial Number can be any number but the script will change it to current date like YYYYMMDDSS, example shown afterwards

  • /temp/reload.sh

 

  • NOTE: This bash script looks in /etc/bind/zones, and will search text ‘Serial’, So make sure your zone file contains ; Serial word in zone file. Example below
$TTL 86400 ; (1 day)
; $ORIGIN example.com
@ IN SOA ns1.example.com. admin (
2022092623 ; Serial
14400 ; refresh (4 hours)
1800 ; retry (30 minutes)
1209600 ; expire (2 weeks)
3600 ; minimum (1 hour)
)

Bash Script named /temp/reload.sh

#!/bin/bash
# This path contains all ZONES files, this is folder created specifically to hold ZONES files
ZONES_PATH="/etc/bind/zones"
DATE=$(date +%Y%m%d)
# we're searching for line containing this comment
NEEDLE="Serial"
for ZONE in $(ls -1 $ZONES_PATH) ; do
curr=$(/bin/grep -e "${NEEDLE}$" $ZONES_PATH/${ZONE} | /bin/sed -n "s/^\s*\([0-9]*\)\s*;\s*${NEEDLE}\s*/\1/p")
# replace if current date is shorter (possibly using different format)
if [ ${#curr} -lt ${#DATE} ]; then
serial="${DATE}00"
else
prefix=${curr::-2}
if [ "$DATE" -eq "$prefix" ]; then # same day
num=${curr: -2} # last two digits from serial number
num=$((10#$num + 1)) # force decimal representation, increment
serial="${DATE}$(printf '%02d' $num )" # format for 2 digits
else
serial="${DATE}00" # just update date
fi
fi
/bin/sed -i -e "s/^\(\s*\)[0-9]\{0,\}\(\s*;\s*${NEEDLE}\)$/\1${serial}\2/" ${ZONES_PATH}/${ZONE}
echo "${ZONE}:"
grep "; ${NEEDLE}$" $ZONES_PATH/${ZONE}
done
# In the END, reload and show status of BIND service
service bind9 reload
service bind9 status | grep running
service bind9 status

Result: After modifying any record, run this script. every time you run the script, it will just increment one number in last 2 digits, (it can be max of 99 till the SOA expires, or single day)

root@ns1:/etc/bind/zones# /temp/test.sh
forwarder.example.com:
2022092604 ; Serial
reverse.example.com:
2022092604 ; Serial

root@ns1:/etc/bind/zones# /temp/test.sh
forwarder.example.com:
2022092605 ; Serial
reverse.example.com:
2022092605 ; Serial

I have tagged this file with webmin, so that when user edit the record file and hit save and close, this file runs and do the job, and all data gets replicate to the SLAVE automatically. Also you schedule it to hourly or as per requirements.


  • SLAVE ZONE FILES RAW FORMAT

With BIND 9.9.x, the slave zone files are now saved in a default raw binary format. This was done to improve performance, but at the sacrifice of being able to easily view the contents of the files. it can  also make debugging at SLAVE level more difficult.  If you want to see the slave zone files in plain text, just simple add following in the the named.conf files for your slave zones to include the line: (This has to be done at MASTER/NS1 dns)

masterfile-format text;
  • REDUCE LOGGING FOR BIND9

BIND uses syslogd before a valid logging clause is available so named.conf parse errors and other information will appear in /var/log/syslog. To separate log from SYSLOG & record them in separate file & fix the log size, you can add following in /etc/bind/named.conf.option at end,

logging{
channel bind9_zaib_log {
file "/var/lib/bind/bind.log" versions 3 size 5m;
severity info;
print-time yes;
print-severity yes;
print-category yes;
};
category default{
bind9_zaib_log;
};
};

now reload BIND9 service

service bind9 reload

check the file contents

tail -f /var/lib/bind/bind.log

Once the file size will cross 5 MB, it will reset to 0 auto and will continue to grow till 5 & reset again 🙂 as shown in below image …

root@ns1:/etc/bind# ls -lh /var/lib/bind/bind.log
-rw-r--r-- 1 bind bind 4.2M Sep 27 11:29 /var/lib/bind/bind.log
root@ns1:/etc/bind# ls -lh /var/lib/bind/bind.log
-rw-r--r-- 1 bind bind 4.2M Sep 27 11:29 /var/lib/bind/bind.log
root@ns1:/etc/bind# ls -lh /var/lib/bind/bind.log
-rw-r--r-- 1 bind bind 4.2M Sep 27 11:30 /var/lib/bind/bind.log
root@ns1:/etc/bind# ls -lh /var/lib/bind/bind.log
-rw-r--r-- 1 bind bind 4.3M Sep 27 11:30 /var/lib/bind/bind.log
root@ns1:/etc/bind# ls -lh /var/lib/bind/bind.log
-rw-r--r-- 1 bind bind 4.3M Sep 27 11:30 /var/lib/bind/bind.log
root@ns1:/etc/bind# ls -lh /var/lib/bind/bind.log
-rw-r--r-- 1 bind bind 13K Sep 27 11:31 /var/lib/bind/bind.log

For more customized info, Please read below …

Configuring Bind9 logs


Different Types of DNS Records With Syntax and Examples

Types of DNS Records

A
AAAA
CNAME
MX
PTR
NS
SOA
SRV
TXT
NAPTR

The above DNS records are mostly used in all DNS Configurations.

A Record Example

Address Record, assigns an IP address to a host, domain or subdomain name

ns1 IN A 10.0.0.1

PTR Record Example

A PTR record or pointer record maps an IPv4 address to the canonical name for that host. This is mostly used as a security and an anti-spam measure wherein most of the webservers or the email servers do a reverse DNS lookup to check if the host is actually coming from where it claims to come from. It is always advisable to have a proper reverse DNS record (PTR) is been setup for your servers especially when you are running a mail / smtp server.

255 IN PTR ns1.example.com.

SPF Record Example

The Sender Policy Framework (SPF) is an email-authentication technique which is used to prevent spammers from sending messages on behalf of your domain. With SPF an organization can publish authorized mail servers. Together with the DMARC related information, this gives the receiver (or receiving systems) information on how trustworthy the origin of an email is. SPF is, just like DMARC, an email authentication technique that uses DNS (Domain Name Service). This gives you, as an email sender, the ability to specify which email servers are permitted to send email on behalf of your domain.

v=spf1 +a +mx +ip4:1.2.3.4 +include:mail.example.com -all

DMARC Record Example

_dmarc.example.com. 14400 TXT "v=DMARC1; p=quarantine; sp=none; rf=afrf; pct=100; ri=86400"

DKIM (MultiLine) Record Example

default._domainkey.example.com. 14400 TXT ( "v=DKIM1; k=rsa; p=GIIBgjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAzEi2OVswNjDwG57Rc14vKSAZNizQpO/KRG96H2N3dc1jnEMi0GCXCIFlFnrQffEVA9RWQ4u6pFjhaQ4s/Uony61CmPwls/O5p/IMdKbmkb0ULrdtwkpoW5Ve0F4C777YXBlXk0aTP2sEenX5e4ou8IGog0zTyq4E5v6DY+juNCKE8yktvM2oJvso/mqS2BJc5"
"X+Xvrhs+l3/qqlLEIwIfA4ep2QBXEOIgElBiXptXSwt6ym6ZmBdOl/eeZipulBZyC1onGFLwR5qvRNo/Q0e1c/H9eBrOyOmIJ65OPy8AtT1Ln3emKy9JAdAVaODHTt1jRbK2X8j3t/cAWX7Fntr1QIDAQAB;" )

August 10, 2022

CallMeBot: Sending Alerts to Various Messaging Apps using APi

Filed under: Linux Related, Mikrotik Related — Tags: , , , , , — Syed Jahanzaib / Pinochio~:) @ 11:00 AM


At multiple ISP’s or networks, I use either GSM Modems or 3rd party SMS API to send various information / alerts messages to admin or users , & it involves minor cost on a per sms basis or as bundle. Another method to send messages on WHATSAPP & its FREE , very simple to use CALLMEBOT APi’s. Its a online FREE service via which you can send basic messages to multiple messaging services like WhatsApp / messenger / Signal / Telegram apps.  The only caveat is that you cannot send messages to EVERY contact. The receiver contact must have the callmebot API KEY , which he can easily get via adding callmebot number in there contact list & send particular message & he will get it instantly. so EACH API EKY is tagged with that particular mobile number only. Once you know the key & its tagged mobile number, you can send message to that contact using the API_KEY & his mobile number using various methods.

In this guide we will be using callmebot API to send various test message to admin WhatsAPP number. Follow the below steps …

  • First, You need to get the API key form the CallMeBot. Add the phone number +34 644 91 96 80 into your Phone Contacts. (Name it it as you wish like CallMeBoT / My_Alerts etc )
  • Using WhatsApp, Send this message to this contact

I allow callmebot to send me messages

  • once it is sent, Wait until you receive the message like “API Activated for your phone number. Your APIKEY is XXXXXX” from the bot. Note down the APIKEY
    Note: If you don’t receive the ApiKey in 2 minutes, please try again after 24hs.
  • The WhatsApp message from the CallMeBot will contain the API key needed to send messages using various API.
  • Now You can send text messages using the API after receiving the confirmation.

Following are few methods, which I am using to send various informational/warning/critical messages to Admin whatsapp


Send message using Browser HTTP APi:

Using browser, you can send message by using below

(As always, change the Phone/APiKey)

https://api.callmebot.com/whatsapp.php?phone=+923333021909&apikey=123123&text=Fiber+Link+is+Down

Example Message:


Linux Examples:

CMD: on Terminal/shell

(As always, change the Phone/APiKey)

curl --insecure 'https://api.callmebot.com/whatsapp.php?phone=+923333021909&apikey=123123&text=Linux+Msg+Test+z'

Example Message:


Linux Bash Script Example using separate Text file as attachment

(As always, change the Phone/APiKey)

#!/bin/bash
# DAILY SMS SCRIPT FOR whatsapp MSG Testing
PATH=/opt/someApp/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
# Script by Syed Jahanzaib / aacable@hotmail.com
# https://aacable.wordpress.com
# Version 2.0 -
# Created in year 2013
# Last modified 20-NOV-2017
set -x
logger DAILY 8 am sms executed for whatsapp testing
#ntpdate -u 91.189.91.157
COMPANY="Independence-Day"
TMP="/tmp/dailysms.txt"
> $TMP
# CHAGNE BELOW API KEY AS PER YOUR'S
API_KEY=123123
# Jahanzaib Cell # CHANGE CELL AS PER YOUR's
CELL1="923333021909"
UPTIME=`uptime | awk -F'( |,|:)+' '{if ($7=="min") m=$6; else {if ($7~/^day/) {d=$6;h=$8;m=$9} else {h=$6;m=$7}}} {print d+0,"days,",h+0,"hours,",m+0,"minutes"}'`
DATE=`date`
FOOTER="Powered By Jz"
# Print total and each vlan users
MSG="$COMPANY - Msg Test !
Daily Msg Test @
UPTIME = $UPTIME
$DATE
$FOOTER"
echo "$MSG" > $TMP
cat $TMP

# Finally SEND MSG to whatsapp using CALLMEBOT
curl --insecure "https://api.callmebot.com/whatsapp.php?phone=+$CELL1&apikey=$API_KEY" -G --data-urlencode text@$TMP

rm $TMP
# THE END

Example Message:


Mikrotik RouterOS Example:

Terminal:

(As always, change the Phone/APiKey)

/tool fetch http-method=get mode=https url="https://api.callmebot.com/whatsapp.php\?&apikey=123123&phone=+923333021909&text=Mikrotik+Router+Whatsapp+Msg+Example+SJZ"

Example Message:


Mikrotik DUDE (for Windows) Example:

First download the windows base WGET utility, copy it in any local folder of dude server. & now add notification like this

On DUDE App, Goto Notifications, Click on + ICON to add new notification, a new popup will appear, fill it as per below

Name: Whatsapp2Zaib
Enabled: Ticked
Type: Execute on Server
& in Command Window section, use below cmd

(As always, change the Phone/APiKey)

C:\wget\wget.exe --no-check-certificate "https://api.callmebot.com/whatsapp.php?phone=+923333021909&apikey=123123&text=Service [Probe.Name] on [Device.Name] is now [Service.Status] ([Service.ProblemDescription])"

Example Message:


Hope it Helps !



Regard’s
Syed Jahanzaib

July 26, 2022

Veeam B&R: Virtual disk size is not a multiple of 1KB

Filed under: Veeam B&R — Tags: , — Syed Jahanzaib / Pinochio~:) @ 11:45 AM

Veeam 1KB Disk Error Solved

We are using Veeam B&R ver 9.x application to backup some esxi guests. From past few days we were getting error at one guest (Windows 2008 R2, converted from P2V (using vmware converter) & it’s backup worked well with veeam for long without issue, but after increasing its disk space, it was giving trouble with following errors.

********************************************************************************************
7/18/2022 5:11:33 PM :: Error: Virtual disk size is not a multiple of 1KB.
********************************************************************************************

I tried the veeam knowledge base mentioned here, changed disk size and recalculated, but still getting errors.

This VM have 2 drives assigned, 1st contains OS (C: drive), 2nd drive (G: Drive) contains Oracle DB DATA. To further narrow down the troubleshooting, I included only drive 1 (OS) in the Veeam backup job, and it worked fine. So it was confirmed that the culprit was drive 2 & it was important drive because it contains oracle database folders (ORACLE & USR).

Workaround:

I tried all solutions but couldn’t figured out. As a last resort I did following. I know this workaround is not the proper solution, but since I failed to sort it using my limited knowledge & get the VM backup was crucial, therefore I did following & it worked for me. Luckily I had enough resources to cater the workaround. 6 drivers SSD in Raid-5 made the transition really fast at about 6-7 Gbps.

STEPS:

  • I Stopped all running services on the VM guest related to Oracle/SAP or any un-necessary apps
  • Added new disk (drive3 / H: ) on the effected VM with same size as existing DB drive (disk2 G: )
  • Cloned the partition from old (disk2 G: ) to this new (disk3 H: ) [I used 3rd party DiskGeniusPro tool for cloning by locking partition access, but u can use Robocopy or other tools as well)
  • Using Disk Management, I changed the drive letter & marked the old (disk2 G: ) OFFLINE.
  • Changed the new disk (disk3 H: ) partition drive letter same as older one that is  G:
  • Rebooted the server, & this time all backups went smoothly & SAP ORACLE synching/services are working fine as well
  • Afterwards I removed the OLD faulty partition from the VM.

Alhamdolillah!


Another workaround I got from a virtualization expert member at spicework community support named  Supaplex was to

A slightly simpler option would be using a P2V approach inside the virtual machine using a free tool like V2V Converter https://www.starwindsoftware.com/starwind-v2v-converter and specifying your faulty G: disk as source and ESXi as the target. The tool would convert the disk into a fresh new VMDK file automatically, and all you would have to do is just attach it to the virtual machine and remove the old one. Hope that may save you some time in the future if you encounter such a problem again.

Thank you


Regard’s
Syed Jahanzaib

June 18, 2022

VMWARE – VCSA 7.0 Migration/Cloning

Filed under: VMware Related — Tags: , — Syed Jahanzaib / Pinochio~:) @ 10:30 PM

Moving VCSA 7.0.0 (Build 16189207) from one ESXI host to other ESXI host
(without VMOTION)

 

Recently we added one new esxi server (server02) with some beefy hardware specs including enterprise ssd storage & wanted to move old vcenter hosted to this new esxi host to get benefits of good speed.

Since we donot have shared storage, therefore I used the CLONE method which worked surprisingly accurate without any hurdle.

  • Login to your Vcsa admin panel (https://your-vcsa-ip)
  • Select the vCenter Server virtual machine from the Inventory (source esxi server01 where VCSA is currently installed)
  • Right-click the VCSA virtual machine and click Clone
  • Select the Destination ESXi host server02 (in my case it was new ssd base esxi server), & follow on the instructions

Once the Clone process is done

  • Power off vCenter Server on the source host (server01 old server)
  • Power on the vCenter Server virtual machine on the destination ESXi host (server02 – new esxi server)

VCSA takes some time to start all of its services, Be patience ! (Follow this guide for VCSA Troubleshooting)

Once done, open the VCSA admin panel, & Login.

Alhamdolillah, in my case it was quite straight forward process! your mileage may vary 😀


Backup Note:

Backup is your friend 🙂 Make sure to take regular backups of VCSA using its management panel that is https://your-vcsa-ip:5480/

We take its backup vis windows shared folder using SMB option. you can  use SMB/FTP or other methods of your choice.

When you will click on BACKUP NOW, it will ask you to enter all the details like smb/ftp etc location etc, you need to configure it first !


Regards
Syed Jahanzaib

May 18, 2022

Lenovo SR650 Corrupt GPT & ESXi install failure

Filed under: IBM Related — Tags: , , , , , , , — Syed Jahanzaib / Pinochio~:) @ 12:40 PM

Recently one of our Lenovo SR650’s disk got faulty. It had 14 x 1.2 TB 10k SAS disks. As a long term solution & to avoid any urgency, we decided to remove 2 disks (faulty one for replacement & one for cold spare backup to be used by same or other similar servers).

Once we re-created new Raid.-10 & rebooted the server , boot screen was showing below error

We tried to follow the Lenovo Note which instructed to go into Setup->System Settings->Recovery->Disk GPT Recovery and set to “Automatic.” but still the error didn’t sorted. To settle it on Server Bios level, we performed following steps

  • Update SR650 UEFI Firmware ( Lenovo Download Link )
  • Removed Raid Config, Re-create Raid Config with Full Initialization
  • Full power cycle the server once above is done.

This sorted the Bios screen error regarding GPT.

But once we started the Vmware ESXI 6.5.x installation , it was failing (between 5% and 8%) with the following error …

“partedUtil failed with message: Error: The primary GPT table states that the backup GPT is located beyond the end of disk. This may happen if the disk has shrunk or partition table is corrupted. … Error: Can’t have a partition outside the disk!  BLAH BLAH BLAH …”


Solution # 1

Boot with any windows ISO ( Must have the RAID controller driver or the OS should have in-built drivers, in my case, windows server 2019 had the raid drivers). You can also use Linux base Boot OS .

Re-create the partition , Format & Booom. Afterwards just boot from ESXI ISO/CD/USB/Networkboot , and the ESXI will install fine.


Solution # 2 (Quick & Recommended for admins)

During ESXi installer at anywhere ,

Press Alt-F1 (which will bring you to shell window asking for credentials)

Use following credentials

  • ID: root
  • Password: No password. Just press enter & you can use the CMD’s to sort the issue

Issue the below CMD which will show you list of disk device names that can be managed by partedUtil

ls -ltrh /vmfs/devices/disks

** Note the disk ‘identifier’ that we want to fix. In my case it was 6.5 TB partition in which we wanted to install the esxi.

Now issue the below cmd

partedUtil mklabel /dev/disks/naa.600062b2031e00402a165add7ff9c3ac msdos

This overwrited the brooked partition table. Now return to the installer screen and continue.

This time, esxi installation went fine without errors.


Regard’s
Syed Jahanzaib

February 14, 2022

RM: Auto Renew User if Deposit available

Filed under: Radius Manager — Tags: , , — Syed Jahanzaib / Pinochio~:) @ 11:04 AM

Revision History:

  1. 24-Jun-2015 / Added Base Script
  2. 29-Jun-2016 / Added Invoice function
  3. 14-Feb-2022 / Added multi check for quota related functions


As requested by an Valenzuela / African friends

Scenario:

In radius manager, there are few options to purchase credits via online payment gateways like paypal or others. If the user account is expired and he purchase service online, it adds the deposit into user account but it does not auto renew the service (as per my assumption, as paypal doesn’t works here in pakistan, so i have very little to no knowledge on it).

Example:

err

 

As a workaround, I made below script that can perform following functions ,,,

  1. Scheduled to run after every 5 (or x) minute(s)
  2. Fetch all users who are expired either by Date or Quota
  3. Check if these users have DEPOSIT available (credits above then 0)
  4. Check the current service price and match it with the available deposit/credits
  5. If deposit is not sufficient as per service price, Then print error
  6. If deposit is enough, renew the service , Add Expiration Days (according to the service package)
  7. Reset the QUOTA …
    * if account is fresh, add new quote as per package
    * if account is old and quota is expired, then reset it add new quota as per package
    * if account date is expired but quota is remaining, then add the new quota in existing quota (addictive mode
    * Show user Before / After Quota
    – If user is online show his IP / NAS IP / Time / NAS-IP]if any and sends email/sms to user about the renewal done by deposit : )

We can further add the SMS/EMAIL function in the script as per requirements to let know about the renewal.

Disclaimer: The script can further be customized according to the requirements. No part of this script is copied from anywhere. You are free to use it, modify it as you like [keep the header intact].
This is my own idea Just to share with anyone who is in similar requirements or just for learning purposes !


SCRIPT!

mkdir /temp
touch /temp/auto.sh
chmod +x /temp/auto.sh
nano /temp/auto.sh

& paste following, make sure to change the SQL Password !

#!/bin/bash
#set -x
clear
# Script to renew user account via check deposit and act accordingly
# For Radius Manager 4.1.x
# Created by Syed Jahanzaib
# https://aacable.wordpress.com / aacable@hotmail.com
# 24th Jun, 2016 , 18 Ramazan, 1437 Hijri
# Last modified on 14-Feb-2022
# Colors Config . . . [[ JZ . . . ]]
ESC_SEQ="\x1b["
COL_RESET=$ESC_SEQ"39;49;00m"
COL_RED=$ESC_SEQ"31;01m"
COL_YELLOW=$ESC_SEQ"33;01m"
COL_GREEN=$ESC_SEQ"32;01m"
SQLUSER="root"
SQLPASS="XXXXXXX"
export MYSQL_PWD=$SQLPASS
DB=radius
CMD="mysql -u$SQLUSER --skip-column-names -s -e"
bytesToHuman() {
b=${1:-0}; d=''; s=0; S=(Bytes {K,M,G,T,P,E,Z,Y}iB)
while ((b > 1024)); do
d="$(printf ".%02d" $((b % 1024 * 100 / 1024)))"
b=$((b / 1024))
let s++
done
echo "$b$d ${S[$s]}"
}
USERLIST="/tmp/deposituserlist.txt"
> $USERLIST
DATE=`date`
#Create list of users which have deposit more then 0.00 value, means valid deposite
$CMD "use $DB; SELECT SQL_CALC_FOUND_ROWS username, firstname, lastname, address, city, zip, country, state, phone, mobile,
email, company, taxid, srvid, downlimit, uplimit, comblimit, expiration, uptimelimit, credits, comment,
enableuser, staticipcpe, staticipcm, ipmodecpe, ipmodecm, srvname, limitdl, limitul, limitcomb, limitexpiration,
limituptime, createdon, verifycode, verified, selfreg, acctype, maccm, LEFT(lastlogoff, 10)
, IF (limitdl = 1, downlimit - COALESCE((SELECT SUM(acctoutputoctets) FROM radacct
WHERE radacct.username = tmp.username) -
(SELECT COALESCE(SUM(dlbytes), 0) FROM rm_radacct
WHERE rm_radacct.username = tmp.username), 0), 0),
IF (limitul = 1, uplimit - COALESCE((SELECT SUM(acctinputoctets) FROM radacct
WHERE radacct.username = tmp.username) -
(SELECT COALESCE(SUM(ulbytes), 0) FROM rm_radacct
WHERE rm_radacct.username = tmp.username), 0), 0),
IF (limitcomb =1, comblimit - COALESCE((SELECT SUM(acctinputoctets + acctoutputoctets) FROM radacct
WHERE radacct.username = tmp.username) -
(SELECT COALESCE(SUM(ulbytes + dlbytes), 0) FROM rm_radacct
WHERE rm_radacct.username = tmp.username), 0), 0),
IF (limituptime = 1, uptimelimit - COALESCE((SELECT SUM(acctsessiontime) FROM radacct
WHERE radacct.username = tmp.username) -
(SELECT COALESCE(SUM(acctsessiontime), 0) FROM rm_radacct
WHERE rm_radacct.username = tmp.username), 0), 0)
FROM
(
SELECT username, firstname, lastname, address, city, zip, country, state, phone, mobile, email, company,
taxid, rm_users.srvid, rm_users.downlimit, rm_users.uplimit, rm_users.comblimit, rm_users.expiration,
rm_users.uptimelimit, credits, comment, enableuser, staticipcpe, staticipcm, ipmodecpe, ipmodecm, srvname, limitdl,
limitul, limitcomb, limitexpiration, limituptime, createdon, verifycode, verified, selfreg, acctype, maccm,
mac, groupid, contractid, contractvalid, rm_users.owner, srvtype, lastlogoff
FROM rm_users
JOIN rm_services USING (srvid)
ORDER BY username ASC
) AS tmp
WHERE 1
AND (tmp.acctype = '0' OR tmp.acctype = '1' OR tmp.acctype = '2' OR tmp.acctype = '3' OR tmp.acctype = '4' OR tmp.acctype = '5' )
AND tmp.enableuser = 1 AND
(IF (limitdl = 1, downlimit - (SELECT COALESCE(SUM(acctoutputoctets), 0)
FROM radacct WHERE radacct.username = tmp.username) - (SELECT COALESCE(SUM(dlbytes), 0)
FROM rm_radacct WHERE rm_radacct.username = tmp.username) , 1) <= 0
OR
IF (limitul = 1, uplimit - (SELECT COALESCE(SUM(acctinputoctets), 0)
FROM radacct WHERE radacct.username = tmp.username) - (SELECT COALESCE(SUM(ulbytes), 0)
FROM rm_radacct WHERE rm_radacct.username = tmp.username) , 1) <= 0
OR
IF (limitcomb = 1, comblimit -
(SELECT COALESCE(SUM(acctinputoctets + acctoutputoctets), 0)
FROM radacct WHERE radacct.username = tmp.username) +
(SELECT COALESCE(SUM(ulbytes + dlbytes), 0)
FROM rm_radacct WHERE rm_radacct.username = tmp.username) , 1) <= 0
OR
IF (limituptime = 1, uptimelimit - (SELECT COALESCE(SUM(acctsessiontime), 0)
FROM radacct WHERE radacct.username = tmp.username) + (SELECT COALESCE(SUM(acctsessiontime), 0)
FROM rm_radacct WHERE rm_radacct.username = tmp.username) , 1) <= 0
OR
IF (limitexpiration=1, UNIX_TIMESTAMP(expiration) - UNIX_TIMESTAMP(NOW()), 1) <= 0) LIMIT 0, 50;" | awk '{print $1}' > $USERLIST
TOTUSR=`$CMD "use $DB; select username from rm_users;" | wc -l`
echo "- INFO: Total Users scanned: $TOTUSR

"
#LOOK FOR VALID USER IN FILE, IF EMPTY THEN EXIT
USRVALID=`cat $USERLIST`
if [ -z "$USRVALID" ]; then
echo "INFO: No user found with Expired Date/Quota package ... Exiting peacefully!"
exit 1
fi
# Apply Formula to read the file in which users list and act accordingly.
num=0
cat $USERLIST | while read users
do
num=$[$num+1]
USR=`echo $users | awk '{print $1}'`
DEPOSIT=`$CMD "use radius; SELECT credits FROM rm_users where username = '$USR';" | sed 's/\..*$//'`
###########################################
# ACCOUNT EXPIRY CHECK and other variables#
###########################################
TODAY=$(date +"%Y-%m-%d")
TODAYHM=$(date +"%Y-%m-%d-%H-%M")
TODAYDIGIT=`echo $TODAY | sed -e 's/-//g'`
MONTH=$(date +"-%m")
CMONTH=`echo $MONTH | sed -e 's/-//g'`
MONTHYEAR=$(date +"%B-%Y")
ALPHAMONTHYEAR=`echo $MONTHYEAR #| sed -e 's/-//g'`
CURR_MONTHYEAR=$(date +"%Y-%m")
SRVEXPIRYFULL=`mysql -u$SQLUSER -p$SQLPASS -e "use radius; SELECT expiration FROM radius.rm_users WHERE username = '$USR';" |awk 'FNR == 2'`
FULLNAME=`mysql -u$SQLUSER -p$SQLPASS -e "use radius; SELECT firstname, lastname FROM radius.rm_users WHERE username = '$USR';" |awk 'FNR == 2'`
MOBILE=`mysql -u$SQLUSER -p$SQLPASS -e "use radius; SELECT mobile FROM radius.rm_users WHERE username = '$USR';" |awk 'FNR == 2'`
COUNTRY=`mysql -u$SQLUSER -p$SQLPASS -e "use radius; SELECT country FROM radius.rm_users WHERE username = '$USR';" |awk 'FNR == 2'`
STATE=`mysql -u$SQLUSER -p$SQLPASS -e "use radius; SELECT state FROM radius.rm_users WHERE username = '$USR';" |awk 'FNR == 2'`
ADDRESS=`mysql -u$SQLUSER -p$SQLPASS -e "use radius; SELECT address FROM radius.rm_users WHERE username = '$USR';" |awk 'FNR == 2'`
LOGOFFDATE=`mysql -u$SQLUSER -p$SQLPASS -e "use radius; SELECT lastlogoff FROM radius.rm_users WHERE username = '$USR';" |awk 'FNR == 2 {print $1,$2}'`
SRVID=`mysql -u$SQLUSER -p$SQLPASS -e "use radius; SELECT srvid FROM radius.rm_users WHERE rm_users.username = '$USR';" |awk 'FNR == 2 {print $1}'`
SRVPRICE=`mysql -u$SQLUSER -p$SQLPASS -e "use radius; SELECT unitprice FROM radius.rm_services WHERE rm_services.srvid = $SRVID;" |awk 'FNR == 2 {print $1}' | cut -f1 -d"."`
SRVEXPIRYFULLD=`mysql -u$SQLUSER -p$SQLPASS --skip-column-names -e "use radius; SELECT expiration FROM radius.rm_users WHERE username = '$USR';" |awk '{print $1}' | sed 's/expiration//'`
SRVEXPIRY=`mysql -u$SQLUSER -p$SQLPASS -e "use radius; SELECT expiration FROM radius.rm_users WHERE username = '$USR';" |awk 'FNR == 2' | sed -e 's/-//g' | sed 's/00:.*//'`
PKGNAME=`mysql -u$SQLUSER -p$SQLPASS -e "use radius; SELECT srvname FROM radius.rm_services WHERE rm_services.srvid = '$SRVID';" |awk 'FNR == 2'`
PKGQUOTA=`mysql -u$SQLUSER -p$SQLPASS -e "use radius; SELECT trafficunitcomb FROM rm_services WHERE srvid= '$SRVID';" |awk 'FNR == 2'`
PKGQUOTAB=$(($PKGQUOTA / 1024))
PKGQUOTABYTES=$(($PKGQUOTA * 1024 * 1024))
USR_CUR_COMBLIMIT=`$CMD "use $DB; SELECT comblimit FROM rm_users WHERE username= '$USR';"`
USR_CUR_COMBLIMIT_HUMAN=`bytesToHuman $USR_CUR_COMBLIMIT`
TOT_DOWN_UP_CURR_MONTH_IN_BYTES=`$CMD "use $DB; SELECT ((SUM(AcctInputOctets)+SUM(AcctOutputOctets))) FROM radacct WHERE username ='$USR' AND acctstarttime LIKE '$CURR_MONTHYEAR-%' LIMIT 0 , 30;"`
TOT_USER_DOWNLOAD_SINCE_LAST_REFRESH_IN_BYTES=`$CMD "use $DB; SELECT downlimit from rm_users where username = '$USR';" |sed 's/[\._-]//g'`
TOT_USER_UPLOAD_SINCE_LAST_REFRESH_IN_BYTES=`$CMD "use $DB; SELECT uplimit from rm_users where username = '$USR';" |sed 's/[\._-]//g'`
RM_CMD_FOR_USER_ACTUAL_LIVE_QUOTA_VALUE=`$CMD "use $DB; SELECT SQL_CALC_FOUND_ROWS IF (limitdl = 1, downlimit - COALESCE((SELECT SUM(acctoutputoctets) FROM radacct WHERE radacct.username = tmp.username) - (SELECT COALESCE(SUM(dlbytes), 0) FROM rm_radacct WHERE rm_radacct.username = tmp.username), 0), 0), IF (limitul = 1, uplimit - COALESCE((SELECT SUM(acctinputoctets) FROM radacct WHERE radacct.username = tmp.username) - (SELECT COALESCE(SUM(ulbytes), 0) FROM rm_radacct WHERE rm_radacct.username = tmp.username), 0), 0), IF (limitcomb =1, comblimit - COALESCE((SELECT SUM(acctinputoctets + acctoutputoctets) FROM radacct WHERE radacct.username = tmp.username) - (SELECT COALESCE(SUM(ulbytes + dlbytes), 0) FROM rm_radacct WHERE rm_radacct.username = tmp.username), 0), 0), IF (limituptime = 1, uptimelimit - COALESCE((SELECT SUM(acctsessiontime) FROM radacct WHERE radacct.username = tmp.username) - (SELECT COALESCE(SUM(acctsessiontime), 0) FROM rm_radacct WHERE rm_radacct.username = tmp.username), 0), 0) FROM ( SELECT username, firstname, lastname, address, city, zip, country, state, phone, mobile, email, company, taxid, rm_users.srvid, rm_users.downlimit, rm_users.uplimit, rm_users.comblimit, rm_users.expiration, rm_users.uptimelimit, credits, comment, enableuser, staticipcpe, staticipcm, ipmodecpe, ipmodecm, srvname, limitdl, limitul, limitcomb, limitexpiration, limituptime, createdon, verifycode, verified, selfreg, acctype, maccm, mac, groupid, contractid, contractvalid, rm_users.owner, srvtype, lastlogoff FROM rm_users JOIN rm_services USING (srvid) ORDER BY username ASC ) AS tmp WHERE 1 AND username LIKE '$USR%' AND (tmp.acctype = '0' OR tmp.acctype = '1' OR tmp.acctype = '2' OR tmp.acctype = '3' OR tmp.acctype = '4' OR tmp.acctype = '5' ) LIMIT 0, 50;" | awk '{print $3}'`
RM_CMD_FOR_USER_ACTUAL_LIVE_QUOTA_VALUE_IN_HUMAN_FRIENDLY_FORMAT=`bytesToHuman $RM_CMD_FOR_USER_ACTUAL_LIVE_QUOTA_VALUE`
IS_QUOTA_LEFT_IN_NEGATIVE=`echo $RM_CMD_FOR_USER_ACTUAL_LIVE_QUOTA_VALUE_IN_HUMAN_FRIENDLY_FORMAT | grep -i -c "-"`
RM_CMD_FOR_USER_ACTUAL_LIVE_QUOTA_NEGATIVE_VALUE_IN_BYTES_WITHOUT_DASH=`echo $RM_CMD_FOR_USER_ACTUAL_LIVE_QUOTA_VALUE_IN_HUMAN_FRIENDLY_FORMAT |sed 's/[\._-]//g'`
RM_CMD_FOR_USER_ACTUAL_LIVE_QUOTA_NEGATIVE_VALUE_IN_HUMAN_FRIENDLY_VALUE=`bytesToHuman $RM_CMD_FOR_USER_ACTUAL_LIVE_QUOTA_NEGATIVE_VALUE_IN_BYTES_WITHOUT_DASH`
RQT=`$CMD "use $DB; SELECT username,
IF (limitcomb =1, comblimit - COALESCE((SELECT SUM(acctinputoctets + acctoutputoctets) FROM radacct
WHERE radacct.username = tmp.username) -
(SELECT COALESCE(SUM(ulbytes + dlbytes), 0) FROM rm_radacct
WHERE rm_radacct.username = tmp.username), 0), 0)
FROM
(
SELECT username, firstname, lastname, address, city, zip, country, state, phone, mobile, email, company,
taxid, rm_users.srvid, rm_users.downlimit, rm_users.uplimit, rm_users.comblimit, rm_users.expiration,
rm_users.uptimelimit, credits, comment, enableuser, staticipcpe, staticipcm, ipmodecpe, ipmodecm, srvname, limitdl,
limitul, limitcomb, limitexpiration, limituptime, createdon, verifycode, verified, selfreg, acctype, maccm,
mac, groupid, contractid, contractvalid, rm_users.owner, srvtype
FROM rm_users JOIN rm_services USING (srvid) ORDER BY username ASC
) AS tmp
WHERE 1 AND (tmp.acctype = '0' OR tmp.acctype = '1' OR tmp.acctype = '3' OR tmp.acctype = '4')
AND username LIKE '$USR%';" | awk '{print $2}'`
RQT_OUT=`echo $RQT`
RQT_RESULT1=`echo $RQT | grep -i -c "-"`
RQT_RESULT_REMOVE_HYPHEN=`echo $RQT | sed -e 's/-//g'`
#exit 1
RQT4=`bytesToHuman $RQT_RESULT_REMOVE_HYPHEN`
QTL_Y_OR_NO=`$CMD "use $DB; SELECT limitcomb FROM rm_services WHERE srvid = '$SRVID';"`
if [ "$DEPOSIT" -eq 0 ]; then
LASTUSRBAL=0
else
LASTUSRBAL=$(($DEPOSIT - $SRVPRICE))
fi
TIMEUNITEXP=`$CMD "use $DB; SELECT timeunitexp FROM radius.rm_services WHERE srvid = '$SRVID';"`
TIMEBASEEXP=`$CMD "use $DB; SELECT timebaseexp FROM radius.rm_services WHERE srvid = '$SRVID';"`
if [ "$TIMEBASEEXP" == "2" ]; then
NEXTEXPIRYADD=$(date +"%Y-%m-%d" -d "+$TIMEUNITEXP days")
EXPERIOD="$TIMEUNITEXP Days"
fi
# Set Quota Limit variable which will be used in the end zzzzzzzzzzzz
if [ "$TIMEBASEEXP" == "3" ]; then
NEXTEXPIRYADD=$(date +"%Y-%m-%d" -d "+$TIMEUNITEXP month")
EXPERIOD="$TIMEUNITEXP Month"
fi
# Set Expiry Date/Month Unit
if [ $PKGQUOTA -eq 0 ]
then
QT="UNLIMITED"
else
QT="$PKGQUOTA MB"
fi
# Check Service Expiry Date, if Active then ignore
IS_USER_EXPIRED=`rmauth 127.0.0.1 $USR 1 |grep -i -c "The account has expired"`
if [ "$IS_USER_EXPIRED" -eq 1 ];then
USREXPORNOT=1
else
USREXPORNOT=0
fi
# Check if user quota is ended or not
IS_USER_QUOTA_END=`rmauth 127.0.0.1 $USR 1 |grep -i -c "Total traffic limit reached"`
if [ "$IS_USER_QUOTA_END" -eq 1 ];then
USRQTORNOT=1
else
USRQTORNOT=0
fi
if [ "$USREXPORNOT" -eq 1 ]; then
DATEEXP="$USR - *** Account Date Expired that is $SRVEXPIRYFULL !! **"
else
DATEEXP="$USR - Account Date is OK that is $SRVEXPIRYFULL."
fi
#IS_QUOTA_LEFT_IN_NEGATIVE zzzzzzzzzzzzzzzzzzz
if [ "$USRQTORNOT" -eq 1 ] && [ "$IS_QUOTA_LEFT_IN_NEGATIVE" -eq 1 ] ; then
RQT_in_NEGATIVE_HUMAN=`bytesToHuman $RQT_RESULT_REMOVE_HYPHEN`
QTEXP="$USR - *** Account QUOTA Expired that is Negative $RQT_in_NEGATIVE_HUMAN !! **"
fi
if [ "$USRQTORNOT" -eq 0 ]; then
RQT_in_POSITIVE_HUMAN=`bytesToHuman $RQT_RESULT_REMOVE_HYPHEN`
QTEXP="$USR - Account QUOTA is OK that is $RQT_in_POSITIVE_HUMAN "
fi
if [ "$DL" == "NULL" ]; then
QTEXP="$USR - NO QUOTA have been added yet! seems fresh account."
fi
if [ "$UL" == "NULL" ]; then
QTEXP="$USR - NO QUOTA have been added yet! seems fresh account."
fi
if [ "$RQT_OUT" == "0" ]; then
QTEXP="$USR - QUOTA INFO: No Quota hase been added yet! seems fresh account."
#echo "$USR - QUOTA INFO: NO quota hase been added yet! seems fresh account."
fi

########### ACCOUNT STATUS EXPIRED BUT NOT ENOUGH DEPOSIT to RENEW ACTION ############
if [ "$DEPOSIT" -eq 0 ]; then
USRDEPORNOT=0
fi
if [ "$DEPOSIT" -lt "$SRVPRICE" ]; then
USRDEPORNOT=0
echo "
$DATEEXP
$QTEXP
$USR - Current Expiry = $SRVEXPIRYFULLD
$USR - Pacakge Name/Price: $PKGNAME | $SRVPRICE PKR
$USR - Current Deposite Available = $DEPOSIT"
echo -e "$COL_RED$USR | ERROR: Insufficient deposit for Renewal ! Current Deposite is $DEPOSIT and SRV renewal price is $SRVPRICE $COL_RESET
"
fi
if [ "$DEPOSIT" -eq "$SRVPRICE" ] || [ "$DEPOSIT" -gt "$SRVPRICE" ]; then
USRDEPORNOT=1
echo -e "$COL_GREEN$USR INFO: | Deposit Balance: $DEPOSIT | Service Name: $PKGNAME | Price $SRVPRICE $COL_RESET"
fi
########### ACCOUNT STATUS EXPIRED and DEPOSIT IS ENOUGH TO RENEW ACTION ############
if [ "$USREXPORNOT" -eq 1 ] || [ "$USRQTORNOT" -eq 1 ]; then
if [ "$USRDEPORNOT" -eq 1 ] ; then
# RENEW USERS IF ALL CONDITIONS MATCHED / PRINT FETCHED VALUES , JUST FOR INFO / ZAIB bbbbb
echo "
$DATEEXP
$QTEXP
$USR - Resetting Date: Current Expiry = $SRVEXPIRYFULLD / Next Expiry: $NEXTEXPIRYADD | UNIT = $EXPERIOD
$USR - Pacakge Name/Price: $PKGNAME | $SRVPRICE PKR
$USR - Current Deposite Available = $DEPOSIT | Deposite Balance after Deduction: $LASTUSRBAL"
# ADD 30 DAYS VALUE TO EXPIRED USER ACCOUNTzzzzzzzzzz
$CMD "use radius; UPDATE rm_users SET expiration = '$NEXTEXPIRYADD' WHERE username = '$USR';"
# ADD COMMENTS
$CMD "use radius; UPDATE rm_users SET comment = '$USR account last renewed from previous DEPOSIT $DATE' WHERE username = '$USR';"
# ADD SYSLOG ENTRY
#$CMD "use radius; INSERT INTO rm_syslog (datetime, ip, name, eventid, data1) VALUES (NOW(), 'n/a', 'DEPOSIT_$USR', '$USR', '$USR renewd service $PKGNAME');"
# UPDATE User Balance
$CMD "use radius; UPDATE rm_users SET credits = '$LASTUSRBAL' WHERE username = '$USR';"
# ADD INVOICE
$CMD "use $DB; INSERT INTO rm_invoices (managername, username, date, bytesdl, bytesul, bytescomb, downlimit, uplimit, comblimit, time, uptimelimit,
days, expiration, capdl, capul, captotal, captime, capdate, service, comment, transid, amount, invnum,
address, city, zip, country, state, fullname, taxid, paymentopt, paymode, invtype, paid, price, tax, remark,
balance, gwtransid, phone, mobile, vatpercent )
VALUES
('admin', '$USR', NOW(), '0', '0', '$PKGQUOTABYTES', '0', '0', '$PKGQUOTABYTES', '0', '0', '30', '$NEXTEXPIRYADD', '0', '0', '1', '0', '1', '$PKGNAME', 'This user service renewed by Deposit/Payment', '577343812eee0', '1', '$TODAYHM', '$ADDRESS', '$CITY', '00000', '$COUNTRY', '$STATE', '$FULLNAME', 'n/a',
DATE_ADD(CURDATE(), INTERVAL '10' DAY), '0', '0', '$TODAY', '$SRVPRICE', '0.000000', '$TODAYHM', '$LASTUSRBAL', '', '$MOBILE', '$MOBILE', '0.00' );"

$CMD "use $DB; INSERT INTO rm_invoices (managername, username, amount, price, tax, vatpercent, balance,
date, service, paymode, invgroup, paymentopt, transid)
VALUES ('admin', 'admin', 1, '-$SRVPRICE', '0', '0.00',
'', NOW(), 'Ref.: C-$TODAYHM', '2', '1', DATE_ADD(CURDATE(), INTERVAL '10' DAY),
'577343812eee0' );"
if [ "$RQT_RESULT1" -eq 1 ] ;then
echo "$USR - Total Data Reamining since last renewal = NO DATA REMAINING !"
else
echo "$USR - Total Data Reamining since last renewal = $RQT4"
fi
######################################################
############## QUOTA SECTION STARTS HERE #############
######################################################
# UPDATE Quota limitations if any, else ignore
DL=`$CMD "use radius; SELECT (SELECT SUM(acctoutputoctets) FROM radacct WHERE username = '$USR') - (SELECT COALESCE(SUM(dlbytes), 0) FROM rm_radacct WHERE username = '$USR');"`
UL=`$CMD "use radius; SELECT (SELECT SUM(acctinputoctets) FROM radacct WHERE username = '$USR') - (SELECT COALESCE(SUM(ulbytes), 0) FROM rm_radacct WHERE username = '$USR');"`
if [ "$DL" == "NULL" ]; then
DL=x
fi
if [ "$UL" == "NULL" ]; then
UL=x
fi
if [ "$RQT_OUT" == "0" ]; then
QTEXP="QUOTA INFO: NO QUOTA have been added yet! seems fresh account."
echo "QUOTA INFO: NO QUOTA have been added yet! seems fresh account."
fi
# QUOTA FRESH ADD MODE FOR NEW ACCOUNT
if [ "$DL" == "x" ] || [ "$UL" == "x" ]; then
echo "*********** QUOTA FRESH ADD MODE FOR NEW ACCOUNT"
echo "$USR - QUOTA INFO: Account Seems Fresh, Adding NEW Quota ..."
#$CMD "use $DB; UPDATE rm_users SET comblimit = '$PKGQUOTABYTES' WHERE username = '$USR';"
ADDICTIVE_COMB_LIMIT=`echo "1+$PKGQUOTABYTES" |bc -l`
$CMD "use $DB; UPDATE rm_users SET comblimit = '$ADDICTIVE_COMB_LIMIT' WHERE username = '$USR';"
fi
#exit 1
# QUOTA RESET MODE for expired QUOTA
IS_USER_QUOTA_END=`rmauth 127.0.0.1 $USR 1 |grep -i -c "Total traffic limit reached"`
if [ "$IS_USER_QUOTA_END" -eq 1 ] && [ "$RQT_RESULT1" -eq 1 ] ;then
DL=`$CMD "use radius; SELECT (SELECT SUM(acctoutputoctets) FROM radacct WHERE username = '$USR') - (SELECT COALESCE(SUM(dlbytes), 0) FROM rm_radacct WHERE username = '$USR');"`
UL=`$CMD "use radius; SELECT (SELECT SUM(acctinputoctets) FROM radacct WHERE username = '$USR') - (SELECT COALESCE(SUM(ulbytes), 0) FROM rm_radacct WHERE username = '$USR');"`
FINAL1_COMB_LIMIT=`echo "$DL+$UL" |bc -l`
FINAL2_COMB_LIMIT=`echo "$FINAL1_COMB_LIMIT+$PKGQUOTABYTES" |bc -l`
# ADDing Quota
echo "$USR - QUOTA info: RESET MODE: Existing Quota limit is Used. Adding New Quota as per package ..."
$CMD "use radius; UPDATE rm_users SET downlimit = '- $DL', uplimit = '- $UL', comblimit = '$FINAL2_COMB_LIMIT' WHERE username = '$USR';"
fi
## QUOTA ADDICTIVE MODE
IS_USER_QUOTA_END=`rmauth 127.0.0.1 $USR 1 |grep -i -c "Total traffic limit reached"`
if [ "$IS_USER_QUOTA_END" -eq 1 ] && [ "$RQT_RESULT_REMOVE_HYPHEN" -gt 0 ] ;then
#echo "********* QUOTA ADDICTIVE MODE"
echo "$USR - QUOTA info: ADDICTIVE MODE: Existing Quota is remaining. Adding New Quota in existing quota as per pacakge (IF ANY) ..."
##echo "$USR - QUOTA setting is - ADDICTIVE ..."
ADDICTIVE_COMB_LIMIT=`echo "$USR_CUR_COMBLIMIT+$PKGQUOTABYTES" |bc -l`
$CMD "use $DB; UPDATE rm_users SET comblimit = '$ADDICTIVE_COMB_LIMIT' WHERE username = '$USR';"
fi
# QUOTA ADDICTIVE MODE for DATE EXPIRED USERS ONLY
IS_USER_QUOTA_END=`rmauth 127.0.0.1 $USR 1 |grep -i -c "Total traffic limit reached"`
if [ "$IS_USER_QUOTA_END" -eq 0 ] && [ "$RQT_RESULT1" -eq 0 ] ;then
echo "$USR - QUOTA info: ADDING quota for DATE EXPIRED USERS ONLY (Existing Quota will be added in new Quota, IF ANY) ..."
ADDICTIVE_COMB_LIMIT=`echo "$USR_CUR_COMBLIMIT+$PKGQUOTABYTES" |bc -l`
$CMD "use $DB; UPDATE rm_users SET comblimit = '$ADDICTIVE_COMB_LIMIT' WHERE username = '$USR';"
fi
#####################################################
############## QUOTA SECTION ENDS HERE ##############
#####################################################
RQT=`$CMD "use radius; SELECT username,
IF (limitcomb =1, comblimit - COALESCE((SELECT SUM(acctinputoctets + acctoutputoctets) FROM radacct
WHERE radacct.username = tmp.username) -
(SELECT COALESCE(SUM(ulbytes + dlbytes), 0) FROM rm_radacct
WHERE rm_radacct.username = tmp.username), 0), 0)
FROM
(
SELECT username, firstname, lastname, address, city, zip, country, state, phone, mobile, email, company,
taxid, rm_users.srvid, rm_users.downlimit, rm_users.uplimit, rm_users.comblimit, rm_users.expiration,
rm_users.uptimelimit, credits, comment, enableuser, staticipcpe, staticipcm, ipmodecpe, ipmodecm, srvname, limitdl,
limitul, limitcomb, limitexpiration, limituptime, createdon, verifycode, verified, selfreg, acctype, maccm,
mac, groupid, contractid, contractvalid, rm_users.owner, srvtype
FROM rm_users JOIN rm_services USING (srvid) ORDER BY username ASC
) AS tmp
WHERE 1 AND (tmp.acctype = '0' OR tmp.acctype = '1' OR tmp.acctype = '3' OR tmp.acctype = '4')
AND username LIKE '$USR%';" | awk '{print $2}'`
RQT_HUMAN_FRIENDLY=`bytesToHuman $RQT`
echo "$USR - Current Updated Quota after the renewal ... : $RQT_HUMAN_FRIENDLY"
USRIP=`$CMD "use $DB; SELECT framedipaddress FROM radacct WHERE acctstoptime IS NULL AND username = '$USR';"`
if [ -z "$USRIP" ]; then
echo "$USR - Online Status: Offline !"
else
NASIP=`$CMD "use $DB; SELECT nasipaddress FROM radacct WHERE username = '$USR' and acctstoptime IS NULL;"`
ONLINE_TIME=`$CMD "use $DB; SELECT acctsessiontime FROM radacct WHERE username = '$USR' and acctstoptime IS NULL;"`
ONLINE_START_TIME=`$CMD "use $DB; SELECT acctstarttime FROM radacct WHERE username = '$USR'and acctstoptime IS NULL;"`
ONLINE_TIME_FIN=`echo "$ONLINE_TIME/60/60" |bc -l | cut -c1-4`
echo "$USR - Online Status: IP: $USRIP | Online Since: $ONLINE_START_TIME | Online Time: $ONLINE_TIME_FIN Hour(s) | NAS: $NASIP "
fi
fi
fi
done


Result:

  • /temp/auto.sh
- INFO: Total Users scanned: 2
INFO: No user found with Expired Date/Quota package ... Exiting peacefully!
  • /temp/auto.sh
- INFO: Total Users scanned: 2

test50 INFO: | Deposit Balance: 99960 | Service Name: 2mb Speed - 30 Days - 1 GB Quota | Price 10

test50 - *** Account Date Expired that is 2022-02-14 00:00:00 !! **
test50 - Account QUOTA is OK that is 2.00 GiB
test50 - Resetting Date: Current Expiry = 2022-02-14 / Next Expiry: 2022-03-16 | UNIT = 30 Days
test50 - Package Name/Price: 2mb Speed - 30 Days - 1 GB Quota | 10 PKR
test50 - Current Deposit Available = 99960 | Deposite Balance after Deduction: 99950
test50 - Total Data Reamaining since last renewal = 2.00 GiB
test50 - QUOTA info: ADDING quota for DATE EXPIRED USERS ONLY (Existing Quota will be added in new Quota, IF ANY) ...
test50 - Current Updated Quota after the renewal ... : 3.00 GiB
test50 - Online Status: Offline !
  • /temp/auto.sh

- INFO: Total Users scanned: 2

test50 INFO: | Deposit Balance: 99950 | Service Name: 2mb Speed - 30 Days - 1 GB Quota | Price 10

test50 - Account Date is OK that is 2022-03-16 00:00:00.
test50 - *** Account QUOTA Expired that is Negative 742.69 MiB !! **
test50 - Resetting Date: Current Expiry = 2022-03-16 / Next Expiry: 2022-03-16 | UNIT = 30 Days
test50 - Pacakge Name/Price: 2mb Speed - 30 Days - 1 GB Quota | 10 PKR
test50 - Current Deposite Available = 99950 | Deposite Balance after Deduction: 99940
test50 - Total Data Reamining since last renewal = NO DATA REMAINING !
test50 - QUOTA info: RESET MODE: Existing Quota limit is Used. Adding New Quota as per package ...
test50 - Current Updated Quota after the renewal ... : 1024.00 MiB
test50 - Online Status: Offline !

January 2, 2022

SANGFOR IAM – Short Notes

Filed under: SANGFOR — Tags: , , , — Syed Jahanzaib / Pinochio~:) @ 7:25 PM

in year 2019,  We acquired SANGOFR IAM m5200 hardware device (along with 3 years support/renewal bundle) as an replacement for Microsoft ISA/TMG 2010 product. Since its acquisition & so far we have good experience with its usage. We tested few other products like Sophos, FortiGate & few other, but IAM was the closest replica for TMG replacement (specially in its integration with the AD) therefore we went for it. As per our core requirements of Compliance/audit, IAM Logging details level is very impressive. It’s local support was very good and responsive & they helped us in initial demo & configuration.

As time will allow, I will try to add some guides/tips and notes for day to day task related to Sangfor IAM.


Fake Online Sessions & Forced Logged out Everyday !

Sangfor have multiple methods to integrate its authentication with the Active Directory DC. We are using LOGIN Script Method (using GPO) along with IWA integration. It works well, but it was creating one BIG issue. When the user logins to the computer, the login script via GPO triggers & the sangfor login exe sends the user login event to the sangfor & sangfor then notified about the user status. But if the user dont do proper shutdown of his workstation, then the user session shows as ONLINE for hours & in the mean while if his dhcp lease expires & some other user gets the same IP , his internet access works without any kind of authentication.

To over come the issue, we set following,

the LOG OUT ALL USERS EVERY DAY option did the trick! However it created another problem that users who donot do proper shutdown, and next when open laptops and resume windows (from hibernation actually) , they get the IAM LOGIN screen on there browser. By setting an S.O.P, we enforced users that its essential that you must properly shutdown when you leave the office, or LOGOFF/LOGIN one time.

Weird !



Allow Office 365 / Outlook related connectivity to Particular AD Group.

In our office, all users are joined with Active Directory Domain. (there are multiple domain with cross forest trust in our company). We have allowed limited internet facility to particular active directory group only. This year we have moved away from on-prem Lotus domino email server to cloud base Microsoft O365 solution, therefore we had to allow internet to every body who is now using Outlook. To limit the internet usage & after doing some extensive R&D & ‘internet activities’ lookup via sangfor , we created following ‘O365’ Object in URL DATABASE, and allowed it  to AD group ‘Internet_for_O365_Group’ & associate outlook users to this group. This way users who doesn’t have internet facility can still use O365 related services in a controlled manner.


*.office365.com
*.office.com
*.office.net
*.outlook.com
*.microsoft.com
*.onmicrosoft.com
*.microsoftstream.com
*.azure.net
*.azureedge.net
*.windows.net
*.live.com
*.atdmt.com
*.ytimg.com
*.windowsazure.com
*.msftidentity.com
*.msidentity.com
*.microsoftonline.com
*.msecnd.net
*.msftauth.net
*.msauth.net
*.azure.com
*.digicert.com
*.agp.com.pk
*.obsagp.com.pk
*.msftconnecttest.com
*.acompli.net
*.sharepoint.com
*.live.net
*.onedrive.com
*.msftstatic.com
*.windows.com
*.s-microsoft.com
*.passport.net
*.msocsp.com
*.msftncsi.com
*.msedge.net


More will be added as per time allow.

Regard’s
Syed Jahanzaib

November 22, 2021

Short Notes for Active Directory SLD to FQDN Cross Forest Migration – Using ADMT tool



FRIST ! Some DRY Theory …

What is a Single Label Domain (SLD)?  This is a term that Microsoft uses to describe domains which have only a single name, and no suffix such as “.local” or “.com.”  For example, your Active Directory domain might have a name like “company.local,” but if it were Single Label, it might be just “company.”

The problem is, Single Label Domains fall into a grey area of Microsoft support.  SLD will cause multiple issues when integrating with other applications, and even when performing something simple like joining a new computer (windows 10 for example) to the domain & you need to modify the registry as well. SKype for business will not work SLD and the long list goes on.


What is the Active Directory Migration Tool (ADMT)?

The Active Directory Migration Tool (ADMT) is a Microsoft software application that helps you manage and perform the necessary operations to move AD objects. You can move objects within the same domain forest (intraforest) or to a different forest (interforest).


Our Story!

Recently we migrated our active directory domain users from Single Label Domain SLD to FQDN. It was cross forest migration & both forest were on same LAN. We were using Lotus Domino Email server on premises & wanted to migrate on Exchange Online (office 365 cloud). To perform the migration , it was strongly recommended by the Microsoft to migrate SLD to FQDN. Our SLD AD domain was running from past 16-17 years started with windows 2000 to 2003 to 2008 to 2012 to 2016, therefore after much planning , we planned the migration , tested it multiple times in virtual labs. There are multiple 3rd party tools available for the smooth migration but for our number of users the estimated cost for tool acquisition was 4 Million+ PkR which was a huge amount, therefore we used the Microsoft free tool called ADMT, which is quite old , BUT still it did the job for us.

I may not be able to write down all details because internet is full of such step by step video tutorials, I will try to provide you details only related to some customization we applied for smoother transition.

Specs used in this guide & few tips to follow before doing ADMT execution

  • Source Domain : SLD with Windows Server 2016
  • Target Domain  : FQDN on Windows Server 2019
  • Workstations : Mostly Windows 10 PRO with multiple builds including v 1909 – OS build 18363.1556 | W10 20H2 OS build 19042.1348
    Note ADMT 3.2 Only support the migration of Operating Systems up to Windows 7, (that doesn’t mean Windows 8 and Windows 10 wont work, it just means they are not supported). Migrating Windows 8 and 10 throws a lot of security translation errors, because of the way it treats ‘Apps’, so I’d recommend you do a LOT of testing before carrying out a live migration.
  • ADMT tool version 3.2 : Installed on Target DC
  • SQL Server 2008 R2 Express / Version 10.52.4000.0 installed on Target DC
  • PSTOOLS library installed on both DC for ease of CMD execution on remote WorkStations/servers
  • To migrate machines, the ADMT Admin user needs “Local” administrative access on all the source machines. The easiest way to do this is via group policy, using ‘Restricted Groups’.  This allows you to take a group (or user) and put put them on the local groups (including administrators) of the targeted machines.

    In the OLD domain, create a group and put the ADMT Admin from the target domain in it. (I put the domain admin from the target domain in it as well, to be on the safe side, but that’s up to you).

    It can be done iva opening group policy and navigate to
    Computer Configuration > Policies  >Windows Settings > Security Settings > Restricted Groups
    Add a new one, select the group you have just created > and add it to ‘Administrators’.

  • ADMT doesn’t have a rollback function. Be sure before you migrate the objects
  • Make sure you have documented all the domain related stuff very well in sheets, Also Ensure you perform at least 1-2 test labs in isolated virtual environments
  • If you want to make life easier during migration, DISABLE windows FIREWALL. To perform computer migrations, (and security translations), ADMT needs to deploy an ‘agent’ to the machines in the OLD domain. The local firewall (if enabled) can stop this, I simply disable the users local firewall via GPO to avoid any inconvenience
    It can be done by opening group policy and navigate to
    Computer Configuration >Policies  > Administrative Templates > Network > Network Connections > Windows Firewall > Domain Profile
    Locate the “Windows Firewall protect all network connections‘ and set it to disabled.
  • Enable Remote Registry Key Service on users workstation via GPO. I have also seen the agent fail to deploy if the ‘Remote Registry Service’ is not running on the target machines, (it’s disabled by default in newer version of windows). So I use the GPO policy to turn that on as well on all workstations
    It can be done by opening group policy and navigate to
    Computer Configuration > Polices > Windows Settings > Security Settings > System Services
    Locate the ‘Remote Registry’ service, and set it’s startup to automatic.
  • It is recommended to Disable Sleep and Hibernation via GPO

Step by Step ADMT

Note: Details can be found over internet as so many guides have already been written & shared

It is assumed that new domain controller is installed along with proper DNS working.

  • Supported Operating System : Source/Target must be running Windows Server 2008 or above
  • ADMT requires a SQL server to store data, Download & install SQL Express on Target DC > Download Link for SQL Express
  • Download & install ADMT tool on Target DC > Microsoft Download Link for ADMT
  • Download & install Password Export Service on Source DC > PSE Download Link from Microsoft
    * Create the encryption key for password migration on the source DC and copy it to the target DC. (user password migration) | Install ADMT PW Migration DLL & reboot source DC could be required
  • Prepare Active Directory for the migration process. There are two main things to prepare, DNS & a domain trust
    * Create DNS CONDITIONAL FORWARDERS on source/target DC pointing to each other  / Add the DNS suffixes for each domain via GPO. The old domain needs to be able to resolve names in the new domain, and the new domain needs to be able to resolve names in the old domain. To achieve this you need to setup ‘Conditional Forwarding’ in each domain for the other one
    * In addition, we want all machines (in both domains) to set their primary DNS Suffix, to their own domain, and their DNS suffix search list to look for their own domain first, then the other domain. The easiest way to do that is via group policy.  On a domain controller > Administrative Tools > Group Policy Management Console, NAVIGATE To
    Computer Configuration > Policies > Administrative Templates > Network > DNS Client
    Setting: DNS Suffix Search List: Set to current domain ‘comma‘ other domain , Example olddomain.com,newdomain.com
    > Repeat the procedure in the new domain, (but the domain names will be the opposite way round)
  • Using, Active Directory Domains and Trusts Tool on source/target DC’s, create two way TRUST
    * You can use below guides to do so , Link#1Link#2
  • Using ADMT on Target DC, start migrating in following numbering

a) Migrate AD Groups
> There is a sequence for doing this though, there are three types of security groups and they HAVE TO be migrated in the correct order,
First: Universal Groups
Second: Global Groups
Third: Domain Local Groups

b) Migrate AD Users Account

c) Migrate Security translation of COMPUTER

Real World Note ons SID migration: This can take a while, (up to an hour for some machines, Luckily our users had most advance laptops/desktops with SSD’s Flash storage so ma time we saw was 20-25 mnts for heavy data user) and it’s best done without anyone being logged in (to prevent any profiles, or registry hives being locked). So take time to plan when this is done – rush it and you will have problems, and the very users who are too busy to be interrupted, are the very ones that shout the loudest if there’s a problem post migration.
d) Migrate COMPUTER objects in the end which will reboot the client PC upon completion, & then after reboot user can login to NEW domain (its a manual process to select OTHER USERS to see the NEW domain in the list, this is one time & next time windows will remember the last sign_IN, to avoid this manual thing, you can read the full guide in the end)

 


Following are few Tips & tricks that we followed or created on the case basis …

Force users to login to NEW domain by using OTHER USER login as by default login option

Once you migrate the user / sid translation & computer migration to new Domain & user PC gets reboot, on next Login user will still see his old domain login screen. In order to login to NEW domain, user have to click on OTHER USER option . This is because windows remembers last signed in USER. [In companies where IT Dept have strong hold on users or management its very easier to just inform all the users via email/contact or other means, but in company were your boss is kind of dept. head is afraid of informing users about change, then you have to follow the long route & in our case we had to do it too.]

  • in Source/Target ADUC, we create new OU named “Pre-Migration-Temporary-OU
  • in Source/Target DC, we created new GPO named “Pre-Migration-GPO” in which we set following setting
    Computer Configuration > Policies > Windows Settings > Security Settings > Security Options
    on right pane, we modified below setting
    “Interactive logon: Don’t display last signed-in” >set it to>  *ENABLED*
  • On source DC, we moved the user COMPUTER object to “Pre-Migration-Temporary-OU“. AS you know that it generally takes 90 minutes or more on user computer to auto refresh the Gpupdate, therefore for instant update we used PSTOOLs command PSEXEC. Using it  we executed cmd “gpupdate /force” on target user PC.PSEXEC sample CMD:
PsExec.exe \\USER1-PC -u DOMAIN\ADMINISTRATOR -p ADMINPASSWORD cmd "/c gpupdate /force"
  • Then on Target NEW DC, using ADMT console, we first moved his user account / SID translation to new DC. Finally we migrated his COMPUTER to NEW DC in “Pre-Migration-Temporary-OU” as target
  • Once his Computer migrated & rebooted, the user is now seeing OTHER USER login window (which is by default NEW DOMAIN) , once he logged IN using his username / password to new DOMAIN & he is able to see old settings desktop etc, we then moved his COMPUTER from Pre-Migration-Temporary-OU to COMPUTERS OU on the NEW DC

Disabling User Must change password After Next Logon

After the user account was migrated to new domain, it had the following option ENABLED by default

  • User must change password after next logon

This was creating problem for us because every user will change password, & as a result he have to change his Lotus Notes / SameTime chat app / Mobile app password too, because lotus notes have SINGLE LOGON option which syncs password from active directory. At the time of migration with hundreds of users & with very limited IT resource, we used following scheduled script to get rid of ti temporarily basis till the migration lasted.

Wrap below CMD in any batch file and schedule it to run every minute

powershell.exe -inputformat none -command "Get-ADUser -Filter * -SearchBase 'OU=MYMAINOU,DC=mydomain,DC=local' | Set-ADUser -ChangePasswordAtLogon:$False"

Few issues & there resolutions after user migration 

We Observed few limitations in the migration process

  • IBM SAMETIME Password wiped: Not all 3rd party applications settings can be fully migrated, this includes some apps saved password. Example we are using IBM lotus SAMETIME lan chat app, its SAVED password didn’t migrated & user have to open and enter his password & tick on remember/auto login option
  • Google chrome settings are not migrated as well. Good part is If the user is logged in to google chrome using his GMAIL account. he can re login & all of his settings will be synced back. but if the user was google chrome as GUEST, then his settings will not be migrated

Missing Profile Problem after user is migrated via ADMT

In some cases, once the user is migrated, his account was showing disabled on NEW DC, (which we didn’t noticed) and once all settings migrated, and user logged in to his PC, his profile was empty like fresh. This created very much hurdles for some executive level users.

If for some reason, if a migrated user gets a NEW FRESH profile (or in other words lost the old profile) you can use the following procedure to re-assign the old profile back to the target account

  1. Run Regedit on USER computer
  2. Go to HKEY_LOCAL_MACHINE\Software\Microsoft\Windows NT\CurrentVersion\ProfileList
  3. Go through the ProfileList and identify the Source account (OLD PROFILE example user.old.domain). Copy the value from the ProfileImagePath key
  4. Again go through the Profile list and identify the Target account example user.new.domain). Paste the ProfileImagePath key value there
  5. Restart the user workstation

The ProfileImagePath key will be same value for both Source and Target user accounts. This ensures both source and target users will receive the same profile which is stored under C:\Documents and Settings\UserName. or c:\users\xyz (in new version of windows)

Calculator was missing from windows 10 pc

First try this

Option#1

Reset Calculator’s data

Get-AppXPackage -AllUsers | Foreach {Add-AppxPackage -DisableDevelopmentMode -Register "$($_.InstallLocation)\AppXManifest.xml"}

DISM /Online /Cleanup-Image /RestoreHealth

Option#2

Type the following 2 commands at the CMD prompt:

powershell

Always Wait for the Network link during Login

During the migration we observed that our users desktop/laptops had modern hardware with new generation i7 / SSD-Flash storage & they boots in just few seconds, & user logged in quickly ever before the ethernet/wifi gets IP from the DHCP. This was creating problems in MAPPED NETWORK DRIVE & new group policy implication , which was necessary for the migration & backup process as well in network drive, therefore we enabled the “Always wait for the network at computer startup and logon” using group policy & applied to all users OU’s.

It was located under

  • Computer Configuration — Policies — Administrative Templates — System — Logon

This solved our drive mapping issue along with domain WELCOME login script as well which executed few other commands as well like NTP sync, Sangfor Login Script, customized login entry in our central location etc.

When the “Always wait for network at computer startup and logon” policy is enabled, it then refers to the policy for how long it should wait for network connectivity, before dumping the policies and logging the user on with cached credentials. If the “Startup policy processing wait time” is not specified, the default is 30 seconds. If the network is already connected, the computer would log in without waiting further , but would take about 30 seconds if there was no network connectivity (or no DC available).

I set it to to wait 50 seconds for network then continue logging in “GPO Processing wait time”. Default with the setting on is 30 seconds which seemed too short.

 

 

October 26, 2021

Veeam B&R: [nbd] Error: Object reference not set to an instance of an object.

Filed under: Veeam B&R — Tags: — Syed Jahanzaib / Pinochio~:) @ 10:45 AM

Scenario:

  • Veeam B&R 9.5 (Ver 9.5.0.1922) installed on Windows Server 2012R2

We were getting failure error on Veeam daily job report for a particular VM guest ,

Preparing backup proxy VMware Backup Proxy for disk Hard disk 1 [nbd]
Error: Object reference not set to an instance of an object.

It was a VM guest that was recently restored via vcenter to another esxi host (under vcsa).

To further troubleshoot, I exported the JOB log using Veeam article, & observed following

[23.10.2021 04:56:01] < 332> cli| Waiting for the next command.
[23.10.2021 04:56:01] < 3936> cli| Next client command: [closeStg].
[23.10.2021 04:56:01] < 3936> cli| :> X:\VMBKP\Daily VM BKP job via VCSA\MY_VM_GUEST.16D2021-10-23T030322.vbk
[23.10.2021 04:56:01] < 3936> cli| Closing storage ‘X:\VMBKP\Daily VM BKP job via VCSA\MY_VM_GUEST.16D2021-10-23T030322.vbk’.
[23.10.2021 04:56:01] < 3936> cli| MTA backup apartment was created successfully, id ‘0x000001EA804A1E10’.
[23.10.2021 04:56:01] < 3048> cli| Thread started. Thread id: 3048, parent id: 3936, role: MTA invoke thread
[23.10.2021 04:56:01] < 3048> cli| Entering MTA invoke thread.
[23.10.2021 04:56:01] < 3048> stg| There is no file ‘HostFS://X:\VMBKP\Daily VM BKP job via VCSA\MY_VM_GUEST.16D2021-10-23T030322.vbk’ in the files cache.
[23.10.2021 04:56:01] < 3936> stg| There is no file ‘HostFS://X:\VMBKP\Daily VM BKP job via VCSA\MY_VM_GUEST.16D2021-10-23T030322.vbk’ in the files cache.
[23.10.2021 04:56:01] < 3936> cli| Storage is closed.

To sort the issue I did following & the issue got resolved

  • Remove the VM from the VEEAM job
  • Remove the VM from inventory from particular ESXI host,
  • Re-added the VM in ESXI
  • Added the VM in VEEAM job

Afterwards, the job ran fine without any error.

 

Veeam B&R: Win32 Error / Code: 67 Cannot connect to the host’s administrative share


Continued post from


Scenario:

  • Veeam B&R 9.5 (Ver 9.5.0.1922) installed on Windows Server 2012R2

 

After running the daily job, we were observing following failure notice for particular windows 7/10 OS.

Processing WIN10-LOG Error: Failed to connect to guest agent. Errors: ‘Cannot connect to the host’s administrative share. Host: [WIN10-LOG]. Account: [admin]. Win32 error:The network name cannot be found. Code: 67 Cannot connect to the host’s administrative share. Host: [19.168.0.10]. Account: [admin]. Win32 error:The specified network name is no longer available.

After some R&D , I had to create a new DWORD value in registry at effected windows , & afterwards the issue got resolved.

Edit the registry on effected Windows & perform following

  1. Click Start, click Run, type regedit, and then press ENTER.
  2. Locate and then click the following registry subkey:HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Policies\System
  3. If the LocalAccountTokenFilterPolicy registry entry doesn’t exist, follow these steps:
    1. On the Edit menu, point to New, and then select DWORD Value.
    2. Type LocalAccountTokenFilterPolicy, and then press ENTER.
  4. Right-click LocalAccountTokenFilterPolicy and then select Modify.
  5. In the Value data box, type 1, and then select OK.
  6. Exit Registry Editor.

On a safe side , perform following as well,

  • Disable windows based firewall
  • Disable windows UAC

Reboot the affected windows machine & now RUN the backup job on Veeam B&R server.


Exporting VEEAM job LOGS for troubleshooting purposes

It’s a good idea to export JOB logs in order to troubleshoot the error in depth.

Follow below veeam article


Regard’s
Syed Jahanzaib

Older Posts »

%d bloggers like this: