Syed Jahanzaib – Personal Blog to Share Knowledge !

January 4, 2024

TIP: Robocopy – Delete Files/Folder larger then 255 characters

Filed under: Microsoft Related — Tags: , , , , — Syed Jahanzaib / Pinochio~:) @ 9:13 AM

We have a windows base file sharing server which have hundreds of users folder having millions of files. Sometimes when any user left the organization, we move his particular data folder to our archive servers/tapes, & then remove that folder from production server. But since it contains year of data, & nested folders inside multiple folder, therefore we receve above error when deleting it. Sometimes if number of folders fro that user folder is  lesser, we simpley rename few larger folders names inside it to shorter length & then delete works, but still in few cases this trick donot works. For such folders we uses classic ROBOCOPY trick. here is the Command line that comes inhandy when you need to remove larger folder.


  • Login to your file server (or computer where that folder is residing)
  • Take backup of that particular folder you want to DELETE (Example target folder is E:\Users\UserX1)
    (just incase you need to recover any data from it)
  • Open command prompt, & Create any Empty Directory like
mkdir E:\EmptyFolder
  • Now issue below command to remove the LONG folder
robocopy E:\EmptyFolder E:\Users\UserX1  /purge
  • Since there are no files or folders in the source directory (E:\EmptyFolder), it simply deletes the files and folders under the target directory (E:\Users\UserX1) recursively.
  • Delete the empty folder E:\EmptyFolder when finished.

(purge is the key)


Regard’s
Syed Jahanzaib

April 20, 2023

Mikrotik IPSEC Tunel with SAP HEC !

Filed under: Mikrotik Related — Tags: , , , , , , , , , , , , — Syed Jahanzaib / Pinochio~:) @ 10:27 PM


Following post contains information on how we configured IPSEC tunnel using our mikrotik router (CCR1036) with SAP HANA (High-performance Analytic Appliance) HEC (Hana Enterprice [private] Cloud). I am documenting some basic ideas along with configuration codes for myself, because soon we will be migrating from our decade old mikrotik router to Fortigate NGFW.

HEC is an infrastructure-as-a-service tool that enables companies to run SAP-based operations in a hosted environment. HANA Enterprise Cloud (HEC) was introduced to tr+ansform, migrate and operate mission critical applications in a private cloud environment.

For SAP HEC , We opted for Policy based VPN tunnel (wiht PSK) which was relatively simpler to configure & doesnt requires BGP/ASN which is required in route based vpn option. SAP team shared one sheet with us asking for ipsec related parameters, our local ip subnet, & one subnet for sap end virutal instance that we will be accessing at there end & our wan IP. they shared IPSEC related parameters as per below,

  • Phase 1: IKEv2-AES256-SHA384-DHgroup20
  • Phase 2: AES256-SHA256-PFSgroup20
  • Pre-Shared Key (PSK)
  • SAP peer IP address: 1.2.3.4

Unlike other types of VPN’s like pptp/etc, Policy-based IPSec doesn’t have routes. With the Mikrotik, IPSec does not create a virtual interface & you don’t need to add any routes. The packets head for the default route, but the IPSec policy matches the source/dst subnets, and route where it it needs to !

  • All VPN/IPSEC parameters must meet exact at both ends i.e HEC & customer end. In beginning, SAP HEC end made some mistakes like incorrect pre-shared key (PSK) was shared with us, our wan ip was not added in there vpn server allowed hosts which cane establish vpn connectivity with there servers. In such cases your local partner ROLE is very important & unfortunatley we didn’t get any active network related support from our local technology parter ! 

Example used in this post for illustration purposes:

  • IPSEC paramenters as metnioned above,
  • SAP HEC VPN Server IP: 1.2.3.4
  • Source LAN IP Pools: 10.1.1.0/24 , 172.16.1.0/24 (our office local subnet, it must be allowed on remote vpn server)
  • Destination SAP HEC Private IP Pool on which SAP HEC is hosted: 192.168.50.0/24 (We asked them to use this pool at there end)

Mikrotik SAP HEC IPSEC Workingn Config Code:

All the configuration regaridng IPSEC parameters must match exactly at both ends (vendor & customer router) else tunnel will not happen. (In some cases it may be required to request for vendor remote assistance & show them your settings so that they can tune it at there end for matching)

# Add IPSEC profile
/ip ipsec profile
add dh-group=modp2048 enc-algorithm=aes-256 hash-algorithm=sha384 name="SAP HEC IPSEC PROFILE" nat-traversal=no

# Add remote IPSEC server IP address with IKEv2 Support
/ip ipsec peer
add address=1.2.3.4/32 exchange-mode=ike2 name="SAP HEC IPSEC PEER" profile="SAP HEC IPSEC PROFILE"

#Add IPSEC proposal
/ip ipsec proposal
add auth-algorithms=sha256 enc-algorithms=aes-256-cbc lifetime=1d name="SAP HEC IPSEC Proposal" pfs-group=modp2048
# Disable default proposal, as not required
set [ find default=yes ] disabled=yes enc-algorithms=aes-128-cbc

# Add IPSEC pre-shared KEY (provided by SAP or you can also create your own key & shar eit to SAP so that they can set it at there end for your IPSEC profile)
/ip ipsec identity
add peer="SAP HEC IPSEC PEER" secret="STRONG-PRE-SHAREd-KEY"

# Add IPSEC Policies, providing information on what local IP subnet will be able to communicate with remote IP pool, (must be allowed at SAP end as well)
/ip ipsec policy
add dst-address=192.168.50.0/24 peer="SAP HEC IPSEC PEER" proposal="SAP HEC IPSEC Proposal" src-address=10.1.1.0/8 tunnel=yes level=unique
add dst-address=192.168.50.0/24 peer="SAP HEC IPSEC PEER" proposal="SAP HEC IPSEC Proposal" src-address=172.16.1.0/24 tunnel=yes level=unique

# Disable default IPSEC policy, as not required
set 3 disabled=yes

Mikrotik Exemption Rules for IPSEC Tunnels

#Bypass PVN pool from other chains

/ip firewall raw
add action=accept chain=prerouting dst-address=192.168.50.0/24

#Below may not be require if you add it in RAW, but documenting it as we made it work with different scenarios
/ip firewall nat add chain=srcnat action=accept place-before=0 src-address=10.1.0.0/24 dst-address=192.168.50.0/24
/ip firewall nat add chain=srcnat action=accept place-before=0 src-address=172.16.1.0/24 dst-address=192.168.50.0/24

# Exempt SAP HEC Pool from mikrotik local NAT rule
/ip firewall nat
add action=src-nat chain=srcnat comment="Default_NAT_Rule_for_Allowed_Users with Exempting IPSEC Tunnel" dst-address=!192.168.50.0/24 out-interface=WAN1 src-address-list=Net_Allowed_users action=masquerade

TIPS:

Following are few tips that I learned in a hard way !

  • If you have multiple local subnets, you need to create POLICIES accordingly , as shown in the example in this guide (code example). Level of every policy should be set “unique” instead of “required”.
    Also these pools must be allowed at remote server. In our case , only one pool was allowed at remote server in beginning & this created many days in wall banging efforts, as a workaround we created a src-nat rule to NAT our other local subnets to src-nat via allowed pool ! example:
/ip firewall nat
add action=src-nat chain=srcnat comment="Workaround - Jugaaroo Rule to make IPSEC Tunnel Work / SJZ" disabled=no dst-address=192.168.50.0/24 src-address=172.16.1.0/24 to-addresses=\
x.x.x.x (my router lan subnet ip which is allowed on remote site)
  • All VPN/IPSEC parameters must meet exact at both ends i.e HEC & customer end. In beginning, SAP HEC end made some mistakes like incorrect pre-shared key (PSK) was shared with us, our wan ip was not added in there vpn server allowed hosts that cane stablish vpn connectivity with there servers. In such cases your local partner ROLE is very important & unfortunatley we didn’t get any active network related support from our local technology parter ! 
  • You also need to exclude the IPSec subnets from the masquerade natting rule. Ensure you EXCLUDE the detination SAP HEC Pool from *any* NATTING rule (show in code example)
  • If you clients can ping the remote servers, but your mikrotik itself cannot, then You can fix it either using srcnat, or with fake routes where you set pref-src. It doesn’t matter where such route points to, it won’t be actually used. It can be e.g. empty bridge.  Route with pref-src is better, because it works for everything. E.g. if you do traceroute from one LAN to another, there will by default be a hole where remote router should be.
  • Seond option, If you are pinging from the Mikrotik itself, make sure you set src-address so that it matches the policy example
    ping 192.168.50.10  src=10.1.1.1 (10.x is IP from allowed pool)
  • Try disabling and re-enabling the identity, as adding policies on the fly behaves funny in some RouterOS versions.

IPsec has as two modes, one is the Tunnel mode and the second is the Transport Mode.

Tunnel mode is used in site to site VPNs, between Gateways in simple words and is the default mode while Transport mode is used for client to site VPNs or end to end, between a computer and a Gateway…

Now, the Tunnel mode adds a New IP Header in front of the IP header of the originating packet…
In transport mode no new IP header is added, so the IP stays the same… Unless NAT is perforfmed…

If you have transport mode IPSec (e.g. for L2TP/IPSec), decrypted packet (L2TP) will have same addresses as encrypted (unless it’s changed by NAT). If you have tunnel mode IPSec (e.g. LAN to LAN tunnel), decrypted packet will have the source address of remote device in LAN (it it came from it) and destination address will be device on local LAN, while encrypted packets will have remote router as source and local router as destination.

IPsec policy kicks in after all other packet processing, including routing and NAT, has been done and the packet is just about to be sent out via an interface. So some route must match the dst-address of the packet, but it is not important which one in particular it is. And to be recognized by the policy, the packets’ src address must not be changed by src-nat if the route sends them out via WAN with src-nat; on the other hand, you may want to src-nat them intentionally in some scenarios.


References: Mikrotik Forum

 

Regard’s
Syed Jahanzaib

March 10, 2023

Story about Mass migration of SLD / VLAN’s / IP Pool / Wifi Mesh Loop / Rectification of Bonjour mDNS using Parallel Topologies







This post is a quick reference notes for myself on how I managed to rectify some of long pending IT related issues which were left by previous network team, including AD Domain S.L.D migration to FQDN / Public IP scheme being used at Private LAN & its migration to multiple /24 Private IP Pools, Single default VLAN-1 migration to Different VLAN’s with smaller subnets.

The number of desktop/laptop/devices were in several hundreds. Downtime was not an option, therefore we managed to complete the task using parallel topologies for each segment.

Our existing network was running on default VLAN with /8 flat subnet, having each & every device on default VLAN-1 generating enormous number of broadcasts flooding causing network delays/timeout. There was no control on any switch segment, All switch edge port were set on Default roles making situation even worst from Spanning-Tree protocol flooding perspective. Above all like Cherry on the TOP, existing admin used public IP scheme with /8 subnet & many websites which were on this subnet over the internet, never worked on our network.

To over come these issues without any Downtime or disruption, I decided to sort issues on a one by one basis by creating parallel network, using below methods


SLD/FQDN Migration:

  • On same network, I added new FQDN A.D Domain, Established two way trust between both domains. Then using Microsoft migration tool (ADMT ), I start migrating users (one by one, targeted activity ) from old SLD to FQDN. More details are documented in another article.

Short Notes for Active Directory SLD to FQDN Cross Forest Migration – Using ADMT tool


*  Network Related *

We have Cisco L3 Stacked switch at data center & all departmental L2 switches are connected via 10G Fiber TRUNK links. To convert the existing Public IP scheme to Private without having any downtime, I I designed different VLAN’s scheme with smaller /24 private IP’s pool on per department basis, Created there SVI’s on main core switch stack. Using IP helper commands. I also created different vlans for management example different vlan for switch / access points / devices / servers & Users. I created DHCP relay pointed towards our domain controller. As I have to create parallel environment to avoid any downtime or any communication disruption, I didn’t changed any server’s device IP, rather On all server’s I create routes using ROUTE cmd) pointing to switch gateway (SVI) so that all devices can two way communicate with each other either its public or private.


Segmentation of different VLAN’s

Slowly gradually, we changed the Server’s/users/devices VLAN form 1 to new vlan scheme, & changed the DNS entries as well (Most of them auto updated there DNS entries at DC DNS upon rebooting or using Group Policy to enforce DNS registration every 30 minutes). New vlan users were able to communicate with older vlan-1 or x users because we configured CLASSLESS for each DHCP pool pointing to Core switch gateway using there corresponding gateways (SVI’s IP’s at Core L3 Switch).

Some Common Tuning we performed resulted in increased reliability of overall network

  • Most important set root bridge priority to 1 on L3 Core switch by setting the bridge priority to 1 then set all the others  switches to higher number. This will ensure the STP is built correctly
  • Subdivide the network into multiple VLAN’s & smaller logical subnets e.g. Servers, Users, Management vlans for switch n access points, devices like printers , time machines & NVR cameras, departmental vlans etc with each having a /24 e.g. 192.168.1.0/24, 192.168.2.0/24 and so on
  • On your core switch allocate SVI’s per required vlan
  • ensure each device can talk to each other using intervlan routing [exception to those who required firewalling/security]
  • Make all trunk uplinks 802.1q trunks [doesnt requires at new cisco IOS]
  • TAG / Associate ports on all switches with the appropriate vlans
  • Ensure all ports roles are defined properly, example edge / trunk ports
  • User connected EDGE port should be in access mode, & Portfast enabled along with BPDU enabled.
  • Disable all UNUSED ports (or make default sandbox VLAN for all unused ports & assign those ports with this sandbox VLAN, to ensure that unknown devices inserted in those ports should remain in sandbox VLAN
  • Make sure your Wifi APs are not meshing or bridging
  • I added all devices in DNS entries like printers etc , and at user end, added the required devices by DNS name, so that next time if any changes are required, modify the DNS entry only & no need to reconfigure the device at user end.

Some DRY theory on ACCESS/TRUNK Mode

  • An access port can have only one VLAN configured on the interface; it can carry traffic for only one VLAN. An access port transmits packets on only one VLAN (traffic is not tagged on this type of port). Portfast feature causes a switch port to enter the spanning tree forwarding state immediately, bypassing the listening and learning states. Portfast on switch ports connected to a single workstation or server allows those devices to connect to the network immediately, instead of waiting for the port to transition from the listening and learning states to the forwarding state.
  • Edge ports are configured such that they immediately go to the forwarding state. However, this does not mean that there is no loop protection. It is assumed that edge ports will connect to end devices, and thus it is convenient for them to go directly to the forwarding state. However, someone can try to plug in a switch on such a port and can try to become the root bridge or may connect to multiple ports and create a loop. That’s where you should use BPDUGuard. On all edge ports, BPDUGuard should be enabled so that as soon as such a port receives a BPDU, it will go into err-disabled state, thus preventing an L2 loop. Now keep in mind that for RSTP, if you don’t enable BPDUGuard, and a BPDU is received on an edge port, the edge port simply loses its edge port status.

Windows / Linux Route CMD’s for OS with Static IP

At user end, all routes were distributed via windows DHCP Classless routes which worked transparently fine. But server OS with static IP I had to provide proper routes according to our network. Example we have two gateways for internet at LAN, one is SANGFOR IAM for end users, second is Router/NGFW for server ends. Some times we have to route user/server at one or other gateway.

First get the Interface number via ROUTE PRINT CMD

  • route print

(Note down the interface name, example Interface number is 8)

Now add the appropriate gateway for intervlan routing & internet routing as well

*** WINDWOS OS / ADD PERMANENT ROUTES using ROUTE command

Note: -p syntax with route cmd adds the route on permanent basis (in registry)

# Below is an example for SERVER which is on 172.16.2.x VLAN , below is for INTERVLAN Routing for local subnet’s intervlan routing . 172.16.2.1 is the CORE Switch Gateway which have SVI’s configured,

route add -p 172.16.0.0 MASK 255.255.0.0 172.16.2.1 metric 1 IF 8
route add -p 192.168.0.0 MASK 255.255.0.0 172.16.2.1 metric 2 IF 8
# Below is for Internet access , 2 Gateways with priority pointing to Sangfor & NGFW Respectively (NGFW route is added as some port forwarding is done which is routed via NGFW)
route add -p 0.0.0.0 MASK 0.0.0.0 172.16.2.2 metric 3 IF 8
route add -p 0.0.0.0 MASK 0.0.0.0 172.16.2.6 metric 4 IF 8

*** LINUX / ADD PERMANENT ROUTES in interfaces FILE [Ubuntu ver 16 or below]

  • For older version of Linux (Ubuntu ver 16 or below) I added below in interfaces file
up route add -net 172.16.0.0 netmask 255.255.0.0 gw 172.16.2.1
down route del -net 172.16.0.0 netmask 255.255.0.0 gw 172.16.2.1
up route add -net 192.168.0.0 netmask 255.255.0.0 gw 172.16.2.1
down route del -net 192.168.0.0.0 netmask 255.255.0.0 gw 172.16.2.1
  • For new version of Ubuntu ver 18 & above, I used below [sample file shown below]

cat /etc/netplan/00-installer-config.yaml

network:
ethernets:
ens160:
dhcp4: false
addresses: [192.168.0.28/24]
nameservers:
# DNS server is on different VLAN
addresses: [192.168.1.71,192.168.1.72,8.8.8.8]

# STATIC ROUTES for INTERVLAN ROUTING, GW pointing to L3 Core Switch corresponding SVI
routes:
- to: 10.11.0.0/16
via: 192.168.0.1
- to: 10.10.0.0/16
via: 192.168.0.1
# FOR INTERNET , GW POINTING TO SANGFORIAM OR NGFW
- to: default
via: 192.168.0.6
version: 2

DHCP for Different VLAN Users

 

On Domain controller (Default route pointed towards L3 Core Switch SVI) I created multiple IP pool for corresponding VLAN users. At departmental L2 switches, I added corresponding VLAN’s & with the help of IP Helper at core switch, users get IP from the Domain controller DHCP server via there corresponding vlan pool. One by One all departments moved to different Vlan’s with smaller IP pool  resulting in dramatically decline in broadcast traffic. This improve network connectivity reliability at a greater extent.

 


IP Helper not working across VLAN’s

I got stucked at one point where ‘IP helper’ was not forwarding the other vlans (like vlan 10/11/12) users dhcp pkts to domain controller dhcp residing on default vlan-1 despite all settings seems correct, I contacted few Cisco / VM experts & they remotely checked in cisco & vm config in details & yet no one  was able to solved it. At Core switch there was this “no service dhcp” set & it was the culprit , as soon as I set it to service dhcp, & Alhamodlillah! all got sorted !

One example of IP helper at Wireless (with mDNS support) VLAN interface on L3 Core Switch

interface VlanXX
description WIFI_XXX-MOBILE-VLAN
ip address 10.0.0.1 255.255.255.0
ip helper-address 192.168.0.1
service-routing mdns-sd
service-policy-query querier 60
service-policy mypermit-all IN
service-policy mypermit-all OUT

VLAN’s in VMWARE ESXI

Configuring VLAN’s in ESXI was easiest part. The only thing at switch level was to configure TRUNK at which ESXI is connected to. On esxi server, I created new network (VLAN) & tag the required vlan to desired VM guest & Done.

Allah Shuker!


Ref:

Howto Add VLAN in ESXI Server

Following is quick short notes on howto add VLAN in ESXI Server

  • Login to ESXI (I am using VCSA to manage all esxi servers).
  • Now Goto Configure
  • Networking > Virtual Switches (NEXT)
  • Add Networking (NEXT)
  • Select ‘Virtual Machine Port Group for a Standard Switch’ (NEXT)
  • Select ‘Select an existing standard switch’ & click on BROWSE & select vSWITCH0 (NEXT)
  • In next window (Connection settings) at NETWORK LABEL , type suitable name like VLAN-100 / & under VLAN ID tab, type in the required VLAN ID example 100 (NEXT)
  • Click on (FINISH).

Done. Now you can assign this vlan to your desired VM Guest in its network adapter settings.


UniFi Access Points with Multiple SSID’s & VLANS

We have many wireless access points installed at various locations & all controlled centrally by UniFi controller software application. After addition of VLAN’s we decided to remove the default vlan1 SSID, & introduce new SSID based on targeted audience like XYZ-Corporate , XYZ-Mobile , XYZ-Guest & each SSID is tagged with different VLAN’s to control the users based on SSID type. The setup was straight forward. First We create separate VLAN for access points management e.g: VLAN100 & at all access points we changed the ips with vlan100 pool & set there default gateway to vlan-100 SVI configured ta Core Switch, Also we changed there management VLAN to VLAN-100.

  • Sample of IP/MGT VLAN setting on per AP Basis

  • Adding new VLAN ID in UniFi Controller Application

To add VLAN in UniFi Controller ,
Goto SETTINGS  > NETWORKS > & create new VLAN with VLAN-ID number,

  • Adding new SSID in UniFi Controller Application

Now to ADD SSID in UniFi Controller Application& tag with NEW VLAN-ID,
Goto SETTINGS  > WiFi > Create NEW Wifi , Now enter the name of SSID like XYZ-Corporate & in Network select the desired VLAN you created in earlier step

Warning: After applying any setting that needs to be synced to all access points in the group, traffic disruption & timeout to access points may occur upto 1or 2 minutes. Therefore perform such operations Only in Off-Peak hours.


UniFi Access Point – Trunk Port Config at Cisco Switch PORT

At Cisco switch where access points ethernet cable is attached. We did below config for that port,

interface GigabitEthernet1/0/20
description UniFi_AP_Finance
switchport mode trunk

This enabled to carry all vlans including AP Mgt / Different Vlan’s based on various SSID’s [each SSID tagged with different VLAN via Unifi Controller application]


UniFi Wireless Meshing Causing Loop/Broadcast

We noticed that whenever we modify any settings which requires syncing with All AP’s, then one by one AP’s freezes packet transmission connectivity & then get back online few 15-20 seconds but few access points uplink gets there uplink connectivity with another AP wireless (Meshing). This was also creating the loop broadcast. Disabling WIRELESS MESHING in Unifi Controller application & then readopting the AP’s reduced such events.


SANGFOR VLAN

To Provide internet access via sangfor to all vlan users, we added corresponding VLAN’s in SANGFOR IAM device, Each vlan sub-interface with corresponding IP to be used as gateway at corresponding VLAN users (all via DC DHCP)

To add corresponding VLAN’s in SANGFOR,

Goto System > Network / Deployment / Settings > & under LAN interface Page add as per below

10/10.10.10.3/255.255.255.0

Whereas 10/ is the cisco VLAN number, and then the IP of this sub interface &  then the subnet

Warning: After modifying any network related settings, SANGFOR Network services will be restarted, & may cause traffic disruption, therefore perform such actions in off-peak hours.


TIPS:

Bonjour (mDNS) discovery across different VLAN’s (Wired/WiFi)

After segregated network into smaller subnets along with separate vlans for wired/wifi users/devices etc, users were not able to *cast/discover* LCD TV /  MFD Printers if both are on different vlans. (they works fine if the device & user are on same vlan/subnet).  If wired user on vlan2 tries to connect with LCD (on wifi)  on vlan3, he cannot. if they are on same vlan they can connect fine. This is because broadcast doesnt’ works across different vlan.

In UniFi Controller, there is an option of enabling mDNS which is a service provided by a Unifi router like USG, UXG Pro or UDM. but since we didn’t had any UniFi router/switch therefore UniFi mDNS didn’t had any mDNS functionality and it didn’t worked. We had to configure the service to repeat broadcasts across VLANs using Cisco switches.

The solution was to enable the mDNS ‘service discovery’ at Core L3 Switch.

At core switch where all SVI’s are configured, I allowed mDNS service on particular VLAN’s (wired/wifi) using below CMD’s & the problem got sorted out.

interface VlanXX
description WIFI_MOBILE-VLAN
ip address X.X.X.X 255.X.X.X
ip helper-address X.X.X.X
service-routing mdns-sd
service-policy-query querier 60
service-policy mypermit-all IN
service-policy mypermit-all OUT
end

I will keep updating this article as I get free time

Regards
Syed Jahanzaib

November 28, 2022

Office365 – Get users inbox/archive size report by email

Filed under: Office365 — Tags: , , — Syed Jahanzaib / Pinochio~:) @ 4:28 PM

Following powershell script can be executed/scheduled to send below information via Email

  1. Users inbox size which are greater then XX GB
  2. Users archive box size (enabled archived only)

Sample:


PowerShell Script:

# POWERSHELL script to connect with O365 vis PS commands, & get inbox/archive size & send email
# I have used my local email smtp servers, you can use o365/Gmail etc to send email
# Scripting by Syed.jahanzaib
# aacable DOT wordpress DOT com
# aacable AT hotmail DOT com
# 28-NOV-2022

#Company Name
$COMPANY = "zabbo"
#Define admin users to connect with O365 session
$ADMIN_USER = "admin@example.com"
$DATE = Get-Date -Format "MM-dd-yyyy"
$DATE_FULL = Get-Date
$PATH = "C:\Office365_zaib"
#If folder is not present, create one
If(!(test-path -PathType container $PATH))
{
New-Item -ItemType Directory -Path $PATH
}
cd $PATH

# DEFINE INBOX FETCHING WHICH ARE GREATER THEN THIS ***
$INBOX_SIZE= "10GB"

# DEFINE Output Path/File Names
$U_INB_OUTFILE = "$PATH\$COMPANY-O365-INBOX-SizeReport-$DATE.csv"
$U_ARCH_OUTFILE = "$PATH\$COMPANY-O365-ARCHIVE-SizeReport-$DATE.csv"

# DEFINE MAIL SUBJECT
$SMTP_SERVER = "mail.example.com"
$SMTP_SERVER_PORT = "25"
$SMTP_FROM = "ALERTS@EXAMPLE.COM"
$TO_1 = "SUPPORT1@example.com"
$TO_2 = "SUPPORT2@example.com"

$FOOTER = "Script runtime = $DATE_FULL Powered by example.com IT DEPT. (SJZ)"
$MAIL_SUB = "$COMPANY - O365 Weekly Custom Report @ $DATE"
$MAIL_BODY = "You can see attached CSV Files to view inbox / archive size "

######################################
### DONOT EDIT BELOW THIS LINE - SJZ #
######################################

#Connect to O-365
Connect-ExchangeOnline -UserPrincipalName $ADMIN_USER

#*** GET MAIL BOX LARGER THEN $INBOX_SIZE GB
echo "*** GET INBOX MAIL BOX LARGER THEN $INBOX_SIZE GB ..."
Get-EXOMailbox -ResultSize Unlimited | Get-EXOMailboxStatistics | Where-Object {[int64]($PSItem.TotalItemSize.Value -replace '.+\(|bytes\)') -gt "$INBOX_SIZE"} | Sort-Object TotalItemSize -Descending | Select-Object DisplayName, ItemCount, TotalItemSize | Export-Csv "$U_INB_OUTFILE" -NoTypeInformation

#*** GET ARCHIVE BOX SIZE, for those whose archive is enabled
# DEFINE VARIABLES TO fetch ACTIVE ARCHIVE box data only
$Result=@()
#Get all user mailboxes
echo "*** GET ARCHIVE MAIL BOX LARGER THEN 10 GB ..."
$mailboxes = Get-Mailbox -ResultSize Unlimited –RecipientTypeDetails UserMailbox
$totalmbx = $mailboxes.Count
$i = 0
$mailboxes | ForEach-Object {
$i++
$mbx = $_
$size = $null

Write-Progress -activity "Processing $mbx" -status "$i out of $totalmbx completed"

if ($mbx.ArchiveStatus -eq "Active"){
#Get archive mailbox statistics
$mbs = Get-MailboxStatistics $mbx.UserPrincipalName -Archive

if ($mbs.TotalItemSize -ne $null){
$size = [math]::Round(($mbs.TotalItemSize.ToString().Split('(')[1].Split(' ')[0].Replace(',','')/1MB),2)
}else{
$size = 0 }
}

$Result += New-Object -TypeName PSObject -Property $([ordered]@{
UserName = $mbx.DisplayName
UserPrincipalName = $mbx.UserPrincipalName
ArchiveStatus =$mbx.ArchiveStatus
ArchiveName =$mbx.ArchiveName
ArchiveState =$mbx.ArchiveState
ArchiveMailboxSizeInMB = $size
ArchiveWarningQuota=if ($mbx.ArchiveStatus -eq "Active") {$mbx.ArchiveWarningQuota} Else { $null}
ArchiveQuota = if ($mbx.ArchiveStatus -eq "Active") {$mbx.ArchiveQuota} Else { $null}
AutoExpandingArchiveEnabled=$mbx.AutoExpandingArchiveEnabled
})
}
#$Result | Export-CSV "$U_ARCH_OUTFILE" -NoTypeInformation -Encoding UTF8

# FINALY FEED THE RESULTS TO OUR OUTPUT FILE (VARIABLES ARE DEFINE ABOVE) (SJZ)
$Result | Where-Object { $_.ArchiveStatus -eq "Active" } | Select UserName, UserPrincipalName, ArchiveMailboxSizeInMB, ArchiveWarningQuota, ArchiveQuota | Export-CSV "$U_ARCH_OUTFILE" -NoTypeInformation -Encoding UTF8

#***SEND EMAIL with MULTI attachments (SJZ)
Send-MailMessage -BodyAsHtml -SmtpServer $SMTP_SERVER -Port $SMTP_SERVER_PORT -From $SMTP_FROM -To $TO_1,$TO_2 -Subject "$MAIL_SUB" -Body "<br /> $MAIL_SUB <br /> <br /> $MAIL_BODY <br /> <br /> $FOOTER ` " -Attachments "$U_INB_OUTFILE","$U_ARCH_OUTFILE"

Task Scheduler Setting:


Regard’s
Syed Jahanzaib

November 24, 2022

September 26, 2022

BIND – Quick Reference Notes for De Beast!



 


Some DRY theory FOR BIND DNS:

BIND can be used to run a caching DNS server or an authoritative name server, and provides features like load balancing, notify, dynamic update, split DNS, DNSSEC, IPv6, and more. Berkeley Internet Name Domain (BIND) is the most popular Domain Name System (DNS) server in use today.


MY NOTES:

Following post contains quick reference notes on howto deploy BIND DNS server as MASTER/SLAVE for LAN/WAN. This post also contains some tweaks / tips to make life little easier for my self. At some of my clients, I deployed BIND (at some placed UNBOUND) & created few custom commands menu so that client uses the webmin gui module of webmin to modify records , service reload , service restart, dns test etc. Webmin panel example for operator. Webmin do contain full module to manage BIND, but to get the minimum to the point panel, I used some commands & bash script & tagged it with webmin menu.


Scenario:

We have a public domain name example.com & we want to host our own authoritative public dns server so that dependency on host provider reduced & we can manage all types of records at our site. All Internal clients can use our DNS to resolve dns related queries & other hosts on  internet can resolve our domain related hosts via our DNS server. In this scenario our clients will use our dns as primary for resolving, and other internet zones resolving will be forwarded to other public DNS.

Assumptions before proceeding …

  • This post contains info for BIND with ipv4 Only
  • Domain Name: example.com
  • We will have two servers which will be designated as our DNS name servers. This guide will refer to these as ns1 and ns2
  • We will have two additional client pc’s records that will be using the DNS infrastructure you create, referred to as
  • We will configure MASTER/SLAVE configuration so that in case primary goes down, secondary will still be available to fulfill resolving.
    Primary DNS: ns1.example.com – 10.0.0.1
    Secondary DNS : ns1.example.com – 10.0.0.2
  • Using your hosting provider CPANEL, goto DNS management section & change the default nameservers pointing to your local DNS (local DNs must have static public ip address & must be accessible/allowed via any firewall u might have), example below
  • For Reverse PTR records, use APNIC panel, & configure Reverse DNS Delegation, example below
  • Every time you modify zone records, make sure to reload the bind9 service or use the bash script which can change the records as per date/seconds & reload bind service which will trigger SLAVE server to replicate from Master.

When configured as a recursive server, BIND will first use the zones for which it is authoritative before walking down the entire DNS tree from the root/forwarder. for other ZONES , we have configured forwarders, (and root servers can be used in absence of forwarder). We can even create multiple zones, the only limitation is that you will need to maintain all the records in zones you create and ensure they are up to date.

Once we configure & test our DNS, next step is to change NS records on your hosting provider cpanel so that internet can resolve your domain name using your hosted DNS server.


Housekeeping Stuff !

Before installing Bind, we need to perform some housekeeping Stuff first,

NTP:

Make sure you have correct date time syncing with any NTP server with appropriate Time zone as well.

apt-get -y install ntp ntpdate
# Following is for KARCAHI/PK zone, u may select yours accordingly
cp /usr/share/zoneinfo/Asia/Karachi /etc/localtime

BIND installation on Ubuntu 16.4:

apt-get update
apt-get install -y bind9 bind9utils bind9-doc dnsutils

Force IPV4 Only

  • OS Level:

Copy Paste below to disable ipv6

echo "net.ipv6.conf.all.disable_ipv6 = 1" >> /etc/sysctl.conf
echo "net.ipv6.conf.default.disable_ipv6 = 1" >> /etc/sysctl.conf
echo "net.ipv6.conf.lo.disable_ipv6 = 1" >> /etc/sysctl.conf
sysctl -p
  • BIND Level ipv4 config:
nano /etc/default/bind9

Example:

# run resolvconf?
RESOLVCONF=no
# startup options for the server
OPTIONS="-4 -u bind"
  • RESOLV.CONF:

Once BIND is installed, you may want to modify the resolv.conf to look something like below
(For permanent changes, edit  appropriate settings in /etc/network/interfaces file)

search example.com # example.com is your domain name
nameserver 10.0.0.1 # your local host ip

** Configuration files for BIND9 – MASTER **

Default installation location folder is /etc/bind9. We will create subfolder named ZONES to hold all zones files separately. Also we will add forwarder/reverse records files location so that BIND should know where our ZONES files are located.

BIND9 DNS Zones Location & Master/Slave Config File


  • named.conf.local [It will contain ZONE Files & its parameters]

Edit below file to enter zone name, type, file location etc

nano /etc/bind/named.conf.local
//The following code defines the forwarder lookup zone.
zone "example.com" {
type master;
allow-update { none; }; //Since this is the primary DNS, it should be none.
allow-transfer { 10.0.0.2; }; //Allow Transfer of zone from the master server
also-notify { 10.0.0.2; }; //Notify slave for zone changes

file "/etc/bind/zones/forwarder.example.com";
};

//The following code defines the reverse lookup zone.
zone "0.0.10.in-addr.arpa" {
type master;
allow-update { none; }; //Since this is the primary DNS, it should be none.
allow-transfer { 10.0.0.2; }; //Allow Zone Transfer from master server to ns2 slave server
also-notify { 10.0.0.2; }; //Notify slave for zone changes

file "/etc/bind/zones/reverse.example.com";
};

  • named.conf.options [It will contain ACL , Forwarders & related parameters]
nano /etc/bind/named.conf.options
# First we will Define IP pool ACL so that only these ips will be allowed to resolve internal/exteral dns resolving
acl "trusted" {
127.0.0.0/8;
10.0.0.0/8;
# or any other pool/x;
};
options {
directory "/var/cache/bind";
recursion yes; # enables recursive queries
allow-recursion { trusted; }; # allows recursive queries from "trusted" clients ONLY, external clients will be able to resolve records related to example.com only, internal clients mentioned in ACL can query all records lan/wan
recursive-clients 10000; # Fine tuning option for server with High number of users
tcp-clients 1000; # Fine tuning option for server with High number of users
version "syed.jahanzaib";
listen-on { 10.0.0.1; }; # ns1 private IP address - listen on private network only, ns1 local ip
listen-on { 127.0.0.1; }; # ns1 private IP address - listen on private network only
allow-transfer { 10.0.0.2; }; # Allow Transfer of zone to secondary dns ns2.example.com
# Forward all other zones queries to following public dns (Only Trusted ACL will be able to resolve external queries
# This can help to make the responses to these queries faster by reducing the load on the local network.
forwarders {
1.1.1.1;
8.8.8.8;
};
dnssec-validation auto;
auth-nxdomain no; # conform to RFC1035
listen-on-v6 { any; };
};

  • ZONE FILES Location and FORWARDER Configuration File:

It will contain zone details , all dns related records

First Create a folder named ZONES which will contain our zones files (forwarders/reverse)

mkdir /etc/bind/zones
### Now create a new file for forwarders records ###
nano /etc/bind/zones/forwarder.example.com

FORWARDER Configuration File:

; BIND FORWARDER A RECORDS
; example.com- LOCLA HOSTED DNS FOR LAN/WAN (RESOLVER/RECURSIVE/CACHING/AUTHORITATIVE)
; 24-SEPTEMBER-202
; BIND DNS Configuration - by Syed Jahanzaib
; aacableAThotmailDOTcom / https://aacableDOTwordpressDOTcom
$ORIGIN example.com.
$TTL 86400
@ IN SOA ns1.example.com. zaib.example.com. (
2022101905 ; Serial
10800 ; REFRESH
3600 ; RETRY
1209600 ; EXPIRE
10800 ; MINIMUM
)
; NS RECORDS
@ IN NS ns1.example.com.
@ IN NS ns2.example.com.
; A RECORDS
@ IN A 10.0.0.1
ns1 IN A 10.0.0.1
ns2 IN A 10.0.0.2
mail IN A 10.0.0.50
; CNAME RECORDS
www IN CNAME example.com.
; MX RECORDS
@ IN MX 10 mail.example.com.

Save & Exit.


REVERSE Configuration File:

It will contain reverse PTR records & also $GENERATE statement for auto generation of unused ips ptr records for 

nano /etc/bind/zones/reverse.example.com

Example File:

; BIND REVERSE PTR RECORDS
; example.com - LOCAl HOSTED DNS FOR LAN/WAN (RESOLVER/RECURSIVE/CACHING/AUTHORITATIVE)
; 24-SEPTEMBER-202
; BIND DNS Configuration - by Syed Jahanzaib
; aacableDOThotmail.com / https://aacableDOTwordpressDOTcom</pre>
$TTL 43200
@ IN SOA example.com. ns1 (
2022101905 ; Serial
14400 ;
1800 ;
1209600 ;
3600 ;
)

@ IN NS ns1.example.com.
1 IN PTR ns1.example.com.
2 IN PTR ns2.example.com.
50 IN PTR mail.example.com.
;generate auto ptr for few hosts as defined below
$GENERATE 150-255 $ IN PTR 10.0.0-$.example.com.
<pre>

Save & Exit.


BIND ZONE FILES CHECK CMD’s

Every time you edit the zone files, its better to run a checkzone cmd to ensure there are no syntax mistakes.

  • a) Main File Syntax Check
named-checkconf

If your named configuration files have no syntax errors, there won’t be any error messages and you will return to your shell prompt. If there are problems with your configuration files, review the error message , troubleshoot & fix it, then try named-checkconf again

  • b) Zone File Syntax Check
named-checkzone example.com /etc/bind/zones/forwarder.example.com

named-checkzone example.com /etc/bind/zones/reverse.example.com

** Configuration files for BIND9 – SLAVE **

Configuring SLAVE is quite simple. Install the BIND on 2nd server (ns2.example.com). set its resolv.conf to its own ip. Edit below files

NOTE: You have to define all Forwarder/Reverse ZONES defined in MASTER ns1 server in this config

nano /etc/bind/named.conf.local

Example File:

//The following code defines the forwarder lookup zone.
zone "example.com" {
type slave;
masters { 10.0.0.1; };
masterfile-format text;
file "/var/cache/bind/forwarder.example.com";
serial-update-method unixtime;
};

//The following code defines the reverse lookup zone.
zone "0.0.10.in-addr.arpa" {
type slave;
masters { 10.0.0.1; };
masterfile-format text;
file "/var/cache/bind/reverse.example.com";
serial-update-method unixtime;
};

Save & Exit.

Now edit the /etc/bind/named.conf.options & copy paste all data from master ns1 named.conf.options file. You have to modify just one parameter listen-on to point it to ns2 ip example

  • listen-on { 10.0.0.2; };

Save & Exit. & restart BIND

service bind9 restart

now look at syslog file to check for any errors

tail -f /var/log/syslog

Sep 26 06:48:06 ns2 named[8496]: client 10.0.0.1#55064: received notify for zone 'example.com'
Sep 26 06:48:06 ns2 named[8496]: zone example.com/IN: notify from 10.0.0.1#55064: zone is up to date
Sep 26 06:48:07 ns2 named[8496]: client 10.0.0.1#44867: received notify for zone '0.0.0.10.in-addr.arpa'
Sep 26 06:48:07 ns2 named[8496]: zone 0.0.0.10.in-addr.arpa/IN: notify from 10.0.0.1#44867: zone is up to date


BIND Service Related CMD’s

*** Find BIND version CMD
named -v
*** BIND Service related CMD's
service bind9 stop
service bind9 start
service bind9 restart
service bind9 status
*** BIND Service Internal STATS
rndc status
RNDC Result Example: version: BIND 9.10.3-P4-Ubuntu <id:ebd72b3> (syed.jahanzaib) boot time: Wed, 19 Oct 2022 10:18:47 GMT last configured: Wed, 19 Oct 2022 10:18:47 GMT CPUs found: 16 worker threads: 16 UDP listeners per interface: 8 number of zones: 115 debug level: 0 xfers running: 0 xfers deferred: 0 soa queries in progress: 0 query logging is OFF recursive clients: 28/9900/10000 tcp clients: 14/1000 server is up and running [/sourcecode]

NSLOOKUP Test: (Windows)

nslookup -query=any example.com
nslookup -query=ns example.com
nslookup -query=soa example.com
nslookup -query=mx example.com
nslookup -10.0.0.1 example.com
nslookup -10.0.0.1 yahoo.com

DIG Test: (Linux)

Perform nslookup/dig on dns server

dig @127.0.0.1 ns1.example.com
dig @127.0.0.1 ns2.example.com
dig -x 10.0.0.1
dig example.com @10.0.0.1 | grep -e "^host" -e ";; flags"
dig +norec example.com @10.0.0.1 | grep -e "^host" -e ";; flags"
dig +norec example.com @10.0.0.1 | sed -n -e '/;; flags/p' -e '/^;; AUTH/,/^$/p'
dig +noall +answer SOA +multi example.com

TIPS:


  • SERIAL of SOA

The serial number of the SOA record is very important to the correct operation of slave servers. Each time you edit your zone, the serial number must be increased so that the slave server can detect that the zone has been updated. Whenever the slave server notices that the serial number has changed, it performs a zone transfer and updates its cached zone file.

Every time, you make any changes in MASTER NS zone file, Ensure following

  • Change Serial Number , else it will not be replicated to Slave.
  • Reload bind by
service bind9 reload

  • Auto Generation of PTR Records pointing to fixed generated IP-Hostname

If we want to reply the reverse PTR records automatically for our unused IP’s, Edit the REVERSE zone & add following line in the end

$GENERATE 1-255 $ IN PTR 10-11-11-$.example.com.

Save & Exit, & Reload BIND9 service by

service bind9 reload

now if you do nslookup for any unused ip/record from your user pool, you will get results as below

#10.0.0.1 is our NS server, and 10.0.0.50,100 are host whose record is not manually added,

C:\>nslookup 10.0.0.50 10.0.0.1
Server: ns1.example.com
Address: 10.0.0.1

Name: 101-11-11-50.example.com
Address: 10.0.0.50

C:\>nslookup 10.0.0.100 10.0.0.1
Server: ns1.example.com
Address: 10.0.0.1

Name: 101-11-11-100.example.com
Address: 10.0.0.100

  • BASH Script to change ZONE SERIAL number with YYMMDDnn format 

If you have Master/Slave Setup, then you need to update the serial number in zones every time you make any modifications, else it will not replicate to SLAVE. A bash script is handy to run when you update any record, it will update serial to YYMMDDSS (SS=sequence Serial) number, and will increment the SS every time u run it. so total of 99 changes u can make in single date.

Old Serial Number can be any number but the script will change it to current date like YYYYMMDDSS, example shown afterwards

  • /temp/reload.sh

 

  • NOTE: This bash script looks in /etc/bind/zones, and will search text ‘Serial’, So make sure your zone file contains ; Serial word in zone file. Example below
$TTL 86400 ; (1 day)
; $ORIGIN example.com
@ IN SOA ns1.example.com. admin (
2022092623 ; Serial
14400 ; refresh (4 hours)
1800 ; retry (30 minutes)
1209600 ; expire (2 weeks)
3600 ; minimum (1 hour)
)

Bash Script named /temp/reload.sh

#!/bin/bash
# This path contains all ZONES files, this is folder created specifically to hold ZONES files
ZONES_PATH="/etc/bind/zones"
DATE=$(date +%Y%m%d)
# we're searching for line containing this comment
NEEDLE="Serial"
for ZONE in $(ls -1 $ZONES_PATH) ; do
curr=$(/bin/grep -e "${NEEDLE}$" $ZONES_PATH/${ZONE} | /bin/sed -n "s/^\s*\([0-9]*\)\s*;\s*${NEEDLE}\s*/\1/p")
# replace if current date is shorter (possibly using different format)
if [ ${#curr} -lt ${#DATE} ]; then
serial="${DATE}00"
else
prefix=${curr::-2}
if [ "$DATE" -eq "$prefix" ]; then # same day
num=${curr: -2} # last two digits from serial number
num=$((10#$num + 1)) # force decimal representation, increment
serial="${DATE}$(printf '%02d' $num )" # format for 2 digits
else
serial="${DATE}00" # just update date
fi
fi
/bin/sed -i -e "s/^\(\s*\)[0-9]\{0,\}\(\s*;\s*${NEEDLE}\)$/\1${serial}\2/" ${ZONES_PATH}/${ZONE}
echo "${ZONE}:"
grep "; ${NEEDLE}$" $ZONES_PATH/${ZONE}
done
# In the END, reload and show status of BIND service
service bind9 reload
service bind9 status | grep running
service bind9 status

Result: After modifying any record, run this script. every time you run the script, it will just increment one number in last 2 digits, (it can be max of 99 till the SOA expires, or single day)

root@ns1:/etc/bind/zones# /temp/test.sh
forwarder.example.com:
2022092604 ; Serial
reverse.example.com:
2022092604 ; Serial

root@ns1:/etc/bind/zones# /temp/test.sh
forwarder.example.com:
2022092605 ; Serial
reverse.example.com:
2022092605 ; Serial

I have tagged this file with webmin, so that when user edit the record file and hit save and close, this file runs and do the job, and all data gets replicate to the SLAVE automatically. Also you schedule it to hourly or as per requirements.


  • SLAVE ZONE FILES RAW FORMAT

With BIND 9.9.x, the slave zone files are now saved in a default raw binary format. This was done to improve performance, but at the sacrifice of being able to easily view the contents of the files. it can  also make debugging at SLAVE level more difficult.  If you want to see the slave zone files in plain text, just simple add following in the the named.conf files for your slave zones to include the line: (This has to be done at MASTER/NS1 dns)

masterfile-format text;
  • REDUCE LOGGING FOR BIND9

BIND uses syslogd before a valid logging clause is available so named.conf parse errors and other information will appear in /var/log/syslog. To separate log from SYSLOG & record them in separate file & fix the log size, you can add following in /etc/bind/named.conf.option at end,

logging{
channel bind9_zaib_log {
file "/var/lib/bind/bind.log" versions 3 size 5m;
severity info;
print-time yes;
print-severity yes;
print-category yes;
};
category default{
bind9_zaib_log;
};
};

now reload BIND9 service

service bind9 reload

check the file contents

tail -f /var/lib/bind/bind.log

Once the file size will cross 5 MB, it will reset to 0 auto and will continue to grow till 5 & reset again 🙂 as shown in below image …

root@ns1:/etc/bind# ls -lh /var/lib/bind/bind.log
-rw-r--r-- 1 bind bind 4.2M Sep 27 11:29 /var/lib/bind/bind.log
root@ns1:/etc/bind# ls -lh /var/lib/bind/bind.log
-rw-r--r-- 1 bind bind 4.2M Sep 27 11:29 /var/lib/bind/bind.log
root@ns1:/etc/bind# ls -lh /var/lib/bind/bind.log
-rw-r--r-- 1 bind bind 4.2M Sep 27 11:30 /var/lib/bind/bind.log
root@ns1:/etc/bind# ls -lh /var/lib/bind/bind.log
-rw-r--r-- 1 bind bind 4.3M Sep 27 11:30 /var/lib/bind/bind.log
root@ns1:/etc/bind# ls -lh /var/lib/bind/bind.log
-rw-r--r-- 1 bind bind 4.3M Sep 27 11:30 /var/lib/bind/bind.log
root@ns1:/etc/bind# ls -lh /var/lib/bind/bind.log
-rw-r--r-- 1 bind bind 13K Sep 27 11:31 /var/lib/bind/bind.log

For more customized info, Please read below …

Configuring Bind9 logs


Different Types of DNS Records With Syntax and Examples

Types of DNS Records

A
AAAA
CNAME
MX
PTR
NS
SOA
SRV
TXT
NAPTR

The above DNS records are mostly used in all DNS Configurations.

A Record Example

Address Record, assigns an IP address to a host, domain or subdomain name

ns1 IN A 10.0.0.1

PTR Record Example

A PTR record or pointer record maps an IPv4 address to the canonical name for that host. This is mostly used as a security and an anti-spam measure wherein most of the webservers or the email servers do a reverse DNS lookup to check if the host is actually coming from where it claims to come from. It is always advisable to have a proper reverse DNS record (PTR) is been setup for your servers especially when you are running a mail / smtp server.

255 IN PTR ns1.example.com.

SPF Record Example

The Sender Policy Framework (SPF) is an email-authentication technique which is used to prevent spammers from sending messages on behalf of your domain. With SPF an organization can publish authorized mail servers. Together with the DMARC related information, this gives the receiver (or receiving systems) information on how trustworthy the origin of an email is. SPF is, just like DMARC, an email authentication technique that uses DNS (Domain Name Service). This gives you, as an email sender, the ability to specify which email servers are permitted to send email on behalf of your domain.

v=spf1 +a +mx +ip4:1.2.3.4 +include:mail.example.com -all

DMARC Record Example

_dmarc.example.com. 14400 TXT "v=DMARC1; p=quarantine; sp=none; rf=afrf; pct=100; ri=86400"

DKIM (MultiLine) Record Example

default._domainkey.example.com. 14400 TXT ( "v=DKIM1; k=rsa; p=GIIBgjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAzEi2OVswNjDwG57Rc14vKSAZNizQpO/KRG96H2N3dc1jnEMi0GCXCIFlFnrQffEVA9RWQ4u6pFjhaQ4s/Uony61CmPwls/O5p/IMdKbmkb0ULrdtwkpoW5Ve0F4C777YXBlXk0aTP2sEenX5e4ou8IGog0zTyq4E5v6DY+juNCKE8yktvM2oJvso/mqS2BJc5"
"X+Xvrhs+l3/qqlLEIwIfA4ep2QBXEOIgElBiXptXSwt6ym6ZmBdOl/eeZipulBZyC1onGFLwR5qvRNo/Q0e1c/H9eBrOyOmIJ65OPy8AtT1Ln3emKy9JAdAVaODHTt1jRbK2X8j3t/cAWX7Fntr1QIDAQAB;" )

August 10, 2022

CallMeBot: Sending Alerts to Various Messaging Apps using APi

Filed under: Linux Related, Mikrotik Related — Tags: , , , , , — Syed Jahanzaib / Pinochio~:) @ 11:00 AM


At multiple ISP’s or networks, I use either GSM Modems or 3rd party SMS API to send various information / alerts messages to admin or users , & it involves minor cost on a per sms basis or as bundle. Another method to send messages on WHATSAPP & its FREE , very simple to use CALLMEBOT APi’s. Its a online FREE service via which you can send basic messages to multiple messaging services like WhatsApp / messenger / Signal / Telegram apps.  The only caveat is that you cannot send messages to EVERY contact. The receiver contact must have the callmebot API KEY , which he can easily get via adding callmebot number in there contact list & send particular message & he will get it instantly. so EACH API EKY is tagged with that particular mobile number only. Once you know the key & its tagged mobile number, you can send message to that contact using the API_KEY & his mobile number using various methods.

In this guide we will be using callmebot API to send various test message to admin WhatsAPP number. Follow the below steps …

  • First, You need to get the API key form the CallMeBot. Add the phone number +34 644 91 96 80 into your Phone Contacts. (Name it it as you wish like CallMeBoT / My_Alerts etc )
  • Using WhatsApp, Send this message to this contact

I allow callmebot to send me messages

  • once it is sent, Wait until you receive the message like “API Activated for your phone number. Your APIKEY is XXXXXX” from the bot. Note down the APIKEY
    Note: If you don’t receive the ApiKey in 2 minutes, please try again after 24hs.
  • The WhatsApp message from the CallMeBot will contain the API key needed to send messages using various API.
  • Now You can send text messages using the API after receiving the confirmation.

Following are few methods, which I am using to send various informational/warning/critical messages to Admin whatsapp


Send message using Browser HTTP APi:

Using browser, you can send message by using below

(As always, change the Phone/APiKey)

https://api.callmebot.com/whatsapp.php?phone=+923333021909&apikey=123123&text=Fiber+Link+is+Down

Example Message:


Linux Examples:

CMD: on Terminal/shell

(As always, change the Phone/APiKey)

curl --insecure 'https://api.callmebot.com/whatsapp.php?phone=+923333021909&apikey=123123&text=Linux+Msg+Test+z'

Example Message:


Linux Bash Script Example using separate Text file as attachment

(As always, change the Phone/APiKey)

#!/bin/bash
# DAILY SMS SCRIPT FOR whatsapp MSG Testing
PATH=/opt/someApp/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
# Script by Syed Jahanzaib / aacable@hotmail.com
# https://aacable.wordpress.com
# Version 2.0 -
# Created in year 2013
# Last modified 20-NOV-2017
set -x
logger DAILY 8 am sms executed for whatsapp testing
#ntpdate -u 91.189.91.157
COMPANY="Independence-Day"
TMP="/tmp/dailysms.txt"
> $TMP
# CHAGNE BELOW API KEY AS PER YOUR'S
API_KEY=123123
# Jahanzaib Cell # CHANGE CELL AS PER YOUR's
CELL1="923333021909"
UPTIME=`uptime | awk -F'( |,|:)+' '{if ($7=="min") m=$6; else {if ($7~/^day/) {d=$6;h=$8;m=$9} else {h=$6;m=$7}}} {print d+0,"days,",h+0,"hours,",m+0,"minutes"}'`
DATE=`date`
FOOTER="Powered By Jz"
# Print total and each vlan users
MSG="$COMPANY - Msg Test !
Daily Msg Test @
UPTIME = $UPTIME
$DATE
$FOOTER"
echo "$MSG" > $TMP
cat $TMP

# Finally SEND MSG to whatsapp using CALLMEBOT
curl --insecure "https://api.callmebot.com/whatsapp.php?phone=+$CELL1&apikey=$API_KEY" -G --data-urlencode text@$TMP

rm $TMP
# THE END

Example Message:


Mikrotik RouterOS Example:

Terminal:

(As always, change the Phone/APiKey)

/tool fetch http-method=get mode=https url="https://api.callmebot.com/whatsapp.php\?&apikey=123123&phone=+923333021909&text=Mikrotik+Router+Whatsapp+Msg+Example+SJZ"

Example Message:


Mikrotik DUDE (for Windows) Example:

First download the windows base WGET utility, copy it in any local folder of dude server. & now add notification like this

On DUDE App, Goto Notifications, Click on + ICON to add new notification, a new popup will appear, fill it as per below

Name: Whatsapp2Zaib
Enabled: Ticked
Type: Execute on Server
& in Command Window section, use below cmd

(As always, change the Phone/APiKey)

C:\wget\wget.exe --no-check-certificate "https://api.callmebot.com/whatsapp.php?phone=+923333021909&apikey=123123&text=Service [Probe.Name] on [Device.Name] is now [Service.Status] ([Service.ProblemDescription])"

Example Message:


Hope it Helps !



Regard’s
Syed Jahanzaib

July 26, 2022

Veeam B&R: Virtual disk size is not a multiple of 1KB

Filed under: Veeam B&R — Tags: , — Syed Jahanzaib / Pinochio~:) @ 11:45 AM

Veeam 1KB Disk Error Solved

We are using Veeam B&R ver 9.x application to backup some esxi guests. From past few days we were getting error at one guest (Windows 2008 R2, converted from P2V (using vmware converter) & it’s backup worked well with veeam for long without issue, but after increasing its disk space, it was giving trouble with following errors.

********************************************************************************************
7/18/2022 5:11:33 PM :: Error: Virtual disk size is not a multiple of 1KB.
********************************************************************************************

I tried the veeam knowledge base mentioned here, changed disk size and recalculated, but still getting errors.

This VM have 2 drives assigned, 1st contains OS (C: drive), 2nd drive (G: Drive) contains Oracle DB DATA. To further narrow down the troubleshooting, I included only drive 1 (OS) in the Veeam backup job, and it worked fine. So it was confirmed that the culprit was drive 2 & it was important drive because it contains oracle database folders (ORACLE & USR).

Workaround:

I tried all solutions but couldn’t figured out. As a last resort I did following. I know this workaround is not the proper solution, but since I failed to sort it using my limited knowledge & get the VM backup was crucial, therefore I did following & it worked for me. Luckily I had enough resources to cater the workaround. 6 drivers SSD in Raid-5 made the transition really fast at about 6-7 Gbps.

STEPS:

  • I Stopped all running services on the VM guest related to Oracle/SAP or any un-necessary apps
  • Added new disk (drive3 / H: ) on the effected VM with same size as existing DB drive (disk2 G: )
  • Cloned the partition from old (disk2 G: ) to this new (disk3 H: ) [I used 3rd party DiskGeniusPro tool for cloning by locking partition access, but u can use Robocopy or other tools as well)
  • Using Disk Management, I changed the drive letter & marked the old (disk2 G: ) OFFLINE.
  • Changed the new disk (disk3 H: ) partition drive letter same as older one that is  G:
  • Rebooted the server, & this time all backups went smoothly & SAP ORACLE synching/services are working fine as well
  • Afterwards I removed the OLD faulty partition from the VM.

Alhamdolillah!


Another workaround I got from a virtualization expert member at spicework community support named  Supaplex was to

A slightly simpler option would be using a P2V approach inside the virtual machine using a free tool like V2V Converter https://www.starwindsoftware.com/starwind-v2v-converter and specifying your faulty G: disk as source and ESXi as the target. The tool would convert the disk into a fresh new VMDK file automatically, and all you would have to do is just attach it to the virtual machine and remove the old one. Hope that may save you some time in the future if you encounter such a problem again.

Thank you


Regard’s
Syed Jahanzaib

June 18, 2022

VMWARE – VCSA 7.0 Migration/Cloning

Filed under: VMware Related — Tags: , — Syed Jahanzaib / Pinochio~:) @ 10:30 PM

Moving VCSA 7.0.0 (Build 16189207) from one ESXI host to other ESXI host
(without VMOTION)

 

Recently we added one new esxi server (server02) with some beefy hardware specs including enterprise ssd storage & wanted to move old vcenter hosted to this new esxi host to get benefits of good speed.

Since we donot have shared storage, therefore I used the CLONE method which worked surprisingly accurate without any hurdle.

  • Login to your Vcsa admin panel (https://your-vcsa-ip)
  • Select the vCenter Server virtual machine from the Inventory (source esxi server01 where VCSA is currently installed)
  • Right-click the VCSA virtual machine and click Clone
  • Select the Destination ESXi host server02 (in my case it was new ssd base esxi server), & follow on the instructions

Once the Clone process is done

  • Power off vCenter Server on the source host (server01 old server)
  • Power on the vCenter Server virtual machine on the destination ESXi host (server02 – new esxi server)

VCSA takes some time to start all of its services, Be patience ! (Follow this guide for VCSA Troubleshooting)

Once done, open the VCSA admin panel, & Login.

Alhamdolillah, in my case it was quite straight forward process! your mileage may vary 😀


Backup Note:

Backup is your friend 🙂 Make sure to take regular backups of VCSA using its management panel that is https://your-vcsa-ip:5480/

We take its backup vis windows shared folder using SMB option. you can  use SMB/FTP or other methods of your choice.

When you will click on BACKUP NOW, it will ask you to enter all the details like smb/ftp etc location etc, you need to configure it first !


Regards
Syed Jahanzaib

May 18, 2022

Lenovo SR650 Corrupt GPT & ESXi install failure

Filed under: IBM Related — Tags: , , , , , , , — Syed Jahanzaib / Pinochio~:) @ 12:40 PM

Recently one of our Lenovo SR650’s disk got faulty. It had 14 x 1.2 TB 10k SAS disks. As a long term solution & to avoid any urgency, we decided to remove 2 disks (faulty one for replacement & one for cold spare backup to be used by same or other similar servers).

Once we re-created new Raid.-10 & rebooted the server , boot screen was showing below error

We tried to follow the Lenovo Note which instructed to go into Setup->System Settings->Recovery->Disk GPT Recovery and set to “Automatic.” but still the error didn’t sorted. To settle it on Server Bios level, we performed following steps

  • Update SR650 UEFI Firmware ( Lenovo Download Link )
  • Removed Raid Config, Re-create Raid Config with Full Initialization
  • Full power cycle the server once above is done.

This sorted the Bios screen error regarding GPT.

But once we started the Vmware ESXI 6.5.x installation , it was failing (between 5% and 8%) with the following error …

“partedUtil failed with message: Error: The primary GPT table states that the backup GPT is located beyond the end of disk. This may happen if the disk has shrunk or partition table is corrupted. … Error: Can’t have a partition outside the disk!  BLAH BLAH BLAH …”


Solution # 1

Boot with any windows ISO ( Must have the RAID controller driver or the OS should have in-built drivers, in my case, windows server 2019 had the raid drivers). You can also use Linux base Boot OS .

Re-create the partition , Format & Booom. Afterwards just boot from ESXI ISO/CD/USB/Networkboot , and the ESXI will install fine.


Solution # 2 (Quick & Recommended for admins)

During ESXi installer at anywhere ,

Press Alt-F1 (which will bring you to shell window asking for credentials)

Use following credentials

  • ID: root
  • Password: No password. Just press enter & you can use the CMD’s to sort the issue

Issue the below CMD which will show you list of disk device names that can be managed by partedUtil

ls -ltrh /vmfs/devices/disks

** Note the disk ‘identifier’ that we want to fix. In my case it was 6.5 TB partition in which we wanted to install the esxi.

Now issue the below cmd

partedUtil mklabel /dev/disks/naa.600062b2031e00402a165add7ff9c3ac msdos

This overwrited the brooked partition table. Now return to the installer screen and continue.

This time, esxi installation went fine without errors.


Regard’s
Syed Jahanzaib

Older Posts »