Active Directory Backup/Restore , Troubleshooting – Short Notes

Ok so far I have experienced following,

• using Microsoft built-in tools, I tried a lot but failed to restore Active Directory Systemstate only backup to dissimilar hardware, Windows doesn’t boot after restoration. (in the past when we had single DC and some components failure occurred & as a last resort Systemstate backup restoration worked , but on the same DC)
• BAREMETAL WORKS ! If we have Full Baremetal type backup via wbadmin/server backup, then we can do bare metal recovery of full system to OTHER dissimilar hardware machine , either via USB/Network share, and Active directory worked fine with all functions. I tested it few times via USB & network, both worked perfect & all AD components / functions working fine along with test user logging successfull with GP applied as well.
• BAREMETAL: At one restoration lab of Baremetal restoration to different vm , I copied the last backup set to ISO image and added it to the new vm guest , but upon starting the restore process I always get the “The System Image Restore Failed. The Parameter Is Incorrect. (0x80070057)“. To sort this. I added second VHD to another system, copied the last backup set in it, and then copied this VHD to the targeted VM where DC needs to be restored & attached it to the new VM. Afterwards the restore process worked well using the default GUI options.
• BAREMETAL: When you attach the USB/ISO/VHD which contains the windowsimage backup set, make sure that you first goto prompt, and using disk part (volumes) make sure that this volume is not the C: letter, change drive letters as required
• Restoration via backup set should be the last option, It’s better to plan multiple Dc’s deployment at various location & if any DC goes down, redo OS installation and promote it as DC via other DC’s replication.

---

1# Backup Active Directory – System State in windows 2008/2016

Following is an small guide for ‘Backup and restore system state in windows 2008/2016‘

# BACKUP

• You cannot store backup on same partition like C: / You must backup the AD in second partition or other drive. (It must be local drive, or network shared folder, but not USB)
• First install the backup server features from the Server Manager or via CLI like Open CMD, type powershell​ and enter
  Import Server-Manager
  Add-WindowsFeature Windows-Server-Backup
• After this Open command prompt and type

wbadmin start systemstatebackup -backuptarget:d:

and press enter. It will ask for confirmation, Type Y to continue & it will start backup your current AD in D: drive

Note: You can use a different backup target of your choosing, it must be a local drive of your server.

When the backup finishes running, you should get a message that the backup completed successfully. Goto your backup drive and you will find folder name WindowsImageBackup with backup data.

---

2# RESTORE Active Directory in Windows 2016 server (Authoritative Mode) on same machine

Scenario: We have single Domain controller running , daily Systemstate backup is done by WBADMIN utility on D: drive. Accidentally components of AD have been damaged or multiple items from the AD have been deleted by mistake and we are unable to restore them. Since the windows was running OK & only the few AD parts were having trouble therefore we decided to restore via Systemstate backup,

To restore systemstate backup on Same machine, follow below

• Boot in DSRM mode,
• Open command prompt , Get backup version number so that you may restore correct version of backup, use the following command to get the version number

wbadmin get versions -backupTarget:D:

# Make sure your backups are in folder structure  like d:\windowsimagebackup\servername – and so on

• Write down the version identifier you need to use.
• To restore AD in AUTHORITATIVE mode, use the following command

wbadmin start systemstaterecovery -backuptarget:d: -version:04/04/2013-15:00 –authsysvol

 

Note:

1. Change the -version: to match your backup number that noted from wbadmin get version command
2. Change -backuptarget:d: to match the partition where backup folders are residing.
3. – To restore AD in non-authoritative mode, remove the –authsysvol syntax at the end of the command

Once the restore is completed, system will reboot automatically. Once booted, you will see all your AD settings restored.

The NETLOGON share is not present after you install Active Directory

After system state restore , if you get error , You may want to tweak the registry,

Read this

https://support.microsoft.com/en-us/help/947022/the-netlogon-share-is-not-present-after-you-install-active-directory-d

And this one too

http://www.virtuallyimpossible.co.uk/when-good-domain-controllers-go-bad/

---

3# Bare Metal Recovery via ‘WBADMIN’ to different hardware using “FULL BACKUP recovery set”

Updated: 28-NOV-2018

Scenario:

We have 2 Domain controllers & both hardware machines have gone faulty. SystemState & FULL backup is being done by WBADMIN utility on a daily basis , & we have the FULL backup set available in external USB (F:\WindowsImageBackup\DC01)

Now we need to restore Active Directory in a new machine. I performed this DR in my LAB & was able to fully recovery the Active Directory to new machine. Previous DC FULL Backup is available in USB (F:\WindowsImageBackup\DC01)  which is attached to this new system.

• Boot from Server 2016 GUI DVD (or USB)
• On first welcome screen, Select NEXT
• On Second Screen, select Repair Your Computer
• on Choose an option screen, select Troubleshoot,
• On Advanced Option screen, select Command Prompt
• On CMD screen, first check what USB drive have your backup folders (in my case its F:)

You can check volume letters by DISKPART tool,

(some reference below which I used, disk and volume was 3 in my case)
diskpart > list disk > select disk 3 > list volume
(to change drive letter if you require, use > assign letter=F

• Now check for available backup version, issue following

wbadmin get versions -backupTarget:f:

• Once you get the version identifier , restore it by using following cmd

wbadmin start sysrecovery -version:11/26/2018-22:00 -backuptarget:f: -machine:DC01 –authsysvol -recreateDisks -restoreAllVolume

Now you have to do Yes in all questions it asks for, once all done system will reboot , &  you will be flying with the AD again 🙂

---

4# Bare-metal Recovery via WBADMIN Restore in new vm using network share [Updated 2-OCT-2018]

Scenerio:

• Domain Controller Name: DC01
• Full Backup Set is available on a network location \FILESERVER\WindowsImageBackup\DC01

If you have taken FULL backup of your server (EXAMPLE DC01) using WBADMIN/ Server Manager Backup wizard and stored in a network location, we can restore it to the new vm [Bare-metal Recovery]

Steps:

• Create a new VM with same settings as old vm [DC01].
• Now boot this vm via windows ISO (in my example it was 2016 gui edition) , and select REPAIR ,
• Open CMD, and issue
  start /w wpeinit

• Test connectivity by

net use \\FILESERVER\WindowsImageBackup\DC01 /user:DOMAIN\ADMIN

• Now get backup version from the shared folder

wbadmin get versions -backupTarget:\\FILESERVER\WindowsImageBackup\DC01

If all goes correct, then it will show you the backup versions, note down the version you want to restore and use it in below CMD …

wbadmin start sysrecovery -version:10/02/2018-22:05 -backuptarget:\\FILESERVER\WindowsImageBackup\DC01 -machine:DC01 -recreateDisks -restoreAllVolumes –authsysvol

Once it shows successfull , then it may ask for reboot, do so ..

---

5# How to Protect/Unprotect Active Directory Objects from Accident Deletion

Enable Protection

1#- Enable protection on all active directory users

Get-ADObject -filter {(ObjectClass -eq "user")} | Set-ADObject -ProtectedFromAccidentalDeletion:$true

2# Enable protection any Organizational Unit where the setting is not already enabled

Get-ADOrganizationalUnit -filter * | Set-ADObject -ProtectedFromAccidentalDeletion:$true

3# Enable protection for groups

Get-ADObject -filter {ObjectClass -eq "user" -or ObjectClass -eq "group"} | Set-ADObject -ProtectedFromAccidentalDeletion:$true

Disable Protection

To remove protection, just change $true to $false in above commands

List Users with bad password count

Get-ADUser -filter * -Properties badPwdCount | where {$_.badpwdcount -gt 1} | Select -Property Name,badpwdcount | sort -Property name

---

6# FSMO roles transfer by powershell cmd

Its now easier to move FSMO roles with Powershell from the 2012/2016 machine.

1) Login to any DC and open powershell as administrator
2) Execute following command

Move-ADDirectoryServerOperationMasterRole -Identity "DC02" -OperationMasterRole 0,1,2,3,4

DC02 is the server on which you want to transfer all fsmo roles, example dc2 is our second DC and we want to move roles from the DC1 to DC2. so we will simply add Target DC which is in our case is DC2. it will become PDC , just for the sake of conversation we are using word PDC

3) Select yes to all 5 roles
4) run netdom query FSMO to check the roles

---

7# Active Directory Health Check

Open CMD as RUN as ADMINISTRATOR, & use following commands to check active directory domain health ,

• DCDiag /Test:DNS
• dcdiag /s:DC1 /v
• repadmin /showrepl
• repadmin /replsummary
• repadmin /showrepl SERVERNAME
• repadmin /showrepl /errorsonly
• repadmin /queue
• repadmin /showoutcalls *
• repadmin /bridgeheads * /verbose
• repadmin /istg * /verbose

Force Sycn /Replication

The following command will Force / push immediate replication to all domain controllers in the Domain:

• repadmin /syncall /AdeP
• OR
• Repadmin /syncall DC_name /APed

Explanation:

• repadmin /syncall with the /A(ll partitions) P(ush) e(nterprise, cross sites) d(istinguished names)

Test FRS / DFSR repolication

• dcdiag /test:frsevent (for FRS)
• dcdiag /test:dfsrevent (for DFSR)

For reference , use following

http://kpytko.pl/active-directory-domain-services/active-directory-and-sysvol-replication-status/

Check policy match on all domains

• gpotool /verbose

to generate group p[policy acquire on client side, use this on client computer

Goto Start/Run

• RSoP.msc

Check from client workstation, Test

nltest /sc_query:MYDOMAIN

---

AD SystemStat Backup Script with Email

I am using BLAT email tool  for sending email result.

@echo off
::::::::::::::::::::::::::::::::::::::::::
:: DC01 DC BACKUP SCRIPT ::
::::::::::::::::::::::::::::::::::::::::::
set srvname=DC01
set description=DC01 - Daily Status of AD Backup Data Copied in FILE_SERVER
set jobname=DC01 - Daily Status of AD Backup Data Copied in FILE_SERVER
set attachment=c:\backup\ad_backup.log
set mail-subject=DC01 - Daily Status of AD Backup Data Copied in FILE_SERVER
set mail-body=DC01 - Daily Status of AD Backup Data Copied in FILE_SERVER
set mail-to=aacableAThotmailDOTcom
set backuppath=D:\WindowsImageBackup\DC01
set footer=DC+AD Automated Backup and Email Logs Script Created by ZAIB (Pvt) Ltd. IS Dept. / Syed Jahanzaib

FOR /F "skip=1 tokens=1-6" %%A IN ('WMIC Path Win32_LocalTime Get Day^,Hour^,Minute^,Second /Format:table ^| findstr /r "."') DO (
set Milisecond=%time:~9,2%
set Day=%%A
set Hour=%%B
set Minute=%%C
set Second=%%D
)
set /a Start=%Day%*8640000+%Hour%*360000+%Minute%*6000+%Second%*100+%Milisecond%

:: PUT COMMANDS HERE
rd /q %attachment%
net use T: \\FILE_SERVER\BKP_FOLDER
echo Now starting AD backup using wbadmin command ...
wbadmin start systemstatebackup -backuptarget:d: -quiet
echo
echo ***** NOW COPYING DC BACKUP DATA TO T:\DC01\AD
echo **************************************************
xcopy.exe D:\WindowsImageBackup\*.* T:\DC01\ad /S /D /C /Y
echo "Deleting OLD Backup Folder older then 10 days - - - - - -- - - - - - - - -- - - - - -"
PowerShell -Command "& Get-ChildItem "T:\DC01\AD\DC01" | Where-Object {$_.PSIsContainer -and $_.LastWriteTime -le [System.DateTime]::Now.AddDays(-10)} | Remove-Item -Recurse -Force"
PowerShell -Command "& Get-ChildItem "D:\WindowsImageBackup" | Where-Object {$_.PSIsContainer -and $_.LastWriteTime -le [System.DateTime]::Now.AddDays(-10)} | Remove-Item -Recurse -Force"
::
::

FOR /F "skip=1 tokens=1-6" %%A IN ('WMIC Path Win32_LocalTime Get Day^,Hour^,Minute^,Second /Format:table ^| findstr /r "."') DO (
set Day=%%A
set Hour=%%B
set Minute=%%C
set Second=%%D
)
set Milisecond=%time:~9,2%
set /a End=%Day%*8640000+%Hour%*360000+%Minute%*6000+%Second%*100+%Milisecond%
set /a Diff=%End%-%Start%
set /a DiffMS=%Diff%%%100
set /a Diff=(%Diff%-%DiffMS%)/100
set /a DiffSec=%Diff%%%60
set /a Diff=(%Diff%-%Diff%%%60)/60
set /a DiffMin=%Diff%%%60
set /a Diff=(%Diff%-%Diff%%%60)/60
set /a DiffHrs=%Diff%

:: format with leading zeroes
if %DiffMS% LSS 10 set DiffMS=0%DiffMS!%
if %DiffSec% LSS 10 set DiffMS=0%DiffSec%
if %DiffMin% LSS 10 set DiffMS=0%DiffMin%
if %DiffHrs% LSS 10 set DiffMS=0%DiffHrs%

echo The Domain Controller DC01 Backup Report > %attachment%
echo.>> %attachment%
echo The Backup Script took %DiffHrs% Hours, %DiffMin% Mnts, %DiffSec% Secs >> %attachment%
echo.>> %attachment%
echo List of Folders >> %attachment%
echo.>> %attachment%
FORFILES /p %backuppath% /S /D +0 /C "cmd /c IF @isdir == TRUE echo @path" >> %attachment%
echo.>> %attachment%
echo %footer% >> %attachment%
c:\blat\blat.exe %attachment% -to %mail-to% -i %srvname% -s "%mail-subject%"

---

Event ID 4771 : Kerberos Pre-Authentication Failed

When troubleshooting AD account lockout issues you can search thru Domain Controller security logs for audit failures and event ID 4771.

These event details will include a result code which will specify exactly what the issue is. Most common are …

• 0x12 – client credentials have been revoked (disabled, expired, locked, etc)
• 0x17 – password has expired
• 0x18 – pre-authentication was invalid (bad password)

in my particular case I modify the Kereberos time in GPO , default maximum tolerance time was 5 minutes, I extend it to 30 minutes. as showed below …

1. Open “Group Policy Management”.
2. Navigate to “Group Policy Objects” in the Domain being reviewed (Forest >> Domains >> Domain).
3. Right click on the “Default Domain Policy”, select “Edit”.
4. Navigate to Computer Configuration >> Policies >> Windows Settings >> Security Settings >> Account Policies >> Kerberos Policy.

“Maximum tolerance for computer clock synchronization” , set this to higher value like 30 minutes.

---

Event ID 4 :  The Kerberos client received a KRB_AP_ERR_MODIFIED error from the server PC01$. The target name used was PC01$

To sort this, Frankly I’d consider disabling machine account password changes on the domain altogether. MS recommends not doing this because of the relaxed security it engenders. but if security is not much concern then try it.

• https://www.sepago.de/blog/the-danger-of-machine-account-password-changes/
• https://community.spiceworks.com/topic/1390899-security-kerberos-event-4-server-2012-cluster

In Default Domain Group Policy

Computer Configuration | Policies | Windows Settings | Admin Templates | Security Options | Domain Member

Play with following …

• Domain member: Disable machine account password changes =  Enabled

OR increase the password age time to 999 days , [2.7 years approx]

• Domain member: Maximum machine account password age  = 999 days

---

Check user group membership etc

whoami /groups

or GPResult

gpresult /r

Reset all Kerberos tickets of the user with this command:

klist purge

---

Event tracking for AD changes

The following are some of the events related to user account management:

Event ID 4720 shows a user account was created.
Event ID 4722 shows a user account was enabled.
Event ID 4740 shows a user account was locked out.
Event ID 4725 shows a user account was disabled.
Event ID 4726 shows a user account was deleted.
Event ID 4738 shows a user account was changed.
Event ID 4781 shows the name of an account was changed.

---

Regard’s
Syed Jahanzaib