T
techjedi2012
Guest
Hello,
I need to report a bug in the Windows Server Backup (wbengine.exe) software on Windows Server 2016.
The issue is that wbengine.exe will crash on start if any drive/volume has been removed from a system since the last time a scheduled job was created and executed. It does not matter if the drive removed is included or excluded from the backup settings.
This crash prevents using the Windows Server Backup management console to view/manage backup jobs or attempt recovery. This also prevents any scheduled backups from starting.
This has happened a few times over the years, but I just deleted the catalog and started over because I fortunately had the data still intact on the drives themselves and I didn't have time to investigate it more. However a recent maintenance on the server for a slowly failing drive brought it up to the point where I have a simplified repro:
In my case, I have the following drives/volumes:
Disk ### Status Size Free Dyn Gpt Drive Letter
-------- ------------- ------- ------- --- --- ------------
Disk 0 Online 238 GB 23 GB D
Disk 1 Online 953 GB 95 GB E
Disk 2 Online 1863 GB 0 B * * F (MIRROR)
Disk 3 Online 1863 GB 0 B * * F (MIRROR)
Disk 4 Online 1863 GB 0 B * G
Disk 5 Online 9 TB 18 MB * NO LETTER - RESERVED FOR BACKUP
Disk 6 Online 119 GB 11 GB * C (BOOT)
I created a backup schedule covering:
Bare metal recovery
System State
EFI System Partition
Local disk (C
D:\SERVICE\*
D:\STORAGE\*
E:\STORAGE\*
F:\STORAGE\*
Recovery
This backup was run twice over two days and the console showed both backup successes. Then, the G drive was removed from the system and the crashes occurred on the next boot even though the G: volume and the underlying disk are not included in this backup.
Repro Steps:
1) Shutdown server
2) Remove Disk 4 (Drive G
3) Restart server
4) Attempt to open the Windows Server Backup console (wbadmin.msc)
5) Observe the error message:
6) Attempt to restart wbadmin.msc and the same occurs repeatedly.
7) Windows Event Viewer application log shows this event for the wbengine.exe crash:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
<System>
<Provider Name="Application Error" />
<EventID Qualifiers="0">1000</EventID>
<Level>2</Level>
<Task>100</Task>
<Keywords>0x80000000000000</Keywords>
<TimeCreated SystemTime="2020-12-23T18:17:52.745598900Z" />
<EventRecordID>44662</EventRecordID>
<Channel>Application</Channel>
<Computer>(redacted)</Computer>
<Security />
</System>
- <EventData>
<Data>wbengine.exe</Data>
<Data>10.0.14393.3750</Data>
<Data>5ed5def3</Data>
<Data>wbengine.exe</Data>
<Data>10.0.14393.3750</Data>
<Data>5ed5def3</Data>
<Data>c0000005</Data>
<Data>00000000000ffb36</Data>
<Data>1620</Data>
<Data>01d6d957ed7e7cc6</Data>
<Data>C:\Windows\system32\wbengine.exe</Data>
<Data>C:\Windows\system32\wbengine.exe</Data>
<Data>81e063c6-1874-4c38-9ffd-f6d0fe6d71e0</Data>
<Data />
<Data />
</EventData>
</Event>
8) Any attempts to run a scheduled backup fail with the same application error.
9) Power down the server
10) Replace the drive that was removed and start server.
11) Observe that the backup management console now works again and scheduled backups can continue
In the cases where you no longer can put the original drive back in the system, the only solution to this problem is to delete your catalogs and start over with "wbadmin delete catalog" from a command prompt. This is very problematic as it is not uncommon for drive changes to be made when a drive is failing. If your intent is to restore to the new drive, you must be careful to first install the new drive and restore prior to disconnecting the any drive. In the case where all drive ports are already being used, this can be tricky as even disconnecting an unrelated drive in the system will trigger this problem.
My system info:
Windows Server 2016 Standard
Microsoft Windows [Version 10.0.14393]
wbengine.exe info:
Name c:\windows\system32\wbengine.exe
Version 10.0.14393.3750
FileSize 1548800
I have also observed that Windows Server Backup can sometimes get into this crash state after new OS updates from Windows Update. There are reports of variations of this problem on Bing/Google search, but the recommendation is always to either do a catalog delete or get a hotfix, but I am on the latest version of Server 2016 and there is no hotfix listed for this version.
I am really not confident in this Backup software as a result, but I don't know of any good simple alternatives that do Full/Incremental backups and keep many copies over time so you can choose to restore a version of the file from a specific day over time. I would pay for a backup software if it was reliable.
Continue reading...
I need to report a bug in the Windows Server Backup (wbengine.exe) software on Windows Server 2016.
The issue is that wbengine.exe will crash on start if any drive/volume has been removed from a system since the last time a scheduled job was created and executed. It does not matter if the drive removed is included or excluded from the backup settings.
This crash prevents using the Windows Server Backup management console to view/manage backup jobs or attempt recovery. This also prevents any scheduled backups from starting.
This has happened a few times over the years, but I just deleted the catalog and started over because I fortunately had the data still intact on the drives themselves and I didn't have time to investigate it more. However a recent maintenance on the server for a slowly failing drive brought it up to the point where I have a simplified repro:
In my case, I have the following drives/volumes:
Disk ### Status Size Free Dyn Gpt Drive Letter
-------- ------------- ------- ------- --- --- ------------
Disk 0 Online 238 GB 23 GB D
Disk 1 Online 953 GB 95 GB E
Disk 2 Online 1863 GB 0 B * * F (MIRROR)
Disk 3 Online 1863 GB 0 B * * F (MIRROR)
Disk 4 Online 1863 GB 0 B * G
Disk 5 Online 9 TB 18 MB * NO LETTER - RESERVED FOR BACKUP
Disk 6 Online 119 GB 11 GB * C (BOOT)
I created a backup schedule covering:
Bare metal recovery
System State
EFI System Partition
Local disk (C
D:\SERVICE\*
D:\STORAGE\*
E:\STORAGE\*
F:\STORAGE\*
Recovery
This backup was run twice over two days and the console showed both backup successes. Then, the G drive was removed from the system and the crashes occurred on the next boot even though the G: volume and the underlying disk are not included in this backup.
Repro Steps:
1) Shutdown server
2) Remove Disk 4 (Drive G
3) Restart server
4) Attempt to open the Windows Server Backup console (wbadmin.msc)
5) Observe the error message:
"A fatal error occurred during a Windows Server Backup snap-in (Wbadmin.msc) operation. Error details: The server threw an exception. Close Wbadmin.msc and then restart it.
6) Attempt to restart wbadmin.msc and the same occurs repeatedly.
7) Windows Event Viewer application log shows this event for the wbengine.exe crash:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
<System>
<Provider Name="Application Error" />
<EventID Qualifiers="0">1000</EventID>
<Level>2</Level>
<Task>100</Task>
<Keywords>0x80000000000000</Keywords>
<TimeCreated SystemTime="2020-12-23T18:17:52.745598900Z" />
<EventRecordID>44662</EventRecordID>
<Channel>Application</Channel>
<Computer>(redacted)</Computer>
<Security />
</System>
- <EventData>
<Data>wbengine.exe</Data>
<Data>10.0.14393.3750</Data>
<Data>5ed5def3</Data>
<Data>wbengine.exe</Data>
<Data>10.0.14393.3750</Data>
<Data>5ed5def3</Data>
<Data>c0000005</Data>
<Data>00000000000ffb36</Data>
<Data>1620</Data>
<Data>01d6d957ed7e7cc6</Data>
<Data>C:\Windows\system32\wbengine.exe</Data>
<Data>C:\Windows\system32\wbengine.exe</Data>
<Data>81e063c6-1874-4c38-9ffd-f6d0fe6d71e0</Data>
<Data />
<Data />
</EventData>
</Event>
8) Any attempts to run a scheduled backup fail with the same application error.
9) Power down the server
10) Replace the drive that was removed and start server.
11) Observe that the backup management console now works again and scheduled backups can continue
In the cases where you no longer can put the original drive back in the system, the only solution to this problem is to delete your catalogs and start over with "wbadmin delete catalog" from a command prompt. This is very problematic as it is not uncommon for drive changes to be made when a drive is failing. If your intent is to restore to the new drive, you must be careful to first install the new drive and restore prior to disconnecting the any drive. In the case where all drive ports are already being used, this can be tricky as even disconnecting an unrelated drive in the system will trigger this problem.
My system info:
Windows Server 2016 Standard
Microsoft Windows [Version 10.0.14393]
wbengine.exe info:
Name c:\windows\system32\wbengine.exe
Version 10.0.14393.3750
FileSize 1548800
I have also observed that Windows Server Backup can sometimes get into this crash state after new OS updates from Windows Update. There are reports of variations of this problem on Bing/Google search, but the recommendation is always to either do a catalog delete or get a hotfix, but I am on the latest version of Server 2016 and there is no hotfix listed for this version.
I am really not confident in this Backup software as a result, but I don't know of any good simple alternatives that do Full/Incremental backups and keep many copies over time so you can choose to restore a version of the file from a specific day over time. I would pay for a backup software if it was reliable.
Continue reading...