D
Dave Schob
Guest
Our automated kernel driver tests run into a problem every couple of weeks on our Windows 10 and Windows Server 2016 VMs. Debugging the issue revealed that verifier.exe is leaking callbacks. Once the 64 callback slots are full, our driver tests fail with error code 87 when registering our callback. At this point a reboot of the VM is required to reset the callback list and get our tests running smoothly again. We’ve also tested this and confirmed that it happens on a physical machine with Windows 10 (v1607) installed.
Description of our test:
Our automated test runs a suite of Google Tests, first with verifier disabled:
(verifier.exe /volatile /removedriver <ourDriver.sys>)
and then again with verifier enabled:
(verifier.exe /volatile /adddriver <ourDriver.sys> /flags 0xfbf).
More details about verifier fault:
Further testing reveals that the verifier flag "Randomized low resources simulation" causes the issue. If that flag is off, the issue does not occur; and if only that flag is on, the issue does occur. We can reproduce the issue even specifying SysMon's driver in the verifier volatile calls (or even a driver name that does not exist.) Running this batch file will exhaust the callback list and cause any subsequent callback registration to fail:
FOR /L %%i IN (1,1,70) DO (
verifier /volatile /removedriver anything.sys
verifier /volatile /adddriver anything.sys /flags 0b100
@echo ITERATION=%%i
)
Continue reading...
Description of our test:
Our automated test runs a suite of Google Tests, first with verifier disabled:
(verifier.exe /volatile /removedriver <ourDriver.sys>)
and then again with verifier enabled:
(verifier.exe /volatile /adddriver <ourDriver.sys> /flags 0xfbf).
More details about verifier fault:
Further testing reveals that the verifier flag "Randomized low resources simulation" causes the issue. If that flag is off, the issue does not occur; and if only that flag is on, the issue does occur. We can reproduce the issue even specifying SysMon's driver in the verifier volatile calls (or even a driver name that does not exist.) Running this batch file will exhaust the callback list and cause any subsequent callback registration to fail:
FOR /L %%i IN (1,1,70) DO (
verifier /volatile /removedriver anything.sys
verifier /volatile /adddriver anything.sys /flags 0b100
@echo ITERATION=%%i
)
Continue reading...