Task: Write an automated test for a detected deadlock

Here is a simplified version of the problem case

public void PingLicense()
{
lock(this)
{
CheckAuthorization();
}
}

public void GetLicense( out License obCurLicense)
{
lock(this)
{
//talk to non-threadsafe 3rd party lib to query the license info
}
}

CheckAuth() function does a GetLicense and raises events to notify others if a change in License occurs.
Ping License is called by a helper thread via a Timer object (every minute )

So here is how the deadlock occurred, if I remove the license h/w device midway. On the next PingLicense call, helper thread acquires the lock on the object and raises the event for everyone to switch to Unauth mode.
One of the subscribers is the UI ( to enable/disable UI elements ). *Any changes to the UI in .NET must be done on the thread in which the UI was created*
So the event handler requests a switch to the UI (Main) Thread. In the event handler, there is a call to GetLicense(). DeadLock !!!
UI thread is stuck requesting a lock for the object (which PingLicense currently holds). Ping License won't relinquish the lock since the event handler hasn't returned.

Agreed that it seems dumb now. But Hindsight is always...

Now the fix is - PingLicense doesn't need a lock block ( a result of a mindless strafing run inserting lock blocks ) . Removing that solved it.
But no code change without a failing test... Also it kinda bugged me that this got thru my AT Test store.



My Attempt

After some discussion with the nice folks on the TDD list, here's my AT (took 10 mins with Console Output verification)

[Test]
public void Test_CR9672_PollingThread_ShouldNotAcquireLock_BlocksEventHandlers_OnGetLicense()
{
LicTestHelper.WriteLicenseConfig(LicTestHelper.LocalHL_LIC_INFO);
LicenseManager.Instance.EvApplicationAuthorized += new EventHandler(On_LicMgr_EvApplicationAuthorized_CR9672);

DynamicMock obMockHL = SetupDynamicMockHLWrapper();
//... set up a mock to mock out the license hardware

GObjectSpy.CallPrivateMethod( LicenseManager.Instance, ReflectionStrings.S_LICMGR_PINGLICENSE_METH, new object[]{null} );
//Code will block here if a deadlock happens.
// If test completes, the test has passed
obMockHL.Verify();

}

private void On_LicMgr_EvApplicationAuthorized_CR9672(object sender, EventArgs e)
{
Thread obAnotherThread = new Thread( new ThreadStart( this.DummyHandler ) );
obAnotherThread.Start();
obAnotherThread.Join();
}

private void DummyHandler()
{
License obCurLicense;
// Query License which blocks indefinitely if PingLicense is holding a lock
LicenseManager.Instance.GetLicense(out obCurLicense);
}



I added a temporary log via Console.WriteLine to prove the test is authentic
# Test Failure log - test hangs
Lock Request by Thread 353 (PingLicense)
GotLock!
First Thread 353
Event Handler Thread 268
Lock Request by Thread 268 (GetLicense)

# Test Success log
Lock Request by Thread 359 (PingLicense) // did not remove Console.WriteLine - actually no lock request
GotLock! // did not remove Console.WriteLine - actually no lock request
First Thread 359
Event Handler Thread 261
Lock Request by Thread 261 (GetLicense)
GotLock!
EventHandler exit
PingLicense exit

Task completed!

No comments:

Post a Comment