While developing an application, I came up against a deficiency of testing frameworks. They don’t seem to handle code which works asynchronously. By this, I mean that it’s difficult to test code which uses callbacks.
I decided that it was important for my framework to be able to automatically discover when user test code had completed. When user code is asynchronous, the test framework cannot simply call one method and assume the test is finished when it is returned control. One way around this is to require explicit test termination by user code, but this is error prone: an exception in a callback may cause the ‘EndTest’ call to be missed, unless the user is careful to use a ‘Finally’ block which does this.
It is possible to work around this problem by using timeouts: assuming a test has failed after a particular period of time. This does not, however, deal with uncaught exceptions in a callback. If the test framework is unable to catch exceptions which weren’t handled by user code, it is unable to run further tests. This would be a problem when a suite of tests were to be run overnight and the first failed.
There seem to be two different ways that .NET uses callbacks: through AsyncCallback and EventHandler.
The two designs are similar, in that they both use delegates.
The first provides Begin- and End- prefixed methods on a class: the user calls BeginSomeOperation(SomeParameter As Something, Callback As System.AsyncCallback, stateObject As Object) and then waits for their callback to be called. The callback takes an IAsyncResult and the stateObject given to BeginSomeOperation.
When the callback is called, the user calls EndSomeOperation(theAsyncResult) and may be given some result of the async operation. The ‘state’ object is simply for the user to make use of to keep track of what’s happening, if they wish to.
An example of the first design can be seen by examining System.Net.Dns.
The second design provides ‘events’. An class may have more than one exposed Event type. A user of a class calls SomeObject.SomeEvent += MyMethod or, in VB.NET, AddHandler SomeObject.SomeEvent, MyMethod. When SomeObject decides to fire SomeEvent, the user’s MyMethod is called.
My goal was to make exception handling work and also know when a test had finished without requiring explicit termination by user code.
My strategy is to have my own code called instead of the user’s callback, and to call their callback myself. Calling their callback myself means I can catch any exceptions it might fail to.
Dealing with the first type of callback system, AsyncCallback, was relatively simple. I require a small change to user code in order to make this work, but the change is in their test code, so it doesn’t affect the code being tested.
Where a user might write:
Dim Result As IAsyncResult = Net.Dns.BeginGetHostByName("rikkus.info", AddressOf MyCallback, Nothing)
I require:
Dim Result As IAsyncResult = Net.Dns.BeginGetHostByName("rikkus.info", WrapCallback(AddressOf MyCallback), Nothing)
That’s all there is to it.
WrapCallback keeps a mapping from the name of the current test case to the user provided callback and returns a new AsyncCallback which points to an internal method. The internal method, when called in place of the user’s callback, calls the user’s callback in a Try...Catch block and then ends the test.
The second type of callback system, Events, was much tricker to deal with.
With the AsyncCallback system, callbacks are delegates with only one method signature, so when I call the user’s callback, I know what to call, and when I register my wrapper for that callback, I only need one wrapper.
With Events, there are many different EventHandlers (an EventHandler is a delegate), with varying signatures. Users can create their own, so it’s not sufficient to create wrapper delegates for every EventHandler in the .NET framework.
My test framework code is called at runtime, obviously, and at that point I don’t have the correct delegate to use for a particular event. The way around this was to create the delegate at runtime. I do this by writing code at runtime and having it compiled by the .NET framework’s built in compiler.
This means that I can create a wrapper for any callback.
There was one other problem with Events: It’s not possible to write a method which takes an Event parameter. I have to accept the object which raises the event and the name of the event, and call Invoke on the add_EventName method of that object. This method is created by the compiler when an Event is declared, and hidden.
My generated code does the same as I do with AsyncCallback: It calls the real callback in a Try...Catch block, and then ends the current test.
This was a fun exercise, as I’ve learned all about .NET delegates and Events. It will also be useful to me, and I hope to others. I’m currently using my mini async testing framework to test my own code (which uses Net.Dns and System.Timers.Timer) and it’s working nicely. I’ll package up the framework as soon as I can, so others can try it out.
Thanks go to Jeremy Hopkin and someone on #c#, Freenode, whose name I’ve forgotten, for helping me understand .NET delegates and Events.