Friday, November 21, 2008

When Testing Collides With Observation

Recently I noticed that a component I have been working on was running slowly in its Release version. The only changes I had made recently were mostly calls to Debug.WriteLine() to get unit-testing information. When I figured out the problem, it reminded me of a term from my Psychology background (my degree in Psychology, not my therapy ;-) ) called the “Observer Effect.”

The Observer Effect is a term in experimental research which basically says that while you are trying to observe something, the very act of observation might affect what you are trying to observe.

Consider the following simple, albeit silly example :

Hypothesis: Water freezes at zero degrees Celsius.
Method of study: Stand in freezer holding small puddle of water in hand.

Results: Water doesn’t freeze at zero degrees Celsius.

Why? Because the method of observation (holding water in hand) prevents the water from freezing because the warmth of your hand (around 98.6F) warms the water sufficiently so that it doesn’t freeze.

So how does that relate to testing?

We all have put calls to System.Diagnostics.Debug.WriteLine() in our code to help with debugging or to validate assumptions. This is our “observation method.” One of the side benefits of using Debug.WriteLine() is that the compiler will ignore these lines when building a Release version of your code. It is as if the code never existed.

Why is this important? Let’s say that you wrote a generic method that loops through a List<>, writing out the contents of the list:

public static void LogList<T>(string name, List<T> items)
Debug.WriteLine("== " + name + ": " + items.Count.ToString() + " items.==");

int counter = 0;
foreach (T item in items)
Debug.WriteLine(String.Format("[{0}] - {1}", counter, item));

Debug.WriteLine("==End of contents of " + name+ "==");

In this case, where you call LogList() in your code, LogList runs both in Debug and Release versions. Only the calls to Debug.WriteLine() are ignored by the compiler. If this is a particularly long list, or if there’s a time consuming part of this method, your Release executable will be slower.

The good news is that there is an easy fix: the Conditional attribute. The conditional attribute can be applied to methods, and looks like the following:

[Conditional("<symbol to test for>")]

When Visual Studio calls the compiler to compile a Debug version, it passes the symbol DEBUG as part of the command line. When Visual Studio calls the compiler to compile a Release version, it passes the symbol RELEASE as part of the command line.

The Conditional attribute tells the compiler to only include this method if the symbol exists. So if we simply add the following conditional attribute to our method:

public static void LogList<T>(string name, List<T> items)

Now the compiler will include this method in the compiled output only if DEBUG is a symbol (the compiler is building a Debug version).

You might be saying to yourself, “Rick, that only takes care of the method itself. What about all the calls to that method, spread out all over my application?”

The beauty of the conditional attribute is that the compiler ignores all of the calls to the function too!