TestProperty only allows one instance of each key

Snag! So it turns out that TestPropertyAttribute only supports one instance of each key, so it would mean that I would be limited to only associating a single category with each test. Oh well, I guess I’ll write my own attribute then 🙂 I was surprised at how easy this was the first time I did it a few years back.


///<summary>
/// An attribute that is used to associate a category with a
/// test
///</summary>
public class TestCategoryAttribute : Attribute
{
///<summary>
/// The Category the test is associated with
///</summary>
public string Category
{
get;
set;
}
///<summary>
/// Associates the test with a category. These categories are
/// then passed by the TestListGenerator in order to produce
/// Category Dependent Test Lists
///</summary>
///<param name="_category">The category to associate this test to</param>
public TestCategoryAttribute(string _category)
{
this.Category = _category;
}
}


Reflections…

So while Visual Studio doesn’t support the [CategoryAttribute], it does have the rather useful [TaskPropertyAttribute]. This attribute allows you to define key/value pairs for a test. So, I guess I am going to be using “Category” as the key. Unfortunately, the command line version of the test tool only supports either listing every test individually (ick) or some weird proprietary TestList format. Oh, and guess what, you can only create TestLists with Visual Studio Team System Test Edition. Somehow, I sense this is going to be _fun_. Fortunately, I have an evaluation of VSTSTE (sounds like a disease) around here somewhere…

Hmm, I think I will leave this part till later, and go have a look in a mirror – yup, you guessed it, time to play with one of the best features of .net, Reflection. First of all, I need a way to load in the assembly I want and then I need to go through and look for Test Classes. Then, for each TestMethod, I need to grab the settings for the TestProperties…

Loading the assembly is easy:

Assembly asm = Assembly.LoadFrom(dllPath);

Now I can get a list of types:

Type[] types = asm.GetExportedTypes();

And for each Type, I can get a list of Methods:

MethodInfo[] type.GetMethodsBindingFlags.Instance | BindingFlags.DeclaredOnly | BindingFlags.Public)

I’m only interested in TestMethods:

mi.GetCustomAttributes(typeof(TestMethodAttribute),true).Length > 0

And in particular, the TestProperties of those methods:

mi.GetCustomAttributes(typeof(TestPropertyAttribute), true)

Sweet – so that part was easy – I’m going to check this in now…

First Step: Source Control

So I decided that I may as well make this project open source and take advantage of http://www.codeplex.com. CodePlex gives you a simplified and free “Visual Studio Team System” to connect to.

I created a new project @ http://www.codeplex.com/testlistgenerator

Visual Studio Team System

So I have been playing around with Visual Studio Team System in my spare time, evaluating if it’s actually any better than using a whole mess of alternatives (Our current setup is NUnit, MSBuild, NCover, CruiseControl & FogBugz). Overall, having everything integrated with everything else is fantastic! It makes assign check ins to bugs really easy, and it’s then easy to associate specific unit tests to work items, so that you can get some background as to why that unit test is there. Awesome.

However, each “component” lacks one thing. For example, the interface to the work items is no where near as polished as FogBugz. You can’t share items in the source control. And today, I found out that you can’t have categories in NUnit tests. This is a make or break item for me. So what am I going to do about it? Fix it, that’s what. This weekend I plan on writing something that will autogenerate test lists and then run them with the console mstest application.