On my last few contracts I have advocated acceptance testing through UI automation, but one of the most common wishes expressed by those coding up tests was for some way to validate that our automation wrappers actually still reflected the control they wrapped in the application under test. Something as simple as renaming a button would break our tests. It wasn’t until the next scheduled run of our acceptance tests that the breakage would be identified.
As mentioned in my last post I am working on a Windows Automation Toolkit (WATKit), I haven’t finished it yet but I figured an early feature should be some kind of build time validation to resolve the above issues. As of today WATKit includes an MSBuild Task and a few attributes to do just that.
WATKitBuildTask
The build task basically compares types in a test assembly that are decorated with AutomationTypeMappingAttribute against elements on types in one or more source assemblies. The test assembly and source assemblies are specified in the build configuration file (aka VS 2010 project file). Here is the what I added to the WATKit.Tests.csproj file in the WATKit source available on GitHub:
1: <UsingTask TaskName="WATKIt.Build.WATKitBuildTask"
2: AssemblyFile="$(TargetDir)\WATKit.dll"/>
3: <Target Name="AfterBuild">
4: <WATKitBuildTask TestAssembly="$(TargetDir)WATKit.Tests.dll"
5: SourceAssemblies="$(SolutionDir)\WATKit.TestApp.WPF\bin\Debug\WATKit.TestApp.exe" />
6: <WATKitBuildTask TestAssembly="$(TargetDir)WATKit.Tests.dll"
7: SourceAssemblies="$(SolutionDir)\WATKit.TestApp.WinForms\bin\Debug\WATKit.TestApp.exe" />
8: </Target>
So the first thing it does is reference the build task with a <UsingTask /> element. With this in place the task can be referenced as an element within any <Target /> element. I simply uncommented the AfterBuild target that is included in most project files. I have the task running to validate against the two test apps in the solution.
The build task has two required properties:
TestAssembly points to the assembly containing the wrappers to be validated. It doesn’t have to be a test assembly, if you keep your wrappers in a separate class library then it is this library you should reference.
SourceAssemblies is a comma separated list of one or more assemblies that contain real controls and windows. It works with WPF and WinForms applications (I haven’t tested it on other project types, I don’t see any reason it won’t work with Silverlight 4.* assemblies, but I haven’t tested it yet)
AutomationTypeMappingAttribute
In the test assembly only types with the AutomationTypeMappingAttribute are checked. Each Property on these types is checked unless it is decorated with an IgnoreAttribute. Methods and Fields are implicitly ignored.
This attribute has a single constructor argument that must be a fully qualified name of the type to validate against in one of the source assemblies. For example the MainWindow wrapper control in the WATKit.Tests project is declared like this:
1: [AutomationTypeMapping("WATKit.TestApp.MainWindow")]
2: public class MainWindow : Window
3: {
4: }
This tells the build task to validate MainWindow in the test assembly against the first WATKit.TestApp.MainWindow it finds in the source assemblies (the search is carried out in the order assemblies are listed)
IgnoreAttribute
By default the build task will check every Property on the wrapper type unless you tell it otherwise by decorating a Property with this attribute. You can optionally specify a Reason for ignoring it, so that others will know why. For example the DynamicButton property of MainWindow mentioned above looks like this:
1: [Ignore(Reason = "Button does not exist until run time")]
2: public Button DynamicButton { get { // removed for brevity } }
Many properties in the base classes in WATKit are decorated with this attribute to avoid build failures.
AutomationMemberAttribute
By default the build task will look for an exact match of the wrapper Property name with the name of a Field (child controls in WPF and WinForms are exposed as fields not properties) in the type it is validating against. If for some reason the names do not match you can decorate the wrapper property with this attribute and specify the name to match. For example the ChangeMyNameButton property of MainWindow mentioned above looks like this:
1: [AutomationMemberMapping("IChangeMyNameButton")]
2: public Button ChangeMyNameButton { get { // removed for brevity } }
This tells the build task to validate the ChangeMyNameButton property against a field named IChangeMyNameButton.
That’s it, no rocket science but it does the job and based on my experiences should help to give early feedback about breaking changes in applications being tested via UI Automation. The build task is not limited to use with tests that use features of WATKit, it should work with any automation framework as long as you create wrappers for your controls and those wrappers expose properties representing child controls and elements exposed by the real types.
If you have any feedback or suggestions for improving this please let me know by commenting here or over at GitHub.