Coverage with Multiple Test Assemblies

With NCover it’s easy to get coverage results for a single test assembly, but in some cases you may have multiple test assemblies related to a single component. In most cases, you would simply run each set of tests and merge the results into a single execution to get a good idea of where you have coverage holes.

But what if you wanted to do it differently? How would you go about using NCover and NUnit to get coverage results in a single execution for multiple test assemblies?

Friend of NCover, Kris Culin, came to us with an answer to that question and was kind enough to share it with our .NET community. This is one approach to answering this question and by no means the only way. We would love to hear your method!

Using NCover and NUnit to Get Coverage Results in a Single Execution for Multiple Test Assemblies:

Make sure you are using at least the 2.6.x version of NUnit, which is available at http://www.nunt.org. Then, make sure you are using the latest available version of NCover Desktop.

To start, build your solution; the test projects should all be part of the same solution.

Next, using the NUnit GUI, create a new NUnit project and add each of your test assemblies to it. Be sure you browse to the bin folder for each test project, so that when the tests execute NUnit finds all relevant dependencies and that relative paths (if they are used) are found correctly.

Go to Project->Add Assembly… to add a new test assembly to your project.

In the screen shot above, note that three (3) test assemblies have been added, each with their own root nodes. If you ran your tests as they are configured now using NCover the result would be a single execution, but covered assemblies would show up multiple times – one for each AppDomain that was executed. In this scenario, each test assembly is considered a separate AppDomain. However, there is an easy solution for this problem.

In NUnit, go to Tools->Settings. Under the Test Loader node, select Assembly Isolation. On the right side, select the radio button for “Use a single AppDomain for all tests” and then check “Merge tests across assemblies.”

When you click OK, you should be prompted with the following:

Click Yes to proceed.

When the NUnit tree is reloaded, it will now contain a single root. All of your unit tests have been merged into a single tree, using common namespaces throughout your test assemblies.

The merge will work best if you have common namespaces between multiple test assemblies. This is not required — how you organize your tests is completely up to you — but this the suggested best practice.

Now, save your Nunit project in a location relative to your test assembly output folders.

With an NUnit project, you can use either the NUnit GUI or the NUnit console with the nunit project filename as the parameter. With NUnit console, you can use the other command line parameters as you would normally use them. NUnit is smart enough to read the NUnit file to get to the test assemblies.

Make sure your NCover project has nunitx86.exe as one of the covered processes. Also, before running the NUnit GUI, make sure you’ve enabled your NCover project.

Once your NCover project is configured, start NUnit, open the NUnit project you created above and run your unit tests. Once testing is complete, check your NCover project and you will find a single execution. That execution will show each of your covered assemblies just once.

Now, you can browse your coverage results and start working on filling in the coverage gaps.

I hope this solution works for you as it does for me. Getting a full picture of coverage from across multiple test assemblies is an important tool in discovering where your coverage gaps appear.

About the Author:

Kris is a 17 year veteran of the software development industry. For the past 10 years he has worked for Bentley Systems, Inc., headquartered in Exton, PA. Kris works with a team of developers from around the globe that writes software for civil engineers for water distribution, storm and sewer network analysis and detention pond design and analysis. Kris is a co-inventor for two patents related to water distribution. Kris is a believer in writing unit tests whenever possible. Kris is also a full supporter and proponent of NCover and uses it whenever possible to help improve his team’s code and eliminate bugs before they are found by the QA team. Kris has been using NCover since it was a free, open source application.  In his free time, Kris enjoys writing code, playing computer games and photography. You can view his photographs at http://www.culin.org.