Code Coverage with NCover and UFT / QTP

Automated testing is key to any productive QA team. We at NCover often work with teams using any number of test automation solutions to simplify their test processes. One of the solutions that we encounter frequently is HP’s Unified Functional Testing (UFT), the updated version of another HP product, Quick Test Professional (QTP). In fact, many teams still use the older QTP solution today and the steps that we’ll share in this post work well with either UFT or QTP.

Background On Working with UFT / QTP

While UFT/ QTP is helpful for increasing the efficiency of the testing process, these solutions can often interfere with coverage collection. The landscape of plugins and configurations possible with UFT/QTP can impose obstacles to the profiling methodology used by NCover. NCover performs in-memory profiling via the CLR profiler interface provide by .Net. The results are mixed when customers tackle this integration. Certain scenarios work well, but frequently the level of integration required by both tool sets is just difficult to orchestrate. The conflicts are various and fleeting at times, making this an integration with the potential of significant maintenance.

The good news is that the introduction of pre-instrumentation in NCover 5 makes these conflicts a thing of past. With pre-instrumentation you can insert coverage instrumentation into the assemblies of an application on disk, rather than in memory, to avoid the CLR Profiler orchestration. Teams using this approach have achieved positive results and improved performance.

How Pre-Instrumentation Works

Pre-instrumenting an assembly is a simple process and can be done at the command line or as part of a scripts with the following command:

c:> ncover instrument myassembly.dll

c:> ncover instrument myassembly.dll myapp.exe myassembly2.dll

c:> ncover instrument *.dll *.exe

Wildcards are available for this command but it is important to be judicious about instrumentation. Avoid instrumenting third party libraries and system assemblies and focus on instrumenting only what is actionable. Selective pre-instrumentation becomes a form of pre-coverage filtering.

Pre-instrumentation is valid for both a single assembly in a solution or multiple assemblies, on a simultaneous basis, within a solution. Through this approach, selective pre-instrumentation becomes an additional coverage filter allowing you to instrument only the assemblies for which coverage collection is needed. It is not necessary to pre-instrument an .exe file in order to collect coverage on a specific .dll file. Instrument only the .dll file and execute testing on the application as usual.

The execution of a pre-instrumented assembly collects coverage counters into a mapped file on disk while the application testing is underway. When the process exits, the binary coverage file remains on disk and can be imported via NCover Desktop, Code Central or Collector. These files carry the .ncprof extension and are raw coverage data. By making this file a raw binary counter file, the file remains valid for import even if your application crashes before you complete testing.

The coverage file follows a naming convention that includes the process name and a timestamp. All pre-instrumented assemblies in the same process store their coverage counters in the same data file. This is similar to the same behavior of in-memory profiling, which places all of the coverage data in a single execution. Each .ncprof file represents a single execution of an pre-instrumented application.

Importing a coverage file is accomplished simply with the following command:

c:> ncover import --project=Project1 --file=MyAppName-[timestamp].ncprof

c:> ncover import --project=”My Coverage Project” --file=*.ncprof

Additional Resources

If you would like to learn more about instrumentation, we would encourage you to visit our posts on Pre-Instrumentation For Windows Store Apps and Integrating NCover Pre-Instrumentation Into Your Build Process.

Code Coverage with NCover and UFT / QTP