Integrating NCover Pre-Instrumentation Into Your Build Process

Pre-Instrumentation for Continuous Integration

In a recent post describing NCover Pre-Instrumentation, we introduced you to coverage collection without an in-memory profiler. The specific use cases and advantages are well-detailed in that post. Now that this tool is in the arsenal, the next question we’ll address is, “How do I integrate this to my build process?” Let’s take a look.

Build scripts, servers, and CI seem to always have a very homegrown feel. No matter the framework for the process — NAnt, MSBuild, Team Foundation, Jenkins, Bamboo, etc. — the implementation always takes on a personality unique to its development team. Pre-instrumented coverage has the flexibility to allow each team to integrate based on that personality. Let’s assume that this discussion pertains to a single build server with a small farm of test clients. These same principles can scale up or down, as needed.

Key Concepts For Pre-Instrumented Coverage

There several key concepts you should keep in mind when integrating pre-instrumentation into your build process.

Pre-instrumentation:

• alters the assembly on disk,
• should not be part of a final release build,
• should be performed on the building machine for access to pdb’s and source and
• can be performed on any machine with a valid NCover license.

Pre-instrumented assemblies:

• must be re-signed if strong-named signing is used,
• can be distributed and run on any other machine with a valid NCover license,
• will create a new coverage file each time a process loads the assembly and
• have an added assembly reference to NCover.PreInstrument.dll.

Choreography

The sequence of work is straightforward, but non-negotiable. The build process should be unchanged up to the point that assemblies are being packaged for distribution. If you use a tool for obfuscation or build an installer, pay close attention to how these distributions are assembled. Obfuscation will be a hindrance to coverage, so save that for the release build only.

Installers can easily contain a pre-instrumented assembly, so if the deployment method to a testing satellite is via installer, just be sure that installer is not the one intended for final release. Pre-instrumenting assemblies before they are bundled for distributed testing requires only the pre-instrument command and re-signing, as needed. When bundling the signed assembly, be sure to include NCover.PreInstrument.dll in the bundle for distribution.

Sample Commands:

Instrument assembly:

c:\dev\myproject\debug>ncover instrument myassembly.dll

Instrument multiple:

c:\dev\myproject\debug>ncover instrument myassembly1.dll [myExe1.exe,...]

Instrument wildcard:

c:\dev\myproject\debug>ncover instrument myassembl*.dll myexe*.exe

Re-Signing Assemblies

The re-signing of assemblies is a well-documented task. Pre-instrumentation of any type alters the binary hash of an assembly, which will alter its signature. Pre-instrumenting all assemblies in an application invalidates all strong-naming, this lucky happenstance allows the application to run without re-signing. Pre-instrumenting selective assemblies will require that those assemblies be re-signed. See more about re-signing assemblies with strong-name tool (sn.exe) at Microsoft.

Below is a sample command for re-signing assemblies:

c:\dev\myproject\debug> sn.exe -R myassembly.dll mykeyfile.snk

Build Server Commands

The following are examples of commands you may use on your build server as part of the pre-instrumentation process:

Build your application:

msbuild /t:Rebuild /p:Configuration=Debug myapp.sln

Instrument output assemblies:

ncover instrument debug\my*.dll debug\my*.exe

Optional re-signing:

for %i in (my*.dll; my*.exe) do sn -R %i mykeyfile.snk

[build distribution bundle or installer, distribute, and test]

Test Machine Commands

Typically the test machine will have NCover Collector installed, and the commands below will check out a floating license for Collector and make sure that the data is synced before releasing that license.

::///////////////////////////////////////////////////////////
:: Deploy software from build server with
:: command(s) unique to your situation
<doDeployCommands>

:: Connect the Collector to Code Central
:: The connection only needs to be completed once
:: but redundant connection is not harmful
ncover connect --server=http://<mycodecentralip>:11235 --user=<login> --password=<pwd>

:: Check out a license for collector
ncover lic-request --floating

:: Disable sync for optimization of collection
ncover sync-disable --project=<Code Central Project for testing>

:: Optionally delete all old coverage data:
del *.ncprof

:: Execute your testing commands and await completion
:: command(s) unique to your situation
<dostuff here>

:: import all coverage files created
ncover import --project=<Code Central Project for testing> --file=*.ncprof

:: re-enable syncing
ncover sync-enable --project=<Code Central Project for testing>
ncover sync-wait --project=<Code Central Project for testing>

:: release the license
ncover lic-release
::///////////////////////////////////////////////////////////

How Does The Data Get To Code Central?

Data imported to the project on the Collector will be synced to Code Central once the .ncprof files have been imported.

Why Should Synchronization Be Disabled During Collection?

Disabling synchronization during testing and importing is a common optimization which allows data to be grouped and minimized to reduce redundant uploads.

Can An Alternate Location For Coverage Files Be Specified?

By default, the collected coverage files are created in the folder containing the application. This location is configurable via the environment variable NCOVER_COVERAGE_FOLDER. In certain scenarios, it is required that the folder be on the same disk volume as the application being tested. Mapped drives will not work with memory mapped files, so they cannot be the location specified for the coverage folder.

Log Messages – Windows Event Log

The nature of writing log messages into a foreign application dictates that NCover has no guarantee of file access or stdout. All errors and information logging pertaining to NCover pre-instrumentation can be found in the machine’s Application Event Log.

More To Come

In coming posts we will present specifics about MSBuild scripting and collecting coverage on Windows Store Apps.