Pre-Instrumentation For Windows Store Apps

Pre-Instrumentation For Windows Store Apps

The last three posts concerning pre-instrumentation for NCover have provided a first look at scripting and project integration. A key factor in the addition of this feature to NCover is the growth in popularity of Windows Store (sandboxed) applications. NCover has specific tools to leverage pre-instrumentation for these types of applications. Here are some ways to use pre-instrumentation for Windows Store apps.

Simple Strategy

The process for capturing coverage follows the outline of instrumentation as it was outlined generally for the build process. Those steps are as follows:

1. Instrument assemblies desired for coverage (re-sign if needed).
2. Run tests and manually test application.
3. Import coverage results.

Complex Caveats

Windows Store Apps complicate the process by sandboxing. This means that the application has limited access to resources on the local machine unless it is run in Debug mode and Allow Network Loopback is enabled. If you’re deep into development of Apps this is familiar territory. NCover pre-instrumentation overcomes the obstacles of a traditional CLR profiler by embedding the coverage instructions in the application assemblies and then collecting coverage data into a file on disk. Remember these key factors:

• Instrumented assemblies must be bundled in the appx.
• PreInstrument.dll must be added to the appx.
• Always use an NCOVER_COVERAGE_FOLDER.

Pre-instrumentation by Task for Windows Store Apps

Apps are built like any other .Net application and contain one or more managed assemblies. These assemblies are processed into an Appx file for deployment. Prior to that deployment, the covered assemblies need to be pre-instrumented. The following pre-defined targets will serve as triggers for our pre-instrumentation choreography:

• _ComputeAppxPackagePayload
• _GenerateAppxPackageFile

Using these two targets, add custom targets to handle pre-instrumentation and packaging of the NCover.PreInstrument.dll (see msbuild integration for more detail on project targets for NCover).

Target to Instrument Assemblies

<!-- Create a target to pre-instrument any assembly for coverage.

    A conditional to make this target optional is a good plan.-->

<Target Name="NCoverPrep" BeforeTargets="_ComputeAppxPackagePayload" Condition="'$(NCOVER_ON)'=='True'">

  <ItemGroup>

     <!-- Limit the Item Group to just those assemblies needed. -->

     <CvgItem Include="$(MSBuildProjectDirectory)\$(OutputPath)\*<mystuff>.dll" />

     <CvgItem Include="$(MSBuildProjectDirectory)\$(OutputPath)\*<mystuff>.exe" />

  </ItemGroup>

  <!-- A little feedback to show our list of items to instrument -->

  <Message Text="Pre-Instrumenting assembly: @(CvgItem)" Importance="High" />

  <!-- Setup useful property for executing the command -->

  <PropertyGroup>

     <NCOVER_EXE>c:\Program Files\<whatever the path>\ncover.exe</NCOVER_EXE>

  </PropertyGroup>

  <!-- Call the command to instrument each assembly -->

  <Exec Command="$(NCOVER_EXE) instrument &quot;%(CvgItem.FullPath)&quot;" ContinueOnError="true" />

  <!-- OPTIONAL cleanup of static analysis files. These files are

        embedded into the instrumented assemblies so aren’t needed past

        the step of instrumentation.  -->

  <ItemGroup>

    <Unneeded Include="$(MSBuildProjectDirectory)\$(OutputPath)*.zip"></Unneeded>

  </ItemGroup>

  <Delete Files="@(Unneeded)" />

</Target>

Target for Packaging the Appx

Apps are delivered in a package, so before that package is assembled be sure to add to the package payload.

<!-- This Target will add PreInstrument DLL to the app package -->

<Target Name="NCoverPackageMap" BeforeTargets="_GenerateAppxPackageFile" Condition="'$(NCOVER_ON)'=='True'>

  <ItemGroup>

     <NCoverPackage Include="$(MSBuildProjectDirectory)\$(OutputPath)NCover.PreInstrument.dll">

       <TargetPath>NCover.PreInstrument.dll</TargetPath>

     </NCoverPackage>

  </ItemGroup>

  <Copy SourceFiles="@(NCoverPackage)" DestinationFolder="$(MSBuildProjectDirectory)\$(OutputPath)AppX" />

  <Message Text="Result $(MSBuildLastTaskResult): Finished modifying @(PackageMapFile)" Importance="High" />

</Target>

These targets will be invoked when building the appx package. If NCOVER_ON is true, the designated assemblies are instrumented and the supplemental NCover.Preinstrument.dll is added to the pkg. This same approach is required for Apps and the associated Testing Apps. Testing Apps are packaged independently of the App being tested, so instrumentation and packaging must be handled in both places.

For more detailed options for the NCover Instrument command, see the online documentation. Any of the options for this command can be added as needed.

Collecting Coverage using NCOVER_COVERAGE_FOLDER

It turns out when you run the specialized testing assemblies built for Apps that the testing framework aggressively cleans up the artifacts. In this case simply modifying the manifest to retain *.ncprof artifacts seems the most likely approach, but it is not possible to map a file in the artifacts folder into the memory of a sandboxed application. Instead, NCover allows the application to create the file in the temp folder during execution. This location is met with the least amount of resistance to the manipulation required. The NCover Windows service will monitor for the creation of these .ncprof files and create a hard link to your specified NCOVER_COVERAGE_FOLDER. When the temp files are cleaned up after a test run, the file will remain in the designated coverage folder. This coverage folder MUST be on the same disk volume as the temp folder used by the executing App or TestApp.

Pre-instrumentation Output for Windows Store Apps

Messages and feedback from sandboxed apps are limited. Creating log files that are guaranteed to stay around after the fact of execution is problematic. For NCover Pre-instrumentation with Windows Store apps, the feedback log message and all error messages can be found the Windows event log.