The NC4 Paradigm

Three significant paradigm shifts have emerged for NCover v4 (NC4), bringing with them a new opportunity to capture and understand coverage data for my applications. The most notable shift is the global nature of the NCover Windows service. Additionally, the data model has been centralized while maintaining discrete data collection for each profiled process, but sharing static metadata for assemblies across all projects. Finally, the scope of an execution has been refined to account for individual app domains (think IIS).  This combination of features breaks new ground for team collaboration and enterprise reporting of coverage data.

NC4 eliminates the need to create complex command line statements. NCover v3.x (NC3) has been the faithful companion of dev teams and build servers around the world for over two years. What it lacked in grace, it made up for in flexibility. In these years of service, it has been no secret that complex command line operation has continually been a hurdle, if not an out-right hassle. NCover Explorer (NCE) put a GUI on the command line operation, but essentially the obstacles remained the same — it could be a struggle to produce a working command line.

NC4 installs the profiler globally and allows me to identify through project definitions which processes to monitor. This paradigm resembles the global registration of NCover in version 3, but with significant improvements. This allows multiple processes to be covered simultaneously and allows me to collect coverage on my application during normal usage without artificial techniques to start services and applications in a specialized context.

Whether it is a unit testing command line, like “nunit-console.exe mytest.dll,” or clicking an icon pinned to the Windows taskbar, every .Net process is monitored on startup, so that it can either be profiled or ignored based on my NC4 project settings. Though all .Net processes are monitored on startup, there is no detectable delay in launching any unmonitored application. This is a core innovation that allows NC4 to live at peace with all applications in my normal Windows environment.  The essential change is that I can launch applications without special invocation, be it from the command line or a launch from the start menu, and collect coverage.

NC4 requires me to think a bit differently about my coverage integration. Many users who mastered the NC3 paradigm — which required me to run NCover to start another application which in turn might run automated tests — will find this change to be a more natural paradigm.  It’s OK. I can adapt to simply running the applications I need to cover. I can forget that I’m collecting coverage and focus on the application itself.  Yes, indeed, I’m happy with that.

Also brand-new for NC4 is the project auto-configure feature (fig. 1). Since NC4 is monitoring the launch of all .Net processes, invoking the auto-configure mode of project creation reports back the launched processes and their loaded assemblies. This makes it much simpler to find the agents and child processes which actually load the assemblies I want to cover. What I’ve generally encountered with this feature is the tendency to cover more modules than I would with NC3.  NC4 does err on the side of giving me more coverage, rather than less. With NC3 coverage was more additive. It covered only the assemblies I specified. NC4 will cover all components of a process by default, and let me choose what to trim away.

Fig.1

NC4 dynamically aggregates execution data for reporting.  Storing data with minimal aggregation allows NC4 to merge data more easily across multiple executions, on the fly. In this way, NCover serves as a data collection engine and the GUI provides the analysis paradigms to break the data down by modules, namespaces, classes, methods, and documents. Coverage reports have become live-rendered html, which can be continually updated by new collections or elective data consolidation.

Data consolidation is accomplished through the Merge functionality. Merge was specifically designed to support long-term testing efforts like manual regression testing. I can test a build for a week, collect multiple executions of the same application and merge them to a single execution of my regression cycle to easily show total coverage. What’s more, I can label each execution with its specific context of capture, such as “Regression: Sales Order Entry”. (fig 2) The merging of data in NC4 is non-destructive — each execution remains intact. A merge itself can be discarded and all the contents retained for independent analysis or re-merging for ad hoc aggregation. By default, only one copy of an execution is retained in order to ensure data integrity. That execution can be merged and re-merged as needed, but it is only reported in one context at a time.

Fig. 2

NC4 isolates individual IIS application coverage.  Finally, an important innovation for IIS development is that each app domain for any process is captured in a distinct execution. Upon process completion, all app domains loaded within the process become elemental executions of a single merge for that process. Expanding that merge in the GUI lets me browse the data by app domain. In IIS, app domains correlate to the app pools under which I run different applications. This is perhaps the most requested feature for NC3, and now with NC4 I can easily separate coverage for each web application simply by assigning them independent app pools in IIS.

These three fundamental paradigm shifts for NC4 are opening new possibilities for how and when I can collect coverage data. Coupled with the new products NCover Collector and NCover Code Central, I can even collect remote coverage from other designated users and merge it with my own. The big hurdle of training a user on how to execute an application under coverage has been removed. Enhancements to the data encapsulation (as executions) provide greater flexibility for reporting and data consolidation. Together, they are the basis for a new way of thinking about code coverage in the team environment.  If you’re just collecting coverage on your automated tests, you’re missing the big picture.