Condition Coverage Explained in IL
NCover calculates code coverage by breaking methods down into blocks of code which are contiguous groups of instructions (e.g. there are no control-flow statements that divert program control into or out of those blocks). These contiguous blocks are called “branch segments.”
Branch segment blocks are connected together by control flow statements that direct program execution to the next block of code. Branches can have multiple “out-edges” as well as multiple “in-edges”. NCover tracks the unique combination of out and in edges for blocks. These are called “condition edges.”
Branch coverage is calculated as the percentage of branch segment blocks that are executed in a method. Condition coverage is calculated as the percentage of condition edges that are executed in a method.
Through the rest of this post, we’ll describe an example of how code gets broken down into these parts during the coverage analysis process in order to calculate code coverage metrics.
In the method below NCover defines 9 branches and 12 conditions:
public string GetBackgroundColor(string foreground_color) {
string background_color = "black";
switch (foreground_color) {
case "Red":
background_color = "Yellow";
break;
case "White":
background_color = "Green";
break;
case "Blue":
background_color = "Gray";
break;
}
return background_color;
}
Natural Language Interpretation
Looking at this from a natural language perspective we see a switch statement with 3 expected case values. If we are astute in our interpretation of C# we would also assume there is an unseen case here for the default or unexpected cases. But generally speaking we’d anticipate this code to certainly have no more than 4 conditions: Red, White, Blue or something unexpected.
CLR Interpretation
This is a prime example of code that looks quite dissimilar when translated to IL. The underlying IL of .Net does have a switch statement, but in the case where a string is the object of the switch the underlying IL is converted to an IF structure and is not an optimized switch statement.
Here is the IL for that method:
.method public hidebysig instance string GetBackgroundColor (
string foreground_color
) cil managed
{
.locals init (
[0] string background_color,
[1] string CS$1$0000,
[2] string CS$4$0001
)
// branch 1 -- initialize return var and skip out of switch if string is empty
IL_0000: nop
IL_0001: ldstr "black"
IL_0006: stloc.0
IL_0007: ldarg.1
IL_0008: stloc.2
IL_0009: ldloc.2
IL_000a: brfalse.s IL_004d // jump over checks if there is no value supplied
// branch 2 -- check to see if the supplied string is Red
IL_000c: ldloc.2
IL_000d: ldstr "Red"
IL_0012: call bool [mscorlib]System.String::op_Equality(string, string)
IL_0017: brtrue.s IL_0035
// branch 3 -- check to see if the supplied string is White
IL_0019: ldloc.2
IL_001a: ldstr "White"
IL_001f: call bool [mscorlib]System.String::op_Equality(string, string)
IL_0024: brtrue.s IL_003d
// branch 4 -- check to see if the supplied string is Blue
IL_0026: ldloc.2
IL_0027: ldstr "Blue"
IL_002c: call bool [mscorlib]System.String::op_Equality(string, string)
IL_0031: brtrue.s IL_0045
// branch 5 -- all other values skip to return
IL_0033: br.s IL_004d
// branch 6 -- set the return to Yellow
IL_0035: ldstr "Yellow"
IL_003a: stloc.0
IL_003b: br.s IL_004d
// branch 7 -- set the return to Green
IL_003d: ldstr "Green"
IL_0042: stloc.0
IL_0043: br.s IL_004d
// branch 8 -- set the return to Gray
IL_0045: ldstr "Gray"
IL_004a: stloc.0
IL_004b: br.s IL_004d
// branch 9 -- set the return to original init value of Black
IL_004d: ldloc.0
IL_004e: stloc.1
IL_004f: br.s IL_0051 // always goes to next instruction no branch taken
IL_0051: ldloc.1
IL_0052: ret
}
NCover defines a branch in the IL as a set of instructions that must be executed in sequence with no chance of leaving the instruction set outside of an exception. So when breaking down a method’s IL it is fairly safe to assume that a branch will end with a “br” instruction or an “ret”. The pattern of the IL above is to check the string value against each string constant and then branch forward to another location that sets our return value. In this case the conditional comparison is a branch separate from the assignment.
Conditional Branching (2 out-edges)
Looking at the following branch from the IL let’s understand the conditional branch.
// branch 2 -- check to see if the supplied string is Red
IL_000c: ldloc.2
IL_000d: ldstr "Red"
IL_0012: call bool [mscorlib]System.String::op_Equality(string, string)
IL_0017: brtrue.s IL_0035
At IL_0017 the branch statement will either fall through to IL_0019 or it will branch forward to IL_0035. These two possibilities are the out-edges of “branch 2”.
Unconditional Branching (1-out-edge)
Looking at the following branch of code from the IL we can see an unconditional branch.
// branch 5 -- all other values skip to return
IL_0033: br.s IL_004d
And unconditional branch at IL_0033 will alway branch forward to IL_004d when this instruction executes. There is no other possible out-edge from this location.
Conditions and Landing Pads
Landing pads for coverage in IL occur at the beginning of each branch. Each “br” out-edge has a target landing pad in NCover Condition Coverage. Each time a conditional “br” is encountered that poses the possibility for 2 out-edges; an unconditional “br” just one. Out-edges that go to the same destination are simplified out of the branch logic as shown at IL_004f above.
In the code above we have 4 conditional branches at the following locations:
// branch 1
IL_000a: brfalse.s IL_004d
// branch 2
IL_0017: brtrue.s IL_0035
// branch 3
IL_0024: brtrue.s IL_003d
// branch 4
IL_0031: brtrue.s IL_0045
There are 4 unconditional branches at these locations:
// branch 5
IL_0033: br.s IL_004d
// branch 6
IL_003b: br.s IL_004d
// branch 7
IL_0043: br.s IL_004d
// branch 8
IL_004b: br.s IL_004d
The 8 conditional branches yield a total of 8 out-edges. The 4 unconditional branches add an additional 4 out-edges. It’s worthy to note that there are 5 distinct conditions than can all get the code to IL_004d. In order to satisfy 100% condition coverage at offset IL_004d all of those out-edges must be followed during execution. In turn all 12 out-edges must be followed to achieve 100% condition coverage for the method.
It’s also important to point out that a method with no conditional branches or switch statements will have 0 conditions but will still contain one branch, or set of instructions that must be executed in order.
Visualizing the Code
Let’s boil this down to a picture for simpler description. The following model shows the execution logic of the IL in a flow diagram. Each box of this diagram represents a branch of code. Each arrow in the diagram represents a condition. Notice that there are 9 boxes and 12 arrows corresponding to the 9 branches and 12 conditions that NCover is reporting for this method.
Addendum — IL Switch
While our example code in C# uses a switch statement it is not compiled to a native switch in the IL. The true switch statement in IL has no practical limit to the out-edges. One switch statement can produce a large number of out-edges rather than just the boolean option we see from conditional branch as described above.
Release Build
When you start looking at IL you will quickly see that Release and Debug build are usually different. The following IL is for the same method from above but built in Release mode:
.method public hidebysig instance string GetBackgroundColor (
string foreground_color
) cil managed
{
.locals init (
[0] string background_color,
[1] string CS$0$0000
)
// branch 1
IL_0000: ldstr "black"
IL_0005: stloc.0
IL_0006: ldarg.1
IL_0007: dup
IL_0008: stloc.1
IL_0009: brfalse.s IL_004a
// branch 2
IL_000b: ldloc.1
IL_000c: ldstr "Red"
IL_0011: call bool [mscorlib]System.String::op_Equality(string, string)
IL_0016: brtrue.s IL_0034
// branch 3
IL_0018: ldloc.1
IL_0019: ldstr "White"
IL_001e: call bool [mscorlib]System.String::op_Equality(string, string)
IL_0023: brtrue.s IL_003c
// branch 4
IL_0025: ldloc.1
IL_0026: ldstr "Blue"
IL_002b: call bool [mscorlib]System.String::op_Equality(string, string)
IL_0030: brtrue.s IL_0044
// branch 5
IL_0032: br.s IL_004a
// branch 6
IL_0034: ldstr "Yellow"
IL_0039: stloc.0
IL_003a: br.s IL_004a
// branch 7
IL_003c: ldstr "Green"
IL_0041: stloc.0
IL_0042: br.s IL_004a
// branch 8
IL_0044: ldstr "Gray"
IL_0049: stloc.0
IL_004a: ldloc.0
IL_004b: ret
}
The IL still has 4 conditional branches but it has optimized away one of the unconditional branches. The release build yields 8 branches and 11 conditions in NCover coverage statistics.
[…] Condition Coverage Explained in IL (Kerry Meade) […]