Unit Testing (part 3) – Running unit tests & code coverage

This is the 3^rd in a series of posts about unit testing:

Unit Testing (part 1) - Without using a mocking framework

Unit Testing (part 2) - Faking the HttpContext and HttpContextBase

Unit Testing (part 3) - Running Unit Tests & Code Coverage

Unit Testing (part 4) - Faking Entity Framework code first DbContext & DbSet

 

The new way of running unit tests from within Visual Studio or as part of a TFS build is by making use of the TestSettings.RunSettings file (the old file was TestSettings.TestSettings).

This is the TestSettings.RunSettings file from Microsoft with our exclusions added to it in yellow.

What this does is when the code coverage tool is run it excludes certain code from that process such as the code first migrations files or 3^rd party DLL’s (as we cannot unit test them) so we need to exclude them otherwise our code coverage stats will never reach or get close to 100%!.

 <?xml version="1.0" encoding="utf-8"?>

<!-- File name extension must be .runsettings -->

<RunSettings>

  <DataCollectionRunSettings>

    <DataCollectors>

      <DataCollector friendlyName="Code Coverage" uri="datacollector://Microsoft/CodeCoverage/2.0" assemblyQualifiedName="Microsoft.VisualStudio.Coverage.DynamicCoverageDataCollector, Microsoft.VisualStudio.TraceCollector, Version=11.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a">

        <Configuration>

          <CodeCoverage>

<!--

Additional paths to search for .pdb (symbol) files. Symbols must be found for modules to be instrumented.

If .pdb files are in the same folder as the .dll or .exe files, they are automatically found. Otherwise, specify them here.

Note that searching for symbols increases code coverage runtime. So keep this small and local.

-->

<!--          

            <SymbolSearchPaths>             

                   <Path>C:\Users\User\Documents\Visual Studio 2012\Projects\ProjectX\bin\Debug</Path>

                   <Path>\\mybuildshare\builds\ProjectX</Path>

            </SymbolSearchPaths>

-->

<!--

About include/exclude lists:

Empty "Include" clauses imply all; empty "Exclude" clauses imply none.

Each element in the list is a regular expression (ECMAScript syntax). See http://msdn.microsoft.com/library/2k3te2cs.aspx.

An item must first match at least one entry in the include list to be included.

Included items must then not match any entries in the exclude list to remain included.

-->

            <!-- Match assembly file paths: -->

            <ModulePaths>

              <Include>

                <ModulePath>.*\.dll$</ModulePath>

                <ModulePath>.*\.exe$</ModulePath>

              </Include>

              <Exclude>

                <ModulePath>.*CPPUnitTestFramework.*</ModulePath>

                <ModulePath>.*stackexchange.*</ModulePath>

                <ModulePath>.*antlr3.*</ModulePath>

                <ModulePath>.*microsoft.practices.*</ModulePath>

              </Exclude>

            </ModulePaths>

 

            <!-- Match fully qualified names of functions: -->

            <!-- (Use "\." to delimit namespaces in C# or Visual Basic, "::" in C++.)  -->

            <Functions>

              <Exclude>

                <Function>^Fabrikam\.UnitTest\..*</Function>        

                <Function>^std::.*</Function>

                <Function>^ATL::.*</Function>

                <Function>.*::__GetTestMethodInfo.*</Function>

                <Function>^Microsoft::VisualStudio::CppCodeCoverageFramework::.*</Function>

                <Function>^Microsoft::VisualStudio::CppUnitTestFramework::.*</Function>

                <Function>^SiteDataAccess\.Core\.Migrations\..*</Function>

              </Exclude>

            </Functions>

 

            <!-- Match attributes on any code element: -->

            <Attributes>

              <Exclude>

                <!--Don't forget "Attribute" at the end of the name -->

                <Attribute>^System.Diagnostics.DebuggerHiddenAttribute$</Attribute>

                <Attribute>^System.Diagnostics.DebuggerNonUserCodeAttribute$</Attribute>

                <Attribute>^System.Runtime.CompilerServices.CompilerGeneratedAttribute$</Attribute>

                <Attribute>^System.CodeDom.Compiler.GeneratedCodeAttribute$</Attribute>

                <Attribute>^System.Diagnostics.CodeAnalysis.ExcludeFromCodeCoverageAttribute$</Attribute>

              </Exclude>

            </Attributes>

 

            <!-- Match the path of the source files in which each method is defined: -->

            <Sources>

              <Exclude>

                <Source>.*\\atlmfc\\.*</Source>

                <Source>.*\\vctools\\.*</Source>

                <Source>.*\\public\\sdk\\.*</Source>

                <Source>.*\\microsoft sdks\\.*</Source>

                <Source>.*\\vc\\include\\.*</Source>

              </Exclude>

            </Sources>

 

            <!-- Match the company name property in the assembly: -->

            <CompanyNames>

              <Exclude>

                <CompanyName>.*microsoft.*</CompanyName>

              </Exclude>

            </CompanyNames>

 

            <!-- Match the public key token of a signed assembly: -->

            <PublicKeyTokens>

              <!-- Exclude Visual Studio extensions: -->

              <Exclude>

                <PublicKeyToken>^B77A5C561934E089$</PublicKeyToken>

                <PublicKeyToken>^B03F5F7F11D50A3A$</PublicKeyToken>

                <PublicKeyToken>^31BF3856AD364E35$</PublicKeyToken>

                <PublicKeyToken>^89845DCD8080CC91$</PublicKeyToken>

                <PublicKeyToken>^71E9BCE111E9429C$</PublicKeyToken>

                <PublicKeyToken>^8F50407C4E9E73B6$</PublicKeyToken>

                <PublicKeyToken>^E361AF139669C375$</PublicKeyToken>

              </Exclude>

            </PublicKeyTokens>

 

            <!-- We recommend you do not change the following values: -->

            <UseVerifiableInstrumentation>True</UseVerifiableInstrumentation>

            <AllowLowIntegrityProcesses>True</AllowLowIntegrityProcesses>

            <CollectFromChildProcesses>True</CollectFromChildProcesses>

            <CollectAspDotNet>False</CollectAspDotNet>

          </CodeCoverage>

        </Configuration>

      </DataCollector>

    </DataCollectors>

  </DataCollectionRunSettings>

</RunSettings>

 

Save this file and then in Visual Studio go to Test -> Test Settings and select this .RunSettings file.  And then run the tests.

If you have Visual Studio Enterprise you can automatically run the tests on every build by clicking this icon

We should be running the unit tests frequently and before we merge our code to the main branch.

To use Code Coverage (again an Enterprise thing) Right click in test explorer and choose ‘Analyze Code Coverage for Selected Tests’.

All the tests will then be run and the ‘Code Coverage Results’ window will appear.

Clicking the icon circled above will show all your code highlighted so at a glance as you navigate your code you can see what is covered by the unit tests.

·         Blue = Unit tests are testing it

·         Red = No tests are testing this code yet!

We should be running code Analysis before we merge to main to ensure our code is being tested.

 

Using the same .RunSettings file on a TFS build shows this:

And in the TFS Build Summary report:

(Note: I only have code coverage and unit tests’ running on main branch merges.)