Building C# project with MSBuild and Jenkins, including quality metrics with SonarQube

In two previous posts I outlined how I set up Jenkins to run a build based on which branch was built in source control and deploy it to a version on AppEngine based on the branch, and also how I set up SonarQube to perform static analysis on the code.  It got plenty of owwws and aaahs from my colleagues but there was a gorilla in the room: This was all based on Java... and most of our projects are in .NET.  It's all well and good for the AppEngine pipeline, but how are the other developers going to leverage this kind of build-test-deploy automation if everything they work on is C# and VB?  Thus was the gauntlet laid down: get this working for Microsoft.

So my challenge was to recreate Push-to-Deploy for .NET, so I had to take a look at the tools I figured I would need (this article was helpful):

• Obviously MSBuild would replace Maven as the build tool.  Of course, I actually knew nothing about how MSBuild actually compiled the projects.  I just hit "build" in Visual Studio.  
• Jenkins had a plugin to run MSBuild, and supported Windows batch files.  But while the Java pipeline could run on the local Jenkins instance (running on Ubuntu Server 14.04), obviously this wasn't going to work here.  Enter the Windows Slave.
• Testing and coverage would be done by NUnit and OpenCover.
• Static Analysis would be done by SonarQube using FxCop and StyleCop.
• Since there was a substantial JavaScript component, run JSLint.

This was a pretty huge undertaking for me as I'd never used most of the tools I was going to need.  So I started simple: Just get the slave node set up and connected, and cloning the repo.  Baby steps.

Initial Set Up of the Windows Slave

Step one was to spin up another VM, this time with Windows Server 2012 Datacenter R2 (NOTE: you can't name your machine "windows-slave"... not because "slave" is offensive, but because "windows" is reserved... just a heads up...). I didn't want to make it too beefy (I wiped out my credits from my Ultimate account last week so I was gun shy), so I went with a Basic A1 (1 core, 1.75G memory).  I figured if it bombed out I could always bump it up.  Since I had stuff I needed to be able to download and install, I turned off the Enhanced Security for IE11 feature (otherwise it would ask me to give permission to every single website I visited, which got super old.  Tried installing Chrome but to my surprise it was super slow compared to IE... go figure.  Installed Git for windows and Java 7 SDK, then started looking into what Jenkins needed.  

The first thing was to set up a slave node. So we navigate through Manage Jenkins -> Manage Nodes and to start we have the master node.  Adding a new Dumb Node and setting it up are pretty straight forward:

At this point I was confused. I was expecting to give Jenkins a URL to point at... I mean, how else would it know where the slave was?  The answer, is "Don't call us, we'll call you"... that is, the slave node contacts Jenkins from the Java Web Start connector to say "Here I am!".  So I RDP into the slave machine, open up the Jenkins page in IE, and navigate to the slave node:

Click the orange "Launch" button and it will launch the JNLP agent, which will connect back to the Jenkins server.  The agent will let you install it as a service, which is handy (don't have to manually start all the time). 

At first I didn't realize where it was getting the port number from, so it was getting fouled up because it was random (and Azure needs specific port numbers to open up).  I got this changed by going to Magange Jenkins -> Configure Global Security and changing "TCP port for JNLP slave agents" from Random to Fixed and specifying a port (it's the option at the very top, you can't miss it).  

So to test the set up I created a job that pointed at my BoxBlaster GitHub repo and just did a pull.  The first few builds failed due to permission issues, which were mentioned in some of the instructions I found but apparently I missed.  So I opened up the properties for the Jenkins Slave service, and in the log no tab gave the service permission to interact with the desktop:

Once I got that figured out, I got my first green ball of happiness, whoohoo! Oh but I was only just getting started...

Firing up MSBuild

Working with MSBuild was a new experience for me, and it took a ton of trial and error to get it working right.  The first thing to do was create an MSBuild "installation" just as I'd done with Maven and SonarQube.  I installed the Win7 and Win8 SDKs on the slave machine, as well as .NET 3.5 (needed by the JNLP agent).  These also include MSBuild, so we can specify the path in the build:

Then we can add an MSBuild Build step to our job:

Don't worry about the parameter for now, I'll cover that when I go over StyleCop

Or course, it isn't going to magically all work right out of the box.  First of all, the "targets" used by MSBuild have to be installed on the machine.  Turns out that in order to get the web targets, you have to install (at least) Visual Studio Express for Web.  Before I figured that out I kept getting this error:

C:\Jenkins\workspace\MSBuildTest\BoxBlaster\BoxBlaster.csproj(125,3): error MSB4019: The imported project "C:\Program Files (x86)\MSBuild\Microsoft\VisualStudio\v11.0\WebApplications\Microsoft.WebApplication.targets" was not found. Confirm that the path in the <Import> declaration is correct, and that the file exists on disk.

I got to build #12 before I finally got MSBuild to work.  Now it was time to include some static analysis in the build process.

Using StyleCop, FxCop, JSLint, and SonarQube

I decided to start with StyleCop, and since I'd never tried to use it before, I fell back to my good friend Google to find out how to use it.  I found my answer on Stack Overflow (per the norm).  I knew that I didn't want to change anything about the project directly to make it work with the build process (so setting stuff up in Visual Studio on the dev machine was out).  But the SO thread pointed me in the right direction; while they were using nuget from the dev console within VS, I knew that nuget also came with a command line interface that could be used from a script.  So I included a couple lines in the batch file that ran before MSBuild:

cd %WORKSPACE%\BoxBlaster
C:\nugetCLI\nuget install stylecop.msbuild

This changed into the BoxBlaster directory and uses the command line nuget to install Stylecop.  This was the easy part to figure out.  In order to run StyleCop, you have to have targets for MSBuild, which normally live in the csproj file.  Installing StyleCop isn't enough, these target elements need to be there for MSBuild to know to run it, but I didn't want to change the project file.  Turns out I didn't need to.  There is a command line argument for MSBuild that will let you specifiy custom targets at runtime; this one got me working:

/p:CustomAfterMicrosoftCSharpTargets=%WORKSPACE%\BoxBlaster\StyleCop.MSBuild.4.7.49.1\build\StyleCop.MSBuild.Targets

Now, in a production version I would probably not want to hard code the path to the target file since a change in the StyleCop version (which is reinstalled for every build) would break this path, but for now it got me going.  The last step for StyleCop was publishing the results, which was easily done with the Violations plugin for Jenkins:

Setting up the Violations plugin.  Some of the options I omitted for brevity...

The Violations Report on the job page

After getting StyleCop working, I started playing around with SonarQube.  First I tried the built in "SonarQube" post-build task, but that only works for Java projects.  Instead, I needed to use the "Invoke Standalone SonarQube Analysis" build step.  You need to set the sonar properties manually here, and I found help in the documentation and on the web. My properties ended up looking like this (I'll get to the code coverage stuff in a bit):

One thing I ended up spending a lot of time on was trying to run StyleCop and FxCop first and then reusing the reports with SonarQube, but it turns out that the Sonar properties allowing that "sonar.stylecop.mode" and "sonar.fxcop.mode" aren't supported anymore (can't find the reference anymore but this seemed to be the case). Turned out it was pointless because StyleCop and FxCop are baked into the SonarRunner, you just have to turn the rules on at the SonarQube server.  It took me a whilt to figure that out... I kept getting these messages in the logging from the runner:

10:14:26.202 INFO  - All StyleCop rules are disabled, skipping its execution.
10:14:26.203 INFO  - All FxCop rules are disabled, skipping its execution.

Once again Stack Overflow came to the rescue.  You have to log in as an admin to the SonarQube server, then go to Rules.  Search by language (C#), for me there were 442 rules.  Then select "Bulk Change" in the upper right corner, "Activate In" and then select "Sonar way - C#".  Boom, all the rules in the book are active.

Before I figured out the rules bit, though, I tried to get JSLint working to analyize my frontend JS code.  This should have been pretty straight forward, but I got hung up because I didn't realize that the "log file" parameter was required, and it would blow up if you didn't specify it.  I got a couple of these nasty exceptions before I found an answer:

ERROR: Build step failed with exception
org.mozilla.javascript.WrappedException: Wrapped java.io.FileNotFoundException: C:\Jenkins\workspace\MSBuildTest (Access is denied) (jslint.js#5732)

Then I broke a couple builds trying to publish the JSLint results with the Violations plugin, but JSLint wasn't cooperating (apparently my JS was so bad it was basically erroring JSLint out, so the report was ill formed for what Violations expected... or something).  After a couple tried I gave up and just published it with teh JSLint plugin.  This is the corrected set up:

After I got JSLint squared away is when I decided to take a serious look at getting FxCop going.  My first attempt failed because I didn't set up an FxCop installation in jenkins (after installing it on the slave):

The setup in the build was pretty straightforward:

 

So I had FxCop working on it's own but I wanted the analysis in SonarQube so I revisited what I had going on with the runner.  Builds #34 and #35 failed because I didn't have the Analysis Bootstrapper plugin installed on the SonarQube server. Finally on build #36 (apparently when I figured out the bit about the rules needing enabled on the server), I was getting the sonar runner to run the StyleCop and FxCop rules and seeing the results on the SonarQube dashboard.  Yay!  Now, about those unit tests...

Nunit and OpenCover (and nuget package restore)

At this point the SonarQube dashboard was giving me dreary grey boxes because it wasn't getting any test coverage data... which had a lot to do with the fact that there were no unit tests.  So the first step was to set up some unit tests.  Mind you, I hadn't made any code changes to this project in over six months, and in the mean time I've learned a thing or two about good repo management.  When I went to add the testing project, the build kept breaking because or different package versions between the main project and the test project. When I updated the main packages, I noticed they were getting pushed to the repo.  I've since come to avoid the practice, so I removed them from source control and pushed the changes.  In the back of my mind, it occured to me that MSBuild doesn't do the package management piece, nuget does, so it came as no surprise when the build broke because none of the packages were found.  I already had nuget around in the project for installing StyleCop, and after some trial and error I figured out to use the "nuget restore BoxBlaster.sln" command.  Problem solved. 

That bit of set up out of the way, I could get to the unit tests.  I wrote up a trivial unit test in the test project (I'd like to do some real SignalR unit tests, but for this I just needed something), and included another windows batch file in the build to call nunit-console, directing the output to an xml file.  After a failed first attempt (gotta quote those paths with spaces) I got it to work on the second run.  Easy peasy.  But now I needed coverage information.

The first step was to install OpenCover, and it turns out nuget will do that.  So added "nuget install OpenCover" to my opening batch file:

This is the final version of the batch file that runs as the first build step.

... and called it later from the batch file I had running nunit.  It took me four builds to get this green, but while it was running, it wasn't "working".  I kept getting this result:

Committing...
No results, this could be for a number of reasons. The most common reasons are:
    1) missing PDBs for the assemblies that match the filter please review the
    output file and refer to the Usage guide (Usage.rtf) about filters.
    2) the profiler may not be registered correctly, please refer to the Usage
    guide and the -register switch.

Which was definitely not what I was looking for.  Using a template I found online, I had the nunit execution happen from a dynamically created bat file which was then set as the target for OpenCover.  My first thought was that the registration wasn't working.  I found some info that seemed promising on OpenCover's github page, so I tried running the JNLP service as an admin user.  No change.  Tried adding "regsvr32 x64\OpenCover.Profiler.dll" to the batch file and it was stalling the build (regsvr32 needs the "-s" option otherwise it pops up a dialog box).  I figured out the "-s" bit, build was running again but still no results.  Some more tinkering and half a dozen builds later I finally figured out that nunit needs the /noshadow option (clue here too). Now I have coverage data. Hurray!  Add the last couple Sonar properties to point it to the reports generated by NUnit and OpenCover, and now I have the bright, colorful landing page again:

Here is the script that makes NUnit and OpenCover work, it runs just before SonarQube:

Still to do...

The only thing left to do in my "Build-Test-Deploy" pipeline here is the "deploy" part.  The simplest way to do this with Azure is to set up source control deployment, in which case deployment is just a git push to the Azure repo.  It SHOULD also be possible to use the "publish profile" to do a deployment, perhaps using PowerShell, but that is a rabbit hole for a future post.

Summary

So, at a glance, here is everything this build job does:

• Connect to Windows slave machine
• Clone the source control repository when notified of a change from GitHub
• Execute a windows batch command [Set up] (Build step)
• set the path to include MSBuild (probably not needed)
• restore nuget packages for solution
• nuget install OpenCover
• cd into main project and nuget install StyleCop
•  Build solution with MSBuild (Build step)
• include CL arg to include StyleCop targets
•  Run JSLing (Build step)
• Don't forget to specify the log file
• Execute FxCop (Build step)
• Requires you set up an FxCop installation in Manage Jenkins -> Configure System
• Execute a windows batch command [Testing] (Build step)
• cd into OpenCover directory (created when we installed earlier)
• dynamically create nunit batch file ( echo contents > nunit.bat )
• register OpenCover profiler with regsvr32 with "-s" option
• run OpenCover with nunit.bat as target, specify output file
• Invoke Standalone SonarQube Analysis (Build step)
• specify properties in "Analysis Properties" section
• enable Visual Studio Bootstrapper plugin with sonar.visualstudio.enable=true
• point nunit and opencover reportPaths to files generated by testing batch file
• Publish JSLint results (Post-build Action)
• Checkstyle results wants the log file we specified above
• Report Violations
• Point at fxcop and stylecop results files generated above.

The Windows slave needed the following prep:

• Install Visual Studio Express for Web (to get targets for MSBuild)
• Install .NET 3.5 (required by JNLP)
• Install Java 7 JDK
• Install JNLP from Jenkins -> Manage Jenkins -> Manage Nodes -> Slave node page 
• Install Win7 and Win8 SDKs
• Install FxCop

SonarQube server needed the following plugins installed

• C#
• StyleCop
• Analysis Bootstrapper for Visual Studio Projects

Jenkins needed the following plugins installed:

• FxCop Runner plugin (for stand alone FxCop execution)
• JSLint Report Plug-in
• MSBuild Plugin
• NUnit plugin (not strictly necessary since I'm reporting on testing through SonarQube)
• Violations plugin
• Windows Slaves plugin