Migrating AppEngine Datastore Entities to a SQL Database: Jenkins Automation

In two previous posts I laid out the process for extracting the entities from the Google AppEngine datastore and then loading them into Azure SQL.  While this process was successful, it was also a bit labor intensive.  I felt pretty certain that the process could be automated, so I set out to script it out and run it from Jenkins.

My initial goal was to complete the entire process, from initiating the datastore backup to pushing to Azure, without modifying the existing app in any way.  So, while the most straight forward way to backup the enties periodically is with a (shocker) cron job, I wanted to try to trigger the backup from outside the app.

Building C# project with MSBuild and Jenkins, including quality metrics with SonarQube

In two previous posts I outlined how I set up Jenkins to run a build based on which branch was built in source control and deploy it to a version on AppEngine based on the branch, and also how I set up SonarQube to perform static analysis on the code.  It got plenty of owwws and aaahs from my colleagues but there was a gorilla in the room: This was all based on Java... and most of our projects are in .NET.  It's all well and good for the AppEngine pipeline, but how are the other developers going to leverage this kind of build-test-deploy automation if everything they work on is C# and VB?  Thus was the gauntlet laid down: get this working for Microsoft.

Automating code coverage and static analysis with Jenkins, JaCoCo, and SonarQube

So now that I had a Maven build running in Jenkins (as outlined in a previous post), I wanted to get some static analysis and code coverage data as part of the build.  So I started looking into SonarQube, and it looked like a good product.  Fired up another VM, found some instructions (and more instructions) and native packages for Ubuntu, and the set up part was relatively painless (relatively being the operative word...).  The hard part was getting it to work with Jenkins.

Multiversion Push To Deploy with Jenkins, BitBucket, and GAE

Round Two in my ongoing battle to get CI to work (Round 1 was a bit of a wash).  I had a pretty straight forward goal in mind this time: Push a branch to BitBucket, initiate a Jenkins build job, and push to a specific version of the app on AppEngine based on which branch was pushed (development, staging, master).

Experimenting with Jenkins, ReviewBoard, and Redmine on Azure

As the developers at work finished up the last loose ends on the Programming Standards we were tasked with creating at work, one of the last things we talked about was how to enforce the standards.  This led to a discussion of code reviews, and I went down the rabbit hole, going so far as to read SmartBear's free 150 page ebook on the subject.  I thought it was a great approach but it seemed that in order to really get the most out of it, you needed a tool.  I wanted to play around with something, so I began looking into what was available for free, and ended up going down a bit of a different rabbit hole: continuous integration.  I wanted to see how Jenkins, ReviewBoard, and Redmine could work together to act as a suite of tools for agile development.  So I spun up some Ubuntu 14.04 instances on my Azure account, and away I went...