Exam Objectives
Quick Overview of Training Materials
Exam Ref 70-486: Developing ASP.NET MVC 4 Web Applications - Objective 4.3
Programming ASP.NET MVC 4 (OReilly) - Chapters 17 (Automated Testing)
Professional ASP.NET MVC 5 (WROX) - Chapter 14 (Unit Testing)
The Art of Unit Testing
Using Browser Link in Visual Studio 2013
Isolating Code Under Test with Microsoft FakesProgramming ASP.NET MVC 4 (OReilly) - Chapters 17 (Automated Testing)
Professional ASP.NET MVC 5 (WROX) - Chapter 14 (Unit Testing)
The Art of Unit Testing
Using Browser Link in Visual Studio 2013
MSDN - Understanding Web Tests
YouTube - Creating Web Test and Load Test using Microsoft Visual Studio
YouTube - Load Testing SharePoint 2013 using Visual Studio 2013
While trivial, the default tests for the home controller actions exhibit the basic qualities of a good unit test (they just need better names).
Automating the running of unit tests is generally pretty easy with modern unit testing frameworks. The NUnit runner can be installed as an add-on to Visual Studio, or the Microsoft Testing Framework runner comes with it already. If tests are fast (as they should be), then running a suite of tests can be done often without causing undue pain to productivity. This is typical output from running a test suite:
One rule of thumb is that a given test should only contain one assertion. This can result in the creation of multiple duplicate tests, however, if you were to create a new test method for every single edge case you were trying to test. NUnit will allow you to create parametrized tests, however, which enables you to run essentially the same test multiple times with different test values. Below is an example that test several different cases:
Microsoft Fakes takes this concept a step further, allowing you to create an entire fake assembly. It's as simple as right clicking on the assembly reference in Solution explorer and selecting "Add Fakes Assembly"
One unique (and potentially hazardous) capability of MS Fakes is the ability to create shims. To understand shims a little better we have to back up a step and talk about stubs and mocks. You see, test doubles come in different flavors. A stub is generally a dumb fake; that is, it implements an interface and just returns what we tell it to return. In other words, a stub is used to control state. Another kind of mock actually keeps track of how it has been interacted with. This sort of fake is used to validate behavior. With these mocks, we might use it in the test and then validate that method xyz was called with such and such parameter 3 times.
Shims take this up a notch. One area where both stubs and mocks can be weak is in situations where there are no obvious seams. A seam, as described in Micheal Feathers book Working Effectively with Legacy Code, is a place where two pieces of software meet and other dependencies can be injected. This is often accomplished with some form of dependency injection, whether it be construction injection, setter injection, factories, what have you. Shims get around a lack of injectibility by replacing code at runtime, so instead of calling dependency X, you call the shim of that code.
This is a very powerful technique, but it has a great deal of potential for abuse. Tests that make use of shims can be fragile, because they aren't black box tests; by their nature they have some knowledge of the implementation details of the code under test, and changes to those implementation details may break the test without actually breaking any externally visible behavior. So shims should definitely be used judiciously. Where they do shine is when adding tests to legacy code that isn't easily testable. They can allow you to get the existing behavior "pinned" with a test suite. The code can then be refactored to be more easily testable, at which point the shim tests can be replaced.
The following code was adapted from Microsoft's examples on MSDN:
In order to use the fake assembly, the offending code must be run from within a ShimsContext. Within this context, the behavior of the shimmed class is defined and the method called. Because DateTime.Now is set to always return Jan 1, 2000, the above test passes.
All the browsers will open your app at the same time.
To refresh all the connected browsers, use the refresh icon next to the browser selector.
PluralSight - Automated ASP.NET MVC Testing: End to End
My four part post on unit testing goes into such depth on the subject, I really struggled with what I could even add here. Whole books have been written on the subject (The Art of Unit Testing is a worthwhile read), so I'll keep it pretty brief and general with a couple quick examples.
Automated tests come in a number of different flavors depending on the level of the application they are working against. Unit tests are the most granular tests, and should only exercise one piece of logic. Unit tests are usually written using a unit testing framework such as the Microsoft Unit Testing Framework or NUnit (my personal choice). The OReilly text gives four criteria for what constitutes a unit test:
- Atomic - tests only one piece of functionality
- Repeatable - test pass/fail should not change based on the environment
- Isolated - does not depend on other tests or other external dependencies (i.e. a database)
- Fast - fast tests can be run often without creating an undue burden
The WROX text has similar criteria, stating that tests should be small (atomic), isolated, and produce an automated pass/fail result. It also notes that tests should only cover the public APIs (in other words, only the public methods on a class). This isolates the test from the internal implementation details and focuses strictly on the contract the unit of code under test is trying to fulfill. If a lot of your classes code is stuffed into private methods that you want to test, it may be worth extracting another class (the class under test may be doing too much).
In The Art of Unit Testing, the author identifies three broad qualities of good unit tests:
In The Art of Unit Testing, the author identifies three broad qualities of good unit tests:
- Trustworthy - when a tests passes or fails, you believe it. No need to double check in debugger
- Maintainable - only test public contracts and keep tests isolated.
- Readable - Tests are stories to the programmer who come after us, they should be easy to understand
The Art of Unit Testing also goes into great depth on why keeping tests small, isolated, and repeatable is beneficial. Definitely well worth a look (much more material than I can try to squeeze in here...).
While trivial, the default tests for the home controller actions exhibit the basic qualities of a good unit test (they just need better names).
[TestMethod] public void Index() { // Arrange HomeController controller = new HomeController(); // Act ViewResult result = controller.Index() as ViewResult; // Assert Assert.IsNotNull(result); }
Automating the running of unit tests is generally pretty easy with modern unit testing frameworks. The NUnit runner can be installed as an add-on to Visual Studio, or the Microsoft Testing Framework runner comes with it already. If tests are fast (as they should be), then running a suite of tests can be done often without causing undue pain to productivity. This is typical output from running a test suite:
One rule of thumb is that a given test should only contain one assertion. This can result in the creation of multiple duplicate tests, however, if you were to create a new test method for every single edge case you were trying to test. NUnit will allow you to create parametrized tests, however, which enables you to run essentially the same test multiple times with different test values. Below is an example that test several different cases:
[Test, Sequential] public void LinearConsumptionFunction_getCurrentSatisfaction_CalculatedCorrectly( [Values(.5, .6, 1.0, .5, .6)] double exhausted, [Values(100, 100, 100, 100, 1000)] int max, [Values(0, 0, 0, 50, 900)] int min, [Values(50, 60, 100, 75, 960)] int expected) { //Arrange LinearConsumptionFunction lcf = new LinearConsumptionFunction(); //Act int result = lcf.currentSatisfaction(max, min, exhausted); //Assert Assert.AreEqual(expected, result); }
Create Mocks
An important aspect of effective unit testing is keeping the tests isolated to the unit of code under test. This can create problems when the code being tested has external dependencies, such as databases or web services. Tests that make use of these external dependencies are more properly called integration tests, and while valuable in their own way, they lack some of the advantages of unit tests. In other cases, it may not even be a problem of communicating with remote services, but the problem may be that collaborator classes haven't been implemented yet. In these cases, these classes could be created easily enough if they existed, but they don't. In both cases, Mocks can be used in place of these resources.
On way of creating a mock is to roll your own. This is how I handled the database dependencies in my four parter on unit testing mentioned above. I created a class that implemented the same interface as the entity framework repository classes, but using an in memory data set instead of the database. Here is a sample of what those tests looked like:
Another way of creating mocks is through a mocking framework. Moq is one such framework. Creating a "Mock" class, passing in the interface being mocked as the type parameter, creates a mock of that interface that can be manipulated for the test. Below is an example that uses Moq to create a test double for a class we hadn't implemented yet:
On way of creating a mock is to roll your own. This is how I handled the database dependencies in my four parter on unit testing mentioned above. I created a class that implemented the same interface as the entity framework repository classes, but using an in memory data set instead of the database. Here is a sample of what those tests looked like:
[Test] public void ApplicationController_Create_Saves_Application_When_Valid() { var db = new FakeUnitOfWork(); IPrincipal fakeUser = FakeUser.GetFakeUser(0); var controller = new ApplicationController(db, fakeUser); controller.Create(TestData.Applications.First()); //asserts Assert.AreEqual(1, ((FakeGenericRepository<Application>)db.Applications).Added.Count); Assert.AreEqual(true, ((FakeGenericRepository<Application>)db.Applications).Saved); }
Another way of creating mocks is through a mocking framework. Moq is one such framework. Creating a "Mock" class, passing in the interface being mocked as the type parameter, creates a mock of that interface that can be manipulated for the test. Below is an example that uses Moq to create a test double for a class we hadn't implemented yet:
[Test, Sequential] public void ConsumableNeed_CalculateSatisfactionTwoTicks_CorrectResult( [Values(0, 6, 10)] int time, [Values(100, 40, 0)] int expected) { //Arrange IConsumptionFunction lcf = new LinearConsumptionFunction(); var testTimeMan = new Mock<ITimeManager>(); testTimeMan.Setup(m => m.getCurrentTime()).Returns(0); var sut = new ConsumableNeed("test", testTimeMan.Object, lcf, 10); testTimeMan.Setup(m => m.getCurrentTime()).Returns(2); sut.tick(); //Act testTimeMan.Setup(m => m.getCurrentTime()).Returns(time); sut.tick(); var result = sut.getSatisfactionLevel(); //Assert Assert.AreEqual(expected, result); }
Microsoft Fakes takes this concept a step further, allowing you to create an entire fake assembly. It's as simple as right clicking on the assembly reference in Solution explorer and selecting "Add Fakes Assembly"
 |
| And just like that, we have a fake System assembly |
One unique (and potentially hazardous) capability of MS Fakes is the ability to create shims. To understand shims a little better we have to back up a step and talk about stubs and mocks. You see, test doubles come in different flavors. A stub is generally a dumb fake; that is, it implements an interface and just returns what we tell it to return. In other words, a stub is used to control state. Another kind of mock actually keeps track of how it has been interacted with. This sort of fake is used to validate behavior. With these mocks, we might use it in the test and then validate that method xyz was called with such and such parameter 3 times.
Shims take this up a notch. One area where both stubs and mocks can be weak is in situations where there are no obvious seams. A seam, as described in Micheal Feathers book Working Effectively with Legacy Code, is a place where two pieces of software meet and other dependencies can be injected. This is often accomplished with some form of dependency injection, whether it be construction injection, setter injection, factories, what have you. Shims get around a lack of injectibility by replacing code at runtime, so instead of calling dependency X, you call the shim of that code.
This is a very powerful technique, but it has a great deal of potential for abuse. Tests that make use of shims can be fragile, because they aren't black box tests; by their nature they have some knowledge of the implementation details of the code under test, and changes to those implementation details may break the test without actually breaking any externally visible behavior. So shims should definitely be used judiciously. Where they do shine is when adding tests to legacy code that isn't easily testable. They can allow you to get the existing behavior "pinned" with a test suite. The code can then be refactored to be more easily testable, at which point the shim tests can be replaced.
The following code was adapted from Microsoft's examples on MSDN:
[TestMethod] public void DateTime_Shim_Test() { int fixedYear = 2000; using (ShimsContext.Create()){ // Arrange: // Detour DateTime.Now to return a fixed date: System.Fakes.ShimDateTime.NowGet = () => { return new DateTime(fixedYear, 1, 1); }; // Instantiate the component under test: var componentUnderTest = new MyComponent(); // Act: int year = componentUnderTest.GetTheCurrentYear(); // Assert: // This will always be true if the component is working: Assert.AreEqual(fixedYear, year); } }
public class MyComponent{ public int GetTheCurrentYear(){ return DateTime.Now.Year; } }
In order to use the fake assembly, the offending code must be run from within a ShimsContext. Within this context, the behavior of the shimmed class is defined and the method called. Because DateTime.Now is set to always return Jan 1, 2000, the above test passes.
Create and run web tests
Visual Studio Ultimate Edition includes the ability to run Web Performance tests. There are a couple ways you can create a new Web Performance test. The first is to add a Web Performance and Load Test project to the solution, which is done the same way you would add any other project to a solution:
Another method is to add a "Web Performance Test" to an existing test project. I used the MVC Music Store application, which had a unit test project already, and just added a Web Performance Test:
Once you've added the test, it's time to record some actions. Selecting the "record" button will open up Internet Explorer with a special browser add in for recording tests. If it doesn't show up, go to Settings and select "Manage Add-Ons" and make sure the recorder add on is enabled:
Then it's just a matter of doing stuff on the site that you want tested. One issue I ran into that through me off for a bit: when I ran the web test the first time, I was seeing tons of extra requests that looked like SignalR polling. This was extra weird because SignalR wasn't installed in this project.. What the heck??
Turns out browser link was the culprit. Once I turned that off, the results were a little bit more like what I expected:
Once the web tests are recorded, they can be used to create Load tests. You add a load test similar to the way you add a web test, except there is a wizard that walks you through setting up the parameters. Before you can actually run the load test, make sure the site is running exactly the way it was set up to run when you recorded the web test. I spun my wheels a bit trying to get Visual Studio to run the app and then run the test but it wouldn't do both at the same time. Trick was not to use debug, but the "View in browser" button right next to it. This is what the load test looks like:
There is a demo on YouTube: Creating Web Test and Load Test using Microsoft Visual Studio, which is an excellent walk through. There is also a longer demo by a couple guys from Rackspace showing how to load test Sharepoint using the load testing functionality in Visual Studio: Load Testing SharePoint 2013 using Visual Studio 2013.
Once you've added the test, it's time to record some actions. Selecting the "record" button will open up Internet Explorer with a special browser add in for recording tests. If it doesn't show up, go to Settings and select "Manage Add-Ons" and make sure the recorder add on is enabled:
Then it's just a matter of doing stuff on the site that you want tested. One issue I ran into that through me off for a bit: when I ran the web test the first time, I was seeing tons of extra requests that looked like SignalR polling. This was extra weird because SignalR wasn't installed in this project.. What the heck??
Turns out browser link was the culprit. Once I turned that off, the results were a little bit more like what I expected:
Once the web tests are recorded, they can be used to create Load tests. You add a load test similar to the way you add a web test, except there is a wizard that walks you through setting up the parameters. Before you can actually run the load test, make sure the site is running exactly the way it was set up to run when you recorded the web test. I spun my wheels a bit trying to get Visual Studio to run the app and then run the test but it wouldn't do both at the same time. Trick was not to use debug, but the "View in browser" button right next to it. This is what the load test looks like:
There is a demo on YouTube: Creating Web Test and Load Test using Microsoft Visual Studio, which is an excellent walk through. There is also a longer demo by a couple guys from Rackspace showing how to load test Sharepoint using the load testing functionality in Visual Studio: Load Testing SharePoint 2013 using Visual Studio 2013.
Debugging in multiple browsers and mobile emulators
It's easy enough to open the application in multiple browsers simultaneously. Simple drop down the browser options, select "Browse With", and select whichever installed browsers you want to open the application with. Some mobile emulators (like electric plum) will show up as options here as well:
All the browsers will open your app at the same time.
To refresh all the connected browsers, use the refresh icon next to the browser selector.
No comments:
Post a Comment