Again, most of these notes are copy-pasted right out of the power points, with my notes written in the //comments:
*******************************************************************************************
MODULE 6
*******************************************************************************************
Define code quality indicators
performing code review, including security and performance analysis
analyzing code for common defects
calculating code coverage
performing architecture validation
Code reviews
- Full workflow backed Code review experience
- Send requests to one or more team members
- Great new Diff experience
- Inline mode
- Side-by-Side mode
- Line highlighting including sub-line highlighting
- Full Intellisense within Diff Tool
- Accept or decline code review requests from others
// understand the business value of code reviews. Shelvesets for code reviews kinda works,
// but now a baked in code review workflow
// TeamReview on codeplex also an old method
// subtle changes to diffing tool shading
Requesting a code review
// in team explorer, in "My Work" click request review, add one or more person, and
// can add description seeking specific guidance.
Accepting and responding
// requests appear under "Code Review", can accept or decline, add comments
// demo
// when you have a pending change, you can make a code review request.
// note that if you aren't using at least TFS2011, you won't see the "My Work" link
// http://social.msdn.microsoft.com/Forums/en-US/3082b94b-cfd8-4f6a-befd-3ba9d0356f64/my-work-section-not-available-in-team-explorer-for-vs11-professional?forum=TFSvnext
//
// code reviews are one of the most effective ways to remove bugs before code ships
// code reviews cannot work against checked in code, so they should be done quickly.
// if you are busy on something else (priority 1 bug) decline so the requester can send
// request to someone else.
// Good way to become a better programmer (code reviewed by senior programmers who can
// point out learning opportunities
Incorporate unit testing
establishing unit test standards
developing a strategy for adding unit tests to existing applications
selecting the unit test type
establishing the code coverage target
Using testing architecture
- Visual Studio Unit Test Explorer
- Command Line Runner
- TeamBuild Unit Test Activity
[Visual Studio Unit Test Platform] (abstraction layer)
- MS-Test Managed
- MS-Test Native
- NUnit
- xUnit.net
- QUnit
// Microsoft unit testing from a decade ago wasn't really keeping up with technology
// abstraction layer allows us to use different testing frameworks using the
// VS Unit Test Platform. Using these frameworks back in the day required writing
// complicated shell scripts.
Developer Focused Unit Test Experience
// anywhere you see a list, there is a search bar at the top. Good for norrowing down
// long lists of unit tests.
// Most important tests are shown first.
// Red-green bar at the top indicates very quickly if there are any tests that failed
// (red) or that all tests passed (green)
// Shows how long the tests took to run
// Failed tests are shown at the top.
// Shows tests from any framework - we don't care what framework the tests are run in,
// so it just shows pass/fail and the test name.
// Summarizes the run details. Time taken and pass/fail rate.
Code coverage in VS2012
Analyze your code coverage with a single click
Analyze for selected tests to help find how specific tests are covering your system
Supports native code (via the MS-Test Native framework)
Works with third party managed & native frameworks
// not economically viable to test 100% of code
// 80% is pretty common
// We want to see the parts that are being tested and not tested.
// we can use this to target our testing strategy to build unit tests for parts of the
// code that are not being tested.
// everytime you have a bug, create a unit test, and you now have a test that covers
// that problem area.
// can even include Coded UI test (even though it isn't a unit test), though these are much
// slower than unit tests and maybe shouldn't be run every build
Code Clone Analysis
New feature in Visual Studio 2012
Looks for semantically similar code using a heuristic search technique
More than just simply searching for exact matches
// copy-paste programmer (junior programmer) - find code that is similar, tweak it to meet
// his requirements.
// can create maintenance problems
// clone analysis finds these similar code bases and targets it for unified testing
//
// finding bugs in production is orders of magnitude more expensive to fix than bugs
// found at built, and finding them in the requirements is even better!
Develop code
- developing maintainable code
- defining coding standards
- defining application architecture
- defining configuration management process and tooling (source control tree, branching
strategy, solution structure, source control/check-in policy)
// trying to get it right the first time will mean spending too much time planning, overly
// complex code, gold plating, etc. Goal should be to accept the change is inevitable and
// to build maintainable code.
// Code standards should be well defined and understood.
Branching Guidance
ALM Rangers Guidance Document
http://vsarbranchingguide.codeplex.com/
// good understanding of branching strategies that are available.
// understanding the role of branching
// release management can be done with branching (version numbers with branches)
// don't put version numbers in a team project, you can't rename it!
// can do powerpoint storyboarding for branching strategy
// no magic solution to branching. Single branch is simplest, but takes a lot of
// maturity. Focus on integration and feature flags. Can be painful. Book "Continuous
// Delivery" covers benefits.
Validate quality
- organizing test artifacts such as test cases, suites, plans, and requirements
- defining test strategy
- managing test execution
- identifying test types and associated tools
- identifying and configuring environments
- Analyzing test runs
// These can be done with MS Test Manager. These will be covered heavily in next exam
Test Plans
Test plans allow you to group your test suites, test cases and configurations into a
convenient group
// Test plan matches an iteration (test plan per sprint in Scrum)
// Test suites could be requirements based, static (we choose exactly what goes in), or
// a query that matches test cases to match with suites.
// Test plan matches configurations. Different OS and Browsers might be defined...
Test Suites
- Test suites are groups of related test cases
- There are three types of Test Suites
- Requirements based test suite
- Static Test Suite
- Query-based Test Suite
Understanding each test type in Visual Studio
In this section we’ll look at each of the following type types
- Unit tests
- Manual tests
- Coded UI tests
- Ordered tests
- Generic tests
- Web performance tests
- Load tests
Unit tests
- Written by programmers for use by programmers
- Can be run manually or automatically, usually as part of a build process. (Eg. Continuous
Integration)
- Become a form of code documentation by detailing the behaviour of application logic
// unit test give us an idea of the expected behavior
Manual tests
- Are lists of steps to be manually performed by a tester
- Manual tests are created as Test Case work items in Team Foundation Server
- Contain expected results that a manual tester must choose to pass or fail
- Manual tests are primarily worked on using Microsoft Test Manager (MTM)
// most testing is done this way. test cases are typically written in excel or word as
// tables. In TFS test cases are work items.
Coded UI tests
- Automated tests of the user interface (UI) are known as coded UI tests.
- Provide functional testing of the UI and validation of UI controls.
- You can create coded UI tests to determine that the UI is functioning correctly after code
changes.
- Can be created by recording your actions or from an action recording on a manual test
// move us away from manual tests. Allows us to drive through the UI. May be a recording
// of a manual test.
// allows testers to keep pace. UI tests can be run in the build.
// in TFS, can easily create a bug task from a failed automated test.
Ordered tests
- An ordered test contains other tests that are meant to be run in a specified order.
- Each test is self-contained and independent.
- Environment state is not retained between each test that is run.
- Do not use an ordered test in which any of the tests have dependencies on the outcome of
previous tests in the ordered test.
// ordered test are kind of chains of tests. testing is better done in as atomic a manner
// as possible
Generic tests
- Generic tests call external programs and tests.
- Use a generic test to wrap an existing test, program, or third-party tool that behaves as
follows:
- It can be run from a command line.
- It returns a value of Pass or Fail.
Load testing
- Used to ensure your web application performs as expected under a range of user loads.
- Can be used for smoke, load and stress testing.
- Visual Studio monitors the system under test and can provide an enormous amount of data.
- Understanding Performance Monitor counters is required to really interpret the test
results.
Web Performance Testing
- Repeatable tests that drive a web application using the HTTP layer.
- Recorded using a web browser but executed using network classes.
- You can analyse the HTTP response.
- Primarily used as the building blocks for Load Tests.
- Should not be confused with Coded UI Tests.
// record with the broswer but not run on the browser. Works well for load testing, because
// it doesn't require the overhead of a browser.
Microsoft Test Manager
- The Microsoft Test Manager (MTM) comprises two Activity Centers;
- Testing Center
- Work with manual test cases
- Lab Center
- Allows you to work with physical or virtual lab environments
- Switch between the two parts using the Activity Center drop down
*** THEY TAKE A BREAK HERE, END OF MODULE VIDEO ***
Customize a team project
- customizing a Work Item
- customizing Work Item Queries
- customizing Team Portal
- identifying areas and iterations
- managing roles and associated permissions
// need to know these in detail for TFS administration
What is a Process Template?
- The template TFS uses when creating new Team Projects
- Defines key aspects of a team project such as work item types, queries and reports
- You can choose a process template during the project creation wizard
- While some aspects of the process can be changed after a project has been created, you
can’t simply change from one process template to another after the project has been
created
// defines workflow, work item types, queries, reports, etc.
Template Architecture
- Process Template Composition
- Build
- Classifications
- Group and Permissions
- Lab
- Reports
- Test Management
- Version Control
- Windows SharePoint Services //stil need .stp file for Sharepoint site to work right
- WorkItem Tracking
- New Team Project Wizard
- used by project leads to create a new team project
- XML Process definition files
- a set of tasks that must run to correctly configure a new team project for the
process.
Available templates
- There are three out-of-the-box templates (at RTM)
- Microsoft Visual Studio Scrum 2.0
- MSF for Agile Software Development 6.0
- MSF for CMMI Process Improvement 6.0
- While the Scrum and Agile templates are similar, the CMMI template is based on a
different MSF process with a different philosophical approach to software development.
Tips before you begin
- Process Template customisation should be planned and well considered before starting
- Always test templates on non-production TFS server instances BEFORE uploading to a
production server
- Avoid changes for the sake of change
- Manage template files in the same way you would work with .NET source code.
- Store in TFS Source repository
- Adopt branching/labelling practises to track versions
// don't ad hoc your customizations. Keep it in version control. Use team project server or
// collection
The customisation process
- Download the default process template that you are either currently using or that most
closely matches what you want
- Rename the template (Important)
- Make a few small changes at a time
- Verify your changes as you go
- Upload to a non-production server to test
- Create a new team project using your template
- Verify the changes you have made are correct
Downloading a template
// Team Explorer -> Process Template Manager -> can download process templates.
ProcessTemplate.xml
// in the root of the process template folder
// <name> Name of the template </name>
// <description> Describe the template </description>
// ignore plugins, groups point to sub-xml elements, which sit in subfolders
// process template root/WorkItem Tracking/TypeDefinitions/ collection of type definitions
// looks at "requirement"
// <FIELD> used to define
// "name" and "refname" must both be globally unique
// "reportable" can be measure, dimension, detail -> affect how the values are reported to
// the data warehouse and OLAP cube
// process template root/Classification/fieldmapping.xml maps from MS Project to TFS
EXAM BEST BETS
- While you do not generally need to know the specifics of how to use every quality tool,
make sure you know;
- what they are
- what value they offer
- what problems/issues they can solve
- You don’t need to know everything about template customization but have a good general
knowledge
*******************************************************************************************
MODULE 7
*******************************************************************************************
Implement an automated deployment process
- architecting an automated build and deploy workflow
- integrating automated tests
- automating deployment into both test and production environments
Implement pre-production environments
- automating the creation of virtual test lab environments
- automating deployment into virtual test environments
// developer may have many different software packages, beta and prerelease versions
What does Production look like?
- Server SOE
- Multiple servers
- Controlled patching
- Firewalls
- Server hardening scripts
- Load balancers
// much different environment between production and developer
Lab Management Workflow
- build
- restore virtual machine env.
// stack of machines, clusters, clients
// can be a mirror of prod
- deploy
- take env. snapshot // lets us get back to known state
- test
Two types of environments
Standard SCVMM
Automated deployment to environment Y Y
Execution of manual and automated tests Y Y
Automated data collection Y Y
Startup and shutdown of virtual machines N Y
Use of Snapshots to revert to different states N Y
Support for Network isolation N Y
Support for physical servers Y N
Support for Hyper-V Y Y
Support for VMWare Y N
// network isolation lets you duplicate the entire environment stack
Lab Environments
- A lab environment is a collection of virtual and physical machines, which you can use to
develop and test applications.
- A lab environment can contain multiple roles needed to test multi-tiered applications,
such as workstations, web servers, and database servers.
- You can use a build-deploy-test workflow with your lab environment to automate the
process of building, deploying, and running automated tests on your application.
Benefits of Lab Environments
- Reduced time between build and when a tester can start testing
- You can use a test plan to run automated tests using Microsoft Test Manager
- You can use a build-deploy-test workflow to test multi-tiered applications automatically
- Collect diagnostic data from all machines, even during manual testing
// advocate for blowing away the envirnoment every night and redeploying apps.
// encourages better behavior and mentality.
Benefits of SCVMM Environments
- Quickly reproduce machine configurations - store collections of VMs that are configured
to recreate typical production environments.
- Reproduce the exact conditions of a bug – When a test run fails, you can store a copy of
the state of your lab environment.
- Run multiple copies of a lab environment at the same time – You can run multiple copies
of your lab environment at the same time without conflicts.
Supported OSes for Virtual Machines
- Windows XP SP3 and later versions
- Windows Vista
- Windows Server 2003
- Windows Server 2008
- Windows Server 2008 R2
- Windows 8 or later versions
// build-deploy-test demo
// use MTM (Microsoft Test Management) to create test environment
// in 2010 creating an environment was a pain, much improved in 2012
Manage feedback between development and operations
- implementing tools to simplify communication
- identifying, measuring, and reporting key analytics and performance indicators such as
cycle time and mean time to repair (MTTR)
- creating appropriate shared artefacts and process to facilitate communication
- standardizing required artefacts for defect reports
- creating strategies for test case reuse (both automated and manual) in production
The Microsoft System Center Family
- Configuration Manager // doesn't have tight integration with TFS
- Data Protection Manager
- EndPoint Protection
- Orchestrator // not likely to be part of ALM story
- Operations Manager
- Service Manager
- Virtual Machine Manager
Troubleshoot production issues
- gathering defect data from bugs in a production setting
- providing actionable data to the development team
- monitoring applications running in production for potential issues
- managing incident response times collaboratively between development and operations
A Common Problem
- Developers typically spend considerable time debugging code
- Often bugs are difficult to find and in many cases we simply can’t reproduce them
- Much time is wasted stepping line by line through code
- IntelliTrace is a feature in Visual Studio Ultimate that is designed to help with this
scenario
// devs need to reproduce environment, and then reproduce the bug. Takes a lot of time
// to find the bug, comparatively little time to fix
Concept
- Basic concept is “capture debugger state information during program execution”
- Think of it like a personal DVR for developers
// records info about debugger state as we run the app
Tester/Developer Scenario
- Tester captures IntelliTrace data while executing test cases.
.iTrace file is attached when the Tester raises a Bug
- Developers are able to find the issue much more quickly with the aid of the
.iTrace file. Once fixed they mark the bug resolved so the tester can retest
Enabling the IntelliTrace Collector
- The tester will enable the IntelliTrace data collector in the Properties for the test plan
Collector Configuration
- The tester can configure the specific information that the data collector will collect
Raising a bug with the IntelliTrace log included
When the tester creates a bug, the iTrace log file is automatically linked if the collector is enabled
Developers receive bug work item
Developers open the .iTrace file
The Importance of Symbol files
- Symbol Files
- .PDB files
- Used during the debugging process
- Includes Source File Names, Line Numbers and Local Variable Names
- You should always keep your symbol files around
- Symbol Server
- Central location for storing your PDB files
- Essentially this is just a file share
EXAM BEST BETS
- Make sure you know the role of Lab Management and what it offers a team
- Run through various Build Definition settings for DefaultTemplate and
LabDefaultTemplate processes
- Be familiar with the capabilities of each System Center product
// Wrap up module
// test taking tips, exam insurance (double shot)
// Pro Scrum Development with MS Visual Studio 2012
No comments:
Post a Comment