NUnit vs MSTest - 2011 Edition

I have tried to be as objective as possible. Disclaimer: NUnit user since 2005-06.

Legend:

• MSTest as an alias for the unit-testing fwk bunded with VS2010 v10.0.30319 throughout this post (although technically it is the just the runner). It's much easier to say than VisualStudio QualityTools UnitTestFramework. For NUnit, I'm using v2.5.9
• Class-Setup/Teardown - to be executed ONCE before/after ALL tests
• Test-Setup/Teardown - to be executed before/after EVERY test

Major Differences

Migrating a test suite between the two runners might be non-trivial depending on your test code. MSTest and NUnit are very different in the way they go about running a set of tests. e.g. Consider a set of 2 classes containing tests - Class A with 2 tests, B with just one. If I insert logging at some critical points.. (I am looking at the default behavior - no customization)

PSR : Free Windows Screen Recording tool

An unsung hero in the MS Toolkit.

It has a weird name - Problem Steps Recorder that is not obvious as a search keyword. It doesn't create a video as the output, rather an HTML containing annotated images. (On the flip side, it doesn't eat up a lot of space and doesn't lose much information).

This one ships with Windows7. To invoke, Start > Run > psr 

This should pop the very intuitive UI. Here's a 3 min video walkthrough. I just learnt you can annotate issues as well. 

Brilliant!

Review : Parasoft DotTest Static Code Analysis

Everyone agrees that static code analysis is good. However it isn't in widespread use. It has become easier with FXCop / Code Analysis built into VS2010. Personally I'm for "educating developers" over SCA. 

Focus on Prevention over Post-facto Inspection, just like Bugs. 

• Make sure that the number of warnings reduce over time.
• Make sure that the code isn't being engulfed by green suppression markups. Refine the set of rules as needed.

Cannot stress this enough, To avoid SCA pain (drowning in a stormy sea of SCA warnings) START EARLY.

I was asked to come up with a good set of SCA rules for acceptance test code. Here's the specific tool I'm looking at -  Parasoft dotTest (I'm using v9.1).

Setup is easy.. Liked the following aspects.

• You can install into a separate profile i.e. avoid messing up your default VS2010 setup. 
• You can get a floating network license that can be shared across users. You need to check "Release automatically when idle" in the License options.

So I created mine by creating a new configuration.

Main Menu - Parasoft - Test Configurations... Click New

Adding categories one at a time by cross-verifying it with a medium sized codebase ~80 files / ~8K LOC. Took about 2 mins. The rules can be found under the "Static" tab on the right, when your configuration is selected. 

• TIP! The search box can be used to filter the hierarchical view below.
• TIP! Right click on a specific rule to bring up its documentation. Or parameterize the rule if that is supported. 

You can export the configuration then to a file that can be shared across your team.



Since I was looking for SCA on test code (which is run within a controlled env), I turned off the following categories

• COM Guidelines
• Casting guidelines
• Implementing Finalize and Dispose
• Security Inspection
• Serialization Guidelines
• Security Policy Rules
• WPF

That accounts for 96 / 444 rules. Next the rules that didn't make the cut (Rule Id in brackets, use search/filter to find a rule.)

• Keep line length within predefined parameters (80) [BRM.MLL-3]
• Includes whitespaces on the left - in short indentation causes rules to cause noise warnings. Unusable.
• Avoid class, struct, or interface names which are more than 14 characters long [BRM.LONGNAMES-4]
• Default limit too short. Cannot customize. I prefer long readable names over this warning. 
• Always provide appropriate file header (copyright information, etc.) [BRM.FILEHEADER-3] 
• Needs a comment to be the first line in the file - even before the using statements. Does not recognize standard XML comments.
• Does not exclude non-public classes. Noise 
• Avoid unused private methods [CMUG.MU.AUPM-2] 
• Noise warnings. Flagged methods that were being called.
• Avoid using get accessor with side effects [CMUG.PRU.AGAS-3]
• Flagged all properties that were internally calling String.Format. Noise.
• Use an array instead of many parameters [CMUG.MU.AIMP-5]
• Noise again. Recommends turning get(string elementId, string keyPropertyName, string valuePropertyName) to take a string array instead of well named parameters. Counter-productive.
• Make events public [CMUG.EVU.MEP-5]
• No explanation given. Doesn't seem to be necessary - Event Design Page MSDN.
• Put using statements in alphabetical order [CS.USO-4]
• VS2010 sorts usings such that System.* namespaces are grouped above the rest.
  This rule doesn't seem to know about this. A workaround is to override this behavior in VS2010 but then that's something that each developer would have to change.
• NAMING
• Avoid language-specific names for members [NG.ATNC.ALSN.MEMBER-5] : flags up "_constaints" for containing "int"
• Avoid method names that conflict with keywords [NG.WC.KEYWORD.METHOD-3]  : Will not let you have methods called Get, Stop, Select - which are sometimes the most intuitive names.
• Use Pascal case for class names [NG.CAPSTY.PASCAL.CLASS-5] : UIDriverBase is a valid name as per Design guidelines but flagged up by this rule. See  Capitalization Rules for Acronyms.
• Use the property's type name as part of the property name [NG.PRN.APNCTN-4] : This is a "Consider" not a "Do" in the guidelines. Embedding type name in the property is risky w.r.t. future refactorings (e.g. if you change the type of the property with name MonitorIPString from String to StringBuilder. Misleading.
• Use the suffix 'EventHandler' for event handler names [NG.EVN.EHNEEH-3] : Broken. Here's a sample - Message: Change type of event 'X.Driver+AppLaunchedEventHander AppLaunched' or add 'EventHandler' suffix to type 'Driver+AppLaunchedEventHander'.
• Use flags attribute for bit field enum [NG.ETN.UFABFE-3] : Flagged simple 1 member enum with non-plural name as bit-field enums. Broken. 
• Avoid too many function calls from a single function [OOM.FCSF-4] 
• Limit of 10 is easily breached. Non-customizable. Includes property access within the method.
• Use 'GetType()' in the 'Equals()' method implementation [PB.EQL-3] 
• Does not exclude overloads of Equals. e.g. Equals(MyType t) does not need to check type of t.
• Use upper case for method name with name length <=2 [NG.CAPSTY.UPPER.MN-5], [NG.CAPSTY.UPPER.PROP-5]
• Asks me to rename a method called At(params) to AT(params). Was not able to find any design guideline to support this.
• Avoid public default constructor when a non-default constructor is present [CMUG.CU.PUBDEFAULTCTOR-3] 
• Contradicts  Exceptions should implement common constructors [ERR.EICC-4]. The exception rule won.
• Follow the limit for Number Of Static Methods [METRICS.NOSM-1] 
• This one flagged a false +ve. Static Class had 1 static var, 3 public and 3 private methods. Rule violated: 10 is not less than 10. Refactoring often leads to lots of small private methods.

Then there was a bunch that 

• was auto-enforced by C# e.g. Name indexer 'item' or Name event accessors 'add' and 'remove'. 
• Some were guidelines not rules. 
• Some were plain out of date e.g. Provide the same accessibility for the property and its accessors [CLS.ACAC-3] This flags up a common pattern of properties whose setters are private. The explanation points to an outdated (.Net 1.1) version. We've have come a long way since then... 

Turned off another 80-100 too. Finally ended up with 196 / 444 rules enabled.
There is also a report out and code-metrics at the end too. But to summarize, I found Parasoft dotTest to be a bit unpolished.. the infrastructure is there. Analysis is quick enough but the rules need to be improved. Not enough documentation online - except from Parasoft itself. 

Verdict: Passable, but for now prefer Code Analysis that comes bundled with Visual Studio 2010.

String Calculator Kata : My first screencast trilogy

This is my attempt at Roy Osherove's popular String Calculator Katas. 


TDD StringCalculator Kata - Part1 from Gishu Pillai on Vimeo.



The Vimeo Album containing all 3 parts.

I'm using C# (VS2010+Resharper), NUnit and an IDE extension called Beacons 
(the wiki might be a bit behind the downloads tab :))


The videos look best when you have the HD option turned on and Scaling (hover on the video - top-right) turned off.
intermediate practitioners
For best results : download the videos and the subtitles. Use your media player with audio turned off and subtitles enabled at 2-4X. 
I don't do this (ever) so wasn't equipped with the proper audio gear (and coupled with the fact that I do not have an internet voice and can't code and speak at the same time, the audio is passable.)

beginners
Practice it a couple of times before watching. You'll get more out of it.


What I learned from it:

• I need to practice more :) 
• 2 modes of learning in a Kata 
• muscle memory - goal: Speed. 
• exploration - goal: Possible solutions. Take different choices and see how it ends up.
• Automated refactoring (e.g. Resharper for C#) rules. Knowing your IDE and keyboard shortcuts goes a long way w.r.t. productivity
• Streamline all routine work and bind them to keyboard shortcuts - like building, running tests, reusable code snippets/templates, etc

For people looking for screencasting tools on windows, I'd recommend

1. BB Flashback Express - the free edition for screencasting. Export to AVI using ffdshow worked great for me.
2. ShowOff - for highlighting keystrokes and chords. Keyboard Jedi wasn't too great with 64bit & keyboard chords (before my patience ran out).

 

The A Team

Contd from the previous post..

So how do we succeed?
Lock up Enemy #1 - Accidental Complexity

Empower teams to choose

All projects / teams are not the same. Different needs, different tools. This may be difficult in BigCo especially if the tools have already been bought. But make some noise - stand up for your tools, the "users" need to channel some feedback upstream to the "purchasers".

• Explore options instead of resigning to the golden hammer. Prefer tools that don't get in your way. Ones that don't require you to learn yet another proprietary language. Ability to write extensions in your team's native language is a +. This also opens avenues for developers to assist with automation work, if required.
  
• Use existing tools instead of writing your own - they're likely to be functional, tried and tested
• Avoid putting all eggs in one basket. Keep tools/test-runners swappable by defining layers. To migrate to a different test runner, you should only need to migrate your thin tests layer which calls into an automation layer below it, which does most of the heavy lifting. More on this soon..

Collaboration
forms a reinforcing loop against Silos/Specialists. Increasing collaboration decreases opportunities for specialization, which in turn facilitates more collaboration. Of course, the reverse is also true - Silos can finish off collaboration. Only one shall survive, you just have to choose.

Outside-in / Test-first + Wishful Thinking
If you've tried the XP Practice of TDD, you'd know the liberating feeling of letting the unit tests drive the design of the production code. You fix the end-goal, you make it work, make it clean and repeat.
Starting with the test prevents any bias (arising from implementation details, existing tools at your disposal, etc.)

ATDD is the corresponding practice at the system level. However it is not an easy practice to quickly latch on to. So work towards it in baby steps
For starters, concentrate on a DONE definition + writing tests first (before implementation) from the users' perspective on a piece of paper. Make sure everyone has the same idea of DONE before you start the iteration.
As the team matures, you can even move up to ATDP (from the BDD world) where you write test before or during iteration planning & use them for estimation.

WHAT over HOW
Ensures that the test is at the right level of abstraction (the ol' forest over trees adage). It makes the tests shorter and readable. It also works beautifully to bring out the intent (as opposed to the implementation) of the test.
Specify the bare minimum ; things that are relevant to the test at hand.. all other details need to be out of sight.

Stable DSL for testing
You employ wishful thinking to imagine the ideal interface you'd like the system to expose for testing. Since the tests are another client to the system, they can also drive beneficial design changes. The tests stand-in for real users, so if the system is difficult to consume for the tests it follows that it might be for the users too. You could start with a plain C# interface to begin with and then work your way up to a custom DSL. It

• abstracts away incidental details like the GUI, underlying technology and other implementation details.
• abstracts away the tools used for automation from the tests.
  
• decouples the people interested in writing tests from the automation personnel. This allows both to play to their strengths and offers the best of both worlds. e.g. the testers could define the automation interface changes for the sprint and the developers could implement them with production-code like quality.
  
• makes it easy to write new tests with relatively little boot-up time. Writing a test then is just combining the reusable building blocks offered by the test DSL. The tests layer is a good training ground for new hires.
  

Imagine (wishful) a robot that will operate the system for you and think of the commands that you'd issue to the robot. The set of commands are your starting point.
e.g. robot.DoX(params) or robot.GetY()

Programming Skills
Automation is programming. Without good programming techniques and discipline, sustainable pace would be difficult.
This means you need to raise the bar for automation personnel and/or leverage devs. If the team lacks the skill-set required, take countermeasures... Training, get some experts onboard, etc. The average skill level of the team can also be increased by frequent pairing.

Refactoring
Your #1 weapon against complexity. Beck's 4 rules for simple design, the techniques from the Refactoring book (Martin Fowler) + the SOLID principles are a must-read. Top that off with an introductory text on programming (e.g. Clean Code - Robert Martin) and you should be good to go.

Good Naming & Discoverable Design
Taking the time to pick good names goes a long long way. Good names make it easy to find things, facilitate understanding, help zone in on a specific area to change & reduce duplication
This also helps in being able to discover the design / API using just the IDE (learn by intellisense) and programmer intuition. Choose names that are likely to be searched. Operate by the principle of least surprise (code that works as expected the first time around); Avoid hidden side-effects. Document and use team conventions to preserve consistency.

Communicate Intent / Distill the essence
This takes WhatOverHow to the next level. Explaining Why e.g. by extracting another "coarse" method to move up one level OR differentiating sets of inputs by using explanatory names. This reduces the test further to the essence - where the tests turn into readable system documentation... the kind that never gets out of date.

Learning Curve
Refactoring well and often keeps accidental complexity down to manageable levels. The supporting cast of Pairing, a discoverable design, intention-revealing code and a good testing DSL make it easy for new team members to learn the ropes.
This inhibits cargo-cult behavior and the changes made are deliberate/intentional rather than hopeful. Another source of complexity wanes.

Test Maintenance - the last frontier
Test Maintenance like complexity can be minimized not eliminated. As complexity decreases, maintenance effort reduces too.
The testDSL makes it possible to write-and-test the blocks once & use anywhere. Simple designs (no duplication, intention-revealing code, minimal classes) make maintenance much easier.

Transitively, the cost of automation goes down as well.

Let's refactor our diagram to remove the accidental nodes and edges and things get clearer now. Refactoring code is even more rewarding.

Towards better acceptance test automation...

This started out as a sketch of a Causal Loop Diagram (CLD) for bad acceptance test suites... and then it got away from me :) The black arrows indicate "+ve links" e.g. Duplication and Complexity increase/decrease together. The blue arrows ending in dots indicate "-ve links" e.g. Refactoring and Duplication work against each other. Increase in Refactoring causes decrease in duplication.
Click on it to be able to zoom in.

Automated tests != Free Lunch

Disclaimer: I'm a proponent of XP and truly believe it has made me a much better programmer. This post just aims to let readers see through the fog-of-agile caused due to data-transfer loss as it passes from person to person. Please do not misinterpret this as an anti-agile rant. I'm just saying it doesn't work out always unless you're willing to put in the effort to make the change.

Legend
What you hear… (good) the Promised Land

• What was left unsaid (bad.. or downright ugly)

You have an automated regression-safety net to make future changes. Make a change, Push a button and you will know if you broke anything. CI can provide near instant-feedback. Better confidence.

• You have 2-3X the code to maintain. If you have people who don’t care/are too busy/ not passionate about code quality and writing good tests, the tests are the first to put a chokehold on your productivity. Bad tests are as good as (or possibly worse than) having no tests. You could see yourself in a situation where a section of your team is permanently siphoned to keeping the build/tests green. This turns into a daily bottleneck. Tests need to be self-checking, thorough, readable, professionally written, independent, repeatable, concise and FAST. All this takes effort!

Documentation - the tests can be "live specs" of the application - They never get out of date like documentation.

• It takes a significant level of discipline and skill to write readable spec-tests. An essential skill :to see the What and Why without getting entangled in the How. Most teams get this wrong... without noticing it.
  
• Sidenote: The practice of ignoring failing tests is criminal (but usually not punished accordingly) and can lead to misleading specs.

Quality & Productivity: Leads to high-quality production code. Fewer Bugs. More features added / unit time (because you spend less time in debugging and manual testing)

• IF you let the tests drive/shape your design (ATDD and TDD). Client-first design is an unstated requirement.
• The quality of the code is usually a direct reflection of the people writing it. This means you need craftsmen (> 30-50% of a team) and NOT armies of cargo-cult programmers.
• If you're using automated tests exclusively for regression (or getting your 'agile badge'), you'll slowly grind to a halt. Writing tests for "untestable blobs implemented from a non-negotiable handed-down paper design" is frustrating. People can be stuck on “how do I test this?” – Usually leads to silent trade-offs & non-thorough test which will let in bugs and put you in the net negative w.r.t. productivity.

Less rework/thrashing: The dialogue / conversation (that you have with the customer to come up with the acceptance tests) makes it highly likely that you’re building the right thing..

• Assumes that the customers want to collaborate with the developers and testers. This is not often true.. Real users are sometimes real hard-to-find. Even if you manage to snag one of them, you can only procure a small slice of their time. Real users rarely want to write tests.


• If the customers give a “vision” and delegate the responsibility of mapping them to executable specs to the technical team (or worse the QA/testers), you still run the risk of “This is not what I asked for” late in the game. Regular demos may help shorten the feedback time.. but you may still waste an iteration.The magic potion here is collaboration and conversation.. the tests are just a beneficial byproduct.

Simple: Red-Green-Refactor. How hard can that be?

• Sounds simple.. but is deceptive. True OO is a minority sport. Refactoring is a skill that you need to work on. Off the job practice is mandatory.
  You may need to "hire" a good coach for an extended period (I'd say 6 months-1 release) to get the team rolling. Spot trainings/Just-in-time learning won't work for most teams.

Enterprise Library Logging vs Log4Net for a rolling file log : jumpstart and a showdown!

I was looking for a rolling file log and couldn't make my mind between the two. So I embarked on a small quest to see which one is easier to learn and work with.

Flash Cards Kata and the transformation priority premise

I began the new year by resolving to complete the PCL (Peter Siebel's) book. Around the same time, another interesting post by Uncle Bob appeared on the WWW. A discussion on the TDD yahoogroups spurred me to try a Kata - two-sided Flash Cards.

Looked easy enough to test my new-found Lisp skills. Although I was not consciously choosing the next test / step based on the transformation priority, I was curious to know if I had been doing it all along without knowing it.

This innocuous looking kata had me stuck initially - what is the right first step? Trash-canned the first couple of attempts. The third attempt was slow but I succeeded. I did it again in C# just to see if the language had any bearing on my method. This time I was faster because I was familiar with the trail now.

1. I am not wiser with regard to the priority premise - I found I used a bunch of transformations that were not part (bold) of the original list (shown in blue below). As a result, their priorities are unknown. This was the proverbial rain on my parade.
2. Also in the C# variant, I found a couple of tests where I needed multiple transformations to get to the next green & I could not figure out a smaller test.

Note: Check the "speaker notes" in case things get dense especially with the Lisp one. Use the menu button at the bottom-right to get to the presentation link

Attempt#1 Flash Cards Kata in Lisp

TEST User should be asked a question

• {} => statement

TEST A right ans should be confirmed

• P5 (statement->statements) adding more unconditional statements.

TEST A wrong ans should be corrected

• P6 (unconditional->if) splitting the execution path

TEST Diff card and a right ans

• P4 (constant->scalar) replacing a constant with a variable or an argument

TEST Diff card and a wrong ans

• constant => expr

TEST Multiple cards

• {} => loop (statement)

TEST Answers are case insensitive

• exprX => exprY

TEST Answers can have leading/trailing whitespace

• value => expr

Attempt#2 Flash Cards Kata in C#
TEST User is asked the question

• P2 nil -> constant

TEST User is asked another question (triangulate)

• {} => statement *** Play()
• constant => field *** LastQuestionAsked
• void => statement *** Ask() - is there a smaller step here?

TEST A correct ans is confirmed

• P2 nil -> constant

TEST A wrong ans is corrected

• P5 (unconditional->if) splitting the execution path *** Play()
• P4 (statement->statements) adding more unconditional statements
• nil => expr *** User.WhoAnswers
• field assignment *** User.ctor
• constant => field *** User.LastConfirmationReceived
• void => statement *** User.Notify

TEST A diff question and a right answer

• P4 constant => scalar arg

TEST A diff question and a wrong ans

• constant => expression

TEST Two cards

• {} => statements

TEST Multiple cards

• {} => loop
• P4 (statement->statements) adding more unconditional statements *** User
• field assignment *** User.ctor

TEST Case insensitive answers

• exprX => exprY

TEST Trim leading/trailing spaces

• value => expr

The way of the unusual architect - Dan North

An enlightening talk on 'how the human mind works and its corollaries on how programmers work' from the wise Dan North - founder-leader of the BDD camp..

...and so you go from something that was fit for purpose but a little bit ugly to something that never quite delivers. Who here has been involved in what I call a 2-year rewrite anti-pattern ? So we go to the business and we say

Ok.We will rewrite out entire system for some period of 2 years and then everything will be good.
'Wait hang on. What will we get after 2 years?'
Exactly what you've got now.
'Right... and during that 2 years..'
Oh we're locked down! We can't do anything else for you.
'Ok. So you're telling us that we should pay you for 2 years to go no further than we are now and you won't be able to do anything else'.
No.. that's pretty much it.
'Okay...'

and we call that enterprise architecture.

He had me there 3 mins into the talk. Recommended

Beacons - a VS 2010 Extension for TDD

Just uploaded an early version of a TDD visualization extension that I've been working on in my free time. Get an early version of Beacons from the google code project here

The extension only works with NUnit currently. You specify a file path to your NUnit test results file path (which is normally a TestResults.xml file in your .dll/.nunit folder) and you're all set to go.
It can't listen to Resharper test-runner yet. (yet.. if I can figure out how to tap in, should be not an issue.)

To see it in action, check out this video - switch to fullscreen HD


TDD StringCalculator Kata - Part1 from Gishu Pillai on Vimeo.

• Shift+Alt+B, H - Change hats (adding behavior hat OR refactoring hat)
• Shift+Alt+B, P - Play/Pause (when you need to take a break)

If you give it a fair try, I'd love to hear from you. Particularly I'm interested in finding patterns (so share end-of-session screenshots)
More ideas, comments, suggestions - feedback of any kind is welcome.. Post away at http://groups.google.com/group/beacons-tdd

ScenarioLibrary, LibraryTables with Slim Scenarios : Avoid fixture method explosion !

Assuming: working knowledge of Fitnesse.

I've been recommending Slim Scenarios for a while (primarily because they are the Fitnesse equivalent of reusable blocks (methods/functions in your favourite programming language) i.e. tables that can be called from other tables. {Actually they are akin to macro expansion.. but that's nitpicking!)

However soon I ran into a problem, the scenario steps translate into method calls onto the backing fixture class.
e.g.

|scenario         |play  movie|pathToMovie|on home theatre|
|enter            |HDMI1      |mode on tv                 |
|enter            |DTV/CBL    |mode on avr                |
|set avr volume to|-25        |dB                         |
|navigate menu to |@pathToMovie                           |
|press            |play       |on media player            |

Now whenever I call this scenario, the fixture methods 'enterModeOnTv', 'enterModeOnAvr', etc. need to be present on the fixture class. So if Fixtures A, B, c call this scenario, All of them need these methods on them. Bummer!

Now the initial response was to extract these methods onto a helper class. Still I'd need to write some delegating methods on each fixture to delegate to the extracted class. So I raise my hand, admit my ignorance and post to the fitnesse forums. The experts answer in a flash!

The magic answer is the Library Table

FrontPage.HomeTheatreTests.SetUp

|library     |
|television  |
|avr         |
|media player|

The Library table specifies a list of class that should be checked in case the fixture does not contain the required method. So in this case, the order in which the classes would be checked would be [Fixture > MediaPlayer > Avr > Television]. If a matching method is found in any of these classes, the step is resolved.. No more delegating methods!

This allows me to have cohesive helper classes that contain related methods which can be mixed into any page.

Another notable mention is ScenarioLibrary pages - these are pages which are libraries of usable scenarios.

FrontPage.HomeTheatreTests.ScenarioLibrary

|scenario         |play  movie|pathToMovie|on home theatre|
|enter            |HDMI1      |mode on tv                 |
|enter            |DTV/CBL    |mode on avr                |
|set avr volume to|-25        |dB                         |
|navigate menu to |@pathToMovie                           |
|press            |play       |on media player            |

|scenario|start home theatre|
|power   |on  |media player |
|power   |on  |avr          |
|power   |on  |tv           |
...

ScenarioLibrary pages are included if they are a brother/uncle of the current page. Also all ScenarioLibrary pages up the hierarchy are included (allowing the possibility of overriding)

All this leads to very small test pages and it all works!

FrontPage.HomeTheatreTests.CanWorkWhileWatchingMovie

# ScenarioLibrary page is included (helper slim scenario definitions)
# Next SetUp page is included (imports and library tables)

|script    |me                                                   |
|start home theatre                                              |
|play movie|File Manager\USB\C\2 Watch\Iron Man 2|on home theatre|
|start working                                                   |
|after     |30                                   |mins           |
|ensure    |work is done                                         |
|press     |stop                                 |on media player|
|shut down home theatre                                          |

In this example, the fixture Me only needs to have methods that pertain to me working. All methods related to the TV, AVR and MediaPlayer are resolved by the corresponding classes in the Fixture.dll due to the Library table.

public class Me
    {
        private bool _workIsDone ... 

        public void startWorking() ...
        public void afterMins(int timeInMinutes) ...
        public bool workIsDone ...
    }

Pretty cool!

What do I use for acceptance tests (ATDD) : NUnit vs Fitnesse ?

Lately I've had some teams in my getting-agile ecosystem debating on which tool to use for acceptance testing. Now my first response is always "It depends" (You really can't go wrong with that. :) ).

Notation: I'm gonna use

• NUnit when I actually mean any member of the xUnit family of unit-testing frameworks. 
• Fit/Fitnesse stands for itself and now Slim/Fitnesse.
• Customer means the XP's Customer role or Scrum's Product Owner role.

Fit-Fitnesse was the de-facto tool for automated acceptance tests till sometime ago.. Around 2009, there were strong murmurs that Fitnesse isn't working that well (read as some teams weren't executing very well and Fitnesse was part of their toolkit ).

So what does it "depend" on ??

How to get Thoughtworks' White to LogStructure

Struggled for some time getting it to work. I've been playing with Thoughtworks' open library for UI Testing for a while.

Sometimes UISpy doesn't get you to the UIElement hierarchy and you need to take out the big guns ala White's window.LogStructure to dump out the UIElement tree.
As always it's simple when you know how.

Step#1: Modify the app.config of your executable to include the sections related to White. See a sample file on White's page here.

Step#2: Add the following line to your AssemblyInfo.cs file

[assembly: log4net.Config.XmlConfigurator(ConfigFile = "Log4Net.config", Watch = true)]

Step#3: Add a file called Log4Net.config under your assembly (configure it such that it is always copied to the output folder). This contains the configuration information for Log4Net (e.g. the following config logs to console and to file named example.log in your output folder.). That's it you should now be able to see/capture window.LogStructure output from White. Godspeed !

<log4net>
  <appender name="Console" type="log4net.Appender.ConsoleAppender">
    <layout type="log4net.Layout.PatternLayout">
      <!-- Pattern to output the caller's file name and line number -->
      <conversionPattern value="%5level [%thread] (%file:%line) - %message%newline" />
    </layout>
  </appender>

  <appender name="RollingFile" type="log4net.Appender.RollingFileAppender">
    <file value="example.log" />
    <appendToFile value="true" />
    <maximumFileSize value="100KB" />
    <maxSizeRollBackups value="2" />

    <layout type="log4net.Layout.PatternLayout">
      <conversionPattern value="%level %thread %logger - %message%newline" />
    </layout>
  </appender>

  <root>
    <level value="DEBUG" />
    <appender-ref ref="Console" />
    <appender-ref ref="RollingFile" />
  </root>
</log4net>

An effective test strategy to grow applications

This is a post to summarize my understanding (as of today) of an effective (low ceremony) way to build applications.
Let's take a look at (a slightly modified version) of Mike Cohn's layered test pyramid.



The choice of a pyramid indicates the robustness/stability of a lower layer directly affects the effectiveness of the upper layer. Also the number of tests decreases as you move up. As the agile testing book says ROI is maximum at the bottom (speed of feedback over time invested) and wanes towards the top.
e.g. without robust unit-tests, DSL tests or GUI tests would catch a bunch of errors without the essential feedback needed to fix it quickly. More and more errors would make it up to the middle and top layers, where it is more time-consuming/expensive to find-n-fix. Nothing replaces well-written, quick, professional unit tests.
To prevent ambiguity and misinterpretation, let's go over each layer.

Review : Growing Object Oriented Software - Guided by Tests: Freeman, Pryce

Recommended... best book on TDD in a while.

Just finished this one today... I got to fess up - that I am a "classicist". I've seen some real spaghetti tests involving mocks.. But I've gotten some periodic tension from the people on the other side of this religious fence of TDD. Since this book comes from "the torch-bearers of the mockist camp", I picked up a copy and tried to keep an open mind. And I although I haven't switched camps.. I now see the light. There is no one right path but the authors have placed some industrial strength beacons on this path and teach you how to use mocks and interaction tests to "grow" applications driven by automated tests. (However I still maintain that in untrained hands, Mocks (like C++) can be very dangerous and quickly lead into tarpits. Easy to abuse. However in the right hands, they can be used to write very expressive code & enable rapid and iterative development. ).

Some of the key takeaways for me were:

• Developing outside-in ATDD: writing a 'walking skeleton' first , spotting objects and roles (relationships) between them, TDDing these objects and "folding them back" into a running app was a new method to me.
• Listening to the tests, Focus on readability, tests with helpful diagnostics / failure messages, eliminating or reducing the irrelevant part of the test, use of builders + drivers to keep the tests concise
• The chapters on threads and asynchronous testing are home-runs. Don't leave for work without it. The last third of the book was worth the price of the book (It isn't yet available in India so had to get it from overseas).

There is a lot of real world knowledge that the authors have distilled down to a kind of 'secret sauce'.

Contract Tests with Slim/Fitnesse : Running a test suite against multiple implementations of an interface

Recently I had a need for something similar to "xUnit contract tests" in Fitnesse (defined as: a test suite, which any object must satisfy so that it can be considered as a valid implementation of a Role. I got the term from one of J.B.Rainsberger's talks. )

To illustrate my point, let me cook up an example.

Imagine a contest for BoneyM impersonators. We would quiz each impersonator with a set of questions that they must answer correctly to win the contest.

i.e. we have a set of tests (questions) for each implementation of an Impersonator (Role). We also have multiple impersonators that I want to validate against the contract tests. The tests need to be dry - since they don't change for each implementation.

Implementing case-insensitive lookup tables (Dictionary) with string keys

The Dictionary type in .Net takes in a comparer as a constructor parameter. So you could pass in StringComparer.OrdinalIgnoreCase and you're done.
I used to upper-case keys while adding it to the lookup table earlier and again when performing a lookup - to achieve case-insensitive string lookups. I remember reading some article about String.ToUpper() being optimized for such scenarios. Now I was unsure - which one is better ?

WPF - Find a child control of a specific type or satisfying a given predicate under a parent control

Ideally you shouldn't be doing this in WPF and MVVM land. However there is an exception to every rule.. mine was I had to set the focus and put a specific cell in a grid into edit mode when the user clicked a button.

Set keyboard focus when the user begins editing a wpf datagrid templatecolumn ?

The problem:
When I begin editing a grid cell, I'd expect the keyboard focus to be set the edit control for the cell. In case of a textbox, I'd also like the existing contents to be selected by default ; so that i can begin typing and set a new value quickly.