My conversations with gullible machines...

NUnit vs MSTest - 2011 Edition

2011-11-14T16:31:00.001+05:30

I have tried to be as objective as possible. Disclaimer: NUnit user since 2005-06.

Legend:

MSTest as an alias for the unit-testing fwk bunded with VS2010 v10.0.30319 throughout this post (although technically it is the just the runner). It's much easier to say than VisualStudio QualityTools UnitTestFramework. For NUnit, I'm using v2.5.9
Class-Setup/Teardown - to be executed ONCE before/after ALL tests
Test-Setup/Teardown - to be executed before/after EVERY test

Major Differences

Migrating a test suite between the two runners might be non-trivial depending on your test code. MSTest and NUnit are very different in the way they go about running a set of tests. e.g. Consider a set of 2 classes containing tests - Class A with 2 tests, B with just one. If I insert logging at some critical points.. (I am looking at the default behavior - no customization)

NUnit

Observations:

Tests are run in sequence (no overlap)
Single threaded : one thread runs all the tests
All tests within a test class are run on the SAME instance
xUnit order :

for each test class

Class setup

for each test

test setup
test
test teardown

Class teardown

MSTest

Observations:

Tests are run in sequence
Multi-threaded even when tests are run sequentially : We can see 4 threads in this example (1 per test + 1 for class-teardowns)
A NEW Instance of the test class is created PER TEST (see 3 diff hashcode values)
MSTest order :

for each test-class

Class setup
for each test

test setup
test
test teardown

Class cleanup for all test classes visited

What does all this mean ?

MSTest differs from xUnit in its implementation of test class setup or teardown. They have to be static methods, which excludes setting up anything in instance variables to be used by subsequent tests. You can promote them to static (aka global) variables but combined with multiple-threads, be careful!
MSTest does not provide deterministic test-class teardown i.e. no guarantees about when it will be run; just that it will be executed. So you need to design your tests to account for this - see what happened in the image above. Instead of Start-Close cycles, I see Start-Start-Close-Close!! I haven't yet figured out a workaround for this.. so if you know please post a comment.

Test-Class Inheritance

xUnit has some patterns that use inheritance to achieve the desired effect. So let's see they fare in this dept.

NUnit : abstract fixture

MSTest : abstract fixture

the abstract fixture pattern - move common methods (e.g. common setup/teardown into a abstract base fixture)

MSTest (right) won't execute Class-Setup/Teardown methods defined in base types. Test-Setup/Teardowns are treated differently. (so you may not be able to layer setup/teardown as shown above ). Also remember Class-Teardown is unpredictable.

the abstract test pattern - all the tests are housed in an abstract class. You then subclass and override a method that provides the actual object to be tested. e.g. run 'contract tests' against multiple implementations of an interface.

NUnit : You can cleanly extract a POCO abstract base type with test methods. MSTest: Additionally, you need to decorate the base type also with a TestClass attribute. (It won't show up in the TestView.) But can be done!

Parameterized Tests (Data driven)

I remember reading (when I first heard of TDD) that tests should be self-contained and shouldn't need to take inputs. Well, we've changed our mind. The need to execute the same test logic with different data sets (without duplication) led to RowTests. NUnit integrated a contributed extension and now there are TestCases.
So let's test if how our client handles authentication.

[TestCase("WrongUser", "Don't care", TestName = "Invalid user")]
[TestCase("Gishu", "Wrong Password", TestName = "Wrong password")]
[TestCase("Gishu", "openSesame", TestName = "Right credentials")]
public void ThrowsExceptionIfCredentialsAreWrong(string username, string password)
{
    var client = new Client();
    var exception = Assert.Throws<ConnectFailedException>(
        () => client.Connect("129.0.0.1", username, password));
    Assert.That(exception, Has.Message.EqualTo("Invalid Credentials"));
}

I love TestNames vs anonymous inline data clumps!!
Lets switch to MSTest. No RowTests built-in. There is a patch but it is the bare minimum (and requires me to mess with the registry) ; more on that after the jump. You can't slot in inline data values - but you can pull in data from a CSV/Xml/DB. Hmm... seems like a Hammer to swat a fly. But I gave that a try too; HOW TO: Create a data driven test. So I use the wizard, more attributes appear. It works but...

Differences:

Does not show me 4 distinct tests in the results (actually the total count circled is wrong..). You cannot name your tests.
You need to pop up test details (another window) for each group to see which tests failed in the group.
Even then, you can't see which inputs failed. Annoying.

The extension mentioned earlier also suffers from the same. So for static inputs, "No great shakes!"
Dynamic Input generation: But we're still not done, what if we want to compute the inputs using code?? Say hello to NUnit's TestCaseSource and ValueSource. You can write methods that generate test cases at run-time based on any arbitrary logic (with all the frills: names, tags/category, expectedResult/Exception).
Once again MSTest - Not Supported.

Parameterized Test Fixtures

Next the big guns.. you don't need them everyday. But when you do, NUnit allows you to really cut down duplication via Parameterized Fixtures and Generic Fixtures. e.g. consider the following convoluted example

[TestFixture(typeof(PhoneX), "Sam", Description = "Model X")]
[TestFixture(typeof(PhoneY), "Htc", Description = "Model Y")]
public class TestAPhone<TPhone> where TPhone : Phone, new()
{
    private TPhone _phoneToBeTested;

    public TestAPhone(string name)
    {
        _phoneToBeTested = new TPhone { Name = name };
    }

    [Test]
    public void TestDialing()
    {
        Console.Out.WriteLine("a phone of type {0} with name {1}", typeof(TPhone), _phoneToBeTested.Name);
    }
    [Test]
    public void TestBatteryLife()...

NUnit interprets that as 2 different test fixtures - It is parameterized.. "Sam" and "Htc" are parameters passed into the suite. It is generic too.. the 2 test fixture attributes caused the entire suite to be run against an object of PhoneX and PhoneY.

MSTest: NOT SUPPORTED (or I haven't found it).

IDE-Integration

A big attraction with most MS offerings is integration; in this case, IDE integration. MSTest integrates out of the box. NUnit needs some assistance; but if you have something like Resharper (which you should), this isn't a deal-breaker.

Speed of Execution

MSTest: This is peculiar. I used Ctrl+R, A to run all tests in solution. It takes 4-5 s to run 3 measly tests.. as compared to NUnit which runs an identical set in 0.1s . Around 4 secs elapse, before any test method's status changes - it seems it is doing some heavy lifting in "preparing" to run the tests.
NUnit: I have NUnit setup to run all my tests when the binaries change. So a successful build triggers a test run. The faster I receive the test results, the less the possibility that I pursue a distraction.

Even in console mode, NUnit was faster. The cold start case takes much longer. However in subsequent runs too, NUnit took 3.5s as compared to 6s for MSTest
NUnit wins the speed test.

Assertion Library

NUnit has a much more evolved assertion library than MSTest. Don't have a detailed comparison here.
Personally, I miss the constraint based Assert Model present in NUnit, Hamcrest, et.all, which I find more readable.
e.g. Assert.That(actualObject, meets_some_constraint)

Asserting on exceptions

MSTest : Expected Exception not thrown!

MSTest: You can use this to state that the corresponding test should throw an exception of Type X. It also takes a message as a second parameter... which is the failure message (to be shown when the test fails) NOT the expected exception message. i.e. you can only check the type of exception.
NUnit: You can check the type, the ExpectedMessage (you can even specify that the actual exception message must contain/start-with/exactly-match/match-a-regex), a custom handler to verify the exception and a failure message.

NUnit : Expected Exception thrown but with incorrect message

However even this approach has been deprecated because you're not sure of the source of the exception, any line in the test could throw the exception and pass the test. So enter Assert.Throws where you narrow the scope down to the Connect call in the following example.

NUnit

MSTest does not have Assert.Throws or an equivalent. This was actually a user-request in 2008. Still pending in 2011. It isn't too hard to write your own version.. but then you shouldn't have to.

public static T Throws<T>(Action codeblock) where T : Exception
{
    try
    {
        codeblock();
    }
    catch(T exception)
    {
        return exception;
    }
    Assert.Fail("Did not throw an exception of type " + typeof(T));
    return null;
}

Another nitpicky example would be the CollectionAssert implementation.
MSTest accepts ICollection for parameters instead of the more general IEnumerable. When you're comparing objects of IEnumerable<T> (LINQ anyone) or IList<T> (another core collection type), you need to create copies so that you can compare. An avoidable design choice. Also note the error messages - it's important to ***fail informatively***. Chalk this one too to NUnit.

IEnumerable<int> enumerableX = new[] {1, 2, 3};
IEnumerable<int> enumerableY = new[] {1, 2, 4}.Concat(new[]{5});
CollectionAssert.AreEqual(enumerableX.ToList(), enumerableY.ToList(), "myError Message");
// Output:
// CollectionAssert.AreEqual failed. myError Message(Different number of elements.)

/*----------Equivalent NUnit error ---------*/
CollectionAssert.AreEqual(enumerableX, enumerableY, "myError Message");

//Output:
//NUnitTestsClient.ConnectTests.TestCollections:
//  myError Message
//  Expected is <...List`1[System.Int32]> with 3 elements, actual is <....List`1[System.Int32]> with 4 elements
//  Values differ at index [2]
//  Expected: 3
//  But was:  4

Code Coverage

Yup. With MSTest, setting up code coverage is a snap. This is how you do it ; the trick is in locating the Configure button :)

That said, getting code coverage for NUnit tests is just a bit more of effort. Create a batch file

vsinstr /coverage MyProduction.dll                   #instrument binaries
vsperfcmd -start:coverage -output:WhatsThe.coverage  #start profiler
[path to nunit-console-x86.exe] MyNUnitTests.dll     #run tests
vsperfcmd -shutdown                                  #stop profiler

That creates a coverage file that you can just open up in the IDE and you get the stats and the line highlighting et. all.

Test Impact

MSTest can point out unit tests that are impacted by a code change, on the basis of function call chains. There is some additional work to avail this benefit (configuration and collecting baselines)
NUnit : not supported.
That said: it works only for unit tests; Personally I'd run all the unit tests but YMMV.. especially if it takes over 10-15 mins to run all of them.
It would have been a HUGE time-saver if it worked for acceptance tests. However at the system level, modules interact via DB, files, networks, etc even though there are no direct function calls ; so doesn't work.

Parallel execution

MSTest seems to be designed to enable executing unit tests in parallel. Here's the how-to.
Note:

I had to open-and-close my solution for the change to take effect.
Also you can have either coverage or parallel execution (not both).

Of course your tests need to be designed accordingly too - eliminate interference. Also would be helpful to verify that the tests are isolated/independent.

NUnit : not supported..yet. However there is PNUnit, an open-source mod. Also it seems likely that parallel execution would be built into the next big release of NUnit. Another approach would be running segments of your tests in parallel on multiple cores/machines.

Misc. Peeves

Some more tiny things that irked me with MSTest

No hierarchical view of test cases. Flat lists everywhere.
Have to open a new "test details" window to see details of every test failure. e.g. console output.
Adds supporting files to the solution (.vsmdi, .testsettings) etc.
Cannot see console output for all tests in one place. (Useful for debugging).
Categories can only be applied to tests. Tedious - applying the attribute in 15 places instead of once on top of the test-class. (Duplication). Filtering on categories is non-intuitive in the IDE. Try doing it without looking at the answer.

Summary (IMHO):

So NUnit is clearly ahead on Speed, Simplicity, Assertion library, Flexibility e.g. Parameterized (data-driven) tests and informative failures. It has had the benefit of time and shows it. If you're already using it, there is no compelling reason to switch.

MSTest has parallel execution & Test Impact as differentiators. However that is not enough to make up for its shortcomings as a unit testing framework. Less frequent releases (tied up with VS editions) and slower pace of evolution. It just has got a lot of small things wrong that make it difficult to like. If you're using TFS, then MSTest would be easier to work with than other unit testing frameworks for obvious reasons. (But then this is your smaller problem.. Sorry couldn't resist that one.)

PSR : Free Windows Screen Recording tool

2011-11-09T13:02:00.000+05:30

An unsung hero in the MS Toolkit.

It has a weird name - Problem Steps Recorder that is not obvious as a search keyword. It doesn't create a video as the output, rather an HTML containing annotated images. (On the flip side, it doesn't eat up a lot of space and doesn't lose much information).

This one ships with Windows7. To invoke, Start > Run > psr

This should pop the very intuitive UI. Here's a 3 min video walkthrough. I just learnt you can annotate issues as well.

Brilliant!

Review : Parasoft DotTest Static Code Analysis

2011-11-08T18:12:00.002+05:30

Everyone agrees that static code analysis is good. However it isn't in widespread use. It has become easier with FXCop / Code Analysis built into VS2010. Personally I'm for "educating developers" over SCA.

Focus on Prevention over Post-facto Inspection, just like Bugs.

Make sure that the number of warnings reduce over time.
Make sure that the code isn't being engulfed by green suppression markups. Refine the set of rules as needed.

Cannot stress this enough, To avoid SCA pain (drowning in a stormy sea of SCA warnings) START EARLY.

I was asked to come up with a good set of SCA rules for acceptance test code. Here's the specific tool I'm looking at - Parasoft dotTest (I'm using v9.1).

Setup is easy.. Liked the following aspects.

You can install into a separate profile i.e. avoid messing up your default VS2010 setup.
You can get a floating network license that can be shared across users. You need to check "Release automatically when idle" in the License options.

So I created mine by creating a new configuration.

Main Menu - Parasoft - Test Configurations... Click New

Adding categories one at a time by cross-verifying it with a medium sized codebase ~80 files / ~8K LOC. Took about 2 mins. The rules can be found under the "Static" tab on the right, when your configuration is selected.

TIP! The search box can be used to filter the hierarchical view below.
TIP! Right click on a specific rule to bring up its documentation. Or parameterize the rule if that is supported.

You can export the configuration then to a file that can be shared across your team.

Since I was looking for SCA on test code (which is run within a controlled env), I turned off the following categories

COM Guidelines
Casting guidelines
Implementing Finalize and Dispose
Security Inspection
Serialization Guidelines
Security Policy Rules
WPF

That accounts for 96 / 444 rules. Next the rules that didn't make the cut (Rule Id in brackets, use search/filter to find a rule.)

Keep line length within predefined parameters (80) [BRM.MLL-3]

Includes whitespaces on the left - in short indentation causes rules to cause noise warnings. Unusable.

Avoid class, struct, or interface names which are more than 14 characters long [BRM.LONGNAMES-4]

Default limit too short. Cannot customize. I prefer long readable names over this warning.

Always provide appropriate file header (copyright information, etc.) [BRM.FILEHEADER-3]

Needs a comment to be the first line in the file - even before the using statements. Does not recognize standard XML comments.
Does not exclude non-public classes. Noise

Avoid unused private methods [CMUG.MU.AUPM-2]

Noise warnings. Flagged methods that were being called.

Avoid using get accessor with side effects [CMUG.PRU.AGAS-3]

Flagged all properties that were internally calling String.Format. Noise.

Use an array instead of many parameters [CMUG.MU.AIMP-5]

Noise again. Recommends turning get(string elementId, string keyPropertyName, string valuePropertyName) to take a string array instead of well named parameters. Counter-productive.

Make events public [CMUG.EVU.MEP-5]

No explanation given. Doesn't seem to be necessary - Event Design Page MSDN.

Put using statements in alphabetical order [CS.USO-4]

VS2010 sorts usings such that System.* namespaces are grouped above the rest.
This rule doesn't seem to know about this. A workaround is to override this behavior in VS2010 but then that's something that each developer would have to change.

NAMING

Avoid language-specific names for members [NG.ATNC.ALSN.MEMBER-5] : flags up "_constaints" for containing "int"
Avoid method names that conflict with keywords [NG.WC.KEYWORD.METHOD-3] : Will not let you have methods called Get, Stop, Select - which are sometimes the most intuitive names.
Use Pascal case for class names [NG.CAPSTY.PASCAL.CLASS-5] : UIDriverBase is a valid name as per Design guidelines but flagged up by this rule. See Capitalization Rules for Acronyms.
Use the property's type name as part of the property name [NG.PRN.APNCTN-4] : This is a "Consider" not a "Do" in the guidelines. Embedding type name in the property is risky w.r.t. future refactorings (e.g. if you change the type of the property with name MonitorIPString from String to StringBuilder. Misleading.
Use the suffix 'EventHandler' for event handler names [NG.EVN.EHNEEH-3] : Broken. Here's a sample - Message: Change type of event 'X.Driver+AppLaunchedEventHander AppLaunched' or add 'EventHandler' suffix to type 'Driver+AppLaunchedEventHander'.
Use flags attribute for bit field enum [NG.ETN.UFABFE-3] : Flagged simple 1 member enum with non-plural name as bit-field enums. Broken.

Avoid too many function calls from a single function [OOM.FCSF-4]

Limit of 10 is easily breached. Non-customizable. Includes property access within the method.

Use 'GetType()' in the 'Equals()' method implementation [PB.EQL-3]

Does not exclude overloads of Equals. e.g. Equals(MyType t) does not need to check type of t.

Use upper case for method name with name length <=2 [NG.CAPSTY.UPPER.MN-5], [NG.CAPSTY.UPPER.PROP-5]

Asks me to rename a method called At(params) to AT(params). Was not able to find any design guideline to support this.

Avoid public default constructor when a non-default constructor is present [CMUG.CU.PUBDEFAULTCTOR-3]

Contradicts Exceptions should implement common constructors [ERR.EICC-4]. The exception rule won.

Follow the limit for Number Of Static Methods [METRICS.NOSM-1]

This one flagged a false +ve. Static Class had 1 static var, 3 public and 3 private methods. Rule violated: 10 is not less than 10. Refactoring often leads to lots of small private methods.

Then there was a bunch that

was auto-enforced by C# e.g. Name indexer 'item' or Name event accessors 'add' and 'remove'.
Some were guidelines not rules.
Some were plain out of date e.g. Provide the same accessibility for the property and its accessors [CLS.ACAC-3] This flags up a common pattern of properties whose setters are private. The explanation points to an outdated (.Net 1.1) version. We've have come a long way since then...

Turned off another 80-100 too. Finally ended up with 196 / 444 rules enabled.
There is also a report out and code-metrics at the end too. But to summarize, I found Parasoft dotTest to be a bit unpolished.. the infrastructure is there. Analysis is quick enough but the rules need to be improved. Not enough documentation online - except from Parasoft itself.

Verdict: Passable, but for now prefer Code Analysis that comes bundled with Visual Studio 2010.

String Calculator Kata : My first screencast trilogy

2011-09-15T12:59:00.001+05:30

This is my attempt at Roy Osherove's popular String Calculator Katas.

TDD StringCalculator Kata - Part1 from Gishu Pillai on Vimeo.

The Vimeo Album containing all 3 parts.

I'm using C# (VS2010+Resharper), NUnit and an IDE extension called Beacons
(the wiki might be a bit behind the downloads tab :))

The videos look best when you have the HD option turned on and Scaling (hover on the video - top-right) turned off.
intermediate practitioners
For best results : download the videos and the subtitles. Use your media player with audio turned off and subtitles enabled at 2-4X.
I don't do this (ever) so wasn't equipped with the proper audio gear (and coupled with the fact that I do not have an internet voice and can't code and speak at the same time, the audio is passable.)

beginners
Practice it a couple of times before watching. You'll get more out of it.

What I learned from it:

I need to practice more :)
2 modes of learning in a Kata

muscle memory - goal: Speed.
exploration - goal: Possible solutions. Take different choices and see how it ends up.

Automated refactoring (e.g. Resharper for C#) rules. Knowing your IDE and keyboard shortcuts goes a long way w.r.t. productivity
Streamline all routine work and bind them to keyboard shortcuts - like building, running tests, reusable code snippets/templates, etc

For people looking for screencasting tools on windows, I'd recommend

BB Flashback Express - the free edition for screencasting. Export to AVI using ffdshow worked great for me.
ShowOff - for highlighting keystrokes and chords. Keyboard Jedi wasn't too great with 64bit & keyboard chords (before my patience ran out).

The A Team

2011-07-30T19:49:00.006+05:30

Contd from the previous post..

So how do we succeed?
Lock up Enemy #1 - Accidental Complexity

Empower teams to choose

All projects / teams are not the same. Different needs, different tools. This may be difficult in BigCo especially if the tools have already been bought. But make some noise - stand up for your tools, the "users" need to channel some feedback upstream to the "purchasers".

Explore options instead of resigning to the golden hammer. Prefer tools that don't get in your way. Ones that don't require you to learn yet another proprietary language. Ability to write extensions in your team's native language is a +. This also opens avenues for developers to assist with automation work, if required.
Use existing tools instead of writing your own - they're likely to be functional, tried and tested
Avoid putting all eggs in one basket. Keep tools/test-runners swappable by defining layers. To migrate to a different test runner, you should only need to migrate your thin tests layer which calls into an automation layer below it, which does most of the heavy lifting. More on this soon..

Collaboration
forms a reinforcing loop against Silos/Specialists. Increasing collaboration decreases opportunities for specialization, which in turn facilitates more collaboration. Of course, the reverse is also true - Silos can finish off collaboration. Only one shall survive, you just have to choose.

Outside-in / Test-first + Wishful Thinking
If you've tried the XP Practice of TDD, you'd know the liberating feeling of letting the unit tests drive the design of the production code. You fix the end-goal, you make it work, make it clean and repeat.
Starting with the test prevents any bias (arising from implementation details, existing tools at your disposal, etc.)

ATDD is the corresponding practice at the system level. However it is not an easy practice to quickly latch on to. So work towards it in baby steps
For starters, concentrate on a DONE definition + writing tests first (before implementation) from the users' perspective on a piece of paper. Make sure everyone has the same idea of DONE before you start the iteration.
As the team matures, you can even move up to ATDP (from the BDD world) where you write test before or during iteration planning & use them for estimation.

WHAT over HOW
Ensures that the test is at the right level of abstraction (the ol' forest over trees adage). It makes the tests shorter and readable. It also works beautifully to bring out the intent (as opposed to the implementation) of the test.
Specify the bare minimum ; things that are relevant to the test at hand.. all other details need to be out of sight.

Stable DSL for testing
You employ wishful thinking to imagine the ideal interface you'd like the system to expose for testing. Since the tests are another client to the system, they can also drive beneficial design changes. The tests stand-in for real users, so if the system is difficult to consume for the tests it follows that it might be for the users too. You could start with a plain C# interface to begin with and then work your way up to a custom DSL. It

abstracts away incidental details like the GUI, underlying technology and other implementation details.
abstracts away the tools used for automation from the tests.
decouples the people interested in writing tests from the automation personnel. This allows both to play to their strengths and offers the best of both worlds. e.g. the testers could define the automation interface changes for the sprint and the developers could implement them with production-code like quality.
makes it easy to write new tests with relatively little boot-up time. Writing a test then is just combining the reusable building blocks offered by the test DSL. The tests layer is a good training ground for new hires.

Imagine (wishful) a robot that will operate the system for you and think of the commands that you'd issue to the robot. The set of commands are your starting point.
e.g. robot.DoX(params) or robot.GetY()

Programming Skills
Automation is programming. Without good programming techniques and discipline, sustainable pace would be difficult.
This means you need to raise the bar for automation personnel and/or leverage devs. If the team lacks the skill-set required, take countermeasures... Training, get some experts onboard, etc. The average skill level of the team can also be increased by frequent pairing.

Refactoring
Your #1 weapon against complexity. Beck's 4 rules for simple design, the techniques from the Refactoring book (Martin Fowler) + the SOLID principles are a must-read. Top that off with an introductory text on programming (e.g. Clean Code - Robert Martin) and you should be good to go.

Good Naming & Discoverable Design
Taking the time to pick good names goes a long long way. Good names make it easy to find things, facilitate understanding, help zone in on a specific area to change & reduce duplication
This also helps in being able to discover the design / API using just the IDE (learn by intellisense) and programmer intuition. Choose names that are likely to be searched. Operate by the principle of least surprise (code that works as expected the first time around); Avoid hidden side-effects. Document and use team conventions to preserve consistency.

Communicate Intent / Distill the essence
This takes WhatOverHow to the next level. Explaining Why e.g. by extracting another "coarse" method to move up one level OR differentiating sets of inputs by using explanatory names. This reduces the test further to the essence - where the tests turn into readable system documentation... the kind that never gets out of date.

Learning Curve
Refactoring well and often keeps accidental complexity down to manageable levels. The supporting cast of Pairing, a discoverable design, intention-revealing code and a good testing DSL make it easy for new team members to learn the ropes.
This inhibits cargo-cult behavior and the changes made are deliberate/intentional rather than hopeful. Another source of complexity wanes.

Test Maintenance - the last frontier
Test Maintenance like complexity can be minimized not eliminated. As complexity decreases, maintenance effort reduces too.
The testDSL makes it possible to write-and-test the blocks once & use anywhere. Simple designs (no duplication, intention-revealing code, minimal classes) make maintenance much easier.

Transitively, the cost of automation goes down as well.

Let's refactor our diagram to remove the accidental nodes and edges and things get clearer now. Refactoring code is even more rewarding.

Towards better acceptance test automation...

2011-07-30T11:24:00.019+05:30

This started out as a sketch of a Causal Loop Diagram (CLD) for bad acceptance test suites... and then it got away from me :) The black arrows indicate "+ve links" e.g. Duplication and Complexity increase/decrease together. The blue arrows ending in dots indicate "-ve links" e.g. Refactoring and Duplication work against each other. Increase in Refactoring causes decrease in duplication.
Click on it to be able to zoom in.

And now the thousand err... few thousand words that correspond to that picture :)

I'll open with the worst-kept secret: Automation is Code. You're asking the computer to perform a set of steps to check something. That sounds like... programming. (There is also considerable overlap with unit testing.) If you have built a few non-trivial systems, you'd admit that the main challenges are minimizing accidental complexity and maximizing communication (with other humans).

All roads lead to accidental complexity.
Complexity is unavoidable for non-trivial programs. Essential complexity is that which is absolutely required to solve the problem at hand. Accidental is everything else (e.g. complexity introduced by the tools or language or algorithm used). The art is in identifying accidental complexity and minimizing it.

If we just concentrate (really hard enough) at the corresponding node in the center, we see the following inward black arrows (causes).

Wrong choice of Tools
You shouldn't use a hammer to unscrew a nail although it might work given the right level of persistence and resistance to pain. Sadly there are a lot of expensive 'golden hammers' being forced on QA teams. A bad tool can

get in your way.
coerce you to go against your better judgement. e.g. forces you to write repeated boilerplate code with no facility for extracting reusable blocks; even though you know 'Duplication is Evil'. You own the tools; don't let it be like the other way around.
lock you in . It is prohibitively expensive to switch. The vendor can also choose to charge for enhancements and fleece you with their well-thought out pricing and support models
refuse to play with other tools. Often this leads to the need for inventing wheels (in-house) ... which end up kinda square. Meanwhile the other folks using those round existing (open-source?) wheels are racing ahead
can breed specialists (for lack of a better term. Thanks Elizabeth H.)
drag in other relatives (tools from the same vendor). Go to bullet 1.

Emergence of the "Elite" / specialists / silos
Cumbersome tools can raise certain specialists to an elite status. These are the only people who dare to touch the codebase ; no one else understands it / knows how to use the tools or run the tests. All the automation work funnels through them. They become a difficult bottleneck limiting team throughput. Imaginary walls pop up, they may have illusions of grandeur and work in isolation unquestioned. Now you have a knowledge silo in the team (god forbid they leave or fall sick or take a vacation :).
Worse as the diagram shows, they form a reinforcing loop with accidental complexity. The complexity precludes the non-elite from making changes reinforcing the specialist status. The specialists continue to add complexity unchecked... widening the gap further.

Cargo-Cults
If this is the first time you've come across this term, read Eric Lippert's excellent post.
They form another reinforcing loop with complexity. Lots of cult members furiously spot-fixing without understanding the design or "Why did that work?" leads to the design turning opaque. Loss of structure leads to BBOMs, which breed the next generation of clueless fixers.

Speculative Architecture
Over-engineering is a leading cause of accidental complexity. Designing for future needs isn't as rampant as on the development side of the fence where the architects live.
Reuse is best when extracted out of 2 or more functioning clients - designing for reuse inside-out usually leads to abstractions that make all clients mad.
Intricate interfaces/classes/abstractions with non-intuitive names all add to the complexity and lead to a steep learning curve for new team members. Not sure who I am quoting here but - "you really can't use what you do not understand" so make sure that the code is within the grasp of the team supposed to work on it.

Duplication
It is the easiest problem to fix and somehow the most widespread code smell. Changing something in one place is a breeze.. changing it in 47 places identically is a mistake-prone chore. Doing this every few weeks is ... inviting bugs over.
Duplication forms another reinforcing loop with accidental complexity. Greater the duplication, greater the confusion / complexity. As the complexity goes up, it can be harder to find the right way/spot reusable pieces. People write their own copy - duplication goes up!

Test-After
This is a throw-back from the way things used to be. Programmers code till they are happy or time runs out. Throw it over the wall to QA. QA finds bugs, throws it back over. This continues for a while.

Cut to year 2011. The wall has been torn down. We can't wait till the development team is done to begin with automation. There won't be enough time. Even if you manage with some end-of-sprint heroics, you're likely to miss out on exploratory / attribute testing. The automation isn't likely to be the best either - in fact it is most probably going to be "the first thing that worked" (TM) considering the pressure and not wanting to be the ones that let the team down. The bugs - they live! (and yes, they're all out of bubblegum)

Test-after tests are usually long granular only-machine-readable scripts that over-specify and coupled to the GUI to boot! After about 4 sprints of heroics, the team is tired and complexity is taking the elevator to the top! Stop and change something.. the next sprint is NOT going to be better.

The heavies : Wrong Tests, Rework, Bad tests, Maintenance, $$$.
Wrong tests
They're usually a symptom of lack of communication/collaboration.
It's bad enough that the team built and wrong thing, you also expended a significant amount time & money in automating it & other testing. Now you got to do it all over again - Rework. That is resources could have been used to give the customer more features. There is no excuse for getting the tests wrong.
Eliminate middle-men & barriers in the communication path. Get the trinity of the customer+tester+developer to start talking! If the customer doesn't have the time to explain things to the team, quantify the implications in monetary terms. You will have someone's attention.

Bad tests
Write a failing test, how hard can that be? But what if I say write a failing test that is automated, thorough, repeatable, independent, precise and employs the best programming ideas? And did I mention it has to run as FAST as possible.
It takes effort, programming skill and discipline. Expecting historically non-coding testers to jump this chasm on their own in a short-time frame is crazy.

Getting this wrong can lead to tests that are

Spotty - the tests aren't thorough. As a result, bugs keep coming through the net of automated tests
Repeatable - the tests give you different results each time you run it. Given enough time, this can irreparably damage trust in the tests & consequently the testers. The tests turn into another hurdle for the development team.
Dependent - the tests have insider information about each other. Run it alone - it fails. Run the whole suite - everything passes. So to check one of them, you need to run and wait for a whole bunch of them.
Fragile - the existing tests keep breaking all the time. A tiny flutter in a remote corner of the system causes loads of tests to break.
Slow - "this thing usually takes 2 mins to comeback. But sometimes it takes upto 3. Let's Thread.Sleep for 5 in the test just to be safe." So "usually" you're twiddling thumbs for 3min/test... and for 100+ tests, that comes to 5 hours of waiting. What's the way out? (Hint: No. The solution is not to run it at night.)
Over-specification - the tests are a cure for insomnia. KLoC after KLoC of prescriptions for the computer.
Messy - It's code. Possibilities are endless... especially if you factor in some constant deadline pressure and no training

A bad test always comes back!

Maintenance and Cost of Automation
Maintaining a bad codebase is only better than maintaining bad tests. No one wants to babysit a tantrum throwing CI build server every day.

Automation costs real money. All "accidental maintenance" is money lost. Since the end-goal is to make money, it follows that automation has to have ROI. It has to pull its weight. If not it might take the team down with it.

Is there any good news at all in this post?
No but the next one does :)

Automated tests != Free Lunch

2011-07-01T10:58:00.006+05:30

Disclaimer: I'm a proponent of XP and truly believe it has made me a much better programmer. This post just aims to let readers see through the fog-of-agile caused due to data-transfer loss as it passes from person to person. Please do not misinterpret this as an anti-agile rant. I'm just saying it doesn't work out always unless you're willing to put in the effort to make the change.

Legend
What you hear… (good) the Promised Land

What was left unsaid (bad.. or downright ugly)

You have an automated regression-safety net to make future changes. Make a change, Push a button and you will know if you broke anything. CI can provide near instant-feedback. Better confidence.

You have 2-3X the code to maintain. If you have people who don’t care/are too busy/ not passionate about code quality and writing good tests, the tests are the first to put a chokehold on your productivity. Bad tests are as good as (or possibly worse than) having no tests. You could see yourself in a situation where a section of your team is permanently siphoned to keeping the build/tests green. This turns into a daily bottleneck. Tests need to be self-checking, thorough, readable, professionally written, independent, repeatable, concise and FAST. All this takes effort!

Documentation - the tests can be "live specs" of the application - They never get out of date like documentation.

It takes a significant level of discipline and skill to write readable spec-tests. An essential skill :to see the What and Why without getting entangled in the How. Most teams get this wrong... without noticing it.
Sidenote: The practice of ignoring failing tests is criminal (but usually not punished accordingly) and can lead to misleading specs.

Quality & Productivity: Leads to high-quality production code. Fewer Bugs. More features added / unit time (because you spend less time in debugging and manual testing)

IF you let the tests drive/shape your design (ATDD and TDD). Client-first design is an unstated requirement.
The quality of the code is usually a direct reflection of the people writing it. This means you need craftsmen (> 30-50% of a team) and NOT armies of cargo-cult programmers.
If you're using automated tests exclusively for regression (or getting your 'agile badge'), you'll slowly grind to a halt. Writing tests for "untestable blobs implemented from a non-negotiable handed-down paper design" is frustrating. People can be stuck on “how do I test this?” – Usually leads to silent trade-offs & non-thorough test which will let in bugs and put you in the net negative w.r.t. productivity.

Less rework/thrashing: The dialogue / conversation (that you have with the customer to come up with the acceptance tests) makes it highly likely that you’re building the right thing..

Assumes that the customers want to collaborate with the developers and testers. This is not often true.. Real users are sometimes real hard-to-find. Even if you manage to snag one of them, you can only procure a small slice of their time. Real users rarely want to write tests.

If the customers give a “vision” and delegate the responsibility of mapping them to executable specs to the technical team (or worse the QA/testers), you still run the risk of “This is not what I asked for” late in the game. Regular demos may help shorten the feedback time.. but you may still waste an iteration.The magic potion here is collaboration and conversation.. the tests are just a beneficial byproduct.

Simple: Red-Green-Refactor. How hard can that be?

Sounds simple.. but is deceptive. True OO is a minority sport. Refactoring is a skill that you need to work on. Off the job practice is mandatory.
You may need to "hire" a good coach for an extended period (I'd say 6 months-1 release) to get the team rolling. Spot trainings/Just-in-time learning won't work for most teams.

Enterprise Library Logging vs Log4Net for a rolling file log : jumpstart and a showdown!

2011-04-03T10:18:00.008+05:30

I was looking for a rolling file log and couldn't make my mind between the two. So I embarked on a small quest to see which one is easier to learn and work with.

First up I'm gonna create a VS2010 Console application.

Enterprise Library
Ok first up we need to download the latest drop of EnterpriseLibrary which is currently 5.0. Run through the install.
Strafe the web a bit. Okay.. we need an app.config file in our project with some XML muck. Thankfully MS has a GUI tool to help us write it - it's named EntLibConfig.NET4.exe and lies in the EntLib install folder. Open it up and we can either create/load an app.config file.

Main Menu > Blocks > Add Logging Block
An Event logging Listener is added by default (2nd column). Delete it and add a Rolling Flat File Trace Listener - click the Plus sign next to 'Logging Target Listeners' caption.
Next go on a property-setting spree. It should end up like this

We'll work our way right to left.
Third Column : TextFormatter.

Template = {timestamp(local)} : {severity} - {message}.

The ellipses button opens up a editor which has a dropdown with all the special placeholder incantations to insert all sorts of informative bits into your log message. The default template by the way is pretty detailed for a default.

Second Column : Rolling flat file trace listener

File Name -> rolling-with-entlib.log
Formatter - choose TextFormatter (which is present by default. Else add one in the third column via the context menu.) and you should see a link between the trace listener and the formatter.
Max Archive Files - set to 2. Older messages will be sent to an archive file.
Message Footer, Header - clear out.
Roll Interval -> None
Roll Size KB -> 200

First column : Logging category.
By default, a 'General' logging category is added. I'll stick to that for simplicity (but you could add more of your own). Configure the Listeners property of the category to point to the rolling flat file listener. You should see another link.
We need one more link (the last I promise) to make the config valid. Open up the 'Logging errors & warnings' and make it point to the same listener. Another link appears and we're done. Save the file.

Jump back into the IDE. Add a reference to the Microsoft.Practices.EnterpriseLibrary.Logging from the .NET tab. Write ourselves a new PSVM method with

Logger.Write("Dilshan flings himself for the catch to send Kohli back!", "General", 0, 0, TraceEventType.Critical);
Logger.Write("MS Dhoni walks in to steady the innings.", "General", 0, 0, TraceEventType.Information);

Log4Net
Looks like we need the binaries and ... yup.. some more XML. But there's no GUI to help us this time around. However there is a kind soul on the internet writing tutorials at BeefyCode.com. Two thumbs way up! By Part 5 I think I know all I need... for now.

   <configSections>
  <section name="log4net" type="log4net.Config.Log4NetConfigurationSectionHandler, log4net"/>
</configSections>
<log4net>

<appender name="RollingFileAppender" type="log4net.Appender.RollingFileAppender">
<layout type="log4net.Layout.PatternLayout">
  <conversionPattern value="%date %-5level- %message%newline"/>
</layout>
<file value="rolling-with-log4net.log" />
<appendToFile value="true" />
<rollingStyle value="Size" />
<maxSizeRollBackups value="1" />
<maximumFileSize value="200KB" />
<staticLogFileName value="true" />
</appender>
<root>
<level value="ALL"/>
<appender-ref ref="RollingFileAppender"/>
</root>

</log4net>

A quick summary: Log4Net has appenders (instead of EntLib's trace listeners) whose properties need to be set. A root node controls the level of logging + a reference to the appender to use. The PatternLayout node controls how the message is logged (akin to EntLib's formatter template property) with the magic placeholders. That's it.

Back to the IDE. Add a reference to log4net.dll.

log4net.Config.XmlConfigurator.Configure();
var logger = log4net.LogManager.GetLogger(typeof(Program)); // Program is the name of my entry point class

logger.Error("Dilshan flings himself for the catch to send Kohli back!");
logger.Info("MS Dhoni walks in to steady the innings.");

That's all there is to it. I had to change the project's target framework to ".Net Fwk 4" for the build to succeed. The client profile doesn't have System.Web that log4net needs for some reason.

Here's how it looks in the log files (I added a running counter into the message)

REM This is Log4Net *********
2011-04-03 10:30:52,871 ERROR- #499577 Dilshan flings himself to send Kohli back!
2011-04-03 10:30:52,911 INFO - #499577 MS Dhoni walks in to steady the innings.

REM and here is EntLib *********
4/3/2011 10:29:44 AM :  Information - #499773 MS Dhoni walks in to steady the innings.
4/3/2011 10:29:44 AM :  Critical - #499774 Dilshan flings himself to send Kohli back!

Showdown!
Things that I noted from this limited experiment:

EntLib has a nice config tool. Log4net could have one too but I couldn't find any. To be fair, the xml muck for log4net is simpler to write by hand and shorter by about 30%.
It's easy to tweak the amount of logging via an xml file edit. You tweak the 'Minimum Severity property for the Logging category in EntLib5.0 ; the root/level element's value attribute in Log4Net.
Log4Net is 3x-5x faster from the rolling file burst logging tests that I did. Although it probably doesn't matter for most purposes.
Log4Net also has a smaller assembly footprint - you just need one DLL. EntLib will drag in 4 dependencies ; additional EntLib DLLs.

logger.Info("We just won the ICC World Cup! Yay!! Logging out...")

Flash Cards Kata and the transformation priority premise

2011-02-01T08:25:00.015+05:30

I began the new year by resolving to complete the PCL (Peter Siebel's) book. Around the same time, another interesting post by Uncle Bob appeared on the WWW. A discussion on the TDD yahoogroups spurred me to try a Kata - two-sided Flash Cards.

Looked easy enough to test my new-found Lisp skills. Although I was not consciously choosing the next test / step based on the transformation priority, I was curious to know if I had been doing it all along without knowing it.

This innocuous looking kata had me stuck initially - what is the right first step? Trash-canned the first couple of attempts. The third attempt was slow but I succeeded. I did it again in C# just to see if the language had any bearing on my method. This time I was faster because I was familiar with the trail now.

I am not wiser with regard to the priority premise - I found I used a bunch of transformations that were not part (bold) of the original list (shown in blue below). As a result, their priorities are unknown. This was the proverbial rain on my parade.
Also in the C# variant, I found a couple of tests where I needed multiple transformations to get to the next green & I could not figure out a smaller test.

Note: Check the "speaker notes" in case things get dense especially with the Lisp one. Use the menu button at the bottom-right to get to the presentation link

Attempt#1 Flash Cards Kata in Lisp

TEST User should be asked a question

{} => statement

TEST A right ans should be confirmed

P5 (statement->statements) adding more unconditional statements.

TEST A wrong ans should be corrected

P6 (unconditional->if) splitting the execution path

TEST Diff card and a right ans

P4 (constant->scalar) replacing a constant with a variable or an argument

TEST Diff card and a wrong ans

constant => expr

TEST Multiple cards

{} => loop (statement)

TEST Answers are case insensitive

exprX => exprY

TEST Answers can have leading/trailing whitespace

value => expr

Attempt#2 Flash Cards Kata in C#
TEST User is asked the question

P2 nil -> constant

TEST User is asked another question (triangulate)

{} => statement *** Play()
constant => field *** LastQuestionAsked
void => statement *** Ask() - is there a smaller step here?

TEST A correct ans is confirmed

P2 nil -> constant

TEST A wrong ans is corrected

P5 (unconditional->if) splitting the execution path *** Play()
P4 (statement->statements) adding more unconditional statements
nil => expr *** User.WhoAnswers
field assignment *** User.ctor
constant => field *** User.LastConfirmationReceived
void => statement *** User.Notify

TEST A diff question and a right answer

P4 constant => scalar arg

TEST A diff question and a wrong ans

constant => expression

TEST Two cards

{} => statements

TEST Multiple cards

{} => loop
P4 (statement->statements) adding more unconditional statements *** User
field assignment *** User.ctor

TEST Case insensitive answers

exprX => exprY

TEST Trim leading/trailing spaces

value => expr

The way of the unusual architect - Dan North

2010-12-15T12:18:00.004+05:30

An enlightening talk on 'how the human mind works and its corollaries on how programmers work' from the wise Dan North - founder-leader of the BDD camp..

...and so you go from something that was fit for purpose but a little bit ugly to something that never quite delivers. Who here has been involved in what I call a 2-year rewrite anti-pattern ? So we go to the business and we say

Ok.We will rewrite out entire system for some period of 2 years and then everything will be good.
'Wait hang on. What will we get after 2 years?'
Exactly what you've got now.
'Right... and during that 2 years..'
Oh we're locked down! We can't do anything else for you.
'Ok. So you're telling us that we should pay you for 2 years to go no further than we are now and you won't be able to do anything else'.
No.. that's pretty much it.
'Okay...'

and we call that enterprise architecture.

He had me there 3 mins into the talk. Recommended

Beacons - a VS 2010 Extension for TDD

2010-12-04T02:45:00.002+05:30

Just uploaded an early version of a TDD visualization extension that I've been working on in my free time. Get an early version of Beacons from the google code project here

The extension only works with NUnit currently. You specify a file path to your NUnit test results file path (which is normally a TestResults.xml file in your .dll/.nunit folder) and you're all set to go.
It can't listen to Resharper test-runner yet. (yet.. if I can figure out how to tap in, should be not an issue.)

To see it in action, check out this video - switch to fullscreen HD

TDD StringCalculator Kata - Part1 from Gishu Pillai on Vimeo.

Shift+Alt+B, H - Change hats (adding behavior hat OR refactoring hat)
Shift+Alt+B, P - Play/Pause (when you need to take a break)

If you give it a fair try, I'd love to hear from you. Particularly I'm interested in finding patterns (so share end-of-session screenshots)
More ideas, comments, suggestions - feedback of any kind is welcome.. Post away at http://groups.google.com/group/beacons-tdd

ScenarioLibrary, LibraryTables with Slim Scenarios : Avoid fixture method explosion !

2010-11-23T17:26:00.005+05:30

Assuming: working knowledge of Fitnesse.

I've been recommending Slim Scenarios for a while (primarily because they are the Fitnesse equivalent of reusable blocks (methods/functions in your favourite programming language) i.e. tables that can be called from other tables. {Actually they are akin to macro expansion.. but that's nitpicking!)

However soon I ran into a problem, the scenario steps translate into method calls onto the backing fixture class.
e.g.


|scenario         |play  movie|pathToMovie|on home theatre|
|enter            |HDMI1      |mode on tv                 |
|enter            |DTV/CBL    |mode on avr                |
|set avr volume to|-25        |dB                         |
|navigate menu to |@pathToMovie                           |
|press            |play       |on media player            |

Now whenever I call this scenario, the fixture methods 'enterModeOnTv', 'enterModeOnAvr', etc. need to be present on the fixture class. So if Fixtures A, B, c call this scenario, All of them need these methods on them. Bummer!

Now the initial response was to extract these methods onto a helper class. Still I'd need to write some delegating methods on each fixture to delegate to the extracted class. So I raise my hand, admit my ignorance and post to the fitnesse forums. The experts answer in a flash!

The magic answer is the Library Table

FrontPage.HomeTheatreTests.SetUp


|library     |
|television  |
|avr         |
|media player|

The Library table specifies a list of class that should be checked in case the fixture does not contain the required method. So in this case, the order in which the classes would be checked would be [Fixture > MediaPlayer > Avr > Television]. If a matching method is found in any of these classes, the step is resolved.. No more delegating methods!

This allows me to have cohesive helper classes that contain related methods which can be mixed into any page.

Another notable mention is ScenarioLibrary pages - these are pages which are libraries of usable scenarios.

FrontPage.HomeTheatreTests.ScenarioLibrary


|scenario         |play  movie|pathToMovie|on home theatre|
|enter            |HDMI1      |mode on tv                 |
|enter            |DTV/CBL    |mode on avr                |
|set avr volume to|-25        |dB                         |
|navigate menu to |@pathToMovie                           |
|press            |play       |on media player            |

|scenario|start home theatre|
|power   |on  |media player |
|power   |on  |avr          |
|power   |on  |tv           |
...

ScenarioLibrary pages are included if they are a brother/uncle of the current page. Also all ScenarioLibrary pages up the hierarchy are included (allowing the possibility of overriding)

All this leads to very small test pages and it all works!

FrontPage.HomeTheatreTests.CanWorkWhileWatchingMovie


# ScenarioLibrary page is included (helper slim scenario definitions)
# Next SetUp page is included (imports and library tables)

|script    |me                                                   |
|start home theatre                                              |
|play movie|File Manager\USB\C\2 Watch\Iron Man 2|on home theatre|
|start working                                                   |
|after     |30                                   |mins           |
|ensure    |work is done                                         |
|press     |stop                                 |on media player|
|shut down home theatre                                          |

In this example, the fixture Me only needs to have methods that pertain to me working. All methods related to the TV, AVR and MediaPlayer are resolved by the corresponding classes in the Fixture.dll due to the Library table.


public class Me
    {
        private bool _workIsDone ... 

        public void startWorking() ...
        public void afterMins(int timeInMinutes) ...
        public bool workIsDone ...
    }

Pretty cool!

What do I use for acceptance tests (ATDD) : NUnit vs Fitnesse ?

2010-11-12T14:39:00.005+05:30

Lately I've had some teams in my getting-agile ecosystem debating on which tool to use for acceptance testing. Now my first response is always "It depends" (You really can't go wrong with that. :) ).

Notation: I'm gonna use

NUnit when I actually mean any member of the xUnit family of unit-testing frameworks.
Fit/Fitnesse stands for itself and now Slim/Fitnesse.
Customer means the XP's Customer role or Scrum's Product Owner role.

Fit-Fitnesse was the de-facto tool for automated acceptance tests till sometime ago.. Around 2009, there were strong murmurs that Fitnesse isn't working that well (read as some teams weren't executing very well and Fitnesse was part of their toolkit ).

So what does it "depend" on ??

Customer Collaboration: Probability of Non-developers reading/modifying individual tests

Is your customer willing/able to collaborate with the team over a piece of paper to extract examples/scenarios for a particular use-case ?
Are customers interested in creating truth-tables/spreadsheets of 'how this rule should work' or story descriptions of 'how the ideal system should work'?
Do Customers / QA-personnel want to add/modify tests (without dev support) to improve their understanding / the system? Is the team using a shared medium to come up with more tests?

If yes, choose Fit-Fitnesse... that is what it was created for. Collaboration.
Alternate Scenario: Developers are given a bunch of requirements and are responsible for conversion to executable tests. Customers/Proxies are available over email to answer questions during ear-marked hours. Test-list generation is done offline (read without an active customer) followed by a verbal/email review. The transformation to executable tests has no participation from the customer. Acceptance tests are being used primarily for closing stories (definition of done) and regression. If a Pass/Fail result is all you need, NUnit might be a better choice.

Positive Pressure on Test readability

As-intended use of Fitnesse, causes test authors to express intent before implementation.. a wiki page is like a blank of paper - wishful thinking ahead! This causes the pages to read like scripts or a sequence of intent-revealing steps with intermediate checks/asserts. If the team is struggling to create readable-n-maintainable tests, use fitnesse as a guardrail.
You could do the exact same thing with NUnit. You just need to see the world from the users' perspective and a professional programmer's instincts for refactoring and simplicity. A blank test is as good as a blank wiki page. Flesh out the step-methods later. A common trait (of developers) is to jump the gun to implementation details at great harm to readability. Curb your enthusiasm and no reason why it shouldn't work

Deployment Footprint

Fitnesse offers a Web interface to easily view/modify/run tests from the comfort of your favourite browser. Point to your fitnesse server URL and voila! Zero install. You can even do 'What if' analysis without much assistance. (e.g. What if number of concurrent clients is raised from 100 to 150)
NUnit can't do it on its own. You would need a CI System to offer the web dashboard to execute/view results. Viewing the internals of a test is still difficult. You would need an IDE and 'coding skills' to modify the tests for any what-if analysis. Of course, if the development team is the only one interacting with the tests, this is a moot point.

Homesickness

If dev >> testers, then it might be easier to teach the testers NUnit - in crunch time, free devs can pitch in to help the test teams get to the finish line. Most developers already know NUnit; Getting all of them to learn another tool for acceptance tests could be a drag (atleast in the short term). Personally (Disclaimer: more dev/NUnit hours than QA/Fitnesse) I find that Fitnesse has a longer learning curve than NUnit (different table styles, reusing sections of a page, etc) before you can use it effectively. NUnit also wins hands down in the refactoring department (moving HTML around can get tiresome very soon to a person used to a statically typed language + a good IDE). Stuff like 'attaching a debugger to a failing test', 'how do I get my pages into source control', 'feedback on progress of test execution' with Fitnesse might restrain/annoy speed-addicted xUnit-developers. NUnit also has a richer assertion library, which I missed during my time in Fitnesse.
On the other hand, if you have customers and testers who can 'converse in fitnesse' and have a few projects under their belt, don't get in their way. Testers can get homesick too. Go with Fitnesse and pair Devs with Testers to diffuse Fitnesse in the team.

End-game

Both are tools - it is how your team executes with these tools, that makes the difference. To quote Ron Jeffries, "people can screw up everything" ; that doesn't reflect badly on the tool they used.
It is a reversible decision. If you design your tests to be independent of the test runner, you can swap out the existing runner at any time. e.g. the idea below (see image) that I've had some success with (for rich desktop clients). Maybe you'd find that none of them work for "your team", look for something new that works (e.g. maybe Cucumber is a better fit). Instead of spending hours in analysis paralysis, choose one, get started, reflect and adapt.

How to get Thoughtworks' White to LogStructure

2010-10-27T16:57:00.003+05:30

Struggled for some time getting it to work. I've been playing with Thoughtworks' open library for UI Testing for a while.

Sometimes UISpy doesn't get you to the UIElement hierarchy and you need to take out the big guns ala White's window.LogStructure to dump out the UIElement tree.
As always it's simple when you know how.

Step#1: Modify the app.config of your executable to include the sections related to White. See a sample file on White's page here.

Step#2: Add the following line to your AssemblyInfo.cs file

[assembly: log4net.Config.XmlConfigurator(ConfigFile = "Log4Net.config", Watch = true)]

Step#3: Add a file called Log4Net.config under your assembly (configure it such that it is always copied to the output folder). This contains the configuration information for Log4Net (e.g. the following config logs to console and to file named example.log in your output folder.). That's it you should now be able to see/capture window.LogStructure output from White. Godspeed !

<log4net>
  <appender name="Console" type="log4net.Appender.ConsoleAppender">
    <layout type="log4net.Layout.PatternLayout">
      <!-- Pattern to output the caller's file name and line number -->
      <conversionPattern value="%5level [%thread] (%file:%line) - %message%newline" />
    </layout>
  </appender>

  <appender name="RollingFile" type="log4net.Appender.RollingFileAppender">
    <file value="example.log" />
    <appendToFile value="true" />
    <maximumFileSize value="100KB" />
    <maxSizeRollBackups value="2" />

    <layout type="log4net.Layout.PatternLayout">
      <conversionPattern value="%level %thread %logger - %message%newline" />
    </layout>
  </appender>

  <root>
    <level value="DEBUG" />
    <appender-ref ref="Console" />
    <appender-ref ref="RollingFile" />
  </root>
</log4net>

An effective test strategy to grow applications

2010-08-20T14:40:00.004+05:30

This is a post to summarize my understanding (as of today) of an effective (low ceremony) way to build applications.
Let's take a look at (a slightly modified version) of Mike Cohn's layered test pyramid.

The choice of a pyramid indicates the robustness/stability of a lower layer directly affects the effectiveness of the upper layer. Also the number of tests decreases as you move up. As the agile testing book says ROI is maximum at the bottom (speed of feedback over time invested) and wanes towards the top.
e.g. without robust unit-tests, DSL tests or GUI tests would catch a bunch of errors without the essential feedback needed to fix it quickly. More and more errors would make it up to the middle and top layers, where it is more time-consuming/expensive to find-n-fix. Nothing replaces well-written, quick, professional unit tests.
To prevent ambiguity and misinterpretation, let's go over each layer.

Layer 0: Unit-Integration-Stress tests

Unit tests should accumulate over time as a side-effect of test driven development. It's a tough art to master -- writing good unit tests but they are invaluable in the long term. They are essential for 'internal quality'. This layer also contains some integration tests and stress tests.
'Integration tests' in this post means (as in the GOOS Book) : 'Let me see if the real collaborator behaves as I expect it to'.

e.g. Let's say Car needs to use the Driver database. Since we live in these times, we'd define a Role(Interface) DriverRepository to abstract away the database from ClassA. In my unit-tests, I'd inject a MockDriverRepository into a Car and test it out. So I know Car is good. Now there are some assumptions implicit in the Role/interface. I need to verify that the real implementation behaves the same - so I'd need to write some 'integration tests' to verify if the real collaborator DriverRepository behaves the same. These tests would round-trip a third party / external subsystem in order to produce a result. TestDriverRepository.GetsSpecifiedDriverInformation() would test if we can get the specified driver information out from a real database. Integration tests are *slower* than unit-tests, so we partition it into a different test-suite that is not run as frequently as the unit-tests. Feedback needs to be as short as possible during development.

'Stress tests' are for asynchronous code - code that has multiple threads running through them. In this case, you'd have to define some invariants - things that must remain true irrespective of the number and scheduling of multiple threads through the code. (Refer to the last couple of chapters of the GOOS book for details).

Layer 1: Under the GUI / DSL-API tests
This is the layer that's rarely given its due and usually missed - its primary reason for existence is scenario testing just under the skin/GUI. These tests are written from the user's perspective. When you get this layer right, its sweet. You will see a DSL emerging. Your tests appear as short scripts written in an application specific DSL, where the steps are things that an actual user would do or perceive. These tests can exercise the whole app similar to a real user, without having to bring up a GUI... as long as you have designed it for testability (use a MVP / MVVM pattern).
These tests involve all real collaborators (no mocks/stubs/fakes). As a result, they are slower than Unit tests. However built on a bedrock of solid unit tests, they can take you to the finish line... almost.

Layer 2: GUI Tests
Even with anemic views, there is a matter of the 'wiring'. Is the GUI Control wired correctly to the right property/action on the backing object? Irrespective of the strength of the layers beneath it, you need a set of GUI tests that go really end-to-end identical to a real user. They're the slowest we've seen so far - so do not test everything through the GUI. The test duration would grow prohibitively large with time.
An interesting idea here is: to just test the code in the GUI in the GUI Tests. I haven't tried this out myself but sounds like something that is just crazy enough to work.

We're almost at the end. There are some aspects that we haven't yet tested.

Exploratory tests: Let some good testers go ad-hoc at your app trying to break things. These tests are manual since they rely on the tester's creativity and insight (not mechanical).
Usability tests: Regular demos and getting some real users to use the latest builds would go a long way to ensuring usability. Testers can act as a proxy as long as they are aligned with the customer's profile and needs. These are manual too since machines can't determine usability.
"ility" tests: These are tests that deal with performance, load, security, reliability, scalability, etc. When a feature is picked up for implementation, if it has any of the above 'ility' needs, they should be noted and a corresponding test be written up to validate them. (The point being.. don't wait till the end to start with this.) The tests can be written in a scripting language and use specialized tools/libraries if required.

Who does what ??

An effective way to implement features is a Top-Down approach, where you begin by writing a GUI Test/DSL-API test first and then work your way down. This way you only write code that a client needs - which leads to simple usable APIs. Once you have a failing red acceptance test,

You can either figure out the top-level design to get it to work and test-drive the classes required. Combine them to make the accceptance test pass.

you can begin by writing the code to make the acceptance test pass. Once green, in the refactoring step, you can now figure out the right "house" for the code. TDD the objects that you now know you need and fold them back into the running application (See the GOOS book if you're interested in this approach).

That's it. This ties up nicely with Brian Marick's test quadrants idea.

Review : Growing Object Oriented Software - Guided by Tests: Freeman, Pryce

2010-08-07T18:48:00.006+05:30

Recommended... best book on TDD in a while.

Just finished this one today... I got to fess up - that I am a "classicist". I've seen some real spaghetti tests involving mocks.. But I've gotten some periodic tension from the people on the other side of this religious fence of TDD. Since this book comes from "the torch-bearers of the mockist camp", I picked up a copy and tried to keep an open mind. And I although I haven't switched camps.. I now see the light. There is no one right path but the authors have placed some industrial strength beacons on this path and teach you how to use mocks and interaction tests to "grow" applications driven by automated tests. (However I still maintain that in untrained hands, Mocks (like C++) can be very dangerous and quickly lead into tarpits. Easy to abuse. However in the right hands, they can be used to write very expressive code & enable rapid and iterative development. ).

Some of the key takeaways for me were:

Developing outside-in ATDD: writing a 'walking skeleton' first , spotting objects and roles (relationships) between them, TDDing these objects and "folding them back" into a running app was a new method to me.
Listening to the tests, Focus on readability, tests with helpful diagnostics / failure messages, eliminating or reducing the irrelevant part of the test, use of builders + drivers to keep the tests concise
The chapters on threads and asynchronous testing are home-runs. Don't leave for work without it. The last third of the book was worth the price of the book (It isn't yet available in India so had to get it from overseas).

There is a lot of real world knowledge that the authors have distilled down to a kind of 'secret sauce'.

As the book rightly says, TDD that isn't sustainable will slowly choke the project, tests turn into maintenance headaches and strangles development progress. This is the part that trips most people up with TDD - Kent Beck's book on TDD and most of the literature out there detail the basic technique (with an assumption that you would figure out the exact steps that you need to take in your specific context). The skill of "listening to the tests" isn't easy to pickup or practice for most people. Freeman, Pryce et. all have come up with a more prescriptive method that should keep beginners on the rails.

I followed along to the AuctionSniper example using my own style (and .net, nUnit, Moq + White instead of java, jUnit, jMock + WindowLicker as in the book) in order to compare and contrast. The example isn't trivial (deals with multiple-threads, third party server and protocol (XMPP) and was well chosen.

The final solution was different from the solution in the book; but still passed all the tests. The authors emphasize test readability, which is a 'Can do better' item for me.

Final word: This book is a must-read for intermediate and advanced programmers interested in gaining mastery over TDD. I'd place it on the top-shelf right next to Beck's XP Book, TDD Book & Fowler's Refactoring book. It doesn't have any radical new ideas however it does teach you the right guiding principles to survive in the real world today. It says a lot of things that needed to be said at this point of time... for that I'd give it a 9/10.

Note: The book also has a companion site, with a responsive discussion forum.

Contract Tests with Slim/Fitnesse : Running a test suite against multiple implementations of an interface

2010-07-27T17:03:00.004+05:30

Recently I had a need for something similar to "xUnit contract tests" in Fitnesse (defined as: a test suite, which any object must satisfy so that it can be considered as a valid implementation of a Role. I got the term from one of J.B.Rainsberger's talks. )

To illustrate my point, let me cook up an example.

Imagine a contest for BoneyM impersonators. We would quiz each impersonator with a set of questions that they must answer correctly to win the contest.

i.e. we have a set of tests (questions) for each implementation of an Impersonator (Role). We also have multiple impersonators that I want to validate against the contract tests. The tests need to be dry - since they don't change for each implementation.

So here are the questions and the expected answers to them. (the contract tests)

Q:How did rasputin die? A:His enemies shot him till he was dead.
Q:Was yesterday sunny? A: No
Q:Who runs the baker family? A: Ma Baker

So the contract for each impersonator is


 public interface BoneyMImpersonator
 {
     string HowDidRasputinDie();
     bool WasYesterdaySunny();
     string WhoRunsTheBakerFamily();
 }

Slim/Fitnesse:
The new addition to the Fitnesse/Fit family is the SLIM runner. From what I've seen in a couple of days - I'm impressed!
For some quick help to get it running, check out this post by Brett Schuchert - http://schuchert.wikispaces.com/FitNesse.Tutorials.0

One of the table-styles that SLIM has introduced is "scenario tables". The thing that is great about them - is that it brings something similar to C++ macros (expansion via text substitution) to Fitnesse. i.e. you can write up a scenario table once and then call it from multiple places in other tables (script/decision tables). Let's see it in action.

Here's my Fitnesse page:
http://code.google.com/p/gishu-util/source/browse/trunk/Blog/ContractTests_FitnessePage.txt

And here's the backing fixture and domain code
http://code.google.com/p/gishu-util/source/browse/trunk/Blog/FixtureCode.cs
http://code.google.com/p/gishu-util/source/browse/trunk/Blog/DomainCode.cs

Refer to this fitnesse doc page for a sample http://fitnesse.org/FitNesse.UserGuide.SliM.ScenarioTable.

There's a collapsed section called "No Peeking" - but you should. Without it, SLIM doesn't have an actor or co-ordinator to talk to - and you'd get cryptic errors like 'Method [name] not found in fitSharp.Slim.Operators.ExecuteBase+NullInstance'. The missing ingredient for me was the |script|quiz| line. With this, you tell Fitnesse that a class named Quiz is who you should talk to.

Next you define the scenario (see the above link for details). In short, a scenario has a name and parameters. Parameters must be used within the scenario prefixed with a @ sign. In our scenario, we use the fully-qualified type name of the implementation as the parameter. In welcome, we create an instance of it via some reflection/activation magic. Next we interrogate the object and compare answers.

Finally we need the scenario to be run against multiple implementations of the contract. For that we have a decision table (equivalent to Fit's ColumnFixture), which must have the exact same name as the scenario and the column headers must match the scenario parameters.
So in effect, each row is replaced by a scenario invocation with the parameters specified in the row. This can be seen when you run the tests. You can see new column with cells, which be expanded to see the scenario details as shown below...

In the future, if you have a new implementation say StonedGuy, just add another row to the Decision table and that's it! The contract tests are DRY-compliant.

Update: Gregor Gramlich points out that the welcome method doesn't need to be implemented in the fixture, just for an instance of the test subject via reflection. Instead a line |start | FullyQualifiedClassName | works just as well. Nice tip. Code samples updated.

Implementing case-insensitive lookup tables (Dictionary) with string keys

2010-04-22T16:13:00.004+05:30

The Dictionary type in .Net takes in a comparer as a constructor parameter. So you could pass in StringComparer.OrdinalIgnoreCase and you're done.
I used to upper-case keys while adding it to the lookup table earlier and again when performing a lookup - to achieve case-insensitive string lookups. I remember reading some article about String.ToUpper() being optimized for such scenarios. Now I was unsure - which one is better ?

So I ran a small experiment.


//void Main()
         var keys_for_lookup = new List<string> { "abc", "dEF", "123", "655", "pQr" };
        for (int i = 0; i < 3; i++)
        {
            var dict1 = new Dictionary<string, object>();
            AddSomeKeys(dict1, new List<string> { "ABC", "DEF", "GHI", "JKL", "MNO", "PQR", "XYZ", "123" });
            TimeIt(delegate { RunTest_Upper(dict1, keys_for_lookup); });

            dict1 = new Dictionary<string, object>(StringComparer.OrdinalIgnoreCase);
            AddSomeKeys(dict1, new List<string> { "abc", "def", "ghi", "jkl", "mno", "pqr", "XYZ", "123" });
            TimeIt(delegate { RunTest(dict1, keys_for_lookup); });
        }

// end of main

And this is what the results looked like :
It looks like using a Dictionary with a comparer injected is

faster
less hassle. You specify it once and forget it. You do not have remember to do UpperCase each time you lookup the map. (e.g. from the p.o.v. of future programmers)
less memory pressure. ToUpper() would create a new String instance per lookup.

From My conversations with gullible machines...

(Setup: .Net 3.5, VS2008, WinXP SP2, Intel T7400 Core 2, 2GB RAM on a normal work afternoon, lots of apps open)

The rest of the code for scrutiny:


private static void AddSomeKeys(Dictionary<string, object> dict1, List<string> keys)
    {
        foreach (var key in keys)
            dict1.Add(key, new Object());
    }

    private static void RunTest_Upper(Dictionary<string, object> dict1, List<string> keys)
    {
        for (int i = 0; i < 1000000; i++)
            foreach (var key in keys)
            { var v = dict1.ContainsKey(key.ToUpper()); }

    }
    private static void RunTest(Dictionary<string, object> dict1, List<string> keys)
    {
        for (int i = 0; i < 1000000; i++)
            foreach (var key in keys)
            { var v = dict1.ContainsKey(key); }
    }

    private static void TimeIt(Action action)
    {
        var s = System.Diagnostics.Stopwatch.StartNew();
        action.Invoke();
        s.Stop();
        Console.WriteLine("Elapsed msecs = {0}", s.ElapsedMilliseconds);
    }

WPF - Find a child control of a specific type or satisfying a given predicate under a parent control

2010-04-20T12:39:00.007+05:30

Ideally you shouldn't be doing this in WPF and MVVM land. However there is an exception to every rule.. mine was I had to set the focus and put a specific cell in a grid into edit mode when the user clicked a button.

Now this is one of the corner cases, for which you need to write some code-behind in the view. Next I needed to get a handle on the datagrid, which was nested within a few expanded data-templates. You can't name this grid, because each expanded data-template would have the same name value.

I did not want to hard code the entire hierarchy path of UI types from the tab to the nested grid. So what I needed was an API like


var grid = tab.FindChild<DataGrid>();

Unfortunately this isn't built-in functionality ; so I look on Stackoverflow and find a relevant question.
However it didn't work for me as is.

Some more debugging and searching later.. I find this post 'Understanding the Visual and Logical tree in WPF' by the always-enlightening WPF Guru Josh Smith. Now I'm not sure I can explain it to a third person..
but here is what I found by tinkering with a test-app.

From My conversations with gullible machines...

A DependencyObject can have m logical (shown in bold above) and n visual children.

The visual and logical children can overlap. e.g. a StackPanel can be a logical and a visual child of a parent at the same time

A logical child can be any object - does not necessarily derive from DependencyObject

With this knowledge, I wrote a few overloads of FindChild - source available here. Pretty self-explanatory.



// find a child textbox with a specific name
var textBox = tab.FindChild<TextBox>( t => t.Name == 'txtName');

// find all child stackpanels meeting a specific criteria (has children)
foreach(var panel in tab.FindChildren<StackPanel>( panel => panel.Children.Count > 0)
{ // do something with panel }

There is also another class - a VM for a Node in the UI Hierarchy, that I used to create the test app. I found it useful for diagnostics. e.g. if you have a reference _contentBox. You can bring up QuickWatch and add the expression 'new UITreeNode(_contentBox)' and visualize the logical and visual hierarchy under it. (Of course you need to paste the class definition into your project temporarily.)

From My conversations with gullible machines...

Set keyboard focus when the user begins editing a wpf datagrid templatecolumn ?

2010-04-16T15:44:00.008+05:30

The problem:
When I begin editing a grid cell, I'd expect the keyboard focus to be set the edit control for the cell. In case of a textbox, I'd also like the existing contents to be selected by default ; so that i can begin typing and set a new value quickly.

This works for the default data grid column types... except for a DataGridTemplateColumn ; a column whose display & edit behavior can be customized. You can use a templateselector to dynamically choose the templates to be used for displaying and editing values.

However if you attempt to tab into a TemplateColumn cell and hit F2 to begin editing, you'd find that you can't start typing in it. You need to click in it once or press Tab once more to do that. Compare the behavior of a TextColumn and TemplateColumn showing the same bound value as shown in the code below. The TextColumn behaves as expected. The edit control receives keyboard focus and all the text in it is selected.

Highly unintuitive.

The fix for the keyboard focus thing turned out to be setting FocusManager.FocusedElement attached property on the desired control in your Data Template for editing. Check out the sample below with/without the FocusedElement property setter :


           ...
           <DataTemplate x:Key="DefaultTitleTemplate">
               <TextBlock Text="{Binding Title}"/>
           </DataTemplate>
           <DataTemplate x:Key="EditTitleTemplate">
                        
                   <TextBox x:Name="Fox"
                       
                        FocusManager.FocusedElement="{Binding RelativeSource={RelativeSource Self}}" 
                        Text="{Binding Path=Title, Mode=TwoWay, UpdateSourceTrigger=PropertyChanged}"
                        Style="{StaticResource HighlightTextBoxStyle}"
                        >
                   </TextBox>
           </DataTemplate>
       </Grid.Resources>
       <DockPanel>
           <TextBox DockPanel.Dock="Top" x:Name="Test" Text="{Binding Path=(FocusManager.FocusedElement).Name, ElementName=MyWindow}"
                    Style="{StaticResource HighlightTextBoxStyle}"/>
           <toolkit:DataGrid ItemsSource="{Binding Items}" AutoGenerateColumns="False">
               <toolkit:DataGrid.Columns>
                   <toolkit:DataGridTemplateColumn Header="Templated Title"
                   CellTemplate="{StaticResource DefaultTitleTemplate}"
                   CellEditingTemplate="{StaticResource EditTitleTemplate}">

                   </toolkit:DataGridTemplateColumn>
                   <toolkit:DataGridTextColumn Header="Title" >
                       <toolkit:DataGridTextColumn.Binding>
                           <Binding Path="Title">
                               <Binding.ValidationRules>
                                   <local:CannotBeginWithX/>
                               </Binding.ValidationRules>
                           </Binding>
                       </toolkit:DataGridTextColumn.Binding>
                   </toolkit:DataGridTextColumn>
           ...

Still struggling to find the
fix to auto-select the text in the textbox in the expanded edit cell template.

Update: Got this to work by subscribing to the PreparingCellForEdit event. The editor is passed in as an event argument. For a templatedcolumn, the grid is unable to guess your editor control type. So you need to cast it to your particular editor type and call Focus()+SelectAll() on it. I subscribed to the event in the ctor and wasn't able to get the editor.. You need to wait till the Loaded event to subscribe. Then it worked.

Humanized's Enso Launcher : Jumpstart

2010-03-20T15:15:00.010+05:30

Enso is a good example of the axiom that 'Search is faster than navigation'. Its one of the tools that I was introduced to by the book 'The productive programmer' by Neal Ford. It has been handy ever since.

The concept of the tool is pretty simple... it redefines the way in which you communicate with your machine. e.g. If I want to install or uninstall a program, I do not want to click through a number of windows and dig through menus (and lose my focus by the time I get there). My intention is 'open add or remove programs' so that I can install that new app. It may be a tiny boost but multiplied by the volume of instances / day, Enso can save you a lot of time. So lets get right to it...

First you need to install Humanized's Enso launcher (12 MB download)
Next it's easier if you are a keyboard person instead of a mouse person. You need some dexterity with your left pinky finger

Enso takes over your capslock key once installed. So keep capslock pressed down with your left pinky (you should see a green floating band at the top-left) and continue typing to issue commands....

Opening/Launching programs

With Vista and Windows7, this is not something to write home about. Both of them have a real-time search box when you invoke the Start Menu - where you can enter a substring/part of the app you want to invoke and the start menu narrows down your choices.

However if you're on WinXP, you're gonna love this. e.g I always had to go hunting for the add/remove programs applet inside the Start Menu. (or pick something else e.g. like a specific version of Visual Studio). Soon I'd be piling on quick launch icons ; and even that required you to reach for the mouse. (I like to be one with the keyboard as long as possible).

Now with enso, reach out your left pinky to the capslock key. Keep it pressed, while you continue to type a substring e.g 'open remove' or 'open 2008' in the green floating band that just popped up. You'll see a dynamically updating dropdown that you can use to select with the arrow keys. When you lift your finger off the capslock key. The selected program shall be opened. No kidding... try it. It's addictive ; No program is ever more than one open command away.
Useful: Absolutely

Associate a file/directory/url/shortcut with a command

Assume you have a directory that you frequently access e.g.
d:\mail\snail\wail\jail
Ensure that the above path is selected (either in the location bar of windows explorer or write it down in notepad and select it). Next use your pinky and type in
learn as open jail
Enso should get back with 'open jail is now a command'

Now try opening like before.
Caps + 'open jail'
and you should see a windows explorer window pointed at the right folder.

You just taught Enso a new command 'jail'.
You can verify this by looking in your MyDocuments folder. You should see a folder created by Enso with all your commands.
This also means that you can carry your enso commands with you across machine (e.g. in a USB storage device), dump it to the local MyDocuments folder and off you go.

Of course Enso just takes over your capslock key, to regain that function, Enso has 2 commands. Keep caps pressed and type
'capslock on' OR
'capslock off'.

You can make Enso unlearn a command by saying
Caps + 'unlearn open jail' Enso wipes its memory cells clean.
Useful: Usually. e.g. you could associate your MRU visual studio solution file with a command 'curproj'

Now try this on selected files/shortcuts.
e.g. Select any exe or shortcut in your file explorer.
Caps + learn as open someexe
Now you can the exe is at your fingertips with
'open somexe'
Works for shortcuts too. You can even create a "rooted view" e.g. create a shortcut for %windir%\explorer.exe /e, /root, D:\tarpit\theNextKillerApp
Select it and learn it as a command. Try it out
Useful: Sometimes

Works for urls as well. Enso would open up your a new instance/tab of your default browser pointed at the learned url.
e.g. select http://www.answers.com/topic/dexterity right here. Teach enso a new command
Caps + 'learn as open dex'
Now you can open this page at any time using
open dex

Useful: Occasionally - since browsers have generally better at this; bookmarks, tags and incremental search in the address bar.

Jump/Switch to an open window or tab in a browser

This one is nice.
Caps + 'go google'
or
go taskjump directly to an open window which has that substring in the window title.
Its works for browser tabs too - if you had a minimized or buried browser with a tab showing google's homepage, the right browser and child tab would be activated. Its a shame it doesn't work with Firefox3.. seems to be a bug that could be fixed. Works with IE8
Useful: Pretty much ; wish I could unlearn alt+tab-ing as easily as Enso.

Open with ...

This one is supereasy but hasn't become second nature to me yet. You can select any file (e.g. in a file explorer)
Caps + 'open with [name of app]'

and Enso does just that. e.g. 'open with notepad' will open up the selected file in notepad.
Useful: Yes. No more Shift+right click > Open with ... > hunt for the exe if !inList. Freedom!!

Close active window/tab

the command is close. I like dismissing windows with
Caps +'cl'
Enso can do minimize and maximise too.

For more inbuilt commands, Use Caps + 'command list' to have a look at inbuilt commands. I believe you can script custom commands too (e.g. 'mail' instead of 'open Outlook') if you know a bit of Python. However that's outside the scope of this post :)

That's all folks!

HOW TO : Setup your Xtreamer to stream content off Windows (Win7) machines via a wifi network

2010-03-17T20:25:00.006+05:30

Update#2 [Apr 2010]: Sent the wireless USB dongle back to their seller in India since it was within the warranty period. They sent me a new dongle, free of cost. Now the whole setup is functional again. The online forum on the other hand was another deal altogether...

Update: My Xtreamer has stopped communicating via wifi after 3 days, without any configuration changes. "Can't get SSID" is the only response I get from the network settings page. The Wifi antenna drivers are proprietary (available for download) and hence I can't isolate the problem to the antenna. 2 subsequent posts to the Xtreamer forum have been mysteriously being deleted without any dialogue/clarification. Verdict as of now: Flaky antenna/connection finder.. wouldn't get this if streaming is your main need.

This post details my skirmishes in getting my Xtreamer (a streaming media player) to wirelessly stream content off my Windows7 laptop via a DLink N-Router. You could hook it up such that the Xtreamer can also access the internet - but I haven't done that. I intend to use the Xtreamer on a local trusted network and internet access on a diff public wireless network.

Step1: Update your Xtreamer firmware. I have version 2.3.1. You download the firmware zip, unzip it to get an .xtr file. Place the file in the root dir of a USB pendrive and connect it to the Xtreamer. Power it on - Main Menu > Settings > System > Upgrade. Browse and select the .xtr file. It should upgrade and reboot.

Step2: Get your network running and get your machine on it. Get your wireless router connected to your PC. Mine came with a CD with a wizard for basic setup.

Next enter the Web Config of the router. e.g. http://192.168.0.1
You would be prompted for the credentials to login into the router. Refer to your memory / the manual as necessary. Make sure you change your router password from the defaults.
For my DLink DIR-615, I navigate via Setup > Wireless Settings > Manual Wireless Conn Setup

SSID / Wireless N.w name = MySecretNetwork
802.11 mode = use mixed b/g/n if you're not sure
visibility = Visible.

You can hide it / set to Invisible after you've scanned the settings into the xtreamer.
Set SecurityMode = None initially ; till you are positive that you have the network working first.
If you have the network working, secure it. My settings are as...

SecurityMode = WPA Personal
WPA Mode = Auto (WPA / WPA2)
Cipher Type = TKIP
GroupKeyInterval = 3600 sec
Pre-sharedKey = supersecretpwd

Save these settings. This should reboot the router. So you'd have to reconnect again to wireless network - MySecretNetwork.
Click on the wireless icon in the tasktray - choose the network. You will be prompted for the password - key in your supersecretpwd

To verify that you're connected. Try getting to your router web config page again in a new browser window. It should work.

Step3: Get the Xtreamer on the network.
From the main menu, Settings > Network > Wireless Setup.

Select a connection profile: Choose a free slot e.g. Connection2 and press Right to go to the next screen.
Get Conn mode: Choose Infrastructure (AP) and go Right
You should see a drop down containing your network/ssid name. Woohoo! Go Right
Next it will prompt you for the WPA Key (for a secure network). Use the onscreen keyboard to key in your supersecretpwd and hit OK.
Choose DHCP IP (Auto) at the next screen.
Now you're at the moment of reckoning. You'd be shown a summary of your choices and asked to confirm. Go Right to test your connection.
If all goes well, you should see Test Ok and Show Net Info (summary of network parameters).
Press Enter to complete task.

You should now see
On*: [AssignedIP] as the value for the Wireless Setup property in the Settings > Network page.

Step4: Gentlemen (n ladies) share your folders

Select the folder c:\MyShare in Win7 and bring up Properties > Sharing > Advanced Sharing...
Enter a name for your share, a comment if you wish.
Next click on Permissions and add a credential (user/pwd/domain) that you'd use to access the share remotely. Use a non-guest account. I was not able to get it to work with the default guest account.

Repeat the above for all shares.
You also need to turn off at the network connection level

Click on the Network and Sharing Center link on the Sharing tab. Verify the following settings for the current (home or work) profile

Set Password protected shares = Off.
Public Folder sharing = On
File and printer sharing = On
Network Discovery = On

Tell your firewall to let in the media: Finally, you need to ensure that you're firewall is not gonna play spoilsport. Bring up your firewall.. Kaspersky in my case
Settings > Protection > Firewall > Settings.
Go to the networks tab.
Select your network (which is currently showing as Public). Right click and mark it as trusted. Easiest way to get this over with. Of course if you are accessing the internet this isn't the way to do it. You'd need to configure it at a more granular level. Ok out. You can verify by setting up 2 machines on the wifi network and pinging each other ; if packets are received correctly, you're all set.

Step5: Time to stream
So now you have the PC and Xtreamer on the network, shares created and reachable.

Press home on your Xtreamer to get to the main menu > Media Library > NET.
Select My_Shortcuts.
Press 1 on your remote. Select Add in the popup.
Use down and right to click the detail button.
Now navigate back up and enter the username, pwd, domain, File Server IP (ignore the other params) Press Enter in each field to bring up the onscreen keyboard. Use the numeric keys on the remote for the IP addr. Remember this is the set of credentials that you used on the Permissions dialog when you shared the folders.
Click OK. Save the network setting.

You should now see a shortcut like fileserverIP:username. Select it by pressing Enter. If you've entered everything correctly. You should see a red "Logon successful" at the top-right; below a listing of all shares on that machine.
Sometimes you maybe prompted for a user-name/pwd for an individual share. Not sure why this happens - however enter and choose 'OK and save'. Xtreamer will remember it for you from now on.

You should now see all files in the shared folder. Choose a movie and hit OK to begin the streaming... and yeah it's all set for you to scream in joy!

NUnit ExpectedException Bug in 2.4.7

2010-03-10T15:57:00.006+05:30

"Bug In NUnit ? It can't be!" Exactly my words ; However it turned out that the behavior is due to a newer NUnit runner (Gui v 2.5.3) running a test fixture referencing an older version of NUnit.Framework.dll (v2.4.7)

Try this in a NUnit test fixture referencing NUnit.Framework v 2.4.7


// 2.4.7 - test passes even if an exception of wrong type is thrown
        [ExpectedException(ExceptionType=typeof(FileNotFoundException))]
        [Test]
        public void Test_ExpectedException()
        {
            throw new ArgumentException();
        }

I found that the test passes if "any" exception is thrown. It doesn't have to match the specified type. A false positive!

To fix this, upgrade your NUnit version to 2.5.2. You'd also need to fix a compile error. 'ExceptionType' property has now become ExpectedException.


//2.5.2 'works as expected'
        [ExpectedException(ExpectedException=typeof(FileNotFoundException))]
        [Test]
        public void Test_ExpectedException()
        {
            throw new ArgumentException();
        }

You live... you learn.

Method resolution in Ruby : OR how to find Mr. Right method?

2010-03-02T14:30:00.007+05:30

Before I begin: Some of the stuff below may be incorrect (Corrections welcome.), however it helps me understand ruby method resolution. I have come up with this model based on my reading of the RubyProgrammingLanguage book (must-read for anyone learning Ruby) by Flanagan & Matz and some tinkering.. it may not be accurate (if that is what you're after)

Basket of Facts:

A Ruby type is an object of type 'Class' (henceforth referred to as a class-object)
An object has a class attribute pointing to the right class-object.
An object has an "eigenclass"/"metaclass" that holds singleton methods (methods that are only available to that instance.)

class-objects are special in a few ways:

a class-object has its class attribute pointing to the 'Class' type in Ruby
a class-object has a superclass
the "eigenclass" of a class-object also has a superclass; the eigenclass of its superclass
a class-object has an included modules list
a class-object has a table of methods (instance methods defined in the class definition)
class methods are singleton methods of the class-object
a class-object is generally associated with a constant viz. it's class name.

So now let's get started. Don't let the following mouse-drawn picture scare you - it'll all make (some?) sense in sometime. In the meantime, right click on it and open it up in a new tab and have the picture handy for reference while you descend into this post.

Next jump right over this big puddle of source code. You can copy it into your editor if you want to try it out with me..

module Kernel
  def breadcrumb
    puts "Kernel#breadcrumb"
    Module.nesting[0].instance_eval{ remove_method :breadcrumb }
  end
end
class Object
  def breadcrumb
    puts "Object#breadcrumb"
    Module.nesting[0].instance_eval{ remove_method :breadcrumb }
  end
  def self.breadcrumb
    puts "Object.breadcrumb"
    class << Module.nesting[0]
      instance_eval { remove_method :breadcrumb }
    end
  end
end

module MyModule
  def breadcrumb
    puts "MyModule#breadcrumb"
    Module.nesting[0].instance_eval{ remove_method :breadcrumb }
  end

end

class Base
  MY_CONST = :b
  include MyModule

  def breadcrumb
    puts "Base#breadcrumb";
    Base.class_eval{ remove_method :breadcrumb }
#     Base.instance_eval{ remove_method :breadcrumb } UH-OH! why does this remove the inst method
  end
  def self.breadcrumb
    puts "Base.breadcrumb"
    class << Module.nesting[0]
      instance_eval{ remove_method :breadcrumb}
    end
  end
end

class Derived < Base
  def breadcrumb
    puts "Derived#breadcrumb";
    Module.nesting[0].instance_eval{ remove_method :breadcrumb }
  end
  def self.breadcrumb
    puts "Derived.breadcrumb"
    class << self
      instance_eval{ remove_method :breadcrumb}
    end
  end
end

class Class
  def breadcrumb
    puts "Class#breadcrumb";
    Class.class_eval{    remove_method :breadcrumb  }
  end
end
class Module
  def breadcrumb
    puts "Module#breadcrumb";
    Module.class_eval{    remove_method :breadcrumb  }
  end
end

****LANDING ZONE****
Nice Jump!

I have defined a new class called 'Derived' which derives from 'Base' and includes 'MyModule'.
Unless a base type is explicitly specified, all classes derive from Object (which includes Kernel) (and derives from BasicObject - new in Ruby 1.9). e.g. Base derives from Object. This can be verified via

d = Derived.new
p d.class.ancestors # => [Derived, Base, MyModule, Object, Kernel, BasicObject]

Base and Derived each have defined a class method (self.breadcrumb) and an instance method (breadcrumb). MyModule just defines a module method of the same name.
Convention: Class/Singleton methods would print receiver.breadcrumb. Instance methods would print receiver#breadcrumb.

Take a deep breath and dive in...

First lets create a new instance of Derived and add a singleton method to it. (i.e. a method that is present only in that instance. Another instance of Derived would not have this method)
Here's how we do that...

d = Derived.new

class << d
def breadcrumb
puts "DerivedObj[id:%d].breadcrumb" % object_id
class << self
 instance_eval{ remove_method :breadcrumb}
end
end
end

'Call once': once the method is called, the method disappears (is 'undefined'). This will help us uncover the order in Ruby tries to find a method. I have added such methods in all types up the hierarchy (and then some) to help us spot the trail.
(Note: Instance methods use class_eval to remove themselves. Singleton/class methods use instance_eval. Refer to the RPL book)

And now for the moment of truth

puts "Method call on an object".center(40, "*")
6.times{ d.breadcrumb }

Output:
********Method call on an object********
DerivedObj[id:6186928].breadcrumb
Derived#breadcrumb
Base#breadcrumb
MyModule#breadcrumb
Object#breadcrumb
Kernel#breadcrumb

Refer to the image (I was reading Why's Nobody know Shoes; which might explain some of the weird imagery and the helpfulness of it all ;) So here's what happened in the sequence illustrated by our 'call-once' breadcrumbs:

To the extreme left is our object d,

Call1: Find the eigenclass of d, look for a singleton method of that name. Found!
Call2: .. not found in eigenclass, find the class-object of d via the class attribute. Look in the methods defined in class Derived. Found!
Call3: .. not found in eigenclass/class. Look in included modules of D (most recently included one first).. there are none. Travel via the superclass portal to reach Base. Look for inst method. Found!
Call4: .. not found in eigen/class/incl module/superclass. Try included modules of superclass e.g. MyModule. Found!
Call5: .. not found in eigen/class-n-its-modules/superclass-n-its-modules. Go down the superclass portal again to reach Object. Look at instance methods. Found!
Call6: .. look in incl. modules of Object. Kernel. Found!

If you try calling the method again, Ruby would not be able to find a method of that name and would now try looking for a method_missing method (in the same manner as it tried looking for breadcrumb. However this time it is guaranteed to find it because Kernel has a method_missing implementation that raises the NoMethodError. Need to see it to believe it?

temp.rb:84:in `block in

': undefined method `breadcrumb' for # (NoMethodError)
from temp.rb:84:in `times'
from temp.rb:84:in `

So lets recap the steps.
1. Look in the eigenclass (golden arrow) for the class
2. If not found, traverse the class attribute (purple arrow) to find the class-object.
3. Look in the instance methods for this class
4. If not found, try included modules of the class (in reverse order of mixin)
5. If not found, check if we have a superclass (pink arrow). If yes, repeat steps 3-5 till we run out of superclasses
6. If still not found, go back to Step1 but this time look for a 'method_missing' method'

By the way, we just saw how instance methods are inherited. (Snuck that one on you.) If Derived doesn't define an instance method, all its ancestors are looked up in order ; inheritance works.

Take a break, have some lemonade before we set out again... Ready? Off we go again. This time instead of calling a method on an object, we call a class method. Comment the previous 2 lines...

puts "Method call on a class-object".center(40, "*")
7.times{  Derived.breadcrumb  }

Output:
*****Method call on a class-object******
Derived.breadcrumb
Base.breadcrumb
Object.breadcrumb
Class#breadcrumb
Module#breadcrumb
Object#breadcrumb
Kernel#breadcrumb

Hey! we see some instance methods being invoked when we call a class method. What the... Deep breaths 1..2..3..pfffhoooo (u got to get the pfffhoooo right)

Let's see if our image can help us out, find the object pointed to by the constant Derived (3rd box from the left - first row)

Call1: Start at Derived. Follow the golden arrow to the Derived's eigenclass - Derived". Look for a singleton method. Found!
Call2: .. not found in eigenclass. But eigenclasses of class-objects have superclasses. Follow the golden arrow to Base's eigenclass. Found!
Call3: .. not found in Derived" or Base". But we still have a golden arrow to Object". Found!
Call4: .. not found in Derived"/Base"/Object".Try BasicObject" too. Not found. Now we're out of superclasses. Now we follow the purple arrow to the class-object of Derived i.e. Class. Look for instance methods defined there. Found!
Call5: .. not found in the eigenclass hierarchy or class-object. Try included modules in Class. not found. But wait, we have a superclass. Down the pink arrow to Module. Look for an instance method... found!
Call6: .. not found in eigenclass hierarchy/Class-n-included-modules/Module. Try included modules in Module. Nope. Down the pink arrow to Object. Found an instance method!
Call7: .. not found in eigenclass hierarchy/Class-n-incl-modules/Module-n-incl-modules/Object. Look in included modules of Object.. aha Kernel. Look for method.. found!

If you make one more call, you should now be guessing that a NoMethodError should show up from Kernel#method_missing(similar to the previous example)

So lets refactor the steps.

Look for the method in the eigenclass (yellow arrow). Until we run out of superclasses for the eigenclass, inspect each eigenclass for the method.
If not found, traverse the class attribute (purple arrow) to find the class-object.
Look in the instance methods for this class
If not found, try included modules of the class (in reverse order of mixin)
If not found, check if we have a superclass (pink arrow). If yes, repeat steps 3-5 till we run out of superclasses
If still not found, go back to Step1 but this time look for a 'method_missing' method

So the only new tidbits were

class methods are singleton methods of class-objects. They live in the green clouds tied with the yellow arrows to the class.
eigenclass of a class-object is special. These special 'green clouds' have superclasses.

And that is the secret behind class method inheritance... See the first three lines of the output. (Snuck another one.. {snicker} No more I promise.)

And that also explains how class method calls can result in instance methods being called. (Lines4-7)

And that's it folks! You will wake up enlightened or confused when I snap my fingers :)

As an exercise to people who like to stay and flex some meta-programming muscles. Try and explain this predicament.

Module defines a constants method, which returns the list of all known constants (incl. class names).
Module.constants => 102
Any user-defined class e.g. Base or Derived inherits this method ; however Base.constants returns only the constants defined inside Base (and its superclasses). e.g. If I add a constant inside Base.
Base.constants => 1
How does Module's constants behave differently in the Base (which inherits this functionality)?

Hint: Check Module.singleton_methods and Module.public instance_methods
Hint2: Hit Stackoverflow and search for the answer once you have taken a good crack at it.

Meet the frameworks : Rhino v Moq v jMock : Part II

2010-01-26T22:14:00.004+05:30

Story so far
We've seen how to
* setup expectations for calls on methods/properties.
* specify the number of times to be called.
* specify values to be returned

Now I move onto more involved scenarios.
#6:Throw an exception in response to an expected call on mockCollaborator.Method()

Rhino Mocks


       [ExpectedException(ExceptionType=typeof(UnableToServeException),
            MatchType=MessageMatch.Contains,
            ExpectedMessage="Sorry for the inconvenience.")]
       [Test]
       public void Test_ExpectExceptionToBeThrown()
       {
           _mockChef.Expect(chef => chef.Bake(CakeFlavors.Pineapple, false)).Throw(new Exception());

           _bakery.PlaceOrder(OrderForOnePineappleCakeNoIcing);
       }

The Throw method tagged onto the end of the Expect does the trick of raising any derivation of Exception (here I've raised a generic Exception)


       [ExpectedException(ExceptionType=typeof(UnableToServeException),
           MatchType=MessageMatch.Contains,
           ExpectedMessage="Sorry for the inconvenience.")]
       [Test]
       public void Test_ExpectExceptionToBeThrown()
       {
           _mockChef.Setup( chef => chef.Bake(CakeFlavors.Pineapple, false)).Throws<Exception>();
           _bakery.PlaceOrder( OrderForOnePineappleCakeNoIcing );
       }

Moq uses a generic Throws() to achieve the result

jMock


 @Test
 public void test_expectAndThrowException() throws OutOfIngredientsException
 {
   final Sequence aSequence = context.sequence("retryBake");
   context.checking( new Expectations() {{
     oneOf(_mockChef).bakeRealWorld( CakeFlavors.Pineapple, false );
       will(throwException(new OutOfIngredientsException() ));
       inSequence(aSequence);
     oneOf(_mockInventory).replenishStocks();
       inSequence(aSequence);
     oneOf(_mockChef).bakeRealWorld( CakeFlavors.Pineapple, false );
       inSequence(aSequence);
   }});
  
   _bakery.placeOrderRealWorld( OrderForOnePineappleCakeNoIcing() ); 
 }

The will(throwException(exceptionInstance) after an expectation does the trick. Ignore the lines in Sequence (I'll get to them in Section#9)

Java's checked exceptions and explicit method specification (all the way up the call-chain) feels a bit too rigid to me (maybe due to my .Net exposure). So I've created an overload of Bake that throws an exception, to avoid adding a throws clause to every test method that has an expectation for Bake(). Worked for me.. although it's not the right thing to do. The joys of coding in a toy example!

void bakeRealWorld(CakeFlavors flavor, boolean withIcing) throws OutOfIngredientsException;

#7:Custom callbacks or blocks of code to execute when a mockCollaborator.Method() is called
Let me cook up a use for this. Let's say I want the first call to Chef.Bake throw an OutOfIngredientsException(), so that I can call ReplenishStocks() and retry. This time, the call to Chef.Bake should go through.

Just the kind of job for... custom Callbacks!

Rhino Mocks


       [Test]
       public void Test_ExpectWithCustomCallback()
       {
           int callbacks = 0;
           _mockChef.Expect(chef => chef.Bake(CakeFlavors.Pineapple, false))
               .Repeat.Times(2)
               .WhenCalled(delegate(MethodInvocation mi) {
                   Console.WriteLine("Callback Params: {0}, {1}", mi.Arguments[0], mi.Arguments[1]);
                   callbacks += 1;
                   if (callbacks == 1)
                       throw new OutOfIngredientsException();

                   return;
               });
           _mockInventory.Expect(inv => inv.ReplenishStocks()).Return(true);
          
           _bakery.PlaceOrder(OrderForOnePineappleCakeNoIcing);
          
           _mockChef.VerifyAllExpectations();
           _mockInventory.VerifyAllExpectations();
       }

Tag on WhenCalled(delegateWithCustomCode) to the expectation. Also note that you have access to the method arguments as shown in the Console trace above.


       [Test]
       public void Test_ExpectWithCustomCallback()
       {
           var callbacks = 0;
           _mockChef.Setup(chef => chef.Bake(CakeFlavors.Pineapple, false))
               .Callback( delegate(CakeFlavors flavor, bool withIcing)
                           {   // any custom code
                               Console.WriteLine("Callback Params: {0}, {1}", flavor, withIcing);
                               callbacks += 1;
                               if (callbacks == 1)
                                   throw new OutOfIngredientsException();

                               return;
                           }
               );
          
           _mockInventory.Setup(inv => inv.ReplenishStocks()).Returns(true);
           _bakery.PlaceOrder( OrderForOnePineappleCakeNoIcing );
          
           _mockChef.VerifyAll();
           _mockInventory.VerifyAll();
       }

Tag on an overload of Callback( customDelegate ) at the end of the Setup to specify custom code to be executed.
Moq gives you strongly typed callbacks with upto 4 parameters (Action is the biggest Action type in .Net; a restriction that will be lifted with .net 4.0). Argument access is a snap.

jMock


// http://www.jmock.org/custom-actions.html
 @Test
 public void test_expectAndInvokeCustomCallback()
 {
   context.checking( new Expectations() { {
     oneOf(_mockChef).bake(CakeFlavors.Pineapple, false );
       will( MyCustomCallback.tracingCallback() );
   } } );
  
   _bakery.placeOrder( new Order(CakeFlavors.Pineapple, false, 1) );
 }
class MyCustomCallback implements Action
{
   public void describeTo(org.hamcrest.Description description)
   {
       description.appendText("a custom callback to do what I deem fit! ");
   }
  
   public Object invoke(Invocation invocation) throws Throwable
   { // any custom code
     System.out.println("Callback invoked with "
       + ((CakeFlavors)invocation.getParameter(0))
       + " and "
       + (((Boolean) invocation.getParameter(1)).booleanValue() ? "" : "no")
       + " icing!");
       return null;
   }
   public static Action tracingCallback()
   {
     return new MyCustomCallback();
   }
}

It may be due to my rusty Java but it turned out to be a whole lot of classes to achieve this with jMock

#8:Raise an event from a mock collaborator
This one is very handy when testing GUIs using a ModelViewPresenter (or related pattern)

Our inventory raises an event when we're running low on a particular ingredient. The bakery can use this notification to tell the Chef to go easy on that ingredient, while we stock our supplies.

Rhino Mocks


       [Test]
       public void Test_RaiseEventFromMock()
       {
           _mockChef.Expect(chef => chef.GoEasyOn("Sugar"));

           _mockInventory.Raise(inv => inv.RunningLowOnIngredient += null,
               this, new StockEventArgs("Sugar"));

           _mockChef.VerifyAllExpectations();
       }

To raise an event, invoke the Raise method on the mock with a null subscription to the event & the arguments to be passed onto the event handler method.


       [Test]
       public void Test_RaiseEventFromMock()
       {
           _mockChef.Setup(chef => chef.GoEasyOn("Sugar"));

           _mockInventory.Raise(inventory => inventory.RunningLowOnIngredient += null, new StockEventArgs("Sugar"));
           _mockChef.VerifyAll();
       }

Identical to Rhino except for the fact that you don't have to specify the 'sender' parameter.

jMock

Java doesn't support events as first-class citizens. You need to manually implement the publisher-subscriber pattern.

#9:Argument Constraints
Sometimes, we do not want to / cannot specify the exact arguments with which a method on a collaborator will be called as part of the expectation. In such cases, we use constraints on the arguments, which must be met for the expectation to be met.
e.g. from the previous example, lets say we don't know the value of the Low-stock-ingredient at compile-time. We have a variable into which we can fetch the ingredient and which we can use to compare against the actual value.
In short we want to specify a Predicate (a function which returns true/false) for an argument as a constraint.

Rhino Mocks


       [Test]
       public void Test_ConstrainingArguments()
       {
           var itemAboutToGoOutOfStock = "Eggs";
           _mockChef.Expect(chef => chef.GoEasyOn(
               Arg<string>.Matches (ingredient => ComplexCheck(itemAboutToGoOutOfStock, ingredient))));

           _mockInventory.Raise(inventory => inventory.RunningLowOnIngredient += null,
               this, new StockEventArgs(itemAboutToGoOutOfStock));
           _mockChef.VerifyAllExpectations();
       }
       private bool ComplexCheck(string expected, string actual)
       {
           return expected.Equals(actual); // any complex test
       }

So here we use Arg.Matches(predicate) to do a complex constraint on the string argument for Chef#GoEasyOn(). There are a bunch of other helpful constraints - explore the Arg type docs (or the intellisense popup) for more.


       [Test]
       public void Test_ConstrainingArguments()
       {
           var itemAboutToGoOutOfStock = "Eggs";
           _mockChef.Setup(chef => chef.GoEasyOn(
               It.Is<string>(ingredient => ComplexCheck(itemAboutToGoOutOfStock, ingredient))));

           _mockInventory.Raise(inventory => inventory.RunningLowOnIngredient += null,
               new StockEventArgs(itemAboutToGoOutOfStock));
           _mockChef.VerifyAll();
       }
       private bool ComplexCheck(string expected, string actual)
       {
           return expected.Equals(actual); // any complex test
       }

Moq has It.Is(predicate) for this purpose. Explore the type documentation (or the intellisense popup) for more helper constraints.

jMock

The example is different. Here we are constraining the arguments to the Bake call with a predicate.


//http://www.jmock.org/custom-matchers.html
 @Test
 public void test_expectAndCheckArgs()
 {
   context.checking( new Expectations() { {
     oneOf(_mockInventory).isEmpty();
       will(returnValue(false));
     oneOf(_mockChef).bake(
       with( FlavorMatcher.aPineappleFlavoredCake() ),
       with( any(boolean.class)));
   }} );
   _bakery.pleaseDonate( OrderForOnePineappleCakeNoIcing() );
 }


class FlavorMatcher extends TypeSafeMatcher<CakeFlavors>
{
 public boolean matchesSafely(CakeFlavors flavor)
 {  // any complex test

     return flavor == CakeFlavors.Pineapple;

 }
 public void describeTo(org.hamcrest.Description description) {
       description.appendText("a cake with pineapple flavor ");
 }

 @Factory
 public static Matcher<CakeFlavors> aPineappleFlavoredCake()
{
   return new FlavorMatcher();
 }
}

once again more classes than I'd liked (Disclaimer: rusty java programmer could be the problem).

Use with(matcherInstance) in place of arguments in the expectation. Derive a new Matcher derived class with your custom check.

#9:Expect a sequence of calls in a specific order
Ordered Expectations: Looking at the test in Section#4 - the test can be satisfied by the following implementation


       public void PleaseDonate(Order order)
       {
           _chef.Bake(order.Flavor, order.WithIcing);
           var isEmpty = _inventory.IsEmpty;
       }

which is not what we were getting at. Seems like the test is missing something... the order of calls! Chef.Bake should be called after Inventory.IsEmpty.

Rhino Mocks


       [Test]
       public void Test_ExpectCallsInOrder()
       {
           var mockCreator = new MockRepository();
           _mockChef = mockCreator.DynamicMock<Chef>();
           _mockInventory = mockCreator.DynamicMock<Inventory>();
           mockCreator.ReplayAll();   // to avoid setting up an expect for an event handler subscribe in the following ctor.

           _bakery = new Bakery(_mockChef, _mockInventory);

           using (mockCreator.Ordered())
           {
               _mockInventory.Expect(inv => inv.IsEmpty).Return(false);
               _mockChef.Expect(chef => chef.Bake(CakeFlavors.Pineapple, false));
           }

           _bakery.PleaseDonate(OrderForOnePineappleCakeNoIcing);

           _mockChef.VerifyAllExpectations();
           _mockInventory.VerifyAllExpectations();
       }

Woooaah! What just happened? Have a seat ; take a deep breath.
Ordered expectations have been a feature of Rhino Mocks; however the v3.5 syntax revamp hasn't made an impact here. This took some doing .. Turns out you need to go old-skool for a bit.
The key:
* Create a new MockRepository instance and invoke the DynamicMock method (instead of the static MockRepository.GenerateMock )
* Use the Ordered block to setup expectations in a specific order


       [Test]
       public void Test_ExpectCallsInOrder()
       {
           var sequence = String.Empty;
           _mockInventory.Setup(inv => inv.IsEmpty).Returns(false)
               .Callback(delegate { sequence += "1"; });
           _mockChef.Setup(chef => chef.Bake(CakeFlavors.Pineapple, false))
               .Callback(delegate { sequence += "2"; });

           _bakery.PleaseDonate(OrderForOnePineappleCakeNoIcing);

           Assert.AreEqual("12", sequence, "Calls not in expected order!");
           _mockChef.VerifyAll();
           _mockInventory.VerifyAll();
       }

Last time I checked (Jan25,2010) Moq does not support this out of the box. However you could work around this with a collecting object. Very naive.. you could come up with better solutions to fit your exact context.

jMock


 @Test
 public void test_expectCallsInSpecificOrder()
 {
   final Sequence pleaseDonateSequence = context.sequence("pleaseDonate");
   context.checking( new Expectations() {{
     oneOf(_mockInventory).isEmpty();
       inSequence(pleaseDonateSequence);
       will(returnValue(false));
     oneOf(_mockChef).bake(CakeFlavors.Pineapple, false);
       inSequence(pleaseDonateSequence);
   }});
  
   _bakery.pleaseDonate( OrderForOnePineappleCakeNoIcing() ); 
 }

jMock has the shortest syntax for ordered expectations. You create a named Sequence object and then tag the name along with each expectation (listed in the expected order).

So there you have it. If it was a showdown, jMock would be ahead of Moq and Rhino Mocks would be just a sliver behind Moq.

JMock:
+ has a nice readable concise API for number of invocations and ordered expectations

Moq:
+ has a nice mechanism for type-safe custom callbacks and argument constraints

Rhino:
+ isn't bad. Does everything. However even with the v3.5 syntax revamp, it is a little unintuitive as compared to the others. The documentation was a bit noob-unfriendly, when I needed it. However it is the only option if can't use .net 3.5 for your tests (which should be rare).

Meet the frameworks : Rhino Mocks v Moq v jMock : Part 1

2010-01-26T17:46:00.016+05:30

If you've been near any TDD-infected person, you're sure to have heard about Mock frameworks (or served as a by-standing casualty in a classicist v Mockist skirmish). Then you have to look. At first you're apprehensive, it looks like too much work (defining interfaces, creating mocks, expectations, et. all). Soon you're cornered by a Data Access object or a Network stream and you have no choice. You dip your feet into the pool of mocks.. soon enough you're splashing around like a silly kid creating mocks willy-nilly (they're free). Unknown to you, the pool has some whirlpools that suck you in when you're having too much fun and not paying any attention. In the end, it's like mock-Children of the Corn..

I'm a sucker for horror stories (and there's that other little matter of being biased towards state-based tests.) As with most things, there is more than one way to use a tool - mock frameworks are no exception. The folks at mockobjects.com have a new book out which explains the right way (not read it yet; maybe I'll still be converted into a mockist.)

Personally my pet-peeve with mock frameworks has been the learning curve. Usually you don't have a dedicated time-slot to learn a mock framework (and that's kind of a good constraint since the mock frameworks are flexible, powerful and consequently huge); I'd bet that most of us learn them while trying to get a test to pass. Nothing like a reluctant mock framework with cryptic error messages between you and a green bar, to get you into real-curse-practice. The only thing that comes a distant second is "Which is better? Rhino Mocks or Moq ?" from people who haven't explored either of them. If both of them scratch your itch, take a pick!

So I've tried to list down the most probable scenarios and try each of them with RhinoMocks, Moq and jMock (always wanted to write some tests in jUnit).

You could use this to make your own choice (which is only yours to make). I've written this down as a self-reference for the next time I'm wondering "Why doesn't this @#&*(@ test fail?' or 'How is this test passing?'

A template for interaction-tests

For state-based tests, I'd usually write my assert first, step back to the action that will lead to the assert and finally arranging everything for the test to work. However this doesn't exactly work for interaction-based tests: In this realm, it's more helpful to write the action first, then the expectations and other steps for arrange and finally assert (which sometimes is just verifying that the expected mocks were called. Some frameworks like jMock do it automatically at the end of the test.)
Arrange

create mock objects for collaborators that you want to mock( not Stub. I hope you've read Martin Fowler's paper [link] on the difference.)
Create your test subject (here a Bakery type) and inject the mock collaborators via ctor/setter methods.
Define the interaction via 'expects'. e.g. I expect the test subject to call methodX on collaboratorA with these parameters.

Act

State the action/behavior that is the primary focus of this test

Assert

Assert any state-change (if applicable)
Verify all expected calls on the collaborators were called.

Source Code:
The source code for this series is available for browse/checkout (you'll need subversion to checkout) at
Dot Net code
Java code

#1: Expect a Call on mockCollaborator.method()

Rhino v3.5 (with NUnit 2.4.8)

Let's see how we do this in Rhino Mocks 3.5 - use the above section to work through this example.
Rhino mocks has changed with a (IMHO noob-friendly/simpler) syntax with version 3.5 - Prior to that an explicit record-replay style was the norm. So if you're using a prior version, try moving to v3.5. (There is a forsaken commented-out source file in the above mentioned link for old-skool Rhino).

To declare that you expect a method to be called, you use the Expect method on the respective mock object. Lambda expressions, Generics and type-inference result in compile-time checks and Intellisense guiding you all the way.
The below expectation reads "On the mockChef, expect Bake to be called with Vanilla and false as arguments." (when someone calls PlaceOrder() on the bakery, which employs the Chef)
I'm not going to post the code to make this pass inline (for honoring the memory of brevity in blog posts). I suggest you try TDDing your way through like me / refer to the source repository.


[Test]
public void Test_ExpectMethodCall()
{  // arrange : create mocks
    var mockChef = MockRepository.GenerateMock<Chef>();

    // arrange: inject dependencies
    Bakery bakery = new Bakery(mockChef);
    // arrange: expect the Bake method on Chef to be called
    mockChef.Expect(chef => chef.Bake(CakeFlavors.Vanilla, false));

    //act
    bakery.PlaceOrder(new Order { Flavor = CakeFlavors.Vanilla, WithIcing = false, Quantity = 1 });

    //assert
    mockChef.VerifyAllExpectations();
}

Moq v 2.0.5 (with NUnit 2.4.8)

Moq was the new kid on the block challenging the leading .Net mock frameworks with .Net3.5 lambda-expressions powered snappy syntax.


  [Test]
  public void Test_ExpectMethodCall()
  {
      //arrange
      var mockChef = new Mock<Chef>();

      //arrange: inject dep
      var bakery = new Bakery(mockChef.Object);
      // arrange: expect
      _mockChef.Setup(chef => chef.Bake(CakeFlavors.Vanilla, true));

      //act
      _bakery.PlaceOrder(new Order { Flavor = CakeFlavors.Vanilla, WithIcing = true, Quantity = 1 });
      //assert
      _mockChef.VerifyAll();
  }

Differences
#1: mockChef.Object is the mock collaborator; this is a common stumble for people new to Moq. I think (unverified) this is due to Moq's design of not having a different Repository or Mockery type.
#2: Moq uses Setup for expecations (instead of Rhino's Expect)

jMock 2.5.1 (with jUnit 4.8.1)

jMock is the Java mock framework distilled from the experience of the gurus at mockobjects.com ; they have a new book out and are the recognized pioneers of the "Mock objects" technique. I highly recommend their second revision of the acclaimed 'Mock Roles not objects' paper.
So let's see how the test looks like in jMock:


Mockery context = new JUnit4Mockery();

@Test
public void test_expectCall()
{
Chef mockChef = context.mock(Chef.class);
Bakery bakery = new Bakery(mockChef);

context.checking( new Expectations() { {
 oneOf(mockChef).bake(CakeFlavors.Pineapple, false );
} } );

bakery.placeOrder( new Order(CakeFlavors.Pineapple, false, 1) );
// mockChef.Verify() - autoverified : framework :)
}

Differences
#1: jMock has a more fluent approach to specifying expectations - look inside the Expectations creation. It reads 'Expect one invocation on mockChef of method bake with parameters Pineapple and false. Refer to the nicely written jMock cheatsheet in case you're feeling a bit lost.
#2: You don't need to explicitly verify expectations - automatically done at the end of a test case.

#2 Expect a call on mockCollaborator.getProperty and return something
I've factored out common code into a SetUp method to shorten the code snippets. In general, .Net properties = one get_property method + one set_property method + one backing field. Syntactic sugar!

So when someone queries Bakery.IsOpen, check to see if the Chef is available.

Rhino Mocks


      [Test]
      public void Test_ExpectPropertyGet()
      {
          _mockChef.Expect(chef => chef.IsAvailable).Return(true);
          Assert.IsTrue(_bakery.IsOpen);

          _mockChef.Expect(chef => chef.IsAvailable).Return(false);
          Assert.IsFalse(_bakery.IsOpen);

          _mockChef.VerifyAllExpectations();
      }

Note the new Return method tagging along with the Expect to signify what should be returned when the Bakery makes a call to Chef#IsAvailable


      [Test]
      public void Test_ExpectPropertyGet()
      {
          _mockChef.Setup(chef => chef.IsAvailable).Returns(true);
          Assert.IsTrue(_bakery.IsOpen);

          _mockChef.Setup(chef => chef.IsAvailable).Returns(false);
          Assert.IsFalse(_bakery.IsOpen);

          _mockChef.VerifyAll();
      }

Moq calls it Returns which IMHO reads better.

jMock

Java does not have properties as first class language features. They are manually implemented as accessor/mutator method pair and a backing field.

#3: Expect n calls to mockCollaborator.Method()
So if I place an Order for 3 Pineapple cakes, I'd expect the Chef#Bake to be called thrice.

Rhino Mocks


      [Test]
      public void Test_ExpectMultipleCalls()
      {
          _mockChef.Expect(chef => chef.Bake(CakeFlavors.Pineapple, true)).Repeat.Times(3);

          _bakery.PlaceOrder(new Order { Flavor = CakeFlavors.Pineapple, WithIcing = true, Quantity = 3 });
        
          _mockChef.VerifyAllExpectations();
      }

Tag on the Repeat property to the Expect if you expect it to be called n times.
Caveat: the test fails if the mock isn't called at least 3 times. If you call it 3 or more times, the test passes. So if you have a need for exactly thrice, dive into the rhino mocks docs and let me know if you figure out a way to make it work.


       [Test]
       public void Test_ExpectMultipleMethodCalls()
       {
           _bakery.PlaceOrder(new Order { Flavor = CakeFlavors.Pineapple, WithIcing = false, Quantity = 3 });
           _mockChef.Verify( chef => chef.Bake(CakeFlavors.Pineapple, false), Times.Exactly(3), "Chef should have been called thrice!");
       }

Differences:
#1. With Moq, you specify this during verification with an overload of Verify (as opposed to while defining expectations)
#2 This test will pass only if Bake is called Exactly thrice (unlike Rhino)

jMock


 @Test
 public void  test_expectMultipleCalls()
 {
   context.checking( new Expectations() {{
     exactly(3).of(_mockChef).bake(CakeFlavors.Pineapple, false);
   }} );
 
   _bakery.placeOrder( new Order(CakeFlavors.Pineapple, false, 3) );
 }

jMock wins this one for readabililty.
#4: Return values from mockCollaborator.Method()

Refer to #2 for Rhino and Moq. You tag on a Return/Returns method call to the expectation.
Let's say the bakery being a charitable organization, responds to a PleaseDonate message at the end of the day from cake-loving orphans. The bakery will ask the Chef to bake one for them after checking to see if they have surplus stocks in inventory.

jMock


 @Test
 public void test_expectCallAndReturnValue()
 {
   context.checking( new Expectations() {{
     oneOf(_mockInventory).isEmpty();
       will(returnValue(false));
     oneOf(_mockChef).bake(CakeFlavors.Pineapple, false);
   }} );     
  
   _bakery.pleaseDonate( OrderForOnePineappleCakeNoIcing() );
 }

jMock has a will method to signify what will happen when the method is invoked. Use the indentation (convention) so that it reads better.

#5:Expect no call (Don't call me!)
When the orphans send a PleaseDonate() and there are no surplus stocks in inventory, you wouldn't want to bother the overworked chef at the end of a weary day.

Rhino Mocks


       [Test]
       public void Test_ExpectNoCall()
       {
           _mockInventory.Expect(inv => inv.IsEmpty).Return(true);
           _mockChef.Expect(chef => chef.Bake(Arg<CakeFlavors>.Is.Anything, Arg<bool>.Is.Anything))
               .Repeat.Never();

           _bakery.PleaseDonate(OrderForOnePineappleCakeNoIcing);

           _mockInventory.VerifyAllExpectations();
           _mockChef.VerifyAllExpectations();
       }

Note: This snippet gives you a sneak peek of Argument Constraints. In short the second line states 'mockChef#Bake should not be called (irrespective of arguments)' Bring the .Repeat.Never() clause into plain view by placing it on a new line with one level of indentation.


       [Test]
       public void Test_ExpectNoCall()
       {
           _mockInventory.Setup(inventory => inventory.IsEmpty).Returns(true);
           _bakery.PleaseDonate(OrderForOnePineappleCakeNoIcing);

           _mockChef.Verify(chef => chef.Bake(It.IsAny<CakeFlavors>(), It.IsAny<bool>()), Times.Never());
           _mockInventory.VerifyAll();
       }

Differences:
#1 Moq takes a different path of leaving the invocation-count check till the very end. Use the Verify Overload with Times.Never() as in Section#3
#2 Argument constaints in Moq read slightly better than Rhino.

jMock


 @Test
 public void test_expectNoCall()
 {
   context.checking( new Expectations() {{
     oneOf(_mockInventory).isEmpty();
       will(returnValue(true));
     never(_mockChef).bake( CakeFlavors.Pineapple, false );

   }});
  
   _bakery.pleaseDonate( OrderForOnePineappleCakeNoIcing() ); 
 }

jMock doesn't hide intent with never() leading the expectation. jMock's API shows the thought that's been put into it.

5 down. 5 for the next part.

My TDD Workshop

2009-10-23T19:31:00.000+05:30

I was recently asked to conduct a workshop (read everyone-off-their-armchairs-and-type-with-me) on Test Driven Development. Now since this is something that I love (could not be any more apt for my 100th blog-post), I kind of "invested all of my self esteem in this presentation" as Dilbert would have said.

It ain't a workshop till you have slides :) Workshop Slide-set
The source code is available for download at Google code as a project

I don't have the code examples complete - I didn't like the way the GUI example turned out ; gonna give it another whirl soon.

Test driven development Workshop 2009 by Gishu Pillai is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License.
Based on a work at docs.google.com.
Permissions beyond the scope of this license may be available at
my-tdd-workshop.html

My guiding principles were

to give the attendees a true taste of TDD (vs automated-tests after vs test-first programming)
to set up beacons to avoid the usual pits, which sink time, effort, people, projects
to condense what I felt were in the "must-know" category

Test "driven" design - emergent design
SOLID Principles
Mocks and Dependency Injection
Model View Presenter / Humble dialog way of building GUIs

to learn myself from the interaction (I've learned from a lot of people online. It 's only fair that I give back)

So I got onto google docs, fired up a presentation, wrote up a rough list of things that I considered important and off I went towards an initial draft. I asked around for a good non-toy problem for the code-alongs... to my surprise I didn't net that many.. so I settled for the Bowling Example of TDD yore (Ron Jeffries, Uncle Bob, et. all)
"Score a complete bowling game as per the standard scoring rules."

I also needed an extension to the problem that involved a bit of Mocks and GUIs (for some real world flavor) - so I extended the problem (of course with my "customer hat" on)
"Create a View/Display for the Bowling Game that shows pins rolled and score updates in real time."

I then tried to solve the problem myself. The second problem turned out to be more time-consuming than I imagined. (A definite possibility is that I had lost my touch with TDD and was rusty due to lack of practice - a shoutout to Naresh Jain, for helping me realize that)
Finally I published the draft online to the testdrivendevelopment yahoo group for scrutiny ; As always I got a lot of help and nice suggestions. (A big thank you! to all of you) I incorporated most of them - including expanding my 1-day plan to a 2-day (and finally a 3-day plan - Charlie Poole can go 'told you so')

I then conducted the first instance of the workshop with 13 attendees.
The first day went as per plan. TDD Origins, SOLID Principles, Bowling Game Part I.
However the second day, the rails came off. They loved the Rhino Mocks jumpstart - however coding along ate up a lot of hours. I skimmed the Moq equivalent exercise and somehow rushed through to the end. The grand-finale of building a GUI (which was supposed to illustrate how you can design incrementally and it's okay if you dont have a 30 page design-document blessed by the powers that be, if you pay attention to design all the time) had to be left as an "exercise for the attendee". I'll need to extend the workshop to a full 3 days next time.

Had the opportunity to do a lot of little experiments - like using a Chess Timer to time how much time I spent writing tests to code. Good fun overall! It also showed me that I need to tune my content a bit more - the anonymous feedback was encouraging. Scored a 100% on "Would you recommend this (session) to someone else" ? Just the impetus I needed to sink another bunch of hours tinkering with the content.

Testing .Net code with Cucumber and IronRuby

2009-09-14T17:09:00.000+05:30

As promised, this is the culmination of this trilogy. The previous two posts were a jump-start on Cucumber to test Ruby code. Next I moved towards using Cucumber's plain text stories to test .Net code.Aslak Hellesoy has a wiki-post on how to do this ; however I found that you get eaten and spat out by a lot of IronRuby dragons along the way. So hold on..

1. Install the new DLR

First up install .Net Framework 4.0 (I have Beta1) - this has the new Dynamic Language Runtime (DLR) that makes things like IronRuby and IronPython possible.

2. Get the latest IronRuby release

Next we need to get IronRuby as a zip. Extract it to say d:\ironruby-0.9.0 Add the path to the bin folder to your PATH environment variable to avoiding lengthy paths. Drop to a command shell, type 'ir' to invoke the interactive IronRuby console. Type something simple to test if it works. It should.

>>>puts "Hello IronRuby"

3. Dragons ahead. Use diversion

But I hit a snag with this version of IronRuby, which had A BUG.

So I had to get the latest (8 Sep 2009 master to be precise) version of the source from the github repository to overcome that. You can download it as a zip or use git as you prefer. I got the zip
ironruby-ironruby-90cdda82fd60f4b7e6d7d940501c586d55954466.zip (Could have used a shorter name)

Extract it to say d:\ir-src
I hit a few path-too-long errors during extraction. Just keep skipping them. I think it's because of the rather large alphanumeric string which is the name of the top-level folder.

Once extracted, navigate to D:\ir-src\ironruby-ironruby-90cdda82fd60f4b7e6d7d940501c586d55954466\Merlin\Main\Languages\Ruby
and open Ruby.sln in VS2008 (I have Dev Edition of VSTS.. Although it should build in VS Express Editions as per the docs - I couldn't get the solution to open in it.)
Build Solution. At the end of it you should find the built binaries in
D:\ir-src\ironruby-ironruby-90cdda82fd60f4b7e6d7d940501c586d55954466\Merlin\Main\bin\Debug

Rename that directory with the huge name to something like "ir".

Copy all built binaries and overwrite the ones from the 0.9.0 release's bin folder i.e. D:\ironruby-0.9.0\bin.

Open ir.exe.config in the same folder and update the Library paths element to the proper paths to folders within your ir-src folder. 'Handle with extreme care' or you'll lose hours chasing error messages. It should read
Path#1 - D:\ir-src\ir\Merlin\Main\Languages\ (NOT D:\ir-src\ir\Merlin\External.LCA_RESTRICTED\Languages)
Path#2 & #3 - D:\ir-src\ir\Merlin\External.LCA_RESTRICTED

My xml looks like
```
<set language="Ruby" option="LibraryPaths" 
value="D:\ir-src\ir\Merlin\Main\Languages\Ruby\libs\;D:\ir-src\ir\Merlin\External.LCA_RESTRICTED
\Languages\Ruby\redist-libs\ruby\site_ruby\1.8\;D:\ir-src\ir\Merlin\External.LCA_RESTRICTED\Lang
uages\Ruby\redist-libs\ruby\1.8\" />
```

4. Wrapper script ICucumber

Lastly we need a wrapper script to invoke cucumber with Ironruby. Create a file called icucumber.bat under your Ruby bin folder i.e. D:\Ruby\bin\icucumber.bat with the following text.

REM Update with appropriate values for GEM_PATH, Ruby bin and the path to ir.exe
@ECHO OFF
REM This is to tell IronRuby where to find gems.
SET GEM_PATH=d:\ruby\lib\ruby\gems\1.8
"D:\ironruby-0.9.0\bin\ir.exe" -D -X:ExceptionDetail "d:\ruby\bin\cucumber" %*

5. Back to our example from the previous two posts.
The .feature file stays unchanged.
Delete c:\cukes\dot_net_features\support\BowlingGame.rb ; since we're going to implement the same in C# this time around as
c:\cukes\dot_net_features\support\BowlingGame.cs


namespace CukesDemo
{
   public class BowlingGame
   {
      public void roll(int pins_knocked_down)
      {
        Score += pins_knocked_down;
      }

      public int Score
      {
         get; set;
      }
      public bool Over
      {
         get { return false; }
      }
   }
}

Also create a small batch file to compile it to a DLL (Assumes csharp compiler is on the PATH).
c:\cukes\dot_net_features\support\Compile.bat

IF EXIST bin GOTO COMPILE
MKDIR bin
:COMPILE
csc /t:library /out:bin/BowlingGame.dll bowling_game.cs

Finally back to our step definitions to check the glue. Four changes needed - explained in comments.
cukes\dot_net_features\step_definitions\bowling_game_steps.rb

# CHANGE 1 : Add bin folder to load-path
$:.unshift(File.dirname(__FILE__) + '/../support/bin')   
# CHANGE 2 : Get BowlingGame.dll
require 'BowlingGame'      

Given /^I am starting a new game$/ do
   # CHANGE 3 : Use Namespace::ClassName.new  
   @game = CukesDemo::BowlingGame.new   
end

When /^I roll (\d+) gutter balls$/ do |count|
  count.to_i.times{
    @game.roll(0)
  }
end

Then /^the score should be (\d+)$/ do |expected_score|
  @game.score.should == expected_score.to_i
end

Then /^the game should be over$/ do
  # CHANGE 4 :  be_over passes even if Over returns false. Don't know what is the equiv of over? 

in .Net
  #~ @game.should be_over == true 
  @game.over.should == true
end

When /^my rolls are (.*)$/ do |rolls|
  rolls.split(' ').each{|roll|
    @game.roll(roll.to_i)
  }
end

Now for the grand finale, run Cucumber to verify our .Net DLL via IronRuby !!!

HTH

Scenario Outlines and Tagging in Cucumber

2009-09-12T16:14:00.000+05:30

Post#2 in this trilogy.

Scenario Outline

Scenario tables are similar to Fit's ColumnFixture and NUnit's RowTest. You run the same scenario with different inputs each time.
Let's go back to our plain text feature and add the following.

Scenario Outline: score should be as per the std rules
  Given I am starting a new game
  When my rolls are <rolls>
  Then the score should be <score>

 Scenarios: lets go bowling
 | rolls   | score |
 |5 2      | 7     |
 |5 5 5    | 15    |

The key things to remember here are the 'Scenario Outline:' marker to indicate that it is an outline. We then have placeholders within angular brackets. The Outline is then followed by one or more tables identified by the marker 'Scenarios:'
The next line should contain column headers which correspond with the placeholders in the outline. Cucumber would substitute the values to run each row against the Outline.

SARC and it should fail. Let's go fix that up in bowling_game.rb

class BowlingGame
 attr_reader :score
 def initialize
   @score = 0
 end
 def roll( pins_knocked_down)
   @score += pins_knocked_down
 end
 def score
   @score
 end
 def over?
   true
 end
end

SARC. Now you should see this nice listing.

Tagging

You can tag a scenario (or a feature) with one (or more) tags.

@important, @fast
Feature: Bowling Game Score Calculation
 In order to let the player know his score

By default, cucumber runs all .feature files in the features subfolder. You can group features in different subfolders as well e.g. I can define a new.feature within a new_features subfolder, which can then be run with

>cucumber new_features

Overtime, it can get crowded with lots of subfolders. But tagging is here to save the day..
To run only features/scenarios marked 'fast'

>cucumber --tags @fast

To run features/scenarios that are not marked 'fast'

>cucumber --tags ~@fast

So that's another way to quickly sort out your tests.

Language Support

The next thing you'd probably want to know (just in case you need it) is that cucumber speaks multiple languages. You can write your .feature file in any language - provided that the necessary entries are made in the resource file - languages.yml.

Next post - getting cucumber to test a .Net app with some help from IronRuby. Piqued?

Resources on BDD / Cucumber (although I felt the majority were too entwined with Rails (e.g. WebRat for testing Web Apps via free step definitions) but then that very well could be the major user-base for cucumber right now.)

ScreenCast by Aslak Hellesoy Executable User Stories with RSpec and BDD

ScreenCast by Ben Mabey Outside in development with Cucumber

Blogpost series by Dave Astels Cucumber: More advanced

RailsCast episodes by Ryan Bates Beginning with Cucumber

Green in 30 mins : Getting Started with Cucumber (a test framework for BDD)

2009-09-12T13:29:00.000+05:30

Prologue:
BDD is TDD with an outside-in (top-down) bent.
The way to tackle any new feature request is to ask Why 5 times? An you should arrive at one of Protect Revenue, Increase Revenue or Manage Cost. If not, chances are you’re building something that is not needed.
A feature can be summarized in a few lines as (ala Mike Cohn’s user story format)

As a <role>
I want <feature>
So that <value>

Dan North began this journey with JBehave.
It evolved over time into a Ruby gem - RSpec courtesy David Chelimsky n co. RSpec consisted of two parts - example runner and a story runner. The story runner runs features (think big cog) written up as plain text files which can be ‘executed’ (via some developer-added connecting-glue code in a separate location). The example runner (think small cog) is a bunch of module specs that make the feature happen. (think xUnit for BDD).
The RSpec story runner has now been simplified/improved and become Cucumber – a really neat little tool by Aslak Hellesoy. There are others too who have contributed to this movement like Dave Astels, et. all

You can still use RSpec example runner in tango with Cucumber or you could stick with xUnit under the hood instead of RSpec.
Enough prologue for today.

Now I've spent 3-4 days chasing hyperlinks and watching screencasts (links at the end of Post#2 in this series), this post should give you a rolling start on cucumber.

Step1# Installation

You need a few Ruby Gems to get rolling. (RSpec & Win32Console not mandatory)

gem install rspec
gem install cucumber
gem install win32console

I have rspec (1.2.8) | cucumber (0.3.99) | win32console (1.2.0). Try ‘cucumber –help’ to verify this step. Win32console is for color output on Windows.

Step2# Lets go bowling

I’ll take the popular ‘Score a bowling game’ TDD Kata as an example.
So we begin with a feature. (I’ll skip the pop-the-why-stack ; this feature falls into the ‘protect revenue’ category). Find a nice empty directory let’s say c:\cukes. Create a features subdirectory within it.
We create a new file : features\bowling_score.feature and type this in. This is called the feature narrative – it’s just a descriptive block of text. However the form is slightly different the context/value clause rises to the top with the prefix In order to <business value>. Spotlight on business value!
Note: Indentation matters! Spaces preferred.

Feature: Bowling Game Score Calculation
  In order to let the player know his score
  As a CEO of TheBowlingInc
  I want to calculate the score at the end of a bowling fame

Next we Save And Run Cucumber from the c:\cukes directory. This step I hereby alias to SARC. This outputs

c:\cukes>cucumber
Feature: Bowling Game Score Calculation
  In order to let the player know his score
  As a CEO of TheBowlingInc
  I want to calculate the score at the end of a bowling fame

0 scenarios
0 steps
0m0.000s

So now that prompts us : we need Scenarios. A Feature may contain multiple scenarios (which collectively validate the feature).
A Scenario is executable. A Scenario takes the form

Scenario: <text description>
  Given <context>
  When <action>
  Then <user-visible outcome>

So we expand our feature like this.

Feature: Bowling Game Score Calculation
  In order to let the player know his score
  As a CEO of TheBowlingInc
  I want to calculate the score at the end of a bowling fame

  Scenario: All Gutters score 0
     Given I am starting a new game
     When I roll 20 gutter balls
     Then the score should be 0
     And the game should be over

In addition you can write And <step> to have multiple steps – the above example is equivalent to 2 Then clauses. You can use it under Given/When too. Save and run cucumber again. (or SARC from here on)

For some reason the snippets don’t show up. So we play a little trick. Create a subfolder under features called step_definitions. Create a new blank file called bowling_game_steps.rb in it. Run cucumber again. Now you should see some snippets. Copy them from the console and paste into the blank file. SARC. You should now see that the first step is shown as a TODO and the rest are skipped (and in a different step-color to boot)

The first snippet looks like

Given /^I am starting a new game$/ do
pending
end

We need to define a “step” – specify the action to be taken i.e. when cucumber encounters a matching Given clause for the regex, what action should it take? You define that as the content of the block. Let’s say we want to create a new instance of a game. So replace `pending` with
`@game = BowlingGame.new`
SARC and now we have a familiar color – Red (of the Red-Green-Refactor fame). We’re notified that our step has failed.

We have no class called Bowling Game yet. Create a new subfolder under features called support to house our app classes. Create a new file in there ; features\support\bowling_game.cs

class BowlingGame; end

SARC and we’re green…. partially.. better than red. Cucumber now points us to the next step.

I’ll define all the steps like this.. while you take a breather. There. The updated version

Given /^I am starting a new game$/ do
  @game = BowlingGame.new
end

When /^I roll (\d+) gutter balls$/ do |count|
  count.to_i.times{
    @game.roll(0)
  }
end

Then /^the score should be (\d+)$/ do |expected_score|
  @game.score.should == expected_score.to_i
end

Then /^the game should be over$/ do
  @game.should be_over
end

When /^my rolls are (.*)$/ do |rolls|
  rolls.split(' ').each{|roll|
    @game.roll(roll.to_i)
  }
end

NOTE:

The second step is an example of a parameterized step. If the regex in the step definition contains groups, the matched contents are passed in as parameters to the block for the step. Parameter passed in are strings so you need to convert it to the right type before use. Standard regexp rules apply for matching groups. So now the step can match both ‘When I roll 5 gutter balls’ and When I roll 20 gutter balls’. You also see parameters highlighted distinctly in cucumber output. Cool!
Then steps use a new should method which is added to all objects. You can read more about such helper methods here and here
Cucumber tells after each step what needs to be done next. This rhythm is similar to the TDD approach.

Here’s the updated bowling_game.rb with ‘the simplest thing that could possibly work’

class BowlingGame
 def roll( pins_knocked_down)
 end
 def score
   0
 end
 def over?
   true
 end
end

SARC this time we specify the no source
option as a CL argument

 >cucumber –s

And we’ve reached the promised land - plain text file story/features that can be verified automatically at the push of a button. That's pretty cool.

Just to reiterate the folder structure.

In the next post, we move up the learning curve with Scenario tables and tagging.

How to talk to a C# / ASP.Net web service via Ruby / Rails

2009-07-28T21:11:00.001+05:30

Maybe not the right answer.. but I took the path of least resistance and succeeded. Yay Ruby!

First I did a dummy Asp.net Xml Web service as shown in this MSDN walkthrough which does F to C temp conversion. This meant that I can access my legacy/existing C# code without any interop hassles. Yes it is SOAP based (and old-school) but it will have to do for now. You can test out the web service via a dummy stub page that exercises the web service. You can also infer the needed urls from this page.

Next I found this uber-blog post by Ryan Heath and it was all downhill after that.

You write a wrapper Ruby class like this which gets the WSDL and sets itself up. You can then just call your Web methods on it!

   require 'soap/wsdlDriver'
    
    class TempConverterWrapper
      attr_accessor :endpoint, :service_name
      def initialize(endpoint, service)
        @endpoint = endpoint
        @service  = service
      end
    
      def convert_fahrenheit_to_celsius(temp)
        soap = wsdl.create_rpc_driver
        response = soap.convert_fahrenheit_to_celsius(:temp_in_fahrenheit => temp)
        soap.reset_stream
        response.convert_fahrenheit_to_celsiusResult
      end
    
    
    private
        def wsdl
          puts "http://#{@endpoint}/#{@service}.asmx?WSDL"
          SOAP::WSDLDriverFactory.new("http://#{@endpoint}/#{@service}.asmx?WSDL")
        end
    end

Your main / irb code to exercise this web service. More bang / LOC!

puts "Enter a fahrenheit value to convert to celsius... via a Web Service"
temp_fahrenheit = gets.to_f

wrapper = TempConverterWrapper.new('localhost:1464', 'TemperatureConverter')
puts "The corresponding celsius value is #{wrapper.convert_fahrenheit_to_celsius(temp_fahrenheit)}"

How to show a Tree (TreeView) in a Rails view?

2009-07-24T10:05:00.000+05:30

Well there seems to be no one-way for displaying a tree in Rails. But it has to be some javascript/AJAX doing its thing with CSS on HTML lists. So I took a look at YUI Tree (the first thing that google threw up) but it felt bloated and the learning curve was a bit too steep. Some time later.. I found a tree plugin for JQuery and also found that there is a RailsCast to use JQuery with Rails (so I knew its possible to interop).
I've been wanting to look at what JQuery is - with the massive good-will it is building up. Did a couple of tutorials 1 and 2 on their site to get started. Downloaded the jquery.js file (1 file for all the goodness !! awesome).
Next it's time to download the zip for the excellent JQuery tree plugin by Jorn Zaefferer - incidentally its the same person who wrote the second tutorial. I felt better about my chances for success.

I'm assuming you have a Rails app and the code for your hieararchy all done to this point. It's going to be a simple tree showing the hierarchy (no fancy AJAX callbacks or delayed loading of children... although I think its supported)

Unzip the treeview plugin folder v1.4 (it has a version of jquery bundled too inside the lib subfolder). I see

jquery.treeview.XXX.js (the javascript files) : Moved them to my public\javascripts folder within my Rails project folder. Also copied the lib subfolder here.
jquery.treeview.css (the CSS stylesheet) : Moved them to my public\stylesheets folder
images subfolder : Moved the images in this folder to a new subfolder public\images\jquery.treeview.images

Now the CSS file has references to the images. So do a find-replace to make sure all the images are reachable in their new locations; "url(images/" becomes "url(../images/jquery.treeview.images/"

I have a show action on my controller that creates @root_node which is a typical Node object with a Children attribute - an array of child nodes. So now we need something to render it as an hierarchical HTML list - a Rails helper method for your controller

  def display_segment( node )
    
    html = "<li>"
    node_class = node.children.length == 0 ? "file" : "folder"
    html << "<span class=\"#{node_class}\">#{h(node.to_s)} </span>"
    html << "<ul id=\"children_of_#{h(node.sid)}\">"
    node.children.each{|child_node|
      
      html << display_segment( child_node )
    }
    html << "</ul></li>"
  end

Let's also take a look at the view show.html.erb


<html>
<head>
<h3>  You are viewing <%= @data_file_path %>  </h3>

<title><%= controller.action_name %></title>
<%= stylesheet_link_tag 'appstylesheet'%>
</head>

<body>
<ul id="my_tree" class="filetree">
<%= display_segment(@root_node) %>
</ul>

</body>
</html>

Things to note

First we need an enclosing ul block with css class set to filetree. This would be picked up our custom javascript snippet to JQuery-magically turn into a tree. So we need to make a note to give it a good id - here my_tree
Next the content is rendered via our recursive helper function. Children of a node are in a nested ul block. Each leaf-node is a li item and has css class as "file". parents have css class as "folder"

So now it should look something like this.

Now the stage is set for JQuery to do its thing. First lets introduce these 2 lines to the head tag of our view.


<%= javascript_include_tag 'lib/jquery', 'jquery.treeview', 'custom' %>
<%= stylesheet_link_tag 'jquery.treeview.css' %>

So we say we want to use the jquery javascript file, followed by the treeview plugin and finally our own custom javascript file that we'll write next.
We also want to use the treeview CSS stylesheet. You can do a "View Source" in firefox to see what the translated HTML looks like - std. stuff.

We're nearing the end here - Our \public\javascripts\custom.js file to sprinkle some JQuery dust on this thing


$(document).ready( function() {
$("#my_tree").treeview();
} );

Here we're giving a function to be executed when the page/document is ready. This function finds our top-level list element with id = "my_tree" and calls the treeview() method on it.

Victory Jig time!

Illustrated How to do a simple C++ DLL project in Visual Studio ?

2009-07-10T12:00:00.000+05:30

Agreed that it isn't a big deal ... however I struggle with this everytime I dip my toes after a long managed stint.. and after a question on stackoverflow - it seems I'm not alone.

So here goes.. Fire up Visual Studio

Ctrl+Shift+N to go to the new project dialog or Right click on the Solution Node - Add - New Project... Press Next instead of Finish to go to the next screen.

So when I do that I have the following structure.

Stdafx.h & stdafx.cpp – defines and includes standard header files. Leave unmodified.
WinHookFacade_VS2005.cpp – defines the DLL Main function (entry point). Leave unmodified unless you have your very own dllmain.cpp with custom code.
Readme.txt – just a text file, you can stuff some info in.

Try to build the project. It should build fine.. do it just to be sure.
Next step include all your .h and .cpp into the project. Right click on the project in solution explorer - Add - Existing item..

You may have build failures.. now its time to enter the maze of C++ Project settings

C++ Settings

If you want additional directories to be looked for resolving header includes. Right click on Project Node - Properties
IMPORTANT: Check the Configuration combobox to read “All Configurations” each time you change a project setting else you risk the debug and release configurations getting out of sync.

Also for some reason, precompiled headers are frowned upon at my workplace. So I turn them off too..

Define any preprocessor symbols you want to.
DETOUR example.. WINHOOKFACADE_EXPORTS is a symbol defined here - which will cause my functions to be exported from here. Referencing projects will not have this symbol defined and will import my functions. The header files begin like this


#ifdef WINHOOKFACADE_EXPORTS
#define WINHOOKFACADE_API __declspec(dllexport)
#else
#define WINHOOKFACADE_API __declspec(dllimport)
#endif

void WINHOOKFACADE_API Hook();

Also in this case, you may want different symbols defined in both configurations. E.g. DEBUG is defined only in Debug configuration and so on.. so change the Configuration Combobox accordingly before making changes.

Linker Settings

To change the default location of the built binary file: E.g. I want the output to be generated in a SolutionDir\bin\Debug or Release. So you can use the helpful macros by pressing the tiny button on the right of the input field

$(SolutionDir)bin\$(ConfigurationName)\$(ProjectName).dll

I also keep the lib files that I need to link to in SolutionDir\bin\lib subfolder. So specify the additional folders to scan for .lib files

To specify the lib files you need to link to...

To specify where the lib file for the current project should be generated. In my case I want it to also go in under SolutionDir\bin\lib and named WinHookFacade_Debug.lib or WinHookFacade_Release.lib
Use the GUI macro helper thingie to come up with your own incantation. In my case it is

$(SolutionDir)bin\lib\$(ProjectName)_$(ConfigurationName).lib

Pre-Post Build steps:
If you’d like to do some tasks (like copying over something) before or after a build, check out the Build events node under Configuration Properties. I’d recommend against using this.. having this in the build script (e.g. ant or nant) makes it more visible – it tends to get hidden and out of sync in the project properties.

Phew! That's it. Take a nap now to recharge those brain cells.

How to compile ruby from source on Windows?

2009-06-17T12:29:00.001+05:30

I wanted to play with the latest version of ruby. I got the win32 binary zip of 1.9.1 p0 - however irb used to popup an error message on startup - used it for sometime but then I decided to fix it. Checked back on ruby's download page and found an updated version p129. Downloaded it but it was a source zip.

I wasn't sure that compiling it from source would be trivial... however it reinforced the simplicity of the ruby way. Summarizing the steps I found on this post at ruby-forum.com

Get the source zip from the download page and unzip it locally to say c:\ruby-src
Open the Visual Studio Command prompt (I have VS2008 on WinXP SP2)
Make a folder for your build say c:\ruby-build. Move into it / change dir
c:\ruby-build> c:\ruby-src\ruby-1.9.1-p129\win32\configure.bat
c:\ruby-build> nmake
c:\ruby-build> nmake test
c:\ruby-build> nmake DESTDIR=c:/ruby-build install

That's it.. you'll find the binaries within a usr folder within the ruby-build folder.

How to create a global (system) windows hook ?

2009-06-16T11:28:00.000+05:30

What are Windows Hooks? You probably don't want to know that :) Good starting points are
The MSDN door to hooks
Win32 Hooks

A global / system hook would be “I want to be called when anyone does X in any process” ; the other kind is when “I want to be called when anyone does X in ThreadID#Y”. Creating a global windows hook cannot be done in managed code except (some keyboard/mouse hooks) via P/Invoke (so save yourself some time although local hooks are possible).
Windows would callback on the Hook function ; also called the filter function. The signature -


LRESULT CALLBACK FilterFunctionForHook(int code, WPARAM wParam, LPARAM lParam)

For global hooks, the filter function should be contained in an unmanaged DLL – I created one called WinHookFacade.dll. This unmanaged DLL would be injected into each process. Also the copy of the DLL loaded into other process cannot be explicitly unloaded – will unload when the process goes down. Also any bad code in the hook function can result in bad scary things and may require the user to go through a logoff-login or reboot cycle. Avoid if possible because global hooks will extract their pound of performance. But when has that stopped us programmers ? ;) Hence exercise caution with global hooks.

To create a global hook, you therefore need to create an exe project that hooks/unhooks the filter function and a dll project that contains the filter function.

The Hook


void Hook()
{
HINSTANCE hCurrentDll = GetModuleHandle(_T("WinHookFacade.dll"));
g_HookHandle = SetWindowsHookEx(WH_CBT,
FilterFunctionForHook,
hCurrentDll,
0);

if (g_HookHandle == NULL)
throw new std::exception("Unable to hook");
}

The key here would be the SetWindowsHookEx Win32 API function. The first parameter is what type of hook you’d like to register for – See this page for available types. In this example, WH_CBT means events related to windows (creation,activation,destructions etc). The second parameter is the name of the Hook/Filter function that shall be called back by the OS – detailed below. The third parameter is a handle to the DLL containing the hook function for global hooks (if null, it indicates a thread-specific hook). The fourth parameter is 0 for global hooks (for thread/local hooks, it is the Thread ID you want to hook into).

The Callback


LRESULT CALLBACK FilterFunctionForHook(int code, WPARAM wParam, LPARAM lParam)
{
if (code >= 0)
{
   //your code e.g.
   switch(code)
   {
      case HCBT_ACTIVATE:
     // log the window title                  
     ...
}
return CallNextHookEx(g_HookHandle, code, wParam, lParam);
}

This function would be called in the process in which the event occurs (e.g. where a window is created). Process Isolation : a process cannot run custom code into another process; hence DLL injection. Also the CallNextHookEx Win32 API function must be called to pass control onto the next filter function in queue passing it the original arguments received to be a good citizen. The first parameter is the Hook Handle obtained when you registered the Hook function.

IMPORTANT: However this value must be the same across all instances of the shared Hook DLL & must be available within the filter function. For this reason, you need to create the “hook handle” in a shared data segment Whaa.. how do I do that? Declare your HHOOK variable like this


#pragma data_seg (".MY_HOOK_DATA")
HHOOK g_HookHandle = 0;
#pragma data_seg()

This function is supposed to call the next function in queue without doing anything if the code argument is less than zero. If not, the function may execute custom code to do specific tasks. The wParam and lParam arguments are pointers to handles/structures containing additional info. The exact type of structure depends on the value of the code parameter – check msdn for the page for the specific alias of the filterfunction e.g. in my case it would be CBTPROC; wParam is a handle to the window and lParam is a pointer to a struct

Cleanup


void Unhook()
{
 if (!UnhookWindowsHookEx(g_HookHandle))
     throw new std::exception("Unhook failed!");

 g_HookHandle = NULL;
}

Here the primary player is the UnhookWindowsHookEx Win32 API Function. In case of an error, you’d need to use the fugly GetLastError() function to know what really went wrong.

Test Drive

Now to take this for a spin, create an exe project referencing the DLL containing the hook function. The main function will call on the exported functions as shown below..


int _tmain(int argc, _TCHAR* argv[])
{
  char buffer[256];
  try
  {
      printf("Hooking...");
      Hook();
      printf("Done.\n");
    
    
      printf("Press Enter to exit\n");      // hook is running
      gets(buffer);

      printf("Unhooking...");
      Unhook();
      printf("Done. Woohoo!!\n");
  …

That’s all there is to it.
CAUTION: Also if you get something wrong, the effects would be felt immediately – in my case, the UI would freeze as soon as my botched hook function was registered; my taskbar would be nuked and random error dialogs from running apps. But nothing a reboot can’t solve! C++ string compiler errors, WinAPI gotchas & hooks -- It was like walking on a tightrope in dark windy conditions. <runs back to the managed side>

Ruby is slow... until you learn to write good code.

2009-04-22T22:52:00.000+05:30

I wrote up a small program to find prime-numbers between x and y ; implemented the Sieve of Eratosthenes.
The SOE is basically where you take all numbers (say 1-100) into an array. Cross out 1. Loop from 2-Sqrt(N) i.e. 2-10. Now cross out every multiple of 2 in the array. Next cross out every multiple of 3.. and so on. Finally all the numbers that aren't crossed out are prime.

However I did a very simplistic implementation (test-driven) to boot. All the tests did pass.. however running it to get all primes between 10K and 100K took 34 secs to execute. And I was on the new n improved ruby 1.9 with a better VM. I switched to the stable ruby 1.8.6 and it took ~60 secs. So now I turned back to my primary Weapon C#. The same steps implemented in C# took 722 msecs. So now there were 2 possibilities
1. Ruby is sloooow (the usual gripe)
2. Something in my algorithm is messed up.

So I posted my source onto StackOverflow - (my current fave time sink :) to get some eyeballs to look at the problem.
So here's my naive implementation

class PrimeGenerator
  def self.get_primes_between( x, y)
    sieve_array = Array.new(y) {|index|
      (index == 0 ? 0 : index+1)
    }

    position_when_we_can_stop_checking = Math.sqrt(y).to_i
    (2..position_when_we_can_stop_checking).each{|factor|
      sieve_array[(factor).. (y-1)].each{|number|
        sieve_array[number-1] = 0 if isMultipleOf(number, factor)
      }
    }
    
    sieve_array.select{|element| 
      ( (element != 0) && ( (x..y).include? element) )
    }
  end
  def self.isMultipleOf(x, y)
    return (x % y) == 0
  end
end

# Benchmarking code


require 'benchmark'
Benchmark.bm(30) do |r|
  r.report("Gishu") { a = PrimeGenerator.get_primes_between( 1_000, 10_000) }
end

A few people got back with what might be the problem.
One of the first and most voted issue was the slicing the array to get a new subset array. That's got to be expensive inside a loop for an array of this size. Obviously I was using for loops in C# (since C# doesn't have Ruby's each)

for index in factor..y-1 do
  sieve_array[index] = 0 if isMultipleOf(index+1, factor)
end

That shaved off 8 secs - 24%. But still 26 secs. Another optimization was to reduce iterations - instead of checking each number with IsMultiple(number, _loopVar), set the 'step' for the loop to equal _loopVar (so from 2 you go 4,6,8,10...)

And finally Mike Woodhouse dispatched a homer. He posted a tiny snippet that did the same thing in under 1 sec. No that's not a typo.

def sieve_to(n)
  s = (0..n).to_a
  s[0]=s[1]=nil
  s.each do |p|
    next unless p
    break if p * p > n
    (p*p).step(n, p) { |m| s[m] = nil }
  end
  s.compact
end

# Usage:
# puts sieve_to(11).select{|x| x>=3}.inspect


        user     system      total        real
Gishu  27.219000   0.000000  27.219000 ( 27.375000)
Mike    0.141000   0.000000   0.141000 (  0.140625)

I then had to know how he did it..

Smart#1: He skips if array[_loopvar] is already 0. whereas if array[6] is crossed out due to 3, I did a scanning run to set all multiples of 6 to 0 which were already 0.(now down to 6 secs)

Smart#2: Use the Numeric#step trick to jump to the next multiple vs traversing entire list with an IsMultiple check (now down to 0.25 secs)

Smart#3: He begins the inner loop with Square of p. e.g if _loopvar is 7, I used to check from 8->end. Mike checks from 49->end. This one was a little tricky to figure out.. the reason is 7x2, 7x3, 7x4, 7x5, 7x6 have already been crossed out when _loopVar was 2,3,4,5,6 respectively. (now down to 0.21 secs)

Smart#4: Smart use of Ruby's nil for crossed out vs my choice of 0. That followed by Array#compact reduces the number of elements for the final select operation to get primes between x and y (now down to 0.18 secs.)

So after all that it's Ruby 0.15 secs and C# (with same optimizations) 0.028 secs, much more easy to take in.

An interesting thought to end this post is that even with the naive first implementation C# performed reasonably well... under a sec. I'd have moved on to the next task. Ruby on the other hand slowed to a crawl forcing me to take another look for optimizations. In a indirect sort of way, C#'s superior performance could be hiding bad/substandard solutions and letting programmers get away with it...

MergeSort in Ruby

2009-04-08T10:22:00.000+05:30

Finally got the ball rolling on the 'Introduction to Algorithms' book.. What I figured was that I implement the sorting algorithms in ruby - help me get better at ruby AND algorithms.
I've got to confess.. I'm test-infected so I'm gonna TDD my way out. This series of posts are not going to be in a walkthrough style of narration.. I'm just going to post complete samples. The order of methods indicate how the code was built up.. (the first method is the 'oldest'.

Here's merge_sort_test.rb [The Test Fixture]
--

require 'test/unit'
require 'merge_sort'
class TestMergeSort < Test::Unit::TestCase
  def test_partition_array_with_even_number_of_elements
    left, right = MergeSort.partition_array [11, 5, 9, 15]
    assert_equal [11, 5], left
    assert_equal [9, 15], right
  end
  def test_partition_array_with_odd_number_of_elements
    left, right = MergeSort.partition_array [11, 5, 45, 9, 15]
    assert_equal [11, 5], left
    assert_equal [45, 9, 15], right
  end
  
  def test_merge_arrays
    assert_equal( [10,20,30,40], 
      MergeSort.merge_arrays( [10,30], [20,40] ) )
    
    assert_equal( [1, 3, 5, 9], 
      MergeSort.merge_arrays( [1, 5, 9], [3] ) )
    assert_equal( [1, 3, 5, 9], 
      MergeSort.merge_arrays( [3], [1,5,9] ) )
  end
    
  def test_sort
    assert_equal [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],  
      MergeSort.sort([ 8, 3, 4, 0, 9, 1, 2, 7, 6, 5])
  end
end

--
merge_sort.rb [The Implementation]
--

class MergeSort 
  def self.partition_array( input_array)
    mid_index = (input_array.length / 2) - 1
    [input_array[0..mid_index], input_array[(mid_index+1)..(input_array.length-1)]]
  end
  
  def self.merge_arrays(array1, array2)
    merged_array = []

    until (array1.empty? || array2.empty?) do
      if (array1[0] < array2[0])
        merged_array << array1.shift
      else
        merged_array << array2.shift
      end
    end
    
    merged_array + array1 + array2
  end
  
  def self.sort( input_array )
    return input_array if (input_array.length < 2)
    
    left, right = self.partition_array input_array
    sortedLeft = self.sort left
    sortedRight = self.sort right
    y = self.merge_arrays( sortedLeft, sortedRight )
  end
end

--
A command line 'runner'/'main' function
--

# Console runner / main
if !ARGV[0].nil?
  input = []
  ARGV.each{|x| input << x.to_i}
  print MergeSort.sort(input).inspect
end

--
>ruby merge_sort.rb 90, 100, 40, 10, 45
[10, 40, 45, 90, 100]

And we're done! IMHO The Ruby Array class really shines here w.r.t. readability.

Dynamic evaluation of an expression

2009-02-03T16:38:00.000+05:30

In this post, I'll tackle Step#2 of evaluating an expression like 2 * (3 + 5) and return 16.

Convert expression from infix to postfix (see the last 2 posts)
Evaluate postfix expression to determine result

As always, write a new test for ExpressionConverter


[Test]
public void TestEvaluate()
{
   ExpressionConverter converter = new ExpressionConverter();
   Assert.AreEqual(10, converter.EvaluateInfixExpression("((1+2) + 3) * 8/4"));
}

The implementation is very simple now that we have the ability to 'convert to postfix'.


public object EvaluateInfixExpression(string sInfixExpression)
{
   List<object> postfixExpression = this.ConvertInfixToPostfix(sInfixExpression);
   Stack<object> stack = new Stack<object();
   foreach (object term in postfixExpression)
   {
       if (term is int)
       {
           stack.Push(term);
       }
       else
       {
          Operator operatorTerm = term as Operator;
          int iValue2 = (int) stack.Pop();
          int iValue1 = (int) stack.Pop();
          stack.Push( operatorTerm.Evaluate(iValue1, iValue2) );
       }
   }
   return stack.Peek();
}

Except that we don't have Operator#Evaluate()... Lets get over that hurdle. Mark this test with a temp Ignore tag. Write a new test in the TestOperator fixture


[Test]
public void TestPlusOperator_Evaluate()
{
   Assert.AreEqual(30, Operator.Plus.Evaluate(10, 20));
}

If each operator could have a associated evaluate code block that evaluated 2 parameters and returned the result, we'd be done. A new member variable, a new parameterized ctor and a new code block later.. Here are the changes


public class Operator
{
   public static readonly Operator Plus = new Operator("+", 1, (x, y) => x + y );
   private Func<int,int,int> m_evaluateDelegate;

   private Operator(string sSymbol, int iPrecedenceLevel, Func<int,int,int> evaluateDelegate):this(sSymbol, iPrecedenceLevel)
   {
       m_evaluateDelegate = evaluateDelegate;
   }

   public int Evaluate(int iValue1, int iValue2)
   {
      return m_evaluateDelegate(iValue1,iValue2);
   }

Green. Ditto for the other operators -, *, /
All done. Finally a console app to do this


static void Main(string[] args)
{
   ExpressionConverter expr = new ExpressionConverter();

   Console.WriteLine("Postfix version is ");
   foreach (object term in expr.ConvertInfixToPostfix(args[0]))
   {   Console.Write(term + " ");   }
   Console.WriteLine();

   Console.WriteLine("Expression evaluated to " + expr.EvaluateInfixExpression(args[0]) );
}

This is how it works.
Release>DynCalc.exe "((1+2) + 3) * 2 - 8/4"
Postfix version is
1 2 + 3 + 2 * 8 4 / -
Expression evaluated to 10

Now ExpressionConverter is no longer a converter.. we should probably rename it to Expression. Also it has no instance state. All methods can therefore be converted to static methods - no need to create an instance to invoke members. Done.
Other than non-happy paths like malformed expressions.. (can be done) it's all good. End of series. I had fun..

Test driving an infix to postfix expression converter (2 of 3)

2009-02-03T14:08:00.000+05:30

So at the end of Post 1 of this series. we've a converter that doesn't yet understand operator precedence.
Let me refresh my memory before proceeding..
An operator token belongs to a class/family of 'Operators', has a string representation like *, +, /, - and finally has a relative precedence associated with it.
1 for +, -
2 for *, /
3 for (, )

Seems like we need an Operator class instead of just a string. I'll mark the failing test with [Ignore("Detour")] tag while we go make some operators.
Create a new test fixture, imagine a magical Operator class.. something like this

TestOperator.cs


[Test]
public void TestParse_PlusOperator()
{
Operator plusOp = Operator.Parse("+");
Assert.IsNotNull(plusOp);
Assert.AreEqual("+", plusOp.ToString());
Assert.AreEqual(1, plusOp.PrecedenceLevel);
}

I turn my ideas about Operators into a Type/Class.


public class Operator
{
private string m_OperatorSymbol;
private int m_iPrecedenceLevel;
private static readonly Operator plusOperator = new Operator("+", 1);

private Operator(string sSymbol, int iPrecedenceLevel)
{
   m_OperatorSymbol = sSymbol;
   m_iPrecedenceLevel = iPrecedenceLevel;
}

public int PrecedenceLevel
{
   get { return m_iPrecedenceLevel; }
}
public override string ToString()
{   return m_OperatorSymbol;    }


public static Operator Parse(string stringToParse)
{   return Operator.plusOperator;    }
}

Green! I took the quick way out with the Parse implementation.. I'll let the coming tests force me to distill that bit of code. I add similar tests for Minus (and then Multiply and Divide). I end up with a need for a 'switch case' in Parse().
Also Parse should probably throw if its passed a token that it doesnt recognize. I'll note that down as a test too.


[Test]
[ExpectedException(ExceptionType=typeof(ArgumentException), ExpectedMessage="Unknown Operator! Cannot parse Gishu")]
public void TestParse_InvalidInput()
{
Operator noOp = Operator.Parse("Gishu");
Assert.Fail("Operator learnt some AI to parse Gishu. This should have blown up by now!");
}

[Furious typing] Alright now Operator.Parse() looks like this.. all its tests pass.


public static Operator Parse(string stringToParse)
{
switch (stringToParse)
{
   case "+":
       return Operator.plusOperator;
   case "-":
       return Operator.minusOperator;
   case "*":
       return Operator.multiplyOperator;
   case "/":
       return Operator.divisionOperator;

   default:
       throw new ArgumentException("Unknown Operator! Cannot parse " + stringToParse);
}
}

Now lets update our tokenizer to use this new type. This looks like it might break a few tests.. but with baby steps and a vigilant eye, it'll all be better by the time I'm done with it. I'll start by modifying one of my initial tokenizer tests...


[RowTest]
[Row("+")]
[Row(" + ")]
public void ReadPlusOperator(string sInput)
{
Tokenizer t = new Tokenizer(sInput);
  
//Assert.AreEqual("+", t.GetNextToken());
Assert.AreEqual(Operator.Plus, t.GetNextToken());
}

This would require us to make the static plusOperator member public. The other option is to check each member of the operator object… which would be duplication of the testfixture for Operator. So lets share our operators with the world.. make em public and rename.


public class Operator
{
public static readonly Operator Plus = new Operator("+", 1);
//...

And we're red...
TestDynCalc.TestTokenizer.ReadPlusOperator( + ):
Expected: <+>
But was: "+"

Instead of making a big bang change.. let’s make an ugly hack inside Tokenizer#GetNextToken() to make just this test pass..


Regex regexOperator = new Regex(@"^\s*([-+*/()])");
obResults = regexOperator.Match(m_sInputData);
if (obResults.Success)
{
Group obGroup = obResults.Groups[1];
RemoveReadCharacters(obGroup);

//return obGroup.Value;
if (obGroup.Value != "+")
    return obGroup.Value;
else
    return Operator.Plus;

}

I broke 2 tests - that tested the tokenizer with complete expressions containining +. Turns out its easy to fix them.


[Test]
public void ReadMultipleTokens()
{
Tokenizer t = new Tokenizer(" 2 + 3 - 34 * 2 / 1 ");
//object[] arrExpectedTokens = new object[] { 2, "+", 3, "-", 34, "*", 2, "/", 1 };
object[] arrExpectedTokens = new object[] { 2, Operator.Plus, 3, "-", 34, "*", 2, "/", 1 };
foreach (object token in arrExpectedTokens)
{
    Assert.AreEqual(token, t.GetNextToken());
}
}

All green. Excuse me while I repeat the procedure for all the other operators. The hack in Tokenizer#GetNextToken() can now be replaced by ...


return Operator.Parse(obGroup.Value);

However we forgot the parantheses bros. while creating our operators. We need more operators! Mark Tokenizer with Ignore(“2 TDDed operators coming right up…”).. [more furious typing] we have new operators. Take off the ignore tag. Similar changes… and Tokenizer is now Operator-aware.
Time to take off the ignore tag on ExpressionConverter tests and end our little detour.
So we're back to red.. but this time, we've operators with precedence. [after some mental acrobatics with a pencil and an eraser on paper]
The key to getting this is to check if the operator on top of stack is not a '(' and has higher precedence than the incoming/current token. If yes, pop the stack, write popped operator to output and retry. If no, push the incoming token onto the stack.
So + should not be pushed if the top of stack[TOS] contains *; however * should be if TOS is +.
So we add some more code to the final else block in ExpressionConverter#ConvertToPostfix..


if (obStack.Count() > 0)
{
Operator termOnTopOfStack = obStack.Peek() as Operator;
if ((termOnTopOfStack != Operator.OpenParantheses)
    && (termOnTopOfStack.PrecedenceLevel > (term as Operator).PrecedenceLevel))
{
    postfixExpr.Add(obStack.Pop());
    continue;
}
}

obStack.Push(term);

All Green! Woohoo! However the method seems to have grown to an uncomfortable size for me... maybe I can refactor.

Rename method to ExpressionConverter#ConvertInfixToPostfix()

Stack always contains Operators.. So we can do away the casting...Stack stackOfOperators

Replace ‘term.ToString() == ")" ‘ with type-based checks

Each of ( and ) is a parenthesis together they make a set of parentheses. http://en.wikipedia.org/wiki/Parenthesis... well you live, you read wikis, you learn. Correct typos

Extract some methods to clarify intent.
Had to promote the stack and postfixExpr variables to class members. (All the extracted methods required the stack and List variables to be passed in)

Looking good. I'll post the refactored version shortly. Let's try another expression/test.


[Row("0-12000-(16*4)-20", "0 12000 - 16 4 * - 20 -")]

Red again! [A few brain cycles later]
This breaks because the converter doesn’t recognize Left to Right precedence for operators at the same level. (In 0 - 5 + 3, - should be performed first to get the result of -2 even though precedence-wise + and - are equivalent.)
But this can be easily solved by replacing > with >= where we compare operator precedence. Here’s the final version of the class.


public class ExpressionConverter
{
 List<object> m_postfixExpr = new List<object>();
 Stack<Operator> m_stackOfOperators = new Stack<Operator>();
 public List<Object> ConvertInfixToPostfix(string sInfixExpr)
 {
     m_postfixExpr.Clear(); m_stackOfOperators.Clear();

     Tokenizer parser = new Tokenizer(sInfixExpr);
     object term = parser.GetNextToken();
     while (term != null)
     {
         if (term is int)
         {
             m_postfixExpr.Add(term);
         }
         else if (term == Operator.CloseParenthesis)
         {
             MoveOperatorsOnStackToPostfixExprTillMatchingOpenParenthesisIsFound();
         }
         else
         {
             Operator operatorTerm = term as Operator;
             if (OperatorOnTopOfStackHasHigherOrSamePrecedenceAs(operatorTerm))
             {
                 m_postfixExpr.Add(m_stackOfOperators.Pop());
                 continue;
             }

             m_stackOfOperators.Push(operatorTerm);
         }

         term = parser.GetNextToken();
     }

     PopAllRemainingOperatorsOnStackToPostfixExpr();
     return m_postfixExpr;
 }

 private bool OperatorOnTopOfStackHasHigherOrSamePrecedenceAs(Operator term)
 {
     if (m_stackOfOperators.Count() == 0)
         return false;

     Operator termOnTopOfStack = m_stackOfOperators.Peek();
     return ((termOnTopOfStack != Operator.OpenParenthesis)
             && (termOnTopOfStack.PrecedenceLevel >= term.PrecedenceLevel));
 }

 private void MoveOperatorsOnStackToPostfixExprTillMatchingOpenParenthesisIsFound()
 {
     Operator term = m_stackOfOperators.Pop();
     while (term != Operator.OpenParenthesis)
     {
         m_postfixExpr.Add(term);
         term = m_stackOfOperators.Pop();
     }
 }

 private void PopAllRemainingOperatorsOnStackToPostfixExpr()
 {
     while (m_stackOfOperators.Count > 0)
         m_postfixExpr.Add(m_stackOfOperators.Pop());
 }
}

Code is good. End of post. In the next post, I'll write a console 'main' app to exercise the converter and evaluate the expression.