Testing .Net code with Cucumber and IronRuby

As promised, this is the culmination of this trilogy. The previous two posts were a jump-start on Cucumber to test Ruby code. Next I moved towards using Cucumber's plain text stories to test .Net code.Aslak Hellesoy has a wiki-post on how to do this ; however I found that you get eaten and spat out by a lot of IronRuby dragons along the way. So hold on..

1. Install the new DLR

First up install .Net Framework 4.0 (I have Beta1) - this has the new Dynamic Language Runtime (DLR) that makes things like IronRuby and IronPython possible.

2. Get the latest IronRuby release

Next we need to get IronRuby as a zip. Extract it to say d:\ironruby-0.9.0 Add the path to the bin folder to your PATH environment variable to avoiding lengthy paths. Drop to a command shell, type 'ir' to invoke the interactive IronRuby console. Type something simple to test if it works. It should.

>>>puts "Hello IronRuby"

3. Dragons ahead. Use diversion

But I hit a snag with this version of IronRuby, which had A BUG.

• So I had to get the latest (8 Sep 2009 master to be precise) version of the source from the github repository to overcome that. You can download it as a zip or use git as you prefer. I got the zip
  ironruby-ironruby-90cdda82fd60f4b7e6d7d940501c586d55954466.zip (Could have used a shorter name)


• Extract it to say d:\ir-src
  I hit a few path-too-long errors during extraction. Just keep skipping them. I think it's because of the rather large alphanumeric string which is the name of the top-level folder.


• Once extracted, navigate to D:\ir-src\ironruby-ironruby-90cdda82fd60f4b7e6d7d940501c586d55954466\Merlin\Main\Languages\Ruby
  and open Ruby.sln in VS2008 (I have Dev Edition of VSTS.. Although it should build in VS Express Editions as per the docs - I couldn't get the solution to open in it.)
  Build Solution. At the end of it you should find the built binaries in
  D:\ir-src\ironruby-ironruby-90cdda82fd60f4b7e6d7d940501c586d55954466\Merlin\Main\bin\Debug


• Rename that directory with the huge name to something like "ir".


• Copy all built binaries and overwrite the ones from the 0.9.0 release's bin folder i.e. D:\ironruby-0.9.0\bin.



• Open ir.exe.config in the same folder and update the Library paths element to the proper paths to folders within your ir-src folder. 'Handle with extreme care' or you'll lose hours chasing error messages. It should read
  Path#1 - D:\ir-src\ir\Merlin\Main\Languages\ (NOT D:\ir-src\ir\Merlin\External.LCA_RESTRICTED\Languages)
  Path#2 & #3 - D:\ir-src\ir\Merlin\External.LCA_RESTRICTED
  
  My xml looks like
  

  <set language="Ruby" option="LibraryPaths" 
  value="D:\ir-src\ir\Merlin\Main\Languages\Ruby\libs\;D:\ir-src\ir\Merlin\External.LCA_RESTRICTED
  \Languages\Ruby\redist-libs\ruby\site_ruby\1.8\;D:\ir-src\ir\Merlin\External.LCA_RESTRICTED\Lang
  uages\Ruby\redist-libs\ruby\1.8\" />

4. Wrapper script ICucumber

Lastly we need a wrapper script to invoke cucumber with Ironruby. Create a file called icucumber.bat under your Ruby bin folder i.e. D:\Ruby\bin\icucumber.bat with the following text.

REM Update with appropriate values for GEM_PATH, Ruby bin and the path to ir.exe
@ECHO OFF
REM This is to tell IronRuby where to find gems.
SET GEM_PATH=d:\ruby\lib\ruby\gems\1.8
"D:\ironruby-0.9.0\bin\ir.exe" -D -X:ExceptionDetail "d:\ruby\bin\cucumber" %* 

5. Back to our example from the previous two posts.
The .feature file stays unchanged.
Delete c:\cukes\dot_net_features\support\BowlingGame.rb ; since we're going to implement the same in C# this time around as
c:\cukes\dot_net_features\support\BowlingGame.cs

namespace CukesDemo
{
   public class BowlingGame
   {
      public void roll(int pins_knocked_down)
      {
        Score += pins_knocked_down;
      }

      public int Score
      {
         get; set;
      }
      public bool Over
      {
         get { return false; }
      }
   }
}

Also create a small batch file to compile it to a DLL (Assumes csharp compiler is on the PATH).
c:\cukes\dot_net_features\support\Compile.bat

IF EXIST bin GOTO COMPILE
MKDIR bin
:COMPILE
csc /t:library /out:bin/BowlingGame.dll bowling_game.cs

Finally back to our step definitions to check the glue. Four changes needed - explained in comments.
cukes\dot_net_features\step_definitions\bowling_game_steps.rb

# CHANGE 1 : Add bin folder to load-path
$:.unshift(File.dirname(__FILE__) + '/../support/bin')   
# CHANGE 2 : Get BowlingGame.dll
require 'BowlingGame'      

Given /^I am starting a new game$/ do
   # CHANGE 3 : Use Namespace::ClassName.new  
   @game = CukesDemo::BowlingGame.new   
end

When /^I roll (\d+) gutter balls$/ do |count|
  count.to_i.times{
    @game.roll(0)
  }
end

Then /^the score should be (\d+)$/ do |expected_score|
  @game.score.should == expected_score.to_i
end

Then /^the game should be over$/ do
  # CHANGE 4 :  be_over passes even if Over returns false. Don't know what is the equiv of over? 

in .Net
  #~ @game.should be_over == true 
  @game.over.should == true
end

When /^my rolls are (.*)$/ do |rolls|
  rolls.split(' ').each{|roll|
    @game.roll(roll.to_i)
  }
end

Now for the grand finale, run Cucumber to verify our .Net DLL via IronRuby !!!


HTH

Scenario Outlines and Tagging in Cucumber

Post#2 in this trilogy.

Scenario Outline

Scenario tables are similar to Fit's ColumnFixture and NUnit's RowTest. You run the same scenario with different inputs each time.
Let's go back to our plain text feature and add the following.

Scenario Outline: score should be as per the std rules
  Given I am starting a new game
  When my rolls are <rolls>
  Then the score should be <score>

 Scenarios: lets go bowling
 | rolls   | score |
 |5 2      | 7     |
 |5 5 5    | 15    |

The key things to remember here are the 'Scenario Outline:' marker to indicate that it is an outline. We then have placeholders within angular brackets. The Outline is then followed by one or more tables identified by the marker 'Scenarios:'
The next line should contain column headers which correspond with the placeholders in the outline. Cucumber would substitute the values to run each row against the Outline.

SARC and it should fail. Let's go fix that up in bowling_game.rb

class BowlingGame
 attr_reader :score
 def initialize
   @score = 0
 end
 def roll( pins_knocked_down)
   @score += pins_knocked_down
 end
 def score
   @score
 end
 def over?
   true
 end
end

SARC. Now you should see this nice listing.


Tagging

You can tag a scenario (or a feature) with one (or more) tags.

@important, @fast
Feature: Bowling Game Score Calculation
 In order to let the player know his score

By default, cucumber runs all .feature files in the features subfolder. You can group features in different subfolders as well e.g. I can define a new.feature within a new_features subfolder, which can then be run with

>cucumber new_features

Overtime, it can get crowded with lots of subfolders. But tagging is here to save the day..
To run only features/scenarios marked 'fast'

>cucumber --tags @fast

To run features/scenarios that are not marked 'fast'

>cucumber --tags ~@fast

So that's another way to quickly sort out your tests.

Language Support

The next thing you'd probably want to know (just in case you need it) is that cucumber speaks multiple languages. You can write your .feature file in any language - provided that the necessary entries are made in the resource file - languages.yml.

Next post - getting cucumber to test a .Net app with some help from IronRuby. Piqued?


Resources on BDD / Cucumber (although I felt the majority were too entwined with Rails (e.g. WebRat for testing Web Apps via free step definitions) but then that very well could be the major user-base for cucumber right now.)

• ScreenCast by Aslak Hellesoy Executable User Stories with RSpec and BDD


• ScreenCast by Ben Mabey Outside in development with Cucumber


• Blogpost series by Dave Astels Cucumber: More advanced


• RailsCast episodes by Ryan Bates Beginning with Cucumber

Green in 30 mins : Getting Started with Cucumber (a test framework for BDD)

Prologue:
BDD is TDD with an outside-in (top-down) bent.
The way to tackle any new feature request is to ask Why 5 times? An you should arrive at one of Protect Revenue, Increase Revenue or Manage Cost. If not, chances are you’re building something that is not needed.
A feature can be summarized in a few lines as (ala Mike Cohn’s user story format)

As a <role>
I want <feature>
So that <value>

Dan North began this journey with JBehave.
It evolved over time into a Ruby gem - RSpec courtesy David Chelimsky n co. RSpec consisted of two parts - example runner and a story runner. The story runner runs features (think big cog) written up as plain text files which can be ‘executed’ (via some developer-added connecting-glue code in a separate location). The example runner (think small cog) is a bunch of module specs that make the feature happen. (think xUnit for BDD).
The RSpec story runner has now been simplified/improved and become Cucumber – a really neat little tool by Aslak Hellesoy. There are others too who have contributed to this movement like Dave Astels, et. all


You can still use RSpec example runner in tango with Cucumber or you could stick with xUnit under the hood instead of RSpec.
Enough prologue for today.

Now I've spent 3-4 days chasing hyperlinks and watching screencasts (links at the end of Post#2 in this series), this post should give you a rolling start on cucumber.

Step1# Installation

You need a few Ruby Gems to get rolling. (RSpec & Win32Console not mandatory)

gem install rspec
gem install cucumber
gem install win32console

I have rspec (1.2.8) | cucumber (0.3.99) | win32console (1.2.0). Try ‘cucumber –help’ to verify this step. Win32console is for color output on Windows.

Step2# Lets go bowling

I’ll take the popular ‘Score a bowling game’ TDD Kata as an example.
So we begin with a feature. (I’ll skip the pop-the-why-stack ; this feature falls into the ‘protect revenue’ category). Find a nice empty directory let’s say c:\cukes. Create a features subdirectory within it.
We create a new file : features\bowling_score.feature and type this in. This is called the feature narrative – it’s just a descriptive block of text. However the form is slightly different the context/value clause rises to the top with the prefix In order to <business value>. Spotlight on business value!
Note: Indentation matters! Spaces preferred.

Feature: Bowling Game Score Calculation
  In order to let the player know his score
  As a CEO of TheBowlingInc
  I want to calculate the score at the end of a bowling fame

Next we Save And Run Cucumber from the c:\cukes directory. This step I hereby alias to SARC. This outputs

c:\cukes>cucumber
Feature: Bowling Game Score Calculation
  In order to let the player know his score
  As a CEO of TheBowlingInc
  I want to calculate the score at the end of a bowling fame

0 scenarios
0 steps
0m0.000s

So now that prompts us : we need Scenarios. A Feature may contain multiple scenarios (which collectively validate the feature).
A Scenario is executable. A Scenario takes the form

Scenario: <text description>
  Given <context>
  When <action>
  Then <user-visible outcome> 

So we expand our feature like this.

Feature: Bowling Game Score Calculation
  In order to let the player know his score
  As a CEO of TheBowlingInc
  I want to calculate the score at the end of a bowling fame

  Scenario: All Gutters score 0
     Given I am starting a new game
     When I roll 20 gutter balls
     Then the score should be 0
     And the game should be over

In addition you can write And <step> to have multiple steps – the above example is equivalent to 2 Then clauses. You can use it under Given/When too. Save and run cucumber again. (or SARC from here on)



For some reason the snippets don’t show up. So we play a little trick. Create a subfolder under features called step_definitions. Create a new blank file called bowling_game_steps.rb in it. Run cucumber again. Now you should see some snippets. Copy them from the console and paste into the blank file. SARC. You should now see that the first step is shown as a TODO and the rest are skipped (and in a different step-color to boot)

The first snippet looks like

Given /^I am starting a new game$/ do
pending
end

We need to define a “step” – specify the action to be taken i.e. when cucumber encounters a matching Given clause for the regex, what action should it take? You define that as the content of the block. Let’s say we want to create a new instance of a game. So replace `pending` with
`@game = BowlingGame.new`
SARC and now we have a familiar color – Red (of the Red-Green-Refactor fame). We’re notified that our step has failed.




We have no class called Bowling Game yet. Create a new subfolder under features called support to house our app classes. Create a new file in there ; features\support\bowling_game.cs

class BowlingGame; end

SARC and we’re green…. partially.. better than red. Cucumber now points us to the next step.



I’ll define all the steps like this.. while you take a breather. There. The updated version

Given /^I am starting a new game$/ do
  @game = BowlingGame.new
end

When /^I roll (\d+) gutter balls$/ do |count|
  count.to_i.times{
    @game.roll(0)
  }
end

Then /^the score should be (\d+)$/ do |expected_score|
  @game.score.should == expected_score.to_i
end

Then /^the game should be over$/ do
  @game.should be_over
end

When /^my rolls are (.*)$/ do |rolls|
  rolls.split(' ').each{|roll|
    @game.roll(roll.to_i)
  }
end

NOTE:

1. The second step is an example of a parameterized step. If the regex in the step definition contains groups, the matched contents are passed in as parameters to the block for the step. Parameter passed in are strings so you need to convert it to the right type before use. Standard regexp rules apply for matching groups. So now the step can match both ‘When I roll 5 gutter balls’ and When I roll 20 gutter balls’. You also see parameters highlighted distinctly in cucumber output. Cool!
2. Then steps use a new should method which is added to all objects. You can read more about such helper methods here and here
3. Cucumber tells after each step what needs to be done next. This rhythm is similar to the TDD approach.

Here’s the updated bowling_game.rb with ‘the simplest thing that could possibly work’

class BowlingGame
 def roll( pins_knocked_down)
 end
 def score
   0
 end
 def over?
   true
 end
end

SARC this time we specify the no source
option as a CL argument

 >cucumber –s 

And we’ve reached the promised land - plain text file story/features that can be verified automatically at the push of a button. That's pretty cool.

Just to reiterate the folder structure.




In the next post, we move up the learning curve with Scenario tables and tagging.