tag:blogger.com,1999:blog-147447162024-03-08T08:17:40.850+05:30My conversations with gullible machines...Gishuhttp://www.blogger.com/profile/17616896114730114577noreply@blogger.comBlogger130125tag:blogger.com,1999:blog-14744716.post-35761712554839670312013-05-05T21:27:00.001+05:302013-05-07T12:19:26.718+05:30Extend MSTest : TestCaseSource aka runtime inputs for test cases<div dir="ltr" style="text-align: left;" trbidi="on">
<div dir="ltr" style="text-align: left;" trbidi="on">
<div dir="ltr" style="text-align: left;" trbidi="on">
<span style="font-family: "Trebuchet MS",sans-serif;"></span><br />
<h2 style="text-align: left;">
<span style="font-family: "Trebuchet MS",sans-serif;">Extending MSTest</span></h2>
<span style="font-family: "Trebuchet MS",sans-serif;"><br />Read this great post first by William Kempf. And It's true!</span><br />
<span style="font-family: "Trebuchet MS",sans-serif;"> <a href="http://www.digitaltapestry.net/blog/extending-mstest">http://www.digitaltapestry.net/blog/extending-mstest</a></span><br />
<br />
<span style="font-family: "Trebuchet MS",sans-serif;"> Next if you're willing to bear with all that, these are the only 2 posts from circa-2009 on the intertubes that give you any hope <br /><br />MSDN Blogs - Bruce Taimana <a href="http://blogs.msdn.com/b/vstsqualitytools/archive/2009/09/04/extending-the-visual-studio-unit-test-type-part-1.aspx">Part 1</a>. <a href="http://blogs.msdn.com/b/vstsqualitytools/archive/2009/09/04/extending-the-visual-studio-unit-test-type-part-2.aspx">Part 2</a><br /> </span><br />
<h3>
<span style="font-family: "Trebuchet MS",sans-serif;">Writing an extension in 2013</span></h3>
<span style="font-family: "Trebuchet MS",sans-serif;">The details have changed since 2009 and combined with the paucity of information, it was three days before I had something to show.<br /><br />So my target extension was going to bring in NUnit's TestCaseSource functionality, whereby you could supply the parameters for a parameterized test via a method (at runtime. Compile-time is already supported via XML and DBs.. though cumbersome.)</span></div>
<pre class="brush: csharp">[TestMethod]
[TestCaseSource("DivideCases")]
public void MultipleParams(int n, int d, int q)
{
Assert.AreEqual( q, n / d );
}
public static object[] DivideCases()
{
return new[]
{
new object[] {12, 3, 4},
new object[] {12, 2, 6},
new object[] {12, 4, 3}
};
}
</pre>
<a name='more'></a><div dir="ltr" style="text-align: left;" trbidi="on">
<h3>
<span style="font-family: "Trebuchet MS",sans-serif;">
Taking it for a spin </span></h3>
<ul style="text-align: left;">
<li><span style="font-family: "Trebuchet MS",sans-serif;">Get the binaries from the <a href="http://code.google.com/p/gishu-util/source/browse/#git%2FMSTestExtensions%2FBinaries">google code repository</a></span><span style="font-family: "Trebuchet MS",sans-serif;"> </span></li>
<li><span style="font-family: "Trebuchet MS",sans-serif;">Copy the extension dll (and pdb) to the Visual studio folder - C:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\IDE\PrivateAssemblies</span><span style="font-family: "Trebuchet MS",sans-serif;"> </span></li>
<li><span style="font-family: "Trebuchet MS",sans-serif;">Next you need to set some registry keys - Install.reg and Uninstall.reg files provided in the binaries folder. Merge the install one.</span><span style="font-family: "Trebuchet MS",sans-serif;"> </span></li>
<li><span style="font-family: "Trebuchet MS",sans-serif;">Phew. Deployment on build machines is gonna be a pain. Start up Visual studio. Create a test project as usual. Reference the extension dll to gain access to the attributes</span><span style="font-family: "Trebuchet MS",sans-serif;"> </span></li>
<li><span style="font-family: "Trebuchet MS",sans-serif;">The changes: </span></li>
<ul>
<li><span style="font-family: "Trebuchet MS",sans-serif;">Mark your test class with the new ParamTestClass attribute. </span></li>
<li><span style="font-family: "Trebuchet MS",sans-serif;">The test method attribute stays the same. Specify inputs that you wish to pass to your test methods.</span></li>
<li><span style="font-family: "Trebuchet MS",sans-serif;">Specify the name of the method that will provide the inputs via the TestCaseSource attribute. It has to be a <b>public </b>instance or static <b>method </b>which returns an IEnumerable, object[] or IEnumerable<testcasedata>. Look at <a href="http://code.google.com/p/gishu-util/source/browse/MSTestExtensions/SampleTest/ExampleTests.cs">the Sample tests</a>. TestCases can take single or multiple parameters ; they can also be named e.g. Test Login 'for admin' 'for normal' </testcasedata></span></li>
</ul>
</ul>
</div>
</div>
<br />
<br />
<a href="https://picasaweb.google.com/lh/photo/yF7EIO9TL780POGtB3_wxfcaujkF9KLSFFuJboHAXZ8?feat=embedwebsite"><img height="393" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg_OarCaXZq6W9oUvtdFn0p9iDpz0tJCqIn2RWcKuwH245u9Jff5RreuU6Zj66Qid3y-og3vFilGMJuXiB1Zz1dlDxJtBGKvrSzsQFwpqfbLKkjQR9tZE74x-vX4XQGQ54IjcQ2/s400/TestCaseSource.jpg" width="400" /></a><br />
<br />
<h3 style="text-align: left;">
<span style="font-family: "Trebuchet MS",sans-serif;">Lessons from writing the extension</span></h3>
<ul style="text-align: left;">
<li><span style="font-family: "Trebuchet MS",sans-serif;">The libraries you need to reference have been renamed since the 2009 post</span></li>
<li><span style="font-family: "Trebuchet MS",sans-serif;">You can't add new tests at runtime.. (as mentioned in the digitaltapestry post) The best you can do is </span></li>
<ul>
<li><span style="font-family: "Trebuchet MS",sans-serif;">control invocation of the test method (e.g. call it in a loop with diff params)</span></li>
<li><span style="font-family: "Trebuchet MS",sans-serif;">add pre/post processing (e.g. collate a single test result)</span></li>
</ul>
<li><span style="font-family: "Trebuchet MS",sans-serif;">I couldn't get the UI extension part to work - it seems like those methods were only for use by the Test Results window (which was a .trx viewer). However both of them <a href="http://connect.microsoft.com/VisualStudio/feedback/details/750184/test-results-window-does-not-show-test-results">seem to have been deprecated</a>. No idea what's the new way to better format the test results. I finally gave up and settled for the console output support. Couldn't figure how to set a fixed font for the VS 2012 'Test Output' for tabular output. So the UI Subfolder in the source is useless</span></li>
<li><span style="font-family: "Trebuchet MS",sans-serif;">Full Source - <a href="http://code.google.com/p/gishu-util/source/browse/#git%2FMSTestExtensions">http://code.google.com/p/gishu-util/source/browse/#git%2FMSTestExtensions</a></span></li>
</ul>
</div>
Gishuhttp://www.blogger.com/profile/17616896114730114577noreply@blogger.com1tag:blogger.com,1999:blog-14744716.post-24061466489151925012013-05-03T11:20:00.001+05:302013-05-07T12:18:11.336+05:30Sweeping the IDisposable minefield<div dir="ltr" style="text-align: left;" trbidi="on">
<div dir="ltr" style="text-align: left;" trbidi="on">
<div dir="ltr" style="text-align: left;" trbidi="on">
<span style="font-family: "Trebuchet MS",sans-serif;">IDisposable has been around since the beginning of .Net.</span><br />
<span style="font-family: "Trebuchet MS",sans-serif;">The basic premise is simple..</span><br />
<span style="font-family: "Trebuchet MS",sans-serif;"><br /></span>
<span style="font-family: "Trebuchet MS",sans-serif;">Developers dont need to manage memory now. The 'Garbage Collector' takes care of reclaiming memory for you.. However the GC is non-deterministic.. you can't predict when it will embark on a 'collection' of unused objects. So far so good..</span><br />
<span style="font-family: "Trebuchet MS",sans-serif;">However there are cases when you want deterministic cleanup, e.g. if DB connections are a premium, you want to reclaim them as soon as possible. Enter IDisposable.. Simple concept. Introduce a public Dispose method that you can call when you deem fit.</span><br />
<br />
<h2 style="text-align: left;">
Creating types</h2>
<br />
<h3 style="text-align: left;">
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;"><span style="color: blue;">Summary:</span></span></h3>
<ol style="text-align: left;"><span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;"><span style="color: blue;">
</span></span>
<li><span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;"><span style="color: blue;">All managed objects do NOT need to implement IDisposable. Just let the GC do its thing. You do <b>NOT </b>need to implement IDisposable if</span></span></li>
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;"><span style="color: blue;">
</span></span><ol>
<li><span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;"><span style="color: blue;">does NOT directly own any unmanaged resources (e.g. native handles, memory, pipes, etc..) </span></span></li>
<li><span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;"><span style="color: blue;">does NOT directly own any managed members that implement IDisposable</span></span></li>
<li><span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;"><span style="color: blue;">does NOT have special cleanup needs that must run on-demand/asap e.g. unsubscribe from notifiers, close DB Connections, etc. </span></span></li>
</ol>
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;"><span style="color: blue;">
</span></span>
<li><span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;"><span style="color: blue;">If you need to implement IDisposable <b>BUT</b> do not own any (direct) unmanaged resources, You should <b>NOT </b>throw in a free finalizer.</span></span></li>
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;"><span style="color: blue;">
</span></span><ol>
<li><span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;"><span style="color: blue;">Consider if you can make your type sealed. This simplifies the implementation a great deal.</span></span></li>
<li><span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;"><span style="color: blue;">If you must have <b>subtypes</b>, then implement the version with the <b>virtual Dispose(bool)</b> overload as detailed below. Again, rethink if you can seal the type.</span></span></li>
</ol>
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;"><span style="color: blue;">
</span></span>
<li><span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;"><span style="color: blue;">If you directly own unmanaged resources that need cleanup, check for a managed wrapper type that you can use. e.g. a SafeFileHandle. If there is, use it and fall back to 2. Still no finalizer</span></span></li>
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;"><span style="color: blue;">
</span></span>
<li><span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;"><span style="color: blue;">If you reach here, you need a finalizer. Finalizer pulls in IDisposable. Ensure that you have a deterministic Dispose implementation, that makes the finalizer redundant and avoids the associated performance penalties. Log an error in the finalizer to call your attention to cases where the clients have forgotten to call Dispose. Fix them.</span></span></li>
<span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;"><span style="color: blue;">
</span></span><ol><span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;"><span style="color: blue;">
</span></span>
<li><span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;"><span style="color: blue;">Consider creating a managed wrapper type because finalizers are hard to get right. e.g. SafeMyNativeTypeWrapper. Deriving from SafeHandle is not recommended - better left to experts</span></span></li>
<li><span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;"><span style="color: blue;">Use GC.AddMemoryPressure and its counterpart to 'help' the GC if you are allocating significant amounts of native memory. Similarly manage handles via the HandleCollector class (e.g. GDI handles). See <a href="http://blogs.msdn.com/b/brada/archive/2003/12/12/50948.aspx">this post</a> for details except I'd move the Remove.. calls into Dispose instead of the finalizer.</span></span></li>
</ol>
<ol>
</ol>
</ol>
<h2 style="text-align: left;">
Programming against types that implement IDisposable</h2>
<ol style="text-align: left;">
<li><span style="font-family: "Trebuchet MS",sans-serif;"><span style="color: blue;">Limit the scope of disposable types to within a method. Wrap them within a using block</span> to ensure that Dispose is called (reliably) when control leaves the using block.</span></li>
<li><span style="font-family: "Trebuchet MS",sans-serif;"><span style="color: blue;">If you need to hold on to a disposable type i.e. as a member field, you need to implement IDisposable on the container type.</span> e.g. If A owns B owns C owns D, where D implements IDisposable, then A,B and C need to implement IDisposable as well.</span></li>
<li><span style="font-family: "Trebuchet MS",sans-serif;"><span style="color: blue;">Do not dispose objects that you don't own. </span>e.g.
if you obtain a reference to an object from a container (e.g. a MEF
container or a Form's controls collection) OR a static/global accessor
you don't own the object. Hence you shouldn't call dispose and break
other clients with ObjectDisposedException. Leave it to the container to
Dispose it. </span></li>
</ol>
<br />
<span style="font-family: "Trebuchet MS",sans-serif;">The long-winded version (with code snippets)</span><br />
<span style="font-family: "Trebuchet MS",sans-serif;"></span></div>
<h3 style="text-align: left;">
<span style="font-family: "Trebuchet MS",sans-serif;">Para1: Avoid implementing IDisposable unless necessary, most objects don't need it.</span></h3>
<span style="font-family: "Trebuchet MS",sans-serif;"><br /></span>
<span style="font-family: "Trebuchet MS",sans-serif;">If your type doesn't need IDispose. You can stop reading here.</span><br />
<span style="font-family: "Trebuchet MS",sans-serif;"><br /></span>
<br />
<h3 style="text-align: left;">
<span style="font-family: "Trebuchet MS",sans-serif;">Para2: If you need deterministic cleanup,
implement IDisposable (mini).</span></h3>
<ul style="text-align: left;"><span style="font-family: "Trebuchet MS",sans-serif;">
</span>
<li><span style="font-family: "Trebuchet MS",sans-serif;"> All public members need to check for _isDisposed == true & throw an ObjectDisposedException</span></li>
<span style="font-family: "Trebuchet MS",sans-serif;">
</span>
<li><span style="font-family: "Trebuchet MS",sans-serif;">Dispose can be called multiple times : once again use _isDisposed and ignore all calls except the first one</span></li>
<li><span style="font-family: "Trebuchet MS",sans-serif;">Dispose should not throw exceptions </span></li>
<span style="font-family: "Trebuchet MS",sans-serif;">
</span>
<li><span style="font-family: "Trebuchet MS",sans-serif;">Call Dispose for disposable managed objects that this type owns. Corollary: Dispose will creep all the way up the object graph. e.g. If TypeA contains B contains C contains D and D implements IDisposable: A,B,C need to implement IDisposable.</span></li>
<span style="font-family: "Trebuchet MS",sans-serif;">
</span>
<li><span style="font-family: "Trebuchet MS",sans-serif;">(Managed Memory Leaks from Events) - Unsubscribe from all events that this object has active subscriptions to. Long-lived Publishers can keep short-lived subscribers alive and prevent them from being collected. Try: clearing subscribers from your own events might be a good idea- set event/delegate to null. Try: using the WeakEventManager/WeakReference type</span></li>
<span style="font-family: "Trebuchet MS",sans-serif;">
</span>
<li><span style="font-family: "Trebuchet MS",sans-serif;">Seal your type - inheritance needs the full blown Dispose pattern (later on in this post). </span></li>
</ul>
<br />
<br />
<pre class="brush: csharp">sealed class MyType : IDisposable
{
// other code
private bool _isDisposed;
public void SomeMethod()
{
if (_isDisposed)
throw new ObjectDisposedException();
// proceed..
}
public void Dispose()
{
if (_isDisposed)
return;
// cleanup
_isDisposed = true;
}
}
</pre>
<br />
<br />
<h3 style="text-align: left;">
<span style="font-family: "Trebuchet MS",sans-serif;">Para3: Avoid finalizers unless necessary.</span></h3>
<span style="font-family: "Trebuchet MS",sans-serif;">When are finalizers necessary ?</span><br />
<ul style="text-align: left;">
<li><span style="font-family: "Trebuchet MS",sans-serif;">When you have directly owned Unmanaged resources that need to be cleaned up AND there isn't a managed wrapper type that has the finalization routine nailed down e.g. a SafeHandle derivation. If you can, you go back to the previous section. </span></li>
</ul>
<span style="font-family: "Trebuchet MS",sans-serif;">Finalizers</span><br />
<ul style="text-align: left;">
<li><span style="font-family: "Trebuchet MS",sans-serif;">slow down the collection process </span></li>
<li><span style="font-family: "Trebuchet MS",sans-serif;">prolong object lifetime - the object moves into the next generation (whose collection is even less frequent. C# in a Nutshell gives a ratio of Gen 0 10:1 Gen 1)</span></li>
<li><span style="font-family: "Trebuchet MS",sans-serif;">are difficult to get right </span></li>
</ul>
<span style="font-family: "Trebuchet MS",sans-serif;"><br /></span>
<span style="font-family: "Trebuchet MS",sans-serif;">Finalizers should</span><br />
<ul style="text-align: left;"><span style="font-family: "Trebuchet MS",sans-serif;">
</span>
<li><span style="font-family: "Trebuchet MS",sans-serif;">not block / throw exceptions</span></li>
<span style="font-family: "Trebuchet MS",sans-serif;">
</span>
<li><span style="font-family: "Trebuchet MS",sans-serif;">must execute quickly</span></li>
<span style="font-family: "Trebuchet MS",sans-serif;">
</span>
<li><span style="font-family: "Trebuchet MS",sans-serif;">not reference other finalizable members (their finalizers may have already run)</span></li>
<span style="font-family: "Trebuchet MS",sans-serif;">
</span>
<li><span style="font-family: "Trebuchet MS",sans-serif;">should log / raise a red flag to indicate Dispose was not called.</span></li>
</ul>
<pre class="brush: csharp">sealed class HasUnmanaged : IDisposable
{
public void Dispose()
{
Dispose(true);
// prevent object from being promoted to next Gen/finalizer call
GC.SuppressFinalize(this);
}
~HasUnmanaged()
{
LogSomeoneForgotToCallDispose();
Dispose(false);
}
private bool _isDisposed;
private void Dispose(bool isDisposing)
{
if (_isDisposed)
return;
if (isDisposing)
{
// dispose of managed resources(can access managed members)
}
// release unmanaged resources
_isDisposed = true;
}
}
</pre>
</div>
<br />
<br />
<h3 style="text-align: left;">
<span style="font-family: "Trebuchet MS",sans-serif;">Para4: Subclassing a Disposable type</span></h3>
<span style="font-family: "Trebuchet MS",sans-serif;">If your type cannot be sealed, then it's time to bring out the big guns. Implement the base type as follows</span><br />
<br />
<pre class="brush: csharp">class BaseType : IDisposable
{
Intptr _unmanagedRes = ... // unmanaged resource
SafeHandle _managedRes = ... // managed resource
public void Dispose()
{
Dispose(true);
GC.SuppressFinalize(this);
}
~BaseType()
{
Dispose(false);
}
private bool _isDisposed;
virtual void Dispose(bool isDisposing)
{
if (_isDisposed)
return;
if (isDisposing)
{
// managed resources dispose
_managedRes.Dispose();
}
// unmanaged resource cleanup
Cleanup(_unmanagedRes);
// null out big fields if any
_unmanagedRes = null;
_isDisposed = true;
}
}
</pre>
<ul style="text-align: left;">
<li><span style="font-family: "Trebuchet MS",sans-serif;">If the derived type has it's own resources that need cleanup, it overrides the virtual member like this</span></li>
<span style="font-family: "Trebuchet MS",sans-serif;">
</span>
<li><span style="font-family: "Trebuchet MS",sans-serif;">If the derived type does not have resources that need cleanup, just inherit from the base. You're done. </span></li>
</ul>
<pre class="brush: csharp">class MessyDerived : BaseType
{
// ... Derived members
private bool _isDisposed;
override void Dispose(bool isDisposing)
{
try
{
if (!_isDisposed)
{
if (isDisposing)
{
// derived managed resources
}
// derived unmanaged resources
_isDisposed = true;
}
}
finally
{
base.Dispose(isDisposing);
}
}
}
class SimpleDerived : BaseType
{
// ... Derived members
}
</pre>
<br />
Of course, there will be edge-cases. But for the most part, this should save you a lot of grief<br />
See also - <a href="http://www.bluebytesoftware.com/blog/PermaLink.aspx?guid=88e62cdf-5919-4ac7-bc33-20c06ae539ae">Joe Duffy's post</a> </div>
Gishuhttp://www.blogger.com/profile/17616896114730114577noreply@blogger.com2tag:blogger.com,1999:blog-14744716.post-24841813468123371012012-07-18T09:14:00.001+05:302012-07-18T13:21:28.101+05:30Book Review: Writing Solid Code by Steve Maguire<div dir="ltr" style="text-align: left;" trbidi="on">
<iframe frameborder="0" marginheight="0" marginwidth="0" scrolling="no" src="http://rcm.amazon.com/e/cm?t=myconverswith-20&o=1&p=8&l=as1&asins=1556155514&ref=tf_til&fc1=000000&IS2=1&lt1=_blank&m=amazon&lc1=0000FF&bc1=000000&bg1=FFFFFF&f=ifr" style="height: 240px; width: 120px;"></iframe>
<br />
<div>
4 out of 5 stars.
<br />
<br /></div>
<div>
This is an MS(!) Press book written by Steve Maguire in the early 90s. Why am I reading it now? Jim Weirich (of Rake fame) recommended this book in an online video that I happened to watch. </div>
<div>
The book was targeted at teams working in C ( distilled from the authors stint at Microsoft )... but the advice is pretty relevant even today if you can breeze through the code snippets. It seems to be out of print but you might luck out and find an old copy like I did</div>
<br />
<div>
<ul style="text-align: left;">
<li><span style="background-color: #d9ead3;">Lean on the compiler</span>.. turn on all the warnings by default. Disabling should be the exception not the rule.</li>
<li>Use SCA tools from Day1. Fix issues regularly instead of letting them pile up.</li>
<li><span style="background-color: #d9ead3;">If you have unit tests, use them</span>.</li>
<li>Maintain a fortified debug version of your product with 'dev/debug mode asserts'. Conditionally compiled so that release version is lean.</li>
<li>Use asserts to <span style="background-color: #d9ead3;">identify 'things that should not happen' as early as possible</span>. <span style="color: blue; font-family: "Helvetica Neue", "Arial", "Helvetica", sans-serif;">(Of course tests are much better :)</span></li>
<li><span style="background-color: #d9ead3;">Reviewing the code (personally at least) before committing changes</span> is the easiest and cheapest way to reduce bugs.</li>
</ul>
</div>
<div>
<b>Design</b></div>
<div>
<ul style="text-align: left;">
<li>Don't create 'candy machine interfaces' - make it hard for clients to make mistakes.</li>
<li>Eliminate 'undefined' behavior.. so that clients do not depend on it</li>
<li>Don't bury error codes in return values - make them hard to ignore</li>
<li>A function should do only ONE thing and do it well - <span style="color: blue; font-family: "Helvetica Neue", "Arial", "Helvetica", sans-serif;">(actually I rephrased it. this line's from Clean Code)</span></li>
<li>Don't use flag arguments. </li>
<li><span style="background-color: #d9ead3;">Do not tradeoff client code readability</span> over ease of implementation.. make code legible at point of call.</li>
<li>Avoid sharing/Passing around global data</li>
<li><span style="background-color: #d9ead3;">Don't tradeoff global or algorithmic efficiency for local efficiency</span></li>
<li>If you have to look it up, it isn't obvious. Make it boring and obvious.</li>
<li>Eliminate as many if branches as possible.</li>
<li>Write code for the "average" programmer. Simple over clever.</li>
</ul>
</div>
<div>
<b>Attitude</b> (IMO The best section of the book)</div>
<div>
<ul style="text-align: left;">
<li>Bugs just don't go away.. track them down.</li>
<li><span style="background-color: #d9ead3;">Fix bugs now</span>.. not later</li>
<li>Don't meddle with legacy code if you don't need to</li>
<li><span style="background-color: #d9ead3;">Don't add features if you don't need to</span>. All flexibility has a cost. (maintenance, testing, learning curve, etc.)</li>
<li>Don't keep trying solutions till one works. Take the time to find the correct one. Don't TRY.. READ.</li>
<li>Don't rely on testers to find your bugs. Don't shoot the messenger when they do find your bugs</li>
<li>Never allow the same bug to bite you twice. <span style="background-color: #d9ead3;">Fix your process to stonewall that type of bug</span>.</li>
</ul>
<div>
The book has aged quite well over 20 years. Combined with on topic anecdotes (multicoloured screwdrivers is a keeper), this one is a good book to casually read while you wait for something to complete/load.<br />
<span style="font-size: x-small;"><br /></span><br />
<span style="font-size: x-small;">Note: The highlighted sections indicate Steve was <em>agile</em> before it became cool/a buzzword.</span></div>
</div>
</div>Gishuhttp://www.blogger.com/profile/17616896114730114577noreply@blogger.com1tag:blogger.com,1999:blog-14744716.post-6316849631661613562012-04-26T21:41:00.000+05:302012-04-26T21:43:19.426+05:30Lean Startup Book Review<div dir="ltr" style="text-align: left;" trbidi="on">
<span style="font-family: "Helvetica Neue", "Arial", "Helvetica", sans-serif;">Rated 3/5</span><span style="font-family: "Helvetica Neue", "Arial", "Helvetica", sans-serif;"><br /></span><br />
<span style="font-family: "Helvetica Neue", "Arial", "Helvetica", sans-serif;"><br /></span><br />
<span style="font-family: "Helvetica Neue", "Arial", "Helvetica", sans-serif;">The book is written in lucid manner with lots of anecdotes thrown in to illustrate the point being made. That said I'd have liked it to be concise (the essence could easily be extracted to a couple of blog posts)</span><br />
<span style="font-family: "Helvetica Neue", "Arial", "Helvetica", sans-serif;"><br /></span><br />
<span style="font-family: "Helvetica Neue", "Arial", "Helvetica", sans-serif;">I did a small experiment. Extracted out things that I felt were important take-aways and tried to see if it is something that I didn't know before (highlighted Blue).</span><br />
<span style="font-family: "Helvetica Neue", "Arial", "Helvetica", sans-serif;"><br /></span><br />
<ul style="text-align: left;">
<li><span style="font-size: small;"><span style="font-family: "Helvetica Neue", "Arial", "Helvetica", sans-serif;">What over How. Deciding what to build is the crucial and more difficult problem (tip of the hat to Mr.Brooks)</span></span></li>
<li><span style="font-size: small;"><span style="font-family: "Helvetica Neue", "Arial", "Helvetica", sans-serif;">Validate your value hypothesis and your growth hypothesis. (Fail fast, regular demos, real customer, et. all. The growth angle was new for me)</span></span></li>
<li><div>
<span style="font-size: small;"><span style="font-family: "Helvetica Neue", "Arial", "Helvetica", sans-serif;"><span style="color: blue;">Cohort Analysis, Split testing</span></span></span></div>
</li>
<li><span style="font-size: small;"><span style="font-family: "Helvetica Neue", "Arial", "Helvetica", sans-serif;"><span style="color: blue;">Build Minimum Viable Product ASAP, Establish baseline, Tune, Validate.</span></span></span><span style="font-size: small;"><span style="font-family: "Helvetica Neue", "Arial", "Helvetica", sans-serif;"><span style="color: blue;"><br /></span></span></span></li>
<li><div>
<span style="font-size: small;"><span style="font-family: "Helvetica Neue", "Arial", "Helvetica", sans-serif;"><span style="color: blue;">Metrics must be </span></span></span><span style="font-size: small;"><span style="font-family: "Helvetica Neue", "Arial", "Helvetica", sans-serif;"><span style="color: blue;">Actionable (demonstrate clear cause and effect), </span></span></span><span style="font-size: small;"><span style="font-family: "Helvetica Neue", "Arial", "Helvetica", sans-serif;"><span style="color: blue;">Accessible (simple, unambiguous, unrestricted), </span></span></span><span style="font-size: small;"><span style="font-family: "Helvetica Neue", "Arial", "Helvetica", sans-serif;"><span style="color: blue;">Auditable (verifiable - ability to test data on demand against reality)</span></span></span></div>
</li>
<ul style="text-align: left;">
</ul>
<li><div>
<span style="font-size: small;"><span style="font-family: "Helvetica Neue", "Arial", "Helvetica", sans-serif;">Reduce batch size , Reduce WIP, Pull model (Lean)</span></span></div>
</li>
<li><span style="font-size: small;"><span style="font-family: "Helvetica Neue", "Arial", "Helvetica", sans-serif;"><span style="color: blue;">Growth engines (the best part of the book IMHO)</span></span></span></li>
<ul style="text-align: left;">
<li><span style="font-size: small;"><span style="font-family: "Helvetica Neue", "Arial", "Helvetica", sans-serif;"><span style="color: blue;">Sustainable growth = new customers result from the actions of past customers.</span></span></span></li>
<li><div>
<span style="font-size: small;"><span style="font-family: "Helvetica Neue", "Arial", "Helvetica", sans-serif;"><span style="color: blue;">Sticky: engaging customers for long-term. KeyMetric: Churn rate = Rate of acquiring new customers - attrition rate</span></span></span></div>
</li>
<li><span style="font-size: small;"><span style="font-family: "Helvetica Neue", "Arial", "Helvetica", sans-serif;"><span style="color: blue;">Viral: Not word of mouth. Referral a natural outcome of using the product. Revenue is usually indirect. KeyMetric: Viral co-efficient. How many referrals does a single new customer beget? > 1 indicates growth.</span></span></span></li>
<li><span style="font-size: small;"><span style="font-family: "Helvetica Neue", "Arial", "Helvetica", sans-serif;"><span style="color: blue;">Paid : Revenue invested to acquire new customers. Keymetric: Revenue over lifetime of customer - Cost per customer Aquisition</span></span></span></li>
</ul>
<li><span style="font-size: small;"><span style="font-family: "Helvetica Neue", "Arial", "Helvetica", sans-serif;">It's usually a human problem. Ask 5 whys - answers usually go from technical to human issues. (Gerald Weinberg)</span></span></li>
<li><span style="font-size: small;"><span style="font-family: "Helvetica Neue", "Arial", "Helvetica", sans-serif;">Retrospectives: </span></span></li>
<ul style="text-align: left;">
<li><span style="font-size: small;"><span style="font-family: "Helvetica Neue", "Arial", "Helvetica", sans-serif;">Be tolerant of all mistakes </span></span></li>
<li><span style="font-size: small;"><span style="font-family: "Helvetica Neue", "Arial", "Helvetica", sans-serif;">Never allow the same mistake to be made again</span></span></li>
<li><span style="font-size: small;"><span style="font-family: "Helvetica Neue", "Arial", "Helvetica", sans-serif;">Shame on us for making it so easy to make a mistake.</span></span></li>
</ul>
</ul>
<span style="font-family: "Helvetica Neue", "Arial", "Helvetica", sans-serif;"></span><br />
<div>
<span style="font-family: "Helvetica Neue", "Arial", "Helvetica", sans-serif;"><br /></span></div>
<div>
<span style="font-family: "Helvetica Neue", "Arial", "Helvetica", sans-serif;"><br /></span></div>
</div>Gishuhttp://www.blogger.com/profile/17616896114730114577noreply@blogger.com0tag:blogger.com,1999:blog-14744716.post-13657780817108659632012-03-28T18:31:00.001+05:302012-03-28T18:31:17.175+05:30VS2010 and Resharper 5.1 cheatsheet<div dir="ltr" style="text-align: left;" trbidi="on">
<div dir="ltr" style="text-align: left;" trbidi="on">
<br />
<span style="font-family: "Trebuchet MS", sans-serif;">You can grab the VS2010 settings file </span><a href="http://code.google.com/p/gishu-util/source/browse/trunk/Blog/vs2010env/Gishu.vssettings"><span style="font-family: "Trebuchet MS", sans-serif;">here </span></a><span style="font-family: "Trebuchet MS", sans-serif;">. Import them via VS2010 Main Menu > Tools > Import and Export Settings... and follow the wizard.</span><br />
<span style="font-family: "Trebuchet MS", sans-serif;"><br /></span><br />
<span style="font-family: "Trebuchet MS", sans-serif;">Here's a cheat sheet of the shortcuts that I love.</span><br />
<span style="font-family: "Trebuchet MS", sans-serif;">C => Control, M => Alt</span><br />
<span style="font-family: "Trebuchet MS", sans-serif;"></span><br />
<a name='more'></a><br />
<br /></div>
<table border="0" cellpadding="0" cellspacing="0" class="tblGenFixed" id="tblMain"><tbody>
<tr class="rShim"><td class="rShim" style="width: 0;"></td><td class="rShim" style="width: 120px;"></td><td class="rShim" style="width: 143px;"></td><td class="rShim" style="width: 317px;"></td><td class="rShim" style="width: 176px;"></td></tr>
<tr><td class="hd"><div style="height: 16px;">
.</div>
</td><td class="s0"></td><td class="s1"></td><td class="s2">Resharper 5.1</td><td class="s2">VS</td></tr>
<tr><td class="hd"><div style="height: 16px;">
.</div>
</td><td class="s3">THE BOMBS</td><td></td><td></td><td></td></tr>
<tr><td class="hd"><div style="height: 16px;">
.</div>
</td><td class="s4"></td><td class="s5">M + Enter</td><td class="s5">Pop up quick fixes</td><td></td></tr>
<tr><td class="hd"><div style="height: 16px;">
.</div>
</td><td class="s4"></td><td class="s5">C + S + R</td><td class="s5">Refactor this! </td><td></td></tr>
<tr><td class="hd"><div style="height: 16px;">
.</div>
</td><td class="s4"></td><td class="s5">M + PgUp/PgDown</td><td class="s5">Cycle thru resharper wiggles/error</td><td></td></tr>
<tr><td class="hd"><div style="height: 16px;">
.</div>
</td><td class="s4"></td><td></td><td></td><td></td></tr>
<tr><td class="hd"><div style="height: 16px;">
.</div>
</td><td class="s3">Intellisense</td><td></td><td></td><td></td></tr>
<tr><td class="hd"><div style="height: 16px;">
.</div>
</td><td class="s4"></td><td class="s5">S + M + Space</td><td class="s5">Find possible completion and add using stmt (if library is referenced atleast somewhere in solution</td><td></td></tr>
<tr><td class="hd"><div style="height: 16px;">
.</div>
</td><td class="s4"></td><td></td><td></td><td></td></tr>
<tr><td class="hd"><div style="height: 16px;">
.</div>
</td><td class="s3">Add</td><td></td><td></td><td></td></tr>
<tr><td class="hd"><div style="height: 16px;">
.</div>
</td><td class="s4"></td><td class="s5">C + S + A</td><td></td><td class="s5">Add new item...</td></tr>
<tr><td class="hd"><div style="height: 16px;">
.</div>
</td><td class="s4"></td><td class="s5">M + Ins</td><td class="s5">Add new item in solution explorer, </td><td></td></tr>
<tr><td class="hd"><div style="height: 16px;">
.</div>
</td><td class="s4"></td><td></td><td class="s5">Add methods if you're within a class in text editor</td><td></td></tr>
<tr><td class="hd"><div style="height: 16px;">
.</div>
</td><td class="s4"></td><td></td><td></td><td></td></tr>
<tr><td class="hd"><div style="height: 16px;">
.</div>
</td><td class="s3">Code strafers!</td><td></td><td></td><td></td></tr>
<tr><td class="hd"><div style="height: 16px;">
.</div>
</td><td class="s3">GO TO</td><td></td><td class="s5">Supports partial / Pascal-case filtering - e.g. search for PJ will find PoorJoke</td><td></td></tr>
<tr><td class="hd"><div style="height: 16px;">
.</div>
</td><td class="s4"></td><td class="s5">C + T </td><td class="s5">Jump to Type</td><td></td></tr>
<tr><td class="hd"><div style="height: 16px;">
.</div>
</td><td class="s4"></td><td class="s5">C + S + T </td><td class="s5">Jump to file</td><td></td></tr>
<tr><td class="hd"><div style="height: 16px;">
.</div>
</td><td class="s4"></td><td class="s5">S + M + T </td><td class="s5">Jump to symbol</td><td></td></tr>
<tr><td class="hd"><div style="height: 16px;">
.</div>
</td><td class="s4"></td><td class="s5">C + , </td><td class="s5">Jump to recent file</td><td></td></tr>
<tr><td class="hd"><div style="height: 16px;">
.</div>
</td><td class="s4"></td><td class="s5">M + Home </td><td class="s5">Jump to base class / method</td><td></td></tr>
<tr><td class="hd"><div style="height: 16px;">
.</div>
</td><td class="s4"></td><td class="s5">M + End </td><td class="s5">Jump to derivation / overridden methods</td><td></td></tr>
<tr><td class="hd"><div style="height: 16px;">
.</div>
</td><td class="s4"></td><td class="s5">S + M + L</td><td class="s5">Jump to file in Solution Explorer</td><td></td></tr>
<tr><td class="hd"><div style="height: 16px;">
.</div>
</td><td class="s4"></td><td></td><td></td><td></td></tr>
<tr><td class="hd"><div style="height: 16px;">
.</div>
</td><td class="s4"></td><td class="s5">F8</td><td></td><td class="s5">Go to next error (or next item in any VS List window)</td></tr>
<tr><td class="hd"><div style="height: 16px;">
.</div>
</td><td class="s4"></td><td class="s5">F12 </td><td></td><td class="s5">Go to definition / Drill down</td></tr>
<tr><td class="hd"><div style="height: 16px;">
.</div>
</td><td class="s4"></td><td class="s5">C + -</td><td></td><td class="s5">Pop Browse context (Opposite of F12)</td></tr>
<tr><td class="hd"><div style="height: 16px;">
.</div>
</td><td class="s4"></td><td></td><td></td><td></td></tr>
<tr><td class="hd"><div style="height: 16px;">
.</div>
</td><td class="s4"></td><td class="s5">S + M + F12</td><td class="s5">Find usages</td><td></td></tr>
<tr><td class="hd"><div style="height: 16px;">
.</div>
</td><td class="s4"></td><td class="s5">C + S + M + F12</td><td class="s5">Find usages advanced</td><td></td></tr>
<tr><td class="hd"><div style="height: 16px;">
.</div>
</td><td class="s4"></td><td></td><td></td><td></td></tr>
<tr><td class="hd"><div style="height: 16px;">
.</div>
</td><td class="s3">Code Movers</td><td></td><td></td><td></td></tr>
<tr><td class="hd"><div style="height: 16px;">
.</div>
</td><td class="s4"></td><td class="s5">C + S + A + Up/Down</td><td class="s5">Move code block (line/method/class) up or down the file</td><td></td></tr>
<tr><td class="hd"><div style="height: 16px;">
.</div>
</td><td class="s4"></td><td class="s5">C + S + A + Left/Right</td><td class="s5">Move code block in / out of scopes e.g foreach / if blocks.</td><td></td></tr>
<tr><td class="hd"><div style="height: 16px;">
.</div>
</td><td class="s4"></td><td></td><td></td><td></td></tr>
<tr><td class="hd"><div style="height: 16px;">
.</div>
</td><td class="s3">Custom bindings</td><td></td><td></td><td></td></tr>
<tr><td class="hd"><div style="height: 16px;">
.</div>
</td><td class="s4"></td><td class="s5">S + M + E</td><td></td><td class="s5">VS File.OpenContainingFolder (Text Editor)</td></tr>
<tr><td class="hd"><div style="height: 16px;">
.</div>
</td><td class="s4"></td><td></td><td></td><td class="s5">VS Project.OpenFolderInWindowsExplorer (Global)</td></tr>
<tr><td class="hd"><div style="height: 16px;">
.</div>
</td><td class="s4"></td><td class="s5">S + M + R</td><td></td><td class="s5">VS Project.AddReference</td></tr>
</tbody></table>
</div>Gishuhttp://www.blogger.com/profile/17616896114730114577noreply@blogger.com0tag:blogger.com,1999:blog-14744716.post-30752383559330248582012-02-10T09:00:00.000+05:302012-02-11T09:27:01.890+05:30GUI Testing rehab : Can we start saying NO?<div dir="ltr" style="text-align: left;" trbidi="on">
<div style="font-family: "Trebuchet MS",sans-serif;">
Testing GUIs has been hard, tedious, painful... just bad. But they have been an occupational hazard due to lack of feasible alternatives..</div>
<div style="font-family: "Trebuchet MS",sans-serif;">
<br /></div>
<div style="font-family: "Trebuchet MS",sans-serif;">
There's a hard-earned confidence you get when you see a dancing UI twisting, turning... testing itself. And vendors smelt that from miles away.. and then they homed in with tools. Over-simplified demos were given, Influential people were influenced, buying decisions made, tools thrust on unsuspecting people... "The horror... the horror..."</div>
<div style="font-family: "Trebuchet MS",sans-serif;">
<br /></div>
<div style="font-family: "Trebuchet MS",sans-serif;">
But I digress. GUI tests were problematic because</div>
<ul style="font-family: "Trebuchet MS",sans-serif; text-align: left;">
<li>Flaky: Running tests would just fail without reason one fine day. Reissuing a test run would pass that test (the flashing lights tests anti-pattern) but could fail cause a different intermittent test failure. Trust goes down.. tests get commented.. a dangerous path to tread.</li>
<li>Fragile: Vulnerable to UI/UX changes - Some test broke because
someone turned a textbox into a combobox.. or worse someone just
redesigned a commonly used dialog. Time to throw someone into "the hole"
again.. record-n-replay/Fix all those broken tests. </li>
<li>Time to develop/write: Writing UI tests = tedium. Getting them to "stabilize" takes a while. But 'we use a record-n-replay tool!'.. put a pin on that one.</li>
<li>Time to execute: Don't hold your breath, these tests take a while. Waiting for windows to pop up, scraping info out of controls, etc.</li>
<li>Quirky controls: There are always some automation party poopers. Third party controls that don't exhibit standard behavior / the tool simply refuses to "see" them. But the UI is already "done".. Time to call in some specialists..</li>
<li>Vendor lock-in and Specialists: Our resident expert has vanished without a trace.. who can write/fix the tests? (Shrugs all around) Instant Emergency: "We surely can't swap tools now. How quickly can we hire someone who speaks ToolX ?"</li>
<li>Misc dept: Handle possible failure error dialogs so that the test
doesn't block or wreck the subsequent tests, test sensitive to type of OS, theme, screen resolution, etc. </li>
</ul>
<br />
<div style="font-family: "Trebuchet MS",sans-serif;">
"Enough!" you say. Is there any hope in this post at all?</div>
<div style="font-family: "Trebuchet MS",sans-serif;">
<br /></div>
<div style="font-family: "Trebuchet MS",sans-serif;">
Let's tackle them one at a time.</div>
<div style="font-family: "Trebuchet MS",sans-serif;">
<span style="font-size: large;">Fragility / UX sensitivity</span></div>
<div style="font-family: "Trebuchet MS",sans-serif;">
What if we could extract named actions ( a set of building blocks ) that we could then use to build up our tests. Think Lego blocks (named actions) combining to become models (tests) limited only by your imagination and time.</div>
<div style="font-family: "Trebuchet MS",sans-serif;">
<br /></div>
<div style="font-family: "Trebuchet MS",sans-serif;">
e.g. Let's say I want to test if my (unnamed) mail client</div>
<br />
<pre class="brush: csharp">test CanReceiveEmails
testMailServer.Setup(DummyEmails).For(username,password)
mailClient.Start()
mailClient.AuthorizeOfflineMailStoreAccess(datafile_password)
mailClient.LoginToMailServerAs(username, password)
mailClient.SendAndReceiveAllFolders()
var actualEmails = mailClient.GetUnreadEmails()
Assert.That(actualEmails.Count).Is.EqualTo(DummyEmails.Count)
// more comprehensive checks for message content...
mailClient.Stop()
</pre>
<br />
<div style="font-family: "Trebuchet MS",sans-serif;">
So there, we have identified the actors (I'll call them as Drivers henceforth) in our test and the corresponding keywords/actions that we need them to offer. How did that help us - you ask?</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEinzFOuDFSqHBF2QWZAdIiJbJ0LgmbXpsh8veISAcqCpAm-ARR_9KHT4OjlOhujPcq5lgNUbH5Esffx-FKeE9HnuYZiAW7AXBJT04Xg8NEo1oY4mfQkYp1NHvGqgWOiCBXrQoTm/s1600/NamedActions-1.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEinzFOuDFSqHBF2QWZAdIiJbJ0LgmbXpsh8veISAcqCpAm-ARR_9KHT4OjlOhujPcq5lgNUbH5Esffx-FKeE9HnuYZiAW7AXBJT04Xg8NEo1oY4mfQkYp1NHvGqgWOiCBXrQoTm/s1600/NamedActions-1.png" /></a></div>
<br />
<div style="font-family: "Trebuchet MS",sans-serif;">
We have removed any traces of the UI out
of the test. So let's say the LoginToMailServerAs changes from a modal
application window to an inline standard widget provided by the specific
mail server implementation. All I need to go fix now is the
implementation of the LoginToMailServerAs action and all my tests should stay unchanged.<br />
Also now everyone can just invoke LoginToMailServerAs as a magic incantation without worrying about how it works...it just does!</div>
<div style="font-family: "Trebuchet MS",sans-serif;">
<br /></div>
<div style="font-family: "Trebuchet MS",sans-serif;">
<span style="font-size: small;"><span style="background-color: #b6d7a8; color: black;">Separate intent (WHAT you want to do) from implementation</span><span style="background-color: #b6d7a8;"> (HOW you're doing it):</span></span><b><span style="font-size: small;"> </span></b>Compare that to a run-of-the-mill UI test, the above test is much much more readable. Easier to read, understand, fix/maintain.</div>
<div style="font-family: "Trebuchet MS",sans-serif;">
<br /></div>
<div style="font-family: "Trebuchet MS",sans-serif;">
<span style="font-size: large;">Time To Write</span> - it still takes time but decreases as the store of named actions grows. Every keyword/action needs to be implemented once.. write once use wherever you need it.</div>
<div style="font-family: "Trebuchet MS",sans-serif;">
<br /></div>
<div style="font-family: "Trebuchet MS",sans-serif;">
We've <span style="background-color: #b6d7a8;">lowered the technical expertise needed to write a test.</span> Given the "drivers":cohesive clumps of named actions, requisite tooling and a brief walkthrough of the existing "drivers", someone can discover the APIs to choreograph a specific script - a test. Focus on testing/thinking rather than automation/coding.</div>
<div style="font-family: "Trebuchet MS",sans-serif;">
</div>
<div style="font-family: "Trebuchet MS",sans-serif;">
<span style="font-size: large;">Vendor lock-in and Specialists </span></div>
<ul style="font-family: "Trebuchet MS",sans-serif; text-align: left;">
<li><span style="background-color: #b6d7a8;">The decline of the specialists</span> : "That looks almost like a xUnit style test!" You're observant. Yes you could leverage whatever it is your developers are using for unit tests - this means anyone can now write a test. No more dependency on specialists, learning curves for mastering a proprietary tool, No magic-tool licenses to buy. More money to distribute among the team (that last part is still fiction.. but I'd bet you'll have a really motivated team the day after :)</li>
<li><span style="background-color: #b6d7a8;">Encapsulate Tools</span> : The tool is bounded by the box exposing the keywords. No one outside this box (the driver) knows that you're using White for instance. This makes the tool replaceable and the choice of tool a reversible decision.</li>
</ul>
<div style="font-family: "Trebuchet MS",sans-serif;">
<br /></div>
<div style="font-family: "Trebuchet MS",sans-serif;">
But how do we implement the HOW i.e. the keywords? The Drivers themselves.</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi_iZZV2MS_Ej4Snpoig-6WZC_6me18THnOfYO6hTHium-bIP6J_d5Y7Pjti7uNPZoOiROKbf9u-wObHAz6mrfanyCK_uKYi2wWqp8dOdrjpuWiWi5osnzEALdxG_QA-0wg7ncd/s1600/NamedActions-Driver.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="119" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi_iZZV2MS_Ej4Snpoig-6WZC_6me18THnOfYO6hTHium-bIP6J_d5Y7Pjti7uNPZoOiROKbf9u-wObHAz6mrfanyCK_uKYi2wWqp8dOdrjpuWiWi5osnzEALdxG_QA-0wg7ncd/s320/NamedActions-Driver.png" width="320" /></a></div>
<br />
<div style="font-family: "Trebuchet MS",sans-serif;">
You could use an open-source library like <a href="http://white.codeplex.com/">White</a> (or equivalent) that can launch/attach to a running instance of a GUI app, find windows/controls and poke them. (Anything that helps you implement the ControlFinder role shown later)<br />
<br /></div>
<div style="font-family: "Trebuchet MS",sans-serif;">
<br /></div>
<div style="font-family: "Trebuchet MS",sans-serif;">
<span style="font-size: large;">Flakiness</span> </div>
<div style="font-family: "Trebuchet MS",sans-serif;">
Depends on your choice of UI controls and your Automation library. e.g. with C#/WPF applications that run on Windows, I've found White to be pretty robust. < 5% chance of White playing truant.</div>
<div style="font-family: "Trebuchet MS",sans-serif;">
<br />
<span style="color: #4c1130; font-size: x-large;">Enter VM/PM Tests</span></div>
<div style="font-family: "Trebuchet MS",sans-serif;">
"That's it ??!! These are still UI Tests! What about writing all that nasty UI Automation code?" White has wrapped the nastiness within a bunch of control-wrapper types. (You could add your own too). However for special controls, you'd still need to get your hands dirty.</div>
<div style="font-family: "Trebuchet MS",sans-serif;">
<br /></div>
<div style="font-family: "Trebuchet MS",sans-serif;">
"But these tests still crawl!"</div>
<div style="font-family: "Trebuchet MS",sans-serif;">
<br /></div>
<div style="font-family: "Trebuchet MS",sans-serif;">
Beyond this, the target application has to be (re-?) structured or as the self-righteous phrase goes 'designed for testability'. Here's one idea that should work... </div>
<div style="font-family: "Trebuchet MS",sans-serif;">
<br /></div>
<div style="font-family: "Trebuchet MS",sans-serif;">
All of the remaining issues are due to the GUI. There are so many types of UI controls to automate. Waiting for windows and finding controls in large hierarchies takes time. What if I slice the UI out ?</div>
<div style="font-family: "Trebuchet MS",sans-serif;">
e.g. let's consider the Login.. named action (which involves bringing up the login dialog, entering the username, password and clicking OK)</div>
<div style="font-family: "Trebuchet MS",sans-serif;">
<br /></div>
<div style="font-family: "Trebuchet MS",sans-serif;">
If we design it such that the UI is thin (devoid of any code/logic) and merely "binds"/maps to fields and methods in a backing class. This means updating the control will trigger the backing field to update and vice-versa. Doing an action like clicking a button will trigger the underlying method.</div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEinL5DJJSgmbC8oj_qm8hutPhOaiRkZb3_CCRekZvatjqmG8vHTfepmYBKsXfUsZmcbbDw2TDg8e2yPyl8TAk0BC2f__Ng5iJPOl6BvdN-tElYtSdUuHo8nMbTpEY2QjiHEMCcd/s1600/MVVM.png" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" height="400" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEinL5DJJSgmbC8oj_qm8hutPhOaiRkZb3_CCRekZvatjqmG8vHTfepmYBKsXfUsZmcbbDw2TDg8e2yPyl8TAk0BC2f__Ng5iJPOl6BvdN-tElYtSdUuHo8nMbTpEY2QjiHEMCcd/s400/MVVM.png" width="277" /></a></div>
<br />
<br />
<div style="font-family: "Trebuchet MS",sans-serif;">
This technique has been known for some time now (<a href="http://martinfowler.com/eaaDev/PresentationModel.html">Presentation Model (MVP)</a> - Martin Fowler (2003) OR Microsoft's variation called <a href="http://msdn.microsoft.com/en-us/magazine/dd419663.aspx">MVVM</a>, which leverages .Net's in-built data-binding feature to make WPF apps faster to develop).</div>
<div style="font-family: "Trebuchet MS",sans-serif;">
<br /></div>
<div style="font-family: "Trebuchet MS",sans-serif;">
The only thing that the UI contains is the layout, the controls and the wiring to the underlying class. (Even that can be automatic if you move into the realm of advanced MVVM - look for Rob Eisenberg's MIX talk which uses convention to auto-bind). The more important thing is that most of the code (and bugs as a corollary) has moved into a testable class - the ViewModel / PresentationModel. The whole app is basically a symphony orchestrated by multiple presenters.</div>
<div style="font-family: "Trebuchet MS",sans-serif;">
<br /></div>
<div style="font-family: "Trebuchet MS",sans-serif;">
So instead of fidgeting with the UI, I can now just assign desired values to corresponding properties and invoke the OK method to simulate the whole Login process. Much better - plain method calls. What if I can load the whole app from the ViewModel layer down in my test process? That'd be great.<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhqozn73MF6TrEdv0y6EHhVOxscJzNxiYtPpFyDbTAuk390QssIIL6Q1yyDlO1nTx0Zour9hyj5HGj64V8PXyyyf4_q0LFDjUdOw5IL41EE6ZCjKtG65O6_Smm3cY2H53TkdIze/s1600/SideBySide.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="520" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhqozn73MF6TrEdv0y6EHhVOxscJzNxiYtPpFyDbTAuk390QssIIL6Q1yyDlO1nTx0Zour9hyj5HGj64V8PXyyyf4_q0LFDjUdOw5IL41EE6ZCjKtG65O6_Smm3cY2H53TkdIze/s640/SideBySide.png" width="640" /></a></div>
</div>
<div style="font-family: "Trebuchet MS",sans-serif;">
<div class="separator" style="clear: both; text-align: center;">
</div>
<br /></div>
<div style="font-family: "Trebuchet MS",sans-serif;">
Benefits:</div>
<ul style="font-family: "Trebuchet MS",sans-serif; text-align: left;">
<li>Time to develop: No need to write UI automation code. Just <span style="background-color: #b6d7a8;">call existing methods and set properties</span> that the developers have (already) created as part of the implementation. Quick, simple and easy.</li>
<li>Time to execute: No more flashing windows, looking for controls and manipulating them. If you're able to load the whole app sans the UI layer within your test process, you are effectively creating a bunch of objects, toying with them and then letting the garbage collector clean them up. It's way slower than a unit test (because you're using all the real services, data stores, devices etc..) but would be <span style="background-color: #b6d7a8;">faster than a traditional GUI test</span> (Presentation Intensive tests will show a bigger gap as compared to something that spends most of its time talking to a slow hardware device. YMMV)</li>
<li><span style="background-color: #b6d7a8;">Quirky anti-automation Controls: Buh-Bye!</span> Instead of grappling with a third party tree/grid that doesn't want to be found, you can just reach into the VM/PM layer & grab a friendly in-memory collection (that it binds to) within the corresponding ViewModel/Presenter.</li>
</ul>
<div style="font-family: "Trebuchet MS",sans-serif;">
<br /></div>
<div style="font-family: "Trebuchet MS",sans-serif;">
But wait it gets better..</div>
<ul style="font-family: "Trebuchet MS",sans-serif; text-align: left;">
<li><span style="background-color: #b6d7a8;">Decoupled the testers from the implementation</span> : This means as long as you give them some key information, the testers can start writing the tests</li>
</ul>
<br />
<pre class="brush: csharp">
public interface ControlLocator
{
Window GetWindow(string title);
T GetControl<T>(Window parentWindow, string automationId) where T : Control;
}
public interface ViewModelLocator
{
T GetViewModel<T>() where T : ViewModel;
}
</pre>
<div style="font-family: "Trebuchet MS",sans-serif;">
What testers need to know to implement the Drivers.</div>
<ol style="font-family: "Trebuchet MS",sans-serif; text-align: left;">
<li><span style="font-size: large;">for
UI Tests: ParentWindow + ControlType + unique ControlId </span></li>
<li><span style="font-size: large;">VM Tests: ViewModelType + PropertyName/CommandName
</span>
</li>
</ol>
<ul style="font-family: "Trebuchet MS",sans-serif; text-align: left;">
<li><t><t><span style="background-color: #b6d7a8;">Enable Test-first</span>:</t></t> they don't have to wait till the whole thing is implemented to write the tests. e.g. With record-and-replay style of tools, you'd have to wait till the development team gives you the running application to begin test automation. Especially important for teams practicing one of the Agile methods. You could now enable the teams to move up to ATDD. </li>
</ul>
<div style="font-family: "Trebuchet MS",sans-serif;">
<t><t>So let's do a recap, </t></t></div>
<div style="font-family: "Trebuchet MS",sans-serif;">
<br /></div>
<t><t><b style="font-family: "Trebuchet MS",sans-serif;">Current: </b><span style="font-family: "Trebuchet MS",sans-serif;">We started at the top where teams have heavy investments in GUI Testing. These tests are work magnets : maintenance-heavy... sucking in team resources... high cost to benefit as compared to the PM/VM Tests.</span></t></t><br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiDU6vibBkJkXHVFbOegVCOKuLA-no-i0Oq1BjkawlFQXzsMOSHbxJybr0EgHbWh3A1FDuDM2lvaqs-prp4aCIj3N7ksjgxiDTrqYM7hfo1ncEKDcGhxguRkWCIu69_eWMB6J9j/s1600/Progression.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="260" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiDU6vibBkJkXHVFbOegVCOKuLA-no-i0Oq1BjkawlFQXzsMOSHbxJybr0EgHbWh3A1FDuDM2lvaqs-prp4aCIj3N7ksjgxiDTrqYM7hfo1ncEKDcGhxguRkWCIu69_eWMB6J9j/s320/Progression.png" width="320" /></a></div>
<t><t> <br /><span style="font-family: "Trebuchet MS",sans-serif;"><b>Target:</b> By identifying reusable actions with intention-revealing names, we can construct tests much faster than before and with less cost (Programmers will recognize this as the Extract Method refactoring). </span><br style="font-family: "Trebuchet MS",sans-serif;" /><span style="font-family: "Trebuchet MS",sans-serif;">Further by peeling off the UI layer, we get a scriptable interface (an API so to speak) for the target application. We can write most of the system-level tests without the UI.. most teams still like to write some UI tests just as backup.</span><br style="font-family: "Trebuchet MS",sans-serif;" /><br style="font-family: "Trebuchet MS",sans-serif;" /><span style="font-family: "Trebuchet MS",sans-serif;"><b>Stretch:</b> Finally IF a team resolves to write comprehensive unit tests (such that most bugs don't make it past the green section), uses VM tests to catch integration defects and makes every defect an opportunity to fix the process: you could STOP writing UI tests at all (James Shore is a proponent and seems to have had success with this). The time saved in UI Automation can be put to better use - exploratory testing. Not all teams will get here.. but if you make it here, you'll never want to go back. You'd be able to deliver more features per unit time.</span><br style="font-family: "Trebuchet MS",sans-serif;" /><br style="font-family: "Trebuchet MS",sans-serif;" /><span style="font-family: "Trebuchet MS",sans-serif;">So what do you need to do:</span><span style="font-family: "Trebuchet MS",sans-serif;"> </span></t></t><br />
<ul style="text-align: left;">
<li><t><t><span style="font-family: "Trebuchet MS",sans-serif;">Get enough user-context to create a library of named actions; called keywords by some. Tests are written in terms of these keywords. Remember What - not how. e.g. EnterUserNameField() or ClickLogin() is bad; ask "WHY?" to chunk up and you should reach Login(username, password)</span></t></t></li>
<li><t><t><span style="font-family: "Trebuchet MS",sans-serif;">Let testers step into the shoes of the user & shape this interface outside-in. Pair them with a good programmer to ensure you have a "Discoverable API" i.e. easier to figure out on your own given tooling support (e.g. IDE Intellisense).</span></t></t></li>
</ul>
<t><t><br style="font-family: "Trebuchet MS",sans-serif;" /><span style="font-family: "Trebuchet MS",sans-serif;">For UI-less tests,</span></t></t><br />
<ul style="text-align: left;">
<li><t><t><span style="font-family: "Trebuchet MS",sans-serif;"></span><span style="font-family: "Trebuchet MS",sans-serif;">Follow a design technique like MVP or MVVM. Minimize code in the UI.. so that it's easier to test.</span></t></t><span style="font-family: "Trebuchet MS",sans-serif;"> </span></li>
<li><span style="font-family: "Trebuchet MS",sans-serif;">Ensure that you do not need the UI to start an instance of your SUT/application. Have a composition-root (e.g. a Main() function where the app comes together)</span><span style="font-family: "Trebuchet MS",sans-serif;"> </span></li>
<li><span style="font-family: "Trebuchet MS",sans-serif;">Abstract out the User-interaction. So you can't pull a MessageBox or ShowDialog() out of thin air in the ViewModel code. You create a Role e.g. User. The production implementation of User will probably pop up dialogs. When you need to test without the UI, you replace it with a fake-object controlled by your test.</span></li>
</ul>
<t><t><br style="font-family: "Trebuchet MS",sans-serif;" /><span style="font-family: "Trebuchet MS",sans-serif;">Now the preceding bullets are easier said than done for zillion-line legacy apps. For greenfield projects, I find this a very enticing alternative - there is no reason to not build it in test by test. We've crossed out most of the perils of UI Tests.</span></t></t><br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi5OYCUKgW8oxdfSgzAUteVXMAqct8h7ngctaDeuivIBLsw_YpJHjiQlmEvxLE6grchIvn41sB0rH3Uzm6LxgzPRLot9FeIAQvDdwqUq3ysogCWqAxlatW6FKo79_0mW890ykTr/s1600/PerilsOfUITesting.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="223" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi5OYCUKgW8oxdfSgzAUteVXMAqct8h7ngctaDeuivIBLsw_YpJHjiQlmEvxLE6grchIvn41sB0rH3Uzm6LxgzPRLot9FeIAQvDdwqUq3ysogCWqAxlatW6FKo79_0mW890ykTr/s320/PerilsOfUITesting.png" width="320" /></a></div>
<br />
<t><t><span style="font-family: "Trebuchet MS",sans-serif;"><br /></span></t></t><br />
Feel free to question / enhance / criticize with objective reasons / list pros and cons.... In the words of the Human Torch: "Flame on!"</div>Gishuhttp://www.blogger.com/profile/17616896114730114577noreply@blogger.com0Service Rd, Sainath Nagar, Majiwada, Thane, Maharashtra, India19.213183175881142 72.98571825027465819.211308675881142 72.983250750274664 19.215057675881141 72.988185750274653tag:blogger.com,1999:blog-14744716.post-83070240492366846242011-11-14T16:31:00.001+05:302013-01-16T22:19:48.835+05:30NUnit vs MSTest - 2011 Edition<div dir="ltr" style="text-align: left;" trbidi="on">
<div dir="ltr" style="text-align: left;" trbidi="on">
<div style="font-family: "Trebuchet MS",sans-serif;">
<span style="font-size: small;">I</span><span style="font-size: small;"> have tried to be as objective as possible. Disclaimer: NUnit user since 2005-06.</span></div>
<span style="font-size: small;"><br /></span>
<span style="font-family: "Trebuchet MS",sans-serif; font-size: small;">Legend:</span><br />
<ul style="text-align: left;">
<li><span style="font-family: "Trebuchet MS",sans-serif; font-size: small;">MSTest as an alias for the unit-testing fwk bunded with VS2010 v10.0.30319 throughout this post (although technically it is the just the runner). It's much easier to say than VisualStudio QualityTools UnitTestFramework. For NUnit, I'm using v2.5.9</span></li>
<li><span style="font-family: "Trebuchet MS",sans-serif; font-size: small;">Class-Setup/Teardown - to be executed ONCE before/after ALL tests </span></li>
<li><span style="font-family: "Trebuchet MS",sans-serif; font-size: small;">Test-Setup/Teardown - to be executed before/after EVERY test</span></li>
</ul>
<br />
<div style="color: #660000;">
<span style="font-family: "Trebuchet MS",sans-serif; font-size: large;">Major Differences</span></div>
<span style="font-family: "Trebuchet MS",sans-serif;">Migrating a test suite between the two runners might be non-trivial depending on your test code. MSTest and NUnit are very different in the way they go about running a set of tests. e.g. Consider a set of 2 classes containing tests - Class A with 2 tests, B with just one. If I insert logging at some critical points.. (I am looking at the default behavior - no customization)</span><br />
<a name='more'></a></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEitI-8rdo5rI3Ek3AHc_vjqwWug2xgqozBjTx60x1ISc_5xZ0jOpTKUgWWoo6IzfIFQvriR1rUmDXMrCjAL1ZgmGdXr01nM6rryLtDEmqCd5_DU47js1oNHGQNFK97pCc_CQ5uH/s1600/NUnit-log.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="236" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEitI-8rdo5rI3Ek3AHc_vjqwWug2xgqozBjTx60x1ISc_5xZ0jOpTKUgWWoo6IzfIFQvriR1rUmDXMrCjAL1ZgmGdXr01nM6rryLtDEmqCd5_DU47js1oNHGQNFK97pCc_CQ5uH/s640/NUnit-log.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">NUni<span style="font-family: "Trebuchet MS",sans-serif;">t</span></td></tr>
</tbody></table>
<div dir="ltr" style="text-align: left;" trbidi="on">
<span style="font-family: "Trebuchet MS",sans-serif;"> Observations:</span><br />
<ul style="text-align: left;">
<li><span style="font-family: "Trebuchet MS",sans-serif;">Tests are run in sequence (no overlap)</span></li>
<li><span style="font-family: "Trebuchet MS",sans-serif;">Single threaded : <b>one thread</b> runs all the tests</span></li>
<li><span style="font-family: "Trebuchet MS",sans-serif;">All tests within a test class are run on the <b>SAME instance</b> <a href="#1" name="top1"><sup>1</sup></a></span></li>
<li><span style="font-family: "Trebuchet MS",sans-serif;">xUnit order :</span></li>
<ul>
<li><span style="font-family: "Trebuchet MS",sans-serif;">for each test class </span></li>
<ul>
<li><span style="font-family: "Trebuchet MS",sans-serif;">Class setup</span></li>
</ul>
<ul>
<li><span style="font-family: "Trebuchet MS",sans-serif;">for each test</span></li>
<ul>
<li><span style="font-family: "Trebuchet MS",sans-serif;">test setup</span></li>
<li><span style="font-family: "Trebuchet MS",sans-serif;">test</span></li>
<li><span style="font-family: "Trebuchet MS",sans-serif;">test teardown</span></li>
</ul>
<li><span style="font-family: "Trebuchet MS",sans-serif;">Class teardown</span></li>
</ul>
</ul>
</ul>
</div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg66BtpurZoKOb9VOT3dk_AJXkxSAgNQo9Sg9ahEZfhAYXOwIKXn0S30TlAsQyUAv4MSIbVMjcyONOJcSlYKxfPXHhWiMNELPsCcY2O9272TJuJjCP0j9Mdsadpz6e_uFVYI_3z/s1600/MSTest-log.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="242" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg66BtpurZoKOb9VOT3dk_AJXkxSAgNQo9Sg9ahEZfhAYXOwIKXn0S30TlAsQyUAv4MSIbVMjcyONOJcSlYKxfPXHhWiMNELPsCcY2O9272TJuJjCP0j9Mdsadpz6e_uFVYI_3z/s640/MSTest-log.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">MSTest</td></tr>
</tbody></table>
<div dir="ltr" style="text-align: left;" trbidi="on">
<span style="font-family: "Trebuchet MS",sans-serif;">Observations: </span></div>
<div dir="ltr" style="text-align: left;" trbidi="on">
<ul style="text-align: left;">
<li><span style="font-family: "Trebuchet MS",sans-serif;">Tests are run in sequence</span></li>
<li><span style="font-family: "Trebuchet MS",sans-serif;"><b>Multi-threaded</b> even when tests are run sequentially : We can see 4 threads in this example (1 per test + 1 for class-teardowns)</span></li>
<li><span style="font-family: "Trebuchet MS",sans-serif;">A <b>NEW Instance</b> of the test class is created <b>PER TEST</b> (see 3 diff hashcode values)</span></li>
<li><span style="font-family: "Trebuchet MS",sans-serif;">MSTest order :</span></li>
<ul>
<li><span style="font-family: "Trebuchet MS",sans-serif;">for each test-class</span></li>
<ul>
<li><span style="font-family: "Trebuchet MS",sans-serif;">Class setup</span></li>
<li><span style="font-family: "Trebuchet MS",sans-serif;">for each test</span></li>
<ul>
<li><span style="font-family: "Trebuchet MS",sans-serif;">test setup</span></li>
<li><span style="font-family: "Trebuchet MS",sans-serif;">test</span></li>
<li><span style="font-family: "Trebuchet MS",sans-serif;">test teardown</span></li>
</ul>
</ul>
<li><span style="font-family: "Trebuchet MS",sans-serif;">Class cleanup for all test classes visited</span></li>
</ul>
</ul>
</div>
<div dir="ltr" style="text-align: left;" trbidi="on">
<span style="font-family: "Trebuchet MS",sans-serif;">What does all this mean ?</span><br />
<ol style="text-align: left;">
<li><span style="font-family: "Trebuchet MS",sans-serif;">MSTest differs from xUnit in its implementation of <span style="background-color: yellow;">test class setup or teardown</span>. They <span style="background-color: yellow;">have to be static methods</span>, which excludes setting up anything in instance variables to be used by subsequent tests. You can promote them to static (aka global) variables but combined with multiple-threads, be careful! </span></li>
<li><span style="font-family: "Trebuchet MS",sans-serif;">MSTest <span style="background-color: yellow; color: red;">does not provide deterministic test-class teardown</span> i.e. no guarantees about when it will be run; just that it will be executed. So you need to design your tests to account for this - see what happened in the image above. Instead of Start-Close cycles, I see Start-Start-Close-Close!! I haven't yet figured out a workaround for this.. so if you know please post a comment.</span></li>
</ol>
<div style="color: #660000;">
<span style="font-family: "Trebuchet MS",sans-serif; font-size: large;">Test-Class Inheritance</span></div>
</div>
<div dir="ltr" style="text-align: left;" trbidi="on">
<span style="font-family: "Trebuchet MS",sans-serif;">xUnit has some patterns that use inheritance to achieve the desired effect. So let's see they fare in this dept.</span></div>
<div dir="ltr" style="text-align: left;" trbidi="on">
<ul style="text-align: left;"></ul>
</div>
<table cellpadding="0" cellspacing="0" class="tr-caption-container" style="float: left; margin-right: 1em; text-align: left;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjlHeBVcn-ataCdQCrf-y0Hsv4xmx0BIlwbxMAYhy6GXkHNC1aM2RvdLwNcop-sk3J1HbfivgvTzt2ssBnqSWnjsjeWyjgSBWhPibtN7Rwbt1U6IBedOdNtu54evo8dt5cuU80x/s1600/NUnit-Inheritance.png" imageanchor="1" style="clear: left; margin-bottom: 1em; margin-left: auto; margin-right: auto;"><img border="0" height="157" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjlHeBVcn-ataCdQCrf-y0Hsv4xmx0BIlwbxMAYhy6GXkHNC1aM2RvdLwNcop-sk3J1HbfivgvTzt2ssBnqSWnjsjeWyjgSBWhPibtN7Rwbt1U6IBedOdNtu54evo8dt5cuU80x/s400/NUnit-Inheritance.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">NUnit : abstract fixture</td></tr>
</tbody></table>
<table cellpadding="0" cellspacing="0" class="tr-caption-container" style="float: left; margin-right: 1em; text-align: left;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjy0wDnw0Yaq0NQQxwkmGiG36a8HId0KTAoC6eSCQIDQ4fRYzj9gsn-7pn-w8C3iOBVZ9DNeWL1jjLl9HrM9ZJEkeH7z4fh8-C0KW98qW0TRR3kp-0hjWSLIAUmvSiZCytfi_hg/s1600/MSTest-Inheritance.png" imageanchor="1" style="clear: right; margin-bottom: 1em; margin-left: auto; margin-right: auto;"><img border="0" height="155" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjy0wDnw0Yaq0NQQxwkmGiG36a8HId0KTAoC6eSCQIDQ4fRYzj9gsn-7pn-w8C3iOBVZ9DNeWL1jjLl9HrM9ZJEkeH7z4fh8-C0KW98qW0TRR3kp-0hjWSLIAUmvSiZCytfi_hg/s400/MSTest-Inheritance.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">MSTest : abstract fixture</td></tr>
</tbody></table>
<div dir="ltr" style="text-align: left;" trbidi="on">
<div dir="ltr" style="text-align: left;">
<ul style="text-align: left;">
<li><span style="font-family: "Trebuchet MS",sans-serif;">the <b>abstract fixture pattern</b> - move common methods (e.g. common setup/teardown into a abstract base fixture)</span></li>
<ul>
<li><span style="font-family: "Trebuchet MS",sans-serif;"><span style="background-color: yellow; color: red;">MSTest (right) won't execute Class-Setup/Teardown methods defined in base types.</span> Test-Setup/Teardowns are treated differently. (so you may not be able to layer setup/teardown as shown above ). Also remember Class-Teardown is unpredictable.</span></li>
</ul>
</ul>
<span style="font-family: "Trebuchet MS",sans-serif;"> </span></div>
<div style="text-align: left;">
<span style="font-family: "Trebuchet MS",sans-serif;"> </span></div>
<div style="text-align: left;">
<span style="font-family: "Trebuchet MS",sans-serif;"> </span></div>
<div dir="ltr" style="text-align: left;">
<ul style="text-align: left;"><ul>
</ul>
<li><span style="font-family: "Trebuchet MS",sans-serif;">the <b>abstract test
pattern</b> - all the tests are housed in an abstract class. You then subclass and override a method that provides the actual object to be
tested. e.g. run 'contract tests' against multiple implementations of an
interface. </span></li>
<ul>
<li><span style="font-family: "Trebuchet MS",sans-serif;">NUnit : You can cleanly extract a POCO abstract base type with test methods. <span style="background-color: white; color: blue;">MSTest:</span> Additionally, you need to decorate the base type also with a TestClass attribute. (It won't show up in the TestView.)<span style="background-color: #d9ead3;"> </span></span><span style="background-color: black; color: blue; font-family: "Trebuchet MS",sans-serif;"><span style="background-color: #d9ead3;">But can be done!</span></span> </li>
</ul>
</ul>
</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgMyRpKXOKr_Lx3VFkOus6_UVCRTHJj7DMPhWdY1vVV-qKLCTaVAEN6KmWe_CrQ0M6P5wS01n5ipFvyXTVbJpMgmEMyZGbF82bt2u7E6tSDplAksydkci0M4v8cuTI1SRJkO7fK/s1600/AbstractTest.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="228" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgMyRpKXOKr_Lx3VFkOus6_UVCRTHJj7DMPhWdY1vVV-qKLCTaVAEN6KmWe_CrQ0M6P5wS01n5ipFvyXTVbJpMgmEMyZGbF82bt2u7E6tSDplAksydkci0M4v8cuTI1SRJkO7fK/s640/AbstractTest.png" width="640" /></a></div>
<span style="font-family: "Trebuchet MS",sans-serif;"> </span><span style="font-family: "Trebuchet MS",sans-serif;"><br /></span><br />
<br />
<div style="color: #660000;">
<br /></div>
<div style="color: #660000;">
<span style="font-size: large;"><span style="font-family: "Trebuchet MS",sans-serif;">Parameterized Tests (Data driven)</span></span></div>
<span style="font-family: "Trebuchet MS",sans-serif;">I remember reading (when I first heard of TDD) that tests should be self-contained and shouldn't need to take inputs. Well, we've changed our mind. The need to execute the same test logic with different data sets (without duplication) led to RowTests. NUnit integrated a contributed extension and now there are <a href="http://www.nunit.org/index.php?p=testCase&r=2.5">TestCases</a>.</span><br />
<span style="font-family: "Trebuchet MS",sans-serif;">So let's test if how our client handles authentication. </span><br />
<span style="font-family: "Trebuchet MS",sans-serif;"><br /></span>
<pre class="brush: csharp">[TestCase("WrongUser", "Don't care", TestName = "Invalid user")]
[TestCase("Gishu", "Wrong Password", TestName = "Wrong password")]
[TestCase("Gishu", "openSesame", TestName = "Right credentials")]
public void ThrowsExceptionIfCredentialsAreWrong(string username, string password)
{
var client = new Client();
var exception = Assert.Throws<ConnectFailedException>(
() => client.Connect("129.0.0.1", username, password));
Assert.That(exception, Has.Message.EqualTo("Invalid Credentials"));
}
</pre>
<span style="font-family: "Trebuchet MS",sans-serif;"> </span><br />
<span id="goog_353621318"></span><span id="goog_353621319"></span><br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhFTaFYpTc60V-CQD9DMG9z051kPuo4lx9yh7EtO3IbWqOuk-Evt0GLzQuwWpP2qdp-ww2BqJpn7troAiUX6YdxBdGdCBP1EWmShvAWG9sBW8LcMxVAVgTfX8MwcKF8vMgHDUkQ/s1600/N-RowTests.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhFTaFYpTc60V-CQD9DMG9z051kPuo4lx9yh7EtO3IbWqOuk-Evt0GLzQuwWpP2qdp-ww2BqJpn7troAiUX6YdxBdGdCBP1EWmShvAWG9sBW8LcMxVAVgTfX8MwcKF8vMgHDUkQ/s1600/N-RowTests.png" /></a></div>
<br />
<span style="font-family: "Trebuchet MS",sans-serif;">I love <b>TestNames </b>vs anonymous inline data clumps!!</span><br />
<span style="font-family: "Trebuchet MS",sans-serif;">Lets switch to<span style="background-color: yellow; color: red;"> MSTest. No RowTests built-in</span>. There is <a href="http://blogs.msdn.com/b/vstsqualitytools/archive/2009/09/04/extending-the-visual-studio-unit-test-type-part-2.aspx">a patch</a> but it is the bare minimum (and requires me to mess with the registry) ; more on that after the jump. You can't slot in inline data values - but you <span style="background-color: #d9ead3; color: blue;">can pull in data from a CSV/Xml/DB</span>. Hmm... seems like a Hammer to swat a fly. But I gave that a try too; <a href="http://msdn.microsoft.com/en-us/library/ms182527.aspx">HOW TO: Create a data driven test</a>. So I use the wizard, more attributes appear. It works but...</span><br />
<span style="font-family: "Trebuchet MS",sans-serif;"><br /></span>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi4DXx_IdjQhPSw-O5qjzazUuC9lMVHGMO9wk7Dy9-HeYRgun1KXGYu8nnj2c1t8v3I5vZZ-tBHfZAoww5uqRxz7HiRt3-rEouxR8jlFwcZMjlGB5nfuFY5Z71ya_g6GqDgBs-W/s1600/MS-DDT.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="307" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi4DXx_IdjQhPSw-O5qjzazUuC9lMVHGMO9wk7Dy9-HeYRgun1KXGYu8nnj2c1t8v3I5vZZ-tBHfZAoww5uqRxz7HiRt3-rEouxR8jlFwcZMjlGB5nfuFY5Z71ya_g6GqDgBs-W/s640/MS-DDT.png" width="640" /></a></div>
<span style="font-family: "Trebuchet MS",sans-serif;"><br /></span>
<span style="font-family: "Trebuchet MS",sans-serif;">Differences:</span><br />
<ol style="text-align: left;">
<li><span style="font-family: "Trebuchet MS",sans-serif;"><span style="background-color: yellow;">Does not show</span> me 4 <span style="background-color: yellow;">distinct tests in the results</span> (actually the total count circled is wrong..). You <span style="background-color: yellow;">cannot name your tests</span>.</span></li>
<li><span style="font-family: "Trebuchet MS",sans-serif;">You need to pop up test details (another window) for each group to see which tests failed in the group. </span></li>
<li><span style="font-family: "Trebuchet MS",sans-serif;">Even then, you can't see which inputs failed. Annoying.</span></li>
</ol>
<span style="font-family: "Trebuchet MS",sans-serif;">The extension mentioned earlier also suffers from the same. So for static inputs, "No great shakes!"</span><br />
<span style="font-family: "Trebuchet MS",sans-serif;"><b> Dynamic Input generation: </b>But we're still not done, what if we want to compute the inputs using code?? Say hello to NUnit's <a href="http://www.nunit.org/index.php?p=testCaseSource&r=2.5">TestCaseSource </a>and <a href="http://www.nunit.org/index.php?p=valueSource&r=2.5">ValueSource</a>. You can write methods that generate test cases at run-time based on any arbitrary logic (with all the frills: names, tags/category, expectedResult/Exception).</span><br />
<span style="font-family: "Trebuchet MS",sans-serif;">Once again <span style="background-color: yellow;">MSTest - Not Supported</span>.</span><br />
<span style="font-family: "Trebuchet MS",sans-serif;"><br /></span>
<span style="font-family: "Trebuchet MS",sans-serif;"> </span><br />
<div style="color: #660000;">
<span style="font-size: large;"><span style="font-family: "Trebuchet MS",sans-serif;">Parameterized Test Fixtures</span></span></div>
<br />
<span style="font-family: "Trebuchet MS",sans-serif;">Next the big guns.. you don't need them everyday. But when you do, NUnit allows you to really cut down duplication via <a href="http://www.nunit.org/index.php?p=testFixture&r=2.5">Parameterized Fixtures and Generic Fixtures</a>. e.g. consider the following convoluted example </span><br />
<pre class="brush: csharp">[TestFixture(typeof(PhoneX), "Sam", Description = "Model X")]
[TestFixture(typeof(PhoneY), "Htc", Description = "Model Y")]
public class TestAPhone<TPhone> where TPhone : Phone, new()
{
private TPhone _phoneToBeTested;
public TestAPhone(string name)
{
_phoneToBeTested = new TPhone { Name = name };
}
[Test]
public void TestDialing()
{
Console.Out.WriteLine("a phone of type {0} with name {1}", typeof(TPhone), _phoneToBeTested.Name);
}
[Test]
public void TestBatteryLife()...
</pre>
<span style="font-family: "Trebuchet MS",sans-serif;">NUnit interprets that as 2 different test fixtures - It is parameterized.. "Sam" and "Htc" are parameters passed into the suite. It is generic too.. the 2 test fixture attributes caused the entire suite to be run against an object of PhoneX and PhoneY.</span><br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiezbjms_Kjo7qbQ8_Tyr0kVwrEvOhXdbP9NuyWEbOHZkZgk6ZqI85OvSd6YhngUk9kyFbTzcJER0r8i1l1k2il7T-i36gX0f1f6FhWPECsgDxrsjL_IKQlBKgBUnS2TObPE3hI/s1600/N-ParamGenericFixtures.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="147" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiezbjms_Kjo7qbQ8_Tyr0kVwrEvOhXdbP9NuyWEbOHZkZgk6ZqI85OvSd6YhngUk9kyFbTzcJER0r8i1l1k2il7T-i36gX0f1f6FhWPECsgDxrsjL_IKQlBKgBUnS2TObPE3hI/s640/N-ParamGenericFixtures.png" width="640" /></a></div>
<span style="font-family: "Trebuchet MS",sans-serif;"><br /></span>
<span style="font-family: "Trebuchet MS",sans-serif;"><span style="background-color: yellow; color: red;">MSTest: NOT SUPPORTED (or I haven't found it)</span>.</span><br />
<br />
<div style="color: #660000;">
<span style="font-size: large;"><span style="font-family: "Trebuchet MS",sans-serif;">IDE-Integration</span></span></div>
<span style="font-family: "Trebuchet MS",sans-serif;">A big attraction with most MS offerings is integration; in this case, IDE integration. <span style="color: blue;">MSTest integrates out of the box. NUnit needs some assistance;</span> but if you have something like Resharper (which you should), this isn't a deal-breaker.</span><br />
<br />
<br />
<div style="color: #660000;">
<span style="font-family: "Trebuchet MS",sans-serif; font-size: large;">Speed of Execution</span></div>
<ul style="text-align: left;">
<li><span style="font-family: "Trebuchet MS",sans-serif;">MSTest: </span><span style="font-family: "Trebuchet MS",sans-serif;">This is peculiar. I used Ctrl+R, A to run all tests in solution. <span style="color: black;">It takes 4-5 s to run 3 measly tests.. as compared to NUnit which runs an identical set in 0.1s .</span> Around 4 secs elapse, before any test method's status changes - it seems it is doing some heavy lifting in "preparing" to run the tests.</span></li>
<li><span style="font-family: "Trebuchet MS",sans-serif;">NUnit: I have NUnit setup to run all my tests when the binaries change. So a successful build triggers a test run. The faster I receive the test results, the less the possibility that I pursue a distraction.</span></li>
</ul>
<br />
<span style="font-family: "Trebuchet MS",sans-serif;">Even in console mode, NUnit was faster. The cold start case takes much longer. However in subsequent runs too, NUnit took 3.5s as compared to 6s for MSTest</span><br />
<span style="color: blue; font-family: "Trebuchet MS",sans-serif;">NUnit wins the speed test.</span><br />
<span style="font-size: large;"><br /></span>
<div style="color: #660000;">
<span style="font-size: large;"><span style="font-family: "Trebuchet MS",sans-serif;">Assertion Library</span></span></div>
<span style="font-family: "Trebuchet MS",sans-serif;">NUnit has a much more evolved assertion library than MSTest. </span><span style="font-family: "Trebuchet MS",sans-serif;">Don't have a detailed comparison here. </span><br />
<span style="font-family: "Trebuchet MS",sans-serif;">Personally, I miss the <b style="background-color: yellow;">constraint based Assert Model</b> present in <a href="http://www.nunit.org/index.php?p=constraintModel&r=2.4">NUnit</a>, <a href="http://code.google.com/p/hamcrest/wiki/Tutorial">Hamcrest</a>, et.all, which I find more readable.</span><br />
<span style="font-family: "Trebuchet MS",sans-serif;">e.g. <span style="font-family: "Courier New",Courier,monospace;">Assert.That(actualObject, meets_some_constraint)</span></span><span style="font-family: "Trebuchet MS",sans-serif;"></span><br />
<br />
<div style="background-color: yellow;">
<span style="font-family: "Trebuchet MS",sans-serif;"><b>Asserting on exceptions</b></span></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhgXrf4uD_hJn_Ynzl13ySU6SKO6jl3y6-hbQOXXZvM4H_AxKCaBqKZ3AMw9g1letABjSEdXMWzenCrEEzykmsyR-o2rnP_mJdFnCoB0jyuWCL1d0G7Km82R-9MKs5DKYW3bAU9/s1600/MSTest-ExpExcep.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="209" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhgXrf4uD_hJn_Ynzl13ySU6SKO6jl3y6-hbQOXXZvM4H_AxKCaBqKZ3AMw9g1letABjSEdXMWzenCrEEzykmsyR-o2rnP_mJdFnCoB0jyuWCL1d0G7Km82R-9MKs5DKYW3bAU9/s640/MSTest-ExpExcep.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">MSTest : Expected Exception not thrown!</td></tr>
</tbody></table>
<span style="font-family: "Trebuchet MS",sans-serif;"><br /></span>
<br />
<ul style="text-align: left;">
<li><span style="font-family: "Trebuchet MS",sans-serif;"> MSTest: You can use this to state that the corresponding test should throw an exception of Type X. It also takes a message as a second parameter... which is the failure message (to be shown when the test fails) <b>NOT</b> the expected exception message. i.e. you can only check the type of exception.</span></li>
<li><span style="font-family: "Trebuchet MS",sans-serif;">NUnit: You can check the type, the ExpectedMessage (you can even specify that the actual exception message must contain/start-with/exactly-match/match-a-regex), a custom handler to verify the exception and a failure message. </span></li>
</ul>
<span style="font-family: "Trebuchet MS",sans-serif;"> </span><br />
<div>
<table cellpadding="0" cellspacing="0" class="tr-caption-container" style="float: left; margin-left: 1em; text-align: left;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhI7QoXZIIfiBMqPga3LBYPI2UncS4zkTKTRKYaA7nnDDkDHeHJ_s52aKT9WImtFniyFNzyTVcQ9o3UOwP9n_7ueFbnklLvnAi7VTFOKYb6Jo0WWixye3AQaUAP1dCArkl9RZt6/s1600/NUnit-ExpExcep.png" imageanchor="1" style="clear: left; margin-bottom: 1em; margin-left: auto; margin-right: auto;"><img border="0" height="273" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhI7QoXZIIfiBMqPga3LBYPI2UncS4zkTKTRKYaA7nnDDkDHeHJ_s52aKT9WImtFniyFNzyTVcQ9o3UOwP9n_7ueFbnklLvnAi7VTFOKYb6Jo0WWixye3AQaUAP1dCArkl9RZt6/s640/NUnit-ExpExcep.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">NUnit : Expected Exception thrown but with incorrect message</td></tr>
</tbody></table>
<span style="font-family: "Trebuchet MS",sans-serif;">However even this approach has been deprecated because you're not sure of the source of the exception, any line in the test could throw the exception and pass the test. So enter Assert.Throws where you narrow the scope down to the Connect call in the following example.</span><br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiN_jeX0XDflwliZI9DIYKYbTG68RIrooQC-SJPsnrhVSXSWTx5CwC-iJrhsFFOCQsrx_RDfqORlKxElHfotp2BIkKEDWiZ7QMt0tUy45w4QHK0o3ln9YK6zAZrYVhTjMBS9Fjc/s1600/NUnit-Throws.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="179" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiN_jeX0XDflwliZI9DIYKYbTG68RIrooQC-SJPsnrhVSXSWTx5CwC-iJrhsFFOCQsrx_RDfqORlKxElHfotp2BIkKEDWiZ7QMt0tUy45w4QHK0o3ln9YK6zAZrYVhTjMBS9Fjc/s640/NUnit-Throws.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">NUnit</td></tr>
</tbody></table>
<span style="font-family: "Trebuchet MS",sans-serif;"><span style="background-color: yellow;">MSTest does not have Assert.Throws</span> or an equivalent. This was actually <a href="http://connect.microsoft.com/VisualStudio/feedback/details/381288">a user-request in 2008</a>. Still pending in 2011. It isn't too hard to write your own version.. but then you shouldn't have to.</span><br />
<br />
<span style="font-family: "Trebuchet MS",sans-serif;"> </span></div>
</div>
<pre class="brush: csharp">public static T Throws<T>(Action codeblock) where T : Exception
{
try
{
codeblock();
}
catch(T exception)
{
return exception;
}
Assert.Fail("Did not throw an exception of type " + typeof(T));
return null;
}</pre>
<br />
<span style="font-family: "Trebuchet MS",sans-serif;">Another nitpicky example would be <span style="background-color: yellow;">the </span><b style="background-color: yellow;">CollectionAssert implementation</b>. </span><br />
<span style="font-family: "Trebuchet MS",sans-serif;">MSTest accepts ICollection for parameters instead of the more general IEnumerable. When you're comparing objects of IEnumerable<t><t><T> (LINQ anyone) or IList<T><t><t> (another core collection type), you need to create copies so that you can compare. An avoidable design choice. Also note the error messages - it's important to ***<span style="background-color: yellow;">fail informatively</span>***. Chalk this one too to NUnit.</t></t></t></t></span><br />
<pre class="brush: csharp">IEnumerable<int> enumerableX = new[] {1, 2, 3};
IEnumerable<int> enumerableY = new[] {1, 2, 4}.Concat(new[]{5});
CollectionAssert.AreEqual(enumerableX.ToList(), enumerableY.ToList(), "myError Message");
// Output:
// CollectionAssert.AreEqual failed. myError Message(Different number of elements.)
/*----------Equivalent NUnit error ---------*/
CollectionAssert.AreEqual(enumerableX, enumerableY, "myError Message");
//Output:
//NUnitTestsClient.ConnectTests.TestCollections:
// myError Message
// Expected is <...List`1[System.Int32]> with 3 elements, actual is <....List`1[System.Int32]> with 4 elements
// Values differ at index [2]
// Expected: 3
// But was: 4
</pre>
<br />
<div style="color: #660000;">
<span style="font-family: "Trebuchet MS",sans-serif; font-size: large;">Code Coverage</span></div>
<span style="font-family: "Trebuchet MS",sans-serif;">Yup. With MSTest, <span style="background-color: #d9ead3;">setting up code coverage is a snap</span>. This is <a href="http://msdn.microsoft.com/en-us/library/dd504821.aspx">how you do it</a> ; the trick is in locating the Configure button :)</span><br />
<br />
<span style="font-family: "Trebuchet MS",sans-serif;">That said, getting code coverage for NUnit tests is just a bit more of effort. Create a batch file</span><br />
<span style="font-family: "Trebuchet MS",sans-serif;"><br /></span>
<div style="font-family: "Courier New",Courier,monospace;">
<span style="font-size: small;">vsinstr /coverage MyProduction.dll </span><wbr></wbr><span style="font-size: small;"> #instrument binaries<br />vsperfcmd -start:coverage -output:WhatsThe.coverage #start profiler<br />[path to <b>nunit-console-x86</b>.exe] MyNUnitTests.dll #run tests<br />
vsperfcmd -shutdown #stop profiler</span></div>
<span style="font-family: "Trebuchet MS",sans-serif;"><br /></span>
<span style="font-family: "Trebuchet MS",sans-serif;">That creates a coverage file that you can just open up in the IDE and you get the stats and the line highlighting et. all.</span><br />
<br />
<div style="color: #660000;">
<span style="font-family: "Trebuchet MS",sans-serif; font-size: large;">Test Impact</span></div>
<span style="font-family: "Trebuchet MS",sans-serif;">MSTest <span style="background-color: #d9ead3;">can point out </span><b style="background-color: #d9ead3;">unit</b><span style="background-color: #d9ead3;"> tests that are impacted by a code change</span>, on the basis of function call chains. There is some additional work to avail this benefit (configuration and collecting baselines)</span><br />
<span style="font-family: "Trebuchet MS",sans-serif;"><span style="background-color: yellow;">NUnit : not supported</span>.</span><br />
<span style="font-family: "Trebuchet MS",sans-serif;">That said: it works only for unit tests; Personally I'd run all the unit tests but YMMV.. especially if it takes over 10-15 mins to run all of them.</span><br />
<span style="font-family: "Trebuchet MS",sans-serif;">It would have been a HUGE time-saver if it worked for acceptance tests. However at the system level, modules interact via DB, files, networks, etc even though there are no direct function calls ; so doesn't work.</span><br />
<span style="font-family: "Trebuchet MS",sans-serif;"><br /></span>
<div style="color: #660000;">
<span style="font-family: "Trebuchet MS",sans-serif; font-size: large;">Parallel execution</span></div>
<span style="font-family: "Trebuchet MS",sans-serif;"><span style="background-color: #d9ead3;">MSTest seems to be designed to enable executing unit tests in parallel</span>. Here's <a href="http://msdn.microsoft.com/en-us/library/ee921484.aspx">the how-to</a>.</span><br />
<span style="font-family: "Trebuchet MS",sans-serif;">Note: </span><br />
<ol style="text-align: left;">
<li><span style="font-family: "Trebuchet MS",sans-serif;">I had to open-and-close my solution for the change to take effect. </span></li>
<li><span style="font-family: "Trebuchet MS",sans-serif;">Also you can have either coverage or parallel execution (not both). </span></li>
</ol>
<span style="font-family: "Trebuchet MS",sans-serif;">Of course your tests need to be designed accordingly too - eliminate interference. Also would be helpful to verify that the tests are isolated/independent.</span><br />
<div>
<span style="font-family: "Trebuchet MS",sans-serif;"><br /></span>
<div>
<span style="font-family: "Trebuchet MS",sans-serif;"><span style="background-color: yellow;">NUnit : not supported..yet</span>. However there is PNUnit, an open-source mod. Also it seems likely that parallel execution would be built into the next big release of NUnit. Another approach would be running segments of your tests in parallel on multiple cores/machines.</span><br />
<div style="color: #660000;">
<br />
<span style="font-family: "Trebuchet MS",sans-serif; font-size: large;">Misc. Peeves</span><span style="font-family: "Trebuchet MS",sans-serif;"> </span></div>
<span style="font-family: "Trebuchet MS",sans-serif;">Some more tiny things that irked me with MSTest </span><br />
<ul style="text-align: left;">
<li><span style="font-family: "Trebuchet MS",sans-serif;">No hierarchical view of test cases. Flat lists everywhere.</span></li>
<li><span style="font-family: "Trebuchet MS",sans-serif;">Have to open a new "test details" window to see details of every test failure. e.g. console output. </span></li>
<li><span style="font-family: "Trebuchet MS",sans-serif;">Adds supporting files to the solution (.vsmdi, .testsettings) etc.</span></li>
<li><span style="font-family: "Trebuchet MS",sans-serif;">Cannot see console output for all tests in one place. (Useful for debugging).</span></li>
<li><span style="font-family: "Trebuchet MS",sans-serif;"> Categories can only be applied to tests. Tedious - applying the attribute in 15 places instead of once on top of the test-class. (Duplication). Filtering on categories is non-intuitive in the IDE. Try doing it without looking at <a href="http://msdn.microsoft.com/en-us/library/dd286683.aspx">the answer</a>. </span></li>
</ul>
<div style="color: #660000;">
<br /></div>
<div style="color: #660000;">
<span style="font-family: "Trebuchet MS",sans-serif; font-size: large;">Summary (IMHO):</span></div>
<div style="color: #660000;">
<span style="font-family: "Trebuchet MS",sans-serif;">So NUnit is clearly ahead on Speed, Simplicity, Assertion library, Flexibility e.g. Parameterized (data-driven) tests and informative failures. It has had the benefit of time and shows it. If you're already using it, there is no compelling reason to switch. </span></div>
<div style="color: #660000;">
<br /></div>
<div style="color: #660000;">
<span style="font-family: "Trebuchet MS",sans-serif;">MSTest has parallel execution & Test Impact as differentiators. However that is not enough to make up for its shortcomings as a unit testing framework. Less frequent releases (tied up with VS editions) and slower pace of evolution. It just has got a lot of small things wrong that make it difficult to like. If you're using TFS, then MSTest would be easier to work with than other unit testing frameworks for obvious reasons. (But then this is your smaller problem.. Sorry couldn't resist that one.)</span></div>
</div>
</div>
</div>
<hr width="80%"><p><span class="Apple-style-span" style="font-size: x-small;"><br />
<a name="1"><b>1 </b></a>You can configure the NUnit test assembly/class/method to run on a different thread via RequiresThread attribute<a href="#top1"><sup>↩</sup></a><br />
</span>Gishuhttp://www.blogger.com/profile/17616896114730114577noreply@blogger.com4tag:blogger.com,1999:blog-14744716.post-51416173043040970822011-11-09T13:02:00.000+05:302011-11-09T13:02:08.581+05:30PSR : Free Windows Screen Recording tool<div dir="ltr" style="text-align: left;" trbidi="on">
<div style="font-family: "Trebuchet MS",sans-serif;">
An unsung hero in the MS Toolkit. </div>
<div style="font-family: "Trebuchet MS",sans-serif;">
It has a weird name - <b>Problem Steps Recorder</b> that is not obvious as a search keyword. It doesn't create a video as the output, rather an HTML containing annotated images. (On the flip side, it doesn't eat up a lot of space and doesn't lose much information).</div>
<div style="font-family: "Trebuchet MS",sans-serif;">
<br /></div>
<div style="font-family: "Trebuchet MS",sans-serif;">
This one ships with Windows7. To invoke, Start > Run > psr </div>
<div style="font-family: "Trebuchet MS",sans-serif;">
This should pop the very intuitive UI. Here's a <a href="http://www.microsoft.com/download/en/details.aspx?id=22292">3 min video walkthrough</a>. I just learnt you can annotate issues as well. </div>
<div style="font-family: "Trebuchet MS",sans-serif;">
<br /></div>
<div style="font-family: "Trebuchet MS",sans-serif;">
<b>Brilliant!</b></div>
<div style="font-family: "Trebuchet MS",sans-serif;">
<br /></div>
</div>Gishuhttp://www.blogger.com/profile/17616896114730114577noreply@blogger.com0Thane, Maharashtra, India19.23184 72.96717819.17187 72.888214 19.291809999999998 73.046142tag:blogger.com,1999:blog-14744716.post-9771036932220301252011-11-08T18:12:00.002+05:302011-11-08T18:23:35.099+05:30Review : Parasoft DotTest Static Code Analysis<div dir="ltr" style="text-align: left;" trbidi="on">
<div style="font-family: "Trebuchet MS",sans-serif;">
Everyone agrees that static code analysis is good. However it isn't in widespread use. It has become easier with FXCop / Code Analysis built into VS2010. Personally I'm for "educating developers" over SCA. </div>
<div style="font-family: "Trebuchet MS",sans-serif;">
<b>Focus on </b><b><span style="font-family: "Trebuchet MS",sans-serif;">Prevention over Post-facto Inspection, just like Bugs. </span></b></div>
<ul style="text-align: left;">
<li><b><span style="font-family: "Trebuchet MS",sans-serif;">Make sure that the number of warnings reduce over time.</span></b></li>
<li><b><span style="font-family: "Trebuchet MS",sans-serif;">Make sure that the code isn't being engulfed by green suppression markups. </span></b><span style="font-family: "Trebuchet MS",sans-serif;">Refine the set of rules </span><b><span style="font-family: "Trebuchet MS",sans-serif;"></span></b>as needed.</li>
</ul>
<div style="font-family: "Trebuchet MS",sans-serif;">
<br /></div>
<div style="font-family: "Trebuchet MS",sans-serif;">
Cannot stress this enough, To avoid SCA pain (drowning in a stormy sea of SCA warnings) START EARLY.</div>
<div style="font-family: "Trebuchet MS",sans-serif;">
<br /></div>
<div style="font-family: "Trebuchet MS",sans-serif;">
I was asked to come up with a good set of SCA rules for acceptance test code. Here's the specific tool I'm looking at - <a href="http://www.parasoft.com/jsp/products/dottest.jsp?itemId=135">Parasoft dotTest</a> (I'm using v9.1). </div>
<div style="font-family: "Trebuchet MS",sans-serif;">
<br /></div>
<div style="font-family: "Trebuchet MS",sans-serif;">
Setup is easy.. Liked the following aspects.</div>
<ul style="text-align: left;">
<li style="font-family: "Trebuchet MS",sans-serif;">You can install into a separate profile i.e. avoid messing up your default VS2010 setup. </li>
<li><span style="font-family: "Trebuchet MS",sans-serif;">You can get a floating network license that can be shared across users. You need to check "Release automatically when idle" in the License options.</span></li>
</ul>
<div style="font-family: "Trebuchet MS",sans-serif;">
<br /></div>
<span style="font-family: "Trebuchet MS",sans-serif;">So I created mine by creating a new configuration.</span><br />
<div style="font-family: "Courier New",Courier,monospace;">
Main Menu - Parasoft - Test Configurations... Click New</div>
<br />
<span style="font-family: "Trebuchet MS",sans-serif;">Adding categories one at a time by cross-verifying it with a medium sized codebase ~80 files / ~8K LOC. Took about 2 mins. The rules can be found under the "Static" tab on the right, when your configuration is selected. </span><br />
<ul style="text-align: left;">
<li><span style="font-family: "Trebuchet MS",sans-serif;">TIP! The search box can be used to filter the hierarchical view below.</span></li>
<li><span style="font-family: "Trebuchet MS",sans-serif;">TIP! Right click on a specific rule to bring up its documentation. Or parameterize the rule if that is supported. </span></li>
</ul>
<span style="font-family: "Trebuchet MS",sans-serif;">You can export the configuration then to a file that can be shared across your team.</span><br />
<span style="font-family: "Trebuchet MS",sans-serif;"> </span><br />
<span style="font-family: "Trebuchet MS",sans-serif;"><br /></span><br />
<span style="font-family: "Trebuchet MS",sans-serif;">Since I was looking for SCA on test code (which is run within a controlled env), I turned off the following categories</span><br />
<ul style="text-align: left;">
<li><span style="font-family: "Trebuchet MS",sans-serif;">COM Guidelines</span></li>
<li><span style="font-family: "Trebuchet MS",sans-serif;">Casting guidelines </span></li>
<li><span style="font-family: "Trebuchet MS",sans-serif;">Implementing Finalize and Dispose</span></li>
<li><span style="font-family: "Trebuchet MS",sans-serif;">Security Inspection</span></li>
<li><span style="font-family: "Trebuchet MS",sans-serif;">Serialization Guidelines</span></li>
<li><span style="font-family: "Trebuchet MS",sans-serif;">Security Policy Rules</span></li>
<li><span style="font-family: "Trebuchet MS",sans-serif;">WPF</span></li>
</ul>
<span style="font-family: "Trebuchet MS",sans-serif;">That accounts for 96 / 444 rules. Next the rules that didn't make the cut (Rule Id in brackets, use search/filter to find a rule.)</span><br />
<ul style="text-align: left;">
<li style="color: blue;"><span style="font-family: "Trebuchet MS",sans-serif;">Keep line length within predefined parameters (80) [BRM.MLL-3] </span></li>
<ul>
<li><span style="font-family: "Trebuchet MS",sans-serif;">Includes whitespaces on the left - in short indentation causes rules to cause noise warnings. Unusable.</span></li>
</ul>
<li style="color: purple;"><span style="font-family: "Trebuchet MS",sans-serif;"><span style="color: blue;">Avoid class, struct, or interface names which are more than 14 characters long [BRM.LONGNAMES-4]</span> </span></li>
<ul>
<li><span style="font-family: "Trebuchet MS",sans-serif;">Default limit too short. Cannot customize. I prefer long readable names over this warning. </span></li>
</ul>
<li style="color: blue;"><span style="font-family: "Trebuchet MS",sans-serif;">Always provide appropriate file header (copyright information, etc.) [BRM.FILEHEADER-3] </span></li>
<ul>
<li><span style="font-family: "Trebuchet MS",sans-serif;">Needs a comment to be the first line in the file - even before the using statements. <b>Does not recognize standard XML comments.</b></span></li>
<li><span style="font-family: "Trebuchet MS",sans-serif;"><b>Does not exclude non-public classes.</b> Noise<b> </b> </span></li>
</ul>
<li style="color: blue;"><span style="font-family: "Trebuchet MS",sans-serif;">Avoid unused private methods [CMUG.MU.AUPM-2] </span></li>
<ul>
<li><span style="font-family: "Trebuchet MS",sans-serif;">Noise warnings. Flagged methods that were being called.</span></li>
</ul>
<li><span style="font-family: "Trebuchet MS",sans-serif;"><span style="color: blue;">Avoid using get accessor with side effects [CMUG.PRU.AGAS-3]</span> </span></li>
<ul>
<li><span style="font-family: "Trebuchet MS",sans-serif;">Flagged all properties that were internally calling String.Format. Noise.</span></li>
</ul>
<li style="color: blue;"><span style="font-family: "Trebuchet MS",sans-serif;">Use an array instead of many parameters [CMUG.MU.AIMP-5]</span></li>
<ul>
<li><span style="font-family: "Trebuchet MS",sans-serif;">Noise again. Recommends turning get(string elementId, string keyPropertyName, string valuePropertyName) to take a string array instead of well named parameters. Counter-productive.</span></li>
</ul>
<li style="color: blue;"><span style="font-family: "Trebuchet MS",sans-serif;">Make events public [CMUG.EVU.MEP-5] </span></li>
<ul>
<li><span style="font-family: "Trebuchet MS",sans-serif;">No explanation given. Doesn't seem to be necessary - <a href="http://msdn.microsoft.com/en-us/library/ms229011.aspx">Event Design Page MSDN</a>.</span></li>
</ul>
<li style="color: blue;"><span style="font-family: "Trebuchet MS",sans-serif;">Put using statements in alphabetical order [CS.USO-4]</span></li>
<ul>
<li><span style="font-family: "Trebuchet MS",sans-serif;">VS2010 sorts usings such that <a href="http://msdn.microsoft.com/en-us/library/bb514113.aspx">System.* namespaces are grouped above the rest</a>.<br />This rule doesn't seem to know about this. A workaround is to override this behavior in VS2010 but then that's something that each developer would have to change.</span></li>
</ul>
<li><span style="font-family: "Trebuchet MS",sans-serif;">NAMING</span></li>
<ul>
<li><span style="font-family: "Trebuchet MS",sans-serif;"><span style="color: blue;">Avoid language-specific names for members [NG.ATNC.ALSN.MEMBER-5]</span> : flags up "_constaints" for containing "int"</span></li>
<li><span style="font-family: "Trebuchet MS",sans-serif;"><span style="color: blue;">Avoid method names that conflict with keywords [NG.WC.KEYWORD.METHOD-3]</span> : Will not let you have methods called Get, Stop, Select - which are sometimes the most intuitive names.</span></li>
<li><span style="font-family: "Trebuchet MS",sans-serif;"><span style="color: blue;">Use Pascal case for class names [NG.CAPSTY.PASCAL.CLASS-5]</span> : UIDriverBase is a valid name as per Design guidelines but flagged up by this rule. See <a href="http://msdn.microsoft.com/en-us/library/ms229043.aspx">Capitalization Rules for Acronyms</a>.</span></li>
<li><span style="font-family: "Trebuchet MS",sans-serif;"><span style="color: blue;">Use the property's type name as part of the property name [NG.PRN.APNCTN-4]</span> : This is a "Consider" not a "Do" in the guidelines. Embedding type name in the property is risky w.r.t. future refactorings (e.g. if you change the type of the property with name MonitorIPString from String to StringBuilder. Misleading.</span></li>
<li><span style="font-family: "Trebuchet MS",sans-serif;"><span style="color: blue;">Use the suffix 'EventHandler' for event handler names [NG.EVN.EHNEEH-3]</span> : Broken. Here's a sample - Message: Change type of event 'X.Driver+AppLaunchedEventHander AppLaunched' or add 'EventHandler' suffix to type 'Driver+AppLaunchedEventHander'.</span></li>
<li><span style="font-family: "Trebuchet MS",sans-serif;"><span style="color: blue;">Use flags attribute for bit field enum [NG.ETN.UFABFE-3]</span> : Flagged simple 1 member enum with non-plural name as bit-field enums. Broken.</span><span style="font-family: "Trebuchet MS",sans-serif;"> </span></li>
</ul>
<li style="color: blue;"><span style="font-family: "Trebuchet MS",sans-serif;">Avoid too many function calls from a single function [OOM.FCSF-4] </span></li>
<ul>
<li><span style="font-family: "Trebuchet MS",sans-serif;">Limit of 10 is easily breached. Non-customizable. Includes property access within the method.</span></li>
</ul>
<li><span style="font-family: "Trebuchet MS",sans-serif;"><span style="color: blue;">Use 'GetType()' in the 'Equals()' method implementation [PB.EQL-3]</span> </span></li>
<ul>
<li><span style="font-family: "Trebuchet MS",sans-serif;">Does not exclude overloads of Equals. e.g. Equals(MyType t) does not need to check type of t. </span></li>
</ul>
<li><span style="font-family: "Trebuchet MS",sans-serif;"><span style="color: blue;">Use upper case for method name with name length <=2 [NG.CAPSTY.UPPER.MN-5], [NG.CAPSTY.UPPER.PROP-5]</span></span></li>
<ul>
<li><span style="font-family: "Trebuchet MS",sans-serif;">Asks me to rename a method called At(params) to AT(params). Was not able to find any design guideline to support this.</span></li>
</ul>
<li><span style="font-family: "Trebuchet MS",sans-serif;"><span style="color: blue;">Avoid public default constructor when a non-default constructor is present [CMUG.CU.PUBDEFAULTCTOR-3]</span> </span></li>
<ul>
<li><span style="font-family: "Trebuchet MS",sans-serif;">Contradicts <span style="color: blue;">Exceptions should implement common constructors [ERR.EICC-4]</span>. The exception rule won.</span></li>
</ul>
<li><span style="font-family: "Trebuchet MS",sans-serif;"><span style="color: blue;">Follow the limit for Number Of Static Methods [METRICS.NOSM-1]</span> </span></li>
<ul>
<li><span style="font-family: "Trebuchet MS",sans-serif;">This one flagged a false +ve. Static Class had 1 static var, 3 public and 3 private methods. Rule violated: 10 is not less than 10. Refactoring often leads to lots of small private methods.</span></li>
</ul>
</ul>
<span style="font-family: "Trebuchet MS",sans-serif;">Then there was a bunch that </span><br />
<ul style="text-align: left;">
<li><span style="font-family: "Trebuchet MS",sans-serif;">was auto-enforced by C# e.g. Name indexer 'item' or Name event accessors 'add' and 'remove'. </span></li>
<li><span style="font-family: "Trebuchet MS",sans-serif;">Some were guidelines not rules. </span></li>
<li><span style="font-family: "Trebuchet MS",sans-serif;">Some were plain out of date e.g. <span style="color: blue;">Provide the same accessibility for the property and its accessors [CLS.ACAC-3] <span style="color: black;"></span></span>This flags up a common pattern of properties whose setters are private. The explanation points to an outdated (.Net 1.1) version. We've have come a long way since then... </span></li>
</ul>
<span style="font-family: "Trebuchet MS",sans-serif;">Turned off another 80-100 too. Finally ended up with <b>196 / 444</b> rules enabled.</span><br />
<span style="font-family: "Trebuchet MS",sans-serif;">There is also a report out and code-metrics at the end too. But to summarize, </span><span style="font-family: "Trebuchet MS",sans-serif;">I found Parasoft dotTest to be a bit unpolished.. the infrastructure is there. </span><b><span style="font-family: "Trebuchet MS",sans-serif;">Analysis is quick enough but t</span></b><span style="font-family: "Trebuchet MS",sans-serif;"><b>he rules need to be improved. </b>Not enough documentation online - except from Parasoft itself. </span><br />
<br />
<span style="font-family: "Trebuchet MS",sans-serif; font-size: large;">Verdict: Passable, but for now prefer Code Analysis that comes bundled with Visual Studio 2010.</span><br />
<span style="font-family: "Trebuchet MS",sans-serif;"><br /></span><br />
<span style="font-family: "Trebuchet MS",sans-serif;"><br /></span></div>Gishuhttp://www.blogger.com/profile/17616896114730114577noreply@blogger.com0Thane, Maharashtra, India19.23184 72.96717819.17187 72.888214 19.291809999999998 73.046142tag:blogger.com,1999:blog-14744716.post-35679290678697036582011-09-15T12:59:00.001+05:302011-09-19T15:01:35.039+05:30String Calculator Kata : My first screencast trilogy<div dir="ltr" style="text-align: left;" trbidi="on">
<span style="font-family: "Trebuchet MS",sans-serif;">This is my attempt at Roy Osherove's popular String Calculator Katas. </span><br />
<br />
<iframe allowfullscreen="" frameborder="0" height="169" src="http://player.vimeo.com/video/27977192?title=0&byline=0&portrait=0" webkitallowfullscreen="" width="300"></iframe><br />
<a href="http://vimeo.com/27977192">TDD StringCalculator Kata - Part1</a> from <a href="http://vimeo.com/gishu">Gishu Pillai</a> on <a href="http://vimeo.com/">Vimeo</a>.<br />
<br />
<span style="font-size: large;"><br /></span><br />
<span style="font-family: "Trebuchet MS",sans-serif; font-size: large;"><a href="http://vimeo.com/album/1693891">The Vimeo Album containing all 3 parts.</a></span><br />
<br />
<span style="font-family: "Trebuchet MS",sans-serif;">I'm using C# (VS2010+Resharper), NUnit and an IDE extension called <a href="https://code.google.com/p/gishu-util/wiki/Beacons">Beacons</a> </span><br />
<span style="font-family: "Trebuchet MS",sans-serif;">(the wiki might be a bit behind the downloads tab :))</span><br />
<br />
<br />
<span style="font-family: "Trebuchet MS",sans-serif;">The videos look best when you have the HD option turned on and Scaling (hover on the video - top-right) turned off.</span><br />
<b><span style="font-family: "Trebuchet MS",sans-serif;">intermediate practitioners</span></b><br />
<span style="font-family: "Trebuchet MS",sans-serif;">For best results : download the videos and <a href="http://goo.gl/JShDQ">the subtitles</a>. Use your media player with audio turned off and subtitles enabled at 2-4X. </span><br />
<span style="font-family: "Trebuchet MS",sans-serif;">I don't do this (ever) so wasn't equipped with the proper audio gear (and coupled with the fact that I do not have an internet voice and can't code and speak at the same time, the audio is passable.)</span><br />
<br />
<b><span style="font-family: "Trebuchet MS",sans-serif;">beginners</span></b><br />
<span style="font-family: "Trebuchet MS",sans-serif;">Practice it a couple of times before watching. You'll get more out of it.</span><br />
<span style="font-family: "Trebuchet MS",sans-serif;"><br /></span><br />
<span style="font-family: "Trebuchet MS",sans-serif;">What I learned from it:</span><br />
<ul style="text-align: left;">
<li><span style="font-family: "Trebuchet MS",sans-serif;">I need to practice more :) </span></li>
<li><span style="font-family: "Trebuchet MS",sans-serif;">2 modes of learning in a Kata </span></li>
<ul>
<li><span style="font-family: "Trebuchet MS",sans-serif;">muscle memory - goal: Speed. </span></li>
<li><span style="font-family: "Trebuchet MS",sans-serif;">exploration - goal: Possible solutions. Take different choices and see how it ends up.</span></li>
</ul>
<li><span style="font-family: "Trebuchet MS",sans-serif;">Automated refactoring (e.g. Resharper for C#) rules. Knowing your IDE and keyboard shortcuts goes a long way w.r.t. productivity</span></li>
<li><span style="font-family: "Trebuchet MS",sans-serif;">Streamline all routine work and bind them to keyboard shortcuts - like building, running tests, reusable code snippets/templates, etc</span></li>
</ul>
<span style="font-family: "Trebuchet MS",sans-serif; font-size: x-small;">For people looking for screencasting tools on windows, I'd recommend</span><br />
<ol style="text-align: left;">
<li><span style="font-family: "Trebuchet MS",sans-serif; font-size: x-small;"><a href="http://www.bbsoftware.co.uk/BBFlashBack_FreePlayer.aspx">BB Flashback Express</a> - the free edition for screencasting. Export to AVI using ffdshow worked great for me.</span></li>
<li><span style="font-family: "Trebuchet MS",sans-serif; font-size: x-small;"><a href="http://www.donationcoder.com/Software/Skrommel/index.html#ShowOff">ShowOff</a> - for highlighting keystrokes and chords. Keyboard Jedi wasn't too great with 64bit & keyboard chords (before my patience ran out).</span></li>
</ol>
<div>
<span style="font-family: "Trebuchet MS",sans-serif;"> </span><span style="font-family: "Trebuchet MS",sans-serif;"> </span></div>
</div>
Gishuhttp://www.blogger.com/profile/17616896114730114577noreply@blogger.com0tag:blogger.com,1999:blog-14744716.post-10393264856033460082011-07-30T19:49:00.006+05:302011-07-31T17:06:15.962+05:30The A Team<span style="font-family:trebuchet ms;">Contd from </span><a style="font-family: trebuchet ms;" href="http://madcoderspeak.blogspot.com/2011/07/towards-better-acceptance-test.html">the previous post</a><span style="font-family:trebuchet ms;">..</span><br /><br /><span style="font-family:trebuchet ms;">So how do we succeed?</span><br /><span style="font-size:130%;"><span style="font-family:trebuchet ms;">Lock up Enemy #1 - Accidental Complexity</span></span><br /><br /><span style="color: rgb(0, 102, 0); font-family:trebuchet ms;font-size:130%;" >Empower teams to choose</span><br /><br /><span style="font-family:trebuchet ms;">All projects / teams are not the same. Different needs, different tools. This may be difficult in BigCo especially if the tools have already been bought. But make some noise - stand up for your tools, the "users" need to channel some feedback upstream to the "purchasers". </span><br /><ul style="font-family: trebuchet ms;"><li>Explore options instead of resigning to the golden hammer. Prefer tools that don't get in your way. Ones that don't require you to learn yet another proprietary language. Ability to write extensions in your team's native language is a +. This also opens avenues for developers to assist with automation work, if required.<br /></li><li>Use existing tools instead of writing your own - they're likely to be functional, tried and tested</li><li>Avoid putting all eggs in one basket. Keep tools/test-runners swappable by defining layers. To migrate to a different test runner, you should only need to migrate your thin tests layer which calls into an automation layer below it, which does most of the heavy lifting. More on this soon..</li></ul><span style="color: rgb(0, 102, 0); font-family:trebuchet ms;font-size:130%;" >Collaboration</span><br /><span style="font-family:trebuchet ms;">forms a reinforcing loop against Silos/Specialists. Increasing collaboration decreases opportunities for specialization, which in turn facilitates more collaboration. Of course, the reverse is also true - Silos can finish off collaboration. Only one shall survive, you just have to choose.</span><br /><br /><span style="color: rgb(0, 102, 0);font-size:130%;" ><span style="font-family:trebuchet ms;">Outside-in / Test-first + Wishful Thinking</span></span><br /><span style="font-family:trebuchet ms;">If you've tried the XP Practice of TDD, you'd know the liberating feeling of letting the unit tests drive the design of the production code. You fix the end-goal, you make it work, make it clean and repeat.</span><br /><span style="font-family:trebuchet ms;">Starting with the test prevents any bias (arising from implementation details, existing tools at your disposal, etc.) </span><br /><br /><span style="font-family:trebuchet ms;">ATDD is the corresponding practice at the system level. However it is not an easy practice to quickly latch on to. So work towards it in baby steps</span><br /><span style="font-family:trebuchet ms;">For starters, concentrate on a DONE definition + writing tests first (before implementation) from the </span><span style="font-weight: bold; font-family:trebuchet ms;" >users' perspective</span><span style="font-family:trebuchet ms;"> on a piece of paper. Make sure everyone has the same idea of DONE before you start the iteration.<br />As the team matures, you can even move up to ATDP (from the BDD world) where you write test before or during iteration planning & use them for estimation.</span><br /><br /><span style="color: rgb(0, 102, 0);font-size:130%;" ><span style="font-family:trebuchet ms;">WHAT over HOW</span></span><br /><span style="font-family:trebuchet ms;">Ensures that the test is at the right level of abstraction (the ol' forest over trees adage). It makes the tests shorter and readable. It also works beautifully to bring out the intent (as opposed to the implementation) of the test.</span><br /><span style="font-family:trebuchet ms;">Specify the bare minimum ; things that are relevant to the test at hand.. all other details need to be out of sight.</span><br /><br /><span style="color: rgb(0, 102, 0);font-size:130%;" ><span style="font-family:trebuchet ms;">Stable DSL for testing</span></span><br /><span style="font-family:trebuchet ms;">You employ wishful thinking to imagine the ideal interface you'd like the system to expose for testing. Since the tests are another client to the system, they can also drive beneficial design changes. The tests stand-in for real users, so if the system is difficult to consume for the tests it follows that it might be for the users too. You could start with a plain C# interface to begin with and then work your way up to a custom DSL. It</span><br /><ul style="font-family: trebuchet ms;"><li>abstracts away incidental details like the GUI, underlying technology and other implementation details.</li><li>abstracts away the tools used for automation from the tests.<br /></li><li>decouples the people interested in writing tests from the automation personnel. This allows both to play to their strengths and offers the best of both worlds. e.g. the testers could define the automation interface changes for the sprint and the developers could implement them with production-code like quality.<br /></li><li>makes it easy to write new tests with relatively little boot-up time. Writing a test then is just combining the reusable building blocks offered by the test DSL. The tests layer is a good training ground for new hires.<br /></li></ul><span style="font-family:trebuchet ms;">Imagine (wishful) a robot that will operate the system for you and think of the commands that you'd issue to the robot. The set of commands are your starting point.</span><br /><span style="font-family:trebuchet ms;">e.g. robot.DoX(params) or robot.GetY()</span><br /><br /><span style="color: rgb(0, 102, 0);font-size:130%;" ><span style="font-family:trebuchet ms;">Programming Skills</span></span><br /><span style="font-family:trebuchet ms;">Automation is programming. Without good programming techniques and discipline, sustainable pace would be difficult.<br />This means you need to raise the bar for automation personnel and/or leverage devs. If the team lacks the skill-set required, take countermeasures... Training, get some experts onboard, etc. The average skill level of the team can also be increased by frequent pairing. </span><br /><br /><span style="color: rgb(0, 102, 0);font-size:130%;" ><span style="font-family:trebuchet ms;">Refactoring</span></span><br /><span style="font-family:trebuchet ms;">Your #1 weapon against complexity. Beck's 4 rules for simple design, the techniques from the Refactoring book (Martin Fowler) + the SOLID principles are a must-read. Top that off with an introductory text on programming (e.g. Clean Code - Robert Martin) and you should be good to go.<br /><br /></span><span style="color: rgb(0, 102, 0);font-size:130%;" ><span style="font-family:trebuchet ms;">Good Naming & Discoverable Design</span></span><br /><span style="font-family:trebuchet ms;">Taking the time to pick good names goes a long long way. Good names make it easy to find things, facilitate understanding, help zone in on a specific area to change & reduce duplication</span><br /><span style="font-family:trebuchet ms;">This also helps in being able to discover the design / API using just the IDE (learn by intellisense) and programmer intuition. Choose names that are likely to be searched. Operate by the principle of least surprise (code that works as expected the first time around); Avoid hidden side-effects. Document and use team conventions to preserve consistency.</span><br /><br /><span style="color: rgb(0, 102, 0);font-size:130%;" ><span style="font-family:trebuchet ms;">Communicate Intent / Distill the essence</span></span><br /><span style="font-family:trebuchet ms;">This takes WhatOverHow to the next level. Explaining Why e.g. by extracting another "coarse" method to move up one level OR differentiating sets of inputs by using explanatory names. This reduces the test further to the essence - where the tests turn into readable system documentation... the kind that never gets out of date.</span><br /><br /><span style="color: rgb(0, 102, 0);font-size:130%;" ><span style="font-family:trebuchet ms;">Learning Curve</span></span><br /><span style="font-family:trebuchet ms;">Refactoring well and often keeps accidental complexity down to manageable levels. The supporting cast of Pairing, a discoverable design, intention-revealing code and a good testing DSL make it easy for new team members to learn the ropes.</span><br /><span style="font-family:trebuchet ms;">This inhibits cargo-cult behavior and the changes made are deliberate/intentional rather than hopeful. Another source of complexity wanes.</span><br /><br /><span style="font-size:180%;"><span style="font-family:trebuchet ms;">Test Maintenance - the last frontier</span></span><br /><span style="font-family:trebuchet ms;">Test Maintenance like complexity can be minimized not eliminated. As complexity decreases, maintenance effort reduces too.</span><br /><span style="font-family:trebuchet ms;">The testDSL makes it possible to write-and-test the blocks once & use anywhere. Simple designs (no duplication, intention-revealing code, minimal classes) make maintenance much easier.</span><br /><br /><span style="font-family:trebuchet ms;">Transitively, the cost of automation goes down as well.</span><br /><br /><span style="font-family:trebuchet ms;">Let's refactor our diagram to remove the accidental nodes and edges and things get clearer now. Refactoring code is even more rewarding.</span><br /><br /><a href="https://picasaweb.google.com/lh/photo/1bFaBnlkrguab-rrPMUmQPcaujkF9KLSFFuJboHAXZ8?feat=embedwebsite"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhSxNLTTilZith7vJpFmzM7FIn2siUYxjLfMraa_kjc8uCsxCrNAnoAWCT9_YIX22iuOGVFbMokAldUlFT3ks5OQ1K_3M6PbxX5B5YMWydSgF0FkyR2Nu3fKCYixqSVb5a6bxtk/s800/CostOfAutomation-Refactored.png" height="663" width="800" /></a>Gishuhttp://www.blogger.com/profile/17616896114730114577noreply@blogger.com4tag:blogger.com,1999:blog-14744716.post-77281679588989341822011-07-30T11:24:00.019+05:302011-07-31T16:48:46.180+05:30Towards better acceptance test automation...This started out as a sketch of a <a href="http://en.wikipedia.org/wiki/Causal_loop_diagram">Causal Loop Diagram</a> (CLD) for bad acceptance test suites... and then it got away from me :) <span style="font-size:100%;">The black arrows indicate "+ve links" e.g. Duplication and Complexity increase/decrease together. The blue arrows ending in dots indicate "-ve links" e.g. Refactoring and Duplication work against each other. Increase in Refactoring causes decrease in duplication.</span><br />Click on it to be able to zoom in.<br /><br /><a href="https://picasaweb.google.com/lh/photo/jF3yC3k6_C-WybIub7zog_caujkF9KLSFFuJboHAXZ8?feat=embedwebsite"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgY15MvR1l65RB65yGhhk9fLMGAUluxlKZXqAIlRuuiXEhNFGXGekYKHUCr8LyBl9dz6ehb5fNyWWjWKBgMlt4rWb7bC06rlHNCcQ6WjcfoXoTx8Ro_dc1klhPf9qaJ7n0haR9C/s800/CostOfAutomation.png" height="637" width="800" /></a><br /><br /><a name='more'></a><br /><br />And now the thousand err... few thousand words that correspond to that picture :)<br /><br />I'll open with the worst-kept secret: Automation is Code. You're asking the computer to perform a set of steps to check something. That sounds like... programming. (There is also considerable overlap with unit testing.) If you have built a few non-trivial systems, you'd admit that the main challenges are minimizing accidental complexity and maximizing communication (with other humans).<br /><br /><span style="font-size:180%;">All roads lead to accidental complexity. </span><br />Complexity is unavoidable for non-trivial programs. Essential complexity is that which is absolutely required to solve the problem at hand. Accidental is everything else (e.g. complexity introduced by the tools or language or algorithm used). The art is in identifying accidental complexity and minimizing it.<br /><br />If we just concentrate (really hard enough) at the corresponding node in the center, we see the following inward black arrows (causes).<br /><br /><span style="font-size:130%;">Wrong choice of Tools</span><br />You shouldn't use a hammer to unscrew a nail although it might work given the right level of persistence and resistance to pain. Sadly there are a lot of expensive 'golden hammers' being forced on QA teams. A bad tool can<br /><ul><li>get in your way.<br /></li><li>coerce you to go against your better judgement. e.g. forces you to write repeated boilerplate code with no facility for extracting reusable blocks; even though you know 'Duplication is Evil'. You own the tools; don't let it be like the other way around.<br /></li><li>lock you in . It is prohibitively expensive to switch. The vendor can also choose to charge for enhancements and fleece you with their well-thought out pricing and support models<br /></li><li>refuse to play with other tools. Often this leads to the need for inventing wheels (in-house) ... which end up kinda square. Meanwhile the other folks using those round existing (open-source?) wheels are racing ahead</li><li>can breed specialists (for lack of a better term. <a href="http://testobsessed.com/2010/07/19/why-test-automation-costs-too-much/">Thanks Elizabeth H.</a>)</li><li>drag in other relatives (tools from the same vendor). Go to bullet 1.<br /></li></ul><span style="font-size:130%;">Emergence of the "Elite" / specialists / silos</span><br />Cumbersome tools can raise certain specialists to an elite status. These are the only people who dare to touch the codebase ; no one else understands it / knows how to use the tools or run the tests. All the automation work funnels through them. They become a difficult bottleneck limiting team throughput. Imaginary walls pop up, they may have illusions of grandeur and work in isolation unquestioned. Now you have a knowledge silo in the team (god forbid they leave or fall sick or take a vacation :).<br />Worse as the diagram shows, they form a <span style="font-weight: bold;">reinforcing loop</span> with accidental complexity. The complexity precludes the non-elite from making changes reinforcing the specialist status. The specialists continue to add complexity unchecked... widening the gap further.<br /><br /><span style="font-size:130%;">Cargo-Cults</span><br />If this is the first time you've come across this term, read <a href="http://blogs.msdn.com/b/ericlippert/archive/2004/03/01/82168.aspx">Eric Lippert's excellent post</a>.<br />They form another <span style="font-weight: bold;">reinforcing loop</span> with complexity. Lots of cult members furiously spot-fixing without understanding the design or "Why did that work?" leads to the design turning opaque. Loss of structure leads to <a href="http://www.laputan.org/mud/mud.html">BBOMs</a>, which breed the next generation of clueless fixers.<br /><br /><span style="font-size:130%;">Speculative Architecture</span><br />Over-engineering is a leading cause of accidental complexity. Designing for future needs isn't as rampant as on the development side of the fence where the architects live.<br />Reuse is best when extracted out of 2 or more functioning clients - designing for reuse inside-out usually leads to abstractions that make all clients mad.<br />Intricate interfaces/classes/abstractions with non-intuitive names all add to the complexity and lead to a steep learning curve for new team members. Not sure who I am quoting here but - "you really can't use what you do not understand" so make sure that the code is within the grasp of the team supposed to work on it.<br /><br /><span style="font-size:130%;">Duplication</span><br />It is the easiest problem to fix and somehow the most widespread code smell. Changing something in one place is a breeze.. changing it in 47 places identically is a mistake-prone chore. Doing this every few weeks is ... inviting bugs over.<br />Duplication forms another <span style="font-weight: bold;">reinforcing loop</span> with accidental complexity. Greater the duplication, greater the confusion / complexity. As the complexity goes up, it can be harder to find the right way/spot reusable pieces. People write their own copy - duplication goes up!<br /><span style="font-size:130%;"><br />Test-After</span><br />This is a throw-back from the way things used to be. Programmers code till they are happy or time runs out. Throw it over the wall to QA. QA finds bugs, throws it back over. This continues for a while.<br /><br />Cut to year 2011. The wall has been torn down. We can't wait till the development team is done to begin with automation. There won't be enough time. Even if you manage with some end-of-sprint heroics, you're likely to miss out on exploratory / attribute testing. The automation isn't likely to be the best either - in fact it is most probably going to be "the first thing that worked" (TM) considering the pressure and not wanting to be the ones that let the team down. The bugs - they live! (and yes, they're all out of bubblegum)<br /><br />Test-after tests are usually long granular only-machine-readable scripts that over-specify and coupled to the GUI to boot! After about 4 sprints of heroics, the team is tired and complexity is taking the elevator to the top! Stop and change something.. the next sprint is NOT going to be better.<br /><br /><span style="font-size:180%;">The heavies : Wrong Tests, Rework, Bad tests, Maintenance, $$$.</span><br /><span style="font-size:130%;">Wrong tests</span><br />They're usually a symptom of lack of communication/collaboration.<br />It's bad enough that the team built and wrong thing, you also expended a significant amount time & money in automating it & other testing. Now you got to do it all over again - Rework. That is resources could have been used to give the customer more features. There is no excuse for getting the tests wrong.<br />Eliminate middle-men & barriers in the communication path. Get the trinity of the customer+tester+developer to start talking! If the customer doesn't have the time to explain things to the team, quantify the implications in monetary terms. You will have someone's attention.<br /><br /><span style="font-size:130%;">Bad tests</span><br />Write a failing test, how hard can that be? But what if I say write a failing test that is automated, thorough, repeatable, independent, precise and employs the best programming ideas? And did I mention it has to run as FAST as possible.<br />It takes effort, programming skill and discipline. Expecting historically non-coding testers to jump this chasm on their own in a short-time frame is crazy.<br /><br />Getting this wrong can lead to tests that are<br /><ul><li>Spotty - the tests aren't thorough. As a result, bugs keep coming through the net of automated tests</li><li>Repeatable - the tests give you different results each time you run it. Given enough time, this can irreparably damage trust in the tests & consequently the testers. The tests turn into another hurdle for the development team.<br /></li><li>Dependent - the tests have insider information about each other. Run it alone - it fails. Run the whole suite - everything passes. So to check one of them, you need to run and wait for a whole bunch of them.<br /></li><li>Fragile - the existing tests keep breaking all the time. A tiny flutter in a remote corner of the system causes loads of tests to break.</li><li>Slow - "this thing usually takes 2 mins to comeback. But sometimes it takes upto 3. Let's Thread.Sleep for 5 in the test just to be safe." So "usually" you're twiddling thumbs for 3min/test... and for 100+ tests, that comes to 5 hours of waiting. What's the way out? (Hint: No. The solution is not to run it at night.)</li><li>Over-specification - the tests are a cure for insomnia. KLoC after KLoC of prescriptions for the computer. </li><li>Messy - It's code. Possibilities are endless... especially if you factor in some constant deadline pressure and no training<br /></li></ul><span style="color: rgb(153, 0, 0); font-weight: bold;">A bad test always comes back! </span><br /><br /><span style="font-size:130%;">Maintenance and Cost of Automation</span><br />Maintaining a bad codebase is only better than maintaining bad tests. No one wants to babysit a tantrum throwing CI build server every day.<br /><br />Automation costs real money. All "accidental maintenance" is money lost. Since the end-goal is to make money, it follows that automation has to have ROI. It has to pull its weight. If not it might take the team down with it.<br /><br /><br /><span style="font-size:180%;">Is there any good news at all in this post?</span><br />No but <a href="http://madcoderspeak.blogspot.com/2011/07/a-team.html">the next one</a> does :)Gishuhttp://www.blogger.com/profile/17616896114730114577noreply@blogger.com0tag:blogger.com,1999:blog-14744716.post-53984315208011037672011-07-01T10:58:00.006+05:302011-07-30T11:24:06.296+05:30Automated tests != Free Lunch<span style=";font-family:verdana;font-size:85%;" >Disclaimer: I'm a proponent of XP and truly believe it has made me a much better programmer. This post just aims to let readers see through the fog-of-agile caused due to data-transfer loss as it passes from person to person. Please do not misinterpret this as an anti-agile rant. I'm just saying it doesn't work out always unless you're willing to put in the effort to make the change.</span><br /><br />Legend<br /><span style="color: rgb(0, 102, 0);font-family:arial;" >What you hear… (good) the Promised Land</span><br /><ul><li><span style="font-family:trebuchet ms;">What was left unsaid (bad.. or downright ugly)</span></li></ul><br /><span style="color: rgb(0, 102, 0);font-family:arial;" >You have an automated regression-safety net to make future changes. Make a change, Push a button and you will know if you broke anything. CI can provide near instant-feedback. Better confidence. </span> <ul style="font-family: trebuchet ms;"><li>You have 2-3X the code to maintain. If you have people who don’t care/are too busy/ not passionate about code quality and writing good tests, the tests are the first to put a chokehold on your productivity. Bad tests are as good as (or possibly worse than) having no tests. You could see yourself in a situation where a section of your team is permanently siphoned to keeping the build/tests green. This turns into a daily bottleneck. <span style="font-weight: bold;">Tests need to be self-checking, thorough, readable, professionally written, independent, repeatable, concise and FAST. All this takes effort!</span></li></ul><br /><span style="color: rgb(0, 102, 0);font-family:arial;" >Documentation - the tests can be "live specs" of the application - They never get out of date like documentation.</span><br /><ul style="font-family: trebuchet ms;"><li>It takes a significant level of discipline and skill to write readable spec-tests. An essential skill :to see the What and Why without getting entangled in the How. Most teams get this wrong... without noticing it.<br /></li><li>Sidenote: The practice of ignoring failing tests is criminal (but usually not punished accordingly) and can lead to misleading specs.</li></ul><br /><br /><span style="color: rgb(0, 102, 0);font-family:arial;" >Quality & Productivity: Leads to high-quality production code. Fewer Bugs. More features added / unit time (because you spend less time in debugging and manual testing)</span><br /><span style="font-family:trebuchet ms;"><br /><ul><li>IF you let the tests drive/shape your design (ATDD and TDD). Client-first design is an unstated requirement.</li><li>The quality of the code is usually a direct reflection of the people writing it. This means you need craftsmen (> 30-50% of a team) and NOT armies of cargo-cult programmers.</li><li>If you're using automated tests exclusively for regression (or getting your 'agile badge'), you'll slowly grind to a halt. Writing tests for "untestable blobs implemented from a non-negotiable handed-down paper design" is frustrating. People can be stuck on “how do I test this?” – Usually leads to silent trade-offs & non-thorough test which will let in bugs and put you in the net negative w.r.t. productivity.</li></ul></span><br /><br /><span style="color: rgb(0, 102, 0);font-family:arial;" >Less rework/thrashing: The dialogue / conversation (that you have with the customer to come up with the acceptance tests) makes it highly likely that you’re building the right thing..</span><br /><span style="font-family:trebuchet ms;"><ul><li>Assumes that the customers want to collaborate with the developers and testers. This is not often true.. Real users are sometimes real hard-to-find. Even if you manage to snag one of them, you can only procure a small slice of their time. Real users rarely want to write tests.</li><br /><li>If the customers give a “vision” and delegate the responsibility of mapping them to executable specs to the technical team (or worse the QA/testers), you still run the risk of “This is not what I asked for” late in the game. Regular demos may help shorten the feedback time.. but you may still waste an iteration.The magic potion here is collaboration and conversation.. the tests are just a beneficial byproduct.</li></ul></span><br /><br /><span style="color: rgb(0, 102, 0);font-family:arial;" >Simple: Red-Green-Refactor. How hard can that be?</span><br /><span style="font-family:trebuchet ms;"><ul><li>Sounds simple.. but is deceptive. True OO is a minority sport. Refactoring is a skill that you need to work on. Off the job practice is mandatory.<br />You may need to "hire" a good coach for an extended period (I'd say 6 months-1 release) to get the team rolling. Spot trainings/Just-in-time learning won't work for most teams. </li></ul></span>Gishuhttp://www.blogger.com/profile/17616896114730114577noreply@blogger.com3tag:blogger.com,1999:blog-14744716.post-50641264773074036852011-04-03T10:18:00.008+05:302011-04-03T14:10:09.537+05:30Enterprise Library Logging vs Log4Net for a rolling file log : jumpstart and a showdown!I was looking for a rolling file log and couldn't make my mind between the two. So I embarked on a small quest to see which one is easier to learn and work with.<br /><br /><a name='more'></a><br />First up I'm gonna create a VS2010 Console application.<br /><br /><span style="font-size:130%;">Enterprise Library</span><br />Ok first up we need to download the latest drop of EnterpriseLibrary which is currently <a href="http://msdn.microsoft.com/en-us/library/ff632023.aspx">5.0</a>. Run through the install.<br />Strafe the web a bit. Okay.. we need an app.config file in our project with some XML muck. Thankfully MS has a GUI tool to help us write it - it's named EntLibConfig.NET4.exe and lies in the EntLib install folder. Open it up and we can either create/load an app.config file.<br /><br /><ol><li>Main Menu > Blocks > Add Logging Block</li><li>An Event logging Listener is added by default (2nd column). Delete it and add a Rolling Flat File Trace Listener - click the Plus sign next to 'Logging Target Listeners' caption.</li><li>Next go on a property-setting spree. It should end up like this</li></ol><br /><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgH6V6Mr3IU374t1f_ALtglB_g_jrkmgOu0lHtVmHTehtTH1qJIONV7mJHtYSjuKiYuovADyIJVJHTSSVQsFq8VddlZ0nKQEfHgiExVYau0pFHaiyqF2-_EHEob9DsjsSuFgjUU/s800/entLib-Config.png"><img style="display:block; margin:0px auto 10px; text-align:center;cursor:pointer; cursor:hand;width: 800px; height: 538px;" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgH6V6Mr3IU374t1f_ALtglB_g_jrkmgOu0lHtVmHTehtTH1qJIONV7mJHtYSjuKiYuovADyIJVJHTSSVQsFq8VddlZ0nKQEfHgiExVYau0pFHaiyqF2-_EHEob9DsjsSuFgjUU/s800/entLib-Config.png" alt="" border="0" /></a>We'll work our way right to left.<br />Third Column : TextFormatter.<br /><ul><li>Template = {timestamp(local)} : {severity} - {message}.</li></ul>The ellipses button opens up a editor which has a dropdown with all the special placeholder incantations to insert all sorts of informative bits into your log message. The default template by the way is pretty detailed for a default.<br /><br />Second Column : Rolling flat file trace listener<br /><ul><li>File Name -> rolling-with-entlib.log</li><li>Formatter - choose TextFormatter (which is present by default. Else add one in the third column via the context menu.) and you should see a link between the trace listener and the formatter.</li><li>Max Archive Files - set to 2. Older messages will be sent to an archive file.</li><li>Message Footer, Header - clear out.</li><li>Roll Interval -> None</li><li>Roll Size KB -> 200</li></ul>First column : Logging category.<br />By default, a 'General' logging category is added. I'll stick to that for simplicity (but you could add more of your own). Configure the Listeners property of the category to point to the rolling flat file listener. You should see another link.<br />We need one more link (the last I promise) to make the config valid. Open up the 'Logging errors & warnings' and make it point to the same listener. Another link appears and we're done. Save the file.<br /><br />Jump back into the IDE. Add a reference to the Microsoft.Practices.EnterpriseLibrary.Logging from the .NET tab. Write ourselves a new PSVM method with<br /><br /><pre class="csharpcode">Logger.Write(<span class="str">"Dilshan flings himself for the catch to send Kohli back!"</span>, <span class="str">"General"</span>, 0, 0, TraceEventType.Critical);<br />Logger.Write(<span class="str">"MS Dhoni walks in to steady the innings."</span>, <span class="str">"General"</span>, 0, 0, TraceEventType.Information);<br /></pre><br /><span style="font-size:130%;">Log4Net</span><br />Looks like we need <a href="http://logging.apache.org/log4net/download.html">the binaries</a> and ... yup.. some more XML. But there's no GUI to help us this time around. However there is a kind soul on the internet writing tutorials <a href="http://www.beefycode.com/post/Log4Net-Tutorial-pt-1-Getting-Started.aspx">at BeefyCode.com</a>. Two thumbs way up! By Part 5 I think I know all I need... for now.<br /><br /><pre class="csharpcode"> <span class="kwrd"><</span><span class="html">configSections</span><span class="kwrd">></span><br /> <span class="kwrd"><</span><span class="html">section</span> <span class="attr">name</span><span class="kwrd">="log4net"</span> <span class="attr">type</span><span class="kwrd">="log4net.Config.Log4NetConfigurationSectionHandler, log4net"</span><span class="kwrd">/></span><br /><span class="kwrd"></</span><span class="html">configSections</span><span class="kwrd">></span><br /><span class="kwrd"><</span><span class="html">log4net</span><span class="kwrd">></span><br /><br /><span class="kwrd"><</span><span class="html">appender</span> <span class="attr">name</span><span class="kwrd">="RollingFileAppender"</span> <span class="attr">type</span><span class="kwrd">="log4net.Appender.RollingFileAppender"</span><span class="kwrd">></span><br /><span class="kwrd"><</span><span class="html">layout</span> <span class="attr">type</span><span class="kwrd">="log4net.Layout.PatternLayout"</span><span class="kwrd">></span><br /> <span class="kwrd"><</span><span class="html">conversionPattern</span> <span class="attr">value</span><span class="kwrd">="%date %-5level- %message%newline"</span><span class="kwrd">/></span><br /><span class="kwrd"></</span><span class="html">layout</span><span class="kwrd">></span><br /><span class="kwrd"><</span><span class="html">file</span> <span class="attr">value</span><span class="kwrd">="rolling-with-log4net.log"</span> <span class="kwrd">/></span><br /><span class="kwrd"><</span><span class="html">appendToFile</span> <span class="attr">value</span><span class="kwrd">="true"</span> <span class="kwrd">/></span><br /><span class="kwrd"><</span><span class="html">rollingStyle</span> <span class="attr">value</span><span class="kwrd">="Size"</span> <span class="kwrd">/></span><br /><span class="kwrd"><</span><span class="html">maxSizeRollBackups</span> <span class="attr">value</span><span class="kwrd">="1"</span> <span class="kwrd">/></span><br /><span class="kwrd"><</span><span class="html">maximumFileSize</span> <span class="attr">value</span><span class="kwrd">="200KB"</span> <span class="kwrd">/></span><br /><span class="kwrd"><</span><span class="html">staticLogFileName</span> <span class="attr">value</span><span class="kwrd">="true"</span> <span class="kwrd">/></span><br /><span class="kwrd"></</span><span class="html">appender</span><span class="kwrd">></span><br /><span class="kwrd"><</span><span class="html">root</span><span class="kwrd">></span><br /><span class="kwrd"><</span><span class="html">level</span> <span class="attr">value</span><span class="kwrd">="ALL"</span><span class="kwrd">/></span><br /><span class="kwrd"><</span><span class="html">appender-ref</span> <span class="attr">ref</span><span class="kwrd">="RollingFileAppender"</span><span class="kwrd">/></span><br /><span class="kwrd"></</span><span class="html">root</span><span class="kwrd">></span><br /><br /><span class="kwrd"></</span><span class="html">log4net</span><span class="kwrd">></span></pre>A quick summary: Log4Net has appenders (instead of EntLib's trace listeners) whose properties need to be set. A root node controls the level of logging + a reference to the appender to use. The PatternLayout node controls how the message is logged (akin to EntLib's formatter template property) with the <a href="http://www.beefycode.com/post/Log4Net-Tutorial-pt-4-Layouts-and-Patterns.aspx">magic placeholders</a>. That's it.<br /><br />Back to the IDE. Add a reference to log4net.dll.<br /><br /><pre class="csharpcode">log4net.Config.XmlConfigurator.Configure();<br />var logger = log4net.LogManager.GetLogger(<span class="kwrd">typeof</span>(Program)); // Program is the name of my entry point class<br /><br />logger.Error(<span class="str">"Dilshan flings himself for the catch to send Kohli back!"</span>);<br />logger.Info(<span class="str">"MS Dhoni walks in to steady the innings."</span>);<br /></pre>That's all there is to it. I had to change the project's target framework to ".Net Fwk 4" for the build to succeed. The client profile doesn't have System.Web that log4net needs for some reason.<br /><br />Here's how it looks in the log files (I added a running counter into the message)<br /><pre>REM This is Log4Net *********<br />2011-04-03 10:30:52,871 ERROR- #499577 Dilshan flings himself to send Kohli back!<br />2011-04-03 10:30:52,911 INFO - #499577 MS Dhoni walks in to steady the innings.<br /><br />REM and here is EntLib *********<br />4/3/2011 10:29:44 AM : Information - #499773 MS Dhoni walks in to steady the innings.<br />4/3/2011 10:29:44 AM : Critical - #499774 Dilshan flings himself to send Kohli back!</pre><br /><span style="font-size:130%;">Showdown!</span><br />Things that I noted from this limited experiment:<br /><ul><li>EntLib has a nice config tool. Log4net could have one too but I couldn't find any. To be fair, the xml muck for log4net is simpler to write by hand and shorter by about 30%.</li><li>It's easy to tweak the amount of logging via an xml file edit. You tweak the 'Minimum Severity property for the Logging category in EntLib5.0 ; the root/level element's value attribute in Log4Net.</li><li>Log4Net is 3x-5x faster from the rolling file burst logging tests that I did. Although it probably doesn't matter for most purposes.</li><li>Log4Net also has a smaller assembly footprint - you just need one DLL. EntLib will drag in 4 dependencies ; additional EntLib DLLs.<br /></li></ul>logger.Info("We just won the ICC World Cup! Yay!! Logging out...")Gishuhttp://www.blogger.com/profile/17616896114730114577noreply@blogger.com2tag:blogger.com,1999:blog-14744716.post-29034276545351086542011-02-01T08:25:00.015+05:302011-02-05T12:13:05.041+05:30Flash Cards Kata and the transformation priority premise<span style=";font-family:trebuchet ms;font-size:130%;" ><span style="font-size:100%;">I began the new year by resolving to complete the <a href="http://www.gigamonkeys.com/book/">PCL (</a></span></span><a href="http://www.gigamonkeys.com/book/"><span style=";font-size:130%;" ><span style="font-size:100%;">Peter Siebel's) </span></span></a><span style=";font-family:trebuchet ms;font-size:130%;" ><span style="font-size:100%;"><a href="http://www.gigamonkeys.com/book/">book</a>. Around the same time, another <a href="http://cleancoder.posterous.com/the-transformation-priority-premise">interesting post by Uncle Bob</a> appeared on the WWW. A discussion on the TDD yahoogroups spurred me to try a Kata - two-sided Flash Cards.<br /><br />Looked easy enough to test my new-found Lisp skills. Although I was not consciously choosing the next test / step based on the transformation priority, I was curious to know if I had been doing it all along without knowing it.<br /><br />This innocuous looking kata had me stuck initially - what is the right first step? Trash-canned the first couple of attempts. The third attempt was slow but I succeeded. I did it again in C# just to see if the language had any bearing on my method. This time I was faster because I was familiar with the trail now.<br /><br /></span></span><ol><li><span style=";font-family:trebuchet ms;font-size:130%;" ><span style="font-size:100%;">I am not wiser with regard to the priority premise - I found I used <span style="font-style: italic;">a bunch of transformations that were not part</span><span> (bold)</span><span style="font-style: italic;"> of the original list </span><span>(shown in blue below)</span><span style="font-style: italic;">. As a result, their priorities are unknown</span>. This was the proverbial rain on my parade.</span></span></li><li><span style=";font-family:trebuchet ms;font-size:130%;" ><span style="font-size:100%;">Also in the C# variant, I found a couple of tests where I needed <span style="font-style: italic;">multiple transformations to get to the next green</span> & I could not figure out a smaller test.</span></span></li></ol><span style=";font-family:trebuchet ms;font-size:130%;" ><span style="font-size:85%;">Note: Check the "speaker notes" in case things get dense especially with the Lisp one. Use the menu button at the bottom-right to get to the presentation link<br /></span><br />Attempt#1 Flash Cards Kata in Lisp</span><br /><iframe src="https://docs.google.com/present/embed?id=dcx49m86_81gv5kcnf3&interval=5&size=l" frameborder="0" height="559" width="700"></iframe><br /><span style="font-family:trebuchet ms;"> TEST User should be asked a question</span><ul><li><span style="font-family:trebuchet ms;"> <span style="font-weight: bold;">{} => statement</span></span></li></ul><span style="font-family:trebuchet ms;">TEST </span><span style="font-family:trebuchet ms;">A right ans should be confirmed</span><ul><li><span style="font-family:trebuchet ms;"> <span style="font-style: italic; font-weight: bold;"><span style="color: rgb(0, 0, 153);">P5 (statement->statements) adding more unconditional statements</span>.</span></span></li></ul><span style="font-family:trebuchet ms;">TEST </span><span style="font-family:trebuchet ms;">A wrong ans should be corrected</span><ul><li><span style="font-family:trebuchet ms;"> <span style="font-weight: bold; font-style: italic; color: rgb(0, 0, 153);">P6 (unconditional->if) splitting the execution path</span></span></li></ul><span style="font-family:trebuchet ms;">TEST </span><span style="font-family:trebuchet ms;">Diff card and a right ans</span><ul><li><span style="font-family:trebuchet ms;"> <span style="font-weight: bold; font-style: italic; color: rgb(0, 0, 153);">P4 (constant->scalar) replacing a constant with a variable or an argument</span></span></li></ul><span style="font-family:trebuchet ms;">TEST </span><span style="font-family:trebuchet ms;">Diff card and a wrong ans</span><ul><li><span style="font-family:trebuchet ms;"> <span style="font-weight: bold;">constant => expr</span></span></li></ul><span style="font-family:trebuchet ms;">TEST </span><span style="font-family:trebuchet ms;">Multiple cards</span><ul><li><span style="font-family:trebuchet ms;"> <span style="font-weight: bold;">{} => loop (statement)</span></span></li></ul><span style="font-family:trebuchet ms;">TEST </span><span style="font-family:trebuchet ms;">Answers are case insensitive</span><ul><li><span style="font-family:trebuchet ms;"> <span style="font-weight: bold;">exprX => exprY</span></span></li></ul><span style="font-family:trebuchet ms;">TEST </span><span style="font-family:trebuchet ms;">Answers can have leading/trailing whitespace</span><ul><li><span style="font-family:trebuchet ms;"> <span style="font-weight: bold;">value => expr</span></span></li></ul> <span style=";font-family:trebuchet ms;font-size:130%;" >Attempt#2 Flash Cards Kata in C#</span><iframe src="https://docs.google.com/present/embed?id=dcx49m86_828sp82df8&interval=5&size=l" frameborder="0" height="559" width="700"></iframe><br /><span style="font-family:trebuchet ms;">TEST </span><span style="font-family:trebuchet ms;">User is asked the question</span><ul><li><span style="font-family:trebuchet ms;"> <span style="font-weight: bold; font-style: italic; color: rgb(0, 0, 153);">P2 nil -> constant</span></span></li></ul><span style="font-family:trebuchet ms;">TEST </span><span style="font-family:trebuchet ms;">User is asked another question (triangulate)</span><ul><li><span style="font-family:trebuchet ms;"> <span style="font-weight: bold;">{} => statement</span> *** Play()</span></li><li><span style="font-family:trebuchet ms;"> <span style="font-weight: bold;">constant => field</span> *** LastQuestionAsked</span></li><li><span style="font-family:trebuchet ms;"> <span style="font-weight: bold;">void => statement</span> *** Ask() - is there a smaller step here?</span></li></ul><span style="font-family:trebuchet ms;">TEST </span><span style="font-family:trebuchet ms;">A correct ans is confirmed</span><ul><li><span style="font-family:trebuchet ms;"> <span style="font-weight: bold; font-style: italic; color: rgb(0, 0, 153);">P2 nil -> constant</span></span></li></ul><span style="font-family:trebuchet ms;">TEST </span><span style="font-family:trebuchet ms;">A wrong ans is corrected</span><ul><li><span style="font-family:trebuchet ms;"> <span style="font-weight: bold; font-style: italic; color: rgb(0, 0, 153);">P5 (unconditional->if) splitting the execution path</span> *** Play()</span></li><li><span style="font-family:trebuchet ms;"> <span style="font-weight: bold; font-style: italic; color: rgb(0, 0, 153);">P4 (statement->statements) adding more unconditional statements</span></span></li><li><span style="font-family:trebuchet ms;"> <span style="font-weight: bold;">nil => expr</span> *** User.WhoAnswers</span></li><li><span style="font-family:trebuchet ms;"> <span style="font-weight: bold;">field assignment</span> *** User.ctor</span></li><li><span style="font-family:trebuchet ms;"> <span style="font-weight: bold;">constant => field</span> *** User.LastConfirmationReceived</span></li><li><span style="font-family:trebuchet ms;"> <span style="font-weight: bold;">void => statement</span> *** User.Notify</span></li></ul><span style="font-family:trebuchet ms;">TEST </span><span style="font-family:trebuchet ms;">A diff question and a right answer</span><ul><li><span style="font-family:trebuchet ms;"> <span style="font-weight: bold; font-style: italic; color: rgb(0, 0, 153);">P4 constant => scalar arg</span></span></li></ul><span style="font-family:trebuchet ms;">TEST </span><span style="font-family:trebuchet ms;">A diff question and a wrong ans</span><ul><li><span style="font-family:trebuchet ms;"> <span style="font-weight: bold;">constant => expression</span></span></li></ul><span style="font-family:trebuchet ms;">TEST </span><span style="font-family:trebuchet ms;">Two cards</span><ul><li><span style="font-family:trebuchet ms;"> <span style="font-weight: bold;">{} => statements</span></span></li></ul><span style="font-family:trebuchet ms;">TEST </span><span style="font-family:trebuchet ms;">Multiple cards</span><ul><li><span style="font-family:trebuchet ms;"> <span style="font-weight: bold;">{} => loop</span></span></li><li><span style="font-family:trebuchet ms;"> <span style="font-weight: bold; font-style: italic; color: rgb(0, 0, 153);">P4 (statement->statements) adding more unconditional statements</span> *** User</span></li><li><span style="font-family:trebuchet ms;"> <span style="font-weight: bold;">field assignment</span> *** User.ctor</span></li></ul><span style="font-family:trebuchet ms;">TEST </span><span style="font-family:trebuchet ms;">Case insensitive answers</span><ul><li><span style="font-family:trebuchet ms;"> <span style="font-weight: bold;">exprX => exprY</span></span></li></ul><span style="font-family:trebuchet ms;">TEST </span><span style="font-family:trebuchet ms;">Trim leading/trailing spaces</span><ul><li><span style="font-family:trebuchet ms;"> <span style="font-weight: bold;">value => expr</span></span></li></ul>Gishuhttp://www.blogger.com/profile/17616896114730114577noreply@blogger.com0tag:blogger.com,1999:blog-14744716.post-40995715148718998572010-12-15T12:18:00.004+05:302010-12-15T12:47:50.091+05:30The way of the unusual architect - Dan NorthAn <a href="http://www.infoq.com/presentations/Simplicity-Architect">enlightening talk</a> on 'how the human mind works and its corollaries on how programmers work' from the wise Dan North - founder-leader of the BDD camp..<br /><br /><blockquote>...and so you go from something that was fit for purpose but a little bit ugly to something that never quite delivers. Who here has been involved in what I call a 2-year rewrite anti-pattern ? So we go to the business and we say<br /><br />Ok.We will rewrite out entire system for some period of 2 years and then everything will be good.<br /><span style="color: rgb(0, 0, 153);">'Wait hang on. What will we get after 2 years?' </span><br />Exactly what you've got now.<br /><span style="color: rgb(0, 0, 153);">'Right... and during that 2 years..' </span><br />Oh we're locked down! We can't do anything else for you.<br /><span style="color: rgb(0, 0, 153);">'Ok. So you're telling us that we should pay you for 2 years to go no further than we are now and you won't be able to do anything else'. </span><br />No.. that's pretty much it.<br /><span style="color: rgb(0, 0, 153);">'Okay...'</span><br /><br />and we call that enterprise architecture.<br /></blockquote><br />He had me there 3 mins into the talk. RecommendedGishuhttp://www.blogger.com/profile/17616896114730114577noreply@blogger.com0tag:blogger.com,1999:blog-14744716.post-805068669264219072010-12-04T02:45:00.002+05:302011-09-15T12:39:22.040+05:30Beacons - a VS 2010 Extension for TDD<div dir="ltr" style="text-align: left;" trbidi="on">
<div style="font-family: "Trebuchet MS",sans-serif;">
Just uploaded an early version of a TDD visualization extension that I've been working on in my free time. Get an early version of Beacons from the google code project <a href="http://code.google.com/p/gishu-util/wiki/Beacons">here</a></div>
<br />
<br />
<div style="font-family: "Trebuchet MS",sans-serif;">
The extension only works with NUnit currently. You specify a file path to your NUnit test results file path (which is normally a TestResults.xml file in your .dll/.nunit folder) and you're all set to go.<br />
It can't listen to Resharper test-runner yet. (yet.. if I can figure out how to tap in, should be not an issue.)</div>
<br />
<div style="font-family: "Trebuchet MS",sans-serif;">
<br />
To see it in action, check out this video - switch to fullscreen HD<br />
<br />
<iframe allowfullscreen="" frameborder="0" height="450" src="http://player.vimeo.com/video/27977192?title=0&byline=0&portrait=0" webkitallowfullscreen="" width="800"></iframe><br />
<a href="http://vimeo.com/27977192">TDD StringCalculator Kata - Part1</a> from <a href="http://vimeo.com/gishu">Gishu Pillai</a> on <a href="http://vimeo.com/">Vimeo</a>.<br />
<ul>
<li>Shift+Alt+B, H - Change hats (adding behavior hat OR refactoring hat)</li>
<li>Shift+Alt+B, P - Play/Pause (when you need to take a break)</li>
</ul>
If you give it a fair try, I'd love to hear from you. Particularly I'm interested in finding patterns (so share end-of-session screenshots)<br />
More ideas, comments, suggestions - feedback of any kind is welcome.. Post away at <a href="http://groups.google.com/group/beacons-tdd">http://groups.google.com/group/beacons-tdd</a></div>
</div>
Gishuhttp://www.blogger.com/profile/17616896114730114577noreply@blogger.com0Thane, Maharashtra, India19.194459565404916 72.9849243164062518.870224065404916 72.518005316406246 19.518695065404916 73.451843316406254tag:blogger.com,1999:blog-14744716.post-55249399291796931012010-11-23T17:26:00.005+05:302010-11-23T17:39:47.387+05:30ScenarioLibrary, LibraryTables with Slim Scenarios : Avoid fixture method explosion !Assuming: working knowledge of Fitnesse.<br /><br />I've been recommending <a href="http://fitnesse.org/FitNesse.UserGuide.SliM.ScenarioTable">Slim Scenarios</a> for a while (primarily because they are the Fitnesse equivalent of reusable blocks (methods/functions in your favourite programming language) i.e. tables that can be called from other tables. {Actually they are akin to macro expansion.. but that's nitpicking!)<br /><br />However soon I ran into a problem, the scenario steps translate into method calls onto the backing fixture class.<br />e.g.<br /><br /><!-- code formatted by http://manoli.net/csharpformat/ --><br /><pre class="csharpcode"><br />|scenario |play movie|pathToMovie|<span class="kwrd">on</span> home theatre|<br />|enter |HDMI1 |mode <span class="kwrd">on</span> tv |<br />|enter |DTV/CBL |mode <span class="kwrd">on</span> avr |<br />|<span class="kwrd">set</span> avr volume <span class="kwrd">to</span>|-25 |dB |<br />|navigate menu <span class="kwrd">to</span> |@pathToMovie |<br />|press |play |<span class="kwrd">on</span> media player |</pre><br /><br />Now whenever I call this scenario, the fixture methods 'enterModeOnTv', 'enterModeOnAvr', etc. need to be present on the fixture class. So if Fixtures A, B, c call this scenario, All of them need these methods on them. Bummer!<br /><br />Now the initial response was to extract these methods onto a helper class. Still I'd need to write some delegating methods on each fixture to delegate to the extracted class. So I raise my hand, admit my ignorance and post to the fitnesse forums. The experts answer in a flash!<br /><br />The magic answer is the <a href="http://fitnesse.org/FitNesse.UserGuide.SliM.LibraryTable">Library Table</a><br /><br /><span style="font-weight:bold;">FrontPage.HomeTheatreTests.SetUp</span><br /><pre class="csharpcode"><br />|library |<br />|television |<br />|avr |<br />|media player|</pre><br /><br />The Library table specifies a list of class that should be checked in case the fixture does not contain the required method. So in this case, the order in which the classes would be checked would be [Fixture > MediaPlayer > Avr > Television]. If a matching method is found in any of these classes, the step is resolved.. No more delegating methods!<br /><br /><span style="font-weight:bold;">This allows me to have cohesive helper classes that contain related methods which can be mixed into any page.</span><br /><br />Another notable mention is <a href="http://fitnesse.org/FitNesse.UserGuide.SpecialPages">ScenarioLibrary</a> pages - these are pages which are libraries of usable scenarios.<br /><br /><span style="font-weight:bold;">FrontPage.HomeTheatreTests.ScenarioLibrary</span><br /><br /><pre class="csharpcode"><br />|scenario |play movie|pathToMovie|<span class="kwrd">on</span> home theatre|<br />|enter |HDMI1 |mode <span class="kwrd">on</span> tv |<br />|enter |DTV/CBL |mode <span class="kwrd">on</span> avr |<br />|<span class="kwrd">set</span> avr volume <span class="kwrd">to</span>|-25 |dB |<br />|navigate menu <span class="kwrd">to</span> |@pathToMovie |<br />|press |play |<span class="kwrd">on</span> media player |<br /><br />|scenario|<span class="kwrd">start</span> home theatre|<br />|power |<span class="kwrd">on</span> |media player |<br />|power |<span class="kwrd">on</span> |avr |<br />|power |<span class="kwrd">on</span> |tv |<br />...<br /><br /></pre><br /><br />ScenarioLibrary pages are included if they are a brother/uncle of the current page. Also all ScenarioLibrary pages up the hierarchy are included (allowing the possibility of overriding)<br /><br />All this leads to very small test pages and it all works!<br /><br /><span style="font-weight:bold;">FrontPage.HomeTheatreTests.CanWorkWhileWatchingMovie</span><br /><br /><pre class="csharpcode"><br /># ScenarioLibrary page <span class="kwrd">is</span> included (helper slim scenario definitions)<br /># <span class="kwrd">Next</span> SetUp page <span class="kwrd">is</span> included (imports <span class="kwrd">and</span> library tables)<br /><br />|script |me |<br />|<span class="kwrd">start</span> home theatre |<br />|play movie|<span class="kwrd">File</span> Manager\USB\C\2 Watch\Iron Man 2|<span class="kwrd">on</span> home theatre|<br />|<span class="kwrd">start</span> working |<br />|<span class="kwrd">after</span> |30 |mins |<br />|ensure |<span class="kwrd">work</span> <span class="kwrd">is</span> done |<br />|press |stop |<span class="kwrd">on</span> media player|<br />|shut down home theatre |<br /><br /></pre><br /><br />In this example, the fixture Me only needs to have methods that pertain to me working. All methods related to the TV, AVR and MediaPlayer are resolved by the corresponding classes in the Fixture.dll due to the Library table.<br /><pre class="csharpcode"><br /><span class="kwrd">public</span> <span class="kwrd">class</span> Me<br /> {<br /> <span class="kwrd">private</span> <span class="kwrd">bool</span> _workIsDone ... <br /><br /> <span class="kwrd">public</span> <span class="kwrd">void</span> startWorking() ...<br /> <span class="kwrd">public</span> <span class="kwrd">void</span> afterMins(<span class="kwrd">int</span> timeInMinutes) ...<br /> <span class="kwrd">public</span> <span class="kwrd">bool</span> workIsDone ...<br /> }</pre><br />Pretty cool!Gishuhttp://www.blogger.com/profile/17616896114730114577noreply@blogger.com0tag:blogger.com,1999:blog-14744716.post-5245540589171773732010-11-12T14:39:00.005+05:302010-11-15T10:48:44.034+05:30What do I use for acceptance tests (ATDD) : NUnit vs Fitnesse ?Lately I've had some teams in my getting-agile ecosystem debating on which tool to use for acceptance testing. Now my first response is always "It depends" (You really can't go wrong with that. :) ). <br />
<br />
<span style="font-size: x-small;">Notation: I'm gonna use</span><br />
<ul><li><span style="font-size: x-small;">NUnit when I actually mean any member of the xUnit family of unit-testing frameworks. </span></li>
<li><span style="font-size: x-small;">Fit/Fitnesse stands for itself and now Slim/Fitnesse.</span></li>
<li><span style="font-size: x-small;">Customer means the XP's Customer role or Scrum's Product Owner role.</span></li>
</ul><br />
Fit-Fitnesse was the de-facto tool for automated acceptance tests till sometime ago.. Around 2009, there were <a href="http://ericlefevre.net/wordpress/2009/03/06/is-fit-dead-a-debate-on-twitter/">strong murmurs</a> that Fitnesse isn't working that well (read as some teams weren't executing very well and Fitnesse was part of their toolkit ).<br />
<br />
So what does it "depend" on ??<br />
<a name='more'></a><br />
<br />
<div style="color: #660000;"><span style="font-size: large;">Customer Collaboration: Probability of Non-developers reading/modifying individual tests</span></div><ul><li>Is your customer willing/able to collaborate with the team over a piece of paper to extract examples/scenarios for a particular use-case ? </li>
<li>Are customers interested in creating truth-tables/spreadsheets of 'how this rule should work' or story descriptions of 'how the ideal system should work'? </li>
<li>Do Customers / QA-personnel want to add/modify tests (without dev support) to improve their understanding / the system? Is the team using a shared medium to come up with more tests? </li>
</ul>If yes, choose Fit-Fitnesse... that is what it was created for. Collaboration.<br />
Alternate Scenario: Developers are given a bunch of requirements and are responsible for conversion to executable tests. Customers/Proxies are available over email to answer questions during ear-marked hours. Test-list generation is done offline (read without an active customer) followed by a verbal/email review. The transformation to executable tests has no participation from the customer. Acceptance tests are being used primarily for closing stories (definition of done) and regression. If a Pass/Fail result is all you need, NUnit might be a better choice.<br />
<br />
<div style="color: #660000;"><span style="font-size: large;">Positive Pressure on Test readability</span></div>As-intended use of Fitnesse, causes test authors to express intent before implementation.. a wiki page is like a blank of paper - wishful thinking ahead! This causes the pages to read like scripts or a sequence of intent-revealing steps with intermediate checks/asserts. If the team is struggling to create readable-n-maintainable tests, use fitnesse as a guardrail.<br />
You could do the exact same thing with NUnit. You just need to see the world from the users' perspective and a professional programmer's instincts for refactoring and simplicity. A blank test is as good as a blank wiki page. Flesh out the step-methods later. A <b>common trait (of developers</b>) is to <b>jump the gun to implementation </b>details at great harm to readability. Curb your enthusiasm and no reason why it shouldn't work<br />
<br />
<div style="color: #660000;"><span style="font-size: large;">Deployment Footprint</span></div>Fitnesse offers <b>a Web interface to easily view/modify/run tests from the comfort of your favourite browser</b>. Point to your fitnesse server URL and voila! Zero install. You can even do 'What if' analysis without much assistance. (e.g. What if number of concurrent clients is raised from 100 to 150)<br />
NUnit can't do it on its own. You would need a CI System to offer the web dashboard to execute/view results. Viewing the internals of a test is still difficult. You would need an IDE and 'coding skills' to modify the tests for any what-if analysis. Of course, if the development team is the only one interacting with the tests, this is a moot point.<br />
<br />
<div style="color: #660000;"><span style="font-size: large;">Homesickness</span></div>If dev >> testers, then it might be easier to teach the testers NUnit - in crunch time, free devs can pitch in to help the test teams get to the finish line. Most developers already know NUnit; Getting all of them to learn another tool for acceptance tests could be a drag (atleast in the short term). Personally (Disclaimer: more dev/NUnit hours than QA/Fitnesse) I find that Fitnesse has <b>a longer learning curve</b> than NUnit (different table styles, reusing sections of a page, etc) before you can use it effectively. NUnit also wins hands down in the <b>refactoring </b>department (moving HTML around can get tiresome very soon to a person used to a statically typed language + a good IDE). Stuff like 'attaching a debugger to a failing test', 'how do I get my pages into source control', 'feedback on progress of test execution' with Fitnesse might restrain/annoy speed-addicted xUnit-developers. NUnit also has <b>a richer assertion library</b>, which I missed during my time in Fitnesse.<br />
On the other hand, if you have customers and testers who can 'converse in fitnesse' and have a few projects under their belt, don't get in their way. Testers can get homesick too. Go with Fitnesse and pair Devs with Testers to diffuse Fitnesse in the team.<br />
<br />
<br />
<div style="color: #274e13;"><span style="font-size: large;">End-game</span></div>Both are tools - <b>it is how your team executes with these tools</b>, that makes the difference. To quote Ron Jeffries, "people can screw up everything" ; that doesn't reflect badly on the tool they used. <br />
It is <b>a reversible decision</b>. If you <b>design your tests to be independent of the test runner</b>, you can swap out the existing runner at any time. e.g. the idea below (see image) that I've had some success with (for rich desktop clients). Maybe you'd find that none of them work for "your team", look for something new that works (e.g. maybe <a href="http://cukes.info/">Cucumber</a> is a better fit). Instead of spending hours in analysis paralysis, choose one, <b>get started, reflect and adapt</b>. <br />
<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjLTN35SNnhLS3zXOwP88Zp2N0505sv57BYjkrVdVpcjGYnQtytnfWOU3fGJtPImkBifJNfCfHNJoTpNow3OjPOeORmGLxTFvLEOYxNV-FH1b3oTPBTN4ygPJFRjU7WtYojXi8x/s1600/UIvsAPITesting.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjLTN35SNnhLS3zXOwP88Zp2N0505sv57BYjkrVdVpcjGYnQtytnfWOU3fGJtPImkBifJNfCfHNJoTpNow3OjPOeORmGLxTFvLEOYxNV-FH1b3oTPBTN4ygPJFRjU7WtYojXi8x/s1600/UIvsAPITesting.jpg" /></a></div>Gishuhttp://www.blogger.com/profile/17616896114730114577noreply@blogger.com0Thane, Maharashtra, India19.194459565404916 72.9629516601562519.032341565404916 72.729492160156255 19.356577565404915 73.196411160156245tag:blogger.com,1999:blog-14744716.post-10506337053954250522010-10-27T16:57:00.003+05:302010-11-11T13:45:39.821+05:30How to get Thoughtworks' White to LogStructure<div style="font-family: "Trebuchet MS",sans-serif;">Struggled for some time getting it to work. I've been playing with Thoughtworks' open library for UI Testing for a while.<br />
<br />
Sometimes <a href="http://white.codeplex.com/wikipage?title=Third%20Party%20Controls">UISpy doesn't get you to the UIElement hierarchy</a> and you need to take out the big guns ala White's window.LogStructure to dump out the UIElement tree.<br />
As always it's simple when you know how.</div><div style="font-family: "Trebuchet MS",sans-serif;"><br />
</div><div style="font-family: "Trebuchet MS",sans-serif;"><b>Step#1</b>: Modify the app.config of your executable to include the sections related to White. See a sample file on White's page <a href="http://white.codeplex.com/wikipage?title=Configuration">here</a>.</div><div style="font-family: "Trebuchet MS",sans-serif;"><br />
</div><div style="font-family: "Trebuchet MS",sans-serif;"><b>Step#2</b>: Add the following line to your AssemblyInfo.cs file</div><div style="font-family: "Trebuchet MS",sans-serif;"><br />
</div><pre class="csharpcode">[assembly: log4net.Config.XmlConfigurator(ConfigFile = <span class="str">"Log4Net.config"</span>, Watch = <span class="kwrd">true</span>)]</pre><br />
<div style="font-family: "Trebuchet MS",sans-serif;"><b>Step#3</b>: Add a file called Log4Net.config under your assembly (configure it such that it is always copied to the output folder). This contains the configuration information for Log4Net (e.g. the following config logs to console and to file named example.log in your output folder.). That's it you should now be able to see/capture window.LogStructure output from White. Godspeed !</div><br />
<br />
<!-- code formatted by http://manoli.net/csharpformat/ --><br />
<pre class="csharpcode"><span class="kwrd"><</span><span class="html">log4net</span><span class="kwrd">></span>
<span class="kwrd"><</span><span class="html">appender</span> <span class="attr">name</span><span class="kwrd">="Console"</span> <span class="attr">type</span><span class="kwrd">="log4net.Appender.ConsoleAppender"</span><span class="kwrd">></span>
<span class="kwrd"><</span><span class="html">layout</span> <span class="attr">type</span><span class="kwrd">="log4net.Layout.PatternLayout"</span><span class="kwrd">></span>
<span class="rem"><!-- Pattern to output the caller's file name and line number --></span>
<span class="kwrd"><</span><span class="html">conversionPattern</span> <span class="attr">value</span><span class="kwrd">="%5level [%thread] (%file:%line) - %message%newline"</span> <span class="kwrd">/></span>
<span class="kwrd"></</span><span class="html">layout</span><span class="kwrd">></span>
<span class="kwrd"></</span><span class="html">appender</span><span class="kwrd">></span>
<span class="kwrd"><</span><span class="html">appender</span> <span class="attr">name</span><span class="kwrd">="RollingFile"</span> <span class="attr">type</span><span class="kwrd">="log4net.Appender.RollingFileAppender"</span><span class="kwrd">></span>
<span class="kwrd"><</span><span class="html">file</span> <span class="attr">value</span><span class="kwrd">="example.log"</span> <span class="kwrd">/></span>
<span class="kwrd"><</span><span class="html">appendToFile</span> <span class="attr">value</span><span class="kwrd">="true"</span> <span class="kwrd">/></span>
<span class="kwrd"><</span><span class="html">maximumFileSize</span> <span class="attr">value</span><span class="kwrd">="100KB"</span> <span class="kwrd">/></span>
<span class="kwrd"><</span><span class="html">maxSizeRollBackups</span> <span class="attr">value</span><span class="kwrd">="2"</span> <span class="kwrd">/></span>
<span class="kwrd"><</span><span class="html">layout</span> <span class="attr">type</span><span class="kwrd">="log4net.Layout.PatternLayout"</span><span class="kwrd">></span>
<span class="kwrd"><</span><span class="html">conversionPattern</span> <span class="attr">value</span><span class="kwrd">="%level %thread %logger - %message%newline"</span> <span class="kwrd">/></span>
<span class="kwrd"></</span><span class="html">layout</span><span class="kwrd">></span>
<span class="kwrd"></</span><span class="html">appender</span><span class="kwrd">></span>
<span class="kwrd"><</span><span class="html">root</span><span class="kwrd">></span>
<span class="kwrd"><</span><span class="html">level</span> <span class="attr">value</span><span class="kwrd">="DEBUG"</span> <span class="kwrd">/></span>
<span class="kwrd"><</span><span class="html">appender-ref</span> <span class="attr">ref</span><span class="kwrd">="Console"</span> <span class="kwrd">/></span>
<span class="kwrd"><</span><span class="html">appender-ref</span> <span class="attr">ref</span><span class="kwrd">="RollingFile"</span> <span class="kwrd">/></span>
<span class="kwrd"></</span><span class="html">root</span><span class="kwrd">></span>
<span class="kwrd"></</span><span class="html">log4net</span><span class="kwrd">></span></pre>Gishuhttp://www.blogger.com/profile/17616896114730114577noreply@blogger.com0tag:blogger.com,1999:blog-14744716.post-18461203129243661692010-08-20T14:40:00.004+05:302010-08-20T15:50:16.387+05:30An effective test strategy to grow applicationsThis is a post to summarize my understanding (as of today) of an effective (low ceremony) way to build applications. <br />Let's take a look at (a slightly modified version) of <a href="http://blog.mountaingoatsoftware.com/the-forgotten-layer-of-the-test-automation-pyramid">Mike Cohn's layered test pyramid</a>. <br /><br /><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiLl7NUyY8XjsB-00YHPyIJNf1asFcbR3n-XTUw_wueNaLCRRDiZvVealJQlmfwo0NUwWXyX2vyU-Zykqp4Zl6v5YxaNT4ztkbUzTqjoSTNtCvWfimi5im_RzzT_Rg9RnOcwW0F/s800/Modified%20Cohn's%20test%20pyramid_Grad.png"><img style="display: block; margin: 0px auto 10px; text-align: center; cursor: pointer; width: 612px; height: 434px;" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiLl7NUyY8XjsB-00YHPyIJNf1asFcbR3n-XTUw_wueNaLCRRDiZvVealJQlmfwo0NUwWXyX2vyU-Zykqp4Zl6v5YxaNT4ztkbUzTqjoSTNtCvWfimi5im_RzzT_Rg9RnOcwW0F/s800/Modified%20Cohn's%20test%20pyramid_Grad.png" alt="" border="0" /></a><br /><br />The choice of a pyramid indicates the robustness/stability of a lower layer directly affects the effectiveness of the upper layer. Also the number of tests decreases as you move up. As the <a href="http://www.agiletester.ca/">agile testing book</a> says ROI is maximum at the bottom (speed of feedback over time invested) and wanes towards the top.<br /><span style="font-size:85%;">e.g. without robust unit-tests, DSL tests or GUI tests would catch a bunch of errors without the essential feedback needed to fix it quickly. More and more errors would make it up to the middle and top layers, where it is more time-consuming/expensive to find-n-fix. Nothing replaces well-written, quick, professional unit tests.</span><br />To prevent ambiguity and misinterpretation, let's go over each layer.<br /><a name='more'></a><br /><br /><span style="font-size:130%;">Layer 0: Unit-Integration-Stress tests</span><br /><ul><li>Unit tests should accumulate over time as a side-effect of test driven development. It's a tough art to master -- writing good unit tests but they are invaluable in the long term. They are essential for 'internal quality'. This layer also contains some integration tests and stress tests.</li><li>'Integration tests' in this post means (as in the GOOS Book) : 'Let me see if the real collaborator behaves as I expect it to'.<br /></li></ul><span style="font-size:85%;">e.g. Let's say Car needs to use the Driver database. Since we live in these times, we'd define a Role(Interface) DriverRepository to abstract away the database from ClassA. In my unit-tests, I'd inject a MockDriverRepository into a Car and test it out. So I know Car is good. Now there are some assumptions implicit in the Role/interface. I need to verify that the real implementation behaves the same - so I'd need to write some 'integration tests' to verify if the real collaborator DriverRepository behaves the same. <span style="font-weight: bold;">These tests would round-trip a third party / external subsystem in order to produce a result. </span>TestDriverRepository.GetsSpecifiedDriverInformation() would test if we can get the specified driver information out from a real database. Integration tests are *slower* than unit-tests, so we partition it into a different test-suite that is not run as frequently as the unit-tests. Feedback needs to be as short as possible during development.<br /></span><br /><ul><li>'Stress tests' are for asynchronous code - code that has multiple threads running through them. In this case, you'd have to define some invariants - things that must remain true irrespective of the number and scheduling of multiple threads through the code. (Refer to the last couple of chapters of the GOOS book for details).</li></ul><br /><br /><span style="font-size:130%;">Layer 1: Under the GUI / DSL-API tests</span><br />This is the layer that's rarely given its due and usually missed - its primary reason for existence is scenario testing just under the skin/GUI. These tests are written from the user's perspective. When you get this layer right, its sweet. You will see a DSL emerging. <span style="font-weight: bold;">Your tests appear as short scripts written in an application specific DSL, where the steps are things that an actual user would do or perceive.</span> These tests can exercise the whole app similar to a real user, without having to bring up a GUI... as long as you have designed it for testability (use a MVP / MVVM pattern).<br />These tests <span style="font-weight: bold;">involve all real collaborators</span> (no mocks/stubs/fakes). As a result, they are <span style="font-weight: bold;">slower than Unit tests</span>. However built on a bedrock of solid unit tests, they can take you to the finish line... almost.<br /><br /><span style="font-size:130%;">Layer 2: GUI Tests</span><br />Even with anemic views, there is a matter of <span style="font-weight: bold;">the 'wiring'</span>. Is the GUI Control wired correctly to the right property/action on the backing object? Irrespective of the strength of the layers beneath it, you need a set of GUI tests that <span style="font-weight: bold;">go really end-to-end identical to a real user</span>. They're the slowest we've seen so far - so <span style="font-weight: bold;">do not test everything through the GUI</span>. The test duration would grow prohibitively large with time.<br />An interesting idea here is: to just test the code in the GUI in the GUI Tests. I haven't tried this out myself but sounds like something that is just crazy enough to work.<br /><br />We're almost at the end. There are some aspects that we haven't yet tested.<br /><ul><li>Exploratory tests: Let some good testers go ad-hoc at your app trying to break things. These tests are manual since they rely on the tester's creativity and insight (not mechanical).</li><li>Usability tests: Regular demos and getting some real users to use the latest builds would go a long way to ensuring usability. Testers can act as a proxy as long as they are aligned with the customer's profile and needs. These are manual too since machines can't determine usability.</li><li>"ility" tests: These are tests that deal with performance, load, security, reliability, scalability, etc. When a feature is picked up for implementation, if it has any of the above 'ility' needs, they should be noted and a corresponding test be written up to validate them. (The point being.. don't wait till the end to start with this.) The tests can be written in a scripting language and <span style="font-weight: bold;">use specialized tools/libraries if required</span>. </li></ul><span style="font-size:130%;">Who does what ??</span><br /><br /><iframe src="https://spreadsheets.google.com/pub?key=0AqQfN_JAz01mdGlBZEg3Q3JuZjY2R1g0WmUtYmxWQWc&hl=en&single=true&gid=0&output=html&widget=true" frameborder="0" height="300" width="500"></iframe><br /><br />An effective way to implement features is a Top-Down approach, where you begin by writing a GUI Test/DSL-API test first and then work your way down. This way you only write code that a client needs - which leads to simple usable APIs. Once you have a failing red acceptance test,<br /><ul><li>You can either figure out the top-level design to get it to work and test-drive the classes required. Combine them to make the accceptance test pass. </li></ul>OR<br /><ul><li>you can begin by writing the code to make the acceptance test pass. Once green, in the refactoring step, you can now figure out the right "house" for the code. TDD the objects that you <span style="font-weight: bold;">now know you need</span> and fold them back into the running application (See the GOOS book if you're interested in this approach).</li></ul><br />That's it. This ties up nicely with <a href="http://www.exampler.com/old-blog/2003/08/21/#agile-testing-project-1">Brian Marick's test quadrants idea</a>.<br /><br /><br /><img style="display: block; margin: 0px auto 10px; text-align: center; cursor: pointer; width: 612px; height: 434px;" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh4rC1BChSPkLru1PiyXKxFG7ynFnZ6he-1v-XiQlR4AMWdL5LYXQZjY5zuCa4R6DTCzh8KgVvHvjb4h8BHjeNoAqc0GA8J5jtiM82Omv-IPV9HDclHqZMlrlHQ9A8MNO0X7kxc8w/s800/Testing.png" alt="" border="0" />Gishuhttp://www.blogger.com/profile/17616896114730114577noreply@blogger.com0tag:blogger.com,1999:blog-14744716.post-10189997911713295872010-08-07T18:48:00.006+05:302010-08-07T20:34:46.280+05:30Review : Growing Object Oriented Software - Guided by Tests: Freeman, Pryce<span style="font-family:trebuchet ms;"><span style="font-weight: bold;">Recommended... best book on TDD in a while.</span></span><br /><br /><span style="font-family:trebuchet ms;">Just finished this one today... I got to fess up - that I am a "classicist". I've seen some real spaghetti tests involving mocks.. But I've gotten some periodic tension from the people on the other side of this religious fence of TDD. Since this book comes from "the torch-bearers of the mockist camp", I picked up a copy and tried to keep an open mind. And I although I haven't switched camps.. I now see the light. There is no one right path but the authors have placed some industrial strength beacons on this path and teach you how to use mocks and interaction tests to "grow" applications driven by automated tests. (</span><span style="font-style: italic;font-family:trebuchet ms;" >However I still maintain that in untrained hands, Mocks (like C++) can be very dangerous and quickly lead into tarpits. Easy to abuse. However in the right hands, they can be used to write very expressive code & enable rapid and iterative development</span>. ).<br /><br />Some of the key takeaways for me were:<br /><ul><li><span style="font-family:trebuchet ms;">Developing outside-in ATDD: writing a 'walking skeleton' first , spotting objects and roles (relationships) between them, TDDing these objects and "folding them back" into a running app was a new method to me.</span></li><li> <span style="font-family:trebuchet ms;">Listening to the tests, Focus on readability, tests with helpful diagnostics / failure messages, eliminating or reducing the irrelevant part of the test, use of builders + drivers to keep the tests concise</span></li><li style="font-family: trebuchet ms;"><span>The chapters on threads and asynchronous testing are home-runs. Don't leave for work without it. The last third of the book was worth the price of the book (It isn't yet available in India so had to get it from overseas).</span></li></ul><span style="font-family:trebuchet ms;">There is a lot of real world knowledge that the authors have distilled down to a kind of 'secret sauce'.<br /><a name='more'></a><br /></span><span style="font-family:trebuchet ms;">As the book rightly says, TDD that isn't sustainable will slowly choke the project, tests turn into maintenance headaches and strangles development progress.</span> <span style="font-family:trebuchet ms;">This is the part that trips most people up with TDD - Kent Beck's book on TDD and most of the literature out there detail the basic technique (with an assumption that you would figure out the exact steps that you need to take in your specific context). The skill of "listening to the tests" isn't easy to pickup or practice for most people. Freeman, Pryce et. all have come up with a more prescriptive method that should keep beginners on the rails.<br /></span><br /><span style="font-family:trebuchet ms;">I followed along to the AuctionSniper example using my own style (and .net, nUnit, Moq + White instead of java, jUnit, jMock + WindowLicker as in the book) in order to compare and contrast. The example isn't trivial (deals with multiple-threads, third party server and protocol (XMPP) and was well chosen.</span><br /><br /><embed type="application/x-shockwave-flash" src="http://picasaweb.google.com/s/c/bin/slideshow.swf" flashvars="host=picasaweb.google.com&hl=en_US&feat=flashalbum&RGB=0x000000&feed=http%3A%2F%2Fpicasaweb.google.com%2Fdata%2Ffeed%2Fapi%2Fuser%2FGishu.Pillai%2Falbumid%2F5502654231587001233%3Falt%3Drss%26kind%3Dphoto%26hl%3Den_US" pluginspage="http://www.macromedia.com/go/getflashplayer" height="400" width="600"></embed><br /><br /><span style="font-family:trebuchet ms;">The final solution was different from the solution in the book; but still passed all the tests. The authors emphasize test readability, which is a 'Can do better' item for me. </span><br /><br /><span style="font-family:trebuchet ms;"><span style="font-weight: bold;">Final word</span>: This book is a must-read for intermediate and advanced programmers interested in gaining mastery over TDD. I'd place it on the top-shelf right next to Beck's XP Book, TDD Book & Fowler's Refactoring book. It doesn't have any radical new ideas however it does teach you the right guiding principles to survive in the real world today. <span style="font-style: italic;">It says a lot of things that needed to be said at this point of time... for that I'd give it a 9/10.</span></span><br /><br /><span style="font-family:trebuchet ms;">Note: The book also has a </span><a style="font-family: trebuchet ms;" href="http://www.growing-object-oriented-software.com/">companion site</a><span style="font-family:trebuchet ms;">, with a responsive discussion forum.</span>Gishuhttp://www.blogger.com/profile/17616896114730114577noreply@blogger.com0tag:blogger.com,1999:blog-14744716.post-1572536820432876272010-07-27T17:03:00.004+05:302010-07-30T16:21:59.824+05:30Contract Tests with Slim/Fitnesse : Running a test suite against multiple implementations of an interface<span style="font-family:trebuchet ms;">Recently I had a need for something similar to "xUnit contract tests" in Fitnesse (defined as: a test suite, which any object must satisfy so that it can be considered as a valid implementation of a Role. I got the term from one of J.B.Rainsberger's talks. ) </span><br /><br /><span style="font-family:trebuchet ms;">To illustrate my point, let me cook up an example.</span><br /><br /><span style="font-family:trebuchet ms;">Imagine a contest for BoneyM impersonators. We would quiz each impersonator with a set of questions that they must answer correctly to win the contest.</span><br /><br /><span style="font-family:trebuchet ms;">i.e. we have a set of tests (questions) for each implementation of an Impersonator (Role). We also have multiple impersonators that I want to validate against the contract tests. The tests need to be dry - since they don't change for each implementation. </span><a name='more'></a><br /><br /><span style="font-family:trebuchet ms;">So here are the questions and the expected answers to them. (the contract tests)</span><br /><br /><ul><li>Q:How did rasputin die? A:His enemies shot him till he was dead.</li><li>Q:Was yesterday sunny? A: No</li><li>Q:Who runs the baker family? A: Ma Baker</li></ul><span style="font-family:trebuchet ms;">So the contract for each impersonator is</span><br /><pre class="csharpcode"><br /> <span class="kwrd">public</span> <span class="kwrd">interface</span> BoneyMImpersonator<br /> {<br /> <span class="kwrd">string</span> HowDidRasputinDie();<br /> <span class="kwrd">bool</span> WasYesterdaySunny();<br /> <span class="kwrd">string</span> WhoRunsTheBakerFamily();<br /> }</pre><br /><br /><span style="font-weight: bold;font-family:trebuchet ms;" >Slim/Fitnesse:</span><br /><span style="font-family:trebuchet ms;">The new addition to the Fitnesse/Fit family is the SLIM runner. From what I've seen in a couple of days - I'm impressed!</span><br /><span style="font-family:trebuchet ms;">For some quick help to get it running, check out this post by Brett Schuchert - <a href="http://schuchert.wikispaces.com/FitNesse.Tutorials.0">http://schuchert.wikispaces.com/FitNesse.Tutorials.0</a> </span><br /><span style="font-family:trebuchet ms;"> </span><br /><span style="font-family:trebuchet ms;">One of the table-styles that SLIM has introduced is "scenario tables". The thing that is great about them - is that it brings something similar to C++ macros (expansion via text substitution) to Fitnesse. i.e. you can write up a scenario table once and then call it from multiple places in other tables (script/decision tables). Let's see it in action.</span><br /><span style="font-family:trebuchet ms;"> </span><br /><br /><span style="font-size:85%;"><span style="font-family:trebuchet ms;">Here's my Fitnesse page:</span><br /><a href="http://code.google.com/p/gishu-util/source/browse/trunk/Blog/ContractTests_FitnessePage.txt"><span style="font-family:trebuchet ms;">http://code.google.com/p/gishu-util/source/browse/trunk/Blog/ContractTests_FitnessePage.txt</span></a><br /><br /><span style="font-family:trebuchet ms;">And here's the backing fixture and domain code</span><br /><a href="http://code.google.com/p/gishu-util/source/browse/trunk/Blog/FixtureCode.cs"><span style="font-family:trebuchet ms;">http://code.google.com/p/gishu-util/source/browse/trunk/Blog/FixtureCode.cs</span></a><br /><a href="http://code.google.com/p/gishu-util/source/browse/trunk/Blog/DomainCode.cs"><span style="font-family:trebuchet ms;">http://code.google.com/p/gishu-util/source/browse/trunk/Blog/DomainCode.cs</span></a><br /><br /></span><br /><span style="font-family:trebuchet ms;">Refer to this fitnesse doc page for a sample <a href="http://fitnesse.org/FitNesse.UserGuide.SliM.ScenarioTable">http://fitnesse.org/FitNesse.UserGuide.SliM.ScenarioTable</a></span>.<br /><span style="font-size:85%;"><br /></span><span style="font-family:trebuchet ms;">There's a collapsed section called "No Peeking" - but you should. Without it, SLIM doesn't have an actor or co-ordinator to talk to - and you'd get cryptic errors like 'Method [name] not found in fitSharp.Slim.Operators.ExecuteBase+NullInstance'</span><span style="font-family:trebuchet ms;">. The missing ingredient for me was the <span style="font-weight: bold;">|script|quiz|</span> line. With this, you tell Fitnesse that a class named Quiz is who you should talk to.</span><br /><br /><span style="font-family:trebuchet ms;">Next you define the scenario (see the above link for details). In short, a scenario has a name and parameters. Parameters must be used within the scenario prefixed with a @ sign. In our scenario, we use the fully-qualified type name of the implementation as the parameter. In welcome, we create an instance of it via some reflection/activation magic. Next we interrogate the object and compare answers.</span><br /><span style="font-family:trebuchet ms;"> </span><br /><span style="font-family:trebuchet ms;">Finally we need the scenario to be run against multiple implementations of the contract. For that we have a decision table (equivalent to Fit's ColumnFixture), which must have the exact same name as the scenario and the column headers must match the scenario parameters.</span><br /><span style="font-family:trebuchet ms;">So in effect, each row is replaced by a scenario invocation with the parameters specified in the row. This can be seen when you run the tests. You can see new column with cells, which be expanded to see the scenario details as shown below...</span><br /><br /><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhK5t4WD5IQKIZYCQ7BRBp37TNgiRjtCSf2b7jKorhv6zQoo27Z3Zdx00q47Xd0RYM6JJtmbya57J4ReyhyphenhyphenCJUccdUefcyutpymqDnZYY_bemFWcesoqNX2HWQ18MsdFgPltyoc/s800/ContractTestsWithSlim.jpg" /><br /><br /><span style="font-family:trebuchet ms;">In the future, if you have a new implementation say StonedGuy, just add another row to the Decision table and that's it! The contract tests are DRY-compliant.<br /><br /><span style="font-weight: bold;">Update: </span>Gregor Gramlich points out that the welcome method doesn't need to be implemented in the fixture, just for an instance of the test subject via reflection. Instead a line <span style="font-weight: bold;">|start | FullyQualifiedClassName |</span> works just as well. Nice tip. Code samples updated.<br /></span>Gishuhttp://www.blogger.com/profile/17616896114730114577noreply@blogger.com0tag:blogger.com,1999:blog-14744716.post-22929505049385641962010-04-22T16:13:00.004+05:302010-07-27T17:00:37.898+05:30Implementing case-insensitive lookup tables (Dictionary) with string keys<span style="font-family:trebuchet ms;">The Dictionary<tkey,> type in .Net takes in a comparer as a constructor parameter. So you could pass in StringComparer.OrdinalIgnoreCase and you're done.</tkey,></span><br /><span style="font-family:trebuchet ms;">I used to upper-case keys while adding it to the lookup table earlier and again when performing a lookup - to achieve case-insensitive string lookups. I remember reading some article about String.ToUpper() being optimized for such scenarios. Now I was unsure - which one is better ?</span> <a name='more'></a><br /><br /><span style="font-family:trebuchet ms;">So I ran a small experiment.</span><br /><pre class="csharpcode"><br /><span class="rem">//void Main()</span><br /> var keys_for_lookup = <span class="kwrd">new</span> List<<span class="kwrd">string</span>> { <span class="str">"abc"</span>, <span class="str">"dEF"</span>, <span class="str">"123"</span>, <span class="str">"655"</span>, <span class="str">"pQr"</span> };<br /> <span class="kwrd">for</span> (<span class="kwrd">int</span> i = 0; i < 3; i++)<br /> {<br /> var dict1 = <span class="kwrd">new</span> Dictionary<<span class="kwrd">string</span>, <span class="kwrd">object</span>>();<br /> AddSomeKeys(dict1, <span class="kwrd">new</span> List<<span class="kwrd">string</span>> { <span class="str">"ABC"</span>, <span class="str">"DEF"</span>, <span class="str">"GHI"</span>, <span class="str">"JKL"</span>, <span class="str">"MNO"</span>, <span class="str">"PQR"</span>, <span class="str">"XYZ"</span>, <span class="str">"123"</span> });<br /> TimeIt(<span class="kwrd">delegate</span> { RunTest_Upper(dict1, keys_for_lookup); });<br /><br /> dict1 = <span class="kwrd">new</span> Dictionary<<span class="kwrd">string</span>, <span class="kwrd">object</span>>(StringComparer.OrdinalIgnoreCase);<br /> AddSomeKeys(dict1, <span class="kwrd">new</span> List<<span class="kwrd">string</span>> { <span class="str">"abc"</span>, <span class="str">"def"</span>, <span class="str">"ghi"</span>, <span class="str">"jkl"</span>, <span class="str">"mno"</span>, <span class="str">"pqr"</span>, <span class="str">"XYZ"</span>, <span class="str">"123"</span> });<br /> TimeIt(<span class="kwrd">delegate</span> { RunTest(dict1, keys_for_lookup); });<br /> }<br /><br /><span class="rem">// end of main</span><br /><br /></pre><br /><br /><span style="font-family:trebuchet ms;">And this is what the results looked like :<br />It looks like using a Dictionary with a comparer injected is<br /></span><ol><li style="font-family:trebuchet ms;"><span style="font-weight: bold;">faster </span><br /></li><li style="font-family:trebuchet ms;"><span style="font-weight: bold;">less hassle</span>. You specify it once and forget it. You do not have remember to do UpperCase each time you lookup the map. (e.g. from the p.o.v. of future programmers)</li><li><span style="font-family:trebuchet ms;"><span style="font-weight: bold;">less memory pressure</span>. ToUpper() would create a new String instance per lookup. </span><br /></li></ol><span style="font-family:trebuchet ms;"></span><br /><br /><table style="width: auto;"><tbody><tr><td><a href="http://picasaweb.google.com/lh/photo/o5Cmdah_ybOcsjB6Tj6W1_caujkF9KLSFFuJboHAXZ8?feat=embedwebsite"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg02-NqoJiJBOMmdeDpCm1Oq02VRh1lmW2IpLCKVAnw3DSvRoGPzAgzV_FIdS-1r1yZ6rVuQ7ahcBjhXLiYQ7rCajFqwTR51hxtBPqJT8IQQnAn_RBQFBGTBZEehXNBKwjDVHAo/s800/insensitive_case_results.png" /></a></td></tr><tr><td style="font-family: arial,sans-serif; font-size: 11px; text-align: right;">From <a href="http://picasaweb.google.com/Gishu.Pillai/MyConversationsWithGullibleMachines?authkey=Gv1sRgCNTm5f7KtOeXBw&feat=embedwebsite">My conversations with gullible machines...</a></td></tr></tbody></table><br />(Setup: .Net 3.5, VS2008, WinXP SP2, Intel T7400 Core 2, 2GB RAM on a normal work afternoon, lots of apps open)<br /><br /><span style="font-family:trebuchet ms;">The rest of the code for scrutiny:</span><br /><br /><pre class="csharpcode"><br /><span class="kwrd">private</span> <span class="kwrd">static</span> <span class="kwrd">void</span> AddSomeKeys(Dictionary<<span class="kwrd">string</span>, <span class="kwrd">object</span>> dict1, List<<span class="kwrd">string</span>> keys)<br /> {<br /> <span class="kwrd">foreach</span> (var key <span class="kwrd">in</span> keys)<br /> dict1.Add(key, <span class="kwrd">new</span> Object());<br /> }<br /><br /> <span class="kwrd">private</span> <span class="kwrd">static</span> <span class="kwrd">void</span> RunTest_Upper(Dictionary<<span class="kwrd">string</span>, <span class="kwrd">object</span>> dict1, List<<span class="kwrd">string</span>> keys)<br /> {<br /> <span class="kwrd">for</span> (<span class="kwrd">int</span> i = 0; i < 1000000; i++)<br /> <span class="kwrd">foreach</span> (var key <span class="kwrd">in</span> keys)<br /> { var v = dict1.ContainsKey(key.ToUpper()); }<br /><br /> }<br /> <span class="kwrd">private</span> <span class="kwrd">static</span> <span class="kwrd">void</span> RunTest(Dictionary<<span class="kwrd">string</span>, <span class="kwrd">object</span>> dict1, List<<span class="kwrd">string</span>> keys)<br /> {<br /> <span class="kwrd">for</span> (<span class="kwrd">int</span> i = 0; i < 1000000; i++)<br /> <span class="kwrd">foreach</span> (var key <span class="kwrd">in</span> keys)<br /> { var v = dict1.ContainsKey(key); }<br /> }<br /><br /> <span class="kwrd">private</span> <span class="kwrd">static</span> <span class="kwrd">void</span> TimeIt(Action action)<br /> {<br /> var s = System.Diagnostics.Stopwatch.StartNew();<br /> action.Invoke();<br /> s.Stop();<br /> Console.WriteLine(<span class="str">"Elapsed msecs = {0}"</span>, s.ElapsedMilliseconds);<br /> }</pre>Gishuhttp://www.blogger.com/profile/17616896114730114577noreply@blogger.com0tag:blogger.com,1999:blog-14744716.post-52280900769542899532010-04-20T12:39:00.007+05:302010-07-27T17:02:48.226+05:30WPF - Find a child control of a specific type or satisfying a given predicate under a parent control<span style="font-family: trebuchet ms;">Ideally you shouldn't be doing this in WPF and MVVM land. However there is an exception to every rule.. mine was I had to set the focus and put a specific cell in a grid into edit mode when the user clicked a button.</span> <a name='more'></a><br /><br /><span style="font-family: trebuchet ms;">Now this is one of the corner cases, for which you need to write some code-behind in the view. Next I needed to get a handle on the datagrid, which was nested within a few expanded data-templates. You can't name this grid, because each expanded data-template would have the same name value.</span><br /><br /><span style="font-family: trebuchet ms;">I did not want to hard code the entire hierarchy path of UI types from the tab to the nested grid. So what I needed was an API like</span><br /><br /><pre class="csharpcode"><br />var grid = tab.FindChild<DataGrid>();</pre><br /><br /><span style="font-family: trebuchet ms;">Unfortunately this isn't built-in functionality ; so I look on Stackoverflow and find a </span><a style="font-family: trebuchet ms;" href="http://stackoverflow.com/questions/636383">relevant question</a><span style="font-family: trebuchet ms;">.</span><br /><span style="font-family: trebuchet ms;">However it didn't work for me as is.</span><br /><br /><span style="font-family: trebuchet ms;">Some more debugging and searching later.. I find this post '</span><a style="font-family: trebuchet ms;" href="http://www.blogger.com/Understanding%20the%20Visual%20Tree%20and%20Logical%20Tree%20in%20WPF">Understanding the Visual and Logical tree in WPF</a><span style="font-family: trebuchet ms;">' by the always-enlightening WPF Guru Josh Smith. Now I'm not sure I can explain it to a third person..</span><br /><span style="font-family: trebuchet ms;">but here is what I found by tinkering with a test-app.</span><br /><br /><table style="width: auto;"><tbody><tr><td><a href="http://picasaweb.google.com/lh/photo/Ig-V8ax1Yf-drJlalUMJUfcaujkF9KLSFFuJboHAXZ8?feat=embedwebsite"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiiLq9hR9wTVEHwT_Nk0DsscWusAgKBSGtNF4TlOFhuDmsCs8l4FJ6tFD7Qu_fdga8gZN-4yJxiGn1VAzrBsQyegHH40kU4sO8vAeRsaKocZ2R3iyPZzBpf3njnkAMVMxRNUDvk/s800/LogicalChildren.jpg" /></a></td></tr><tr><td style="font-family: arial,sans-serif; font-size: 11px; text-align: right;">From <a href="http://picasaweb.google.com/Gishu.Pillai/MyConversationsWithGullibleMachines?authkey=Gv1sRgCNTm5f7KtOeXBw&feat=embedwebsite">My conversations with gullible machines...</a></td></tr></tbody></table><br /><br /><ul><li style="font-family: trebuchet ms;">A DependencyObject can have m logical (shown in bold above) and n visual children.</li><br /><li style="font-family: trebuchet ms;">The visual and logical children can overlap. e.g. a StackPanel can be a logical and a visual child of a parent at the same time</li><br /><li style="font-family: trebuchet ms;">A logical child can be any object - does not necessarily derive from DependencyObject</li><br /></ul><br /><br />With this knowledge, I wrote a few overloads of FindChild - source <a href="http://code.google.com/p/gishu-util/source/browse/trunk/WPF/Utilities/DepObjExtn.cs">available here</a>. Pretty self-explanatory.<br /><pre class="csharpcode"><br /><br /><span class="rem">// find a child textbox with a specific name</span><br />var textBox = tab.FindChild<TextBox>( t => t.Name == <span class="str">'txtName'</span>);<br /><br /><span class="rem">// find all child stackpanels meeting a specific criteria (has children)</span><br /><span class="kwrd">foreach</span>(var panel <span class="kwrd">in</span> tab.FindChildren<StackPanel>( panel => panel.Children.Count > 0)<br />{ // <span class="kwrd">do</span> something with panel }</pre><br /><br /><span style="font-family: trebuchet ms;">There is also another class - a </span><a style="font-family: trebuchet ms;" href="http://code.google.com/p/gishu-util/source/browse/trunk/WPF/Utilities/UITreeNode.cs">VM for a Node</a><span style="font-family: trebuchet ms;"> in the UI Hierarchy, that I used to create the test app. I found it useful for diagnostics. e.g. if you have a reference _contentBox. You can bring up QuickWatch and add the expression 'new UITreeNode(_contentBox)' and visualize the logical and visual hierarchy under it. (Of course you need to paste the class definition into your project temporarily.)</span><br /><br /><table style="width: auto;"><tbody><tr><td><a href="http://picasaweb.google.com/lh/photo/onobwCbEem8n5dmTpocgGPcaujkF9KLSFFuJboHAXZ8?feat=embedwebsite"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgF94YyacdsS-C2lMkRIDAvnXGIztaQfnQ71mb85lyKHmxqtEvFRLcm25koIPY81CX7jtVgqM1We5yk1P3zhskZrF-XSBDtRFK_KmyW56NfFpdqhQxCawymkYT3vXY4OEGIBa9W/s800/UITreeNode_QuickWatch.JPG" /></a></td></tr><tr><td style="font-family: arial,sans-serif; font-size: 11px; text-align: right;">From <a href="http://picasaweb.google.com/Gishu.Pillai/MyConversationsWithGullibleMachines?authkey=Gv1sRgCNTm5f7KtOeXBw&feat=embedwebsite">My conversations with gullible machines...</a></td></tr></tbody></table>Gishuhttp://www.blogger.com/profile/17616896114730114577noreply@blogger.com1