TDD Kata 2 - Interaction based testing

tdd-kata-2TDD Katas has become very popular in a small segment of the development community and we call our selves software craftsmen. We are passionate about software development as a craft and engage in different activities to better our selves and our peers.

My first kata cast, for instance, has been viewed close to 10k times on Vimeo since its publication. Much of the attention is of course due to Roy Osherove linking to my blog post from his TDD Kata 1 page. This time Roy initiated a sequel, meant to introduce interaction based testing using mocks and possibly stubs, and continue the teaching process of TDD and unit testing practices.

The Kata Cast

The following screen cast covers the entire kata in .NET, complete with Osherove’s three steps as well as manual UI testing at the end.

For best viewing experience I recommend watching it on in HD

String Calculator TDD Kata 2 - Interactions from Rickard Nilsson on Vimeo.

The tools I use are Visual Studio, ReSharper, TestDriven.NET, Moq for mocking, and NUnit.

The code

The code and Visual Studio solution for the finished Kata can be downloaded from GitHub:

Download source


As Osherove mentions in his instructions, this kata is not as simple as the first part, nor as simple as most katas out there. The reason is the element of interaction based unit testing involved, which is quite difficult to wrap you mind around, and it took quite a while to get the steps right. I thought I should share my path to the kata in its present form for others to learn from and comment on.

The Kata

Step 1. Everytime you call Add(string) it also outputs the number result of the calculation in a new line to the terminal or console. (remember to try and do this test first!)

As I did this test first I started out pretty much as how it ended up in the cast. However, after a while I tried to take a step back and see if there were any smells in the code I had not yet discovered.

I found that I didn’t really like the mixed responsibilities that the Calculator class got when I introduced writing to the console. This could be seen as a logging feature and thus a perfect candidate to become an aspect (in AOP). I started playing around with PostSharp and ended up with the following solution which is quite clean.

   1: [Serializable] 
   2: public class OutputAttribute : OnMethodBoundaryAspect 
   3: { 
   4:     [NonSerialized] 
   5:     private IContainer container;
   7:     public override void OnExit(MethodExecutionArgs args) 
   8:     { 
   9:         var console = container.Resolve<IConsole>(); 
  10:         console.WriteLine(args.ReturnValue.ToString()); 
  11:     }
  13:     [OnDeserialized] 
  14:     public void OnDeserialized(StreamingContext context) 
  15:     { 
  16:         container = ContainerFactory.Current; 
  17:     } 
  18: }

Figure 1. Output aspect

Which, at most, leaves the mark of a custom attribute in the Calculator class:

   1: public class Calculator 
   2: { 
   3:     [Output] 
   4:     public int Add(string value) 
   5:     {  
   6:         ... 
   7:     } 
   8: }

Figure 2. Calculator class with Output aspect applied

The problem with this solution is the way PostSharp works. It does all its magic as a post compilation step so everything is pretty much static. This is a problem in a testing scenario when we need to inject the mocked console in this case, hence the smelly ContainerFactory.Current stuff.

Another problem with this solution is that in part three, the console app, we need to disable or override what is outputed. This ends up becoming a static mess which did not feel right at all. If you have another view on this please leave a comment.

Step 2. Create a program (test first)that uses string calculator, which the user can invoke through the terminal/console by calling “scalc ‘1,2,3’” and will output the following line before exiting: “The result is 6”

For step 2 and 3 I thought a bit about refactoring to a UI design pattern like MVP, MVC or MVVM but finally decided to drop it, mainly because I didn’t know any framework like that for console applications. If the application grows I think this is the right way to go, but for the known requirements it’s an overkill, especially considering how small the solution is.

Step 3. Instead of exiting after the first result, the program will ask the user for“another input please” and print the result of the new user input out as well, until the user gives no input and just presses enter. in that case it will exit.

I played around a bit with SpecFlow, which has ha free form Given/When/Then specification syntax, on the later part of the kata. However, I felt that I lost velocity so I dropped it as well. Maybe, if I had some way of conducting complete acceptance testing through a real console, I would have pursued this further. It was simply too much to write, for example:

Scenario: Prompt user for another input
	Given a new string calculator
	And the user has entered: a valid input
	When the program has outputed The result is 1
	Then the user is prompted for another input 
Scenario: Quit on empty input
	Given a new string calculator
	And the user is prompted for another input
	When the user hits enter
	Then the program should exit

Figure 3. SpecFlow Feature specification for the console app

If you have any thoughts, comments, suggestions, or any other feedback please leave them below or ping me on twitter.

Comments (4) -

  • Sal

    2/8/2014 7:10:30 PM | Reply

    Really nice job!

    I almost went to the IConsole abstraction approach, but Roy's suggestion to use Console.SetOut and supply a StringBuilder based writer seemed cleaner. I hate adding unnecessary abstractions just for the purpose of testing. With the builder, you can just verify that it contains the expected output from the calculator. You can't really verify that a certain method was called like with Moq, but I think comparing the text is equivalent.

    This seems like less friction to me. What do you think?

    • Rickard

      2/8/2014 7:11:30 PM | Reply


      I tried the Console.SetOut path as well but I had some trouble with it. I’m using TestDriven.NET as test runner, which uses standard output to log the test result to the output window. I found that the test runner messed up the output and made the assertions fail for the wrong reasons.

      However, I consider this path to be a border line integration test. I’d rather have no dependency on System.Console at all in my logic classes and I don’t mind a thin abstraction like IConsole. For example, it is much easier to abstract, rename or refactor it, when the requirements change.

      On the other hand, I prefer state based testing over interaction based testing, so had I been successful in using Console.SetOut I would probably have done them both, e.g. a few, end to end, integration tests with Console.SetOut as complement.

      Thanks for your feedback!

  • Sal

    2/8/2014 7:16:14 PM | Reply

    Good point on the fact that a thin abstraction allows for more flexibility when requirements change.

  • Sergej Koščejev

    2/8/2014 7:17:43 PM | Reply

    One thing I didn't like in the kata is that you are (implicitly) testing the calculator as part of the tests for the console output. What if the requirements change so that the calculator should perform multiplication instead of addition and the default output should be 1? You'd have to change every test including the output ones which don't care about the algorithm at all.

    You also mentioned you tried out AOP, this seems a bit overkill to me here, but I have another proposal: perhaps output could be cleanly separated from the logic by using a delegate or a wrapper class that would call the calculator and output its result?

Disclaimer: The opinions expressed herein are my own personal opinions and do not represent my employer's view in any way.