Introducing SodaTest: Spreadsheet-Driven Integration Testing for Scala and Java

I'd like to reveal what I've been working on in my spare time for the last couple of months.

The Announcement

SodaTest: "Spreadsheet-Driven Integration Testing", is an open-source framework for creating Executable Requirements for Integration and Acceptance testing in Scala and Java.

The impetus for starting this project was to attempt to create a tool that improves on Ward Cunningham's Framework for Integration Testing, "FIT". As an 'Executable Requirements' testing tool, it can also be considered as an alternative to Fitnesse, Concordion, RSpec, Cucumber, JDave, JBehave, SpecFlow, and Thoughtworks' Twist.

The Background

We used FIT in anger when I first arrived at my current workplace over four years ago. Since then I've watched the team become first dissatisfied with it, then scathing and fearing of it, before abandoning development of new FIT tests in place of Integration Tests written in JUnit. Nevertheless, we all still felt that Executable Requirements were a Good Thing and made a couple of failed restarts in trying to get back on the wagon.

From my own perspective, many of our issues (but not all) were to do with the tool. I identified two main issues that I'd seen hurt people over and over: the input format and the programming model. So I set out to create something that solved these two problems while remaining fairly close to what I think is a great foundation in FIT.

The Result

The result is a tool I've called SodaTest. It uses spreadsheets, in the form of comma-separated value (CSV) files, as the input format. The contents of the spreadsheet are basically small individual tables, much like what would appear in a FIT test. There is a required but simple structure to the tables and a minimum of special words and symbols to provide some context to the parser.

I also aimed to keep the programming model as simple as possible. I tried to make sure there is only noe sensible way things could be done, so as not to confuse developers with options. I've gone to lengths to ensure developers of Fixtures will have to do a minimum of work by having the framework do a lot of boilerplate work for them. Lastly, I've structured and named everything in a way that I believe will guide developers in the right direction (by which I mean away from mistakes that I have made in the past while writing FIT tests).

I've also taken the time to add a little sugar here and there that I thought was missing from FIT, for example integration with JUnit and a more comprehensive set of built-in coercion strategies for converting strings into strong types.

I'm quite pleased with the result. (I wouldn't be telling people about it if I wasn't!) Yesterday I released version 0.1 of SodaTest and people should be able to use this first release of SodaTest to create tests that do almost everything that FIT could do, but with less effort in creating the tests, writing the Fixtures, and getting the whole lot executing in their environment.

You can find out more about the motivation for SodaTest and the features it includes by reading the README on GitHub.

While SodaTest is written almost entirely in Scala (and the most benefit will be gained by using Scala as the language for writing Fixtures), I've also written the sodatest-api-java extension that allows SodaTest Fixtures to be written in Java. There is one limitation where Scala is (currently) still needed, but I reckon 95% of Fixtures should be able to be written entirely in Java if that is something you care about.

The Next Steps

The next steps for SodaTest are clear to me: Dogfood, Broadcast, Feedback and Evolve.

I want to convince my team at work to start experimenting with Executable Requirements again using my home-grown tool; I'd love it if other people in the Scala and Java communities could download this tool and give it a little tryout during their next iteration or two; I want to hear Feedback from people about what's good about SodaTest, what needs more work and whether there's parts that are just plain horrible; and, if the feedback is positive enough to consider SodaTest a preliminary success, I want to continue improving it in the areas where it's still holding people back.

There is already a Roadmap of possible features to add, but now it's really time to get it in people's hands and find out from users what is the next most important thing it needs to do.

Try It Out!

If you're a Scala or Java software developer and Executable Requirements are either a passion of yours or something you've been wanting to try out, why don't you give SodaTest a try? You don't have to commit to it, just write a couple of tests with it, get them running, then passing, and send me some feedback. All feedback is useful, even if you think it sucks! (As long as you tell me why.)

To get started with SodaTest, I suggest you:

• Read the SodaTest Project README
• Use Git to clone and have a look at the examples included with the project
• Download SodaTest and write a test. If you're using Maven, you can use the repository and dependencies below, otherwise you can download JARs from GitHub.

<repositories>
   <repository>
       <id>sodatest-github-repo</id>
       <name>SodaTest repository on GitHub</name>
       <url>http://grahamlea.github.com/SodaTest/m2repository/ </url>
   </repository>
</repositories>
<dependencies>
   <dependency>
       <groupId>org.sodatest</groupId>
       <artifactId>sodatest-api</artifactId>
       <version>0.1</version>
       <scope>test</scope>
   </dependency>
   <dependency>
       <groupId>org.sodatest</groupId>
       <artifactId>sodatest-runtime</artifactId>
       <version>0.1</version>
       <scope>test</scope>
   </dependency>
</dependencies>