One thing that became pretty darned obvious to us early on with these apps is that a solid regression testing system is a necessity.

One area I felt the pain was in some of the work I did with refactoring the first version of the Channel Manager.  I’d fix a bug on Windows and introduce a bug on Mac OS X, then go edit stuff on a Mac and break it on Windows, and so on. Putting in a file abstraction layer to smooth over some of the differences between Windows and the rest of world helped, but one truth remains: for non trivial development, you need to have regression testing in place. Without it, it’s tough to know what you’re breaking as you develop new features.

We have a lot of regression testing in place in the core of Iguana, but we don’t have a really nice blueprint for how to do unit testing with Lua in the Iguana translator. So when our QA engineer, Wade, announced he’d run out of work last week, I taught him how to write an Iguana web app and pointed him at the spinner library Bret wrote for our first regression testing application. In less than a day Wade had a working prototype of what has the potential to be an amazing regression testing system.

The spinner library basically makes it a doddle to spin up a translator instance on a remote Iguana instance, populate it with Lua code, and breathe the beastie into life (Prometheus unbound anyone 😉 ).

Wade was able to do is clip that library together with a unit test I wrote for the file abstraction library and build an app that at the click of a button:

1. Loads the unit test off disk.
2. Installs it on to a mini farm of Iguana servers running on Windows, Linux and Mac.
3. Call the unit test on each server.
4. Brings back the results and reports them.  Ta da!

Obviously it’s a starting point, since we want to make this run automatically as commits come into the repository. But we should have no trouble doing all sorts of neat things with it. Because it’s so easy to grab data using HTTP, it should be possible to point it at an arbitrary repo with Iguana channels in it and have it run tests.

The application is in the Iguana app repository now.  Wade has made it configurable so you can add as many Iguana hosts to test against as you like:

Wade has also styled the output to go into a data table:

We may change this over to use a tree control later.  Wade is working on adding support for triggering the unit test suite when GIT commits are made.

You can try out this tool today from the Iguana app repository using the Channel Manager.  There are two channels called Unit Test – 1. GUI and Unit Test – 2. Tests that you will need to grab to try it out.