Saturday, November 24, 2007

RFC: A mock too far?

I love using mocks as it makes it possible to write unit tests for small parts of a bigger system without depending on it. Using mocks is very simple, just create a mock of an interface (or class, schhhh) , set up the expectations on it, and then execute the test. When using mocks it is easy to get 100% coverage without relying on others. But I have come to the conclusion that this can be taken too far, way too far.

I have tests that do more setting up the mock expectations than actually testing the code. This makes the tests much harder to read, and I'm having problems going back to 6 months old code and understand what a test actually tests. And in my eyes this is a big problem because unit test code should be as simple as possible.

So what I'm interested in is how should I do mock testing, without drowning in mocking expectations. Does anyone have any ideas on how to do it? Have I gone a mock too far?

As a simple example I will show my unit tests of my Clear case implementation for the Hudson CI server. (But my production code looks very similar unfortunately).

I've selected to mock out the clear tool command execution in the test, ie I dont want to issue real commands using the clear case command line tool (and I don't have a Clear case server or client at home). In the below example I want to verify that when the plugin polls the CC server for changes, then it should issue a "lshistory" method call. But to get it to the point where I can run the tests I have to mock out the Hudson dependencies, ie get the latest build, check the time stamp of it, mock a list of changes that the "lshistory" should return.

public void testPollChanges() throws Exception {
final ArrayList list = new ArrayList();
list.add(new String[] { "A" });
final Calendar mockedCalendar = Calendar.getInstance();

context.checking(new Expectations() {
with(equal("viewname")), with(equal("branch")));
classContext.checking(new Expectations() {

ClearCaseSCM scm = new ClearCaseSCM(clearTool, "branch",
"configspec", "viewname", true, "vob", false, "");
boolean hasChanges = scm.pollChanges(project, launcher,
workspace, taskListener);
assertTrue("The first time should always return true", hasChanges);

As you can see the mock expectations take up at least two thirds of the test. Of course I can refactor the tests by adding helper methods that will set up the expections, but many times the expectations are similar but not equal.