Hi Petr,

The article you referred to is interesting indeed. I liked the calibration comparison :-) My understanding in regards to your question was more along the lines of the “do stuff in small increments, so that less could go wrong” kind of thinking. I consider the process of writing a test and an implementation a very iterative one where you can’t (or is error prone to) write the complete test/implementation at once (especially if you don’t know how exactly to do it yet).

I’ll give you an example how I implemented TcpSocketBuffer. Please have a look at TcpSocketBufferTest and more specially at the order in which the test functions are listed – I think that is pretty much the order in which I incrementally wrote the tests and their corresponding implementation. Note also that some of the tests took me more than one iteration (red/green/refactor). The cases when I needed more iterations was when I needed a few steps before I reach the final version of the test.

Let’s start with the very first test function:

	void testWritingZeroBytesAlwaysSucceeds()
	{
		refWriteBytesSucceeds (QByteArray(), true);
		refWriteBytesSucceeds (QByteArray(), false);
	}

It expresses that writing a QByteArray() to the buffer with the flush flag set to true will succeed and that writing a QByteArray() to the buffer with the flush flag set to false will also succeed.

But that was not the initial version of the test! The first version probably was something like:

	{
		call (buffer->write (QByteArray(), true))
		.willReturn (true);
	}

I think this test even failed because I was not returning a value in write(), so just some random bool was returned which happened to be false :-) I’ve even had tests that crash because no value is actually returned and a QByteArray was supposed to be returned – also a kind of red bar :-)

The simples way to make the test run is to hardcore the return value of true. This implementation would allow even the first (new) version of testWriteBufferSizeBytesAtOnce() to pass without any further modifications to the implementation:

	{
		call (buffer->write (QByteArray (WriteBufferSize, '1', true))
		.willReturn (true);
	}

adding the new testWriteMoreThanBufferSize() would fail however because we always return true. The simplest “fix” would be:

	{
		if (bytes.size() < = writeBufferSize)
			return true;
		else
			return false;
	}

Abstracting yourself from your “implementation” you continue to think about black-box tests that enumerate the different usage scenarios (not a single function call but a sequence of calls the produce certain effect or result) and their important cases. That leads to a new and more precise version of testWriteBufferSizeBytesAtOnce():

	{
		call (buffer->write (QByteArray (WriteBufferSize, '1', true))
		.willReturn (true);
		call (buffer->write (QByteArray (“1”, true))
		.willReturn (false);
	}

We try to write WriteBufferSize bytes first, then try to write 1 more byte, which should return false but it doesn’t! The simplest “fix” would be:

	{
		if (writeBuffer.size() + bytes.size() < = writeBufferSize) {
			writeBuffer += bytes;
			return true;
		}
		else
			return false;
	}

At this point you might already be worrying about the code duplication in your tests. Extract Method seems like a good idea:

	void refWriteBytesSucceeds (const QByteArray &bytes, bool flush = true)
	{
		call (buffer->write (bytes, flush))
		.willReturn (true);
	}
	...
	void testWritingZeroBytesAlwaysSucceeds()
	{
		refWriteBytesSucceeds (QByteArray(), true);
	}

Hey, we’d better also check the case when flush is false:

	void testWritingZeroBytesAlwaysSucceeds()
	{
		refWriteBytesSucceeds (QByteArray(), true);
		refWriteBytesSucceeds (QByteArray(), false);
	}

I could continue the example (which is a bit speculative because I don’t quite remember the exact way I went, but that is the way I would have done it now) over at least a few more pages and I would do so if you want me to! I think you already got the mechanism though.

Now the reasoning! Once you start working on a new piece of functionality you don’t know how exactly to use it, neither how to implement it. Rather then inventing and implementing stuff, you let the process guide you. You could use bigger steps and they would also do the job! The benefit of doing small steps is that you make a lot of checkpoints or reference points along the way. If a small change you do “breaks” the implementation, you will figure that out if at least one of the existing tests fails. If none fails, then one of the ones you are about to write will fail. If a small change breaks your tests, then the broken ones should fail. It is important to never change tests and implementation at the same time because you loose your reference point - breaking both tests and implementation could result in all tests still passing even though neither is correct!

Take revision control as an example - each time you do some piece of work that you consider OK you commit. This way you know that even if you mess things up you could easily revert to a good state. It’s the same concept with unit tests and TDD - they just tell you what is a good and what is a bad state.

The number two comment to the article you referred to was an example. If you compare it to my example above you’ll probably notice that the example in the article’s comments uses bigger steps – I sometimes make bigger steps too – but that is when I believe I know what I’m doing. If I have no idea yet what tests and implementation would look like, then I will take the smallest steps possible. I think that the more you do TDD, the more comfortable you would feel about doing small steps. You will also get a feeling when you can afford to do bigger ones and if you see the step was too big you can always “revert” to the checkpoint and try a smaller one.

Short comment about black-box and white-box tests. I’ve been thinking a lot about that! I try to come up with the black-box tests first. If you think about it you don’t have much choice actually as your initial implementation is mostly fake. I also think that for quite a long time I had a misconception about what white-box tests actually are. I used to think that white-box tests are ones you write after studying the implementation. One problem with that thinking is that if you think about the tests while looking at the code you are likely to write tests that pass but should have failed – that is because you wrote tests “derived” from a faulty implementation. My understanding now is that one should try to write an extensive suite of black-box tests consisting of at least one test for each usage scenario. Then, white-box testing could be used to choose more tests for each of the usage scenarios in order to achieve a better test coverage. I think that white-box tests are more about coverage (inputs producing a certain flow through the implementation and the expected output), whereas black-box tests are more about the usage scenarios.

I hope you find this reply helpful! It’d be great if we’d discuss more on the topic!

Btw, could you point to locations in the Introduction to TDD developer manual where you think some more information/clarification would be helpful? For example, places where you had questions which were never answered.

Best regards,

Peter