Dear Atul,
Dear passionate-about-unit-testing-developers ;-),

Sorry for the delayed replay, I wanted to take my time to write down my thoughts hoping to open a discussion. Actually I’m not addressing the new tests for BinaryReader directly, I’m addressing the general topic and concepts after studying the tests both of us have written. I would say they are quite different and quite interesting ;-).

Among the unit tests I have written or seen I think I can identify different types:

• black box tests use the public interface to test a class
• white box tests use the private interface and knowledge about the implementation to test a class
• tests that test single functions of a class
• tests that test partial/complete lifecycles by calling a sequence of functions

Now I am wondering what tests and how many of which type should be written to achieve a good coverage and make us fearless about changing the existing code ;-).

If we go the TDD way, we will first concentrate on the interface of the class and its behavior prior to writing the implementation. This would mean that the first test we’ll most probably intuitively get to write are black box ones. Depending on the type of the class these would be single function (e.g. parse a packet) or lifecycle (e.g. buffering a packet in a PacketSession and writing it in a Connection) tests.

The advantage of black box tests is that they allow us to drastically modify the internals of a class while keeping the interface. In this case the black box tests will remain compilable and semantically correct. The black box tests will continue to assert the same contract (see Design by Contract). If we had white-box test mostly, then we would have had to modify both the tests and the implementation. We should be very cautious about modifying existing tests because the new code may run OK with the modified tests, but the old client code may not work anymore. In such a case we would probably get other tests to fail though.

I think white box tests are quite important as well and are a good addition to black box tests. My guess is that black box tests should be simple, yet strict enough, whereas white box tests should be more extensive and focus on the boundary conditions identified in the implementation. If a function from the interface is implemented by three private functions then it would probably make sense to test each of the functions separately.

My guess about black box tests would be that they should tests different scenarios from an object’s lifecycle. Such tests would be perfect examples for (developer) users how the class can be used. White box test should then maybe concentrate on testing single functions. Lifecycle tests would provide some assurance that the single tested functions also work together in different scenarios.

Would anybody have an idea how to “measure” how good a class is tested? What about how many tests are “enough”? What about tests, which are redundant?

I hope some of you would share some thoughts or experiences. I really want to understand unit tests better and be able to write good tests. Unit tests are clearly an extremely powerful weapon, one which should be used wisely in order to exploit its full power.

Best regards,

Peter

P.S. If you had the patience to read the whole post, which ended up being quite long, then here is a small two minute dessert for you ;-)

http://video.google.com/videoplay?docid=6960295146523698319

atul wrote:Dear all

Here are some more tests I added…

Btw, I am reading a fantastic book “Agile Java” by Jeff Langr. It is a fascinating read…
Jeff says “… one big reason for writing tests is so you can change the code with impunity. Unit tests allow you to fearlessly change code…”

Very true ;-)

( This book, btw, teaches TDD very well … I will write a separate post on TDD, and about Bob Martin’s A Programming Episode, tomorrow ;-)
– cheerio atul