September 21, 2007

...Learn TDD with Codemanship

Getting The Most Test Coverage With The Least Test Code

Legacy code!

I bet that scared you, didn't it? It certainly scares the crap out of me.

Okay, here's a problem for you to ponder:

You have inherited a buggy, unmaintainable code base with 0% unit test coverage and have been asked to "sort it out" as quickly as possible.

Quite rightly, your first instinct is to get some smoke tests in place place to provide enough coverage to at least get you started. Time is short, and the pubs are nearly open, so you need to get as much coverage with as little test code as possible.

Do you:

a. Test the most critical methods? (the methods that most other methods are directly or indirectly depending upon?)

b. Test the most "promiscuous" methods? (the methods that call the most other methods (directly or indirectly)?

I think it's b. But it's a bit more complicated than that, I suspect. Firstly, more promiscuous methods tend to be larger and more complex, and that means they may require more test code to get meaningful results. Secondly, should I factor in the amount opf code contained in the methods being called as well as their promiscuity?

What would be a useful metric to help us pinpoint these methods?

I've noodled with NDepend, which has Method Efferent Coupling ("efferent" refers to outgoing dependencies - i.e, methods I call, as oppose to methods that call me), which is a start. But I want a metric that's more like centrality (e.g., like Google Page Rank), only in reverse.

BTW, NDepend has a Method Rank metric, which would be very useful for pinpointing the most critical methods. Arguably, these would need to be the most reliable methods, since a bug in a highly ranked method could be invoked by many other parts of the code. But it doesn't solve our problem of getting the most coverage with the least test code. Maybe after we've got our smoke tests in place, we should turn our attention to these?

And should I factor code size into that? And what about complexity? Should I take into account the number of paths through each method, and the number of tests it would take to reach those paths?

It's an interesting problem. What do you think?
Posted 10 years, 10 months ago on September 21, 2007