Software People Inspiring

The World-Famous Legacy Code Singleton Fudge

When refactoring legacy code that relies on static methods to access external systems - for example, for data access - our first goal is usually to make the code unit-testable. Therefore, we seek to invert that dependency on a static method to make it substitutible.

I demonstrated in previous blog posts how we do this in fairly simple situations. The dance is pretty straightfoward: you turn the static method into an instance method and then do a "Find and Replace" to swap references to the class that method is on into with "new ClassName().", so the target of invocation is now an instance. We then give client code a way to do the old polymorphic switcheroo by injecting that instance into, say, the constructor of the class where it's being used.

As easy as cake.

What often comes up though is a more complex, and less than ideal situation. What if our static method is being accessed by many, many classes? We'd have to introduce dependency injection into every class that uses it, and then - if those classes are some way down the call stack - into the every link in the chain to pass references from the top down to where they're used. This could be a lot of work, and I've not found a quick automated way of doing that. And what if - horror of horros - the methods that access our static data access method are also static?

Take this example. Here's a pretty nasty data access class that offers static methods for updating and retreiving object state to an external database.



Imagine that -for reasons best known to themselves - the developers use these methods inside every single business object in their middle tier. Like:



Our goal is to get decent automated unit test assurance in place quickly, so we can then safely set about refactoring this whole can of worms properly.

In this situation, I've often used a fudge to get my tests in place. The fudge being to deliberately introduce a Singleton. (Gasp!)



I first extract a new class that contains the implementations of the Update and Fetch methods, and leave delegate methods in our original DataAccess class. Then I extract an interface from this new DataAccessImpl class (IDataAccess) so we can make those methods polymorphic.

Here comes the fudge - I then introduce a static method for setting the data access implementation at runtime. so our unit tests can set it as a mock or a stub, and our production code can set it once as the real McCoy.

Yes, pretty rank. But we've made our application logic unit-testable at a relatively low expense, and can now start writing those tests that will be the safety net for unpicking this whole mess once and for all. To unpick it first and then write the tests would be too risky, in my experience.

Don't Sell Your Customer Short With Code Quackery

There was a time when nobody had heard of "germs".

Back in the days before microbiology, life was - for most people - short and brutal. And folk believed all sorts of nonsense about what was causing their illnesses and how to cure themselves, most of which was about effective as eating scraps of paper with the word "cure" scribbled on them.

Then, one day, someone took a look at some "clean" water under a new invention - a sort of backwards telescope - and saw millions of tiny little animals swimming about in it (and, no doubt, peeing and pooing in it, too; the dirty little monkeys!)

We now understand that these single-celled organisms are able to invade our bodies via various conveniently-placed orifices and do unspeakable things to the cells inside us, producing symptoms ranging from a mild case of the lergy all the way up to severe bouts of chronic death.

We've learned that we can "bug-proof" our bodies against the effects of these nasty little critters by exposing our immune systems to milder, more polite versions of them. And, apart from one or two slightly ill-informed swimsuit models, most right-minded people accept that immunisation is the way to go if we want to live long, happy, death-free lives.

If you were to fall through a time warp and end up in the days before microscopes and microbiology and try to explain to someone with, say, a bad head cold that their symptoms are caused by tiny invisible creatures attacking the insides of their bodies, they wouldn't have believed you.

Head colds are obviously caused by demonic possession, they'd say, before promptly having you shipped off to the funny farm to have holes drilled into your mind by part-time hairdressers.

As software developers, we live in pre-enlightenment times where our customers and our managers are concerned.

Most non-technical stakeholders in software applications are well aware of the symptoms of poor code quality, but our attempts to convince them that schedule delays, cost overruns, high bug counts, spiralling support and maintenance costs and, ultimately, product death can be caused by tiny, invisible code problems like missing unit tests, switch statements and copy-and-paste inheritance have largely failed.

Programmers like me have observed these software germs under our microscopes and witnessed them attacking the cells of our application, turning young healthy modules into wheezy old decrepit blobs. And we've seen how, as the infection spreads, symptoms start to manifest at the system level and beyond. We have seen entire businesses killed by code quality problems.

But the average customer or senior manager is often unwilling to take it on trust that the problems they're suffering could have any connection to whether or not programmers chose meaningful variable names, or broke down complex functions to make them simpler and more testable.

They don't understand that code needs to be immunised against the worst of these problems with high automated test assurance and continuous monitoring of code quality. Nor do they understand the need to maintain a healthy immune system through practices like refactoring - the programming equivalent of your 5 a day.

They're currently happy to continue with their superstitions and software quackery - things they can see and believe in, like hiring more programmers or creating more documentation, even if there's no hard evidence that they work, and plenty of evidence that they make the symptoms worse.

We'd do well to remember that we are the doctors and they are the patients. We must stop humouring them and asking them what treatment they want us to give them, and instead focus on proper and thorough diagnosis, and offering them treatments that we know from evidence tend to work.

They may reject our advice and seek the help of witch doctors and voodoo, but the solution is not to throw up our hands in defeat and become witch doctors ourselves. Every time we do that, we reinforce their superstitions and our hypocrisy is paid for with the blood of a million dead code bases and hundreds of billions of pounds a year in lost business opportunities.

The Right Way To Do Code Reviews Is Not To Do Code Reviews

Code reviews are up there with major surgery and moving house among life's stressful experiences.

The typical code review, based on my own experience and war stories from friends, seems to be that the team gets together in a windowless, airless room with a copy of the code, and expresses opinions about what other people have written. And, as we all know, anything that involves programmers expressing opinions is unlikely to end well.

Adding insult to injury, code reviews tend not to lead to actual improvements in code or design quality. It's rather like being judged on X Factor: the judges have their say, but since music is actually not discussed in any meaningful or constructive way, you leave non the wiser as to what to change.

It doesn't have to be this way, of course. here are my tips for more constructive and productive code reviews:

1. DON'T HOLD CODE REVIEWS!

What? Well, the fact that you're having meetings about code quality is an indication that you're already doing code quality wrong. Do you have Unit Test Execution Meetings, too, where the team gets together and decides which tests are failing? Code review is a continual, ongoing activity. Not a meeting.

2. Set clear quality goals for code and design

It all gets a little less arbitrary and opinion-based when the team have agreed what it is they're actually aiming for. Are you aiming for simplicity? Then when reviewing code, you should be looking for lack of simplicity. Aiming for readability? Seek out code that is hard to understand. Aiming for scalability? Look for processes that can't be easily run in parallel, or load-balanced or clustered. Need a small memory footprint? Need fast response times? Etc etc.

3. Write tests for your quality goals

Need a small memory footprint? How small? Write a test that will fail if the footprint exceeds tolerances. Need code to be as simple as possible? Write a test that will fail when a method or class or package or system (or database schema, or config file) gets too big or too fiddly.

4. Run the tests for your quality goals at last every time you check your code in. The build is broken if any of them fail.

5. Review your code quality tests regularly. If the goals change, change the tests. But remember, raising the quality bar as the code evolves is much, much harder than lowering it. Which is why you'd best start with bar set as high as you can realistically manage, and try and keep it there.

6. Program in pairs. If code's not worth continuous review, it's not worth writing and maintaining.

Clean Code Helps Prevent Bugs

Another quick thought-dump for the weekend.

In his papers on OO design principles, Uncle Bob Martin uses the terms "rigid" and "brittle" to describe code that's hard to change and easy to break.

It probbaly needs rearticulating that there's a direct relationship between those two things: that is, code can be easy to break for the same reasons that it is hard to change.

When we look at factors that make code harder to change, we can see how this might be so.

The readability, or comprehensibility, of code is one obvious common factor. Modifying code we don't understand is a bit like performing keyhole surgery wearing dark glasses - the capacity for unwitting mistakes is significantly higher.

Complexity is another. There's no doubt that code that is more complex tends to be buggier, and the reason for this is simple: more complex code is more likely to be wrong because there are more ways for it to be wrong. Just as we're more likely to throw a seven when rolling a pair of dice, we're more likely to make a mistake when rolling complex code. The exponential explosion of logical combinations of wrong code very quickly overpowers the slow, linear growth of correct combinations. It's basic probability.

Duplication also adds scope for more errors. In particular, it adds a risk that when we need to change common, duplicated logic, we overlook one of the duplicates, leading to inconsistencies and contradictions in our code.

Unmanaged dependencies can also make code more likely to be defective. the risk that a change to one part of the software will inadvertantly cause another part (or parts) to break is magnified significantly when there are more dependencies along which effects can propogate.

If our code is simpler, more comprehensible, low in duplication and organised to minimise the "ripple effect" of changes, it will tend to be less buggy. So we should view Clean Code as not just an enabler of change, but as an aid to defect prevention.

Continuous Delivery is a Platform for Excellence, Not Excellence Itself

In case anyone was wondering, I tend to experience a sort of "heirarchy of needs" in software development. When I meet teams, I usually find out where they are on this ladder and ask them to climb up to the next rung.

It goes a little like this:

0. Are you using a version control system for your code? No? Okay, things really are bad. Sort this out first. You'd be surprised how much relies on that later. Without the ability to go back to previous versions of your code, everything you do will carry a much higher risk. This is your seatbelt.

1. Do you produce working software on a regular basis (e.g., weekly) that you can get customer feedback on? No? Okay, start here. Do small releases and short iterations.

2. How closely do you collaborate with the customer and the end users? If the answer is "infrequently", "not at all", or "oh, we pay a BA to do that", then I urge them to get regular direct collaboration with the customer - this means programmers talking to customers. Anything else is a fudge.

3. Do you agree acceptance tests with the customer so you know if you've delivered what they wanted? No? Okay, then you should start doing this. "Customer collaboration" can be massively more effective when we make things explicit. Teams need a testable definition of "done": it makes things much more focused and predictable and can save an enormous amount of time. Writing working code is a great way to figure out what the customer really needed, but it's a very expensive way to find out what they wanted.

4. Do you automate your tests? No? Well, the effect of test automation can be profound. I've watched teams go round and round in circles trying to stabilise their code for a release, wasting hundreds of thousands of pounds. The problem with manual testing (or little or noe testing at all), is that you get very long feedback cycles between a programmer making a mistake and that mistake being discovered. It becomes very easy to break the code without finding out until weeks or even months later, and the cost of fixing those problems escalates dramatically the later they're discovered. Start automating your acceptance tests at the very least. The extra effort will more than pay for itself. i've never seen an instance when it didn't.

5. Do your programmers integrate their code frequently, and is there any kind of automated process for building and deploying the software? No? Software development has a sort of metabolism. Automated builds and continuous integration are like high fibre diets. You'd be surprised how many symptoms of dysfunctional software development miraculously vanish when programmers start checking inevery hour or three. It will also be the foundation for that Holy Grail of software development, which will come to later.

6. Do your programmers write the tests first, and do they only write code to pass failing tests? No? Okay, this is where it gets more serious. Adopting Test-driven Design is a none-trivial undertaking, but the benefits are becoming well-understood. Teams that do TDD tend to produce mucyh more reliable code. They tend to deliver more predictably, and, in many cases, a bit sooner and with less hassle. They also often produce code that's a bit simpler and cleaner. Most importantly, the feedback we get from developer tests (unit tests) is often the most useful of all. When an acceptance test fails, we have to debug an entire call stack to figure out what went wrong and pinpoint the bug. Well-written unit tests can significantly narrow it down. We also get feedback far sooner from small unit tests than we do from big end-to-end tests, because we write far less code to pass each test. Getting this feedback sooner has a big effect on our ability to safely change our code, and is a cornerstone in sustaining the pace of development long enough for us to learn valuable lessons from it.

Now, before we continue, notice that I called it "Test-driven Design", and not "Test-driven Development". Test-driven Development is defined as "Test-driven Design + Refactoring", which brings us neatly on to...

7. Do you refactor your code to keep it clean? The thing about Agile that too many teams overlook is that being responsive to change is in no small way dependent on our ability to change the code. As code grows and evolves, there's a tendency for what we call "code smells" to creep in. A "code smell" is a design flaw in the code that indicates the onset of entropy - growing disorder in the code. Examples of code smells include things like long and complex methods, big classes or classes that do too many things, classes that depend too much on other classes, and so on. All these things have a tendency to make the code harder to change. By aggressively eliminating code smells, we can keep our code simple and malleable enough to allow us to keep on delivering those valuable changes.

8. Do you collect hard data to help objectively measure how well you're doing 1-7? If you come to me and ask me to help you diet (though God knows why you would), the first thing I'm going to do is recommend you buy a set of bathroom scales and a tape measure. Too many teams rely on highly subjective personal feelings and instincts when assessing how well they do stuff. Conversely, some teams - a much smaller number - rely too heavily on metrics and reject their own experience and judgement when the numbers disagree with their perceptions. Strike a balance here: don't rely entirely on voodoo, but don't treat statistics as gospel either. Use the data to inform your judgement. At best, it will help you ask the right questions, which is a good start towards 9.

9. Do you look at how you're doing - in particular at the quality of the end product - and ask yourselves "how could we do this better?" And do you actually follow up on those ideas for improving? Yes, yes, I know. Most Agile coaches would probably introduce retrospectives at stage 0 in their heirarchy of needs. I find, though, that until we have climbed a few rungs up that ladder, discussion is moot. Teams may well need them for clearing the air and for personal validation and ego-massaging and having a good old moan, but I've seen far too many teams abuse retrospectives by slagging everything off left, right and centre and then doing absolutely nothing about it afterwards. I find retrospectives far more productive when they're introduced to teams who are actually not doing too badly, actually, thanks very much. and I always temper 9 with 8 - too many retrospectives are guided by healing crystals and necromancy, and not enough benefit from the revealing light of empiricism. Joe may well think that Jim's code is crap, but a dig around with NDepend may reveal a different picture. You'd be amazed how many truly awful programmers genuinely believe it's everybody elses' code that sucks.

10. Can your customer deploy the latest working version of the software at the click of a mouse whenever they choose to, and as often as they choose to? You see, when the code is always working, and when what's in source control is never more than maybe an hour or two away from what's on the programmer's desktops, and when making changes to the code is relatively straightfoward, and when rolling back to previous versions - any previous version - is a safe and simple process, then deployment becomes a business decision. They're not waiting for you to debug it enough for it to be usable. They're not waiting for smal changes that should have taken hours but for some reason seem to take weeks or months. They can ask for feature X in the morning, and if the team says X is ready at 5pm then they can be sure that it is indeed ready and, if they choose to, they can release feature X to the end users straight away. This is the Holy Grail - continuous, sustained delivery. Short cycle times with little or no latency. The ability to learn your way to the most valuable solutions, one lesson at a time. The ability to keep on learning and keep on evolving the solution indefinitely. To get to this rung on my ladder, you cannot skip 1-9. There's little point in even trying continuous delivery if you're not 99.99% confident that the software works and that it will be easy to change, or that it can be deployed and rolled back if necessary at the touch of a button.

Now at this point you're probably wondering what happened to user experience, scalability, security, or what about safety-critical systems, or what about blah blah blah etc etc. I do not deny that these things can be very important. But I've learned from experience that these are things that come after 1-10 in my heirarchy of needs for programmers. That's not to say they can't be more important to customers and end users - indeed, user experience is often 1 on their list. But to achieve a great user experience, software that works and that can evolve is essential, since it's user feedback that will help us find the optimal user experience.

To put it another way, on my list, 10 is actually still at the bottom of the ladder. Continuous delivery and ongoing optmisation of our working practices is a platform for true excellence, not excellence itself. 10 is where your journey starts. Everything before that is just packing and booking your flights.

Slow & Dirty - A Rant (slides from SPA2011)

Okay, so here's a PDF version of my slides from today's invited rant at SPA2011. With thanks to the SPA team for allowing me to vent, and giving me booze.

Before you read it, a disclaimer: the value of metrics can go down as well as up. 'nuff said.

"Sufficient Design" - Something To Aspire To For The 99% Of Us Who Aren't Even Capable Of That

So I'm not at the Lean Software & Systems conference in California, but it's hard to miss the tweets.

Joshua Kerievsky's talk seems to have gone down very well, with many tweets about "sufficient design". The basic idea is that our software only needs to be good enough, and time and effort spent makming it better than that is time and effort wasted.

There's an implication buried somewhere in there that "software craftspeople" take quality too far, and are therefore potentially wasting time and effort making their code too good.

I completely agree in making code only as good as it needs to be. But I think maybe people - especially people who have a bit of a chip on their shoulders about craftsmanship - latch on to the idea of "sufficient design" as a justification for not bothering to make their own truly not-good-enough-and-then-some code any better.

The reality is that "good enough" code is a rare thing, and code that's better than it needs to be is almost unheard of. I've never seen code that was too good for its purpose.

The craftsmanship movement is full of people who aspire to one day write code that's as good as it makes sense to make it, and possibly to even be capable of making it better than that, just in case.

Businesses that buy into the mythology of "quick and dirty" to get something to market or to solve a problem sooner are buying into a lie. In the vast majority of cases, taking more care over quality would actually enable teams to go faster. I'm not just saying that: the weight of evidence is on my side. Studies done by IBM, the Software Engineering Institute, Microsoft and many other esteemed (and maybe slightly less esteemed) bodies have provided us with a mountain of hard data that indicates beyind any reasonable doubt that it doesn't cost more to deliver software that's more reliable and more maintainable. In many cases it actually costs less. As Robert C. Martin is fond of saying: the way to go fast is to go well.

Let's cut to the chase. I know from personal experience that when I cut corners and "kludge" a solution, it takes me longer to get it working. So if I made a "business decision" (and, seriously, when did we all turn into Alan Sugar?) to lower the quality bar, I'd be deliberately choosing to deliver a less reliable, less maintainable solution and to do it at greater cost in time and money.

Here's what I think. I think people like Joshua Kerievsky and others who talk of "sufficient design" have actually passed that point where greater quality can be delivered at lower cost. They are in a very small minority who've been applying a level of skill and discipline that does not always make sense economically.

The rest of us are probably on the other side of that curve, and we should be eyeing them enviably. We should certainly not make the mistake of thinking that their advice on sufficient design applies to us. Not yet. Not until we've passed that well-documented "sweet spot" of maximum quality and productivity.

Until that day, we should continue to focus on doing it better and learn to accept that "sufficient design", to the rest of us, is something we should aspire to.

You see, for apprentices like you and me, worrying about making our code too good is as misguided as worrying about being too good in bed. And, of course, if you give a talk about "sufficient lovemaking" to an audience of mostly men, you're going to see a lot of people nodding in agreement. In that hypothetical situation, would you presume that the guys nodding are the ones who have genuinely experienced this problem of being too good in the sack?


No, me neither.

The Dependable Dependencies Principle - Draft Paper

For over 15 years we have known of design principles to help us manage dependencies in software to limit the impact of making changes. Another consequence of dependencies has been overlooked, namely the relationship between dependencies and risk of failure in our code. The more depended-upon code is, the greater the potential impact of its failure. We should therefore desire that code this is more depended-upon be more reliable. This paper explores the relationship between dependencies and reliability, and proposes a new design principle, with a first attempt at an accompanying set of metrics, to help us limit the impact of failure.

Download the full draft paper here.

[screencast] Codemanship Refactoring Assault Course - Take 2

Took it a little further in a second attempt.

"But What If We're Building The Wrong Thing?", And Other Lame Excuses For Writing Shitty Code

I had a very interesting couple of days in Finland, hosted by those lovely folk who organise the Scandinavian Agile conference. (Which was jolly good fun, by the way.)

In the bar after the conference, I got into a familiar discussion about software craftsmanship, which goes like this (sing along if you know the words):

"But does craftsmanship matter if we're building the wrong thing?"

(I should stress the chap asking was playing Devil's Advocate, and nt advocating actually writing shitty code. But it got me thinking all the same, as I get asked this question A LOT.)

A reasonable enough question. If you've just spent a squillion dollars on software nobody's going to use then it probably doesn't matter if the code is reliable, readable, simple, modular and all those other things us craftsmen - and craftswomen - bleat about all the time.

Except... well, let's think this through.

First of all, did we know we were building the wrong thing while we were building it? If we did, then why did we build it at all?

If we didn't know that we were building the wrong thing at the time, then why would we risk the future of that software - which we hope will be successful - by cutting corners on code quality?

In the vast majority of cases, software is either worth writing well, or it's not worth writing at all.

This is especially true because, as we've learned from decades of experience and dozens of large industry studies, it doesn't necessarily take longer or cost more to write cleaner, more reliable code.

To me, it would seem irrational to choose to cut corners. Given that we can produce better code at no extra cost, and given that if code is worth writing at all then it's worth doing a good job of it, why on Earth would we choose to do a worse job?

A similar argument goes something like this:

"Sometimes you just need to do something quick-and-dirty to get it to market and learn some lessons"

I'm calling bullshit on this, too. In software development, we've found - and the jury really isn't out on this one - that the reality is actually quick-and-clean vs. slow-and-dirty, even on relatively small and simple problems.

But let's say I'm wrong. (I know, perish the thought!)

Well, let's say we did a bad job of building the wrong thing. What next? The user feedback starts flooding in, but we struggle to respond because we're devoting most of our time to bug-fixing, and making changes to the code is expensive and risks breaking existing features. And to make the software useful and usable enough, we have to make a heck of a lot of changes. So the code may likely-as-not get thrown away and we start again. But was every single line of it without value? Are we being forced to throw the baby out with the bathwater? or could we have adapted it into something more useful, had we not had such a mountain of technical debt standing in our way?

And what about our quick-and-dirty (which was probably actually slow-and-dirty, but let's pretend for a moment) prototype? Well, we'd better hope we got it 100% right with that first release, because, again, if it needs to evolve, there'll be a mountain of technical debt standing in our way.

Now maybe, in both cases, throwing the code away and starting again really is the best medicine. But wouldn't it be less risky to give the customer a choice? I've only ever seen code bases being ditched out of absolute necessity - always as a last resort on the part of frustrated and angry businesses. Our shitty code backed them into a corner.

Uncle Bob Martin, Steve McConnell and others correctly assert that the quality vs. time/effort trade-off in software development is largely a myth. And demonstrably so.

In that respect, software craftsmanship can give customers more choice by essentially giving them more throws of the dice at the craps table which is software development, and at no extra cost or risk. If we believe that most of the real value in software is discovered through feedback and learning (which is what Agile was all about), then craftsmanship is value-focused because it enables more feedback and for longer. It enables businesses to learn faster.