October 15, 2016
If Your Code Was Broken, Would You Know?I've been running a little straw poll among friends and clients, as well as on social media, to get a feel for what percentage of development teams routinely (or continuously) measure the level of assurance their automated regression tests give them.
For me, it's a fundamental question: if my code was broken, would I know?
The straw poll suggests that about 90% of teams don't ask that question often, and 80% don't ask it at all.
The whole point of automated tests is to give us early, cheap detection of new bugs that we might have introduced as we change the code. So profound is their impact, potentially, that Michael Feathers - in his book Working Effectively With Legacy Code - defines "legacy code" as code for which we have no automated tests.
I've witnessed first-hand the impact automating regression tests can have on delivery schedules and development costs. Which is why that question is often on my mind.
The best techniques I know for "testing your tests" are:
1. A combination of the "golden rule" of Test-Driven Development (only write source code if a failing test requires it, so all the code is executed by tests), and running tests to make sure their assertions fail when the result is wrong.
2. Mutation testing - deliberately introducing programming errors to see if the tests catch them
I put considerable emphasis on the first practice. As far as I'm concerned, it's fundamental to TDD, and a habit test-driven developers need to get into. Before you write the simplest code to pass a test, make sure it's a good test. If the answer was wrong, would this test fail?
The second practice, mutation testing, is rarely applied by teams. Which is a shame, because it's a very powerful technique. Code coverage tools only tell what code definitely isn't being executed in tests. Mutation testing tells us what code isn't being meaningfully tested, even if it is being executed by tests. It specifically asks "If I broke this line of code, would any tests fail?"
The tools for automated mutation testing have improved greatly in recent years, and support across programming languages is growing. If you genuinely want to know how much assurance your tests can give you - i.e., how much confidence you can have that the code really works - then you need to give mutation testing a proper look.
Here are some mutation testing tools that might be worth having a play with:
Java - PIT
C# - VisualMutator
C/C++ - Plextest
Ruby - Mutant
Python - Cosmic Ray
PHP - Humbug
October 14, 2016
101 TDD Tips, via @CodemanshipIf you're on the Twitters, you can follow my "101 TDD Tips" series through the hashtag #101TddTips
October 12, 2016
Evil FizzBuzzThe major upgrade to the Codemanship TDD training workshop launched at the weekend, and - while much has changed - what would a TDD workshop be without FizzBuzz?
But the practical exercises in TDD 2.0 have been turbocharged...
Evil FizzBuzz (or "Continuous FizzBuzz") is my take on a Continuous Integration kata - we just don't do enough of those, right? - where each par of the FizzBuzz problem has been isolated in its own user story.
Teams must split up the work, with each developer (or pair of developers) picking up one user story at a time, test-driving the code just for that story, and continuously integrating with the other members of the team who are simultaneously working on other stories.
Committing to the same GitHub repository, with a cloud build pipeline set up to compile the code and run all the tests, this is a challenge in not tripping over each others' feet. It takes effective coordination and communication. Again, something we just don't practice enough.
The goal is to complete it quickly without breaking the build. To make it even more interesting, assign a single developer to take the challenge by themselves and see who finishes faster.
TDD 2.0 - London, Jan 11th
Here's an idea for what the boss could get your team for Christmas; the first full TDD 2.0 public workshop is happening in London on January 11-13.
There's a 1, 2 and 3-day option, and every attendee gets a copy of the exclusive new TDD book to take away and continue their TDD journey with.
October 11, 2016
Code Quality is a Requirements IssueThe most fundamental aspect of Agile Software Development is responding to change. Whatever software we deliver today, the real value is in what we can learn from that, so we can deliver something even better tomorrow.
It's nature's search algorithm: evolutionary design. So it should come as no surprise that the cost of changing software is pivotal to our ability to succeed. The more change costs, the less change we can accommodate. The less change we can accommodate, the slower we learn. Simples.
In this respect, the cost of changing software can be thought of as a requirements discipline - ever bit as much as user stories and Specification By Example. Indeed, if we're being genuinely Agile, iterating is the requirements discipline, and everything else is about tweaking our seed values to make the search a little more efficient.
As so many have commented in recent years, failing on code quality means failing on agility. You can Lean ScrumBan all you like. But there's more to Agile than turning up to planning meetings.
October 9, 2016
TDD 2.0 LaunchesYesterday saw the launch of the new Codemanship TDD training workshop in London. Thirty keen code crafters joined in South Wimbledon to put themselves through their TDD paces, and get their hands on my new book, exclusive to workshop attendees.
It was a packed day - maybe a bit too packed, with everything we crammed in - and everyone rose to the challenge admirably. As always, it's enormously rewarding to spend a day with developers who care about their craft.
The full version of the workshop comes in 1, 2 and 3-day varieties, with more time to explore each exercise, and at a more leisurely pace. Every attendee gets a copy of the 200-page book, which covers everything from TDD basics (red-green-refactor) to advanced topics like property-based testing and non-functional TDD. Further reading's clearly signposted, making the book and the workshop a great gateway into a lifetime of TDD study and practice, as well as a pick-me-up for experienced TDD practitioners, with many ideas not usually covered in TDD books and courses.
You can find out more about the workshop, and download the first seven chapters of the book for free, by visiting http://www.codemanship.com/tdd.html
September 30, 2016
Software Development Doesn't Scale. Dev Culture DoesFor a couple of decades now, the Standish Group have published an annual "CHAOS" reported, detailing the results of surveys taken by IT managers about the outcomes of IT projects.
One clear trend that emerged - and remains as true today as in 1995 - is that the bigger they are, the harder they fall. The risk of an IT project failing outright rises rapidly with project size and cost. When they reach a certain size - and it's much smaller than you may think - failure is almost guaranteed.
The reality of software development is that, once we get above a dozen or so people working for a year or two on the same product or system, the prognosis does not look good at all.
This is chiefly because - and how many times do we need to say this, folks? - software development does not scale.
If that's true, though, how do big software products come into existence?
The answer lies in city planning. A city is made up of hundreds of thousands of buildings, on thousands of streets, with miles of sewers and underground railways and electrical cabling and lawns and trees and shops and traffic lights and etc etc.
How do such massively complex structures happen? Is a city planned and constructed by a single massive team of architects and builders as a single project with a single set of goals?
No, obviously not. Rome was not built in a day. By the same guys. Reporting to one boss. With a single plan.
Cities appear over many, many decades. The suburbs of London were once, not all that long ago, villages outside London. An organic process of development, undertaken by hundreds of thousands of people and organisations all working towards their own unique goals, and co-operating or compromising when goals aligned or conflicted, produced the sprawling metropolis that is now London.
Trillions of pounds has been spent creating the London of today. Most of that investment is nowhere to be seen any more, having been knocked down (or bombed) and built over many times. You could probably create a "London" for a fraction of the cost in a fraction of the time, if it were possible to coordinate such a feat.
And that's my point: it simply isn't possible to coordinate such a feat, not on that scale. An office complex? Sure. A housing estate? Why not? A new rail line with new train stations running across North London? With a few tens of billions and a few decades, it's do-able.
But those big projects exist right the edge of what is manageable. They invariably go way over budget, and are completed late. If they were much bigger, they'd fail altogether.
Cities are a product of many lifetimes, working towards many goals, with no single clear end goal, and with massive inefficiency.
And yet, somehow, London mostly looks like London. Toronto mostly looks like Toronto. European cities mostly look like European cities. Russian cities mostly look like Russian cities. It all just sort of, kind of, works. A weird conceptual cohesion emerges from the near-chaos.
This is the product of culture. Yes, London has hundreds of thousands of buildings, designed by thousands of people. But those people didn't work in bubbles, completely oblivious to each others' work. They could look at other buildings. Read about their design and their designers. Learn a thousand and one lessons about what worked and what didn't without having to repeat the mistakes that earned that knowledge.
And knowledge is weightless. It travels fast and travels cheaply. Hence, St Petersburg looks like the palaces of Versailles, and that area above Leicester Square looks like 19th century Hong Kong.
Tens of thousands of architects and builders, guided by organising principles plucked from the experience of others who came before.
Likewise, with big software products. Many teams, with many goals, building on top of each other, cooperating when it makes sense, compromising when there are conflicts. But, essentially, each team is doing their own thing for their own reasons. Any attempt to standardise, or impose order from above, fails. Every. Single. Time.
Better to focus on scaling up developer culture, which - those of us who participate in the global dev community can attest - scales beautifully. We have no common goal, no shared boss; but, somehow, I find myself working with the same tools, applying the same practices and principles, as thousands of developers around the world, most of whom I've never met.
Instead of having an overriding architecture for your large system, try to spread shared organising principles, like Simple Design and S.O.L.I.D. It's not a coincidence that hundreds of thousands developers use dependency injection to make external dependencies swappable. We visit the same websites, watch the same screencasts, read the same books. On a 10,000-person programme, your architect isn't the one who sits in the Big Chair at head office drawing UMLL diagrams. Your architect is Uncle Bob. Or Michael Feathers. Or Rebecca Whirfs-Brock. Or Barbara Liskov. Or Steve Freeman. Or even me (a shocking thought!)
But it's true. I probably have more influence over the design of some systems than the people getting paid to design it. And all I did was blog, or record a screencast, or speak at a conference. Culture - in this web age - spreads fast, and scales rapidly. You, too, can use these tools to build bridges between teams, share ideas, and exert tacit influence. You just have to let go of having explicit top-down control.
And that's how you scale software development.
September 29, 2016
The Afternoon I Learned to Fly a PlaneWhen I was younger, my Dad was an amateur pilot. One time, he took us up for a flight, and when he'd got us up in the air and on our way - on a lovely calm and clear day - he let me take the controls for a little while. I remember after we landed being cockahoop that I'd learned how to "fly a plane".
Imagine, now, that I harboured ambitions to start my own airline. The only barrier to this plan is that I'm not a pilot, and hiring pilots is expensive. But, fear not, because - remember - on that calm and clear summer afternoon I learned how to "fly a plane".
Sure, I didn't learn how to take off. Or how to land. Or how to navigate. Or what to do when visibility is poor, or when the skies are choppy. And I didn't learn what all the knobs and dials and switches do, or what it means when that particular red light is blinking and what I should do in response. I didn't learn about basic aerodynamics, or how aeroplanes work or which bits of the plane do what, and how to check that everything's working before I take off. I didn't learn about the fuel the plane uses, where to get some, and how to calculate how much fuel I might need for my journey. I didn't learn how to land in an emergency. I didn't learn about speaking to air traffic control, nor how to use the radio for any purpose. I didn't learn about flight plans, or how to read aviation charts. I didn't learn about "air corridors", or about CAA rules.
But I had flown a plane, and was therefore surely only a couple more lessons away from being a pilot. I mean, it's easy. You just hold the steering wheel and keep the nose of the plane pointing to where you want to go. Right? What's that? It's called a "yoke"? Okay, I didn't learn that either.
This is all absurd, of course. It takes many hours of experience, and much study, before you're safe to be in charge of an aeroplane. There's so much that can go wrong, and you can't just pull over on to the hard shoulder when it does. when I had control, there was a qualified, experienced pilot sitting right next to me, and - in reality - he was in control. He talked me through it all the way, and if necessary could take over at a second's notice. I just did exactly what he told me to do.
But if I were a cynical opportunist, I might exploit this initial moment of euphoria that I experienced to make me believe that I can fly a plane. I can see the ads now: "Thinking of starting an airline? Why pay for expensive so-called 'pilots' when anyone can learn to fly on our 1-day course." This would be an exemplary commercialisation of the Dunning-Kruger effect: they don't know that they haven't learned to fly a plane, and we ain't about to tell them.
What that short flight with Dad did do, though, was get us excited about learning to fly. So much so, that for a good few years it was my brother's primary ambition to be a pilot.
A course claiming to teach you to fly in a day - or even an hour - would end up in court, of course. Grown-ups know it just isn't as simple as holding the stick and going in a straight line for 5 minutes.
So, why don't grown-ups know this about writing software?
September 24, 2016
10 Great Reasons to Learn TDD1. Rising Demand - Test-Driven Development has been growing in popularity over the last decade, with demand rising by a fairly steady 6% each year here in the UK. In 2016, roughly a quarter of all developer jobs advertised ask for TDD skills and experience. It's not unthinkable that within the next decade, TDD will become a mandatory skill for software developers.
2. Better Pay - according to data on itjobswatch.co.uk, the median salary for a software developer in the UK is £40,000. For devs with TDD skills, it rises a whopping 27.5% to £51,000. Overall, developer pay in Britain has been stagnating, with a real-terms fall in earnings over the last decade. Salaries for jobs requiring TDD skills have consistently outperformed inflation, rising about 4.5% every year for the last 3 years.
3. Better Software - the jury's not out on this one. Studies done into the effects of TDD show quite conclusively that test-driven code is less buggy - by as much as 90% less buggy. But that's not the only effect on code quality; studies done at the BBC and by Keith Braithwaite of Zuhlke Engineering show clear gains in code maintainability, with test-driven code being simpler and containing less duplication. Other studies show test-driven code tends to have lower class coupling and higher class cohesion.
4. Faster Cycle Times - the Holy Grail of Continuous Delivery is code that's always shippable, so when a feature's ready, the business can decide to release it straight away, instead of having to wait for a long and tortuous acceptance testing and stabilisation phase to make it fit for purpose. TDD, done well, delivers on this promise by continuously testing the code to ensure it always works. It's no coincidence that all the teams doing Continuous Delivery successfully are also doing TDD. TDD enables Continuous Delivery.
5. Sustainable Innovation - not only can TDD help teams deliver value sooner, it can also help them deliver value on the same code base for longer. I've seen some big organisations brought to their knees by the crippling cost of changing software products and systems, and having to rewrite software from scratch many times just to make small gains in new functionality. The technical benefits of TDD - continuous testing and cleaner code - tend to flatten out the cost of change over time, which typically rises exponentially otherwise.
6. It Doesn't Cost More - despite what some TDD naysayers claim, it actually doesn't cost significantly more to test-drive the design of your code, and in some cases improved productivity. Many teams report a slowdown when adopting TDD, and this is because of the not-inconsiderable learning curve. It takes 4-6 months to get the hang of TDD, and a lot of teams don't make it that far, which is why I strongly recommend adopting TDD "under the radar".
7. Puts The Problem Horse Before The Solution Cart - one of the biggest failings of software development teams is our tendency to be solution-driven and not problem-driven. TDD forces us to start by explicitly articulating the problem we want to solve, and then encourages us to find the simplest solution. As a result, test-driven code has a tendency to be more useful.
8. Free of Fear - I've seen first-hand the profound effect TDD can have on a developer's confidence. Firstly, using precise examples helps us to clear the fog that usually hangs over requirements, obscuring our view going forward. And continuous automated regression testing - a nifty side-effect of doing TDD - gives us confidence looking back that a change we make hasn't broken the software. The net effect is to make us more us more courageous as we work, more willing to try new ideas, less frightened of failing. Writing software is a learning process, and - to quote from Dune - "fear is the mind-killer".
9. Getting Everyone "On The Same Page" - exploring customer requirements using test examples is a very powerful way to clear up ambiguities and potential misunderstandings, by making everything clear and concrete. And not just for developers: testers, ops teams, UX designers, security experts, technical documentation writers, marketers... we can all benefit from these examples, getting the entire team on the same page.
10. A Gateway To High-Integrity Code - TDD, done well, can open the door to more advanced kinds of software testing at relatively low extra costs. For example, if you're in the habit of refactoring duplicate tests into parameterised tests, a few extra lines of code can add an exhaustive data-driven test. Increasingly, tools and techniques normally associated with high-integrity code (e.g., model checking) are being brought into the unit testing arena. NASA JPL's Pathfinder model checker, for example, can drive tests using JUnit Theories. And there are many versions of Haskell's QuickCheck being ported for other languages and xUnit implementations. Finally, after years of high-integrity software being a largely "academic" domain, we stand on the precipise of it going mainstream.
If you're interested in learning TDD, or are a TDD practitioner looking to distinguish yourself and advance your skills, visit www.codemanship.com
September 13, 2016
4 Things You SHOULDN'T Do When The Schedule's SlippingIt takes real nerve to do the right thing when your delivery date's looming and you're behind on your plan.
Here are four things you should really probably avoid when the schedule's slipping:
1. Hire more developers
It's been over 40 years since the publication of Fred L. Brooks' 'The Mythical Man-Month'. This means that our industry has known for almost my entire life that adding developers to a late project makes it later.
Not only is this born out by data on team size vs. productivity, but we also have a pretty good idea what the causal mechanism is.
Like climate change, people who reject this advice should not be called "skeptics" any more. In the face of the overwhelming evidence, they're Small Team Deniers.
Hiring more devs when the schedule's slipping is like prescribing cigarettes, boxed sets and bacon for a patient with high blood pressure.
2. Cut corners
Still counterintuitively, for most software managers, the relationship between software quality and the time and cost of delivery is not what most of us think it is.
Common sense might lead us to believe that more reliable software takes longer, but the mountain of industry data on this clearly shows the opposite in the vast majority of cases.
To a point - and it's a point 99% of teams are in no danger of crossing - it actually takes less effort to deliver more reliable software.
Again, the causal mechanism for this is well understood. And, again, anyone who rejects the evidence is not a "skeptic"; they're a Defect Prevention Denier.
The way to go faster on 99% of projects is to slow down, and take more care.
3. Work longer hours
Another management myth that's been roundly debunked by the evidence is that, when a software delivery schedule's slipping significantly, teams can get back on track by working longer hours.
The data very clearly shows that - for most kinds of work - longer hours is a false economy. But it's especially true for writing software, which requires a level of concentration and focus that most jobs don't.
Short spurts of extra effort - maybe the odd weekend or late night - can make a small difference in the short term, but day after day, week after week overtime will burn your developers out faster than you can say "get a life". They'll make stupid, easily avoidable mistakes. And, as we've seen, mistakes cost exponentially more to fix than to avoid. This is why teams who routinely work overtime tend to have lower overall productivity: they're too busy fighting their own self-inflicted fires.
You can't "cram" software development. Like your physics final exams, if you're nowhere near ready a week before, then you're not gong to be ready, and no amount of midnight oil and caffeine is going to fix that.
You'll get more done with teams who are rested, energised, feeling positive, and focused.
4. Bribe the team to hit the deadline
Given the first three points we've covered here, promising to shower the team with money and other rewards to hit a deadline is just going to encourage them to make those mistakes for you.
Rewarding teams for hitting deadlines fosters a very 1-dimensional view of software development success. It places extra pressure on developers to do the wrong things: to grow the size of their teams, to cut corners, and to work silly hours. It therefore has a tendency to make things worse.
The standard wheeze, of course, is for teams to pretend that they hit the deadline by delivering something that looks like finished software. The rot under the bonnet quickly becomes apparent when the business then expects a second release. Now the team are bogged down in all the technical debt they took on for the first release, often to the extent that new features and change requests become out of the question.
Yes, we hit the deadline. No, we can't make it any better. You want changes? Then you'll have to pay us to do it all over again.
Granted, it takes real nerve, when the schedule's slipping and the customer is baying for blood, to keep the team small, to slow down and take more care, and to leave the office at 5pm.
Ultimately, the fate of teams rests with the company cultures that encourage and reward doing the wrong thing. Managers get rewarded for managing bigger teams. Developers get rewarded for being at their desk after everyone else has gone home, and appearing to hit deadlines. Perversely, as an industry, it's easier to rise to the top by doing the wrong thing in these situations. Until we stop rewarding that behaviour, little will change.