March 16, 2018
Lamenting the Golden Age of High-Integrity Software That Never CameWhen I was a much younger programmer, I read a paper that had a big impact on the way I thought about software integrity.
Up to then, I - like so many - believed that "software has bugs". It seemed inevitable. Because all the software I'd seen had bugs. And all the software I'd written had bugs. We just have to live with it, right?
And then along came this paper on a thing called Cleanroom Software Engineering, and my mind was blown.
IBM wrote a COBOL pre-compiler that had about 85,000 lines of code and zero bugs reported in production. Not one. Ever. And what really struck me is that - bearing in mind how primitive dev tools were in the 1980s - it only took a team of six, achieving an average dev productivity that was measurably higher than the industry average. Also, the cost of maintaining the product - typically a lot higher than the cost of initial development - was relatively low; just one developer-year per year. Because nobody was bug fixing.
Now, of course, compared to software today 85 KLOC isn't much. But it's not insignificant, statistically. Maybe an equivalent product today would have 20x as much code. But what's 20x zero?
A single paper turned my whole worldview about software integrity (vs. productivity) upside-down. I've been lucky enough to experience this kind of approach - not specifically Cleanroom, but along similar lines - since, and seen the results for myself. Seeing is believing, and - praise Knuth! - I'm a believer!
So you can probably imagine my frustration to see how, 20 years later, the "software has bugs" paradigm still dominates. Who out there is producing very high-integrity code? Vanishingly few. I've waited and waited for high-integrity development techniques to catch on. I've even stirred the pot a few times myself with attempts at training products and talks with various publishers about a book that updates the ideas for the hipster Agile generation. To no avail. Still, vanishingly few are interested.
It's not as if there isn't a compelling business case. More reliable code, for little to no extra cost (you might even save time and money)? Lower maintenance costs? Happier customers? A world of digital stuff we can rely on? What's not to like? It's not as if these techniques are incompatible with Agile, either. I've done both at the same time, for real.
But for every person like me out there selling the dream, there are 10 more actively briefing against it. "Quick and dirty". "Move fast and break stuff". "Perfection is the enemy of good enough." Etc etc etc.
It's an easy sell to managers who don't understand the relationship between quality, time and cost. Cut some corners, get there sooner, save some money. A much harder proposition is "take more care, get there sooner, save some money". Bosses don't believe it. Heck, most devs don't believe it, despite the mountain of strong evidence to back it up.
I still live in hope that - one day - high-integrity software will go mainstream. The tools and techniques are not, despite what you may have heard, rocket science. Most devs are smart, and most devs could learn to do this. I did, so it can't be that difficult.
March 11, 2018
Proposing The xUnit "Meta-Kata"A while back I ruminated on refactoring old-fashioned "test it all with a main method" test code (like we did back in the day) into the xUnit unit test framework pattern.
It occurred to me that this might make a good code kata. TDD a well-known kata (e.g., FizzBuzz, Bowling Game, "Rock, Paper, Scissors"), but starting without a unit testing framework doing it all in a single main method.
As the code evolves, refactor the test code to remove code smells like methods testing more than one thing, classes testing more than one "unit" or "feature" (depending on how you roll), high-level modules that depend directly on low-level test fixtures, multiple tests being different examples of the same test, and so on.
It could be an interesting exercise in discovering frameworks. Perhaps it'll take multiple katas for a complete xUnit framework to reveal itself, just as so many great and useful frameworks don't really take shape until they've been reused a few times on other problems.
It might also be an exercise in applying the TDD discipline on two problems simultaneously; a test of your Craft Fu.
Really, it would be a kata within a kata: a meta-kata, if you like. And as such, I think it could be really interesting and rather challenging. I'll hopefully be giving it a go - well, probably a few goes - when I pair with my "apprentice" Will Price soon. When I think we've cracked it, I'll post a screencast and the code (with version history, so you can play it back).
March 9, 2018
S.O.L.I.D. C# - Online Training Event, Sat Apr 14thDetails of another upcoming 1-day live interactive training workshop for C# developers looking to take their design skills to the next level.
I'll be helping you get to grips with S.O.L.I.D. and much more besides with practical hands-on tutorials and exercises.
Places are limited. You can find out more and grab your place at https://www.eventbrite.co.uk/e/solid-c-tickets-44018827498
March 7, 2018
C# Refactoring with Resharper - Live Online Training Workshop, March 17thJust a quick note to plug a live online training event that's happening on Saturday March 17th. If you're a C# developer wants to get to grips with the refactoring discipline and the Resharper plug-in for Visual Studio, this is the perfect kick start.
Refactoring's an essential skill for a code crafter, but still something a lot of even quite experienced developers have yet to wrap their heads around. Take your code craft to the next level.
There's a morning and afternoon session, each lasting 3 hours. Places are limited to 8 in each session, to ensure you get more attention from your trainer (me) while you tackle the exercises.
Find out more and book your place at https://www.eventbrite.co.uk/e/c-refactoring-with-resharper-tickets-43923282721
February 4, 2018
Don't Bake In Yesterday's Business Model With Unmaintainable CodeI'm running a little poll on the Codemanship Twitter account asking whether code craft skills should be something every professional developer should have.
Every professional software developer should be able to write good unit tests, to use version control & do Continuous Integration, to apply design principles and to refactor code safely. Do you...— Codemanship (@codemanship) February 3, 2018
I've always seen these skills as foundational for a career as a developer. Once we've learned to write code that kind of works, the next step in our learning should be to develop the skills needed to write reliable and maintainable code. The responses so far suggest that about 95% of us agree (more than 70% of us strongly).
Some enlightened employers recognise the need for these skills, and address the lack of them when taking on new graduates. Those new hires are the lucky ones, though. Most employers offer no training in unit testing, TDD, refactoring, Continuous Integration or design principles at all. They also often have nobody more experienced who could mentor developers in those things. It's still sadly very much the case that many software developers go through their careers without ever being exposed to code craft.
This translates into a majority of code being less reliable and less maintainable, which has a knock-on effect in the wider economy caused by the dramatically higher cost of changing that code. It's not the actual £ cost that has the impact, of course. It's the "drag factor" that hard-to-change code has on the pace of innovation. Bosses routinely cite IT as being a major factor in impeding progress. I'm sure we can all think of businesses that were held back by their inability to change their software and their systems.
For all our talk of "business agility", only a small percentage of organisations come anywhere close. It's not because they haven't bought into the idea of being agile. The management magazines are now full of chatter about agility. No shortage of companies that aspire to be more responsive to change. They just can't respond fast enough when things change. The code that helped them scale up their operations simultaneously bakes in a status quo, making it much harder to evolve the way they do business. Software giveth, and software taketh away. I see many businesses now achieving ever greater efficiencies at doing things the way they needed to be done 5, 10 or 20 years ago, but unable to adapt to the way things are today and might be tomorrow.
I see this is finance, in retail, in media, in telecoms, in law, in all manner of private sector organisations. And I see it in the public sector, too. "IT delays" is increasingly the reason why government policies are massively delayed or fail to be rolled out altogether. It's a pincer movement: we can't do X at the scale we need to without code, and we can't change the code to do X+1 for a rapidly changing business landscape.
I've always maintained that code craft is a business imperative. I might even go as far as to say a societal imperative, as software seeps into every nook and cranny of our lives. If we don't address issues like how easy to change our code is, we risk baking in the past, relying on inflexible and unreliable systems that are as anachronistic to the way things need to be in the future as our tired old and no-longer-fit-for-purpose systems of governance. An even bigger risk is that other countries will steal a march on us, in much the same way that more agile tech start-ups can steam ahead of established market players simply because they're not dragging millions of lines of legacy code behind them.
While the fashion today is for "digital transformations", encoding all our core operations in software, we must be mindful that legacy code = legacy business model.
So what is your company doing to improve their code craft?
February 1, 2018
BDD & Specification By Example - Where Did We Go Wrong?I've been saving this post up for a while, but with a bit of pre-dinner free time I wanted to put it out there now.
I meet a lot of teams, and one thing many of them tell me is that the "customer tests" they've been driving their designs from are actually written by the developers, not the customer.
DEV TEAMS who do BDD/ATDD: who writes your Cucumber/FitNesse/RSpec etc tests?— Codemanship (@codemanship) July 18, 2016
Sure, they're written using a "Behaviour-Driven Development" or "Acceptance Testing" tool like Cucumber or Fitnesse. But just because you've built a "granny annex" on your house, if there's no granny living in it, it's just an "annex".
We've dropped the ball on this. The CHAOS report, published every year by the Standish Group, consistently cites lack of customer involvement as the number one factor in project failure. A tool won't fix that.
Especially when that tool wasn't designed with customer collaboration in mind. When your "Getting Started" guide begins "First, install Visual Studio..." or requires your customer to learn a mark-up language or to use version control, arguably you're bound to have a hard time getting them to engage in the process.
Increasingly, I work with teams who want to somehow connect the way their customer actually prefers to capture examples with the way devs like to automate tests. 90% of the time, that means pulling data out of Excel spreadsheets - still the most widely used tool in both communities - into unit tests. Some unit testing frameworks even have that facility built in (e.g., MSTest for .NET). But reading data from spreadsheets is child's play for most developers. With OLD DB or JDBC, for example, a spreadsheet's just a database.
But, regardless of the tools, the problem most teams need to solve is a people problem. I've found that close customer involvement is so critical to the chances of a team succeeding at solving the customer's problems that I actually stop development until they engage at the level we need them to. No play? No code.
The mistake many of us make is to give them a choice. "Would you like to spend a lot of time with us discussing requirements and playing with candidate releases and giving us feedback?" "No thanks, ta very much. See you in a year's time."
We made a rod for our backs by allowing them to be absentee partners and trying to figure out what they want and need for them. Specification By Example presents us with an opportunity to make the relationship clearer. The customer has to be "trained" to understand that if they haven't agreed a test for it, they ain't gonna get it.
A Bit of Old School BDD with NUnit & MS ExcelI'm going Old School this morning with my pairing partner, and while she's popped out for a meeting, I thought I'd quickly jot down what we've been working on.
Back in the good old days before BDD/ATDD frameworks, when we wanted to automate customer tests we just captured the customer's example data in something like MS Excel and then wrote a bit of code to read that data into a unit test. (That, essentially, is what SBE tools do, just with some bells and whistles.)
For example, imagine our customer wants to be able to calculate square roots using the software. We could agree an acceptance test, in the trendy hipster "Given...When...Then..." style, and put that in a spreadsheet, like so.
If we name the cell range containing the example data "examples" (for ease of extracting using OLE DB), and save this spreadsheet in the root directory of our Visual Studio test project, then we can relatively easily suck out that data to provide NUnit test cases for a parameterised test with arguments that match the data in the table.
Here's a complete source listing for our basic spike.
(We're going to try and refine this a bit, and see if it can't be made more general. One of the downsides of using a custom TestCaseSource is that we can't parameterise it easily to specify different Excel files and different ranges. Though why such a mechanism doesn't already exist is a bit of a mystery, after 15+ years of NUnit.)
January 26, 2018
Good Code Speaks The Customer's LanguageSomething we devote time to on the Codemanship TDD training course is the importance of choosing good names for the stuff in our code.
Names are the best way to convey the meaning - the intent - of our code. A good method name clearly and concisely describes what that method does. A good class name clearly describes what that class represents. A good interface name clearly describes what role an object's playing when it implements that interface. And so on.
I strongly encourage developers to write code using the language of the customer. Not only should other developers be able to understand your code, your customers should be able to follow the gist of it, too.
Take this piece of mystery code:
What is this for? What the heck is a "Place Repository" when it's at home? For whom or for what are we "allocating" places?
Perhaps a look at the original user story will shed some light.
The passenger selects the flight they want to reserve a seat on.
They choose the seat by row and seat number (e.g., row A, seat 1) and reserve it.
We create a reservation for that passenger in that seat.
Now the mist clears. Let's refactor the code so that it speaks the customers language.
This code does exactly what it did before, but makes a lot more sense now. The impact of choosing better names can be profound, in terms of making the code easier to understand and therefore easier to change. And it's something we all need to work much harder at.
January 23, 2018
Without Improving Code Craft, Your Agile Transformation Will Fail"You must be really busy!" is what people tend to say when I tell them what I do.
It stands to reason. If software is "eating the world", then code craft skills must be highly in demand, and therefore training and coaching for developers in those skills must be selling like hotcakes.
Well, you'd think so, wouldn't you?
The reality, though, is that code craft is critically undervalued. The skills needed to deliver reliable, maintainable software at a sustainable pace - allowing businesses to maintain the pace of innovation - are not in high demand.
We can see this both in the quality of code being produced by the majority of teams, and in where organisations focus their attentions and where they choose to invest in developing skills and capabilities.
"Agile transformations" are common. Some huge organisations are attempting them on a grand scale, sending their people on high-priced training courses and drafting in hundreds of Agile coaches - mostly Scrum-certified - to assist, at great expense.
Only a small minority invest in code craft at the same time, and typically they invest a fraction of the time, effort and money they budget for Agile training and coaching.
The end result is software that's difficult to change, and an inability to respond to new and changing requirements. Which is kind of the whole point of Agile.
Let me spell it out in bold capital letters:
IF CODE CRAFT ISN'T A SIGNIFICANT PART OF YOUR AGILE TRANSFORMATION, YOU WILL NEVER ACHIEVE AGILITY.
You can't be responsive to change if your code is expensive to change. It's that simple.
While you build your capability in product management, agile planning and all that scrummy agile goodness, you also need to be addressing the factors that increase the cost of changing code. Skills like unit testing, TDD, refactoring, SOLID, CI/CD are a vital part of agility. They are hard skills to crack. A 3-day Certified Code Crafter course ain't gonna cut the mustard. Developers need ongoing learning and practice, with the guidance of experienced code crafters. I was lucky enough to get that early in my career. Many other developers are not so lucky.
That's why I built Codemanship; to help developers get to grips with the code-facing skills that few other training and coaching companies focus on.
But, I'll level with you: even though I love what I'm doing, commercially it's a struggle. The reason so few others offer this kind of training and coaching is because there's little money it. Decision makers don't have code craft on their radars. There's been many occasions when I've thought "May as well just get Scrum-certified". I'm not going to go down without a fight, but what I really need (apart from them to cancel Brexit) is for a shift in the priorities of business who are currently investing millions on Agile transformations that are all but ignoring this crucial area.
Of course, those are my problems, and I made my choices. I'm very happy doing what I'm doing. But it's indicative of a wider problem that affects us all. Getting from A to B is about more than just map reading and route planning. You need a well-oiled engine to get you there, and to get you wherever you want to go next. Too many Agile transformations end up broken down by the side of the road, unable to go anywhere.
January 21, 2018
Delegating "Junior" Development Tasks. (SPOILER ALERT: It doesn't work)When I first took on a leadership role on a software development team 20 years ago, from the reading I did, I learned that the key to managing successfully was apparently delegation.
I would break the work down - GUI, core logic, persistence, etc - and assign it to the people I believed had the necessary skills. The hard stuff I delegated to the most experienced and knowledgeable developers, The "easy" stuff, I left to the juniors.
It only took me a few months to realise that this model of team management simply doesn't work for software development. In code, the devil is in the detail. To delegate a task, I had to explain precisely what I wanted that code to do, and how I wanted it to be (in terms of coding standards, our architecture, and so on).
If the task was trivial enough to give to a "junior" dev, it was usually quicker to do it myself. I spent a lot more time cleaning up after them than I thought I was saving by delegating.
So I changed my focus. I delegated work in big enough chucks to make it worthwhile, which meant it was no longer "junior" work.
Looking back with the benefit of 20 years of hindsight, I realise now that delegating "junior" dev tasks is absurd. It's like a lead screenwriter delegating the easy words to a junior screenwriter. It would also probably be a very frustrating learning experience for them. I'm very glad I never went through a phase in my early career of doing "junior" work (although I probably wrote plenty of "junior" code!)
The value in bringing inexperienced developers in to a team is to give them the opportunity to learn from more seasoned developers. I got that chance, and it was invaluable. Now, I recommend to managers that their noobs pair up with the old hands on proper actual software development, and allow for the fact that it will take them longer.
This necessitates - if you want the team to be productive as a whole - that the experienced developers outnumber the juniors. Actually, let's not call them that. The trainees.
Over time - months and years - the level of mentoring required will fall, until eventually they can be left to get on with it. And to mentor new developers coming in.
But I still see and hear from many, many people who are stuck in the hell of a Thousand Junior Programmers, where senior people - often called "architects" - are greatly outnumbered by people still wet behind the ears, to whom all the "painting by numbers" is delegated. This mindset is deeply embedded in the cultures of some major software companies. The result is invariably software that's much worse, and costs much more.
It also leads to some pretty demoralised developers. This is not the movie industry. We don't need runners to fetch our coffee.
ADDENDUM: It also just occurred to me, while I'm recalling, that whenever I examined those "junior" dev tasks more closely, their entire existence was caused by a problem in the way we were doing things (e.g., bugginess, lack of separation of presentation and logic, duplication in data access code, etc). These days, when it feels like "grunt" work - repetitive grind - I stop and ask myself why.