February 10, 2016
Two Good Reasons To Refactor To Parameterised Tests (Plus, How To Use Algorithmically Generated Test Data)For training workshops and other fun activities, I've been playing around with unit testing frameworks that generate test data algorithmically. For example, generating test inputs for JUnit parameterised tests using JCheck, and noodling with the built-in features of NUnit that do similar good stuff.
I continue to promote the value of refactoring our test code into parameterised tests when the opportunity arises (2 or more tests that are essentially different examples of the same behaviour or rule). Once you have parameterised tests, you can buy yourself quite mind-boggling levels of coverage very cheaply.
To illustrate, this morning I took a bit of date-driven logic that had unit tests, and using JCheck, wrote a custom test data generator that enabled me to run that test over a range of 2 million days (about 5,000 years). It was about 10 extra lines of code, and the tests cases took roughly 0.5 seconds to run. That's 5,000 years of testing for the price of 10 LOC and 0.5 seconds to execute. That seems like a small price to pay. Hence my enthusiasm for parameterised tests.
Parameterising tests also presents me with an opportunity to create test method names that read like a true specification.
Take this little example of a test fixture for a Fibonacci number calculator:
Each individual test in an example of a behaviour. I could refactor this code into parameterised tests to remove some of the duplication, but I can use the test method names to more clearly describe the rules of my Fibonacci calculator:
And now that I have some parameterised tests, I can get extra mileage from them using a tool like JCheck:
With a bit of extra code, I can squeeze out a million test cases (randomly generated), reusing the original test code. If - for some bizarre reason - the Fibonacci calculator was critical to our business, I believe that's a heck of a lot of extra assurance for very little extra effort. Those test cases took about 2.5 seconds to run.
Notice, too, that when we algorithmically generate test inputs, we have to algorithmically generate the expected outputs. This can cause us problems, of course. For that big dollop of extra assurance when we really need it, I'm willing to pay the price of having two implementations that calculate Fibonacci numbers. Redundancy is a tried-and-tested strategy for safety-critical systems.
Where the real danger lies would be if I used the same algorithm to generate the expected outputs. If the algorithm was wrong, the tests would still agree with the actual results. So I've used a different algorithm to generate the expected outputs. The odds of both giving exactly the same wrong answers by two sufficiently different routes are very remote.
Other examples might be using Quicksort to test the results of an implementation of Bubble Sort, or calculating 3D transformations in spherical polar coordinates for an implementation that calculates them in Cartesian coordinates. And so on. The basic idea is, don't test logic using the same logic. It's a similar idea to double-entry book keeping in accounting; if we come to the same answers by two different routes, they're very unlikely to be identically wrong.
So there you have it. Refactoring test code to parameterised tests can not only help to make the tests more self-explanatory, but they can give us a jumping off point to very high levels of test assurance relatively cheaply, when it's really needed.
January 20, 2016
Developer Archetypes #29: The Serial Monogamist"I'm a great husband. I must be: I've been married ten times." Or so the joke goes.
And so it is with some software developers, who hop and skip in blissful ignorance from "greenfield project" to "greenfield project", blind to the consequences of their decisions because they never stick around long enough to witness them.
But the fact is that most developers spend most of their time living with those consequences, working on legacy code. It's estimated that we spend anywhere between 50%-80% of our time just reading code. Those developers who have managed to engineer careers that sidestep legacy code can often be oblivious to the profound importance of things like readability, or the benefits of fast-running automated tests, and other factors that can make changing legacy code much, much easier.
Read all you like about design patterns and OOA/D and "enterprise architecture" - if only to impress and intimidate in meetings - but I posit that, until you've lived with the consequences of your design decisions, you know nothing about design. Well, nothing that makes a difference.
If you are one of those developers who never have to look back, then the next time an employer or recruiter approaches you with a "shitty legacy code project", stop to consider whether it might actually be a great learning opportunity in disguise.
January 14, 2016
Intensive TDD Workshop, London Saturday March 19th
The first Intensive TDD workshop of 2016 is now open for registration.
It will be at our usual venue in South Wimbledon, at the amazingly low price of £79 (just 10% of the average price for a day's developer training), on Saturday March 19th. All the training, just without the frills. Bring a packed lunch!
Details and tickets can be found here.
December 19, 2015
The Multi-threaded Santa TDD ChallengeIf you've hunted on the Interwebs for TDD katas, you will no doubt have noticed that they tend to be based on relatively straightforward problems. In particular, there are no - as far as I'm aware - TDD katas that set more challenging high-volume, multi-threaded, multi-process problems to solve.
In the spirit of Christmas - if you celebrate such a thing (and this year, I won't be) - I set some students just such a problem, which I'm calling the Multi-threaded Santa TDD Challenge.
Now, read this very carefully. It goes something like this:
Multi-threaded Santa TDD Challenge
The goal is to make, wrap, load and deliver 1 million presents on Xmas Eve. Play this challenge in pairs, with multiple pairs competing to do it for the lowest cost.
Santa and his elves have to make, wrap, load and deliver 1 million presents. They work in 4 teams:
Each team must be implemented in its own process (e.g, a web service for each activity, or a daemon, etc etc).
Each team either sends to or takes from (or both) one of 4 queues, and each queue has a maximum size, beyond which no more presents can be added:
1. Made (max size = 1000)
2. Wrapped (max size = 2000)
3. Loaded (max size = 5000 - these are all the presents on Santa's sleigh for delivery)
4. Delivered (no size limit)
How does Santa deliver all those presents in just one night, though? Simple. Santa Time works differently to our time. An hour in Santa Time is just 1 second (1,000ms) of human time. So 3600 hours of Santa Time elapse in 1 hour of human time.
You must implement these time delays in a Present class, with 4 distinct lifecycle event methods:
* Present.make() - wait 50ms
* Present.wrap() - wait 10ms
* Present.load() - wait 5ms
* Present.deliver() - well, we'll get to that...
To perform work on a present, a worker thread must invoke the appropriate method, incurring a time cost for that work within that thread. Only after that work has been performed can the present be sent to the queue for the next process to take from. e.g. one thread in the Making process called present.make(), then sends that present to the Made queue, where the Wrapping process can pick it up.
In each process, there are one or more elves (worker threads). Elves are managed by a fifth process: the Elf Pool. To add an elf (worker thread) to a process (e.g., wrapping), you must request an Elf ID from the Elf Pool.
The Elf Pool records every elf assigned to every process for the duration of the work.
Because of the very strict rules of the Elf Union, elves - once assigned to the work - cannot be laid off. So the total number of elves employed can only go up, not down, until all presents have been delivered.
Elves can, however, be swapped between the 4 work processes: so an elf assigned to wrapping can be re-assigned to loading, for example. This, too, must be managed through the Elf Pool.
One worker process (e.g. wrapping) signals an intention to reassign one of its elves to the Elf Pool. The elf remains in that process until another worker process (e.g., Loading) requests another elf, at which point - instead of assigning a new elf from the pool - that elf is re-assigned instead. So the total number of elves employed remains the same. If the Elf Pool has no elves awaiting re-assignment, it assigns a new elf when one is requested.
There is only one Santa, and only one sleigh!
Okay, here's your bottleneck. There can only be one worker thread in the Delivering process, and the delivery round takes a fixed amount of time each time the sleigh goes out, regardless of how many presents or elves are assigned to Delivery. It takes Santa 500ms to make one round of deliveries. i.e. delivering all the presents on Santa's sleigh - no matter how many there are - incurs a time delay of 500ms. At the end of that time, all the presents in the sleigh (i.e., on the Loaded queue) are delivered. This could, for example, be achieved with a Sleigh class that implements the time delay in a deliver() method.
Be advised that no presents can be loaded for delivery while Santa's sleigh is out on a round. This is your bottleneck! For 500ms (at least), the elves doing the loading will be idle. So you may want to reassign them to making or wrapping. But... hmmm... how long will that take?
How to score?
The Elf Pool has another job in this challenge: elves don't work for nothing. They work for cookies. Specifically, one elf assigned for one hour of Santa Time (1s of human time) gets paid 1 cookie. So 100 elves assigned for 1000s of human time would cost 100,000 cookies.
The Elf Pool calculates the cookie payroll, multiplying the number of assigned elves by the time it takes to deliver all the presents in human seconds.
To win, your team needs to deliver all 1 million presents for less cookies than your competitors.
Never Give In. Never Surrender.
Now, here's the kicker: once the start whistle blows, there is NO STOPPING. If any of your worker processes falls over, the work becomes blocked, the clock keeps on ticking, and you keep burning cookies. Restarting any worker process must re-attach all of the elves that were assigned to it via the Elf Pool before it can begin working on presents again.
If your Elf Pool falls over: you are disqualified, and Santa gets to kick you in the backside for all eternity for ruining Xmas!!!
So, there you have it: 1 million presents, 4 worker processes, as many worker threads as you like (within the rules), 4 work queues, and one manager process keeping track of the elves and the cookies and the time.
Easy as cheese. Average time for students to complete was 4 hours. My advice: keep it as simple as you can. I've already piled on plenty-much complexity for you to deal with!
December 12, 2015
The "Just F**king Do It" ManifestoI've noticed there's a new manifesto doing the rounds on social media. Well, not so much a new manifesto as a reaffirmation of one part of an old manifesto.
The "12th principlists", as they're calling themselves, are putting their names to a manifesto based on the twelfth Agile principle - that teams should be self-organising and empowered.
Who, among Agile developers, disagrees with that?
The problem, I feel, is not that developers and teams don't sign up to the 12th principle. Signing up to things that sound good is easy.
The problem has always been that - even with their best intentions - it's beyond the power of teams to make it happen. Or so many teams seem to think.
A core Agile value is courage, and you need a big set of brass balls to manage upwards and outwards so that your team really is given the trust and the authority to decide how development needs to be done.
Don't make excuses. Don't look to Mom & Dad to give you permission. Stand firm and take responsibility. If you think you should be writing tests first, then write tests first. If you think developers should be working in pairs, work in pairs. If you think you need to agree acceptance tests with the customer, then make it absolutely 100% crystal clear that if there isn't a test for it they ain't getting it. If you need whiteboards, get whiteboards. If you need two monitors, get two monitors.
I'm being glib, of course. You may not have the authority to buy a second monitor. The customer may feel their time is better spent doing anything else except agreeing tests with you. And so on.
And this is where the rubber meets the road. In my experience, if development teams don't have the authority to decide how best they should work - within reason, of course - it's because we let things get that way.
At home, who decides if you can have a second monitor? You do. And how big a decision is it? It's trivial. Whip out the old credit card and you'll have it tomorrow. Because you are a grown-up.
At work, though, there's a tendency for teams to be treated like children. Want a second monitor? Ask Mummy and Daddy. Want to pair program? Check it's okay with mummy and Daddy first.
Teams must not seek permission from above for the way they work. That's the road to infantilisation and micromanaged madness. Agree as a team on what needs to be done, and then just f**king do it.
The other side of that equation is delivering. Do what needs to be done, but you have to deliver. Teams failing to meet customer expectations is why they put Mommy and Daddy in charge in the first place. Your team has to demonstrate that you are grown-ups, and that Mummy and Daddy aren't needed. And what better way than by consistently delivering value?
Here's the rub: Mummy and Daddy's way of doing things may not be conducive to delivering working software that satisfies customers' expectations. So, ironically, if you do it their way, you may lose even more trust, and be micromanaged ever tighter. And round and round we go.
When I come across teams who have almost no say in how they do their work, there's usually a long backstory of micromanagement leading to disappointment leading to even worse micromanagement, until developers have to go through a long chain of command to get a pack of index cards.
The kind of personality who can manage upwards to prevent or reverse that situation is relatively rare. We're not famous, as a community, for attracting non-risk-averse assertive individuals with the necessary people skills to protect the team from interference. Typically, those kind of people - often from non-technical backgrounds - are hired to be Mummy and Daddy, and tell the team what to do. It's also a personality type that likes to be in charge and likes things done their way, and teams are easily dominated and exploited by them.
This may stick in the craw of the many free market capitalists working in software development, but what teams need far more than strong leaders is solidarity. Leadership doesn't have to be one person calling all the shots. A team can be democratic. We can throw many ideas around, then form a consensus on what direction to take. And then present a united front on that consensus.
Yes. Software developers agreeing with each other. The impossible dream!
So, teams don't get empowered. They empower themselves. And that does, regrettably, mean that your bottom line will need to be "if you don't want us to work this way, then find someone else". The moment you flinch and show that, really, you'll do whatever they tell you, you're sunk - at least as far as the 12th principle is concerned.
November 21, 2015
Getting Software Teams Back On TrackA recent tweet about things teams do when the schedule is slipping that can actually make things worse seems to have resonated with a lot of people.
The 3 things almost every team does to meet deadlines that make things worse: 1. Hire more developers 2. Cut corners 3. Work long hours— jasongorman (@jasongorman) November 20, 2015
It naturally begs the question: what could we do to get us back on track?
Well, the first - and most painful lesson - is to accept that just maybe we can't get back on track. And therefore we'll have to rethink the plan. There are two ways we can remake a plan that we know can work:
1. Move the delivery date back to fit all the work in
2. Deliver less to meet the delivery date
It's not a conversation a lot of teams relish having with their customer, but there is some silver living re: #2
When I say "deliver less", I mean "less software". If we're clear in our minds that software and value aren't the same thing, it becomes possible to deliver most of the value a customer needs - to solve their key problem, essentially - without delivering all the features we planned to.
This is the software development equivalent of saying "The house won't be finished on time, but you will be able to move in on that date".
If you're doing Agile Software Development, this may already have happened. And if it hasn't, then maybe you're not really doing Agile Software Development.
This solution does require us to have a clear idea of what the real end goals of the software are. You have to shift the emphasis from delivering a shopping list of features to solving a set of problems. Then you can buy yourself flexibility on design. More choices can be a powerful thing when you need to solve a problem in a fixed amount of time.
When I've inherited team leader roles on compromised projects, the first thing I've done is stop everything and get the team to step back and ask "What are we trying to achieve here? Is there a simpler way?" So be prepared to ditch the design and to ditch the plan that goes with it. In my own experience of saving compromised projects, this is the key. Prepare for some full and frank conversations with your customer.
But what of the three things I tweeted about that we know can make late deliveries even later?
Hiring More Developers has long been known to make things worse.
The reasons why hiring more developers exacerbates the problem are explained beautifully in Fred Brook's famous book The Mythical Man-Month. I believe every software developer and every software development manager should read it.
Developers who were busy getting shit done suddenly become busy getting new team members up to speed. It's like promoting your most effective sales people to management roles; watch those sales plummet!
Add to that an exponential explosion in the number of lines of communication within the team required to keep everyone abreast of what everyone else is doing - which, sadly, in software is a necessity because it's all connected - and we are fighting a double productivity hit of losing your most productive developers to mentoring duties and the rapidly diminishing returns of bigger teams.
So, the advice is very simply DON'T DO IT. No matter how tempted you are. No matter how much pressure you're under from above. DON'T DO IT.
But what if the team's already too big? What if the reason you're not getting shit done is because the team overhead is holding you back? Here, you may have to make very difficult decisions, ranging in severity from moving people on to other projects - if they exist - or side-lining people who are getting in the way, to letting people go as a last resort.
If you are a bit top-heavy, or have a few too many trainees, it's possible to absorb the hit. If, on the other hand, you have 3 architects, 2 project managers, 6 tech leads, 4 business analysts, 30 developers and 10 testers... well, as far as I'm concerned a lot of those people are enjoying what is effectively a kind of in-work welfare. They're getting paid - and possibly paid very well - to not just add no value, but to positively suck value out of the team. You're paying them to slow you down. We all know it, but few dare say it.
In those cases when the team is very obviously too big, then you'll have very little choice but to simplify it if you want to deliver something of value when the customer needs it.
Alas, these kinds of decisions - to hire or to redistribute or to fire - are usually out of the hands of the likes of me and you, and managing upwards in these situations carries great risk; specifically the risk that the person going will be the one who blows the whistle. The default behaviour in these situations is to quite knowingly choose failure; perversely, for the good of the team.
Cutting Corners is a classic mistake. For about as long as we've known that hiring more developers will make things worse, we've also known that skimping on quality actually makes us go slower.
The reasons are simple, well-understood and backed up by a veritable Himalayas of hard evidence; bugs cost exponentially more time and money to fix the longer they go undetected. Less testing, less often, therefore, creates a downstream tsunami of necessary bug fixes that will take 10-100x times as long to fix than if we'd caught them sooner.
You can spot the teams that have been doing what we call "code-and-fix" development not just by the size of their bug list, but by the sheer amount of time and effort it takes them to "stabilise" the software to make it fit for release. I've watched teams go round and round in circles for months trying to make code good enough to ship.
This is why teams in the know, when the schedule's slipping, do more testing, more rigorously, and more often. Yes, it's more work now. But it's 10-100x less work later.
For sure, there are no silver bullets in software development. But when it comes to putting teams back on track, creating fast-running automated tests is the closest thing I've seen. It can completely transform the outlook.
Likewise moving to Specification By Example in our discussions about requirements with the customer can have a transformative effect. You'd be amazed how many misunderstandings you catch when you ask for concrete examples to clarify what they mean. A couple of hours spent clarifying a requirement with executable acceptance tests could save you weeks of unnecessary rework.
Same goes for the maintainability of the code. Less thought and care put into making our code easy to understand and easy to change (without breaking) can cost us dearly, and sooner than we think. I've found myself being hampered in the afternoon by code I wrote in the morning.
As in golf, the way to get a better score faster is to take more care over our shots.
This, too, is a tough sell both to team mates and to managers. The entirely debunked myth that better quality software takes longer to deliver still persists. And there may be new skills that have to be learned to make a defect prevention and "clean code" approach work. If the team's not already up-to-speed on the principles and practices, then - well, you're a bit buggered, really, because you're months (possibly years) away from conquering that learning curve.
If the team, on the whole, does have the necessary skills, then that would beg the question: why weren't you already doing these things? If the answer is "the boss won't let us", then here comes another difficult conversation, potentially.
My own experience, and the anecdotes I hear from people in that situation, is that you should never have that conversation. If you're having that conversation, all is very possibly already lost. It rarely goes the way you wish it to. By giving managers and customers a choice over quality, 99 times out of 100, they'll veto it in favour of "features". Even though, as we know, skimping on quality will deliver less features. But the perceived conflict between those two concerns beats most teams who make the mistake of offering it as a choice.
So, when it comes to meeting delivery dates, make quality non-negotiable. And the best way to do that is to never talk about it outside of the team. If you do TDD, for example, than that is the way you deliver working software. Just as surely as using a text editor is. When did you last offer the boss the choice of whether or not you should use a text editor?
And when things are slipping, you need to pull on those quality levers even harder. If you've been taking baby steps, take even babier steps. If you've been building and testing daily, build and test hourly. If you've been devoting an average 20 minutes to discussing a user story with the customer, devote an hour and agree executable tests, including all the edge cases you can think of. If you're averaging one unit test for 10 lines of code, aim to make your unit tests even more focused. If you're averaging one team code review in each iteration, do peer code reviews before every commit. And so on.
Finally, Working Long Hours - we know from many studies done over the last century - is not sustainable, hurts teams and projects, and is a false economy. Going as far back as Henry Ford, who discovered that when he reduced factory workers' hours from 50 to 40 a week, they got more done, we've known that hours worked != useful output.
Software development in particular is susceptible to burnout when we push teams into "crunch mode". It's an activity that requires focus and clarity, and focus and clarity are perishable goods that need to be replenished frequently. From my own experiments with teams, and with myself, I learned that the average developer will achieve roughly as much in 4 hours a day as she does in 8, and more than she would in 12 hours.
Some folk say "it's okay in the short term", but I've found no evidence anywhere that even this is true. Maybe an assembly line worker can work a few long shifts to meet a deadline, but that ain't software development.
We're also very susceptible to the illusion of productivity, and I think this explains where the "it helps in the short-term" myth comes from. Sure, we feel like we're getting shit done. But that may well be because we are blind to the downstream consequences of the mistakes we make along the way. Towards the end of an 8-hour day, I find - when I measure it objectively - that I'm making more mistakes. And so I've learned to quit while I'm ahead, and have found it beneficial to have teams do likewise. Learn to recognise when you're passing from a zone of "flow" into a zone of "liability".
And so, when I lead teams, I have a very low tolerance for longer hours as any kind of solution - even a short-term one.
And, yes, this too is a very tough sell to management. And to many developers still enraptured by the illusion of their own productivity. It takes real nerve - when a deadline is looming and everyone's frantically firefighting - to calmly get up from your desk at 5-6pm and go home. But it is, as far as I can tell, the best thing you can do for the project.
Similar - and closely related to the need to take more care - is the need to stay even more refreshed. Take more short breaks, have a proper relaxing (but not carb-heavy, because that'll make you sleepy) lunch. Be extra diligent about leaving at a reasonable time and having an even better work-life balance.
And then turn your focus on to what you actually do when you are at your desk instead of how long you spend at your desk.
One thing that really annoys me is how many organisations insist on long hours, but then merrily burn those hours on meaningless distractions like meetings and dysfunctional working environments like open-plan offices. I tend to find a strong correlation between the two. Wanna' get more done? Tackle the working environment, not the hours.
So, there you have it. To summarise; to improve your chances of meeting a delivery date:
1. Manage expectations of about what will be delivered and when. Simplify the product, simplify the plan. Focus on the end goal, not the shopping list.
2. Resist the temptation to hire more developers. If your team is already far too big, have that difficult conversation.
3. Turn the quality dial UP. Make quality non-negotiable.
4. Work FEWER hours. Take MORE breaks. Instead of focusing on hours, focus on what gets done in those hours, and remove obstacles to getting things done, like the working environment.
Of course, I say all these things safe in the knowledge that most teams will be too afraid to try them. It's easy on paper. But in practice, it takes a certain kind of team with a certain kind of leadership to make these things happen.
5. HAVE STRONG TECHNICAL LEADERSHIP
This doesn't necessarily mean someone bullish like me fighting the team's battles for them (though I have done it many times). But somehow, from somewhere within, the team needs to find this strength and self-belief and assert themselves on key issues like scope, deadlines, team size, working hours, office environments and so on. They've made you responsible for delivering: now assert the authority that has to accompany that responsibility if it's got to work.
November 4, 2015
The X Factor-isation of Programming May Attract The Wrong KidsI've watched a few documentaries recently about kids getting into "tech" (y'know, tech... like the spinning jenny and the seed drill).
The overriding theme is not one of tech itself - that tends to get edited out - but of entrepreneurialism. These things have become so closely intertwined, especially in the media, that many seem to view them as the same thing. Why would you write an "app" if not to make money?
Well, for millions of us who rode the first wave of home computing in the 1980s, the answer is very simply: for the sheer heck of it.
Why does a kid paint, or draw, or bake bread, or make sandcastles? Is it part of some business plan for a painting/drawing/baking/sandcastle start-up? As a kid, I painted because I enjoyed it. And I programmed because I enjoyed it. It was its own reward.
I worry now for this X Factor-isation of computer programming. Partly because it sets very unrealistic expectations - just like real X Factor. yes, some people get rich and famous making music. The vast majority don't. And the reasons for that are beyond their control. Getting good at making music, on the other hand, is within our control. We can practice; we can learn; we can get lots of experience trying different things.
But how many X Factor contestants are prepared to put in the hours to get good at making music? How many leave those auditions resolved to learn more about harmony and to get better at playing the piano or the guitar?
Without an intrinsic love of what you're doing - be it making music or writing computer programs - learning to do it better will likely as not take up far more time and require far more application than greed and willpower would motivate them to do. If there's no joy in it for you, then you won't go past the basics. It's just too much trouble.
And this is why I believe all the emphasis on start-ups and business plans and marketing and pitching and all the rest of it is a mistake. Want to be in business? Then be in business. Computer programming isn't business. Any more than playing the piano is.
Yes, some people make money programming computers. And some people make money playing the piano. But if you hire a session musician, I guarantee you'll be working with someone who - first and foremost - loves to make music. They might not love making your music, but then they'll go back to their home studio and extract the necessary joy from working on their own. They'll get a job to finance it. Just like so many programmers wade through 9-5 data processing drudgery and then go home and work on a personal project that excites them.
It troubles me deeply that we're dangling the wrong carrot for kids to aspire to. Indeed, it troubles me that we're dangling carrots at all. Programming is its own carrot for those who will enjoy it.
October 29, 2015
Software Developers: Are You Tired Of Unqualified People Telling You How To Do Your Job?Are you a software developer? A real one, who does actual programming and technical architecture and build automation and testing and stuff?
Have you - in some language(s) on some platform(s) - created valuable working software for a real customer in the real world by yourself? Have you created valuable working software for a real customer in the real world working in a team?
Are you tired of being told what to do, and being spoken for, by people who are not software developers and don't understand software development? Do you wish you, your team and your profession had more of a say in things; more of a voice?
Do you believe that software developers get the best results when:
a. We work in small teams with other skilled developers?
b. We work directly with the customer and end users?
c. We are left to make the technical decisions that we are best qualified to make?
d. We have control over who we work with, who can take leadership on what based on their expertise, and make developer hiring decisions?
e. We are not micromanaged and are responsible for how we use our time?
f. We work the way we believe works best, and are measured solely on what we deliver?
g. We decide how to bring new and inexperienced junior developers up to speed, and we decide when they're ready to work unsupervised, which we believe any real software developer should be able to?
If you answered "yes" to all these questions, then you're not alone. I've yet to meet a software developer who didn't have any war stories about clueless managers, micromanagement, lack of respect for them and their team, being forced to work with trainees who didn't know they were trainees, and being forced to work in dysfunctional ways because they had absolutely no say in it.
I think it's high time we took responsibility for our fate. I believe these things happen because we let them happen. But I appreciate just how seemingly impossible it is to be the lone voice on a team - the one person who stands up and says "this is all wrong". The typical end result is a rapid exit from the team, with no improvement in the team's prospects.
At the risk of summoning the ghosts of Marx and Lenin, what's needed is a bit of basic solidarity.
90% of software developers are perfectly happy producing crap and not rocking the boat. And they and their customers are welcome to it.
But the remaining 10% of us who actually give a damn need to be a bit more organised, lest we doom ourselves to frustrating careers locking swords fruitlessly with the other 90%. Been there, done that, wiped the blood off the t-shirt.
So - and I'm sure I'll live to regret this - I'm wondering if the time has come for the 10% to get together and join a sort of "club". I'm not sure what form this club might take, or exactly how we gain entrance to it, but I am increasingly convinced that the club is needed.
Call it a "guild", call it an "alliance", or an "institute": nay matter. What matters is the need for a network of moral support, congregating under a banner defined by the desire and the ability to do real software development, unhindered by unqualified managers and incompetent team members trying to drive the bus.
Who's with me?
October 28, 2015
Focusing On Typing Speed Not Only Misses The Point, But Can Exclude Talented DevelopersAfter harumpty-three years of programming computers, I've long ago abandoned the notion that how fast I can type out code is a limiting factor on how soon I can deliver valuable working software.
Typing, as an activity, is a such a tiny part of what software developers do, that you could be a really slow typist, working in a clunky language full of redundancy and clutter - like Visual Basic - and still get more done than someone who can bash out 100 w.p.m in a tight, streamlined language like Ruby or Haskell.
If you don't believe me, look back at your commits for the last week, count the number of words you typed, and divide that by the number of minutes you worked. I average about two words a minute, which adds up to about 800 LOC a week, or 40 KLOC a year.
Now, my boss - who's sitting with me right now - may mistakenly believe that I must be pressing a key once every thirty seconds. And 40 KLOC doesn't sound like very much at all, until you qualify that with "on legacy code, which most developers spend most of their time working on, in a team, with customers and so on". In other words; real software development, with all the usual obstacles. Starting with a clean slate, working alone, with no customer to seek input and feedback from, I move faster - maybe 100-200 KLOC a year. But, arguably, a lot faster towards solutions of less value. Which opens up a whole debate about what "developer productivity" is anyway...
My actual typing speed - out on the open on a flat surface with the wind behind me - is about 30 w.p.m. What's actually happening here is that I spend very little of my time - less than 5% - typing code.
If I was a 60 w.p.m typist, my productivity - in terms of working software delivered - would not double. It would barely change, because how much I achieve per hour/day/week year is not constrained by the physical act of typing in the code to any appreciable extent. An average of 2 w.p.m. is a reflection of how fast I make decisions about the code. Every line of code contains decisions - names, types, dependencies, formulas and expressions, strategies for solving problems etc. And then there are all the changes: I rarely leave a line of code untouched once it's been written. Refactoring accounts for a lot of the time I spend working on code.
But - and this is backed up by a mountain of evidence - by far the bulk of mine and most developers' time is spend reading code; typically between 50-80%. Which is why my efforts are most focused - after worrying about whether or not it works - on how easy the code is to read.
So, maybe 5% or less of my focus is on write-ability of code, compared to 50% or more on readability. To put it more bluntly, readability matters 10x as much to me.
But our ingénue profession still obsesses mostly over write-ability of code, not on readability. And those of us who can look back on many wasted years servicing this obsession - and it is years, over the lifetime of a career - may sound the warning bells for younger developers still chasing the illusion of productivity. But, it seems, the message "go slow to go fast" still hasn't broken through into the collective Developer Borg consciousness, even within the software craftsmanship community.
But, hey, it's your time and your customer, and who am I to say how you should waste it?
Where I do have a problem, though, is in how this obsession with how fast we type the code can exclude developers with disabilities that make faster typing an impossibility.
I've coached and mentored developers with disabilities, and the first thing I learned is that as much, if not more, can get done. Typing isn't the bottleneck; thinking is the bottleneck; feedback is the bottleneck; cost of change is the bottleneck. Typing is, at worst, a minor irritation to impatient people.
In the UK, and many other developed nations, you're not allowed to discriminate against a software developer on the grounds of disability. But, sadly, I've seen teams do exactly this, entirely on the basis that they are too slow at typing. "They wouldn't be able to keep up. Nobody would want to pair with them. They'll hold the team back."
It's bullshit. And dangerous bullshit, because it restricts opportunities to some very talented developers, which is bad news for them and bad news for you and bad news for the world who could have benefitted from the fruits of their labours.
Just as we need to get past the face-time culture of many development organisations to open the doors to talented programmers with other commitments, like parenting, we need to get past this obsession with typing speed because:
a. It really doesn't make much difference, and
b. It focuses us on an relatively unimportant skill that can exclude some really great developers from joining the party
10 w.p.m. is plenty. It's 5x what you need. It's not they who need to speed up. It's we who need to slow down. Take a deep breath, and focus on the stuff that makes a difference. Hey, maybe think about the problem a bit more? (I know - kerrrrazy!)
October 19, 2015
Software Craftsmanship 2016 - Would You Like To Host An Official Event Where You Are?So, finally, evil plans are taking shape for the revival of the original software craftsmanship conference.
Started in 2009, SC20xx ran for 5 very successful years, and saw innovative new hands-on sessions from some of our best code craft thought leaders.
The unusual "less talking, more coding" format and the distinctly community-led nature of the events made for a fun-packed and challenging day, and my aim is to build on that going forward.
At SC2016, talks will be banned altogether. Nope, not even Open Space talks, or Lightning Talks, or Talks Wearing Various Naval Headwear. We're making the final progression, and transcending traditional conference formats altogether. No talking, just coding.
SC2016 will bring together likeminded passionate programmers, simply to work on pre-set problems, pair program, share ideas and have a laugh - all without the annoyance of someone (probably like me) talking at you.
The aim will be to develop a DVD-ROM or USB stick crammed full of project ideas and exercises for code crafters - some short and self-contained, some larger and more ambitious. You come along, grab a coffee, find someone you like the look of, and get to work. We'll provide desks, chairs, Wi-Fi, doors, windows, walls, ceilings, etc - and coffee, obviously - and mentors on hand to help out with any technical issues.
And here comes the best part: because the conference will effectively come on a disc, it suddenly becomes possible to unshackle the event from its geophysical constraints. To put it more bluntly, other folk can run an SC2016 event in their area. You will need to find some suitable space (e.g., a cow shed, or dry lake bed which inexplicably has Internet and power sockets), provide some basic refreshments (water, tea/coffee, a simple buffet lunch if you can stretch to it), and - provided you stick to the rules - we'll list your event among the official ones on the SC website.
More details will be finalised in the coming few weeks, but here's what I already know:
When? Saturday May 14th
Where? Anywhere suitable (tables, chairs, Wi-Fi - basically, an office or meeting space)
What? A packed day meeting likeminded craftspersons and having fun tackling handpicked projects of all shapes and sizes
Who? Anyone who gives a shit about writing good software, and about getting better at it
How much? The plan now is to try to offer places for free, though organisers of individual events may choose to charge a small fee to cover their nominal costs (e.g., if they can't find a sponsor to pay for catering).
And the expenses? Costs will be managed at a local level by event organisers. How much it costs will depend on where you host it, what you do about catering, and so on. Logistically, SC2016 events will be very similar to the kind of meet-ups and Code Retreats groups run locally all the time, and we expect costs to be in the order of £100-300, assuming a venue is provided for free. Of course, if you have 10,000 at your SC2016 event, and want to feed them lobster for lunch, your costs will be significantly higher!
If you're outside London (I'll be running it here), and interested in hosting or running an SC2016 event where you are, please get in touch.
Remember - can be anywhere in the world: India, the US, Canada, Spain, France, Narnia, Gallifrey or Trumpton.
Drop us a line.