Software People Inspiring

ICT500 - Finding & Nurturing The 0.1% Who Could Be Great Software Developers

For 2 decades, I've worked as a software developer, as well as training and coaching developers in the disciplines of software development.

Through my work and through conferences, workshops and other events, I come into contact with hundreds of software developers every year, and I get a good overview of what's going on in our profession.

And a "profession" it most certainly is. While computer programming is relatively easy to pick up, writing software commercially on any appreciable scale takes decades to master.

There is just so much to know, and so many practical skills that take thousands of hours of practice to really get to grips with. As a discipline, software development is every bit as complex and challenging as physics, medicine or teaching, and requires a lifelong commitment to learning, practicing and improving.

Beyond the obvious and many technical skills developers need to keep up to date with, software development also requires a high standard of general education in maths, written and verbal communication skills, and strong-enough general knowledge from which to quickly build a computable understanding of a wide range of problems that software is used to solve.

Year on year, I see the standard of software developers joining the profession slipping. They tend to be not quite so good with the technical skills, not quite so good at communicating, not quite so proficient with maths and logic, and not quite as well-informed and worldy-wise. Companies that hire young software developers are finding it harder and harder every year to recruit people of the right calibre. I work with some clients on recruitment, and I've been finding that - on average - perhaps 1 in 100 software developers that might apply for a job are really up to snuff. You have to kiss a lot of frogs to find your Prince Coding.

The impact of this on the businesses who rely on software developers, which, these days, is pretty much all of the FTSE500, can be profoundly damaging. While the media cries over the lack of good talent available to the computer games industry and "digital creative media", they completely overlook the fact that the bulk of our economy has been "digital" for decades - with computers playing an increasingly central role in the operations of all organisations that have to do things on a large scale.

If Acme Supermarket Plc has to wait to start a software project until they've found a team good enough to deliver, then Acme have to wait to roll out changes to the way their business operates. I've seen first-hand household British brands brought to their knees by delays in new IT. These days, your ability to create and adapt software systems is a very limiting factor on the ability of your business to adapt and stay competitive.

The importance of software development to the UK goes far beyond business, though. Our entire scientific and engineering base is very heavily reliant on software that has to be written specially. Increasingly, we find the lack of software development skills in science is holding UK R&D back. Our future prosperity and quality of life depends on our ability to discover and to innovate.

For all these reasons, I see programming in schools as an imperative. We desperately need new talent. Not just games companies like Eidos, or special effects companies like The Mill. We all need the decline in Britain's software development capability to be reversed.

My theory is this: hidden among the millions of children in school right now are maybe 1,000 who would make amazing software developers. And they, and we, may never know it. Not only could they be robbed of a potentially very rewarding and personally fulfilling career, but we would all be robbed of the fruits of their labour.

1,000 really great software developers entering the profession every year could, over a decade or so, tip the balance.

Firstly because the best software developers tend to be an order of magnitude more productive and creative than the average, so 1,000 great developers could achieve the same as 10,000 so-so developers.

Secondly because the more great developers come into the profession, making it easier for employers to find good people, the more the so-so developers will be forced to raise their game to stay in work.

We need to find out who these 1,000 kids are. We need to give as many children as possible the chance to find out for themselves whether programming is for them. For the vast majority, it won't be. But for that 0.1% who may find they really enjoy it, and have a real talent for it, we would be shooting our economy in the foot by not giving them the chance to discover programming - and ultimately software development - for themselves.

And once they've been identified, we must move mountains to fuel their passion and remove all obstacles to their development as, well, developers. This will require focused, targeted collaborations between practitioners like me, and the software development community of which I'm a part, as well as schools, teachers, parents, employers, the media, and everyone else who has a stake in the outcome.

I don't much care for kids being made to learn "computer science" (which is not the same thing, by the way). That sort of thing can come later, for those children who - fuelled by burning curiosity and desire to do more, go further and build better software - seek it out. It should be there waiting for them.

First, we need to give as many children as possible the chance to try programming. Not for its own sake, but because you can make cool stuff with code. Programming as "making cool stuff" should be the dominant model until that 0.1% reveal themselves, and then we can start talking theory and maths and discipline.

For my own part, I'll be focusing this year on getting kids coding, by finding support for teachers who want to learn more through a teacher-practitioner coaching programme, and also by devising "cool projects" for kids and for schools with themes ranging from pop music to alien hunting.

I'll also be working with the developer community to start fleshing out a framework for identifying and nurturing that little nugget of raw talent that may emerge.

Non-functional Test-Driven Development

It's the question that comes up everytime I introduce someone to Test-driven Development: "But what about performance?"

The thing about TDD is that adage "be careful what you wish for" applies. The solution we end up with is constrained by tests. There may be a million and one ways of achieving a goal, and some will perform better than others. The trick with TDD is to ask the right questions.

What I like about TDD - and similar precise approaches to defining requirements - is that it forces us to be explicit and unambiguous about what we want from our software.

So, my stock in trade reply to the question "But what about performance?" is "yes, what about performance?"

Software performance has different dimensions, and if it's important then we need to define exactly what performance we require in specific scenarios. A great way to do this is using non-functional tests.

There's the dimension of time, for example. How long should it take for the code to run?

Imagine a search algorithm that looks for a customer name in a sorted list. We could just loop through the list, and if there are only 1,000 customers and the occasional search, that might be fine. But if there are 10,000,000 customers and users are frequently searching, then a simple loop probably isn't going to cut the mustard.

We can constrain our search algorithm with a basic timing test, like the one below, that makes it explicit that our worst case search - the customer we're looking for isn't in the list - should take a maximum of 1 millisecond to complete.



Execution time is only one dimension, of course. What if we need to constrain the memory footprint of our code while it's running? In Java, we can use the JVM to get information about memory usage, and we can create a multithreaded test to monitor how much more memory is being eaten up as our code executes. Let's imagine we need to constrain the memory footprint when sorting our list of 10 million customers by name, forcing us to use an in-place sorting algorithm that uses up a maximum of another 10KB of memory:



And here, with massive caveats for my less-than-amazing knowledge of the Java Runtime (I make no warranties, the value of shares can go down as well as up, etc etc):



Leaving aside the fact that my brute-force method for calculating memory footprint is a bit hokey (and on running the tests several times, quite variable, it seems), the basic idea is hopefully useful. No doubt some fine fellow will point out a much better way.

You may be able to envisage now how we could use tests to explicitly constrain other non-functional runtime qualities of our code. But we can also often find ways to constrain code at design time, too.

We might have a requirement that our methods should be short and simple. Static code analysis tools like XDepend and Checkstyle can give us hooks into the structure of our code and enable us to create tests that, when code fails to live up to our quality standards, alert us to that fact early enough to do something meaningful about it.

Using executable tests, we can steer our software between acceptable limits of performance, scalability, portability, maintainability, and a whole heap of other -ilities we might care about.

But what about the more, how shall we put this, etheric -ilities, like usability, accessibility and so on? These things tend to be pretty ill-defined and qualitative. Can we make them explicit and testable, just like execution time or memory footprint?

I believe that we can, and to without reason, because I've done it and seen it done. We could, say, define a test that fails if a carefully selected group of target users (e.g., legal secretaries with more than 2 years Windows and web browsing experience), when presented with our application for the first time, fail to get their heads around it fast enough to complete certain tasks we set them within a specified time, without any help or documentation.

With a bit of imagination and lateral thinking, it's possible to meaningfully test many more software qualities than we usually do. And my experience of non-functional TDD is that we tend to get what we have tests for, and we tend not to get what don't have tests for. So agreeing executable non-functional tests tends to lead to better non-functional software quality, if it's done well.

As I warned before, though, be very careful what you wish for.

Jason's Handy Guide To Evaluating Software Packages

I get asked this question a lot, but it never occurred to me to write down my usual answer.

How do we evaulate shrink-wrapped software against our needs?

Well, that's easy. You still need to do the usual business requirements analysis. Identify who will be using this system, and what their goals will be for using it. In the good old days, we called these "Use Cases". Yep, even if you're buying and not building the software, you still need use cases.

The next step is to flesh out the design of your use cases, as we might normally do, by describing how the user interacts with the software to achieve their goal.

When we're describing software we haven't built yet, this is design. When we're describing how we'll use software that already exists, this is a process of validation. Can the user achieve their goal using the software we're evaluating?

Even with the most feature-rich packages, we tend to find we don't get an exact match. It's not always possible to achieve every user goal using the software. So as we validate the software against our use cases, we may identify gaps. There are almost always gaps.

The next question we need to answer is can we fill those gaps? Let's say we're evaluating Microsoft PowerPoint for our training business. It doesn't do everything we need out of the box. Let's pretend we have a use case where the trainer needs to populate a slide with an organisation chart showing the reporting structure of the group attending the course. She has a spreadsheet with those names listed in alphabetical order and with information about who reports to whom. using PowerPoint's built-in scripting language, Visual Basic for Applications (VBA), it is indeed possible to take that information and automatically generate an Org Chart.

So that gap could be plugged, with some work. Write a reminder about it down on a blank index card. This is now a potential "User Story" for some programming work that would need to be done if we went the PowerPoint route.

Of course, people identify gaps in software all the time, and it's possible that someone somewhere has already found a solution to plugging some of your gaps with handy tools and utilities. Google is your friend here: search for solutions before you think about reinventing the wheel. If you find one, and there's money involved, write down roughly how much on the index card.

Finally, don't forget the non-functional requirements. A package may offer the right features, but it may not be able to handle a high-enough volume of users, or it may not be secure enough for your purposes, or it may take a long time for users to learn. Evaluate thye software against these criteria, too. Be as explicit as you can. Handwavy requirements like "it must be scalable" aren't very helpful for validating software. What do you mean by "scalable" - a certain number of users at any one time, or a certain number of transactions per second, or the ability to run it on more servers?

All too often, businesses buy a solution and then validate that it does what they need - often by actually trying to roll it out. Whether buying or building, the key is to have clear, testable requirements and to validate the software against them. Don't be seduced by their sales patter and let them lead you like a donkey to the slaughter to their feature list. What their software does is far less important than what we can do with their software.

New Bletchley Park CEO, And A Tribute To Simon Greenish

It's been announced today that Bletchley Park has a new CEO, Iain Standen, a recently retired Colonel from the British army.

Before he disappears into his lovingly tended garden and a life of well-deserved leisure, I just wanted to pay tribute to Simon Greenish, who has been the key factor in the turnaround of Bletchley Park's fortunes since he took over in 2006.


Simon shows Colossus to kids' TV genius Johnny Ball, while the late Tony Sale does his famous velociraptor impression

Five years ago, Bletchley Park's finances were in bad shape. There was a very real danger of it folding due to lack of funds. I know first-hand how hard Simon has worked - harder than most people could even begin to imagine - and he's received very little recognition outside of the staff and volunteers there.


Two maths legends. And Alan Turing.

For every pound raised by enthusiastic supporters like myself, Simon and his amazing team have raised £50, working madly long days and suffering untold stresses and strains to secure funding from a range of public and private bodies. It's this work - often done in secret, and often not reported - that has earned Bletchley Park it's future, and for that I'm hugely grateful. I have no doubt that, were it not for Simon Greenish, Bletchley Park would now be in the hands of property developers.

As it is, Bletchley Park's future looks very positive indeed. Like the best boy scouts, Simon has left it in a much better condition than he found it, and - while there'll be many more challenges to come for Iain and the team - things are definitely on the up.

So, I raise a glass to you, Dr Simon Greenish, saviour of Bletchley Park. Top bloke, that Greenish chap!

What Can We Learn From Dream Theater's Drummer Auditions About Hiring Great Software Developers?

I'm sure I can't be the only software developer who thinks the way we hire for our teams is completely f**ked up in a lot of cases.

When I compare it to how, say, bands hire musicians it seems we care a heck of a lot less about who we work with. I know some of you will complain that hiring is often taken out of our hands, and therefore how can we to be to blame? Well, if we let such important decisions be taken out of the hands of the people who can tell the difference, then maybe it's our fault all the same.

One of the most technically demanding jobs in music is playing drums for prog rockers Dream Theater. After their previous drummer and co-founder, Mike Portnoy, quit the band, the search began to find a replacement. These were very big shoes to fill. There may only be a few dozen drummers in the world who play those parts well enough, and fewer still with the right temperament to fit in with a long-established band.

How does a band like Dream Theater go about finding that person? Do they pick up the phone and call Drum People Inc. and ask for a prog rock and metal drummer to start rehearsals on Monday? I can just see it now - the agent persuading traditional jazz drummers to add "prog", "metal" and "double bass" to their CVs.

And did Dream Theater leave the hiring process to their Director of Human Resources - a tone deaf, musically illiterate Justin Bieber fan who has not even the most basic appreciation of what it is Dream Theater actually do?

No, that would be silly.

How does a band like Dream Theater go about finding the right drummer, then? Well, it starts with drumming, oddly enough. That is, they listen to a lot of audition tapes sent in by drummers from all around the world. They must have received hundreds of them, so that's a few days of dedicated listening. Luckily, you only have to hear a drummer for a few minutes to know whether they'd be up to the standard of someone like Mike Portnoy. But it's still a lot of listening.

But this is a critically important decision for the band. In a 20+ year career - mostly with the same brilliant drummer - what's a few days listening to audition tapes?

The ones who stand out go into the "maybe" pile. And, of course, most drummers of any note have things like web sites and blogs and Twitter accounts and YouTube channels these days, and they and their bands are discussed in various message boards and on other kinds of social media. That is to say, they have a searchable public profile, and catalogue of music we can check out, and a reputation.

They can even watch drummers rehearsing for their audition.



By checking out the portfolio of work the drummer's done and researching them online, Dream Theater were able to whittle down hundreds of auditionees down to the three drummers they thought would be worth auditioning in person.



You can tell a lot about someone from their work, but what you can't tell is how they'd work with you. That's really what an audition should be about - not "can you play this tune?" but "what does it sound like when you play this tune with us?" and "can we write a new tune together?"

Auditions aren't about technical ability - that should already have been established before you even think of inviting that person in and taking up everyone's valuable time. Auditions are about the dynamic: how does this band perform with this person integrated into it? What would Dream Theater with Peter Wildoer, celebrated Swedish death metaller and jazz fusioner, sound like? Possibly a bit heavier - a bit more technical death metal and a little less Asia or Yes? That might be why he didn't get the gig. There's no doubt he's up to the standard of Portnoy. Wildoer's one of the best. He just wasn't the fit they were looking for.

Most notably, when I watched the band's video of the auditions, Wildoer was not spitting feathers because they'd rejected him. To him, being one of one only three drummers in the world even invited to audition was a great experience. God knows I'd be flattered if they even listened to my audution tape for more than 30 seconds!

He was at their level musically and technically, and they weren't wasting his time. The audition video's been viewed 1,000,000 times. That's good exposure and good experience. To get to play and jam with musicians of that calibre, and with that level of passion and professionalism, and have a good time to boot, is not really a waste of anyone's time.

If a great team flattered me by inviting me to "audition" for them, and my technical ability was not under question because we'd already established I was technically good enough to work with them based on my portfolio and reputation, and they said "Hey, Jason, come and spend a day with us and let's play some of the old tunes together and maybe jam a few new ones", and they paid my expenses and put me at my ease - then if I didn't get the job, I'd still feel like I got something out of it.

TDD Is Neither Necessary Nor Sufficient For Good Design

Dan North (@tastapod on That Twitter) rightly points out that TDD is neither necessary or sufficient for good software design.

I make no bones about it. TDD is not the only way to achieve high quality, reliable, maintainable code.

He tweets "Some of the worst code in the worst codebases I've ever seen was 'strictly TDD'."

Flip it over, for balance, and I can also honestly claim that some of the most horrendous code I've ever seen was not test-driven. Based on admittedly small-scale studies, TDD'd code tends to be simpler. But good design is about a lot more than that.

Which is why my TDD course focuses on these aspects of the discipline - the simplicity, the refactoring (LOTS of refactoring!), readability, minimising duplication, localising dependencies ("Tell, Don't Ask"). These basics should be core to any TDD text or course. If more programmers paid attention to just those basics - simplicity, readability, duplication and localising dependencies (darn it, if they just put more effort into readability!), the software world would probably be a better place.

But I also make it clear that those basics will only get you so far, which is why I have 3 other courses that all - from one angle or another - focus on software design. All of my courses are really design courses.

In the refactoring course, we learn about common code smells and how to safely eliminate them. In the OO Design course, we learn about SOLID and other dependency management principles, as well as how to more objectively identify dependency issues, and then refactor the code to address them.

In the Agile design course, teams use a bit of collaborative up-front to establish a shared understanding of the basic architecture and co-ordinate their efforts to flesh out and implement different features of the same software. When they implement it, they test-drive it, and continuously integrate, and refactor and etc etc.

It all dovetails nicely together to hopefully present a more rounded picture of software design in an Extreme programming sort of approach. But altogether they are still not sufficient.

There's a knack to good software design, and a whole heap of other stuff you need to know. I've been a student of software design for decades, and I'm just getting started.

So Dan's quite right. You need more than TDD, and there are other ways of achieving good design.

TDD & Binary Search II - A Divide-And-Conquer Test List

Still with the Binary Search problem from the previous blog post. Let's go back to where the problems started:



I think my mistake was to proceed from here in a linear fashion. It is, after all, a divide-and-conquer algorithm. Binary Search works a bit like splitting a deck of cards and then looking at the card there in the middle. if that's not our card, we ask whether our card is higher or lower. If it's higher, we discard the middle card and the bottom of the pack. then we split the top half in the middle, rinse and repeat until the card in the middle is the one we're looking for. (Or we run out of cards.)

So it feels intuitively to me that I should be "splitting the pack" with each new test case, and that, for here on in, we should be finding the number we're searching for.



The simplest implementation I could think of was:



This was encouraging. It's starting to look a bit like the iterative Binary Search algorithm I was going for. We now have this "middle" variable, and we do the comparison there.

Next, I thought, what if our item is not in the middle of the list, but in the middle of the top half of the list?



Which brought me to introduce a movable "bottom" to the array and introduce the loop (since I now had two iterations):



And what about if our item is in the middle of the bottom half of the list?



This test doesn't terminate if you run it with the existing model code, by the way. Just thought I should mention that.

I can pass this test easily by checking if the key is lower than the middle number, and by introducing a movable "top" for the next iteration so we can discard the top half.



Now, I could be wrong - it is late on a Saturday night, and I've been staring at a spreadsheet most of the day - but that's Binary Search, isn't it?

It still feels a little hokey. Especially the leap to iterating. I suspect there are smaller steps in between some of these steps, and will be looking for those next. But the gist of it feels right. The "knack" to test-driving a divide-and-conquer algorithm might be to divide-and-conquer with your tests, rather than follow a naive linear path.

The next big question is: how would I know, if I didn't know I was shooting for a non-linear algorithm, to follow this kind of roadmap?

I think the answer lies in doing some thinking about performamce. If we're looking at potentially larges lists, then a time complexity of O(N) sounds undesirable when N is a big, big number. Basically, we use our experience and a bit of design ken. No surprise there, as that's the key to cracking any problem in software design. There's no algorithm for algorithm design. I might have interspersed these tests with the kind of complexity tests I demonstrated in the previous blog post.

Progress?

ADDENDUM:

For those of you who favour recursion over iteration, it's a 2 minute job to refactor to:



Good job we wrote those tests, eh?

TDD & Binary Search - Not As Easy As I'd Thought

So I was noodling about test-driving an implementation of Binary Search (to see how a divide-and-conquer algorithm could be TDD'd), and something interesting came up.

My usual starting point is the simplest failing test I can think of - that is, the one that would be easiest to get passing. In this case, I started with the case where the item I'm searching for isn't in the list:



Triangulation begins by very simply returning the value it expects: -1



So far, so easy.

Then I looked for the next nearest failing test. I figured an array of only one element, where the element is the item we're looking for:



And pass it thus:



The next nearest failing test I could think of was an array of 2 numbers. The first number couldn't be the key value we're searching for, because our code would already pass that test, so I make it the second item in the array:



(With a little bit of clean-up to localise the knowledge of how to interact with our BinarySearch object, of course. I'm a good Boy Scout.)

Now here's where I hit a snag. The simplest way to pass this test I instinctively thought of was to use a loop. This follows Uncle Bob's "Transformation Priority Premise", sort of. I went from a hard-coded value to looking at a variable and an IF, to using a loop. So, to my TDD instincts, this feels right:



It's simple. It passes the tests.

And that's just it. It will now pass all of my tests. Provided the array is sorted and contains unique integers (my pre-condition), any array and any key value will be either correctly found, or not found. No more failing tests.

But I ain't done yet. This obviously isn't Binary Search. If I created an array of, say, 1,000,000 integers, it might have to do 1,000,000 comparisons in the worst case. In terms of algorithmic complexity, it's O(N) complex. I want it be O(log2(N)).

But where TDD's concerned, I'm done. No more failing tests. Surely?

I've come up against problems like this before, and what I will ususally do when we realise that an implementation is going to be too inefficient is to start writing tests about the performance or efficiency we require.

As folk on That Twitter were pointing out (over and over again - curse you Twitter for having such a short memory!), we can actually make assertions about algorithmic complexity by finding a way to expose the number of comparisons our search does.

For example, how many comparisons should it need to do to discover that some key value isn't included in an arbitrary array of 10 integers? It should be no more than 4.

A simple brutish way of exposing this information could be to add a feature to BinarySearch that tells us how many comparisons it did for the last search:



And we can test it like this:



We could, of course, do something more sophisticated using mock objects that wouldn't require this slightly unsightly addition to the BinarySearch interface. But with integer arrays, we're a bit stuffed, frankly. So ugly and brutish it is. (I actually used a test spy injected into the constructor that collected the comparison count when I first did it, but then thought "oh, sod it - it's just integers!"). Mocks present another problem. If the things we're comparing are mock objects, there's some considerable hill-climbing to do to set up a sorted collection of unique comparable mock objects. Fiddly, and I'm not sure it's worth the effort.

I can now write tests that force me to optimise my search implemention until it's properly O(log2(N)). But these new tests don't give me any real clues as to what that final algorithm could look like, just that it has to be X efficient. So, while it's a step towards test-driven algorithm design, it's not test-driven algorithm design. The tests don't tell me what code I need to write.

Now, this is an artificial exercise. I know I wanted to end up with Binary Search. If I know the answer I want to get, technically it's not "design" and not really TDD. But that wasn't my goal. I wanted to try and figure out how I could have test-driven Binary Search, had I not known that what I was going to end up with that specific algorithm.

Is TDD a useful tool for algorithm design? To be honest, I've come away from this little exercise with my doubts that it is, in this case.

Some algorithms wear their hearts on their sleeves, when the internal design of the algorithm is a close match for it's externally-visible behaviour. e.g., an algorithm to convert integers into roman numerals (or Farenheit into Celcius, etc).

Binary Search, on the other hand, is a secretive little bugger. You can't tell it's Binary Search just by what it does, and therefore it's hard to drive its design from the outside.

Some argue - rightly, I think - that the O(log2(N) requirement is a feature of Binary Search. I'm inclined to agree, but it doesn't point towards that algorithm, any more than "I want to pay income tax less than X%" points towards moving to Switzerland. The requirement and the implementation are more orthogonal.

Now I took a different tack doing it again this morning, and made a little progress towards understanding how, had I been tasked with implementing an efficient search algorithm I might have let a different sequence of tests lead me towards Binary search, or some other divide-and-conquer algorithm. So I haven't lost all hope yet that TDD isn't going to help in designing new algorithms. And certainly adding tests that fail if our algorithm isn't performing as we need it to can help constrain us, steering us away from inefficient algorithms early in the process.

It's been a fascinating process, and as I noodle with other kinds of well-known algorithms I'm fully expecting now to come up against other challenges. And while it can be frustrating, I take that as a sign that I'm learning something.

My hope is that having 250 smart and passionate programmers working on algorithms and the cross-over with craftsmanship and Clean Code will throw up all sorts of interesting problems, and potentially new solutions. To quote Simon Pegg in Star Trek: "It's exciting!"

Finally, as an aside, here's another approach I've used many times to constrain test-driven designs with non-functional requirements, which is very simple and equally brutish, but allows us to think in terms of things a user will experience (like "what's the maximum time this search can take?"):





I've also used similar techniques to write tests that fail if a certain memory footprint is exceeded, and I'm sure it could be applied to other kinds of performance requirements. IF YOU HAVE THEM - I should add. (Don't go making them up and optimising willy-nilly!)

Why Evidence And Reason Haven't Changed Software Development

As 2012 approaches, my mind lingers on matters of faith. More particularly, our limitless capacity for believing the impossible, even in the face of overwhelming evidence that what we believe isn't true.

A classic case study in hypercredulity is that of self-proclaimed UFO contactee Billy Meier. Billy was a Swiss farmer who, in the 1970's, came forward with tall tales about visits from beautiful - and suspiciously Swedish-looking - aliens who took him into their spacecraft and whisked him away for tours of the universe (and, eventually, the distant past).

Meier claimed the visitors came from the Pleiades (astronomers may know it as Messier object M45), and he supported his stories with a series of very spurious - and really rather comical - still photographs and home movies.

Putting aside the fact that the Pleiades star cluster is dated by astronomers as being between 75 and 150 milions years old - mere newborns compared to our Sun - and that it's very unlikely such an advanced technological civilisation could have emerged in so short a time, Meiers physical evidence has been roundly debunked with little effort.

Photos of Pleiadian visitors have been positively identified as stills from Meier's favourite TV show, Rowan & Martin's Laugh-in (and, yes, I too can believe that Goldie Hawn was sent from heaven, but certainly not from the Pleiades.) Photos of dinosaurs allegedly taken in Earth's prehistoric past (as well as dinosaurs photographed on other prehistoric planets - yes, his claims really were that naive) have also been positively identified as illustrations from a children's book.

His flying saucer stills and movies not only look laughably fake, but have been very accurately reproduced by amateurs with no model-making or special effects experience using materials that were readily available to Meier. One especially keen debunker even took the trouble to perform his recreations using only one arm, as Meier must have done.

Meier's own ex-wife has denounced him as a fraud. And the list continues, piling up a mountain of evidence that the Billy Meier UFO contact case is a hoax - and not even a very good one.

And yet, millions still believe every word of it. You can point them directly to the evidence of a hoax, and they'll believe his story even more. The more they see that contradicts his claims, it seems, the more fervently they believe them.

Faith is like one of those knots that, the more you struggle to get free, the tighter the knot becomes. Thousands of years of organised religion have conditioned many people to see faith - unquestioning belief without evidence (or even in the face of contradictory evidence) - as a virtue. When people can view reality through a lense that tells them that when the facts contradict the belief, they should believe it even harder, we get ourselves into all sorts of pickles.

It's as though we have some special compartment inside our brains that's insulated from things like reality and logic, where we can ring-fence off certain cherished beliefs so that the real world can't get at them. One wonders whether chimpanzees have a similar ability.

And, like it or not, I have to remind myself that:

a. I have one of these compartments, and should therefore be more mindful about which of my beliefs enjoy this special protection, and...

b. There are things people believe about software development that have obviously somehow ended up in their reality-proof mental "panic room", that are impervious to evidence or reason

For example, the whole Mythical Man-Month thing. We have a pretty overwhelming body of evidence now that adding more developers to a team won't produce the software any faster. And yet managers still have this knee-jerk reaction when the schedule's slipping to hire more devs. I've had this discussion with managers many times, presenting the evidence and reasoning about the causes logically and dispassionately. But usually to no avail. Most managers believe that a bigger team will get more done, and the more that belief is undermined, the harder they cling to it. So much so that, after adding a few more developers makes the situation worse, their solution is to hire even more. And then more again, until we've got a team of 100 developers tripping over each other's feet and struggling to produce the useful output of a team of 6.

Earlier this month, I posted a link to the proceedings of a DoD workshop on "software engineering" from the late 1960's, in which many of the issues we face today were discussed, and solutions proposed that bear an uncanny resemblence to what experts recommend today. In the intervening 43 years, little has changed except the scale of the problems. If small and frequent releases, close customer collaboration or test automation was considered advisable then, it's 100-1,000 times as advisable now. And yet, as an industry, we still resist, clinging on to the same old misconceptions that were tripping teams up since the dawn of modern computing, and in the face of a large body of evidence and experience that's grown in the meantime.

To me, it's surely not that hard. Just as I can look at Billy Meier's photos and see them as obvious and derisary fakes, I can see so very clearly the handful of genuinely simple things software teams could literally just decide to get right that would boost their chances of success enormously.

But I have to accept that there are people who look at what I look at, and just don't see it. They can't. It's like trying to make someone to see the tiger in a Magic Eye picture. There's a part of their brain that won't allow it, and my well-meaning attempts to penetrate their Fortress of Woo and ground projects in reality will more often than not just make them believe what they believe even more.



(And, no, the world very probably isn't going to end next December. In case you were wondering.)

SC2012 - Computer Science For Software Craftsmen

I promised myself I wouldn't work until the New Year, but just wanted to put this idea out there.

June 14th is a date to put in the diary for software craftsmen. The fourth Software Craftsmanship conference (SC2012) will be at it's spiritual home, Bletchley Park, just a week or so before the 100th anniversary of Alan Turing's birth.

As well as Turing, education's very much on my mind. So I've been kicking around an idea that could address an important educational gap in UK software development - the yawning divide between theory and practice.

I work closely with many employers, large and small, who are looking for great software developers. Many - probably most - UK developers are self-taught, including me. What employers tell me is that the self-taught programmers tend to hit the ground running, compared to programmers with computer science degrees (a surprising number of whom struggle to program at all, having had so little hands-on practice during their studies.)

But I also note from my own experience that many self-taught programmers have big gaps in their understanding of things like logic, discrete maths, data structures and algorithms, languages and compilers and so on. I include myself in that. I've cherry-picked elements of computer science as and when I needed them, so while I know a fair amount of theory, it's by no means comprehensive.

We end up with CS graduates who've covered the theory, but often don't have a practical grasp of the applications of the theory (and, therefore, don't really understand it). And we have a majority of self-taught programmers who, when asked to sort a list of strings into alphabetical order, might by default stick them in a database table and use SQL to do something that I believe any programmer should know how to do. (I've seen that several times, believe it or not.)

I've watched teams run scared of anything remotely theoretical that comes up on their projects, and I've watched teams of CS graduates hack out some pretty shitty code because nobody taught them how to, say, write good unit tests.

Looking to the future, our software army needs to be better equipped. Of that I have no doubt. There's little competitive edge in spewing out apps that any programmer could build, nor in spewing out clever, but unreliable and unmaintainable, apps. These two worlds need to be brought closer together to produce more well-rounded software developers.

With this in mind, I want to make a proposal. With a Turing theme in mind, I'd like to focus the software craftsmanship conference - and the community that supports it - on marrying computer science and craftsmanship. I've been noodling with code katas based on simple CS ideas - e.g., test-drive an implementation of binary search - and I think this could be a way to go in bridging the gap.

For the practicing craftsman, we could leverage their practical skills to introduce some computer science in the very practical, concrete way that self-taught programmers tend to respond to.

For computer scientists, we could leverage their theoretical understanding to introduce elements of craftsmanship, and give them a practical, hands-on way to learn the theory by applying it, and get the necessary practice at programming many CS graduates seem to miss out on.

I'm proposing that the SC conference community put our heads together and write a book - or a series of books - under the banner of "Computer Science For The Software Craftsman". It would cover the basics that most CS graduates would learn, but every concept would be presented in the form of a code kata or other practical exercise that allows programmers to learn the theory by tackling applications in code, whilst ensuring a high-quality implementation using the techniques we know and love.

It could be a valuable learning resource for self-taught programmers like me, and a valuable practice resource for computer science students (and teachers).

We could complement it with online resources like screencasts and wotnot, so people can watch and learn and share - perhaps forming themselves into peer groups and working through the material learning from each other.

I believe this would be a non-profit resource owned and published by the community, and in the spirit of the conference, any money raised after costs would go directly to Bletchley Park.

I think a good place to start might be data structures and algorithms. We could organise into groups, each taking a specific topic (e.g., search algorithms) and work in pairs to design and test katas and write explanatory notes. As a community, we'd vote to select the katas we'd like to see demonstrated at the conference, based on screencasts done by each pair, and ensuring coverage of a sufficient selection for a book on the subject. (Yes, we'd use a divide-and-conquer algorithm to write our book on data structures and algorithms!)

I'm actively exploring ways to increase space at SC2012, so hopefully there could be 250+ attending - making 125+ pairs. If more than half of us comes up with something, we'll easily have enough useful material to make a rather grand book and a dandy accompanying web site.

Of course, if you think it's a crazy idea, you can voice your concerns to me on That Twitter - @jasongorman

But if folk are generally in support, we'll make a start on this in January, when tickets will also go on sale for the biggest, baddest SC conference yet.

And with that said, it's back to Xmas telly and mince pies for me. See you in 2012.