Software People Inspiring

SC2012 - Resources on Data Structures & Algorithms

A few people have asked about books and resources about data structures and algorithms aimed at programmers.

There's a wealth of material out there on the subject, some of it better than others. Some will offer a good theoretical overview, others are aimed at people like me who ultimately need to see a bit of code to get my head around it.

When I'm studying this sort of thing, it's sometimes good to hook up with other people who can learn it with me. If you're thinking of running a session at Software Craftsmanship 2012, you might want to pair up with somebody else who's going to make it a bit more social and a bit less work for one person. (And remember, you only need to learn one algorithm to run a session.)

Here are a few you might find useful:

Books on Data Structures & Algorithms

Data Structures & Algorithms in Java

Data Structures & Algorithms in C++

Algorithms In A Nutshell

Data Structures & Algorithms using C#

Data Structures & Algorithms using Python

Data Structures & Algorithms Made Easy

Notes On Data Structures & Algorithms In Ruby

(TIP: If you know Java or C++, you're spoiled for choice in this area. Many books available.)


Web Resources

Data Structures & Algorithms in Java (PDF version of book)

Data Structures & Algorithms Java Applets

data Structures & Algorithms in JavaScript (code on GitHub)

Animations for Data Structures & Algorithms

Videos

User risersheriff has posted some good videos on algorithms

Xoax.net's channel features lots of explanations of algorithms

MIT's Open Courseware has a series on algorithms

The Big Bang Theory - Friendship Algorithm (free drinks to anyone who runs a session on this one!)

I know that "data structures and algorithms" has a slightly daunting sound to it, but don't forget that you are creating data structures and algorithms all the time. The "people who bought X also bought Y" algorithm may seem daunting to someone who's an expert in DSP algorithms. I often find that once I get stuck into learning an algorithm, when it's explained clearly and I can see an example in code, it's just, well, more code.

Software Craftsmanship 2012 - Open For Registration

Now in its fourth year, SC2012 - in aid of Bletchley Park - is part of the official celebrations for the centenary of Alan Turing’s birth.




It’s going to be bigger and bolder, with an extra 100 places available for passionate programmers.

The theme is Computer Science For Software Craftsmen, and the money raised will be used to support two very exciting Turing-related projects at Bletchley Park.

For more information and to register, please visit the conference web site.

My Name's Jason Gorman, And I'm A Computer Programmer

My name's Jason Gorman, and I'm a computer programmer.

Like many with my addiction, it all started innocently enough. Like most kids, I experimented with a bit of coding now and then, but never thought it would take over my life in the way that it has. It was just a bit of fun, y'know, to make the weekends go with a bit of a swing. I thought I could handle it. But then, doesn't everyone?

The trouble really started when I fell in with a bad crowd.


A bad crowd, yesterday

With their encouragement I quickly progressed from solving simple coding problems involving just a few hundred lines of code, to working on whole systems made up of hundreds of thousands of lines. By the time I was in my mid-twenties, to my shame, I was doing hundreds of lines every day. And not just in the office to be sociable. I would wake up every morning and do a few lines before even thinking about breakfast. I would do lines at home, by myself - the sure sign of an addict, I later learned.

I realised just how bad it had gotten when I started getting offered jobs that didn't involve any computer programming at all. Apparantly, some hardcore addicts progressed from programming on to even harder drugs with streetnames like "enterprise architecture" and "Scrum coaching". Thankfully, I never got that bad. But things were bad enough. Within a few days of starting these new non-programming jobs I found myself getting the cravings again, and would sneak off to a quiet spot somewhere to program in secret. Some bosses were very angry when they found out I hadn't kicked the habit, and that I was still coding behind their backs. They said they felt betrayed.

Doctors tell me that my addiction cannot be cured. But the condition can be managed. They recommended I start a self-help group, which I did 3 years ago. We meet regularly - about once a year - to share our experiences and learn from each other. We also write code, but it's proper medical-grade, quality code and not that crap you often see being peddled on the streets.

If, like me, you're addicted to programming, and will only settle for the best-quality code available, why not pop along to our next meeting? It's going to be on June 14th at Bletchley Park. We promise a relaxed, non-judgemental environment where you can talk openly and honestly with other programming addicts and you'll be able to feed your addiction in a safe and sustainable way.

There'll be sandwiches, too.

Kids - You Don't Need Anyone's Permission To Learn To Program

I've been hearing from various concerned parents scattered around and about the countryside that their kids are being blocked from studying computing by their schools.

I had no doubt that there'd be schools for whom programming and computer science was not going to be a priority, but it's disappointing to hear there are schools that aren't prepared to lift a finger to help kids who really want to do it.

It seems extraordinary to me that a school that offers, say, a GCSE in Dance can't muster a few crumbs to help a handful of pupils take a GCSE in Computing. But hey ho. It is what it is.

If you're one of those parents, don't lose heart. I have some very good news for you.

I didn't study computing at school. Or at university. There. I've said it.

And I'm doing just dandy as a software developer. How did I achieve this feat? Easy. The same way the majority of professional software developers did it.

When many - well, most - employers are hiring programmers, a computing qualification is actually quite low down on their list of requirements. What's usually top of their list is relevant experience.

A programmer programs. And there are too many computing graduates going to the market clutching pieces of paper who can't actually program. Not really. There are plenty more who don't have that piece of paper who can program, and demonstrably have programmed. A lot.

How do they demonstrate they've programmed? By programming, that's how. Luckily for us, you can tell how good a computer programmer is by looking at their computer programs. It's a bit like painting or singing or juggling in that respect.

Most people who program for a living are self-taught - and that includes the ones who have pieces of paper.

If you want to be a software developer, you do need a high level of general education. You need to be literate, numerate, logical and worldly-wise. But many companies would be just as likely to hire you with a degree in, say, physics, and demonstrable coding skills as they would if you had a specialised computing degree. (And if you study any of the physical sciences, there will be programming, trust me.)

The door won't be closed to you because you didn't get to do GCSE Computing or a BSc in Computer Science. If you teach yourself to code in your own time, and keep at it, by the time you're 21, the fact that you have a degree in Astronomy or Economics probably won't matter to many employers. The fact that you've contributed to Open Source projects, or written XBox games, or iPhone apps, will. I'll take someone with a degree in Caribbean Studies and 20,000 lines of good, clean code under their belt over a computer scientist any day.

If you want to learn how to program, learn how to program. the Interweb is chock full of free programming tools and tutorials and screencasts. Just google "learn to program".

And if you're the kind of kid who needs a bit of motivation, hook up with some like-minded friends and learn and code together.

You don't need anyone's permission to get good at programming.

101 Programming Projects (for Schools & Kids)

There are some great coding tools designed for beginners of all ages. Greenfoot, Scratch, Kodu, Codecademy - the list is growing every month, it seems.

These tools are freely available, and there are oodles of tutorials and screencasts and wotnot out there for anyone who wants to learn.

A significant percentage of children have access to a computer they could learn to code on.

And yet they don't. Why is that?

This is the big question. For your average 8-16 year old - if such a thing exists - what's stopping them?

The answer is, in every practical sense, absolutely nothing. Anyone who wants to learn to program can.

Presumably, they just don't want to. Nobody's given them a compelling enough reason to type gobbledygook into a text editor and press "RUN".

When I started programming way back in the 1790's when the first steam-driven difference engines started coming into the home, it wasn't to learn how to program. For most of us, coding was not an end in itself. I learned to code because I wanted to try to write games. Manic Frogger, Jet Set Elite and Donkey Willy - those were the games I wanted to emulate. Well, maybe. It was a long time ago.

For the first few years, programming was a means to an end, and it was the end that fascinated me. First games, then graphics, then physics and astronomy. Finally, as I spent more and more time at the codeface, I began to see the beauty in code and could become excited about programming done well.

First we revel in banging the drum. Only later do we obsess over the optimum placement of the microphone and getting the EQ just right.

To get kids coding, we need to give kids a reason to code again. It's got to be a compelling reason. They can run games and operate TVs and text their mates without using an interpreted language. The reasons we learned to code aren't going to wash with the GUI and the plug-and-play generation.

Remember those "101 Electronics Projects" you could buy? (You still can, but whatever.) Some of the applications were pretty cool - crystal radio sets and digital clocks and knobs that could reverse time and so on.

I'm proposing 101 Programming Projects. There would be a tool - like Greenfoot, for example. And there would be 101 "recipes" for making your computer do really cool stuff by writing code using the tool. And I mean REALLY COOL STUFF. Like finding aliens. Building augmented reality drum machines. Or a Simon Cowell proximity detector. That sort of thing. Well, hopefully you get the picture.

Each project would have a cool outcome, and result in software that you can't buy or download anywhere. Because if you could, they wouldn't need to write it.

And, after all, that's the real joy of programming. You can make your computer do things that nobody has made their computers do before. You can create in the purest sense of the word.

I don't know if this would be a book, or a web site, or a downloadable thingummyjig, or all of these things and a commemorative plate, too.

What I do know is that we need something like this.

I also know that this will be too much for one man to take on. Even someone as brilliant and handsome as me. I intend to try, but I'm going to need help.

If 101 of us rolled up our sleeves, by developing one project each we would have our 101 Programming Projects. Many hands make lights work. Or something.

It would be great if something could be created in time for the next school year in September. That's a window of 6 months if we started about now. (Or... now! How about now?) Well, soon, anyway.

If you would like to help, and maybe go down in history as one of one hundred and one people who did a thing once, then let's talk. I'll be organising a meeting in London very soon, but we can do this remotely if you're one of those inconsiderate people who's chosen to live somewhere else.

And there may well be a bigger event, with sandwiches and everything, in early April where we can talk about this and the whole question of what software developers can do to get kids developing software. I mean, it's about time we had our say.

ICT500 - Finding & Nurturing The 0.1% Who Could Be Great Software Developers

For 2 decades, I've worked as a software developer, as well as training and coaching developers in the disciplines of software development.

Through my work and through conferences, workshops and other events, I come into contact with hundreds of software developers every year, and I get a good overview of what's going on in our profession.

And a "profession" it most certainly is. While computer programming is relatively easy to pick up, writing software commercially on any appreciable scale takes decades to master.

There is just so much to know, and so many practical skills that take thousands of hours of practice to really get to grips with. As a discipline, software development is every bit as complex and challenging as physics, medicine or teaching, and requires a lifelong commitment to learning, practicing and improving.

Beyond the obvious and many technical skills developers need to keep up to date with, software development also requires a high standard of general education in maths, written and verbal communication skills, and strong-enough general knowledge from which to quickly build a computable understanding of a wide range of problems that software is used to solve.

Year on year, I see the standard of software developers joining the profession slipping. They tend to be not quite so good with the technical skills, not quite so good at communicating, not quite so proficient with maths and logic, and not quite as well-informed and worldy-wise. Companies that hire young software developers are finding it harder and harder every year to recruit people of the right calibre. I work with some clients on recruitment, and I've been finding that - on average - perhaps 1 in 100 software developers that might apply for a job are really up to snuff. You have to kiss a lot of frogs to find your Prince Coding.

The impact of this on the businesses who rely on software developers, which, these days, is pretty much all of the FTSE500, can be profoundly damaging. While the media cries over the lack of good talent available to the computer games industry and "digital creative media", they completely overlook the fact that the bulk of our economy has been "digital" for decades - with computers playing an increasingly central role in the operations of all organisations that have to do things on a large scale.

If Acme Supermarket Plc has to wait to start a software project until they've found a team good enough to deliver, then Acme have to wait to roll out changes to the way their business operates. I've seen first-hand household British brands brought to their knees by delays in new IT. These days, your ability to create and adapt software systems is a very limiting factor on the ability of your business to adapt and stay competitive.

The importance of software development to the UK goes far beyond business, though. Our entire scientific and engineering base is very heavily reliant on software that has to be written specially. Increasingly, we find the lack of software development skills in science is holding UK R&D back. Our future prosperity and quality of life depends on our ability to discover and to innovate.

For all these reasons, I see programming in schools as an imperative. We desperately need new talent. Not just games companies like Eidos, or special effects companies like The Mill. We all need the decline in Britain's software development capability to be reversed.

My theory is this: hidden among the millions of children in school right now are maybe 1,000 who would make amazing software developers. And they, and we, may never know it. Not only could they be robbed of a potentially very rewarding and personally fulfilling career, but we would all be robbed of the fruits of their labour.

1,000 really great software developers entering the profession every year could, over a decade or so, tip the balance.

Firstly because the best software developers tend to be an order of magnitude more productive and creative than the average, so 1,000 great developers could achieve the same as 10,000 so-so developers.

Secondly because the more great developers come into the profession, making it easier for employers to find good people, the more the so-so developers will be forced to raise their game to stay in work.

We need to find out who these 1,000 kids are. We need to give as many children as possible the chance to find out for themselves whether programming is for them. For the vast majority, it won't be. But for that 0.1% who may find they really enjoy it, and have a real talent for it, we would be shooting our economy in the foot by not giving them the chance to discover programming - and ultimately software development - for themselves.

And once they've been identified, we must move mountains to fuel their passion and remove all obstacles to their development as, well, developers. This will require focused, targeted collaborations between practitioners like me, and the software development community of which I'm a part, as well as schools, teachers, parents, employers, the media, and everyone else who has a stake in the outcome.

I don't much care for kids being made to learn "computer science" (which is not the same thing, by the way). That sort of thing can come later, for those children who - fuelled by burning curiosity and desire to do more, go further and build better software - seek it out. It should be there waiting for them.

First, we need to give as many children as possible the chance to try programming. Not for its own sake, but because you can make cool stuff with code. Programming as "making cool stuff" should be the dominant model until that 0.1% reveal themselves, and then we can start talking theory and maths and discipline.

For my own part, I'll be focusing this year on getting kids coding, by finding support for teachers who want to learn more through a teacher-practitioner coaching programme, and also by devising "cool projects" for kids and for schools with themes ranging from pop music to alien hunting.

I'll also be working with the developer community to start fleshing out a framework for identifying and nurturing that little nugget of raw talent that may emerge.

Non-functional Test-Driven Development

It's the question that comes up everytime I introduce someone to Test-driven Development: "But what about performance?"

The thing about TDD is that adage "be careful what you wish for" applies. The solution we end up with is constrained by tests. There may be a million and one ways of achieving a goal, and some will perform better than others. The trick with TDD is to ask the right questions.

What I like about TDD - and similar precise approaches to defining requirements - is that it forces us to be explicit and unambiguous about what we want from our software.

So, my stock in trade reply to the question "But what about performance?" is "yes, what about performance?"

Software performance has different dimensions, and if it's important then we need to define exactly what performance we require in specific scenarios. A great way to do this is using non-functional tests.

There's the dimension of time, for example. How long should it take for the code to run?

Imagine a search algorithm that looks for a customer name in a sorted list. We could just loop through the list, and if there are only 1,000 customers and the occasional search, that might be fine. But if there are 10,000,000 customers and users are frequently searching, then a simple loop probably isn't going to cut the mustard.

We can constrain our search algorithm with a basic timing test, like the one below, that makes it explicit that our worst case search - the customer we're looking for isn't in the list - should take a maximum of 1 millisecond to complete.



Execution time is only one dimension, of course. What if we need to constrain the memory footprint of our code while it's running? In Java, we can use the JVM to get information about memory usage, and we can create a multithreaded test to monitor how much more memory is being eaten up as our code executes. Let's imagine we need to constrain the memory footprint when sorting our list of 10 million customers by name, forcing us to use an in-place sorting algorithm that uses up a maximum of another 10KB of memory:



And here, with massive caveats for my less-than-amazing knowledge of the Java Runtime (I make no warranties, the value of shares can go down as well as up, etc etc):



Leaving aside the fact that my brute-force method for calculating memory footprint is a bit hokey (and on running the tests several times, quite variable, it seems), the basic idea is hopefully useful. No doubt some fine fellow will point out a much better way.

You may be able to envisage now how we could use tests to explicitly constrain other non-functional runtime qualities of our code. But we can also often find ways to constrain code at design time, too.

We might have a requirement that our methods should be short and simple. Static code analysis tools like XDepend and Checkstyle can give us hooks into the structure of our code and enable us to create tests that, when code fails to live up to our quality standards, alert us to that fact early enough to do something meaningful about it.

Using executable tests, we can steer our software between acceptable limits of performance, scalability, portability, maintainability, and a whole heap of other -ilities we might care about.

But what about the more, how shall we put this, etheric -ilities, like usability, accessibility and so on? These things tend to be pretty ill-defined and qualitative. Can we make them explicit and testable, just like execution time or memory footprint?

I believe that we can, and to without reason, because I've done it and seen it done. We could, say, define a test that fails if a carefully selected group of target users (e.g., legal secretaries with more than 2 years Windows and web browsing experience), when presented with our application for the first time, fail to get their heads around it fast enough to complete certain tasks we set them within a specified time, without any help or documentation.

With a bit of imagination and lateral thinking, it's possible to meaningfully test many more software qualities than we usually do. And my experience of non-functional TDD is that we tend to get what we have tests for, and we tend not to get what don't have tests for. So agreeing executable non-functional tests tends to lead to better non-functional software quality, if it's done well.

As I warned before, though, be very careful what you wish for.

Jason's Handy Guide To Evaluating Software Packages

I get asked this question a lot, but it never occurred to me to write down my usual answer.

How do we evaulate shrink-wrapped software against our needs?

Well, that's easy. You still need to do the usual business requirements analysis. Identify who will be using this system, and what their goals will be for using it. In the good old days, we called these "Use Cases". Yep, even if you're buying and not building the software, you still need use cases.

The next step is to flesh out the design of your use cases, as we might normally do, by describing how the user interacts with the software to achieve their goal.

When we're describing software we haven't built yet, this is design. When we're describing how we'll use software that already exists, this is a process of validation. Can the user achieve their goal using the software we're evaluating?

Even with the most feature-rich packages, we tend to find we don't get an exact match. It's not always possible to achieve every user goal using the software. So as we validate the software against our use cases, we may identify gaps. There are almost always gaps.

The next question we need to answer is can we fill those gaps? Let's say we're evaluating Microsoft PowerPoint for our training business. It doesn't do everything we need out of the box. Let's pretend we have a use case where the trainer needs to populate a slide with an organisation chart showing the reporting structure of the group attending the course. She has a spreadsheet with those names listed in alphabetical order and with information about who reports to whom. using PowerPoint's built-in scripting language, Visual Basic for Applications (VBA), it is indeed possible to take that information and automatically generate an Org Chart.

So that gap could be plugged, with some work. Write a reminder about it down on a blank index card. This is now a potential "User Story" for some programming work that would need to be done if we went the PowerPoint route.

Of course, people identify gaps in software all the time, and it's possible that someone somewhere has already found a solution to plugging some of your gaps with handy tools and utilities. Google is your friend here: search for solutions before you think about reinventing the wheel. If you find one, and there's money involved, write down roughly how much on the index card.

Finally, don't forget the non-functional requirements. A package may offer the right features, but it may not be able to handle a high-enough volume of users, or it may not be secure enough for your purposes, or it may take a long time for users to learn. Evaluate thye software against these criteria, too. Be as explicit as you can. Handwavy requirements like "it must be scalable" aren't very helpful for validating software. What do you mean by "scalable" - a certain number of users at any one time, or a certain number of transactions per second, or the ability to run it on more servers?

All too often, businesses buy a solution and then validate that it does what they need - often by actually trying to roll it out. Whether buying or building, the key is to have clear, testable requirements and to validate the software against them. Don't be seduced by their sales patter and let them lead you like a donkey to the slaughter to their feature list. What their software does is far less important than what we can do with their software.

New Bletchley Park CEO, And A Tribute To Simon Greenish

It's been announced today that Bletchley Park has a new CEO, Iain Standen, a recently retired Colonel from the British army.

Before he disappears into his lovingly tended garden and a life of well-deserved leisure, I just wanted to pay tribute to Simon Greenish, who has been the key factor in the turnaround of Bletchley Park's fortunes since he took over in 2006.


Simon shows Colossus to kids' TV genius Johnny Ball, while the late Tony Sale does his famous velociraptor impression

Five years ago, Bletchley Park's finances were in bad shape. There was a very real danger of it folding due to lack of funds. I know first-hand how hard Simon has worked - harder than most people could even begin to imagine - and he's received very little recognition outside of the staff and volunteers there.


Two maths legends. And Alan Turing.

For every pound raised by enthusiastic supporters like myself, Simon and his amazing team have raised £50, working madly long days and suffering untold stresses and strains to secure funding from a range of public and private bodies. It's this work - often done in secret, and often not reported - that has earned Bletchley Park it's future, and for that I'm hugely grateful. I have no doubt that, were it not for Simon Greenish, Bletchley Park would now be in the hands of property developers.

As it is, Bletchley Park's future looks very positive indeed. Like the best boy scouts, Simon has left it in a much better condition than he found it, and - while there'll be many more challenges to come for Iain and the team - things are definitely on the up.

So, I raise a glass to you, Dr Simon Greenish, saviour of Bletchley Park. Top bloke, that Greenish chap!

What Can We Learn From Dream Theater's Drummer Auditions About Hiring Great Software Developers?

I'm sure I can't be the only software developer who thinks the way we hire for our teams is completely f**ked up in a lot of cases.

When I compare it to how, say, bands hire musicians it seems we care a heck of a lot less about who we work with. I know some of you will complain that hiring is often taken out of our hands, and therefore how can we to be to blame? Well, if we let such important decisions be taken out of the hands of the people who can tell the difference, then maybe it's our fault all the same.

One of the most technically demanding jobs in music is playing drums for prog rockers Dream Theater. After their previous drummer and co-founder, Mike Portnoy, quit the band, the search began to find a replacement. These were very big shoes to fill. There may only be a few dozen drummers in the world who play those parts well enough, and fewer still with the right temperament to fit in with a long-established band.

How does a band like Dream Theater go about finding that person? Do they pick up the phone and call Drum People Inc. and ask for a prog rock and metal drummer to start rehearsals on Monday? I can just see it now - the agent persuading traditional jazz drummers to add "prog", "metal" and "double bass" to their CVs.

And did Dream Theater leave the hiring process to their Director of Human Resources - a tone deaf, musically illiterate Justin Bieber fan who has not even the most basic appreciation of what it is Dream Theater actually do?

No, that would be silly.

How does a band like Dream Theater go about finding the right drummer, then? Well, it starts with drumming, oddly enough. That is, they listen to a lot of audition tapes sent in by drummers from all around the world. They must have received hundreds of them, so that's a few days of dedicated listening. Luckily, you only have to hear a drummer for a few minutes to know whether they'd be up to the standard of someone like Mike Portnoy. But it's still a lot of listening.

But this is a critically important decision for the band. In a 20+ year career - mostly with the same brilliant drummer - what's a few days listening to audition tapes?

The ones who stand out go into the "maybe" pile. And, of course, most drummers of any note have things like web sites and blogs and Twitter accounts and YouTube channels these days, and they and their bands are discussed in various message boards and on other kinds of social media. That is to say, they have a searchable public profile, and catalogue of music we can check out, and a reputation.

They can even watch drummers rehearsing for their audition.



By checking out the portfolio of work the drummer's done and researching them online, Dream Theater were able to whittle down hundreds of auditionees down to the three drummers they thought would be worth auditioning in person.



You can tell a lot about someone from their work, but what you can't tell is how they'd work with you. That's really what an audition should be about - not "can you play this tune?" but "what does it sound like when you play this tune with us?" and "can we write a new tune together?"

Auditions aren't about technical ability - that should already have been established before you even think of inviting that person in and taking up everyone's valuable time. Auditions are about the dynamic: how does this band perform with this person integrated into it? What would Dream Theater with Peter Wildoer, celebrated Swedish death metaller and jazz fusioner, sound like? Possibly a bit heavier - a bit more technical death metal and a little less Asia or Yes? That might be why he didn't get the gig. There's no doubt he's up to the standard of Portnoy. Wildoer's one of the best. He just wasn't the fit they were looking for.

Most notably, when I watched the band's video of the auditions, Wildoer was not spitting feathers because they'd rejected him. To him, being one of one only three drummers in the world even invited to audition was a great experience. God knows I'd be flattered if they even listened to my audution tape for more than 30 seconds!

He was at their level musically and technically, and they weren't wasting his time. The audition video's been viewed 1,000,000 times. That's good exposure and good experience. To get to play and jam with musicians of that calibre, and with that level of passion and professionalism, and have a good time to boot, is not really a waste of anyone's time.

If a great team flattered me by inviting me to "audition" for them, and my technical ability was not under question because we'd already established I was technically good enough to work with them based on my portfolio and reputation, and they said "Hey, Jason, come and spend a day with us and let's play some of the old tunes together and maybe jam a few new ones", and they paid my expenses and put me at my ease - then if I didn't get the job, I'd still feel like I got something out of it.