Quality Assurance: it doesn't assure quality

Software testing is an honorable profession; if you want to produce a quality product then you need to have smart and committed software testers in your team. But somewhere in the evolution of our industry, this profession morphed in to a process and organization called QA, or sometimes Integration and Verification. For the most part, this has not been a good thing.

In many companies, quality is added to a product after it has been coded. The developers write code, then sling it into QA where it is beaten into shape. The results can be quite good: I've seen some reasonable products emerge from this strange process including, alarmingly, quite a bit of the software in aircraft systems. But it's a horribly inefficient way to work: it's impossible to predict how long the software beatings will last, it does nothing for teamwork, and it usually results in mediocre product. Where's the professional pride of the coders and testers in such a process?

The way to go is to to have most testing done as the software is being written and the best way to accomplish that is in self-organizing teams of coders,  testers and analysts. There is still usually a need for a final test before release, as part of a rapid "end-game", but that's a time for polishing, not beating.

The net result is faster progress in development and better products. In my experience I've found that testers have valuable insights that can make products simpler and more useable - a dialogue with the coder and analyst at the time the software is being written draws this out. And removing the perception of a QA "safety-net" is good for coders and analysts - it keeps them concentrated on the product goal.

Careful now!

One of the indicators that all is well with this set-up is when team-members prioritise the product goal over their own tasks. So if testing is falling behind, the developers and analysts roll up their sleeves and help out with the testing effort.  At the end of each sprint, it’s better to produce something that works than to have untested code or unused analysis, neither of which are of much value.

QA is about cleaning up a mess that shouldn't have been made in the first place. But when properly deployed, software testing professionals add real value to products.

Size doesn't matter if you're agile...

I'm a published software guru you know. The January 2007 issue of IEEE Software contained an article written by yours truly, thanks to an invitation from guest editor Dr. Ita Richardson. She saw me present at agile conferences in Ireland and thought I was absolutely the right guy to start a fight. So that's what the article is: a point - counterpoint argument, where I taunt a genuine software guru Wolfgang Strigel and he knocks my lights out. Thankfully it's behind the IEEE Software paywall so my ignominious defeat is well hidden.

The point I was arguing - and I'm still right, Wolfgang - was that agile methodologies can be used on both big and small projects, in small companies and in large enterprises. Size should not be a factor in accepting or rejecting agile methods because even the biggest software development project must be broken down into smaller coherent chunks, and developing these in self-organizing teams is almost always the way to go.

But that's too polite. I'm going to remove my gloves and take a bigger swing, like I should have done in that article.

Most very large software projects are disastrous failures: they cost too much, they're late and they don't achieve their goals. This presentation by Roger Sessions explores the relationship between project size and project failure - this slide is an extract.

(Yes, I know the Standish metrics are contentious these days - but even if they're only half-right the conclusion would be the same.)

The traditional methods of managing large projects, all those document-centric processes, don't work very well. My advice: if you're unable to break down a project into agile-sized chunks then either do the company a favour and call a halt to it or run very far away as fast as your skinny little software developer legs can carry you.

Sometimes, Agile is a bad fit for a project but it's not because of size. I can think of two characteristics that would mitigate against agile:

• A very detailed specification has been contracted with the customer. I don't just mean a description of the features, but a specification that leaves no room to manoeuvre, which is fairly rare, thankfully. (Though see below!)
  
• The development organization is completely distributed with few co-located developers. How to  manage virtual self-organizing teams is a problem I haven't been able to crack yet, but the co-ordination mechanisms in agile such as stand-up meets, story-boards and solution white-boarding aren't adapted for this

I encountered the first characteristic in Cúram software when we were contracted to deliver a system to the UK Department of Work and Pensions. A large part of the requirements were described in laws passed by parliament. The only way to understand these requirements was for some unfortunate analyst to swallow all the legislation and regurgitate it for developers to eat (sorry), highlighting overlaps, commonality and inconsistencies. I couldn't find a way of doing this in an agile fashion; we ended up with a project structure that was quite waterfall where a team of analysts provided the requirements to rules developers, and test scenarios to testers. We delivered the system according to the contract and were well paid - but it has never been put in to service. Coincidence?

(So, you may ask, why didn't I run away from this project like I recommend above? Well dear reader, if I knew then what I know now I would have done things differently. Hence this blog.)

When I'm starting a new project I'll always try to apply an agile development approach, breaking it down into smaller chunks. I avoid projects with either of those nasty characteristics.

Code Wins!

Back in the early 2000's, the app server team at IONA had a mantra: "Code Wins!". It was a way of resolving arguments about the merits of a design. Instead of debating via emails or PowerPoints, go and prove your point in code with a prototype.

That team was one of the most creative I ever worked with, and the iPortal Application Server evolved at a tremendous pace.

I notice that almost the same phrase "Code Wins Arguments" shows up in the Facebook registration statement, filed yesterday with the Security and Exchange Commission. It's a powerful idea. Here's the extract from the statement:

The Hacker Way

As part of building a strong company, we work hard at making Facebook the best place for great people to have a big impact on the world and learn from other great people. We have cultivated a unique culture and management approach that we call the Hacker Way.

The word “hacker” has an unfairly negative connotation from being portrayed in the media as people who break into computers. In reality, hacking just means building something quickly or testing the boundaries of what can be done. Like most things, it can be used for good or bad, but the vast majority of hackers I’ve met tend to be idealistic people who want to have a positive impact on the world.

The Hacker Way is an approach to building that involves continuous improvement and iteration. Hackers believe that something can always be better, and that nothing is ever complete. They just have to go fix it — often in the face of people who say it’s impossible or are content with the status quo.

Hackers try to build the best services over the long term by quickly releasing and learning from smaller iterations rather than trying to get everything right all at once. To support this, we have built a testing framework that at any given time can try out thousands of versions of Facebook. We have the words “Done is better than perfect” painted on our walls to remind ourselves to always keep shipping.

Hacking is also an inherently hands-on and active discipline. Instead of debating for days whether a new idea is possible or what the best way to build something is, hackers would rather just prototype something and see what works. There’s a hacker mantra that you’ll hear a lot around Facebook offices: “Code wins arguments.”

Hacker culture is also extremely open and meritocratic. Hackers believe that the best idea and implementation should always win — not the person who is best at lobbying for an idea or the person who manages the most people.

To encourage this approach, every few months we have a hackathon, where everyone builds prototypes for new ideas they have. At the end, the whole team gets together and looks at everything that has been built. Many of our most successful products came out of hackathons, including Timeline, chat, video, our mobile development framework and some of our most important infrastructure like the HipHop compiler.

To make sure all our engineers share this approach, we require all new engineers — even managers whose primary job will not be to write code — to go through a program called Bootcamp where they learn our codebase, our tools and our approach. There are a lot of folks in the industry who manage engineers and don’t want to code themselves, but the type of hands-on people we’re looking for are willing and able to go through Bootcamp.