Are you ready to ship?

Everyone knows you need to test software. No question. A good thing. But it often doesn’t happen. Or teams start with the best of intentions, which fall by the wayside in the face of pressure to deliver above all else. Because software is a fairly intangible thing, it’s hard to spot the cracks starting to appear. So let’s consider the importance of software testing from an Agile/Lean perspective. Maybe something here will resonate with you, your developers, your stakeholders.

Good software testing is a vital part of an Agile/Lean product development process. Without it, it becomes harder and harder to adhere to some of the core principles of these methodologies, causing the development process to become less efficient and, perhaps more importantly, less innovative over time.

Let’s begin with the Lean principle that work in progress (WIP) should be minimised. Rooted in manufacturing, the principle is equally applicable to software development, but less well understood. In manufacturing, WIP isĀ  visible. It’s a pile of components at one station in the manufacturing process that are awaiting processing by that station. Big piles = Lots of WIP = Loss of efficiency in the system as a whole. Inventory is lying dormant, and that’s not a good thing in manufacturing. Inventory lying around isn’t making money. In fact it’s costing money.

But what is WIP in software development? It’s the features and bug fixes that a development team are delivering for a software product. The key word here is delivering. In other words, they’ve been implemented but have not yet been delivered to the end users. It has cost money to implement the features and fix the bugs, but they are not delivering a return to the business until they are in the hands of end users. Not a very tangible thing, even to the developers delivering those features and writing those lines of code each day. Agile attempts to make WIP more visible.

The sprint backlog makes WIP visible by giving the team a visual representation of the work they have commited to complete in the next sprint (normally a couple of a weeks). At the end of a sprint, there should be a potentially shippable product. These two elements together are key to minimising WIP. Teams generally manage to adopt the sprint backlog. It’s the potentially shippable product element where things often fall apart.

What does having a potentially shippable product at the end of a sprint mean in relation to the underlying Lean principle of reducing WIP? Let’s take a common example. A development team has committed to delivering a number of new features in their software product AmazingApp. They implement the features over a couple of weeks and deliver them into quality assurance (QA) and user acceptance testing (UAT) environments.

Does the team have a potentially shippable product at the end of the two weeks? Unlikely. Let’s consider why, by asking a series of questions.

  • Have QA testers and/or users confirmed that the new features meet their acceptance criteria?
  • Does the system still meet all existing non-functional requirements (generally performance and capacity related)?
  • Do all of the existing features still work the way they used to?

Unless the answer to all of these questions is a confident “Yes!”, the team does not have a potentially shippable product. There might still be WIP:

  • one of the new feature doesn’t meet the acceptance criteria
  • the performance of the product is no longer acceptable
  • a couple of existing features no longer meet their acceptance criteria

So what does good software testing look like? What processes are required to deliver a potentially shippable product at the end of each sprint. I’ve summarised what I think are the key requirements below:

  1. Automate Deployment. Reduce the friction of promoting a change into production. Perhaps you have one or more non-production environments, and a production environment. The faster you can promote a change, the sooner it can be made available to QA and UA testers.
  2. Automate Regression Testing (Functional and Non-Functional). Promoting a change quickly to QA and UA testers is not enough. They (and the development team) need to be confident they’re testing a product that hasn’t been broken in existing areas, whether they be functional or non-functional.
  3. Create an Agile QA/UAT Process. QA and UA testing should happen iteratively in parallel with the development process, and in close collaboration with the development team. Implementing new features during a sprint and then handing them over to testers at the end means there is no potentially shippable product.

Lack of automated regression testing reduces efficiency and innovation. The effort required by QA and UA testers to test new features is fairly constant through the life of a software product. But over time, the number of features and complexity of a product grows. If regression testing is a manual process, the effort required to ensure a growing list of features still work grows with every increment of the product.

Developers become fearful of making a change because there is no quick feedback to indicate whether their change broke something. Innovation suffers as a result. Developers focus more on not breaking the product than on delivering ground-breaking and value-adding new features. And the features that do get implemented take longer and longer to deliver into production as they have to go through an ever increasing regression test cycle.

Too much invisible work in progress is killing efficiency and innovation in software development, and one of the main causes is bad software testing.

Image source: By Jef Poskanzer (originally posted to Flickr as smash) [CC BY 2.0 (], via Wikimedia Commons


And so it begins

Welcome. Well done for finding me, and thanks for visiting. As this is my first post its fair to say there isn’t a huge amount of material that would have inspired you to visit. Your pioneering spirit is alive and well.

So, content. What is it that I hope will keep you coming back, and inspire others to visit in the future? They say write about what you know, and for me that’s IT. I’ve been in the industry for a long time, delivering software products for clients. I started as a developer and evolved into a project and programme manager, a fairly typical career path. More recently I’ve worked as part of CTO groups looking at wider IT strategies.

I’ve always worked in big companies (and I mean over 10,000 employees big). I’ve never created or worked for a technology start-up or small software house. But what interests me isn’t unique to a particular size of company. It started with a desire to understand more about how people, process and culture impact the ability to deliver software products. How do you influence those aspects for the better? What motivates people and encourages them to “do the right thing”? And do it together collectively? What product development processes work best and help foster a productive and innovative culture? What causes things to go wrong?

In “The Lean Startup”, Eric Ries makes an important point that when you talk about people, process and culture, they don’t just apply to a particular type of company. The same concepts can be applied as much within large, established companies and government programmes as shiny new technology start-ups. They don’t even just apply to technology companies. That’s what really interests me. How do you apply what are starting to be recognised as good product development practices in these different environments, and help ensure your company thrives? Changing large companies is hard, akin to turning a supertanker. There’s a lot of existing momentum to fight against. But new start-ups can head down the wrong path quickly too, and the results tend to be rather more immediate and catastrophic for those involved.

So, if like me, you think Agile and Lean have a place in technology organisations. That they might just be able to foster innovation and creativity, and allow companies to thrive and survive. Then remember this blog. Bookmark it. Come back and visit. And join in the debate.