You know what’s the best way to start the year? Going over success rates of projects!

The Standish Group has been publishing the CHAOS report for more than twenty years now. I wasn’t surprised to find that they still use the same criteria for success and failure of projects. Here’s a summary of the CHAOS definition:

Resolution Type 1, or project success: The project is completed on-time and on-budget, with all features and functions as initially specified.

Resolution Type 2, or project challenged: The project is completed and operational but over-budget, over the time estimate, and offers fewer features and functions than originally specified.

Resolution Type 3, or project failed: The project is cancelled at some point during the development cycle or delivered but was not used.

As we can see, the iron triangle still rules, agile or not. We’re still measuring success on how well we’ve estimated the initial cost and duration, based on the initial scope. As half of the projects in the study fall into the challenged resolution, and about 20 percent more failed, our industry is not doing that much better than before (although the success numbers are rising slightly and the number of failed projects are decreasing slightly since 2004).

How is that possible when everyone’s going agile? Wasn’t agile supposed to bring the productivity and effectiveness everyone was looking for? Wasn’t agile supposed to save all the doomed projects?

Well, first of all, no. That wasn’t, and still isn’t agile’s promise. Agility means the ability to react effectively to market needs.

What is success anyway?

If that’s how we describe agility, shouldn’t we consider project’s success based on the same market needs?

A project success means that customers use the product, and eventually and sustainably money gets into the business. Given this definitton, an agile project (whatever that is) changes in content, duration and costs to fit the market needs in order to be successful. If we dropped the excess features, and release the product with what the customer actually needs, does that mean we’re not successful?

Or maybe it took a lot more time to until there was an actual product/market fit. We’ve run over budget and over the time estimate. But eventually our start-up was bought by MegaCorp with a lot of fanfare and bucks? Is this a challenged project?

Last but not least, we’re still setting the success goals in the beginning of the project. When we knew the least about it, we didn’t identify all the known unknowns yet (not to mention the unknown unknowns). We put a pin on an empty map, and we get high marks for reaching it for moderate economic success, while the big economic success could be miles away.

We almost always start on the wrong foot, and we’re motivated to do so.

Which leads to me to estimates (and #NoEstimates), because that’s what we do when we don’t have all that information. We estimate time and cost, and then we make decisions. A go/no go for the project. Delay that feature because it will take too much time to develop (or will it? It’s still a guess).

At Agile Practitioners 2015, I’m going to give the talk “To Estimate Or #NoEstimates”. Are estimates the only way to drive decisions? Are there other alternatives? And are we destined to continue being graded on the quality of our guesses?

You don’t want to miss that one.