AmbySoft.com

Ambysoft Logo
IT Project Success Rates: 2011 Open Research

 This open research into IT project success rates was performed during the last two weeks of October and there was 178 respondents. The survey was announced in my October 2011 DDJ column, my DDJ blog, on the Ambysoft announcements list, my Twitter feed, and several LinkedIn discussion forums (IASA, TDWI, Enterprise Architecture Network, Greater IBM connection, and  Considerate Enterprise Architecture Group).

The Survey Results

The survey results are summarized in my November 2011 Dr. Dobb’s Journal column How Successful are IT Projects, Really?.

Some findings include:

  • As you can see in Figure 1, Agile and Iterative project teams have statistically identical success rates
  • As you can see in Figure 1, ad-hoc project teams (no defined process) and traditional project teams have lower success rates that agile/iterative project teams
  • Figure 2 compares the effectiveness of the five paradigms for delivering in a timely manner, for providing good ROI, for delivering value to the stakeholders, and for producing a quality product.
  • When it comes to time/schedule, 20% prefer to deliver on time according to the schedule, 26% prefer to deliver when the system is ready to be shipped, and 51% say both are equally important
  • When it comes to ROI, 15% prefer to deliver within budget, 60% prefer to provide good return on investment (ROI), and 25% say both are equally important
  • When it comes to stakeholder value, 4% prefer to build the system to specification and 80% prefer to meet the actual needs of stakeholders, and 16% say both are equally important
  • When it comes to quality, 4% prefer to deliver on time and on budget and 57% prefer to deliver high-quality, easy-to-maintain systems, and 40% say both are equally important
  • Only 12% of respondents indicated that their definition of success on their most recent project included all three of delivering according to schedule, within budget, and to the specification (answers where both was indicated were included in this calculation).

 

Figure 1. Perceived IT project success rates by paradigm.

 

 

Figure 2. Success factors by paradigm (Scale is from -10 to +10).

 

 

Downloads

The Survey Questions

Raw Data

Summary Presentation

 

What You May Do With This Information

You may use this data as you see fit, but may not sell it in whole or in part. You may publish summaries of the findings, but if you do so you must  reference the survey accordingly (include the name and the URL to this page). Feel free to contact me with questions. Better yet, if you publish, please let me know so I can link to your work.

 

Discussion of the Results

  1. It’s difficult to get a good estimate of project success rates because there isn’t a standard definition of success (nor will there ever be). So, if I define success specifically, for example as “reasonably on time, on budget, to specification” that definition will be applicable for some projects but not others. So, I could presumably get an accurate estimate of how well we’re doing against that criteria but it wouldn’t be the actual industry success rate. If, however, I define allow people to define success in terms of how it was defined for the actual projects then I’ll get a much more accurate estimate of project success rates but I won’t know exactly
  2. This survey was huge, up to 50 questions, so the response rate was a bit lower than usual.
  3. This was the first year we asked about lean, and we only had 40 respondents working in organizations applying lean techniques. So, the lean  results aren’t statistically significant (although to be fair there’s been many academic papers published based on much smaller result sets).
  4. This survey suffers from the fundamental challenges faced by all surveys.

 

Links to Other Articles/Surveys

  1. My other surveys

 

Why Share This Much Information?

I’m sharing the results, and in particular the source data, of my surveys for several reasons:

  1. Other people can do a much better job of analysis than I can. If they publish online, I am more than happy to include links to their articles/papers.
  2. Once I’ve published my column summarizing the data in DDJ, I really don’t have any reason not to share the information.
  3. Too many traditionalists out there like to use the “where’s the proof” question as an excuse not to adopt agile techniques. By providing some  evidence that a wide range of organizations seem to be adopting these techniques maybe we can get them to rethink things a bit.
  4. I think that it’s a good thing to do and I invite others to do the same.