The Current State of Data Management Survey Results: July 2006

Scott W. Ambler + Associates
Home | Articles | Books | IT Surveys | Podcasts | Contact Us | Announcements | Site Map
How to Measure Anything 2nd Edition This survey was performed during the first three weeks of July 2006 and there was 1,176 respondents. The survey was sent out to the Dr. Dobb's Journal mailing list.

The Survey Results

The results of this survey are summarized in the article Whence Data Management? in the November 2006 of Dr. Dobb's Journal. The article also proposes a new vision for data management which you may find interesting.

Here's a few interesting results:

Figure 1. Reasons why development teams go around data groups.


Downloads

Survey

The Survey (162K)

Survey Data File

Raw Data (162K)

Survey Presentation

Summary Presentation (155K)


What You May Do With This Information

You may use this data as you see fit, but may not sell it in whole or in part. You may publish summaries of the findings, but if you do so you must reference the survey accordingly (include the name and the URL to this page). Feel free to contact me with questions. Better yet, if you publish, please let me know so I can link to your work.


Discussion of the Results

  1. I'm not convinced that respondents understood my questions about database testing. The numbers were a lot higher than I suspected, a little over 40% indicated that they're doing some form of database testing, but in practice my experience is that data testing is very spotty. The Data Quality Techniques survey explores this issue and I will report the results in early January.
  2. I'm not convinced that respondents understood what I meant about fixing existing legacy problems via database refactoring. I'd love to think that the technique has been adopted to the extent that the survey indicates, but I'm just not seeing that in practice. The Data Quality Techniques survey also explores this issue.
  3. There was a fair representation of organization sizes. The results didn't appear to vary substantially based on organization size, with the exception that larger organizations were more likely to have data groups than smaller organizations.
  4. We got a fair representation of respondents. Roughly half (618) were developers, but 188 identified themselves as IT management, 133 as project managers, and 98 as data professionals.
  5. At the beginning of the survey the respondents thought they were getting adequate service from their data groups (avg rating of 3.6 out of 5, see Figure 2) but after thinking about the quality issues explored during the survey rated their data groups lower (avg rating of 3.31 out of 5, see Figure 3). It leads me to believe that data groups in many organizations may currently benefit from people having low expectations about what a data group can actually accomplish. As more and more organizations adopt agile techniques, and thereby start adopting quality-oriented techniques which provide concrete results, data groups may find that the bar is slowly being raised on them. They can and should start adopting agile data techniques.
  6. The data professionals rated themselves slightly higher on average than any other group, but it wasn't a big difference. The trend was still the same, even data professionals felt that their data groups weren't doing as good of a job by the end of the survey.
  7. This survey suffers from the fundamental challenges faced by all surveys.

Figure 2. What people initially thought of their data groups.

Figure 3. What people thought of their data groups at the end of the survey.



Links to Other Articles/Surveys

From what I can tell, nobody has ever done a similar survey to this one. However, there are some interesting resources available:


Why Share This Much Information?

I'm sharing the results, and in particular the source data, of my surveys for several reasons:

  1. Other people can do a much better job of analysis than I can. If they publish online, I am more than happy to include links to their articles/papers.
  2. Once I've published my column summarizing the data in DDJ, I really don't have any reason not to share the information.
  3. I think that it's a good thing to do and I invite others to do the same.


Disciplined Agile Delivery: The Foundation for Scaling Agile Agile Modeling: Practices for Scaling Agile Agile Data: Practices for Scaling Agile EnterpriseUP: Agility at Scale Software Development Practices Advisor Scott Ambler + Associates Follow @scottwambler on Twitter!


Copyright © 2006-2014 Scott W. Ambler

This site owned by Ambysoft Inc.