Improvement (SPI) Best Practices
- Be aware of your organization’s current culture. One of the
significant forces that affect the success of your process improvement
efforts is the culture of your organization. Organizations with cultures
that are positive toward process improvement are likely to want to supply a
quality product with reasonable business returns, have middle managers that
are willing to set and work toward targets of meeting your organization’s
needs and business goals, and have senior management leadership that is
willing to launch and sustain a long-term change effort.
- Expect to change your organization’s structure and culture.
Process, organizational structure, and corporate culture all go hand-in-hand
– change one and you will affect the others.
- Keep it simple. A common mistake that organizations make is to
over specify the processes that they intend to follow. Never forget that
your goal is to produce working software that meets the needs of your user
community and that your staff likely has a pretty good idea of how to do
this although they could help with a little guidance from time to time.
Give them just enough guidance for their needs. An example of a well
defined, yet simply defined, process is the
- Align your software process with business goals and objectives.
You could have the best process in the world, but if it doesn’t meet your
organization’s goals then it doesn’t matter. Do you intend to build a
portfolio of applications that integrate with, and build upon, one another?
If so then portfolio management is important. Do you intend to sell
shrink-wrapped software to be used by millions of users? If so then
architecture may be less important to you in favor of getting a product to
- Regularly hold retrospectives. A
retrospective is a process
improvement meeting where you ask four fundamental questions: What did we do
well that if we don't discuss we may forget? What did we learn?
What should we do differently next time? What still puzzles us?
Retrospectives can be simple 15 minute discussions or formal meetings over a
day or two. The goal is to learn from your experiences.
- There's a serious difference between "lessons learned" and "lessons
indicated". The result of a retrospective is a collection of
lessons indicated, they don't become "learned" until you actually improve
your process. I've been in organizations where we've held a
retrospective, sometimes called a post mortem or "lessons learned" session,
which resulted in a list of really good process improvement suggestions.
Always one to cause trouble, I then asked if the client had any similar
documents from four or five years ago. Comparing the documents, we've
often discovered significant overlap between the lists.
- Keep the real goal in mind. My experience has been that software
processes, when applied intelligently, increase the productivity of
developers. My experience has also been that when processes are applied less
than intelligently, or when the paper pushers have too much influence within
an organization, processes can also decrease your productivity.
Organizations that keep the end goal in mind – that of developing,
maintaining, and supporting software that fulfills the needs of their user
community – will be successful with implementing software processes. Those
that follow processes simply for the sake of doing so are likely to fail.
- Recognize that the fundamentals remain the same, the details vary.
Contrary to popular belief, the fundamentals of software development have
been known for many years. You need to perform requirements engineering.
You need to model. You need to write code. You need to test. You need to
perform change control. You get the picture. Every successful software
organization will have a similar set of processes but the way that your
organization brings them in and how they implement them will differ. Your
requirements process may be slightly different than your competitors, but
you will both have one that will generally do the same sort of thing.
- You need more than one process. You wouldn't run a project
team of thirty people the same way that you would a team of three people.
Nor would you run an outsourcing project the same way that you're run a
project developed by a co-located team. Nor would you run a data
warehouse project the same way that you'd run a .NET application. You
need different processes, or at least different flavors of your process, for
different situations. Use the
process for the job.
- Run a trailblazer project to validate your new processes.
Regardless of how well you define a process, no process is perfect. Test
your new software process using a trailblazer/pilit project, one that is
given the extra resources required to try new techniques and to update them
- Treat process improvement like a project. Have an experienced
project manager, ideally someone with experience in both process-oriented
and object-oriented development. Define the requirements for your processes,
model them, implement them, test them with a trailblazer project, and then
improve the processes.
- Improve your processes in priority order. The reality of process
improvement is that you cannot make all of the changes that you want to
immediately; it is simply too great a change for your organization to absorb
at once. This is why we have efforts such as the Software Engineering
Institute’s (SEI’s) Capability Maturity Model Integrated (CMMI) efforts and
the Software Process Improvement Capability Determination (SPICE) efforts of
the International Standards Organization (ISO). Both of these organizations
suggest that you prioritize the process improvements that your organization
needs to make, expect that it will take several years to make the needed
changes, and expect that you will experience difficulties while doing so.
There are five maturity levels in the CMMI for a reason: you need to
progress in order from level one to level two to level three and so on. The
implication is that by knowing which aspects of a software process map to
which CMM maturity levels you have a rough idea of the order in which you
should introduce those processes to your staff. Experience shows that
organizations that try to make immediate, large-scale process changes are
likely to fail doing so. The reality is that it takes time, often several
years, to permanently improve the productivity of your software development
efforts. There is not a simple, quick fix to your problems.
- Communicate your plan. It is essential that you let others
know what are you doing, why are you doing it, the business case for it,
success stories, etc. Keeping them informed on how things are going, even if
you are encountering difficulties, will help get them (and keep them) on
board. There are various options for this: a newsletter, a web site,
periodic emails to key personnel, etc. Trumpet your successes and share your
lessons learned with the appropriate people.
- Accept that the big picture is overwhelming. Because of the
complex nature of software development most developers specialize in one
aspect of it and focus solely on that. This becomes a problem for
organizations that wish to tailor a software process for their exact needs
because when they put the individual pieces together the overall process
becomes very large. For example, the
Rational Unified Process (RUP) is over
3,000 HTML pages in size, and it is only a development process. Once
developers see how large your organization’s software process is they often
go into denial, claiming that you can not possibly achieve your goals. Yet
when you ask an individual which part of their process they can simplify
they’ll often balk at the idea. An alternative approach might be
something such as the
Process (AUP), which is a bit smaller (it's around 30 HTML pages).
- Democracies do not always work, nor do dictatorships.
Organizations that wish to reach consensus regarding their software process
tend to flounder. You’ll never achieve complete agreement on how things
should be done, although organizations that dictate processes from above
tend to fail as well. Effective process improvement efforts seek consensus
at some points and dictate things at other points.
- Identify the consumers and suppliers for each process. Every
process has inputs and outputs, and you need to ensure that there is a
supplier for each input and a consumer for each output. Fundamentally, if
nobody is going to use an artifact that is produced by a given process then
why bother producing it? You also need to look at collections of processes
to see if the artifacts that they produce add value in combination.
- Defining a process is the easy part. Many organizations are very
successful at defining a software process, often producing binders (or web
pages) of documentation. However, when I've revisited these
organizations a few months after introducing a new process it will be all
but forgotten. Getting people to accept your new process, and making the
changes that go along with it, will take significant time and effort to
accomplish. Writing a process is the easy part, following it is the hard
- Introduce a Software Engineering Process Group (SEPG) to your
organization. The sole responsibility of your SEPG is to support the
definition and improvement of your organization’s software process. The SEPG
should be kept small – as a rule of thumb, we suggest one SEPG member for
every one hundred developers in your organization. SEPG efforts are a
component of the EUP's
Software Process Improvement (SPI) discipline.
- Staff your SEPG with actual practitioners. The people who
best understand how to develop software are the people who are very good at
developing software. Yes, that sounds blindingly obvious, but what
isn't obvious is that they are usually the best ones suited to define your
software process. Try to staff your SEPG mostly with current
practitioners, people who are now managers but had did some COBOL
programming 20 years ago are often poor candidates, although include some
people with a process background to ensure that they're not reinventing the
- Reuse existing process materials. There is a
wealth of process materials out there, you very likely don't need to
write your own.
- Avoid “fire hazard processes". A common mistake is to produce
volumes of documentation describing your processes. Your goal is simply to
describe your process materials to such a level that they can be given to a
professional skilled in the techniques of that process so they can work the
- Adopt processes because they make sense. If a process makes
sense to you, and you believe it will add value to your effort, then adopt
it. Otherwise, do not.
- Hold everyone responsible for process improvement. Senior
management must be willing to actively support and sustain process
improvement, project managers must be held responsible for ensuring that
their teams follow the defined processes, and developers must be held
responsible for learning and then following the processes. This is often a
difficult task because senior management often demands immediate results,
whereas process improvement often takes years. Project managers resent
diverting scarce resources from their projects, and developers often resent
being told how to do their jobs.
- Bring in an expert to advise you. Process improvement is a
complex and difficult endeavor, one for which you are likely to need help to
accomplish. You can increase your chance of success by bringing in a
consultant who has both a process background and an OO development
background – someone who has been actively involved in a process improvement
program and who has worked on large-scale, mission-critical software
development projects using OO technology.
- Do not think that everyone is on board. There is likely to be a
small core of people within your organization who do not want to use object
technology for large, mission-critical projects, and these people will
actively undermine your efforts. You need to identify these dissenters and
work together with them to help them see the advantages of working with
object technology and of following a set of defined process patterns to help
in the development OO software.
- A fool with a process is still a fool. For your organization to
be successful with a software process your software professionals will need
to understand the processes, the concepts, the techniques, and the problem
domain. Implementing a new process in your organization involves more than
going out and purchasing a couple of new books and development tools.
- Develop a user guide for your process. You can make it easy for
your staff to learn your chosen processes by providing a well-written
overview of your process as it is to be implemented in your organization.
In fact, this may be all the process material that you need.
- Have patience. Progress will be slow at first, slower than you
hoped or expected. Introducing a software process into an organization takes
time – the required culture shift often takes years to complete.
- Don’t flounder in bureaucratic requirements. Too many process
efforts run aground because of preconceptions forced on them by senior
management, an overly burdensome documentation and review process, or
unrealistic requirements to achieve consensus.
- Define your process early. The longer you leave process
definition the bigger the mess you will have to clean up. Without
direction, developers will typically go and do what they think is right, the
only problem being that each person has their own idea of what “right” is.
This book, Disciplined Agile Delivery: A Practitioner's Guide to Agile Software Delivery in the Enterprise describes the Disciplined Agile Delivery (DAD) process decision framework.
The DAD framework is a people-first, learning-oriented hybrid agile approach to IT solution delivery. It has a risk-value delivery lifecycle, is goal-driven, is enterprise aware, and provides the foundation for scaling agile.
This book is particularly important for anyone who wants to understand how agile works from end-to-end within an enterprise setting.
Data professionals will find it interesting because it shows how agile modeling and agile database techniques fit into the overall solution delivery process.
Enterprise professionals will find it interesting beause it explicitly promotes the idea that disciplined agile teams should be enterprise aware and therefore work closely with enterprise teams.
Existing agile developers will find it interesting because it shows how to extend Scrum-based and Kanban-based strategies to provide a coherent, end-to-end streamlined delivery process.
Enterprise Unified Process: Extending the Rational Unified Process
Scott W. Ambler,
Vizdos. Whereas the
RUP defines a software development lifecycle, the EUP
extends it to cover the entire information technology (IT) lifecycle.
The extensions include two new phases,
Retirement, and several new disciplines:
Operations and Support and the seven enterprise disciplines (Enterprise Business Modeling,
Software Process Improvement).
We actively work with clients around the world to
improve their information technology (IT) practices,
typically in the role of mentor/coach, team lead, or trainer. A full
description of what we do, and how to contact us, can be
found at Scott W.
Ambler + Associates.