Defeating the Forces of Nature -
Two Workshops on Spiral Development

Dr. Wilfred J. Hansen
Software Engineering Institute
Carnegie Mellon University

Two workshops in 2000 explored the meaning of Spiral Development (SD) and Evolutionary Acquisition (EA) for Department of Defense acquisition. This note summarizes the reports from those workshops. It expands on the theme to explore how EA and SD can help defeat the "forces of nature" that deter projects from being on-time, within budget, and high quality.

Evolutionary Acquisition (EA) and Spiral Development (SD) are key elements of the current DoD effort--the 5000 series [USD 01]--to define an acquisition process that consistently attains that uncomfortably small apex on the pyramid striving toward low cost, rapid delivery, and high quality.  Guarding the apex are arrayed the forces of nature, as shown in Figure 1.


Figure 1: The forces of nature guard the apex of process achievement

In anticipation of the DoD 5000 series, the CMU Software Engineering Institute (SEI) and the USC Center for Software Engineering (CSE) held workshops in February and September, 2000, with the objective of identifying a set of critical success factors and recommending approaches for spiral development and its application to evolutionary acquisition.  Their results appear in two reports, [Hansen 00] and [Hansen 01] and are available on the workshop website http://www.sei.cmu.edu/cbs/spiral2000.  This note is a summary of those workshops.

There was no dissent at either the February or the September workshop to the notion that EA/SD can help attain the apex more often than other process models.  Attendees were generally enthusiastic about the prospects.  The majority of speakers at the February workshop reported successes, as shown in Table 1, with some of these projects dating as far back as twenty years.  The September workshop had no project reports because it focused on EA which is too new to have yet been used for completed projects.  However, there was one report of successful use of spiral development for management of Joint Expeditionary Forces Experiment (JEFX) [Tillotson 00].

Despite the successes, a casual observer might read the workshops reports and conclude, with some cynicism, that they say:

We can best understand the fallacies of this interpretation by considering each phrase in turn, examining its roots and the reasons for hope.

1. We don’t understand EA and SD

The notion that EA and SD are not well understood could be garnered from the fact that their definitions were one of the themes of the February presentations and the subject of work groups at both workshops.  In fact, the work group sessions were intended to sharpen the definitions rather than to create new ones. The February work group did formulate four recommendations about further sharpening the definition, but the September group only formulated recommendations calling for further promulgation of the definitions.

Largely, the lack of September recommendations reflects the existence of definitions that have came into being between the workshops. Spiral development is defined in a recent paper [Boehm 00, 01] as

The spiral development model is a risk-driven process model generator.  It is used to guide multi-stakeholder concurrent engineering of software-intensive systems.  It has two main distinguishing features.  One is a cyclic approach for incrementally growing a system's degree of definition and implementation while decreasing its degree of risk.  The other is a set of anchor point milestones for ensuring stakeholder commitment to feasible and mutually satisfactory system solutions.

The papers go on to sharpen the definitions by defining the highlighted terms and describing "essential" properties of spiral processes together with allowable "variants" for each. (Earlier versions of the definition used the term invariant instead of essential.)

One common confusion among development practitioners is to confuse spiral development with any other method where the product is developed in a cyclic sequence of efforts rather than one single waterfall.  Usually such efforts are conducted without risk management reviews that may re-direct the project at the beginning of each cycle.  One example of this sort of confusion is the phrase in DoDI 5000.2 where it calls for an "iterative spiral approach" for development. Does this mean that there are spiral approaches which are not iterative?  Or does it try to clarify the meaning of “spiral approach” by equating it with an iterative approach?  There is no way to tell.  The first is slightly incorrect insomuch as a first iteration risk analysis may determine that only a single iteration is needed.  The second is more incorrect in that there are many iterative process models which are not spiral processes. 

Evolutionary Acquisition is an overarching strategy promulgated in DoDI 5000.2 [USD 01].  It is part of a phased strategy, as shown in Figure 2.  Initial Science and Technology (S&T) project(s) are conducted to advance a concept up the scale of technological readiness defined in Government Accounting Office report GAO [99].  When warranted (technology readiness level 6 or 7), the concept enters the Demonstration phase and is explored in an ATD, an ACTD (Advanced [Concept] Technology Demonstration), or a JWE (Joint Warfare Exercise). Somewhere in this phase a decision is reached to undertake an Acquisition Program for the technology.  This final phase is carried out by Evolutionary Acquisition.

In DoDI 5000.2, evolutionary acquisition is defined this way:

In an evolutionary approach, the ultimate capability delivered to the user is divided into two or more blocks, with increasing increments of capability. Deliveries for each block may extend over months or years. Block 1 provides the initial deployment capability (a usable increment of capability called for in the ORD). There are two approaches to treatment of subsequent blocks:

    The ORD includes a firm definition of full capability, as well as a firm definition of requirements to be satisfied by each block. ...
or
    The ORD includes a firm definition of the first block, but does not allocate to specific subsequent blocks the remaining requirements that must be met to achieve full capability. ...

Thus the project is implemented over the course of a sequence of funding blocks.

2. We aren’t convinced EA and SD work

The work groups suggest an uncertainty with the validity of SD and EA by asking for various kinds of demonstrations. These were the subject of at least 10 recommendations by six different work groups. Typically these recommendations called for case studies, experimental validation, pilot projects, and return-on-investment or business case analyses.

In point of fact, workshop participants were not themselves unconvinced as to merits of SD and EA. The spate of recommendations was intended to provide the sort of concrete evidence that is the best argument to present to skeptical and reluctant potential adopters.  Without a strong body of evidence, no amount of regulations and documentation can break a large, long-lived system free from the clutches of time-blessed procedures.

3. Here’s how to improve EA and SD

Although participants generally agreed that EA and SD were valuable techniques, they had suggestions for improvement in several directions. On the "people" dimension these suggestions ranged from working to understand the human behavioral and cultural down to the practical task of devising methods to encourage the identification and revelation of risks. On the "process" dimension, suggestions included development of tools, testing practices, and project assessment methodologies.  One particular suggestion called for a catalog of risk patterns and the corresponding appropriate process pattern.  At least two of the presentations described such catalogs in commercial use (Kitaoka in February and Jordano in September).  These suggestions were all aimed at making EA and SD easier to use and thus more attractive for future projects.

4. Here’s how to align DoD practices to EA and SD

Given that EA and SD are seen as attractive strategies, and given the fact that no other strategies have been particularly successful, workshop participants determined that DoD should encourage these policies. However, many factors in existing DoD policies, practices, and in-grained culture were seen as deterrents to the successful adoption of EA and SD.  It is no wonder then, that the work groups proposed a number of changes for the DoD.  These changes have been detailed in the work group reports and summarized under the categories:

Noteworthy recommendations appeared in all these areas. In contracting, a recommendation sought to dispel the parochial atmosphere: "Include requirements and incentives for sharing information and products in all appropriate contracts." In the policy area recommendations called for coordination of personnel changes with spiral cycles and focusing program oversight on risk management plans instead of the amorphous notion of status versus plan.  In the funding area, the September “Barriers” work group expressed the opinion that there were no policy barriers to writing contracts to implement EA and SD.  Nonetheless, a number of far-reaching recommendations concerning funding were made by various groups:

5. Here’s how to make teams work

Teamwork is a military hallmark.  Without teamwork, battles are lost before begun. Consider the D-Day  teamwork needed to move 176,000 troops on 4100 ships to Normandy beachhead.  Despite this glorious history, acquisition more often seems an adversarial skirmish than a team working to accomplish a task. Of course, contractors have adversaries; this is guaranteed by the system of competitive bidding. The adversarial attitude has, too often, been carried forward into the execution of the contract, once awarded. Contractors are loathe to share information with other contractors or even with the customer whose contract explicitly states a right to information.

Many efforts have been made to forestall adversarial interactions, notably "integrated product teams" formed from stakeholders including the acquirers, the contractors, the users, and many more.  This concept has even been adopted as part of the Spiral Development model.  Nonetheless, it does not always work to the full satisfaction of all. Observing this, the work groups had many suggestions about teamwork. Among them, the most vital is that:

IPTs should be given the authority to negotiate requirements, test strategy, and budget allocations

Other recommendations described steps to take to ensure a representative and engaged team.  The plan for the team should include team-building efforts.  If the team is distributed over a number of locations, it should have collaboration tools and various other recommended steps should be taken to maximize success.

6. Here’s how to ensure use of EA and SD

Given the workshop enthusiasm for SD and EA, it is not surprising that participants devoted attention to how to foster the use and growth of these methods for military acquisition.  Changing existing behaviors is always difficult.  It might seem helpful that military personnel can be ordered to change or that contracts can require behaviors;  in reality, neither stricture matters when people sit down alone to tackle their daily work and unconscious forces urge reversion to hoary-aged habits.

The emphasis on promulgation of EA and SD was evident in the workshops in that it was a theme of the September presentations and the subject of many recommendations at both workshops. However, there were no really innovative ideas about how to accomplish the change, other than the work of the "Operationalization" group in September.  The majority of the recommendations called for writing guidebooks, case studies, or other materials--all with no real incentives for people to actually read the results.  A second, more promising, collection of recommendations looked at the longer term and suggested emphasis on SD in various courses that acquisition personnel may encounter,  including undergraduate education, Professional Military Education (PME), Defense Acquisition University (DAU), and the Project Management Institute (PMI).

Other recommendations were to:

As part of analyzing the recommendations for this summary, one stood apart.  Several groups at both workshops recommended that a single focal point for EA/SD improvement and adoption be established within the DoD. Its purpose should be to shepherd the recommendations of the workshop and other efforts at improving EA, SD, and their use in acquiring military systems.  Such an office is imperative to avoid duplicate efforts and neglected opportunities. Project managers will come to see this office as the source of training materials they have used and the source of useful suggestions and tools they can apply to the current tasks.

7. The real situation

We started with a fairly cynical vision.  We can now interpolate this with a more realistic evaluation:

We don’t understand EA and SD Although not widely understood, the definitions are well established.
and we aren’t convinced they work, Real success are documented, but systematic study is indeed needed.
but, we want them anyway so here’s how to improve them, align DoD practices, make teams work, and The workshops produced valuable suggestions in all three of these areas.
ensure their use. Additional documents may help, but education is the only real solution.

EA and SD provide good process models for acquisition by the military and development by contractors.  These tasks are notoriously prone to descent on the slopes of Figure 1 to budget overruns, dilatory delivery, and ill-designed, poorly-executed products.  Thus EA and SD are worth pursuing for the promise shown by successful experiences reaching back, in some case, over one or two decades.

These two Spiral Development workshops have generated interest in EA and SD, widened knowledge of their merits and applicability, and produced numerous suggestions for improvement of all these. As a result, and with the aid of the focal office recommended by the workshops, it is possible to foresee the general maturation of EA and SD leading to a time when projects are routinely economical, timely, and useful.


Returning to the triple apex problem of attaining early-cheap-good, we see that, in a sense, spiral development guarantees two of these.  At the start of each iteration, risk analysis determines what is to be done and how long to take.  Budget considerations are part of this, so nothing will be attempted which cannot be paid for. Given these constraints it may not be possible to meet all the requirements.  That is, choosing spiral development is tantamount to accepting that the result may be less than full functionality; future funding increments and contracts may be needed. Thus even if a contract is fixed-price, it has a flexible outcome and does not doom a contractor to an inevitable financial loss. The risk management team--composed of members from the contracting, acquiring, and using communities--is responsible for delivery of as much as possible within the time and budget allotted. The contract will be completed on-time and within budget. Within those constraints, it will deliver a result acceptable to the users. Thus with Spiral Development within Evolutionary Acquisition the apex of success can always be attained!

 


References

[USD 01]

USD(AT&L), "DoDI 5000.2: Operation of the Defense Acquisition System", (Jan 01) http://www.acq.osd.mil/ap/dodi_5000_2_final_version_april_05_2002_Instruction.doc

[Hansen 00]

Hansen, F.; Foreman, J.; Carney, D; Forrester, E.; Graettinger, C.; Peterson, W.; and Place, P., "Spiral Development-Building the Culture: A Report on the CSE-SEI Workshop February, 2000." Software Engineering Institute, Carnegie Mellon University, Special Report CMU/SEI-2000-SR-006, July 2000. http://www/cbs/spiral2000/february2000/finalreport.html

[Hansen 01]

Hansen, F.; Foreman, J.; Albert, C.; Axelband, E.; Brownsword, L. and Forrester, E., "Spiral Development and Evolutionary Acquisition: The SEI-CSE Workshop, September, 2000," Software Engineering Institute, Carnegie Mellon University, Special Report CMU/SEI-01-SR-05, May, 2001, http://www/cbs/spiral2000/september/finalreport.html

[Tillotson 00]

Tillotson, Col. David F., "Joint Expeditionary Forces Experiment (JEFX)." http://www/cbs/spiral2000/september/Tillotson/index.htm

[Boehm 00]

Boehm, B. Spiral Development: Experience, Principles, and Refinements (CMU/SEI-00-SR-008) Pittsburgh, PA: Software Engineering Institute, Carnegie Mellon University, 2000. http://www.sei.cmu.edu/cbs/spiral2000/february2000/BoehmSR.html

[Boehm 01]

Boehm, B., Hansen, W. J., "Understanding the Spiral Model as a Tool for Evolutionary Acquisition," Crosstalk,  May, 2001, pp. 4-11. http://www.stsc.hill.af.mil/crosstalk/2001/may/boehm.asp

[GAO 99]

Government Accounting Office, BEST PRACTICES: Better Management of Technology Development Can Improve Weapon System Outcomes.(Chapter Report, 07/30/1999, GAO/NSIAD-99-162). http://www.gao.gov/archive/1999/ns991620.pdf