Final
CALS and EC/EDI Report

for the

OSD CALS IWSDB PROJECT

An MVP Joint Venture

July 22, 1994

Submitted by
ManTech International Corporation
Technology Applications Operations Center
1313 Locust Avenue
Fairmont, West Virginia

In support of
Contract DAAB10-89-D-0503
And in compliance with
CDRL Sequence Number A025

SOW Number 3.7

TABLE OF CONTENTS

LIST OF FIGURES
1.0  INTRODUCTION

1.1  Summary
1.2  Conclusions

2.0  OVERVIEW OF THE LIFE-CYCLE MODEL

2.1  Critical Success Factors
2.2  Characteristics of Successful Implementations
2.3  Barriers To Successful Implementation

REFERENCE LIST

LIST OF FIGURES

Figure 2.2-1  CALS Concept of Operation

1.0  INTRODUCTION

This report contains our analysis of the lessons learned in all of the case studies included in our economic studies library.¹ This task is addressing the need for a consistent approach for presenting different investment alternatives in order to establish an Integrated Weapon System Database (IWSDB) capability. Our approach, which we call a "business case," is similar to a Functional Economic Analysis (FEA) as promulgated by the DoD Corporate Information Management (CIM) Initiative. While an FEA is a type of business case having a functional focus, the IWSDB business case offers another perspective, that is, the perspective of a weapon system program office (SPO). The SPO must consider constraints resulting from the current weapon system life-cycle acquisition phase, the cross-functional nature of the weapon system, and the concept of operations for Continuous Acquisition and Life-Cycle Support/Electronic Data Interchange (CALS/EDI) support capabilities.

In preparing this report, we performed the following:

1. We built a library of government and industry case studies focused on the tangible economic benefits derived from implementing a shared technical data environment and/or using emerging technologies such as Concurrent Engineering (CE) and Flexible Computer Integrated Manufacturing (FCIM). When referenced in this report, the selected studies are shown in boldface type.

2. We reviewed additional related material on process improvement, re-engineering, lean production, cost systems, and program management that along with our experience and knowledge defined the context of our analysis. We have referenced this material throughout the report and have included a bibliography as Appendix A.

3. We analyzed relevant information from these materials in order to identify lessons learned which relate to critical success factors, performance metrics, and cost drivers.

1.1  Summary

The weapon system acquisition process follows the classic waterfall model which is a sequential approach to management of product/system development. The weapon system development cycle has been increasing in duration. However, the future of weapon system management will be characterized by less of everything, e.g., less funding, less time, fewer support personnel, less competition. Process improvements and business re-engineering will become increasingly necessary in order to enable management to meet enterprise objectives in the new lean environment. The SPO and contractors need information systems to support the implementation of these improvements in the management of weapons systems. The process improvements will be accomplished by initiatives that incorporate the objectives, information management needs, and strategies of the weapons system program offices.

Critical success factors (CSF) are causes of success or failure of an initiative. The CALS Strategic Plan, April 1994 Approval Draft² describes five CSFs that will be used to measure the success of CALS implementation. The five CSFs are shown below.

"Acquisition reform, adoption of dual-use technology, and adoption of a common, integrated information infrastructure all depend on widespread adoption of commercial standards and practices by DoD."

"The resulting information architectures need to be free of redundancy and to directly reflect the information needs of the processes and users they are designed to support."

DoD must have the "widespread capability to acquire, distribute, review, accept, store, access, maintain, and use data in a fully digital and standardized way."

"[R]eal improvements in DoD and business processes...will require major restructuring of jobs, organizations, policies and laws, an activity that demands and warrants significant management attention."

Our analysis confirms that these five critical success factors give a top level perspective to the CALS strategy. We looked at the economic studies and related material for factors at the implementation level and identified both positive and negative success factors. The critical success factors extracted from the lessons represent either characteristics of successful implementations or barriers to successful implementation. Characteristics are positive critical success factors that describe desirable elements of process improvement, re-engineering, and automation. Barriers are negative factors that need to be overcome by some action in order to achieve the desired "TO-BE" environment.

Most of the studies focus on the characteristics of successful implementations. For example, overall life-cycle costs are primarily established in the concept and early design engineering phases. Therefore, shared database implementations must begin in the earliest phases of weapons system development to reduce life-cycle costs.³ The lower costs are a highly desirable characteristic of automated systems such as the IWSDB. The downsizing of DoD will result in fewer people to support the procurement process. Information technology is a means of providing for the processes for procurement with fewer people. Re-engineering or process improvement needs to accompany the CIM initiatives. Fewer processes should result from the implementation of the initiatives. Businesses that have achieved successful process improvements have shown that team efforts produce the best results. Cross functional teams are allowed to make decisions unhindered by the bureaucratic hierarchy of management that slows down old fashioned enterprises.

The economic analyses presented in the studies show high rates of return on investment. In spite of what appear to be overwhelming arguments for investment in CIM technologies, weapon system procurement is paper driven with scattered islands of automation. Our analysis reveals that barriers to success are not overcome by either powerful technology or attractive characteristics. We also find no technical barriers that have prevented the implementation of automation projects. Instead, organizational barriers that dominate the implementations are ingrained in the current culture of DoD and commercial enterprises we have studied. Overcoming these barriers will be a challenge to management, which in several studies is the key component in the implementation. To obtain the support of middle management, top management must be committed to process improvement.

1.2  Conclusions

Initiatives need to concentrate on characteristics and barriers. Studies show that an economic justification for investment in automation based on "AS-IS" and "TO-BE" activity analysis does not, by itself, result in successful implementation. We believe this is because barriers to success are not actively removed by the initiatives. The business case cannot assume that achievement of the desired characteristics, along with the adoption of CALS standards, will be possible without action to remove these barriers.

The downsizing of DoD will result in fewer producers of military weapons. DoD procurement will be a smaller percentage of each producer's income. Therefore, DoD will not be able to use competitive pressure on contractors to reduce cost. Other cost-reduction methods need to be emphasized in the procurement process. One cost-reduction method is reduced time to market, using concurrent engineering practices. Thus, the weapon system life-cycle model of development will have to be streamlined or replaced with a new management approach.

Increased use of EC/EDI as a means to reduce cost is an appropriate initiative; however, DoD cannot reach out beyond the large prime contractors with its CIM initiatives because the smaller suppliers cannot economically meet special government requirements for electronic interchange. Increased use of commercial technology and lower cost implementation approaches are also requirements for cost effective use of EC/EDI technology.

DoD can achieve process improvement only with the commitment of management, especially mid-level management. Studies show that top management support is necessary, but not sufficient to implement change. Strong participation by middle managers is a requirement for success.⁷ This view is supported by the fact that companies that successfully re-engineer their processes are often battling for survival, creating an environment where radical changes are accepted.⁸ DoD's management needs to recognize the critical need for change.

Introduction of new technology is not enough. Existing policies, procedures, and processes were developed to fill a perceived need. If the users of new technology have the same needs, they will find ways to adopt the new technology to the old process and procedures. DoD must re-engineer its process and change policies and procedures that do not support the "TO-BE" environment. Fundamental policy changes are required in order to promote change. Although incentives to re-engineer exist, they are driven by cost reduction which does not generate enthusiasm or commitment.

The complexity of the weapons system life-cycle is increasing. This increased complexity comes from the knowledge burden of increasingly complex weapons and the logistics systems to support them.⁹ The business case must present a concept of operations that clearly describes how the IWSDB will support the knowledge of the weapons system. This knowledge will exist across the enterprise and throughout the contractor and supplier base. By incorporating the knowledge of the weapon system into a shared database, we can achieve economies of knowledge, replacing economies of scale once achieved by high volumes.

The business case must focus on a reduction of management activity in the "TO-BE" environment. Effective implementation of information technology will reduce the number of weapon systems management processes, not just low level activities. Average organizations will use automation to reduce low level activity such as clerical work, but over-achievers will obtain better value from technology through increased management productivity.¹⁰

2.0  OVERVIEW OF THE LIFE-CYCLE MODEL

The weapon system acquisition life-cycle follows a waterfall model. This model is based on the classic, sequential engineering approach to product/system development. It describes phases of the development process that occur with a minimum of parallelism. Figure 2.0-1 shows the model as it applies to the acquisition process.

Figure 2.0-1  The Waterfall Model of Weapon System Acquisition

Although institutionalized, the waterfall life-cycle model and sequential engineering are not rigorously applied, because of the bulky, paper-oriented implementation currently employed and the pressure to reduce schedules and cost. Few organizations have the time or stamina to strictly adhere to the process. As currently practiced, sequential engineering has a number of problems and weaknesses.¹¹

It forces cost reductions on the most important phases of production and operation. The manpower and management complexity are focused in the last two phases. By this point in the life-cycle, management can do little to contain costs, except to cut production volume or operation and support. When downstream activities experience increases in cost because of upstream actions, these actions are cost drivers that the business case should describe.

Current implementations are paper based. Most implementations use paper forms, documents, specifications, drawings, test reports, etc. Policies that require paper delivery need to be reevaluated.

It is difficult to trace requirements. As information is passed downstream, it can be garbled, lost, and embellished. The result can be either the failure to meet the requirements set forth in the specifications, or unrealistic expectations on the part of users and planners.

It delays the detection of errors until the end. Error detection is reserved for the formal testing phase of the program. Analysis or design errors are extremely difficult and expensive to correct.

It does not promote reuse. The waterfall life-cycle does not forbid or preclude reuse; it just does not promote or encourage the concept. Many DoD functional areas that can identify reuse opportunities become involved too late in the life-cycle.

It does not promote prototyping. By its very nature, the waterfall approach assumes that the analysis phase can be conducted once and only once. Proof of concept or other prototypes are seldom used as tools to explore alternative designs or requirements.

It fosters fragmented development. The separate phases theoretically allow competition at each phase, an anachronistic approach that is politically sound, but not suited to the reality of today's defense industrial base or the best business practices. Life-cycle requirements cannot be ascertained through a process of experimentation or trial that would allow iterations that improve the system.

These weaknesses are all within the scope of CALS and the IWSDB business case. The model is the viewpoint from which most of the studies and related material have been presented. With that in mind, we have organized our findings into two main categories that overlap the phases of the development process. These categories are characteristics and barriers.

2.1  Critical Success Factors

Critical success factors are derived from a systematic extraction of ground truth principles taken from lessons learned in many previous projects, both successful and unsuccessful. Each of these factors basically falls into one of two possible categories, either characteristics of successful implementations (of emerging technologies) or barriers to successful implementation. After identification and categorization of the critical success factors, the next step in the process will be to build an IWSDB business case template and to develop implementation strategies. These strategies will be aimed at the implementation of core requirements of the IWSDB and will coalesce around those critical success factors appropriate to the particular requirement. In this way, the lessons learned from past projects will be mapped and applied to future projects.

2.2  Characteristics of Successful Implementations

In the context of this report, the phrase "characteristics of successful implementations" refers to those actions and decisions which tend to fall within the span of control of the implementors and which positively influenced the outcome of previous attempts to adopt new technologies. Similar technology implementation efforts can achieve enhanced results by emulating the characteristics of previous successful attempts. By cross-referencing the characteristics identified in the lessons learned with a matrix of implementation strategies, the IWSDB business case will present an excellent starter set of strategies and supporting references for consideration by the SPO planning an IWSDB implementation.

Improved Procurement Practices
One of the most significant characteristics of successful implementations is the ability to recognize and the willingness to eliminate non-value activities. Nearly every study and report describing DoD activities or initiatives begins with a preface statement about the downsizing of DoD, a euphemism for reducing the number of people through termination or attrition. DoD faces a future where fewer people will be involved in the procurement of weapons, yet DoD will be challenged to better manage these procurements to meet more flexible requirements. Whether this is "lean" management or an "agile" enterprise can be debated, but the reality of fewer people is undeniable. It is only a question of "How many fewer?" Therefore, the business case must support the accomplishment of process improvement to support the "TO-BE" activities of the weapon system program using people, technology, or other means. The business case should not try to describe staffing levels. Rather, it should present alternatives that describe streamlined processes, because elimination of non-value added activities is a first principle of process improvement.

Another important characteristic of successful implementations is the migration away from physical custody of data, especially in hardcopy form. The B2 CITIS case study showed that delivery-in-place can provide the necessary access to data, replacing physical custody in the form of hard copy deliverables. While service infrastructures are currently oriented towards physical custody, this creates significant problems of quality. Once data is detached from its source, it becomes an artifact in time. Because changes to the original data are not automatically reflected in the copies, the correctness and integrity of the data rapidly disintegrate. Ensuring that information in an electronic database is current requires more emphasis on configuration management because of the need to distribute approved changes to the users sharing the data for the correct configuration or version.¹³

Migrating away from physical custody also has noteworthy cost implications. As the amount of redundant data grows, so does the cost of redundant administration and the data configuration management organization required to support it. Eventually, even the cost of physical storage becomes unreasonable, particularly in light of the decreasing value of the data.¹⁴

Figure 2.2-1  CALS Concept of Operation

Successful implementations begin with comprehensive, clearly stated requirements. Because IWSDB business case requirements are derived from the Government Concept of Operations (GCO) (see CALS GCO Figure 2.2-1), inaccuracies in the GCO can profoundly affect the business case. The GCO should focus on what data is required, how it will be used, and what the objectives are, not on how data is created, accepted, or transmitted. In other words, the GCO must be truly conceptual and should not include embedded methods or procedures derived from the "AS-IS" model. Without a clearly defined and accurate GCO, the business case will be based on best guesses and assumptions which may have no basis in reality.¹⁵

Successful implementations incorporate "best of breed" commercial practices. Studies often refer to the special management requirements of weapons systems. A careful examination of these requirements is necessary. The special requirement may not improve the efficiency or quality of weapons system acquisition and may be based on a policy or statute that should be re-evaluated.

Use of Standards
Adoption of and adherence to standards is critical to successful implementations. Sharing data between the government and multiple contractors (and their suppliers) requires standards for electronic data interchange (EDI). Implementation has been focused on business transactions, primarily shipping, billing, and payment, but is being extended to include engineering data. Continued extension of EDI standards to include all digital data is necessary to achieve the required shared data environment for CALS.¹⁷ Substantial savings can be realized by satisfying DoD requirements with commercial technology. Especially important is the adoption of commercial standards by DoD whenever possible.¹⁸

The probability of success is increased by the adoption of standards that address broad areas of scope. For example, Computer-Aided Design/Computer-Aided Manufacturing (CAD/CAM) and Production Management systems are frequently deployed on incompatible computer systems. Because CAD/CAM systems evolved from manual drafting to include "post-processing" applications such as creation of numerical control instructions or finite-element-analysis, the applications tend to be engineering oriented and typically do not integrate functions. The main thrust of Production Management systems has been material/inventory management and standard cost reporting, both of which are application driven, not data driven. The scope of these applications needs to be broadened into total enterprise, data driven, management systems.¹⁹ Existing standards provide for data identification and modeling, which are necessary in order to implement a totally integrated environment. Standards for representation of data and communication are necessary to successfully implement repositories of data, whether they are centralized or distributed. STEP/PDES is an example of a standard for product representation. EC/EDI systems are being extended to make engineering data exchange another type of transaction supported by standard EDI. The B2 CITIS case studies concurred with other studies that the largest share of weapon system life cycle cost is in operational support.

This support requires large repositories of system data, including engineering data.²⁰ In normal day-to-day repository operations, spare parts reprocurement represents by far the largest demand for engineering data. For example, the data required for bid sets account for about 62% of the B2's repository output. The time that elapses between the identification of data needed from the contractor and the time the end user has the data in hand is measured in weeks and months.²¹

Previous implementations have successfully demonstrated that the use of electronic standards can reduce contractor costs. The Prototype RAMP Implementation in Small Manufacturing (PRISM) study demonstrated that the standardized electronic format of PDES could eliminate the need to generate a drawing from aperture cards. The subject company experienced an average of 1.00 hour per Request for Quote (RFQ) to obtain the procurement data package associated with the PDES parts as compared to an average of 1.34 hours to obtain the procurement data associated with the same parts through their conventional process. This equates to a 25 percent reduction in time or a cost savings of $9.88 per RFQ processed. With the PDES parts, the RFQ contained the necessary technical data at the time it was received. With the conventional process, the company had to request an aperture card after receiving the RFQ and then had to send the aperture card to an outside service to produce the drawings. Particularly significant was the reduction of several days of calendar lead time because of the electronic transfer versus the mail system.²²

Previous implementations have also demonstrated that the use of electronic standards can reduce government costs as well. The Prototype RAMP Implementation in Small Manufacturing (PRISM) study demonstrated that PDES could save the government time in compilation of technical data packages. The government's total administrative time averaged 9.54 hours per PDES job (ranging from 5.45 to 15.65 hours). Equivalent non-PDES jobs averaged 10.04 hours in government administrative time. This represents a time savings of ½ hour per order, equating to an 8_ improvement. This improvement was attributable to the use of the PDES file as the technical data package and the impact on the required effort to assemble the package to be transmitted to the bidder.²³ By using PDES to prepare the technical data package, PRISM demonstrated significant lead-time reductions. Most of the activities in the procurement process were still done manually, so the savings were a small percentage of the total cost. These small savings are really just the beginning, however. Once the training pipeline and processes are in place to maximize the capabilities of this new technology, the savings will grow enormously.

Cross-Functional Involvement
Successful implementations take into account the impact of an investment on the entire life-cycle of a system. For example, the benefits of CALS will be realized only when functional activities have the infrastructure support of digital data. However, because functions deal with multiple weapon systems, it is difficult to justify the infrastructure investment for an individual weapon system. The "Catch 22" is program managers are reluctant to acquire digital data when the receiving activities are unable to process those data and the receiving activities are unwilling to invest in digital processing capabilities until digital data are available.²⁴ Broader perspectives are needed to overcome the stove pipe effect and islands of automation currently found in functional activities.

Training
Training, a particularly crucial success factor, is identified as an important element in almost every study. However, most technology implementation efforts dedicate too small a percentage of their budget to training. Few detailed training cost figures are available in the studies, but some summary data are available, such as the On Line Text Retrieval System which places training costs at $10M for a $310M investment.²⁵ Some studies do not include training expense as part of the investment, and yet training is identified as a primary reason for failure of process improvement/business re-engineering projects.

Another important aspect of training as a success factor is the need to include current topic areas such as Total Quality Management. For example, continuing education is required to support continuous process improvement. To manage acquisition programs, personnel will need an awareness of the paramount importance of quality, the presence of variation, the importance of process, and the array of available problem solving tools.

Addressing Cultural Issues

Concurrent Engineering Practices
Another element of successful implementations is the willingness to adapt to the requirements of new methodologies. For example, concurrent engineering uses co-location of project teams to promote team formation and information sharing. Proximity promotes interaction between personnel from different disciplines, which has a major positive effect by itself. Management should change the current practice of locating specialists within their functions. Although CE practices are crucial to execution, concurrent engineering is more a philosophy of management than a bag of tools. Training needs to focus on the philosophy, then follow up with the techniques.²⁸

This willingness to adapt must be fostered beyond the confines of a given contractor's organization. Clearly, DoD should not attempt to codify concurrent engineering. Industry and government participants in the study of The Role of Concurrent Engineering in Weapons Systems Acquisition strongly preferred consolidation, simplification, and coordination of existing standards and regulations. Concurrent engineering is capable of producing improvements when it is part of an integrated corporate competitiveness plan. It does not eliminate any engineering function, but requires downstream processes to be co-designed. It does try to replace repeated "test and fix" cycles with one-pass designs.²⁹

Definition of Scope
The degree of success in any implementation is inherently dependent on how clearly the scope of the effort is defined at the outset. This is just as true for the IWSDB business case as for any technology-based endeavor. In fact, it is just as important to specify what is outside the scope as what is within the scope. Therefore, activities not included in the business case as well as external inputs, controls, outputs, and mechanisms need to be specifically identified as being outside of the business case.³⁰

Formation of Virtual Corporations
Successful implementations adopt modern business strategies. One example is the pre-qualification of contractors and fast formation of virtual corporations required for Agile Manufacturing. The response of legal departments is key to the rapid creation of contracts. Cooperation will require sharing of what is now considered company proprietary data. The formation of "virtual corporations" replaces the traditional economies of scale with economies of knowledge, which enables the inclusion of expertise in the enterprise that does not exist in any one company.³¹ An example of the shortened lead and reaction times possible is the quick development and use of the bunker buster, GBU-28, during the Persian Gulf War. The highly successful project operated outside of the procurement procedures, with distributed decision making and close cooperation of all parties.

Use of Metrics
Successful implementations acknowledge the critical relationship between measurement and quality. In fact, nearly all of the studies address the need for measurement. The business case should describe how metrics will be gathered and used. The IWSDB would be a good place to store metrics, and ideally would automate the measurement process.

Use of Activity-Based-Costing
Another long range key to success is the implementation of management cost systems that provide appropriate activity process cost information to management. Present accounting systems are oriented to reporting spending by expense type. The accounting systems do not provide management with information in a form that is readily usable to manage day-to-day activities. Managers need to monitor and evaluate process improvements. To effectively control activities, managers must have an activity/process view of the cost information. Such views are currently developed on an ad hoc basis by individual managers to satisfy their specific needs and do not provide the program wide visibility required. Implementation of management systems that provide an activity/process view of spending across the entire program is required.

2.3  Barriers To Successful Implementation

In the context of this report, the phrase "barriers to successful implementation" refers to those situations in the development environment that tend to be outside the span of control of the developers and that have prevented the desired outcomes of previous attempts to adopt new technologies. Similar technology implementation efforts must find ways to overcome or bypass these barriers in order to achieve closure. By cross-referencing the barriers identified in the lessons learned with a matrix of implementation strategies, the IWSDB business case will present an excellent starter set of strategies and supporting references for consideration by the SPO planning an IWSDB implementation.

Traditional Procurement Practices
One significant barrier to contemporary development approaches, particularly concurrent engineering, is the inertia of traditional acquisition paradigms. The government could be held hostage by contractors for proprietary data, software, and processes.³² In The Machine that Changed the World, Womack and others show how such data can be accessible, even among competitors, when all parties share a vision of continuous improvement and when supplier relationships are cooperative. The culture of total "hands-off" procurement will need to be replaced by a new paradigm based on trust and disclosure. The replacement by a new paradigm is especially important to concurrent engineering principles. Because life-cycle costs are primarily determined early in the concept and design engineering phases, shared database implementations must begin in the earliest phases of weapons system development, perhaps before contracts have been established, and must include multiple contractors.
Concurrent engineering requires DoD to accept increased costs and accelerated schedules in the earlier acquisition phases to gain significant overall life-cycle savings. A RAND Corporation report identified "[i]nsufficient emphasis placed on production considerations during the development period" as a major problem. One cause of the problem is unrealistically low cost estimates by contractors and government managers. The result is insufficient funding for simultaneous design and production engineering. The procurement process is subject to manipulation through the separation of life-cycle phases for funding purposes, which encourages segregation of costs. For example, expenditures for production tooling may be postponed to show a reduction in development costs.

Resistance to Change
A significant hurdle to the implementation of IWSDBs is the existing psychological investment in legacy systems. Unlike the analysis of cost issues involved with updating hardware and software, the analysis of cultural issues is not so easy. Past efforts to implement new systems often overlooked the attitudes and concerns of the personnel involved. Experience shows that this can prove to be the biggest inhibitor to achieving the desired improvements in productivity. Another issue is the difficulty of quantifying costs of cultural migration through leadership and training.

The greatest risk in the implementation of automation is the risk that the necessary cultural changes will not happen. None of the business cases reviewed had a plan for implementation of cultural change, other than "training" in a general sense. The primary cultural inhibitors were:

· The effect of the new processes on the user's compensation, job security, opportunities for advancement, evaluations, etc.
· The changes needed in the procurement "rules" (written and unwritten) and how they should be implemented.
· The effect of the new processes on the profitability of the contractors and the environment of competition.

Lack of Common Approach
Another barrier to successful implementation is the difficulty of analyzing the economic results of previous implementation attempts. Studies calling themselves business cases do not follow a common methodology, do not take a common approach to presenting the economic analysis, and do not present conclusions or recommendations in a consistent manner. Some studies use traditional cost/benefit analysis. When activity-base-costing is referenced, the cost data is unavailable because accounting systems do not collect the "AS-IS" data. No risk adjustment is made to projected savings, although in some cases the life-cycle savings are estimated with a discount for inflation. No consistent estimated time frame is used in projecting future savings.

Knowledge Burden
Another significant obstacle to successful technological advancement is the sheer burden of complexity associated with emerging technologies. Complex machines and organizations embody knowledge. The greater the complexity of the system, the greater the knowledge burden involved and the greater the amount of information needed when surprising outcomes occur. Trying to avoid or mitigate these "rogue" outcomes, organizations attempt to acquire more knowledge from the outside or develop additional internal procedures, which in turn makes the organization more complex. Complexity can be assessed by an index of three elements: the number of components, the degree of differentiation among them, and the degree of interdependence among the components.³⁶

Analysis Conclusion
Our analysis has shown that initiatives within the DoD need to concentrate on characteristics and barriers. The studies indicate that an economic justification for investment in automation based on "AS-IS" and "TO-BE" activity analysis does not, by itself, result in successful implementation. Again, recognition and understanding of the characteristics of successful implementation for future projects and initiatives are paramount. The development of management strategies to avoid, overcome, or circumvent the barriers to successful implementation are also crucial. Projects and initiatives that take a careful look at these two subjects, we believe, will be more likely to achieve successful implementation.

APPENDIX A  REFERENCE LIST

Adelsberger, Heimo H., and John J. Kanet. "The Leitstand - A New Tool for Computer-Integrated Manufacturing." Production and Inventory Management Journal. (First Quarter 1991): pages 43-47.

Appleton, Daniel S. "Bringing the DoD into the 21st Century." CALS Journal. Volume 2, Number 2, (Summer 1993): pages 37-42.

Barbour, A. A. Cost Implications of Transferring Strategic Airlift C-141s to the Air Reserve Forces. Santa Monica, California: RAND Corporation, 1985.

Beazley, William G. "CALS and Its Impact." Inform. Volume 4, Issue 9, (Oct. 1990): pages 12-18.

Bennett, D.W. Technology Modernization Assessment Flexible Automation. Richland, Washington: Pacific Northwest Laboratory, 1990.

Berstein, T.N. Integrated Data Strategy/Baseline Concept Description. Wright Patterson Air Force Base: CALS Technical Center, 1993.

Buede, Dennis M., and Terry A. Bresnick. "Applications of Decision Analysis to the Military Systems Acquisition Process." Interfaces. Volume 22, Issue 6, (Nov/Dec 1992): pages 110-125.

Burley, Blaine F., and Kirk A. Phillips. A Decision Support Model Using Life Cycle Cost (LCC) Analysis to Select Cost-Effective Alternatives for Hazardous Materials, Thesis. Wright Patterson Air Force Base, Ohio: Air Force Institute of Technology, September 1993.

CALS Industry Steering Group. CALS Technical Report 002, Application of Concurrent Engineering to Mechanical Systems Design. Washington, DC: CALS Industry Steering Group, 1989.

Clark, Philip W., Gerald T. Kelly, and Richard W. Modrowski. "How CALS Can Improve the DoD Weapon System Acquisition Process." CALS Journal. Volume 2, Number 3, (Fall 1993): pages 32-37.

Clark, Philip W., Gerald T. Kelly, and Richard W. Modrowski. How CALS Can Improve the DoD Weapon System Acquisition Process. Bethesda, Maryland: Logistics Management Institute, 1992.

Concurrent Engineering Research Center. Showcasing Concurrent Engineering, A Scenario for Wahlco Power Products Operation. Morgantown, West Virginia: CERC Technical Report Series Research Note, 1992.

Crow, Kenneth. Effective Product Development Teams: Lessons Learned. Palos Verdes, California: DRM Associates, 1992.

Davidovich, Major Stevan M., and Dr. Rodney J. Heisterberg. "Attacking CALS Culture Shock." CALS Journal Volume 2, Number 2, (Summer 1993): pages 48-54.

Davis, Ted. "Adopting a Policy of Reuse." IEEE Spectrum. (June 1994).

DeMarco, Tom, and Timothy Lister. Peopleware: Productive Teams and Projects. New York, New York: Dorset House Publishing Co., 1987.

Demchak, Chris C. "Complexity, Rogue Outcomes and Weapon Systems." Public Administration Review. Volume 52, Issue 4, (Jul/Aug 1992): pages 347-355.

Dominach, Richard. "Design Reviews at a Distance Special Report: Concurrent Engineering." IEEE Spectrum. (June 1994).

Dryden, J., T. Britt, and S. Binnings-DePriester. An Analysis of Combat Aircraft Avionics Production Costs. Santa Monica, California: RAND Corporation, 1981.

Engineering Research Center. Lessons Learned at CE Pilot Sites for the DICE. West Virginia University, Morgantown, West Virginia: Prepared for DARPA, November 1992.

Fiorello, Marco. Estimating Life-Cycle Costs: A Case Study of the A-7D. Santa Monica, California: RAND Corporation, 1975.

GE Aircraft Engines, Concurrent Engineering Programs. DARPA Initiative in Concurrent Engineering (DICE); GE Pilot Project, Case Study Report. Cincinnati, Ohio: GE Aircraft Engines, 1992.

Gilbane, Frank. "Integrating New Technologies: Relational & Object Databases." CALS Journal. Volume 2, Number 3, (Fall 1993): pages 49-54.

Goss, Joseph R. and James Brunke. "Rockwell Implements CALS." CALS Journal Volume 2, Number 4 (Winter 1993): pages 42-45.

Groupe Secor Inc. Case Study: Newport News Shipbuilding. Groupe Secor Inc. No Date.

Hails, Robert E. Economic Analysis Model Evaluation for Technology Modernization Programs, Wright Patterson Air Force Base: Air Force Institute of Technology, 1983.

Hall, Gene, Jim Rosenthal, and Judy Wade. "How to Make Reengineering Really Work." Harvard Business Review (Nov/Dec1993): pages 119-131.

Hammer, Michael, and James Champy. Re-engineering the Corporation, Harper Collins Publishers, New York, NY, 1993.

Ingalls, Captain Stephen A. Application of Concurrent Engineering Methods to the Design of an Autonomous Aerial Robot. Georgia Institute of Technology, 1991.

Jurkus, Anthony F. "Requiem for a Lightweight: The Northrop F-20 Strategic Initiative." Strategic Management Journal. Volume 11, Issue 1, (Jan 1990): pages 59-68.

Kharbanda, Om P., and Ernest A. Stallworthy. "Lessons from Project Disasters." Industrial Management & Data Systems. Volume 92, Number 3, (1992): pages 3-44.

Kidwell, Robert S., and Charles Parisot. EDI's Role in Strategy for Digital Data Exchange. MAP/TOP Steering Committee Report to the OASD/CALS Policy Office, 1989.

Kuenzli, Wayne H., and Vincent R. O'Mahony,. "The Structure of Engineering Drawings: A Simple Example of Data Modeling." CALS Journal. Volume 2, Number 2, (Summer 1993): pages 55-58.

Kumar, Sanjiv, and Sherry Gelenius. "Integration of Legacy Data Within a CALS Environment - Issues, Concerns, and Approaches." CALS Journal. Volume 2, Number 2, (Summer 1993): pages 76-79.

Large, Joseph P. Bias in Initial Cost Estimates: How Low Estimates Can Increase the Cost of Acquiring Weapon Systems. Santa Monica, California: RAND Corporation, 1974.

Large, Joseph P. Development of Parametric Cost Models for Weapon Systems. Santa Monica, California: RAND Corporation, 1981.

Levinson, Harry, and Nan Stone. "The Case of the Perplexing Promotion." Harvard Business Review. (Jan/Feb 1990): pages 11-21.

Mackey, Wayne A., and John C. Carter. "Measure the Steps to Success." IEEE Spectrum. Volume 31, Number 6, (June 1994): pages 33-40.

Mansfield, Edwin. "New Evidence on the Economic Effects and Diffusion of FMS." IEEE Transactions on Engineering Management. Volume 40, Number 1, (Feb 1993): pages 76-78.

Massimini, Esther Marx. "Product Information Management Systems at Honeywell." CALS Journal. Volume 2, Number 3, (Fall 1993): pages 38-42.

McCormick, Janice. "The Case of the Not-So Supermarket." Harvard Business Review (Mar/Apr 1989): pages 15-28.

McGrath, Michael F. "DoD's Affordability Agenda - Making Manufacturing Lean and Agile." CALS Journal. Volume 2, Number 4, (Winter 1993.): pages 28-31.

Merrow, Edward W. Understanding the Outcomes of Megaprojects: A Quantitative Analysis of Very Large Civilian Projects. Santa Monica, California: RAND Corporation, 1988.

Mooney, Mike. "EDMICS." Inform. Volume 5, Issue 10, (November/December 1991): pages 12-15, 46-48.

Nagel, Robert N. "Understanding Agile Competition." CALS Journal (Winter 1993): pages 75-81.

Nagel, Roger., and Rick Dove. 21st Century Manufacturing Enterprise Strategy, Volumes 1 & 2. Bethlehem, Pennsylvania: Iacocca Institute, Lehigh University, 1991.

National Materials Advisory Board, National Research Council. Enabling Technologies for Unified Life-Cycle Engineering of Structural Components. Washington, DC: National Academy Press, 1991.

National Research Council. Dispelling the Manufacturing Myth: American Factories Can Compete in the Global Marketplace. Washington, DC: National Academy Press, 1992.

National Research Council. Improving Engineering Design: Designing for Competitive Advantage. Washington, DC: National Academy Press, 1991.

National Research Council. The Competitive Edge. Washington, DC: National Academy Press, 1991.

National Research Council. Toward a New Era in U.S. Manufacturing: The Need for a National Vision. Washington, DC: National Academy Press, 1986.

Neel, Colonel Timothy E. Concurrent Engineering: A New Paradigm. Carlisle Barracks, Pennsylvania: US Army War College, March 1991.

Newport News Shipbuilding Logistics & Technical Services. CALS Represents the Opportunity to Initiate a Continuing Process of Organization Renewal at Newport News Shipbuilding. Newport News, Virginia: Newport News Shipbuilding Logistics & Technical Services, No Date.

Niven, Daniel. "When Times Get Tough, What Happens to TQM?" Harvard Business Review. (May/June 1993): pages 20-34.

Northrop B-2 Division. Contractor Integrated Technical Information Service (CITIS) "Business Case" Feasibility Study. California: Northrop, Aug 1992.

Office of the Secretary of Defense, Continuous Acquisition and Life-Cycle Support. CALS Strategic Plan (Approval Draft). Washington, DC: Department of Defense, April 1994

Oliver, Judith B., Steven Kauder, and Richard G. Stieglitz Ph.D. "SEAWOLF Implements CALS Techniques." CALS Journal. (Spring 1993).

Patun, Ronald J., and Vijay Pandiorajan PhD. "Flexible Computer Integrated Manufacturing Using CALS Standards." CALS Journal. Volume 2, Number 4, (Winter 1993): pages 57-62.

Psihas, George P. "Maintaining the Defense Industrial Base: Evolving National Policies Place Defense in Great Jeopardy." Vital Speeches. Volume 59, Issue 6, (Jan 1993): pages 165-168.

Rockwood Informatics Corporation. Implementation Of OLTRS Within DND Business Case. Ottawa, Canada: Rockwood Informatics Corporation, 1991.

Rupp, Dale. "CALS-An Initiative for Creating a Digital Standard Format." IMC Journal. Volume 26, Issue 4, (July/Aug. 1990): pages 6-8.

Shavelson, Richard, and John D. Winkler. Can Implementation of Computers Be Justified on Cost-Effectiveness Grounds? Santa Monica, California: RAND Corporation, 1982.

Smith, Roy A. "Information Infrastructure: Corporate Strategic Weapon or Exploding Cigar?" CALS Journal. Volume 2, Number 4, (Winter 1993): pages 83-85.

Snodgrass, B. Neil. A Guide to Business Case Analysis for STEP Implementation. North Charleston, South Carolina: PDES, Inc., 1993.

South Carolina Research Authority. PDES, Inc., Sheet Metal Project Business Case Analysis Report. New Charleston, South Carolina: South Carolina Research Authority, 1992.

South Carolina Research Authority. Prototype RAMP Implementation in Small Manufacturing (PRISM). New Charleston, South Carolina: South Carolina Research Authority, 1992.

Steele, Lowell W. "Technology Maturation and Technology Substitution." Engineering Management Review. (March 1990): pages 11-24.

Strassmann, Paul A. The Business Value of Computers. New Canaan, Connecticut: The Information Economics Press, 1990.

Thompson, Arthur M., and Robert M. Mason. The Business Value of Integrated Information Systems In Engineering-Production-Logistics. Cleveland, Ohio: Center for the Management of Science and Technology, Case Western Reserve University, 1993.

Tripp, Robert S. "A Decision Support System for Assessing and Controlling the Effectiveness of Multi-Echelon Logistics Actions." Interfaces. Volume 21, Issue 4, (Jul/Aug 1991): pages 11-25.

United States of America. Department of Defense. Corporate Information Management: Functional Economic Analysis Guidebook. Arlington, Virginia: SRA Corporation, Jan 1993.

United States of America. Director of Defense Information, Office of the Secretary of Defense. DoD Enterprise Model Volume I. Arlington, Virginia: Department of Defense, 16-Feb-1993.

United States of America. Director of Defense Information, Office of the Secretary of Defense. DoD Enterprise, Model Volume II. Arlington, Virginia: Department of Defense, 11-Jan-1994.

United States of America. General Accounting Office. Automotive Industry: The Competitive Challenge to U.S. Companies. GAO/t-NSIAD-92-7. Washington, DC: General Accounting Office, January 1992.

United States of America. General Accounting Office. Comparison of Operational Capabilities and Support Costs for 15 Versus 20 Aircraft. GAO/NSIAD-93-209. Washington, DC: General Accounting Office, August 1993.

United States of America. General Accounting Office. Consider All Alternatives Before Proceeding With the F/A-18E/F. GAO/NSIAD-93-144. Washington, DC: General Accounting Office, August 1993.

United States of America. General Accounting Office. Defense Management: Stronger Support Needed for Corporate Information Management Initiative To Succeed. GAO/AIMD/NSIAD-94-101. Washington, DC: General Accounting Office, April 1994.

United States of America. General Accounting Office. DoD Initiatives Related to Cutting Costs. T-NSIAD-92-24. Washington, DC: General Accounting Office, March 1992.

United States of America. General Accounting Office. DoD's JUSTIS Decision. GAO/IMTEC-92-47R. Washington, DC: General Accounting Office, April 1992.

United States of America. General Accounting Office. DoDs Manufacturing Technology Program Needs Systematic Evaluation. Washington, DC: General Accounting Office, March 1992.

United States of America. General Accounting Office. Japanese Firms Involved in F-15 Co-Production and Civil Aircraft Programs. GAO/NSIAD-92-178. Washington, DC: General Accounting Office, June 1992.

United States of America. General Accounting Office. Navy Total Quality Management. GAO/GGD-93-29R. Washington, DC: Government Accounting Office, April 1993.

United States of America. General Accounting Office. Problems With the Sense and Destroy Armor Munition. GAO/NSIAD-94-59. Washington, DC: General Accounting Office, November 1993.

United States of America. General Accounting Office. Reliability of Weapon System Cost Reports Is Highly Questionable. GAO/AIMD-94-10. Washington, DC: General Accounting Office, October 1993.

United States of America. General Accounting Office. TSSAM Production Should Not Be Started as Planned. GAO/NSIAD-94-52. Washington, DC: General Accounting Office, October 1993.

United States of America. General Accounting Office. Weapons Acquisition: A Rare Opportunity for Lasting Change. GAO/NSIAD-93-15. Washington, DC: General Accounting Office, December 1992.

United States of America. Office of the Assistant Secretary of Defense Command, Control, Communications, and Intelligence. A Benchmarking Report on Functional Process Improvement, Office of the Assistant Secretary of Defense Command, Control, Communications, and Intelligence, November 1993.

Wells, Wayne. Economic Modeling for Manufacturing of Composite Parts. SAE Technical Paper 880433. Detroit, Michigan: SAE, 1988.

Willigan, Geraldine E. "The Case of the Expensive Expansion." Harvard Business Review. Jan/Feb 1989: pages 10-14, 16, 18, 22-25.

Winner, Robert I., James P. Pennell, Harold E. Betrand, and Marko M. Slusarczuk. The Role of Concurrent Engineering in Weapons System Acquisition. Alexandria, Virginia: Institute of Defense Analysis, 1988.

Womack, James, Daniel Hones, and Daniel Roos. The Machine That Changed the World. New York: Rawson Associates, 1990.

Yourdon, Edward. The Decline and Fall of the American Programmer. Englewood Cliffs, New Jersey: Prentice Hall Inc., 1993.

Zipkin, Paul H. "Does Manufacturing Need a JIT Revolution?" Harvard Business Review (Jan/Feb 1991): pages 40-50.

1 ManTech International Corporation. Preliminary Studies Library Report for the OSD CALS IWSDB PROJECT (Fairmont, WV: ManTech International Corporation, 1994)

2 Office of the Secretary of Defense, Continuous Acquisition and Life-Cycle Support. CALS Strategic Plan (Approval Draft). (Washington, DC: Department of Defense, April 1994)

3 Robert I. Winner, James P. Pennell, Harold E. Betrand, and Marko M. Slusarczuk. The Role of Concurrent Engineering in Weapons System Acquisition. (Alexandria, Virginia: Institute of Defense Analysis, 1988)

4 South Carolina Research Authority. Prototype RAMP Implementation in Small Manufacturing (PRISM). (New Charleston, South Carolina: South Carolina Research Authority, 1992)

5 GE Aircraft Engines, Concurrent Engineering Programs. DARPA Initiative in Concurrent Engineering (DICE); GE Pilot Project, Case Study Report. (Cincinnati, Ohio: GE Aircraft Engines, 1992)

6 United States of America. Department of Defense. Corporate Information Management: Functional Economic Analysis Guidebook. (Arlington, Virginia: SRA Corporation, Jan 1993)

7 United States of America. General Accounting Office. Defense Management: Stronger Support Needed for Corporate Information Management Initiative To Succeed. GAO/AIMD/NSIAD-94-101. (Washington DC: General Accounting Office, April 1994)

8 Michael Hammer and James Champy. Re-engineering the Corporation, (New York, NY: Harper Collins Publishers 1993)

9 Chris C. Demchak. "Complexity, Rogue Outcomes and Weapon Systems." Public Administration Review. Volume 52, Issue 4 (Jul/Aug 1992): pages 347-355.

10 Paul A. Strassmann. The Business Value of Computers. (New Canaan, Connecticut: The Information Economics Press, 1990)

11 Edward Yourdon. The Decline and Fall of the American Programmer. (Englewood Cliffs, New Jersey: Prentice Hall Inc., 1993)

12 Wayne A. Mackey and John C. Carter. "Measure the Steps to Success." IEEE Spectrum Volume 31, Number 6 (June 1994): pages 33-40.

13 Northrop B-2 Division. Contractor Integrated Technical Information Service (CITIS) "Business Case" Feasibility Study. (California: Northrop, August 1992)

14 Northrop B-2 Division. Contractor Integrated Technical Information Service (CITIS) "Business Case" Feasibility Study. (California: Northrop, August 1992)

15 Northrop B-2 Division. Contractor Integrated Technical Information Service (CITIS) "Business Case" Feasibility Study. (California: Northrop, August 1992)

16 Gene Hall, Jim Rosenthal, and Judy Wade. "How to Make Reengineering Really Work." Harvard Business Review (Nov/Dec 1993): pages 119-131.

17 Robert S. Kidwell and Charles Parisot. EDI's Role in Strategy for Digital Data Exchange. (MAP/TOP Steering Committee report to the OASD/CALS Policy Office, 1989)

18 Roger Nagel and Rick Dove. 21st Century Manufacturing Enterprise Strategy, Volumes 1 & 2. (Bethlehem, Pennsylvania: Iacocca Institute, Lehigh University, 1991)

19 Arthur M. Thompson and Robert M. Mason. The Business Value of Integrated Information Systems In Engineering-Production-Logistics. (Cleveland, Ohio: Center for the Management of Science and Technology, Case Western Reserve University, 1993)

20 Data needs to be made available in information-rich format. For example, contractors frequently have to reinvent computer-aided-design models from aperture cards or raster images to facilitate numerical control part programming. South Carolina Research Authority. Prototype RAMP Implementation in Small Manufacturing (PRISM). (New Charleston, South Carolina: South Carolina Research Authority, 1992)

21 Northrop B-2 Division. Contractor Integrated Technical Information Service (CITIS) "Business Case" Feasibility Study. (California: Northrop, August 1992)

22 South Carolina Research Authority. Prototype RAMP Implementation in Small Manufacturing (PRISM). (New Charleston, South Carolina: South Carolina Research Authority, 1992)

23 Ibid.

24 Philip W. Clark, Gerald T. Kelly, and Richard W. Modrowski. How CALS Can Improve the DoD Weapon System Acquisition Process. (Bethesda, Maryland: Logistics Management Institute, 1992)

25 Rockwood Informatics Corporation. Implementation of OLTRS Within DND Business Case. (Ottawa, Canada: Rockwood Informatics Corporation, 1991)

26 Gene Hall, Jim Rosenthal, and Judy Wade. "How to Make Re-engineering Really Work." Harvard Business Review (Nov/Dec 1993): pages 119-131.

27 GE Aircraft Engines, Concurrent Engineering Programs. DARPA Initiative in Concurrent Engineering (DICE); GE Pilot Project, Case Study Report. (Cincinnati, Ohio: GE Aircraft Engines, 1992)

28 Robert I. Winner, James P. Pennell, Harold E. Betrand, and Marko M. Slusarczuk. The Role of Concurrent Engineering in Weapons System Acquisition. (Alexandria, Virginia: Institute of Defense Analysis, 1988)

29 Robert I. Winner, James P. Pennell, Harold E. Betrand, and Marko M. Slusarczuk. The Role of Concurrent Engineering in Weapons System Acquisition. (Alexandria, Virginia: Institute of Defense Analysis, 1988)

30 South Carolina Research Authority. PDES, Inc., Sheet Metal Project Business Case Analysis Report. (New Charleston, South Carolina: South Carolina Research Authority, 1992)

31 Roger Nagel and Rick Dove. 21st Century Manufacturing Enterprise Strategy, Volumes 1 & 2. (Bethlehem, Pennsylvania: Iacocca Institute, Lehigh University, 1991)

32 T.N. Berstein. Integrated Data Strategy/Baseline Concept Description. (Wright Patterson Air Force Base: CALS Technical Center, 1993)

33 Gene Hall, Jim Rosenthal, and Judy Wade. "How to Make Reengineering Really Work." Harvard Business Review (Nov/Dec 1993): pages 119-131.

34 Arthur M. Thompson and Robert M. Mason. The Business Value of Integrated Information Systems In Engineering-Production-Logistics. (Cleveland, Ohio: Center for the Management of Science and Technology, Case Western Reserve University, 1993)

35 Roger Nagel and Rick Dove. 21st Century Manufacturing Enterprise Strategy, Volumes 1 & 2. (Bethlehem, Pennsylvania: Iacocca Institute, Lehigh University, 1991)

36 Chris C. Demchak. "Complexity, Rogue Outcomes and Weapon Systems." Public Administration Review. Volume 52, Issue 4 (Jul/Aug 1992): pages 347-355.