The adoption of CALS technologies and techniques is assumed to enable the delivery of benefits which make the strategic military/industrial aims of NATO significantly more likely to be achieved. Moreover, the CALS community has consistently championed the concept of the crucial value of information throughout the life of weapons systems in achieving these goals, a concept around which NATO is now re-aligning it's acquisition and logistics focus. However, there has always been a difficulty in establishing a direct causal link between the adoption of CALS tools and techniques and delivered benefits and outcomes- answering the question `how do I know when I've implemented CALS?"
In 1999 the CNAD commissioned a report from the NATO CALS Management Board (NCMB) to suggest appropriate metrics which would indicate the extent to which CALS has become embedded within core NATO processes.
The Objective of this paper is to establish a Metrics framework for the adoption of CALS tools and techniques within the NATO acquisition and logistics context; to make recommendations on how this framework should be implemented; to identify the likely resource requirements needed to sustain the activity; and to provide an illustrative example.
1.3. Working Method and Participants:
Following CNAD direction, the NCMB commissioned a Metrics Working Group (MWG), operating on a Project basis with an agreed project plan. The MWG was open to all members of the NCMB, the NATO CALS Office, and Industry members from the NIAG. The NCO acted as a Project Support Office throughout the activity. The MWG met on 3 occasions, in advance of NCMB meetings, in order that progress against the project plan could be reported through the NCMB to CNAD. Throughout its activities, the MWG has maintained close liaison with other interested authorities and Cadre Working Groups (such as Life Cycle Costing and Life Cycle Management) to ensure that the results are, as far as feasible, compatible with the emerging findings of these groups). The MWG also called on national sources from both industry and government to assist with data collection.
The Metrics WG (MWG) recognized the high degree of convergence between this activity and that of others such as the Life Cycle Working Group (LCWG), and that therefore Metrics should be developed within a common conceptual framework and approach to the rapidly emerging NATO concepts of Defense System Life Cycle Management. The LCWG have adopted and recommended to CNAD a standards-based approach to describing Life Cycle Management, based on recognized standards such as ISO 15288 and AAP-20 (PAPS). This approach attempts to reconcile the potentially conflicting viewpoints and roles of process and phase in the delivery, support and sustainment of Defense Systems, and has provided the MWG with a framework on which to map indicative performance goals and measurements.
The MWG recognised from the outset of the task that it would not be possible to provide definitive metrics to CNAD. Whilst the Group could determine that Depot Turn-Around time, for example, would be a sensible indicator of performance, it would be impractical to suggest a definitive target for NATO at this time. To do so would require that NATO nations share a common definition of their processes and expected outcomes. Much work is currently underway in precisely this area, through the Life Cycle Working Group, the Life Cycle Costing Group, the NATO Standardisation Agency, etc. and it is not unrealistic to assume that if nations could harmonise on a common Work Breakdown Structure for acquisition, for example, they could also agree definitive performance and target metrics. Until this time, the metrics (at Annex A) suggested by the MWG are necessarily generic.
