· CNAD accept this report and close out the task
· CNAD recognize and sponsor the remaining work identified in this report.
· CNAD assign responsibility for this work to a central group.
A. ILLUSTRATIVE METRICS - APPLYING THE MODEL
The purpose of establishing and tracking metrics is to provide a means of evaluating how well a particular activity or process is performing against a set criterion. This necessarily implies that the desired output or outcome is clearly defined, is measurable, and has an unambiguous stated value to be achieved.
A process or activity is based upon a set of inputs and a change mechanism leading to one or more desired outputs. The process or activity of interest must be fully understood before practical and useful metrics can be established. All processes or activities have a limit as to what can be achieved when operating optimally. In quality terms this is referred to as "The process being in control" and this can usually be measured. Once the process owner has determined what his/her process or activity is capable of delivering, then he/she must decide if this is acceptable. If it is acceptable then the task is to monitor via metrics to make sure it stays in control. If this is unacceptable then the process or activity necessarily must be reengineered to improve performance. This is usually when technology is introduced.
It is up to each individual, business entity, organisation or nation to determine what they are trying to achieve and then design a set of measures to validate their objective.
Thus far the concept of how metrics can validate the achievement of NATO's strategic objectives for improved defence system availability while saving time, reducing costs and improving quality has been examined. This will be further developed by an example.
The example below assumes that, based on a sound business case, NATO has decided to digitise its core business processes for Defence system life cycle acquisition and support.
To accomplish this, the format described in this paper will be used to develop 3 sets of metrics: 1) Environment; 2) Technology; 3) Process. This example will only develop a high level subset of metrics in order to demonstrate the process. As previously stated each nation does business their way and will have to develop their particular set of metrics to the degree necessary to meets their individual needs. Likewise, joint projects and programs will have to agree to a set of metrics that meets the specific need.
Based upon an analysis of current activity NATO has determined that it needs a program of Business Process Improvement. In order to meet its strategic objectives of improving efficiency and effectiveness while at the same time reducing time, cost and improving quality, NATO has decided to issue a policy to digitise the entire Defence System Life Cycle process by the end of 2005. The following steps will be taken:
A.2. Project Plan:
|
Objective |
Procedure |
Metric |
Digitise DS LC functions |
Policy |
Yes/No |
· Digitise all DS Acquisition functions by 2005 |
Strategy |
Yes/No |
· Digitise contracting function · Payment and accounting
|
Plan Plan
Plan
Plan/
Plan/ |
Yes/No Yes/No Yes/No
Yes/No
Yes/No
Yes/No |
· Digitise Material handling function by 2005 Decompose |
Strategy
|
Yes/No
|
· Digitise Maintenance and Repair function by 2005 Decompose |
Strategy
|
Yes/No
|
The above matrix is designed to demonstrate a structured approach to introducing business process change. While this change is being implemented, a set of metrics needs to be established which measures the degree of improvement resulting from moving from the AS-IS to a TO-BE environment.
Metrics must address all areas of the business environment in order to assess the overall business impact when change is introduced. To avoid sup-optimisation, a business needs managerial targets and control metrics, which provide a functional view as well as a process view. To fully assess the impact of change, it must be measured from the top to the bottom of the organisation. Very large organisations (Service, National, and International) may require several level of metrics to support effective management. The next matrix is designed to illustrate some top-level managerial metrics.
NATO/Nation/Service |
Environments |
Technology |
||
Resources |
Available Skills
|
· Training Targets · Applicability
· Bench Marking · Relevant range assessment
|
NATO/Nation/Service |
Environments |
Technology |
||
Infrastructure |
Infrastructure
· Office
|
· Benchmark · Technology cycle
|
A.4. Program/Project Level Metrics
The next level of metrics focuses on how well an activity is performing against its designed objectives of consistently providing top quality products in the shortest possible time and at the most economical cost to the customer. It is self-evident that these metrics should be appropriate to the activity examined, whilst supporting the overall managerial targets and metrics outlined above.
Phase |
Environments |
Technology |
||
Integrated Project Team
|
· Time
| |||
Digital environment
|
· Time
| |||
Develop Requirement
|
· Time
|
Phase |
Environments |
Technology |
||
Design & Develop |
Time, cost, quality | |||
Integrated Product teams
|
· Time
| |||
Digital environment
|
· Policy compliance
|
Phase |
Environments |
Technology |
||
System Design
· Performance
· Contract compliance and Audit
· Change Management · Concurrent review and Approval |
Time
· People
Quality
· Scrap and rework · Maintainability and Supportability (Predicted) · IETMS
· Availability · IETMS
· MTBF |
Phase |
Environments |
Technology |
||
Build |
Time, cost, quality | |||
Integrated Project Team
· Benchmarking
|
· Time
| |||
Digital environment
|
· Time
|
Phase |
Environments |
Technology |
||
System Manufacture
· Performance
· Financial Mgt
· Change Management · Production control
|
Time
· People
Quality
· Scrap and rework · Maintainability and Supportability (Tested) · IETMS
· Availability (Tested) · IETMS
· MTBF (Tested) |
Phase |
Environments |
Technology |
Process |
Metric |
Operate/Maintain/
|
Time, cost, quality,
| |||
Integrated Project Team
|
· Time
|
Phase |
Environments |
Technology |
||
Digital environment
|
· Time
|
Phase |
Environments |
Technology |
Process |
Metric |
Operations, Maintenance, & Support
|
· Time
|
