Submitted by

In support of

Contract DAAB10-89-D-0503

And in compliance with

CDRL Sequence Numbers A004 and A007

SOW Numbers 3.1.1, 3.1.2

TABLE OF CONTENTS

   

List of Figures

List of Tables

1.0  INTRODUCTION

2.0  OVERVIEW OF TESTING TOOLS

3.0  Technical Discussion

APPENDIX A:  EVALUATED TOOLS

APPENDIX B:  SGML SOFTWARE TOOLS

APPENDIX C:  BIBLIOGRAPHY

APPENDIX D:  CALS ACRONYMS AND ABBREVIATIONS

List of Figures

      

Figure 1.0-1 1840 and 28000 Series Military Specifications

List of Tables

      

Table 2.1-1 MIL-STD-1840B Conformance Checking Tools

Table 2.2-1 MIL-D-28000 Test and Validation Tools

Table 2.3-1 MIL-M-28001 Test and Validation Tools

Table 2.4-1 Test Tools

Table 2.5-1 MIL-D-28003 Test and Validations Tools

1.0  INTRODUCTION

      

This document outlines a study of test and validation tools used for the MIL-STD-1840 and MIL-STD-28000 series of standards. The tools are a means of establishing the conformance of digital data to the respective military standard. The establishment of a set of comprehensive tools to ensure compliance of contractor digital data deliverables to the Continuous Acquisition and Life-Cycle Support (CALS) formats will be the ultimate goal. The testing tools are a critical component of the CALS data acquisition process, because they ensure that only those components that meet the CALS guidelines are able to enter the system, for example the Integrated Weapon Systems DataBase (IWSDB) or a CALS reuse repository. The tools will serve as a "front door" in serving as a quality check upon which future functionality of those items allowed into a database or repository is based. A brief description of the standards follows.

The primary objective of this task is to provide the weapon system's Program Manager and acquisition personnel with the ability to effectively determine whether digital deliverables conform to their relevant CALS standards. This report discusses the current CALS Core Standards for technical information exchange and the test and validation tools being used to test compliance/conformance to the standard.

To ascertain the acceptance level that might be attainable given the test and validation tools in use today, we will outline procedures that investigate several areas designed to assist the acquisition/program manager in better determining which tools are available in a specific operating environment and to provide a pilot CALS Test and Validation Tool Reuse Repository. The repository would provide up-to-date information on CALS Test and Validation Tools from a reuse perspective, without duplicating capabilities and services presently being provided by other entities.

This report involves the results of surveying existing Commercial-Off-The-Shelf (COTS), Government-Off-The-Shelf (GOTS), and public domain CALS test and validation tools and establishing how they are being used today. Also, we will discuss the CALS Core Requirements standards and describe the levels of testing observed at the various Government agencies visited.

At the heart of the CALS initiative has been the exchange of technical product information in digital format. A key objective was and is the ability to enter data once and have it accessed and used over and over as a weapon system progresses through its life-cycle. The only reasonable way to discuss meeting the objectives of CALS is through the utilization of certain Core Technical Data Exchange Standards.

The creation of an IWSDB requires the capability to exchange technical information among diverse systems. Standards allow this to occur by defining neutral data formats that can be used to define data in a comprehensive, unambiguous manner enabling interoperability among Government and industry systems of many types. Because CALS is conceptually built around the ability to exchange digital data files between linked systems or to access data in a physically distributed but logically integrated database, CALS Standards for product data interchange provide the common rules that allow the whole process to move forward.

The following section describes the core CALS standards being used today for exchanging technical product data in neutral format. Figure 1.0-1 shows the 1840 and 28000 series of Military specifications.

1.1  Overview

This section details the standards, their architecture, the current status and implementation issues.

1.1.1  MIL-STD-1840B - Automated Interchange of Technical Information

MIL-STD-1840B serves as a central standard for the CALS environment. It supersedes and enhances MIL-STD-1840A and standardizes formats for the exchange of digital information between organizations or systems in order to facilitate the development and logistic support of defense systems throughout their entire life cycle. MIL-STD-1840B defines the format of the data to be exchanged in the CALS environment as well as the mechanisms and the parameters of those mechanisms, required for the exchange to take place. This standard is applied wherever information is exchanged. There are no restrictions to its application. It can be used for the exchange of American Standard Code for Information Interchange (ASCII) text files, product definition data files, raster image files, or vector graphics files.

Information is exchanged as a transfer unit. The method of encoding for the exchanged information is specified by contract. A transfer unit consists of a transfer unit declaration file, the appropriate transfer unit data file header records, and at least one transfer unit data file. The transfer unit declaration file is an ASCII file that provides all the needed information to uniquely identify the contents of the transfer unit. It provides such information as where the files came from, where they are going, what type of data is included, and how much data is being exchanged. All of these records must be present in the file and in the order stated by the standard. The medium of delivery is 9-track magnetic tape unless the involved parties mutually consent that an alternate method of exchange, such as floppy disks, shall be used. Alternate methods of exchange could include media such as floppy disks, Digital Audio Tape (DAT), magneto-optical disks, etc. The electromagnetic media will be packaged such that it is protected against dirt, moisture, or electrostatic discharges.

The MIL-STD-1840B standard defines, by reference, the formats and structures of the data files used for the transfer and archival of technical data in digital form. It clearly defines the formats, standardized header records, contents of the files used for the exchange of data as well as requirements during shipment for the labeling, protection, packaging, and the marking of media. The provisions for "protection" of media require that electromagnetically inscribed information transfer media such as encoded magnetic tapes and disks be protected against dirt, moisture, and electrostatic discharge damage during shipment.

This standard is not designed to be usable for specific applications, but is not restricted in any way in its application. This military standard is intended for technical information that includes product data, product acquisition and implementation data, and product support data. Product data includes engineering drawings, specifications, as well as new and evolving digital data forms that provide the data in a platform independent form directly usable by computer applications. Product acquisition and implementation data includes parameters and data necessary to manufacture and/or acquire an entire defense system. Product support information includes training and maintenance manuals with their associated illustrations needed to maintain a defense system in a required state of readiness. The scope of this data covers the entire life cycle of a weapon system.

MIL-STD-1840B provides general requirements for Technical Publication, Product Data, and Electrical/Electronic Application Data File document types. The Technical Publications document type includes files that contain MIL-M-28001 Standard Generalized Markup Language (SGML), Document Type Declaration with no text, Formatting Output Specification Instance (FOSI), SGML text entity, MIL-D-28000 Initial Graphics Exchange Specification (IGES), MIL-R-28002 Raster, MIL-D-28003 Computer Graphics Metafile (CGM), Page Description Language (PDL), gray scale or color illustration, or special word data.

In any typical application, the hardware and software must prepare the files for transfer on the sending system by adding header information and writing the files to the selected media type. The hardware and software on the receiving system must process the files by reading the files from the selected media type and stripping the added header information. The media must also be labeled, protected, packaged, and marked appropriately during shipment in accordance with the MIL-STD-1840B standard.

1.1.1.1  Architecture

MIL-STD-1840B is composed of the following six sections:

1.1.1.2  Status and Planned Extensions

MIL-STD-1840B was released on November 3, 1992. It supersedes MIL-STD-1840A which was released 5 years earlier on December 22, 1987. The next version of this standard is expected in late 1995.

MIL-STD-1840B is required for the delivery of technical CALS data. It is approved for use by all agencies of the Department of Defense (DoD). It has been implemented and used successfully by numerous vendors. The use of MIL-STD-1840B will undoubtedly continue to grow because this standard is at the heart of the CALS interchange environment. The Department of Defense will continue to require that CALS procurements adhere to the requirements of this standard.

The future revisions of MIL-STD-1840B will address solid modeling for system design, interactive retrieval and use of technical information, and other potential computer applications for defense systems of the future. The interactive retrieval and use of information will be necessary for the implementation of the Contractors Integrated Technical Information Service (CITIS) and the IWSDB. It is expected that expert systems will be included in future revisions.

MIL-STD-1840 continues to recognize the format required for IGES information but does not attempt to address the Standard for the Exchange of Product data model (STEP) or Product Data Exchange standard using STEP (PDES) formats. This will be done in future revisions of MIL-STD-1840.

Many of the standards and specifications required or referenced by MIL-STD-1840B are evolving significantly due to rapidly advancing technologies. These will have to be further implemented in future revisions of this standard.

Efforts are currently under way to form a recognized working group with international participation. A Target Capability (TCAP) document is being developed for circulation among the CALS Industry Steering Group (ISG). It recommends that the following areas be addressed in the next revision of MIL-STD-1840:  

1.1.1.3  Advantages

MIL-STD-1840B, a standard for interchanging digital technical data, is a core standard for the CALS environment. In addition to the advantage of being required for the delivery of CALS data, this standard has the advantage of having an essential function which facilitates the sharing of technical data within and among autonomous organizations.

This standard clearly defines the mechanisms for exchanging digital data with formats defined in other CALS specifications (MIL-D-28000, MIL-M-28001, MIL-R-28002, and MIL-D-28003). The reference to and use of other standards and specifications allow this standard to evolve, as those standards and specifications evolve, in order to take advantage of advances in current technologies or to use new technologies.

MIL-STD-1840B contains specific detailed instructions for using 9-track magnetic tapes as a medium for exchange of digital data. It contains flexible provisions that rely on agreements between sender and receiver for using other media such as diskettes, Write Once Read Many-times (WORM) optical disks, and Compact Disk Read Only Memory (CD-ROM) for the exchange of technical data. This reliance on agreements provides MIL-STD-1840B with the advantage of being accommodating to changing user needs as well as being able to adapt to new requirements and/or guidelines.

Another advantage of MIL-STD-1840B is that it clearly defines the formats, standardized header records, and the contents of the files used for the exchange of data as well as requirements for the labeling, protection, packaging, and the marking of media during shipment. The standard also addresses electronic product data, new packaging of data for electronic trade business transactions, and electronic product data technology.

1.1.1.4  Implementation Issues

MIL-STD-1840B is required for the delivery of CALS technical data. However, the media for delivery must still be selected. This standard specifies the media for exchange of CALS information and includes provisions for using nine-track magnetic tapes, diskettes, WORM optical disks, and CD-ROM.

Although given recognition as viable CALS media, actual specifications for CD-ROM and WORM optical disks are not clearly defined in this standard. The generic description of requirements leaves room for portability problems in the areas of placement, arrangement, and structuring of files on media other than 9-track magnetic tapes. Informal agreements between sender and receiver must be made to ensure complete portability when CALS data is interchanged on any medium other than 9-track tapes. In addition, other media such as the Bernoulli removable disk drives are not considered, thereby leaving the impression they are not to be used.

Corresponding hardware and software must be present at the preparing (source, sending) site and the receiving (destination) site to accommodate the selected media for the information interchange. The files on the sending system must be prepared for transfer by adding header information to them and writing them to the selected media type. The files must be processed on the receiving system by reading them from the selected media type and stripping the added header information. The configuration management functionality of MIL-STD-1840B requires careful management within the software to assure that correct information is added to the file headers.

The CALS Test Network (CTN) was established to perform end-to-end testing of the exchange of digital data using MIL-STD-1840B. "CALS-compliant" systems must have the capability of reading and generating data or media conforming to MIL-STD-1840B. Because MIL-STD-1840B is at the heart of the CALS interchange environment, DoD will continue to require that CALS procurements adhere to the requirements of this standard.

MIL-STD-1840B was developed by the National Institute of Standards and Technology (NIST) under the direction of the Office of the Secretary of Defense (OSD) CALS Office. All of the CALS standards including MIL-STD-1840B are currently being managed by the CALS Digital Standards Office (CDSO) at the Wright-Patterson Air Force Base, Dayton, Ohio.

1.1.2  MIL-D-28000A - Digital Representation for Communication of Product Data (IGES)

This specification defines the CALS compliant subsets of the IGES for the interchange of vector graphics. The focus of this standard is Computer-Aided-Design (CAD) information, primarily the digital representation of geometry. IGES is a vector-based neutral format for the exchange of product model data. IGES is the most widely used product data exchange specification in the US. It was designed specifically for CAD/Computer Aided Manufacturing (CAM) systems for the exchange of geometric models and drawings. The data format is in ASCII with a fixed line length. MIL-D-28000 IGES is organized into five classes by application area to meet the delivery need of that application. These classes are:

1.1.2.1  Application Subsets

The first four classes specify the entities needed for specific application subsets. In this way, the recipient of a MIL-D-28000 data file may specify the class of data needed without becoming an IGES expert. The Class V application protocol is for the exchange of product data for 3D piping system models only.

Most CAD systems have some type of IGES translator, and even some non-CAD systems support the IGES specification. The support for MIL-D-28000 (subsets) is not as widespread as the support for the full IGES standard. The greatest stated support of the subsets comes from the commercial flavoring software and syntax checking software. MIL-D-28000 Class II, engineering drawings, is the most commonly supported class.

The first four classes of MIL-D-28000A specify the entities needed for specific application subsets. In this way, the recipient of a MIL-D-28000A data file may specify the class of data needed without becoming an expert on the IGES. The MIL-D-28000A application subsets specify the entities allowed in that class through a list of ASME Y14.26M entities given in table form. Limits on the entities are given through notes on that table. Rules are also given for the entity construction. Guidance is provided for MIL-D-28000A file construction requirements for each section (start section, global section, directory entry section, parameter data section, and terminate section) of an IGES file for each class.

The four application subsets defined within MIL-D-28000A are described in the following paragraphs.

1.1.2.2  Application Protocols

An Application Protocol (AP) is a way to transfer defined product data through IGES. An AP documents the user requirements for an application in a graphical model called an Application Reference Model (ARM). The requirements in the ARM are then represented by specific IGES entities in a given AP (the AIM). APs enable IGES to be used to transfer product data reliably until PDES/STEP is available from the commercial CAD vendors. APs provide a defined and more reliable method for transferring product data through IGES.

An AP is composed of the following elements:

APs are very specific in nature. For example, the 3D Piping AP (Class V) exclusively supports the exchange of product data for 3D piping system models. It does not support piping engineering drawings. A user wishing to transfer an engineering drawing of a piping system would have to use an Engineering Drawing AP. Also, only CAD/CAM systems supporting piping will be able to support the piping AP. A CAD/CAM system that does not support piping does not have the appropriate constructs within its database to either output data in the Piping AP or input the data reliably. APs will provide increased information transfer, but with a much narrowed scope in the information that is transferred.

1.1.2.3  Status and Planned Extensions

MIL-D-28000A is based upon an underlying American National Standard (ASME Y14.26M - 1989) and the IGES Version 5.1, both of which were developed by the IPO. As such, the Office of the Assistant Secretary of Defense (OASD) CALS Evaluation and Integration Office (EIO) cooperates with the voluntary IPO through the IPO's CALS/IGES Special Interest Group (SIG). The CALS/IGES SIG reviews proposed changes and suggests new changes at the IPO meetings. The CALS/IGES SIG is a source of IGES technical expertise and is instrumental in ensuring the quality of revisions or amendments to MIL-D-28000A.

A new application protocol, the Engineering Drawings Application Protocol (AP), is being developed for MIL-D-28000A. It is being created by members of the Navy Industry Digital Data Exchange Standards Committee (NIDDESC) and the IGES/PDES Organization. The AP will be submitted to the OASD CALS EIO for inclusion in the next amendment or revision of MIL-D-28000.

1.1.2.4  Implementation Issues

The method by which the senders of a MIL-D-28000A file will produce application subsets is a possible concern for the implementation of MIL-D-28000A. The preferred method is for the originator's CAD system's IGES translator to produce the MIL-D-28000A file. An alternative method is for the CAD system to produce the IGES file that is subsequently run through commercial flavoring software to produce the MIL-D-28000A compliant file. This method must be performed very carefully to prevent any loss of the file's underlying structure, and is not suited for the transfer of application protocols.

Even perfect transfer of the application subset does not ensure that all of the information in the original CAD model will be translated. For example, a CAD system may recognize objects such as resisters and capacitors in its internal database, but because IGES has no standard way to represent such objects, these objects may be transferred as a grouping of points, lines, and curves that represent the object. The concept that a group of entities represent an object is not necessarily conveyed by the subset to the receiving CAD system. MIL-D-28000A displays an awareness of this problem by specifying that "it is the intent of this specification to evolve in the direction of application protocols to ensure quality data exchanges." The application protocol work is being developed within the IGES/PDES Organization to transfer objects within an application area instead of merely a geometric representation with little standardized intelligence attached. The 3D Piping AP is the first AP developed by the IPO.

Most IGES entities are general purpose in nature. They can be combined to create constructs needed for product data transfer, such as a circuit in an electrical application, but they do not rigorously define how this is done. This can be a problem in transfer, because unless the receiving system knows how the IGES entities were combined to create the construct, the receiving system may not be able to interpret it. The basic data will be translated, but all the information needed to translate product data for the application will not be available.

MIL-D-28000 does not contain any rationale for why a specific set of entities was chosen for an application subset over another set of entities. This can make extensions to the subsets laborious, and raises many questions from the user and vendor community.

1.1.2.5  Testing and Validation

The CTN was established to demonstrate the CALS standards, test their effectiveness, and identify needed improvements to the standards. Currently, the CTN tests only the CALS data interchange standards. The CTN uses naturally occurring tests and structured tests. After the test results have been reviewed and approved by the CTN, the test reports are put on the CALS Bulletin Board for public view. The CTN testing performed by Industry participants and lead testbeds is directed by the CALS Test Network Office reporting to the Standards and Technology Division of the CALS EIO.

The CTN has test packets for testing MIL-D-28000 Class I and II, which currently need to be upgraded to the MIL-D-28000A version. Test cases for Class V are currently under development. The CTN does not have test packets for Classes III or IV. The main contents of the CTN test packets are IGES files to process into the CAD system, scripts to follow in creating the data on the CAD system for subsequent output, and plots to show the expected results. The test results from the naturally occurring tests are called "Quick Short Test Reports" (QSTRs) and are available from the CTN or the CTN Bulletin Board.

1.1.3  MIL-M-28001B - Markup Requirements and Generic Style Specifications for Electronic Printed Output and Exchange of Text (SGML)

This standard defines CALS requirements for automated, page-oriented, ASCII-text-based documents. Included in 28001 are SGML for source data, output specifications for composition processing, and page description language options for composed document display.

The SGML was developed to address the problems of information exchange and management challenges associated with electronic printed output and exchange of textual information. The ISO participated in the development of the SGML standard and adopted it in 1986 as ISO 8879. This standard completely defines the terms and syntax to specify the structure and content of a document. ISO 8879 does not provide functionality for specifying appearance or formatting requirements.

The DoD application of SGML is specified in MIL-M-28001. ISO 8879 defines SGML as a meta-language. Many different SGML languages satisfying the grammar and syntax specified in ISO 8879 can be obtained by choosing from among different features and options afforded by ISO 8879. MIL-M-28001 specifies the DoD CALS SGML implementation by choosing certain SGML features and setting certain parameters. MIL-M-28001 also uses SGML to provide a capability for specifying the formatting of CALS SGML documents.

An ISO 8879 SGML document contains three parts; the SGML declaration, the Document Type Definition (DTD), and the Document instance. A CALS SGML document contains these parts and may also contain a Formatting Output Specification Instance (FOSI). The SGML declaration defines what characters will be allowed in the rest of the document and how they will be encoded. The SGML declaration also specifies how certain characters are to be interpreted along with other special rules and definitions. The DTD defines the content and structure of a document. It refers to how the information in a class of documents such as technical manuals, books, memos, etc., are related, and defines any dependencies. Markup of a document provides an unambiguous definition of its structure and content, allowing automated data processing software to process the document in a predictable manner. Also defined in the DTD are entity declarations. The document instance is the information content of the document, marked up in accordance with a DTD. This markup may include declarations, tags, and entity references. The FOSI specifies the desired appearance of the information content of the document. This output formatting description capability is not contained in ISO 8879 but is found in MIL-M-28001. MIL-M-28001 contains an Output Specification (OS) DTD that defines how FOSIs are to be developed and interpreted.

"SGML markup" (or an "SGML document instance") consists of unformatted text with inserted SGML "tags" that correspond to the elements and attributes of the DTD. These tags identify elements of the text (e.g., titles, paragraphs, tables, footnotes) defined in the document's DTD. The "marked up" document (or SGML document instance) can then be "parsed" using special software to determine if the document's tagging conforms to the DTD.

Unlike MIL-M-28001A which contained the DTD for documents conforming to MIL-M-38784B and 12 DTDs based on that DTD, MIL-M-28001B does not contain any DTDs to be used for delivering data to the Government. Like MIL-M-28001A, MIL-M-28001B will provide the so-called "template" DTD in Appendix A. Its chief function is to serve as a "toolkit" for the construction of DTDs by providing SGML coding that can be incorporated or modified in DTDs being developed. The template DTD also provides a set of elements and attributes for use in new DTDs.

A "declaration subset" is used to define a new DTD in terms of changes to an existing DTD. The implementation of the changes in a declaration subset results in a complete and different DTD for the corresponding military specification. The use of such "declaration subsets" in creating new DTDs this way actually allows tighter control over the number of distinct DTDs. While DoD wishes MIL-M-28001B to be implemented in a wide variety of applications, DoD is also quite concerned with the uncontrolled proliferation of DTDs.

New DTDs must be developed for all applications of automated technical publications for which existing CALS DTDs are inadequate. New DTDs should be constructed using those elements and attributes of the "template" DTD as defined in Appendix A of MIL-M-28001B whenever possible. This appendix provides general guidance for development of DTDs. The DTD for a given class of documents such as technical manuals will either be provided in the governing specifications for such documents or else be completely specified within the governing specifications.

1.1.3.1  Applications

The development of those CALS DTDs contained in earlier versions of MIL-M-28001 and based on the DTD for MIL-M-38784B conforming documents was a coordinated effort of the Navy, Army, Air Force, and Industry. MIL-M-38784B has been superseded by MIL-M-38784C, and the DTD for documents conforming to MIL-M-38784C is provided in Appendix B of MIL-M-38784C. In addition, numerous other DTDs were developed, including those DTDs developed by the Air Force under the Technical Manual Specifications Standardization (TMSS) program, as well as a Work Package DTD developed for Naval Air Systems Command (NAVAIR). These DTDs will be included or specified in the appropriate functional specifications.

Currently, MIL-M-28001B is concerned with the digital interchange of paper-based manuals. However, efforts are underway to define the digital interchange of "paperless," i.e., screen medium technical publications. The MIL-D-87269 specification uses SGML to specify a revisable database for the support of interactive electronic technical manuals.

1.1.3.2  Architecture

MIL-M-28001B is composed of the following six sections and 3 appendices:

1.1.3.3  Status and Planned Extensions

It is in the interest of both DoD and industry to agree on the widest applicable set of conventions for the preparation and interchange of publications for defense and non-defense use. MIL-M-28001B has identified applications that provide a more comprehensive specification for the interchange of ASCII data. Such applications include the specifications for an Output Specification, Electronic Review, and Partial Document delivery. These specifications represent the chief enhancements of the MIL-M-28001 specification achieved in its "B" version. They are briefly discussed below.

1.1.3.3.1  Output Specification

In order to format an SGML source file, associated formatting information must be provided. This associated formatting information must define formatting characteristics such as a page model, font and family characteristics, point size, indenting, etc. In addition, these formatting characteristics must be responsive to certain SGML tags. For example, a "paragraph" tag may trigger a change in the line leading, or a "chapter title" tag may trigger "bolding" and "center" functions. The Electronic Publishing Committee of the CALS Industry Standards Working Group formed a "MIL-M-28001 Output Specification Ad Hoc Group" to develop a standard language for providing the associated formatting information of SGML instances. It was decided to use SGML itself for this purpose and provide the associated formatting information in the form of an OS DTD. Appendix B, Section 50 of MIL-M-28001B contains the CALS paper medium OS DTD.

The OS DTD defines a finite set of formatting characteristics used to rigorously describe the composition processing functions to be performed with respect to the tags of a SGML source file. A FOSI is an instance of the OS DTD. The FOSI defines values for the formatting characteristics defined in the OS DTD for every SGML element used in the document DTD, taking into account every context in which the SGML element has a unique formatting requirement. For example, a title of a Technical Manual (TM) chapter is formatted differently than a title of a TM subparagraph. The objective of the FOSI is to rigorously define the format style of the document to be produced from the SGML tagged source file, as required by the appropriate functional specification (MIL-M-38784C, etc.).

A FOSI should be developed for each DTD to describe all default formatting characteristics necessary to compose and publish a document authored according to that DTD. The FOSI should be delivered with the SGML source tagged file. Because all FOSIs will be written with respect to the standard OS (paper medium), vendors will be able to develop software that can accept and process FOSIs and interface with the publishing software. However, such automatic processing of a FOSI is not a requirement of MIL-M-28001B.

1.1.3.3.2  Electronic Review

Section 80 of Appendix C of MIL-M-28001B provides a mechanism that enables an electronic review and comment capability for SGML tagged source files. This capability allows reviewers located in diverse environments to make and exchange comments electronically on multiple copies of a document file over a network. The comments may then be sorted, processed, and incorporated into the document by the file "owner."

The mechanism for electronic review of SGML tagged source files consists of certain SGML constructs that are incorporated into a DTD for a given document type. These SGML constructs have been defined as generally as possible to take into account the many kinds of reviews such as internal contractor reviews, Government reviews, contractor/Government reviews, specification reviews, etc.

Plans for future extensions of electronic review include both a CALS graphics comment capability using SGML tagged for the comments, and a capability to link SGML text and CALS graphics files for related changes. Efforts will also be made to develop a more precise addressing mechanism for indicating the location within document elements of a proposed change.

1.1.3.3.3  Partial Documents

Partial document delivery is used to transmit source SGML tagged source data either as an interim deliverable or as an update package containing data for a document that has been previously delivered. Its purpose is to minimize the retransmittal of unchanged data or to indicate incomplete data. Partial document delivery is not intended to address the issues of page integrity or fidelity, nor is it intended to include specific change pages. The intent of this methodology is to allow the delivery of certain portions of a source document such that the receiving system can identify the location of the information in the original document and perform the appropriate addition, deletion, or replacement operations. Both the manner in which this is accomplished and the effect of the change on composition depends on the receiving system.

1.1.3.3.4  CALS SGML Registry/CALS SGML Library

One of the Ad Hoc Groups of the Electronic Publishing Committee is investigating the requirements for the development and maintenance of a CALS SGML Library (CSL).

It is envisioned that a CALS SGML Registrar will administer the CALS SGML Registry (CSR), a central registry office where DTDs and FOSIs will be registered. The CSR will maintain a CALS SGML Library that will be an on-line database containing all SGML elements and attributes that have been defined, with cross references to DTDs and governing military specifications. It is anticipated that all CSR/CSL requirements will be specified in a future revision of MIL-M-28001.

The CALS SGML Registry will require adherence to basic guidelines for acceptance of SGML tags/attributes. Some of these guidelines are:

1.1.3.4  Advantages

MIL-M-28001B provides SGML applications or "conventions" that can be applied in the automation of document production and management. These applications include the Output Specification, Electronic Review, Partial Document Delivery, and the "template" DTD tagset of elements and attributes. These innovations that are oriented toward DoD and industry needs will provide a comprehensive specification for the interchange of ASCII data. They constitute both a far-sighted adaptation and a wide-ranging application of ISO 8879 to the rapidly changing technology of the printing and publishing industry.

1.1.3.5  Implementation Issues

MIL-M-28001 has undergone extensive revisions since its initial publication on February 26, 1988. MIL-M-28001B contains various specifications and/or recommendations for the interchange of data. Some of these have not been thoroughly tested, including:

Currently, there are no tests for vendor products claiming conformance to MIL-M-28001. Such MIL-M-28001 product conformance testing will depend upon the product's function. For instance, conformance testing of SGML parsers entails the correct interpretation of ISO 8879. Conformance testing of "auto-taggers" or "authoring stations" would be limited to determining the parseability of the instances generated, and again would involve correct ISO 8879 interpretation. With few exceptions, there is no disagreement regarding the correct interpretation of ISO 8879.

However, conformance testing of CALS SGML publishing systems involves MIL-M-28001 compliance, but MIL-M-28001 does not rigorously define system requirements. For example, while MIL-M-28001 specifies the requirements of a FOSI, it does not require a system to automatically process such a FOSI. Therefore, MIL-M-28001B conformance testing of a publishing system would likely be limited to tests for CALS data acceptance and valid document formatting.

1.1.3.6  Extent and Nature of User and Vendor Support

The vendor community is aware of the evolving nature of MIL-M-28001. Some vendors are waiting until the standard is finalized, while other vendors are undertaking full implementations at the present time. A large vendor community is represented on the Electronic Publishing Committee. For the CALS environment, vendors supporting MIL-M-28001 should not "hard-code" their systems to process only a single DTD or FOSI. Certainly, most users will be processing a variety of technical publications that must conform to multiple DTDs and will require a system that can be configured to adapt to new and changing requirements as they arise.

Currently, there are various types of SGML software products on the

market. These include:

1.1.4  MIL-R-28002B - Requirements for Raster Graphics Representation in Binary Format (Raster)

The MIL-R-28002 specification establishes requirements for a standard interchange file format and raster encoding scheme for raster data. MIL-R-28002 (Type I and Type II as defined in NISTIR 88-4017) was first issued in December 1988. It was revised by MIL-R-28002A in November of 1990 and again by MIL-R-28002B, December 14, 1992. This specification identifies the requirements to be met when raster data represented in digital, binary format is delivered to the Government.

Raster graphics involves the digital processing, storage, exchange, and reproduction of images. This technology supports the binary representation of a two-dimensional image as an array of picture elements, also known as pels. Each pel of the array contains information about that portion of the image. This information may include its lightness, darkness, gray-level, and color. The quality of the image depends directly on the density of pels within the array, also known as resolution density or pel transmission density. A high resolution density provides a high quality image, but at the expense of higher storage costs. Data compression, in which an encoding scheme is used to represent redundant data bits of information, can alleviate this storage problem to some extent. MIL-R-28002 restricts such compression to Group 4 encoding as defined in Consultative Committee on International Telephone and Telegraph (CCITT) Recommendation T.6 (Federal Information Processing Standard (FIPS) PUB 150) in order to conform with developing industry standards.

MIL-R-28002 permits two types of digital representation of raster data, called Type I and Type II in the specification. The Type I file format is used for raster data contained in a single monolithic block of compressed data and is called untiled raster data. The Type II file format is an Open Document Architecture (ODA) document (as specified by ISO 8613 ODA) conforming to a special application profile for raster. Type II may be tiled raster data or may consist of a single compressed block of data as in Type I, but with all ODA parameters and data structuring included.

Tiled raster data consists of an image that is subdivided into non-overlapping regions known as tiles where each tile is treated as a separate pel array. This method is especially useful for mechanical drawings in which there are large open areas of space. Within a single image, tiles are equal in size, and their dimensions, specified in terms of pels, have certain limitations. Tiles can be compressed and manipulated to obtain an optimal raster file. MIL-R-28002 specifies that individual tiles be digitized and the data compressed in accordance with Group 4 encoding defined in Recommendation T.6 (FIPS PUB 150).

Type I raster data interchange is intended to be used in procuring data for systems that use only untiled raster data representations. Examples of such systems include typical technical documentation systems, the Air Force Engineering Data Computer-Assisted Retrieval System (EDCARS) and the Army Digital Storage and Retrieval Engineering Data System (DSREDS). A set of graphics attributes specifying the details necessary for processing and reproducing the image must be included in a header record at the beginning of the raster file. These attributes include the size of the original image, the scanning resolution, the image orientation (portrait or landscape), the starting position on the page, and the spacing between the pels and also between the lines containing the pels. These attributes are used in reproducing the image and apply to both Type I and Type II raster data files.

Type II raster data interchange is intended to be used in procuring data for systems that need the flexibility to use tiled or a mixture of tiled and untiled raster data representations. Tiled representations are best applied in systems handling large format drawings or illustrations typically associated with engineering design. The subdivision of a drawing into tiles allows the use of only those portions of an image required at a given time by the application. This can result in reduced requirements for workstation memory and display. The attributes required for Type I are also required for Type II data and are encoded in the ODA data stream as specified by the ODA Raster DAP (explained below). For Type II data, additional attribute information must be included to cover the size of each tile, the number of tiles in the array (image), the method of tile ordering, and the method of tile coding. This information is stored in the header record of an image file during the scanning process and is essential for reproducing the image.

1.1.4.1  Typical Applications

MIL-R-28002 was created for the storage and interchange of scanned engineering drawings but applies to other documents as well, such as technical manuals and illustration in raster form. Appendix A of MIL-R-28002B contains the ODA Raster Document Application Profile (DAP). The DAP specifies an interchange format suitable for transfer of structured documents between equipment designed for raster processing. The documents supported by the Raster DAP are based on the paradigm of an electronic drawing or illustration. Such documents contain one or more pages. Each page consists of an image in the form of a bi-tonal raster graphic content. The features of a document that can be interchanged using the Raster DAP fall into the following categories:

1.1.4.2  Architecture

MIL-R-28002 identifies the requirements to be met when raster data represented in digital, binary format is delivered to the Federal Government. This specification identifies the storage and transmission format of raster data and tiling conventions for document pages and large format engineering drawings interchanged as raster images. All digital raster data files complying with MIL-R-28002 shall conform to either the Type I or Type II binary formats defined in the specification.

As specified in MIL-STD-1840, a set of graphics attributes specifying the details necessary for processing and reproducing the image is contained in a header record at the beginning of a raster file. These attributes include an indication of the raster data type, the size of the original image, the scanning resolution, the image orientation (whether it be portrait or landscape), the spacing between the pels, the spacing between the lines containing the pels, and the bit ordering. These attributes are used in reproducing the image and apply to both the Type I and Type II raster data formats.

Type I data is CCITT T.6 encoded data for an entire scan representation enclosed within MIL-STD-1840 header information. The CCITT T.6 encoding of raster data is defined in FIPS PUB 150, "Telecommunications:  Facsimile Coding Schemes and Coding Control Functions for Group 4 Facsimile Apparatus." Type I has no support for tiling, but has the virtue of simplicity.

Type II data is a MIL-STD-1840 header wrapped around an ODA document as specified in the ODA Raster DAP. The ODA document may consist of a single compressed block of data as in Type I, or it may be tiled. ODA parameters and data structuring must be included.

1.1.4.3  Status and Planned Extensions

MIL-R-28002B was published in December 1992. As a result of international harmonization activities, changes were made to the March 1993 version of the ODA Raster DAP. The "application comments" attribute will now consist of two fields to be compatible with other International Profiles, and the value for the "ODA version" attribute was changed. DoD decided that these changes were significant enough to warrant an amendment to MIL-R-28002B. The amendment will contain changes to Appendix A (the ODA Raster DAP) and is not expected to impact the MIL-R-28002B implementation of the DAP. Therefore, current implementations of MIL-R-28002B will be upwardly compatible with the amendment. This amendment is expected before the end of the 1993 fiscal year.

NIST is planning to issue the ODA Raster DAP as a FIPS PUB by the end of 1994. Once the FIPS is in place, MIL-R-28002 will no longer contain the ODA Raster DAP as an appendix, but will reference the appropriate FIPS.

1.1.4.4  Implementation Issues

The development of Type I data capabilities has been evolving for some time and has stabilized. The CTN digital raster data interchange testing for Type I data has aided many present and potential DoD contractors in their efforts to develop hardware and software that are technically capable of accomplishing MIL-R-28002 Type I interchanges. In particular, the CTN Engineering Data Transfer Test with the Engineering Data Management Information and Control System (Navy) (EDMICS) using MIL-R-28002 (May 1992) reported that MIL-STD-1840A tapes generated by EDCARS and DSREDS were successfully converted from CALS (MIL-R-28002) to EDMICS native format and visually displayed on EDMICS. In principle, this demonstration supported the viability of tri-service data interoperability via CALS media. CTN has published several test reports that describe Type I testing results for a variety of vendor implementations. NIST has begun Type I conformance test development.

Type II data, on the other hand, is a considerably newer and more complex environment. ODA implementors maintain that a detailed understanding of the ASN.1 Basic Encoding Rules (required by ODA) is not required to understand MIL-R-28002B Type II. A vendor or user need only implement the Raster DAP, not the entire ODA standard, to achieve a MIL-R-28002 Type II implementation. Users may employ a library of ASN.1 routines to avoid having to fully understand encodings at the bit or byte level. The (NIST) report, NISTIR 5108, "Raster Graphics:  A Tutorial and Implementation Guide," is an excellent resource for those needing a better understanding of MIL-R-28002 and how to implement it.

A system that can receive (read) and output (write) MIL-R-28002B Type I and Type II data will be considered a compliant CALS implementation. The internal raster data format of the system need not use ODA. Cost comparisons between the implementation of data translators verses designing the system's internal format based on ODA should be performed to determine what is the best MIL-R-28002B (Type II) implementation strategy for each Navy raster system.

The CTN Raster Test Bed at the Lawrence Livermore National Laboratory (LLNL) and David Taylor Model Basin, Carderock Division of the Naval Surface Warfare Center (CDNSWC) have been involved with the testing and validation of raster image data provided by Industry and DoD. InterLinear Technology, in support of LLNL, has developed a Test tool called "ODATOOL" to evaluate CALS raster image data formatted according to the MIL-R-28002A. CTN has tasked InterLinear to update the ODATOOL in accordance with MIL-R-28002B. InterLinear offers commercial products to write, convert, and read ODA files. At present, only the Modular Electronic Document Information Solution (MEDIS) systems by InterLinear Technology support MIL-R-28002 Type I and II.

1.1.5  MIL-D-28003A - Digital Representation for Communication of Illustration Data:  CGM Application Profile (CGM)

MIL-D-28003A is a proposed military specification defining an application profile for delivery of two-dimensional technical publication illustrations using the CGM standard. CGM application profile requirements defined by MIL-D-28003 address the first of several classes of conforming basic meta-files, a conforming basic metafile generator, and both a minimum-level and a publication-level conforming basic metafile interpreter. Generator and interpreter requirements are specified to provide necessary controls over the creation and validation of conforming metafiles and to remove implementation dependencies. The metafile definition itself includes the allowable output primitives and attributes that may be used to compose the illustration. Although the CGM standards that MIL-D-28003 implements define a complete and unambiguous syntax, they intentionally under specify the semantics, or meaning, of the CGM primitives and attributes to accommodate a wider range of existing systems, and to make the standard more adaptable to the requirements of diverse applications and users. The CALS application profile provides an unambiguous interpretation of those semantics appropriate for illustration of technical publications.

The Computer Graphics Metafile standard is a published international standard (International Organization for Standardization/International Electrotechnical Commission (ISO/IEC) 8632), a Federal Information Processing Standard (FIPS 128), and has been adopted by the American National Standard Institute (ANSI). The CGM standard is being developed and maintained through a coordinated effort of ISO SC24 and ANSI X3H3. The U.S. and international standards are identical.

The overall intent of the CGM standard is to provide the lowest level of drawing functionality required to capture and reproduce a picture, in a way that is portable across applications and devices. The CGM standard specifies two-dimensional graphics data interchange in a file format that can be created independently of device requirements and translated into formats needed by specific output devices, graphics systems, and computer systems.

A metafile is a device-independent, application-independent storage format for the exchange of the data that makes up a picture ("picture data"). The metafile definition in ISO/IEC 8632 includes a definition of output primitives and attributes that may be used to compose an illustration, but in an intentionally under-specified semantics (meaning). This was done to accommodate a wide range of existing systems, and to make the standard more adaptable to the requirements of diverse applications and users. The application software wishes to store a picture on a metafile. To do this, it has to write all the information required to a file. The software that performs these writing actions is called the generator. The software that reads a metafile back into an application is called an interpreter.

A profile addresses implementation conformance requirements for the generator and interpreter. For generators, the graphical characteristics of the picture are mapped onto a set of CGM elements that define the pictures within the accuracy and latitude defined by the implementation requirements in the profile. For interpreters, the graphical characteristics of the CGM elements are rendered into a graphical image or picture within the latitude defined by the implementation requirements in the profile. Without a profile, one can address only the syntactical correctness of the data stream. With a profile, one can address and test that the picture is correct. A profile provides a way of standardizing and publicly specifying the options that a vendor supports and how they are to be supported.

The three CGM encodings meet different needs, but all may be interchanged without loss of information. The binary encoding facilitates rapid graphic data processing. The character encoding is compact and transportable. The clear text encoding is human readable and editable. Only the binary format is approved for use by MIL-D-28003.

ISO/IEC 8632 CGM is an upwardly compatible standard format, developed in three versions that offer steps in capability. Version 1 includes elements of ISO 8632 CGM:1987. Version 2 is also based on ISO 8632:1987, but adds capabilities through Amendment 1:1990. Version 3 incorporates ISO 8632:1987, Amendment 1 and Amendment 3:1991. This has resulted in major increases in capabilities that are represented by ISO/IEC 8632:1992, the current version of the CGM standard. ISO/IEC 8632:1992 provides a clear definition of Version 1, 2, and 3 metafiles.

The Military specification, MIL-D-28003, "Digital Representation of Illustration Data:  Computer Graphics Metafile," first published December 20, 1988, was superseded by MIL-D-28003A, published November 15, 1991. Amendment 1 to MIL-D-28003A was published August 14, 1992.

MIL-D-28003 is the CALS Application Profile for CGM, developed to provide more information and requirements for specific application areas that relate to the CALS environment. It is designed for a multi-system environment that involves a wide range of applications. It is also designed to increase the conformance requirements for CGM, to guarantee final results when the pictures are drawn. All CALS CGMs conform to ISO/IEC 8632. It is important to note that CALS applications that use CGM conform to the MIL-D-28003 Application Profile, and not simply to the CGM standard.

1.1.5.1  Typical Applications

MIL-D-28003A is intended for use in computer graphics applications in the following situations:

FIPS 128 in conjunction with MIL-D-28003 should be used when the representation of graphical information in digital form is to be used in technical illustrations and publications, and when the use of a general-purpose, graphical interchange mechanism is required. The current version of FIPS 128 is FIPS 128-1, May 11, 1993. This FIPS adopts the redesignated version of the CGM standard known as ANSI/ISO 8632.1-4:1992.

ISO/IEC 8632 is the recommended standard to:

1.1.5.2  Architecture

The CGM standard (ISO 8632-1987) (FIPS PUB 128), "Computer Graphics - Metafile for the Storage and Transfer of Picture Description Information" is composed of 4 parts. MIL-D-28003A utilizes Part 1 and Part 3 of the standard's four-part architecture.

1.1.5.3  Status and Planned Extensions

Work is in progress to produce several International Standardization Profiles (ISPs) for CGM. The profiles will be based on ISO 8632:1992 and Amendment 1 which specifies profile rules and a model profile. It is proposed that there will be three profiles based on current profiles (including MIL-D-28003 the CALS CGM AP), which range from basic scientific and technical graphics to advanced presentation and visualization.

The Industry Steering Group/Drawing and Graphics Committee is proposing an Amendment 2 to MIL-D-28003A that will include metafile descriptions, clarification of restricted text, order of precedence, and rules for profiles. The overall intent of this amendment is to permit the majority of CGM software procured under MIL-D-28003 to meet the requirements of MIL-D-28003A.

ISO/IEC 8632 CGM:1992 Amendment 1 defines rules for writing profiles. This amendment includes rules for profiles, a model profile, and conformance requirements. ISO/IEC 8632 CGM:1992 Amendment 1 has been approved and will be published in late 1994. It contains rules information that will be used to revise the CALS Application Profile (MIL-D-28003) for CGM. The "Model Profile" of Amendment 1 is similar to MIL-D-28003A. The revision of 28003A, probably MIL-D-28003B, would be upward compatible with 28003A.

ISO/IEC 8632 CGM:1992 Amendment 2 Application Structuring contains structure and intelligent graphics information. Amendment 2 has been given a Draft International Profile status and is being balloted over a six-month period. This amendment contains only change pages; therefore, one should have the full standard in order to use and understand the amendment.

The latest, 1992, edition of the international standard for CGM provides critical capabilities for the CALS environment, which include:

The purpose of this edition of the standard is to extend CGM to fulfill requirements of engineering drawings, the preparation of graphic arts quality presentation materials, cartography, and technical publishing. To a large degree, this work was directly in response to CALS requirements.

Of particular interest to the CALS environment is the work under way within the CGM standards organization (X3H3) to provide "intelligent graphics." "Intelligent Graphics" is defined as adding information to graphics that is not graphical information, but that attaches application intelligence or semantics to the graphics. Requirements are associations of comments with graphics elements, associations of comments with element groups (hierarchical), native format editing, version control, and text-to-graphics links. This requirement was originally introduced by the "electronic Review" work of the CALS ISG Electronic Publications Committee, where SGML-tagged documents are used to provide a commenting capability. The CGM additions will allow for SGML tags to be attached to objects within the CGM file.

Possible future extensions to the international standard that could be of considerable interest to CALS include the formulation of an object-structured grammar. This has been requested from, and will be of major use to, CGM users in commercial aviation (intelligent graphics); CALS electronic review (review comments in graphics and stronger links to text); and hypermedia (smart objects in graphics databases).

1.1.5.4  Advantages

The CGM contains device-independent, digitally-encoded vector and raster graphics data. CGM files are easily transferred and displayed on a wide variety of hardcopy devices (dot-matrix, ink-jet, electrostatic, and laser printers, pen plotters, and film cameras). CGM files can also be easily previewed on an extensive range of softcopy terminals. In comparison to Raster, CGM is easily modifiable, generally of much smaller size, and not dependent upon resolution of the output device. In comparison to IGES (2D data), CGM is faster to interpret and display, and again more compact. The selection of which of the CALS graphic standards (raster, IGES, or CGM) that best fits the application, should be the result of the thorough examination of the processes involved in the application.

1.1.5.5  Implementation Issues

Although MIL-D-28003 has been available for some time, vendors have not fully implemented it, thereby causing confusion with users. Vendors who state they are "CGM-conforming" may not be MIL-D-28003 conforming. Some vendors actually have a problem with importing and exporting the same image; if a CGM file is imported and then immediately exported, it is changed. In other cases, CGM generators provide functionally satisfactory CGMs, but fail to provide the required identification elements.

Standards testing, conformance testing, and interoperability testing are essential steps towards achieving successful interoperability. Standards testing examines the design, construction, and details of the standard, and tests to see if it is correct and well-defined. Conformance testing is a way of determining if a CGM product or metafile correctly implements the CGM standard and MIL-D-28003A the CALS CGM AP. Interoperability testing demonstrates that a given pair of systems (or products) can interoperate. In other words, standards testing assures that the standard is well-defined; conformance testing ensures that the product is "built correctly"; and interoperability testing ensures that there are no hidden problems in system interfaces resulting from laxity in defining the specification.

Standards testing and a small part of interoperability testing have been assigned to the CTN. The CTN tests and evaluates the effectiveness of the CALS standards and specifications. CTN also tests data interchange files for compliance with the CALS specifications. This is, in a small sense, a kind of interoperability test of the standard, but it is in no way a full interoperability test. No organization is performing full interoperability testing of MIL-D-28003 in the sense of testing interoperability between pairs of systems. Conformance testing for MIL-D-28003, on the other hand, has been formally assigned to the NIST.

NIST offers three CGM Test services; metafile testing, generator testing, and interpreter testing. The purpose of the test services is to determine the degree to which the metafile, generator, or interpreter conforms to FIPS 128-1 and MIL-D-28003A. A certificate of validation is issued for metafiles, generator implementations, or interpreter implementations that have been tested and are in compliance with FIPS 128-1 and MIL-D-28003A. Both the generator and interpreter test suites are available as part of the validation service or may be purchased separately. NIST provides the validation test service on a cost-reimbursable basis.

Metafiles should be tested when a specific picture or set of illustrations is being acquired. Conformance of a metafile does not necessarily imply conformance of the generator or interpreter. Generators should be tested to ensure that only correct metafiles are proposed. If the generator has been certified, then it is not necessary to individually test metafiles created by the generator product. Interpreters should be tested to ensure that any conforming CGM can be read and produce the correct picture.

The CTN CGM Test Packet consists of several computer graphics metafiles together with procedures for conducting a CGM interpreter test and for evaluating the resultant images. The Air Force CALS Test Network Test Bed at Lawrence Livermore National Laboratory has been testing Version 1 CGM interchange for several years, but has only recently acquired a CGM generator library that will permit the construction of Version 2 and Version 3 CGMs.

The Application Profile provides common usage across a user community, in this case the community CALS and CALS-related organizations. An advantage of having profiles is that profiles provide a way of standardizing and publicly specifying the options that are supported and how they have been implemented. Another advantage of Application Profiles is that profiles address implementation conformance requirements on generators and interpreters. Without a profile you cannot test the generator or interpreter, which is a step toward assuring successful interchange.

1.2  Summary

This report will provide the results a survey of existing COTS, GOTS and public domain test and validation tools for the above stated standards. Criteria for the qualification of the tools have been established with underlying principles governing reusable software assets. The tools will be analyzed and evaluated against domain specific and general reuse criteria to arrive at a comprehensive set that will be present in a repository of CALS test and validation tools. This report gives a complete view of the tools, their descriptions, their functionality, the availability, and the results of the evaluation against the common and domain-specific reuse criteria established.

Testing tools are a critical component of the CALS process, because they ensure that only those components that meet the CALS guidelines are able to enter the system. This directly supports the implementation of CALS and attainment of key CALS needs. The ability to quickly confirm or deny the compliance and usability of deliverables in digital data format drives directly at the worth of the tool. It also establishes the credibility of the CALS test and validation tools repository. Considering the potential volume of digitized data and reuse of such data by the CALS community, it is imperative that vendor and Government-supplied digital data conform to applicable standards. This conformity must be obtained in language, graphics, and applicable type-of-weapon defense platforms, as well as in the CALS 28000 series specifications and MIL-STD-1840.

In short, the CALS test and validation tools repository must be accessible for the CALS community to conform and validate vendor deliverables. Those deliverables that fail to meet compliance standards will not be permitted to enter the proposed IWSDB. Additionally, both the Government and the user community must be assured that the test results using the tool or tools are valid and accurate.

As a preliminary exercise in conducting our survey of existing CALS test and validation tools, the work performed under the CTN and the NIST was reviewed. The CTN is a DoD-sponsored confederation of voluntary participants from industry and Government. The primary objective of the CTN is to evaluate the effectiveness of the CALS standards (specifications) for technical information (data) exchange. The CTN is utilized to demonstrate the technical capabilities and operational characteristics of each emerging standard in conjunction with software products (COTS) that claim to support and conform to each standard.

Because standards for technical data interchange are one of the key enablers of CALS objectives, one must address the meaning of "conforming" to the standard. Products marketed that claim to conform to the CALS Standard or to be CALS compliant should be subject to a formal testing scenario that would assure the marketplace and public and private sectors that the standard is in fact being met. Companies offer COTS products that claim to perform that validation. Commercial/public test tools are programs that are used to test data formats versus a known standard or to execute test suites to validate that the CALS COTS products are producing data that conforms to the standard.

2.0  OVERVIEW OF TESTING TOOLS

      

In the following sections, we will look into the test and validation tools that have been identified from our survey. In essence, the tools considered will be tools that ensure the conformance to the respective standard. Analysis regarding complimentary software and ancillary tools that do not perform conformance checks for the standard will be minimal, as they do not provide the necessary function of ensuring conformance to the standard that is necessary for an entity to enter an Integrated Weapon Systems DataBase. These tools, nonetheless, have a function in aiding the tools qualification team responsible for the maintenance of the weapon systems database. They provide a means to cross check the results of and ensure the "degree of confidence" in entering the asset into the database system.

2.1  MIL-STD-1840B Testing

MIL-STD-1840B is the "parent" CALS standard. It provides the rules for organizing files of digital data that conform to the respective 28000 series standard. MIL-STD-1840B specifies CALS delivery, format, and organization including header file requirements, data file types, etc. This standard is applied wherever information is exchanged. There are no restrictions to its application. It can be used for the exchange of American Standard Code for Information Interchange (ASCII) text files, product definition data files, raster image files, or vector graphics files. Table 2.1-1 represents a list of identified tools used for checking conformance to MIL-STD-1840B.

TapeTool, available free of charge from CTN, is the most widely used tool for the validation of data that is exchanged according to the MIL-STD-1840 standard. CTN claims, based on a survey of the CALS community, that it is impossible to validate MIL-STD-1840 data properly with any other tool. In October 1993, Version 1.2.10 of TapeTool was released. With this revision of the tool, more data that is valid is passing the validation process. Before this revision, some data that was valid according to MIL-STD-1840 was being rejected. TapeTool v1.2.10 complies with MIL-STD-1840A. A beta version of TapeTool that complies with MIL-STD-1840B is now available through CTN. The common criteria and domain-specific evaluation for TapeTool Version 1.2.10 can be reviewed in Appendix A under 1840B evaluations.

This tool is for handling 9-track tape data in accordance with the MIL-STD-1840 standard. This tool could not be evaluated against our reuse criteria because an evaluation copy was never received.

This version of TapeTool was specifically written for Texas Instruments and is not as complete as the CTN TapeTool. This version was found to have software bugs and hence is not recommended for use. It does not provide the functionality and quality of testing that is provided by the CTN TapeTool. Please note that the evaluation for the TI TapeTool is encompassed by the evaluations of the CTN TapeTool.

2.2  IGES Testing Tools

MIL-D-28000A This specification defines the CALS compliant subsets of the IGES for the interchange of vector graphics. IGES testing is being used to improve the accuracy of data interchange. Errors in the data exchange can require extensive re-work by the CAD system user at the receiving system to portions of the CAD file to correct the translation errors. By identifying those errors in the test cycle, the sender information can correct the errors or identify alternate path. Organizations that extensively use data exchange can experience productivity gains by eliminating much of this re-work step. The two most common types of testing are loop testing and end-to-end testing. These are covered in detail in the "MIL-D-28000 Usage Guide" published by the Carderock Division, Naval Surface Warfare Center (formerly David Taylor Research Center) for the CALS Test Network, Wright-Patterson Air Force Base, Dayton, Ohio, on September 23,1993.

In our evaluation of the tools against the reuse criteria, we felt it necessary to evaluate all the tools from the survey with the common qualification criteria and restrict the domain specific reuse criteria to the Parsers/Verifiers in the IGES domain. This action is justified because the criteria established for domain specific reuse pertained to the specific evaluation of key conformance issues only in the domain. The specific qualification criteria can be found in the Final Criteria and Test Procedures Report.

To test IGES data for CALS conformance, a Parser/Verifier is used. In some cases, the file must be edited or reworked before using a Parser/Verifier. Therefore, tools for IGES testing include both Parser/Verifiers and Viewers. The following was divided in such classes. A list of MIL-D-28000 test and validation tools is shown in Table 2.2-1.

2.2.1  Tools (Parser/Verifiers)

IGES Data Analysis (IDA)

IDA offers a magnitude of software tools that claim to be CALS compliant. IDA's IGES Tool kit is composed of three intuitive and easy to use products for the identification, repair, and flavoring of IGES files. IGESVIEW is a graphic viewer for the display and manipulation of IGES files in their graphical form. The IGES Parser/Verifier is a detailed analysis utility for checking conformance to the IGES specification and to the CALS subsets. IGESXpert is a powerful IGES file editor for the examination, modification, and repair of IGES files. These products can be purchased separately or as an integrated package.

The IGES Tool kit offers a complete solution for testing and debugging high-end engineering applications. IDA is one of the companies that developed a Parser/Verifier that is now considered the industry standard for checking IGES file's conformance to the IGES specification. IGES Parser/Verifier performs a rigorous syntax, structure, and semantics check on an IGES file, producing several detailed file statistic, warning, and error message reports. The software performs two types of analysis. The first type checks the syntax and structure of the IGES file, while the second type checks the structure and semantics of the file. A CALS conformance option can be purchased for the Parser/Verifier, providing additional checking to verify an IGES file's conformance to the CALS IGES subset. Files can be checked against Class I, II, III, or IV as defined in MIL-D-28000A. CALS conformance checking is performed in conjunction with checking all versions of the IGES standard. Any errors are reported with the same detailed references to the original file.

The common criteria and domain specific evaluation for IDA's Parser/Verifier can be reviewed in Appendix A under IGES evaluations.

2.2.2  Viewers/Translators

Arbor Text, Inc.

Arbor Text, Inc. developed an advanced electronic publishing system designed to meet the publishing requirements of all phases of the CALS initiative. IGES2draw is an IGES viewer/translator that converts a Multi-drawing IGES file into Arbor Text IslandDraw format. Draw2IGES does just the opposite. It will convert Arbor Text IslandDraw format to IGES version 3 or 4.

Arbor Text will operate on a large variety of platforms such as SUN Sparc, DEC Alpha, IMB Risc, and just recently they released an MS Windows version.

The common criteria evaluation for Arbor Text, INC. IGES2draw/Draw2IG