1. JSF Modeling Information Management JSF Modeling Information Management NDIA Systems Engineering Conference San Diego, California October 2003 Ref: JSF03-03-0649 Jim Hollenbach Simulation Strategies, Inc. jimh@simstrat.comjimh@simstrat.com, 703.360.3902 JSFPO Technical Lead Modeling Information Sources Action Team W. Henson Graves Lockheed Martin Aeronautics Company henson.graves@lmco.comhenson.graves@lmco.com, 817.777.1856 LM Aero Technical Lead Modeling Information Sources Action Team DISTRIBUTION STATEMENT A: Approved for Public Release; Distribution is Unlimited.

2. 2 Simulation Based Acquisition (SBA): Shared Information is the Hub of IPPD JSF Program Commitment: Availability of complete consistent digital product data 24/7 access and navigation of digital product data by the entire team Data interoperability between team and suppliers Configuration management and publishing in-progress designs Accreditation and traceability of digital product data DISTRIBUTION STATEMENT A: Approved for Public Release; Distribution is Unlimited.

3. 3 Consistent, Authoritative and Readily-Available Information is Vital  To produce the JSF  Coordinate development activities, assess development progress  Reduce the confusion and risks that arise if operating on wrong or logically inconsistent information  Establish validity of M&S-based analyses by tracing results back to authoritative information  To conserve resources  Avoid having to repeatedly find, produce and/or translate the same information  Enable more timely analyses, resulting in shorter decision cycle times and more efficient systems engineering  Avoid the costs of mistakes that arise from bad information DISTRIBUTION STATEMENT A: Approved for Public Release; Distribution is Unlimited.

4. 4 JSF Simulation Based Acquisition Implementation Team SBA Implementation Council M&S User Action Team Functional needs & processes AnalysisTrainingTest M&S Tools Action Team Software applications Modeling Information Sources Action Team Authoritative data V&V Action Team Quality control/risk reduction Mission Systems M&S working group Associated work groups DISTRIBUTION STATEMENT A: Approved for Public Release; Distribution is Unlimited.

5. 5 Data Inputs Engineering Campaign Mission Engagement Context-independent data Externally or internally generated Primitive Aggregated Context-dependent data Internally derived DISTRIBUTION STATEMENT A: Approved for Public Release; Distribution is Unlimited.

6. 6 Data Lineage Example (Surface-to Air Missile Vulnerability Analysis - simplified) ENGAGEMENT MISSION CAMPAIGN ACQUIRE COVART FASTGEN HITRAN RCHEM SPIRITS ESAMS ASM DIADS BRAWLERSHAZAM MIL-AASPEM RADGUNS MOSAIC JSF Virtual Simulator JIMM GIANT EADSIM LCOMCFAM THUNDER ACAD CATIA NASTRAN PGEN 4 Suppressor WINFIRE AFMSS (CLOAR) Live Fire Test FLIR 92MODTRAN ENGINEERING DISTRIBUTION STATEMENT A: Approved for Public Release; Distribution is Unlimited.

7. 7 Inverse Abstraction Relationship Engineering level simulations Campaign level simulations Mission level simulations Engagement level simulations Context-dependent data item alternatives Degree of abstraction Number of instances  Data applicability narrows as context dependencies increase  Standard contexts reduce instances that must be managed  Include context in metadata to guard against inappropriate use DISTRIBUTION STATEMENT A: Approved for Public Release; Distribution is Unlimited.

8. 8 JSF Authoritative Modeling Information System (JAMIS) Information Flow Concept Version 16, 31 Jan 03 MNS ORD Concept Exploration & Definition Demonstration / Validation Engineering Manufacturing & Development Production & Deployment Upgrade and Replace nt PHASE 1000 100 10 MGMT Jan Apr Jul Oct Authoring tools (e.g., architecture, CAD) DSMs / relevant info in tool-specific form Study-specific input datasets & software code Analyst action Raw M&S results Threat & friendly systems [ASDB] Operational context Natural environment, infrastructure JSF [DPD] Interactions (P d, P k,etc.) Linking & translation services (mapping, semantics/syntax) Study- specific archives MNS ORD Concept Exploration & Definition Demonstration / Validation Engineering Manufacturing & Development Production & Deployment Upgrade and Replace nt PHASE 1000 100 10 MGMT Jan Apr Jul Oct Analysis tools (e.g., models, simulations) Register DT & OT test results LM Aero Team, Operational Test Authorities Resource Access System (RAS) NIMA, DTRA, NRO, etc. Program Offices Intel Centers Services, JFCOM, etc. Threat C&P information Operational context info Natural environment & infrastructure C&P info Blue system C&P info Register authoritative data (JSFPO) Color key: JSFPO provided LM Aero provided Design change MoMs JSF control boards Digital System Models Register smoothed / analyzed results User interface admin data JSF design change decisions Register design decision Design change orders Register proposed designs (LM Aero) DISTRIBUTION STATEMENT A: Approved for Public Release; Distribution is Unlimited.

9. 9 JAMIS Operation Roles and Responsibilities Version 12, 30 Jan 03 Review data & metadata Register data Author metadata View data & metadata Assess & mark (in metadata) JSF SORA & LM Aero Tier III IPT directors: Grant privileges Consumers : JSF Enter- prise, others TBD Analyses and testing Technology development JSF Authoritative Modeling Information System External references (e.g., Intel Centers) External references (e.g., Intel Centers) View data & metadata Download & use data Implement privileges Provide training and technical support Manager : LM Aero, in cooperation with JAMIS database managers Examiners : VV&A Action Team, others Producers : Designers, JSFPO, analysts, testers DISTRIBUTION STATEMENT A: Approved for Public Release; Distribution is Unlimited.

10. Blue systems Threat systems Natural environment Civil/military infrastructure Operational Context (scenarios, etc.) Government provided context information LM Aero provided Product Information Model (DPD) (Variants = layered instance values) Functional allocation trade studies Manufacturing/ supplier capabilities Mobility Specialized views: OML, Cost, BOM, mass properties, etc. Coherency & completeness cross-checks Solid lines = primary Dashed lines = secondary Transportation capabilities Assemblies & components Software Spatial & electrical relationships Connections Part records System/ subsystem topology Functional allocation Technologies Interfaces Air vehicle performance Mission system capabilities Autonomic logistics Signatures Reliability JSF Product Development Information Model Scope Lethality/ effectiveness Survivability Manufacturing process Physical architecture Support infrastructure Maintainability Performance Personnel rqmts Interactions Logical architecture (Requirements are recorded in appropriate information domain) DISTRIBUTION STATEMENT A: Approved for Public Release; Distribution is Unlimited.

11. 11 JAMIS Architecture – System View Information model as means to locate, navigate, and access distributed data Resource Access System Metaphase MASCOT PVCS Other databases Information Model (info taxonomy and decomposition, interrelationships) & data locations ASDB DOORS RAS application server Web Browser (RAS GUI) Database servers DISTRIBUTION STATEMENT A: Approved for Public Release; Distribution is Unlimited.

12. 12 Challenge: Information Mapping Professional disciplines, and M&S developers, represent (model) things in widely differing ways  Conceptual views (abstract ways of thinking about what’s involved) What entities, attributes, actions and interactions are represented, at what level of granularity, etc.  Contexts in which information is valid  Semantics and syntax of individual data items DISTRIBUTION STATEMENT A: Approved for Public Release; Distribution is Unlimited.

13. 13 Dimensions of Information Coherency Temporal coherence  The extent to which the data being used for different tools and sites has the same time stamp Organizational coherence  The extent to which all enterprise organizations are using the same set of databases Layer (tier, level, stratum) coherence  The extent to which all the information at a given level of granularity (e.g., engagement level) is logically consistent Perspective coherence  The extent to which information from different perspectives (e.g., logical, physical, behavioral) is logically consistent Abstraction coherence  The extent to which the information at different levels of granularity is logically consistent (the context dependency issue) Hard Very Hard DISTRIBUTION STATEMENT A: Approved for Public Release; Distribution is Unlimited.

14. 14 Development Strategy  Due to the absence of MLS and two-way guards, we are establishing separate JAMIS instances at each security classification level  Unclassified, SECRET Collateral, TOP SECRET SAR  Spiral development  A series of builds, prioritized per program needs, to support: An increasing amount of information An increasing numbers of tools Interfaced to an increasing number of organizations  Version 3 in development now DISTRIBUTION STATEMENT A: Approved for Public Release; Distribution is Unlimited.

15. 15 JPO TOP LEVEL SCHEDULE DISTRIBUTION A – Approved for Public Release; Distribution is Unlimited Version 8a – current as of 28 Feb 03 1 2 3 4 CY2002200320042005200620072008200920102011 Pre-SDDPre-SDDSDDSDD SDDSDD PAC ASRR IBR AS CDR 1 AS CDR 2 AS CDR 3 Blk 1 FTR Block 2 Blocks 0 & 1 Block 3 0A 0B 0C ISR Development and Qualification CTOLSTOVLCV Blk 1 RFT Blk 2 RFTBlk 3 RFT LRIP I (10) LRIP II (22) LRIP III (54) LRIP IV (90) LRIP V (117) LRIP VI (166) Multi Year Production 6 CTOL 4 STOVL 14 CTOL 8 STOVL 20 CTOL 20 STOVL 9 CV 5 UK 30 CTOL 32 STOVL 20 CV 8 UK DAE IPR 1 DAE IPR 2 DAE IPR 3 DAE IPR 4 DAE IPR 5 DAB Review ORD IOC USMC ORD IOC USN ORD IOC USAF LLFF LLFF LLFF LLFF LLFF LLFF PDR FETT AS PDR Blk 2 FTR Blk 3 FTR FY200220032004200520062007200820092010201120122013 CTOL/CV IFR OCR CTOL/CVSTOVL Autonomic Logistics Development Low Rate Initial Production (LRIP) Air System Program Events Propulsion Test and Development 44 CTOL 32 STOVL 32 CV 9 UK 72 CTOL 36 STOVL 48 CV 10 UK 1 2 2013 110 CTOL 36 STOVL 48 CV 12 UK CDR FETT STOVL F135 F136 1 2 3 4 2012 Mission & Vehicle Systems IBR CTOL Trng Sys PDR ALIS PDR TSys/ALIS CDR Blk 1 RFO Blk 2 Flt Test Blk 0 & 1 Flt Test Blk 3 Flt Test CTOL 1st Flt CV 1st Flt STOVL 1st Flt OT&E OA (4 CTOL, 2 STOVL) (4 CTOL, 4 STOVL, 4 CV) EOA OA Flt Test Bed MILCON Test Facilities (2 CTOL, 2 STOVL, 2 CV) CDR FETT IFR FETTPDRCDR ISROCR PDRIBR DISTRIBUTION STATEMENT A: Approved for Public Release; Distribution is Unlimited.

16. 16 A Few Challenges  Fostering broad understanding of the need and the solution concept  Integrating into the system engineering process  Finding authoritative data sources  Overcoming reluctance to release data  Replicating less classified data at higher levels  Providing lower level representations of highly- classified data  Accomplishing the data engineering DISTRIBUTION STATEMENT A: Approved for Public Release; Distribution is Unlimited.

17. 17 Discussion DISTRIBUTION STATEMENT A: Approved for Public Release; Distribution is Unlimited.

18. 18 Summary  Breaking new ground in management of modeling information  Information technology advances don’t solve all challenges  Holistic approach and good systems engineering are required  Promises to advance state of the practice, impact SBA concepts  Still a work in progress  Incremental deliveries, incremental changes for users  Wish had started sooner!  Many challenges common to other acquisition programs  DoD/Service-level actions could help DISTRIBUTION STATEMENT A: Approved for Public Release; Distribution is Unlimited.

19. 19 Information Types  Parametric data  Algorithms  Software code  Publications  Subject Matter Expert knowledge Initial focus has been on parametric data DISTRIBUTION STATEMENT A: Approved for Public Release; Distribution is Unlimited.

20. 20 Modeling Information Characterization  Regardless of its subject, data may be characterized by its:  Trustworthiness: “authoritative” or “non-authoritative”  Evidence in the real world: “empirical” or “derived”  Context dependency: “context-independent” or “context-dependent”  Granularity: “primitive” or “aggregated”  Structure: “atomic” or “complex”  Source (relative to the enterprise): “external” or “internal”  Above characterizations are orthogonal  A data item could be described by any combination DISTRIBUTION STATEMENT A: Approved for Public Release; Distribution is Unlimited.

21. 21 Operational Context DB Scope (Enterprise-wide standards for analysis; in priority order) 1. Scenarios  Road to war  Blue TPFD  Orders of battle (UOB, EOB, etc.)  Geographic area  Infrastructure targets, location 2. Mission Groups 3. Missions 4. Vignettes (created by JSFPO) 5. Use Cases (created by JSF IPTs)  Initiating inputs, required actions, MOPs 6. Natural environment conditions  Time of day, weather 7. Concept of Operations (CONOPS) 8. Force doctrine 9. Tactics 10. Rules of engagement (ROE) 11. Firing doctrine 12. Infrastructure lay down (other than targets) Not in Operational Context DB:  Study-specific archives Input data sets Raw execution outputs Discretionary analyst decisions –e.g., Blue mission routes  System C&P data  Natural environment instance data  Infrastructure C&P DISTRIBUTION STATEMENT A: Approved for Public Release; Distribution is Unlimited.

22. 22 JAMIS Architecture - System View PDM System Resource Access System Product Development Info Model + Storage Locations Metaphase PVCS Other DBs ASDB DOORS JDL DPD data DPD data DPD data Internet Explorer (RAS GUI) Other develop. data DPD data DPD data DISTRIBUTION STATEMENT A: Approved for Public Release; Distribution is Unlimited.

23. 23 Resource Access System The JSF Product Development Information Model provides a catalog structure for the DPD represented in the RAS interface Metaphase PVCS Other DBs DOORS JDL DPD data DPD data DPD data DPD data DPD data System View The PDM System (PDMS) is a federated set of data management systems for development and release control of product data Product Development Information Model The Resource Access System provides 24/7 Web-access to a single, logically unified JSF representation of variants, and configurations The DPD is the release controlled product data contained in the Product Data Management System PDMS Product Data currently in JDL will transition to Metaphase ASDB DISTRIBUTION STATEMENT A: Approved for Public Release; Distribution is Unlimited.

24. 24 Impact Analysis and Consistency DI 1 PE 1 DI 2 DI 3 DI 6 PE 3 DI 5 PE 4 DI 8 DI 7 PE 2 DI 4 Impact Analysis: A change in D1 potentially necessitates the re- execution of PE1 and PE3 to produce DI5 and DI8. DI 1 PE 1 DI 2 DI 3 DI 6 PE 3 DI 5 PE 4 DI 8 DI 7 PE 2 DI 4 Consistency: A consistent version of DI3 must be used to eventually produce a coherent version of DI8. DI 1 PE 1 DI 2 DI 3 DI 6 PE 3 DI 5 PE 4 DI 8 DI 7 PE 2 DI 4 DI – Data Item PE – Process Execution DISTRIBUTION STATEMENT A: Approved for Public Release; Distribution is Unlimited.

25. 25 Rationale for a Integrated Information Model Data source 1 Data source n Data source 2 Data source 3 Model format 1 Model format 3 Model format 2 Model format n...... Data source 1 Data source n Data source 2 Data source 3 Model format 1 Model format 3 Model format 2 Model format n...... Integrated Info Model (common semantics & syntax) Many translators (~N 2 for N tools) Difficult configuration management Fewer, less fragile translators Easier configuration management DISTRIBUTION STATEMENT A: Approved for Public Release; Distribution is Unlimited.

26. 26 Practical Limitations  REDIS process works in domains with high semantic similarities  Similar ways of looking at the world  Synthesizing an integrated information model requires an intimate understanding of the producing and consuming tools (models, simulations)  Usually a time-consuming task  Conceptual model & assumptions of the tools are often unclear  Developer may not have documented this  May be a function of the analysis to be performed Automatic translation of source data into input data sets has practical limits Automatic translation of source data into input data sets has practical limits - Much responsibility must be left with the analyst - Much responsibility must be left with the analyst DISTRIBUTION STATEMENT A: Approved for Public Release; Distribution is Unlimited.

27. 27 Information Model is Maintained as a UML Class Model Database Server RAS Server RAS Client Data Views Data Base Connectors Data formatting driven By XSLT rules UML Class Model XML Schemas & XSLT rules Utility program …which is used to generate XML schemas that are loaded into RAS DISTRIBUTION STATEMENT A: Approved for Public Release; Distribution is Unlimited.

28. 28 Recommendations: DoD Data Policy  Information is a DoD corporate resource  Establish means to capture data needs  similar to Intel Center ‘Production Requests’  Crisply define production “swim lanes” & responsible organizations  Government data rights for all reusable data (contracting policy)  Business/funding rules  Report cards  Define user responsibilities and liabilities  Establish means to appeal data release denials  Provide visibility/awareness means  Internet and SIPRNet registries of core metadata about the data assets, not the data itself Database description, format, security classification, date, POC, etc. DISTRIBUTION STATEMENT A: Approved for Public Release; Distribution is Unlimited.

29. 29 Recommendations: Standards  Need tool standards, not standard tools!  Tool standardization is impractical, costly, negates power of marketplace  Define standard data interchange formats (DIFs)  Need more, better DIF standards (at the ‘payload’ level) Well-defined semantics & syntax Standard metadata, including context dependencies More granular, less overlapping  Need DoD to use its ‘bully pulpit’, be engaged  Define DoD-sanctioned venues for standards development Commercial standards landscape is quite confused, both nationally and internationally NIST can help  Define full range of standard operational contexts  Reduce context-dependent data alternatives to a manageable level  Start with Design Reference Missions for each mission area, flesh out down to tactics DISTRIBUTION STATEMENT A: Approved for Public Release; Distribution is Unlimited.

30. 30 Three JAMIS Architectural Views Operational View JSF team interactions for digital product design and assessment Systems View An integrated set of information systems and supporting applications Technical View Web information services and standards JAMIS DISTRIBUTION STATEMENT A: Approved for Public Release; Distribution is Unlimited.

31. 31 Sources of this Requirement  JSF Modeling and Simulation Support Plan (MSSP)  a government-produced contract reference document  JSF Acquisition Strategy  LM Aero proposal  SDD contract  Integrated Master Plan events  e.g., JSF Distributed Product Description (DPD) is a DAE IPR & DAB review exit criterion DISTRIBUTION STATEMENT A: Approved for Public Release; Distribution is Unlimited.

32. 32 Simulation Based Acquisition (SBA): Shared Information is the Hub of IPPD DISTRIBUTION STATEMENT A: Approved for Public Release; Distribution is Unlimited.

33. 33 Components of the JSF Modeling & Simulation Toolset JSF Suite of Models and Simulations (SoM&S) JSF Strike Warfare Collaborative Environment (SWCE) Toolset (28 models & sims) JSF Engineering and Manufacturing Collaborative Environment SWCESoM&SEMCESoM&S JSF Distributed Product Description (DPD) MSDB Scenarios Databases Authoritative Systems Data Base (ASDB) MSDB Scenarios Databases JSF Resource Access System (RAS) (EMCE) Toolset (100+ models & sims) (JSF M&S Support Plan Fig. 5-1, annotated) DISTRIBUTION STATEMENT A: Approved for Public Release; Distribution is Unlimited.

34. 34 3 Paths to System Representations Compile Load Develop a software component Develop a simulation application Input data sets Executable applications Model or simulation modified to represent a battlespace Model or simulation developed to represent a system Model or simulation initialized to represent a battlespace (Federate) execute Load Compile Data-driven system representation in another’s tool (e.g., Thunder, JIMM) Software representation within another’s tool (e.g., ESAMS, Brawler) System representation as a dedicated application (e.g., JSF Virtual Simulator) Information mapping, translation, pre-processing Authoritative info in its original form DIFs Most authoritative system information Information in application-specific form DISTRIBUTION STATEMENT A: Approved for Public Release; Distribution is Unlimited.

35. 35 RAS User Interface Example Data Item icons appear here The Data Items associated with a navigation node are listed by data item type Each navigation node has metadata DISTRIBUTION STATEMENT A: Approved for Public Release; Distribution is Unlimited.

36. 36 Data Engineering Process Reverse Engineering for Data Integration and Sharing (REDIS) - simplified Data Interchange Format specs Integrated Database Integrated Information Model Information model InformationmodelInformationmodelInformationmodel Informationmodel Source 1 data files Model 1 data files Model 2 data files system n data files Source 2 data files ReverseEngineering Integration/Synthesis ForwardEngineering Data Dictionary … … DISTRIBUTION STATEMENT A: Approved for Public Release; Distribution is Unlimited.

37. 37 A Derivation Trace Preliminary or Detailed Analysis CATIA files Outer Mold Line Conversion Process Facet file Xpatch/ 3DMoM Azimuth Frequency Elevation Polarization Material Assignment Process … RCS Values … Brawler Aircraft Char. AndPerf. Parameters Materials database Facet file with Material IDs DISTRIBUTION STATEMENT A: Approved for Public Release; Distribution is Unlimited.

38. 38 Sample Aircraft Performance Attributes  Lift coefficient vs. angle-of-attack Temperature vs. mach, altitude, throttle Post-stall maneuver capability Min and Max g capacity Max mach number and calibrated airspeed Afterburner delay time Turbine spin-up and spin-down times Correlation factor for turbine to afterburner thrust Max mach number with a drag device Constraints on thrust reversal employment Time to max thrust reversal Thrust reversal efficiency Damping ratio for angle-of-attack response Roll acceleration factor due to angle-of-attack Stability axis roll rate limits by AOA and dynamic pressure Roll acceleration increment for positive AOA acceleration. limit Radar cross section by azimuth, elevation and wavelength Infrared signature characteristics Physical attributes Weapons quantities, operating modes and locations Sensor quantities, operating modes and locations DISTRIBUTION STATEMENT A: Approved for Public Release; Distribution is Unlimited.

39. 39 Shared Responsibilities During SDD The Government will:The SDD contractor shall: Establish and maintain a Strike Warfare Collaborative Environment in which the DPD-based JSF representations will operate, that spans the collaboration focus areas of mission effectiveness analysis, autonomic logistics analysis, engineering and manufacturing analysis and cost analysis Establish and manage an Engineering and Manufacturing Collaborative Environment Provide configuration-managed versions of a JSF SWCE Suite of Models and Simulations Develop, populate, and manage the JSF DPD, which, as a minimum, shall provide, via a DIF, all the JSF information needed to represent the JSF in the SWCE and the EMCE SoM&S. Develop, maintain, and manage the configuration of a library of SWCE boxed sets. Develop translations and Digital System Models (DSMs) not provided by the WSC. Develop the software that may be necessary to convert DPD information into the form needed to initialize or configure an individual model or simulation for that portion of the SWCE SoM&S that the WSC uses. This shall include translations and JSF Digital System Models. Provide an Authoritative Systems Database (ASDB) which is accessible through the JSF Resource Access System Develop and maintain a web-based JSF Resource Access System JSF M&S Support Plan Table 1-1 DISTRIBUTION STATEMENT A: Approved for Public Release; Distribution is Unlimited.