Design your System with Object-Process Methodology – OPM the New ISO 19450 Dov Dori Massachusetts Institute of Technology Technion, Israel Institute of Technology SRI International Princeton, NJ July 20, 2015

How was this talk initiated? What will it be about? Correspondence and Skype with Dr. Maneesh Singh of SRI after getting invitation here I asked for an example to be used in the talk I received a specification of OSRVT: Video Moving Target Indication Capability (coming up) I will first present Object-Process Methodology – OPM, the New ISO 19450 Then we will model the system using OPM and discuss potential benefits

OSRVT: Video Moving Target Indication Capability Presented to SRI International November 10, 2014

One System Remote Video Terminal System MTI

Objective Video Moving Target Indication* (VMTI) capability for OSRVT using platform data stream (e.g., MPEG2 TS) *A computational process of locating a moving object (or several ones) in a video frame. No ID reported. Note: Introduction of such a capability will have little or no impact on other OSRVT operations.

OSRVT VMTI System HMI CSCI (Front End) VMTI CSCI (Back End) 6

VMTI Module STAB MTI OSRVT OSRVT Input Imagery MTI ROI’s Metadata (to) Screened Imagery Screened Imagery Screened Imagery Con t r o l e I m a g e Q u l i t y STAB O v e r l a y D t c MTI VMTI Imagery OSRVT OSRVT Overlay Mask MTI ROI’s (per frame) Synchronized F2F Alignment Metadata F2F Alignment MTI Params Enable/Disable F2F Alignment (from) (to) Generates MTI params from metadata Screens input frames for image defects Aligns consecutive frames for stabilized stream Direct method Feature based Detects image overlays, generates mask Detects moving targets outside of overlay mask in screened, stabilized imagery Options

Moving Target Indication Detection Verification Video frames (stabilized) with moving targets Blob Extraction Laplacian Pyramid Generation Change/ Foreground Detection Filtering + Video frames (original) with moving target indications (VMTI imagery) Moving Target Indication Core Overlay mask imagery Generates multi-resolution features (Laplacian) Detects pixel-based spatio-temporal changes, relative to stabilized “background,” due to “foreground” features from a particular pyramid level (as specified by MTI parameters) Extracts “blobs” or connected foreground “change” pixels Filters “blobs” fulfilling motion consistency (temporal) check + MTI parameters

MTI Core Details + + Video frames (stabilized) with moving targets Foreground Detection (optional) Adaptive Background Modeling Foreground/Background Segmentation Video frames (stabilized) with moving targets Laplacian Pyramid Generation Compute spatiotemporal image gradients Compute Normal Flow values Threshold Normal Flow values Temporal Filtering for Consistent Pixels Fn RFn NFn + Normal Flow Based Change Detection MFn Connected Component Labeling Extract pixel groups using labels Compute Optical Flow Reverse flow warp Insp Blobs, Check Temporal Consistency Check Motion Significance Overlay mask imagery BFn FFn + MTI parameters Blob Extraction Optical Flow Based Blob Filtering Visual Odometry (Camera R, T) From Optical Flow De-rotate Optical Flow Field Compute Epipole And Enforce Epipolar Constraint on Blob Pixels Update Blobs After Removing Pixels Failing Epipolar Constraint PFn Video frames (original) with moving target indications (VMTI imagery) Parallax Detection (optional) Fn: Histogram equalized frame w/ stab params RFn: Laplacian of Fn+ Stabilized ref frame NFn: Fn+ Binarized Fn of change pixels MFn: Fn+ Binarized Fn of consistent change pixels BFn: Fn+ Blobs of change pixels FFn: Fn + consistent blobs + Optical Flow Field PFn: Fn + consistent blobs (following Parallax Detection)

What is Conceptual Modeling? A systematic, formalized process of describing, specifying, designing or explaining ideas, systems, products or processes through a model. Applicable to both Science – Studying what is known and what is missing to satisfy human thirst for knowledge, and Engineering – Designing systems to benefit humans, based on sound scientific principles Science can be thought of as reverse engineering of nature

Why Conceptual Modeling? Convert tacit, fragmented knowledge into explicit, integrative knowledge. Construct concise models – mental pictures of natural systems [science] and artificial systems [engineering] while integrating structure and behaviour at all detail levels. Communicate the model to stakeholders through formal, unambiguous, actionable descriptions.

A conceptual modeling language that is simple yet expressive, and intuitive yet formal Let the search begin!

Universal Ontology Ontology: a set of concepts for describing a domain (industry, banking, military, botany, healthcare…) and systems within it. Universal Ontology: a domain-independent set of concepts for describing systems in the universe, both natural and man-made.

Fundamental question 1: What is needed to describe the universe? Answer: Describing the universe requires things and relations among them. Object-Process Methodology – OPM, is a holistic systems paradigm which enables specification, explanation, understanding and communicating natural and artificial systems in any domain and at any level of complexity. OPM enables constructing a single unifying system model in two complementary, equivalent modalities: graphics and natural language. This dual representation provides for the ability to represent domain-independent knowledge clearly and explicitly, making it possible for people with very little experience to explain and understand the function, structure and behavior of systems in nature, the environment, and man-made systems. Using OPCAT - an elaborate OPM-supporting software environment, humans can engage in educational and creative activities that are facilitated by the intuitive, albeit formal, dual graphic and textual representations. Natural language text is automatically generated on the fly form the graphic input and vice versa. Applications include constructing system models, architecting and modeling artificial multidisciplinary systems, generating and testing them, and supporting them throughout their lifecycles. The generation is made possible, among other things, by automatically generating the code associated with the informational part of the system, while testing it is done by animated simulation of the system's behavior. 9/18/2018

Question 2: What can these things “do”? Answer: Things can exist or happen. Object-Process Methodology – OPM, is a holistic systems paradigm which enables specification, explanation, understanding and communicating natural and artificial systems in any domain and at any level of complexity. OPM enables constructing a single unifying system model in two complementary, equivalent modalities: graphics and natural language. This dual representation provides for the ability to represent domain-independent knowledge clearly and explicitly, making it possible for people with very little experience to explain and understand the function, structure and behavior of systems in nature, the environment, and man-made systems. Using OPCAT - an elaborate OPM-supporting software environment, humans can engage in educational and creative activities that are facilitated by the intuitive, albeit formal, dual graphic and textual representations. Natural language text is automatically generated on the fly form the graphic input and vice versa. Applications include constructing system models, architecting and modeling artificial multidisciplinary systems, generating and testing them, and supporting them throughout their lifecycles. The generation is made possible, among other things, by automatically generating the code associated with the informational part of the system, while testing it is done by animated simulation of the system's behavior. 9/18/2018

Question 3: What are the things that exist in the world? Answer: Objects exist. They are static. Object-Process Methodology – OPM, is a holistic systems paradigm which enables specification, explanation, understanding and communicating natural and artificial systems in any domain and at any level of complexity. OPM enables constructing a single unifying system model in two complementary, equivalent modalities: graphics and natural language. This dual representation provides for the ability to represent domain-independent knowledge clearly and explicitly, making it possible for people with very little experience to explain and understand the function, structure and behavior of systems in nature, the environment, and man-made systems. Using OPCAT - an elaborate OPM-supporting software environment, humans can engage in educational and creative activities that are facilitated by the intuitive, albeit formal, dual graphic and textual representations. Natural language text is automatically generated on the fly form the graphic input and vice versa. Applications include constructing system models, architecting and modeling artificial multidisciplinary systems, generating and testing them, and supporting them throughout their lifecycles. The generation is made possible, among other things, by automatically generating the code associated with the informational part of the system, while testing it is done by animated simulation of the system's behavior. 9/18/2018

Question 4: What are the things that happen in the world? Answer: Processes happen. They are dynamic. Object-Process Methodology – OPM, is a holistic systems paradigm which enables specification, explanation, understanding and communicating natural and artificial systems in any domain and at any level of complexity. OPM enables constructing a single unifying system model in two complementary, equivalent modalities: graphics and natural language. This dual representation provides for the ability to represent domain-independent knowledge clearly and explicitly, making it possible for people with very little experience to explain and understand the function, structure and behavior of systems in nature, the environment, and man-made systems. Using OPCAT - an elaborate OPM-supporting software environment, humans can engage in educational and creative activities that are facilitated by the intuitive, albeit formal, dual graphic and textual representations. Natural language text is automatically generated on the fly form the graphic input and vice versa. Applications include constructing system models, architecting and modeling artificial multidisciplinary systems, generating and testing them, and supporting them throughout their lifecycles. The generation is made possible, among other things, by automatically generating the code associated with the informational part of the system, while testing it is done by animated simulation of the system's behavior. 9/18/2018

Question 5: How do objects and processes relate? Answer: Processes happen to objects. While happening, processes transform objects. Object-Process Methodology – OPM, is a holistic systems paradigm which enables specification, explanation, understanding and communicating natural and artificial systems in any domain and at any level of complexity. OPM enables constructing a single unifying system model in two complementary, equivalent modalities: graphics and natural language. This dual representation provides for the ability to represent domain-independent knowledge clearly and explicitly, making it possible for people with very little experience to explain and understand the function, structure and behavior of systems in nature, the environment, and man-made systems. Using OPCAT - an elaborate OPM-supporting software environment, humans can engage in educational and creative activities that are facilitated by the intuitive, albeit formal, dual graphic and textual representations. Natural language text is automatically generated on the fly form the graphic input and vice versa. Applications include constructing system models, architecting and modeling artificial multidisciplinary systems, generating and testing them, and supporting them throughout their lifecycles. The generation is made possible, among other things, by automatically generating the code associated with the informational part of the system, while testing it is done by animated simulation of the system's behavior. 9/18/2018

OPM Things: Objects and Processes Object: A thing that exists or might exist physically or informatically. Process: A thing that transforms one or more objects.

Physical vs. Informatical Things

OPM’s only two building blocks: 1. Stateful Object 2. Process All the other elements are relations between things, expressed graphically as links.

processes transform objects. Transform? What does that mean? Transforming means creating an object or destroying an object or affecting an object. Object-Process Methodology – OPM, is a holistic systems paradigm which enables specification, explanation, understanding and communicating natural and artificial systems in any domain and at any level of complexity. OPM enables constructing a single unifying system model in two complementary, equivalent modalities: graphics and natural language. This dual representation provides for the ability to represent domain-independent knowledge clearly and explicitly, making it possible for people with very little experience to explain and understand the function, structure and behavior of systems in nature, the environment, and man-made systems. Using OPCAT - an elaborate OPM-supporting software environment, humans can engage in educational and creative activities that are facilitated by the intuitive, albeit formal, dual graphic and textual representations. Natural language text is automatically generated on the fly form the graphic input and vice versa. Applications include constructing system models, architecting and modeling artificial multidisciplinary systems, generating and testing them, and supporting them throughout their lifecycles. The generation is made possible, among other things, by automatically generating the code associated with the informational part of the system, while testing it is done by animated simulation of the system's behavior. 9/18/2018

Transforming an object by a process can be done in three ways (1) Process consumes the object

(2) Process creates the object Consumption Creation

processes affect objects. Affecting? What does that mean? A process affects an object by changing its state. Hence, objects must be stateful – they must have states. Object-Process Methodology – OPM, is a holistic systems paradigm which enables specification, explanation, understanding and communicating natural and artificial systems in any domain and at any level of complexity. OPM enables constructing a single unifying system model in two complementary, equivalent modalities: graphics and natural language. This dual representation provides for the ability to represent domain-independent knowledge clearly and explicitly, making it possible for people with very little experience to explain and understand the function, structure and behavior of systems in nature, the environment, and man-made systems. Using OPCAT - an elaborate OPM-supporting software environment, humans can engage in educational and creative activities that are facilitated by the intuitive, albeit formal, dual graphic and textual representations. Natural language text is automatically generated on the fly form the graphic input and vice versa. Applications include constructing system models, architecting and modeling artificial multidisciplinary systems, generating and testing them, and supporting them throughout their lifecycles. The generation is made possible, among other things, by automatically generating the code associated with the informational part of the system, while testing it is done by animated simulation of the system's behavior. 9/18/2018

(3) Process affects object by changing the object’s state: The third and last kind of object transformation: (3) Process affects object by changing the object’s state:

The three transformation kinds Consumption: Creation: OPM uses a single type of diagram – Object-Process Diagram (OPD) Graphic edit operations are translated on the fly to natural language – Object-Process Language (OPL) Catering to dual channel processing State Change:

The graphics-text equivalence OPM principle Any model fact expressed graphically in an OPD is also expressed textually in the corresponding OPL paragraph. Caters to the dual channel cognitive assumption (Mayer, 2010)

What are the two major aspects of any system? Structure – the static aspect: what the system is made of. Time-independent Behavior – the dynamic aspect: how the system changes over time. Time-dependent Object-Process Methodology – OPM, is a holistic systems paradigm which enables specification, explanation, understanding and communicating natural and artificial systems in any domain and at any level of complexity. OPM enables constructing a single unifying system model in two complementary, equivalent modalities: graphics and natural language. This dual representation provides for the ability to represent domain-independent knowledge clearly and explicitly, making it possible for people with very little experience to explain and understand the function, structure and behavior of systems in nature, the environment, and man-made systems. Using OPCAT - an elaborate OPM-supporting software environment, humans can engage in educational and creative activities that are facilitated by the intuitive, albeit formal, dual graphic and textual representations. Natural language text is automatically generated on the fly form the graphic input and vice versa. Applications include constructing system models, architecting and modeling artificial multidisciplinary systems, generating and testing them, and supporting them throughout their lifecycles. The generation is made possible, among other things, by automatically generating the code associated with the informational part of the system, while testing it is done by animated simulation of the system's behavior. 9/18/2018

What third aspect is specific to man-made systems? Function – the utilitarian, subjective aspect: Why is the system built? For whom is the system built? Who benefits from operating the system? Object-Process Methodology – OPM, is a holistic systems paradigm which enables specification, explanation, understanding and communicating natural and artificial systems in any domain and at any level of complexity. OPM enables constructing a single unifying system model in two complementary, equivalent modalities: graphics and natural language. This dual representation provides for the ability to represent domain-independent knowledge clearly and explicitly, making it possible for people with very little experience to explain and understand the function, structure and behavior of systems in nature, the environment, and man-made systems. Using OPCAT - an elaborate OPM-supporting software environment, humans can engage in educational and creative activities that are facilitated by the intuitive, albeit formal, dual graphic and textual representations. Natural language text is automatically generated on the fly form the graphic input and vice versa. Applications include constructing system models, architecting and modeling artificial multidisciplinary systems, generating and testing them, and supporting them throughout their lifecycles. The generation is made possible, among other things, by automatically generating the code associated with the informational part of the system, while testing it is done by animated simulation of the system's behavior. 9/18/2018

The Object-Process Theorem Stateful objects, processes, and relations among them constitute a necessary and sufficient universal ontology. Object-Process Methodology – OPM, is a holistic systems paradigm which enables specification, explanation, understanding and communicating natural and artificial systems in any domain and at any level of complexity. OPM enables constructing a single unifying system model in two complementary, equivalent modalities: graphics and natural language. This dual representation provides for the ability to represent domain-independent knowledge clearly and explicitly, making it possible for people with very little experience to explain and understand the function, structure and behavior of systems in nature, the environment, and man-made systems. Using OPCAT - an elaborate OPM-supporting software environment, humans can engage in educational and creative activities that are facilitated by the intuitive, albeit formal, dual graphic and textual representations. Natural language text is automatically generated on the fly form the graphic input and vice versa. Applications include constructing system models, architecting and modeling artificial multidisciplinary systems, generating and testing them, and supporting them throughout their lifecycles. The generation is made possible, among other things, by automatically generating the code associated with the informational part of the system, while testing it is done by animated simulation of the system's behavior. 9/18/2018

Complexity Management with OPM Systems are inherently complex. To alleviate this complexity, in OPM, it is managed by detail decomposition through three refinement-abstraction: In-zooming – Out-zooming Unfolding – Folding State expression – suppression.

In-zooming – Out-zooming Example Process Performance Controlling - a metamodel from ISO 19450 All the OPDs, at any detail level, are self-similar. They contain only stateful objects, processes, and relations.

Back to OSRVT – Moving Target Indicator: What is the Function of this system? Describe in three words, the last being a verb ending with ing (gerund) This will be our starting point of the OPM model

The Function: Moving Target Indicating

Who is the Beneficiary? Who benefits from operating the system?

What attribute of War Fighter changes value by operating the system, such that benefit is created?

What are the system’s input and output?

What is the name of the system we are developing?

The next detail level: Zooming into the Moving Target Indicating Function

The Auto-Generated OPL Text:

Summary: OPM Aspect Unification The three system aspects: Function (why the system is built), Structure (static aspect: what is the system made of), and Behavior (dynamic aspect: how the system changes over time) Are expressed bi-modally, in graphics and equivalent text In a single model

Value Proposition to SRI International Model requirements together with the customer Use this model as a basis for concept generation and their evaluation and selection of best one Achieve shared understanding and agreement of multidisciplinary engineering team Communicate the solution model with the customer Use the model across all the system lifecycle: detailed design, integration, testing, deployment, maintenance, retirement…

OPM Resources: Book: Object-Process Methodology - A Holistic Systems Paradigm, Springer Verlag, Berlin, Heidelberg, New York, 2002. Website: Enterprise Systems Modeling Laboratory contains journal & conference papers, free OPCAT software, presentations, projects, and more.

Questions and (hopefully) Answers Join the growing OPM community Here! https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=OPM&A=1 Contact: Dov Dori – dori@mit.edu Questions and (hopefully) Answers

Essence and Affiliation Essence pertains to the thing’s nature: denotes whether the thing is physical or informatical. Affiliation pertains to the thing’s scope: denotes whether the thing is systemic, i.e. part of the system, or environmental, i.e. part of the system’s environment The Essence-Affiliation attribute value combinations

Cyber-Physical Systems: Characteristics Software-controlled physical systems Include physical and cybernetic components An agent – a human decision-maker or an information & decision-making system – is the cybernetic component Hardware (motors, actuators, VLSI chips…) is the physical component Physical processes signal and induce cybernetic events Cybernetic processes signal and induce physical events

Essence is key to modeling Cyber-Physical systems Physical objects in the model represent what is really “out there” – actual states and values of objects Informatical objects represent information about their corresponding physical objects Only informatical objects are available to a decision making agent (human or artificial)

Cyber-Physical Gap A cyber-physical gap exists when the state of the informatical object incorrectly indicates the state of the physical object is supposed to represent

The cyber-physical gap – a critical factor in modern systems design It must be accounted for when designing systems, notably safety-critical ones OPM is most suitable for modeling cyber-physical gaps This is due to its notion of essence – physical vs. informatical things

Cyber-physical gap example: Three-Mile Island Accident First cyber-physical gap – Incorrect instrument reading: PORV is (stuck) open, but due to the false PORV closed indication, the Crew determines PORV is closed! A critical conflict between reality and its cybernetic mirroring! Full presentation in http://www.csdm2014.csdm.fr/-Program-.html#13

Appendix: SysML and OPM – a brief comparison Feature SysML OPM Theoretical foundation UML; Object-Oriented paradigm Minimal universal ontology; Object-Process Theorem Standard documentation number of pages 1670 (700 + 700 + 270) 130 (100 + 30) Standardization body OMG (2006) ISO (2014) Number of diagram kinds 9 1 Graphic modality yes Textual modality no Physical-informatical distinction Systemic-environmental distinction