Defence Research and Development Canada Recherche et développement pour la défense Canada Canada Multinational Experiment 4 on Effects Based Approach to Operations A CFEC Report Philip S. E. Farrell, Ph.D. A presentation for IAMWG Meeting Friday 11 May 2007

Defence R&D Canada R & D pour la défense Canada Presentation Introduction – Human Experimentation MNE 4 CFEC Report Discussion – Can war-fighting experimentation be done in a controlled laboratory setting?

Defence R&D Canada R & D pour la défense Canada Introduction Not “The Bear” Now “The Ball of Snakes” New concepts need to be tested

Defence R&D Canada R & D pour la défense Canada Scientific Method High School Science Report –Background –Purpose –Hypothesis (related to concept) –Analysis –Apparatus –Procedures –Results –Discussion –Conclusions

Defence R&D Canada R & D pour la défense Canada Human Experimentation

Defence R&D Canada R & D pour la défense Canada Human Experimentation Job Similarity Index concept TTPs Recommendations Crew Selection Experiment (2006)

Defence R&D Canada R & D pour la défense Canada Human Experimentation in MNE 4 Too many people? Too many variables? Too many confounds? Too many constraints?

Defence R&D Canada R & D pour la défense Canada MNE 4 Experiment Effects Based Approaches to Operations Refinement and Assessment of EBAO POT Desired End State Planning (desired effects) + _ Execution Assessment (effects) Operational Environment Current situations World states desired actions disturbances ++

Defence R&D Canada R & D pour la défense Canada Analysis Function KBD, SOSA, KM MNE 4 Experiment (CONOPS 0.9) + + ESAED RG ADRM SPRPREPOC&C AAMOP W AAMOE QCE MMA EBAP + _ + _ + _ IDI “action” comparators CA “situation” and “effects” comparators RsRs CsCs S RaRa ReRe CeCe CaCa AD ++ Planning Assessment Execution

Defence R&D Canada R & D pour la défense Canada MNE 4 Experiment Experiment Aim Experiment Objectives Hypothesis Critical Operational Issues Measures Data Experiment Execution Data Collection

Defence R&D Canada R & D pour la défense Canada MNE 4 Experiment Competing Aims! to explore concepts and supporting tools for EBAO to address the Canadian MNE 4 objectives

Defence R&D Canada R & D pour la défense Canada MNE 4 Experiment Canadian Objectives –Refine, assess, and analyze elements of EBAO that Canada and the CF could apply to achieve integrated strategic effort in operations. –Determine whether EBAO represents a means of applying the Canadian 3-D approach to operations –Develop and refine a Multinational Knowledge Management Plan that can be tailored and applied to meet national operational requirements

Defence R&D Canada R & D pour la défense Canada MNE 4 Experiment The Hypothesis of this experiment was: If an operational commander and staff employ effects based operations, then they will have the ability to: Create a holistic understanding (KBD) Develop and refine an operational plan (EBP) Harmonize the full spectrum of DIME planning (MNIG) Coordinate, direct, and monitor operations (EBE) Assess actions, effects, and end state (EBA)

Defence R&D Canada R & D pour la défense Canada MNE 4 Experiment

Defence R&D Canada R & D pour la défense Canada MNE 4 Experiment Experimental Design

Defence R&D Canada R & D pour la défense Canada MNE 4 Experiment Types of Analyses Observations –JOT (Joint Observation Tool) –JDCAT (Joint Battle Centre Data Collection Analysis Tool)

Defence R&D Canada R & D pour la défense Canada MNE 4 Experiment Types of Analyses User surveys and network data

Defence R&D Canada R & D pour la défense Canada MNE 4 Experiment Types of Analyses Box Plots (KBD) Figure C.1 Distribution of correct responses for SA surveys, by organization

Defence R&D Canada R & D pour la défense Canada MNE 4 Experiment Types of Analyses Multi-Dimensional Scaling (KBD) Figure C.4: Multi-dimensional scaling of KBD3 answers by Group Reference Point (KBD 3) Reported same incorrect answers

Defence R&D Canada R & D pour la défense Canada MNE 4 Experiment Types of Analyses Count data (KM)

Defence R&D Canada R & D pour la défense Canada MNE 4 Experiment Types of Analyses Whitney – U (Culture) Q3.2b. To what degree is the decision of the leader contingent on the work and advice of others? (this question was for those who did not answer Leaderless Team in Q3.2a) Mann-Whitney U = 2212, p <.07 Not at allLittleSomewhatMuchCompletelyNot applicable 22 NRF0%5%44%36%11%4%  2 = 38.18, p <.001 CTF1%8%19%54%15%3%  2 = , p <.001

Defence R&D Canada R & D pour la défense Canada MNE 4 Experiment Types of Analyses Vector Method (Common Intent) DateNameVector 28 Febv1 (7, 5, 6, 7) 2 Marv2 (7, 6, 5, 7) 7 Marv3 (7, 6, 7, 7) 9 Marv4 (7, 7, 7, 6) 14 Marv5 (7, 6, 7, 7) 16 Marv6 (7, 7, 7, 6) Referencer (7, 7, 7, 7)

Defence R&D Canada R & D pour la défense Canada MNE 4 Experiment Types of Analyses Vector Method (Common Intent) v1v2v3v4v5v6r v v v v v v r Table 3. Angle between vector pairs in degrees (Duncan & Farrell, 2007)

Defence R&D Canada R & D pour la défense Canada MNE 4 Experiment Results, Discussion, Conclusions, and Recommendations are just as fuzzy as the objectives and hypothesis.

Defence R&D Canada R & D pour la défense Canada MNE 4 Experiment Insights related to Canadian Objectives The notion of planning, execution, and assessment in an iterative fashion until the Current State reaches the End State is a robust concept. The relationship between MNIG, EBAO, and the 3- D concepts requires further development and experimentation. Some of the basic KM and KBD definitions and concepts still have not been resolved.

Defence R&D Canada R & D pour la défense Canada MNE 4 Experiment Recommendations More research is required!

Defence R&D Canada R & D pour la défense Canada Discussion Can war-fighting experimentation be done in a laboratory controlled setting?

Defence R&D Canada R & D pour la défense Canada Discussion Military buy-in vs. Scientific reputation Do you need SMEs to conduct the experiment?

Defence R&D Canada R & D pour la défense Canada Discussion Human Experimentation Ethics Is voluntary consent necessary?

Defence R&D Canada R & D pour la défense Canada Discussion political Considerations Who makes the decisions?

Defence R&D Canada R & D pour la défense Canada Discussion Concept complexity Are there too many variables?

Defence R&D Canada R & D pour la défense Canada Discussion Experimental Design Whatever happened to Hypothesis-testing?

Defence R&D Canada R & D pour la défense Canada Summary Human Experimentation is not new. MNE 4 has refined our processes for war-fighting experimentation. We need to pay attention to the scientific method if our recommendations are to be credible

Defence R&D Canada R & D pour la défense Canada Questions