Applying the Churchman/Ackoff Value Estimation Procedure to Spatial Modeling Susan L. Ose MGIS Capstone Presentation Penn State University - World Campus.

Slides:



Advertisements
Similar presentations
Salt Marsh Restoration Site Selection Tool An Example Application: Ranking Potential Salt Marsh Restoration Sites Using Social and Environmental Factors.
Advertisements

The Nominal Group Technique Chapter 42 Research Methodologies.
ANALYSIS OF VARIANCE (ONE WAY)
Online Rubric Assessment Tool for Marine Engineering Course
STUDENT GUIDE. Go to the PUC Homepage located at From the Student drop-down menu, move cursor over the myPUC link and click myPUC Portal.
OECD/INFE High-level Principles for the evaluation of financial education programmes Adele Atkinson, PhD OECD With the support of the Russian/World Bank/OECD.
Project Selection (Ch 4)
Grid-based Map Analysis Techniques and Modeling Workshop Part 1 – Maps as Data Part 2– Surface Modeling Part 3 – Spatial Data Mining Part 4 – Spatial.
Sampling Distributions
By Joseph K. Berry W. M. Keck Scholar, University of Denver – August, An Overview of GIS-based Corridor.
Decisional Guidance for Computer based Decision support A Review by Cheickna Sylla Spring ‘97.
Using Rubrics for Evaluating Student Learning. Purpose To review the development of rubrics for the purpose of assessment To share an example of how a.
What is Forecasting? A forecast is an estimate of what is likely to happen in the future. Forecasts are concerned with determining what the future will.
Peer assessment of group work using WebPA Neil Gordon Symposium on the Benefits of eLearning Technologies University of Manchester, in conjunction with.
Local Agricultural Supply Chain Improvement Lindsey Arita, Joseph Lee, Julia Mahon, Jay Raghavan Advisor: Dr. Eric Bruun ESE Senior Design Demo.
Sampling.
Ensemble Learning: An Introduction
Using Rubrics for Evaluating Student Learning Office of Assessment and Accreditation Indiana State University.
Academic Advisor: Prof. Ronen Brafman Team Members: Ran Isenberg Mirit Markovich Noa Aharon Alon Furman.
Information Literacy and Assessment: The What, Why, Who, Where, When, and (most importantly) How.
11 Populations and Samples.
QMS 6351 Statistics and Research Methods Chapter 7 Sampling and Sampling Distributions Prof. Vera Adamchik.
FOUNDATIONS OF NURSING RESEARCH Sixth Edition CHAPTER Copyright ©2012 by Pearson Education, Inc. All rights reserved. Foundations of Nursing Research,
Agenda: Block Watch: Random Assignment, Outcomes, and indicators Issues in Impact and Random Assignment: Youth Transition Demonstration –Who is randomized?
Federal Consulting Group August 2004 Department of Labor Civil Rights Center 2004 Satisfaction Study - Recipients.
O NLINE R EPORTING S YSTEM (R EQUIRED FOR DTC S ).
Comparison of Classification Methods for Customer Attrition Analysis Xiaohua Hu, Ph.D. Drexel University Philadelphia, PA, 19104
McGraw-Hill/Irwin Copyright 2006 by The McGraw-Hill Companies, Inc.
COLLECTING QUANTITATIVE DATA: Sampling and Data collection
The Tutorial of Principal Component Analysis, Hierarchical Clustering, and Multidimensional Scaling Wenshan Wang.
Tulane University 1 Tulane University Employee Satisfaction Survey Results October 2012.
1 1 Slide © 2005 Thomson/South-Western Slides Prepared by JOHN S. LOUCKS St. Edward’s University Slides Prepared by JOHN S. LOUCKS St. Edward’s University.
Map Analysis Procedures and Applications in GIS Modeling Topic 19, Routing and Optimal Paths Online book written by Joseph K. Berry www/innovativegis.com/Basis/MapAnalysis/
HIT241 - RISK MANAGEMENT Introduction
Select and Develop Top Performing Sales Professionals Introducing SalesMax Revised 10/2005.
1 1 Slide © 2004 Thomson/South-Western Chapter 17 Multicriteria Decisions n Goal Programming n Goal Programming: Formulation and Graphical Solution and.
1 1 Slide Chapter 7 (b) – Point Estimation and Sampling Distributions Point estimation is a form of statistical inference. Point estimation is a form of.
Smith/Davis (c) 2005 Prentice Hall Chapter Four Basic Statistical Concepts, Frequency Tables, Graphs, Frequency Distributions, and Measures of Central.
OECD/INFE Tools for evaluating financial education programmes Adele Atkinson, PhD Policy Analyst OECD With the support of the Russian/World Bank/OECD Trust.
 Closing the loop: Providing test developers with performance level descriptors so standard setters can do their job Amanda A. Wolkowitz Alpine Testing.
AQUIFER VULNERABILITY ASSESSMENT: PROJECT REVIEW COLUMBIA COUNTY June 11, 2009 Columbia County, Florida Advanced GeoSpatial Inc. Alan Baker, P.G. James.
Sponsor: Dr. K.C. Chang Tony Chen Ehsan Esmaeilzadeh Ali Jarvandi Ning Lin Ryan O’Neil Spring 2010.
Air Travel Forecast Problem 1 Objectives Introduction to forecasting methods Experience with Delphi Experience with consensus-seeking techniques Strength/weaknesses.
Copyright © 1994 Carnegie Mellon University Disciplined Software Engineering - Lecture 3 1 Software Size Estimation I Material adapted from: Disciplined.
1 of 27 How to invest in Information for Development An Introduction Introduction This question is the focus of our examination of the information management.
Assessment Literacy Interim Assessment Kansas State Department of Education ASSESSMENT LITERACY PROJECT1.
 Job evaluation is the process of systematically determining the relative worth of jobs to create a job structure for the organization  The evaluation.
Disciplined Software Engineering Lecture #3 Software Engineering Institute Carnegie Mellon University Pittsburgh, PA Sponsored by the U.S. Department.
Lecture 11: Compensation. Strategic Issues and Compensation  Why do dome employers pay more than other employers?  Why are different jobs within the.
Printing: This poster is 48” wide by 36” high. It’s designed to be printed on a large-format printer. Customizing the Content: The placeholders in this.
Copyright 2010, The World Bank Group. All Rights Reserved. Testing and Documentation Part II.
Sampling Fundamentals 2 Sampling Process Identify Target Population Select Sampling Procedure Determine Sampling Frame Determine Sample Size.
EBM --- Journal Reading Presenter :呂宥達 Date : 2005/10/27.
An Overview of the Objectives, Approach, and Components of ComET™ Mr. Paul Price The LifeLine Group All slides and material Copyright protected.
Ensemble Methods Construct a set of classifiers from the training data Predict class label of previously unseen records by aggregating predictions made.
Model Calibration and Weighting Avoid areas of… High Housing Density Far from Roads In or Near Sensitive Areas High Visual Exposure …what is “high” housing.
 Galorath Incorporated 2004 AccuScope – A Tool For Relative Comparisons Lee Fischman & Kenneth Fong Galorath Incorporated SCEA 2004 Manhattan Beach,
The RutDel Methodology Presented by: Danielle Lombardi.
ESTIMATING WEIGHT Course: Special Topics in Remote Sensing & GIS Mirza Muhammad Waqar Contact: EXT:2257 RG712.
STAT MINI- PROJECT NKU Executive Doctoral Co-hort August, 2012 Dot Perkins.
Dr. Bea Bourne 1. 2 If you have any troubles in seminar, please do call Tech Support at: They can assist if you get “bumped” from the seminar.
Audition. Based on - Audition is a leading edge people matching tool, transforming selection and recruitment into a science more than an art. What is.
IMPACT EVALUATION PBAF 526 Class 5, October 31, 2011.
CPH Dr. Charnigo Chap. 11 Notes Figure 11.2 provides a diagram which shows, at a glance, what a neural network does. Inputs X 1, X 2,.., X P are.
To flip or not to flip: An exploratory analysis into student attitudes towards the flipped classroom approach to learning Enhancement Themes conference,
Chapter 11: Project Risk Management Information Technology Project Management, Fifth Edition.
Chapter 7 (b) – Point Estimation and Sampling Distributions
Selecting a Solution Path
Selecting a Solution Path
Choose the best soltution Design Process Presentation Name Course Name
Presentation transcript:

Applying the Churchman/Ackoff Value Estimation Procedure to Spatial Modeling Susan L. Ose MGIS Capstone Presentation Penn State University - World Campus 17 June 2008

GIS Models - A Refresher GIS models provide –Decision Support –Prediction –Cost assessment, etc. Weighting values –Emphasize/De-emphasize impact of inputs –Model adjusted via tweaking weights How does an analyst determine weights? Figure source: Joseph K. Berry, University of Denver

Estimating Weights - Gathering Expert Opinions via Delphi Process ? ? Compile Results Experts Questionnaire Discussion Anonymous Questionnaires Statistical aggregation Controlled feedback Avoids groupthink Consensus determines number of rounds

Questionnaire Format The ideal questionnaire: –Guides the user through the weight estimation process –Focuses on expert knowledge - not statistics –Emphasizes relative not numeric values Software module –Stores and compiles results –Allows easy distribution to expert group –Can be used in online meeting

Churchman/Ackoff Procedure Focuses on relative value of layers –Compares layer against combination of other layers –Uses recursive procedures –Breaks down larger groups of layers for easier assessment Recommended for group decision making Project adapted method to browser-based software module

Churchman/Ackoff Procedure Step 1: Rank Layers Layers are dragged and dropped into desired order

Churchman/Ackoff Procedure Step 2: Assign Initial Weights Layers divided subdivided into equal groups of no more than four. Control layer randomly chosen, assigned value of 50, added to each group Expert inputs initial estimate of weight Group 1 Group 2

Churchman/Ackoff Procedure Step 3: Judge Importance Expert chooses one of the three conditions and clicks on it. Algorithm adjusts values accordingly.

Churchman/Ackoff Procedure : Results Screen

Procedure Test Case Potential for rain-fed agriculture in Liberia taking into account cost to market Expert panel consisted of MDA Federal, Inc. employees experienced in modeling Used two Delphi rounds –Compiled statistics –Ran model using average, high, and low values

Model Diagram

First Round Results Reference layers received lowest scores Proximity to water and land layers received highest scores Model output using average score and 8 highest weighted layers consistent with results obtained using different methodology Discussion focused on experience with procedure Panel agreed on top 7 layers to include in final model

Second Round Results Results reflected post-first round discussion –Access to water considered key, water layers grouped near top of ranking –Land cover shows potential land to be transformed to agriculture, thus higher score than previous Higher variance in scores than in first round –Probably due to score distribution among less layers –First round lower variance may not have happened if expert could discard layers Model more definitive around water features

Round Comparison

Model Results - Average 2nd round1st round

Does the method work? Model result viable Considering one vs. many values more difficult than considering one vs. one (pairwise method) –Deliberately designed that way to challenge one's opinions –May frustrate participants - "weighting fatigue" Random groupings confused some participants –Difficult to recall original ranking, weight –However randomness focuses judgment on subgroup Can assist individual in examining own conclusions

What happens when someone joins the group later? One expert did not attend discussions, used tool with minimal guidance First round results changed slightly Average Z-score highest of the group

Second Round with Additional Person Pronounced difference in results Greater difference in layer order in mid to lower layers Average Z-score still highest of the group Demonstrates importance of group discussion Z-Scores New expertExpert 1Expert 2Expert 3Expert 4Expert 5 First Round Second Round

Further Development/Study Improve the software tool –Rework the math algorithm that calculates the weights –Offer opportunity to discard layers –Provide comparison of start/finish results –Provide progress bar Application of the method –Categorical values within layers need to be rated first –Orientation meetings important Conduct an experiment using both this method and a pairwise approach and compare results

Acknowledgements Dr. Gregory Koeln, President, MDA Federal Inc. for sponsoring this project Michael Schreiber, Webmaster, MDA Federal Inc. for transforming the procedure from paper to software Dr. Todd Bacastow and Dr. Douglas Miller, Penn State World Campus, for their advice and guidance Dr. Douglas Way, MDA Federal Inc. for his invaluable guidance David Cunningham, Dr. Anna Oldak, Dr. François Smith, and Dr. Andrew Ralowicz, MDA Federal Inc. for their expertise Gregory A. Ose, for his unfailing support