Evaluating GIS Systems Todd Bacastow Geography 468

Evaluation Previously attention on a design process Evaluation is needed to ensure progress Evaluation is typically categorized into two areas Formative evaluations focus on developing and refining designs Summative evaluations compare an implemented system to an alternative system

Methods – Heuristic examinations measure user responses to the system based on a set of common system design criteria – Surveys use open or close-ended questions to identify system needs or areas for improvement – Focus groups involve group discussion of design options, user experiences, or other topics to inform or critique a design – Interviews make use of one-on-one questioning with users or customers to explore design options or gather feedback on tools

Methods – Card-sorting is an activity in which users organize system tools / functions using paper cards to suggest interface organization – Expert evaluation has system design and usability experts critique designs, prototypes, or final systems – Field Exercises put the system through a "test run" using realistic data, scenarios, and tasks – Cost-Benefit Analysis uses metrics to measure the costs of developing/using a system versus the benefits associated with its products

Cost-Benefit Analysis Is the project worthwhile financially? Is it the best option? Should it be undertaken at all? 15-5

Cost-Benefit For an individual – Might rely on a “gut feel” for the right choice. For a company – Concerned with the profit earning capacity and income flow For the Government – Expected to consider at least neutrality of the costing – Must also include consideration of the social cost and benefits – Act within the political environment 15-6

Measurement Estimating Intangible costs Several costs & benefits associated Affected by market Uneven distribution of benfit. 15-8

Time Problems Tackling future time problems by discounting future costs and benefits. OR calculating the correct rate for future dollar value as well as accounting for additional benefits and costs associated. 16-9

Method Identify all possible alternatives. Prepare table showing year to year cost of project including: – Capital – Operating – Maintenance costs – Social and other tangible costs Establish total benefits to be obtained from project by way of sales of goods and services including value of social benefits. Cost calculated at rate of interest Ranking in order of [benefit-cost] or [benefit / cost] 16-10