Maggie Faber, Jeremy Buhler, Frankie Wilson

Slides:



Advertisements
Similar presentations
Developing an Outcomes Assessment Plan. Part One: Asking a Meaningful Question OA is not hard science as we are doing it. Data that you collect is only.
Advertisements

Improve your Club’s project design and implementation Presented by: Ulrike Neubert SIE PD Ann-Christine Soderlund SIE APD.
0 Tactics and Technologies for Stakeholder Engagement With Keith Ellis CEO, Enfocus Solutions.
EngageNY.org Making Sense of Classroom Data: “Trackers” and The Atlas Protocol Session 1, December 2014 NTI.
Design Research Intelligent questioning for effective designs.
University of the West of England, Bristol of the West of England Quality Assurance in the UK Stephen Ryrie Associate.
Examples of using SERU/UCUES Results and Strategies for Increasing Usage on Campus Ron Huesman, Discussion Chair.
Gender Aware Monitoring and Evaluation. Amsterdam, The Netherlands Presentation overview This presentation is comprised of the following sections:
Obtaining reliable feedback from students about teaching
Sharon Walpole University of Delaware Michael C. McKenna University of Virginia Literacy Coaches in Action: Strategies for Crafting Building- Level Support.
INTRODUCTION TO THE MINISTRY OF EDUCATION’s ACHIEVEMENT CHART Bedford Park PS September 2013.
Robin Sabo, Clear Creek ISD. What is rigor? Or is rigor more that that But what about in education? Harder Challenging Focused on higher thinking levels.
Outcome Based Evaluation for Digital Library Projects and Services
OECD/INFE Tools for evaluating financial education programmes Adele Atkinson, PhD Policy Analyst OECD With the support of the Russian/World Bank/OECD Trust.
Overview The Importance of Data: As easy as balancing your checkbook.
1 Business Communication Process and Product Brief Canadian Edition, Mary Ellen Guffey Kathleen Rhodes Patricia Rogin (c) 2003 Nelson, a division of Thomson.
CSCI 4163 / CSCI 6904 – Winter Housekeeping  Clarification about due date for reading comments/questions  Skills sheet  Active listening handout.
Project Management MLP, Careers & Employability Division.
Strategic Planning Presented by: Amy Bocko Laura Hankin Luca Manna Goals. Actions. Future.
Charting Library Service Quality Sheri Downer Auburn University Libraries.
Consultant Advance Research Team. Outline UNDERSTANDING M&E DATA NEEDS PEOPLE, PARTNERSHIP AND PLANNING 1.Organizational structures with HIV M&E functions.
Session 3 Delivering a catechetical session: things to consider.
Requirements Engineering Processes. Syllabus l Definition of Requirement engineering process (REP) l Phases of Requirements Engineering Process: Requirements.
Developing Monitoring & Evaluation Frameworks: Process or Product? Anne Markiewicz.
AssessPlanDo Review QuestionYesNo? Do I know what I want to evaluate and why? Consider drivers and audience Do I already know the answer to my evaluation.
Developing a Monitoring and Evaluation (M&E) Plan.
From Program Logic to Monitoring & Evaluation (M&E) Plan.
Smarter Balanced Scores & Reports. The new assessment, Smarter Balanced, replaces our previous statewide assessment, the New England Common Assessment.
Insert name of presentation on Master Slide The Quality Improvement Guide Insert Date here Presenter:
Supplemental Math Digital Tool: Dreambox
What about the Assessment System?
Helping Students Examine Their Reasoning
Rebecca McQuaid Evaluation 101 …a focus on Programs Rebecca McQuaid
& How to Study When Your Professor Doesn’t Give a Study Guide
Why do we use Milepost? “one stop shop for data”
Monitoring and Evaluation
External sales , From good to great strategies for success
Ask students to write on an index card individually
What is research? Marie Hobson,
Quarterly Meeting Focus
Building the foundations for innovation
Creating a new story: A library strategy
Confronting Conflict s34n Team Resolution.
Magothy River Middle School
Advocacy and CampaiGning
Studying.
Strategic Prevention Framework - Evaluation
Dr Anna Stodter FST Department of Sport and Exercise Sciences
Tips on Applying to the AS&E Diversity Fund
Big Data Inside the Museum
Senior Phase Course Choice
Monitoring and Evaluation using the
LearnSmart Achieve™ Adaptive Test Prep
Hannah Hirschland, LMSW, Managing Director of Analytics & Evaluation
Utilizing Internal Audit Metrics to Advance Your Department
Lesson 14.1 The Money You Earn 2/10
Resource 1. Evaluation Planning Template
Helen Jefferis, Soraya Kouadri & Elaine Thomas
JCPS Success Skills Prepared and Resilient Learner
Good morning-thanks for being here
Participatory Data-Gathering and Community Building
Ask students to write on an index card individually
NEPf-Aligned Student Perception Survey Implementation
Time 1.
Senior Phase Course Choice
KEY INITIATIVE Financial Data and Analytics
How to Use Learning Analytics in Moodle
Quality Framework Overview
The Teacher Work Sample: An Authentic Assessment
Driving Successful Projects
Presentation transcript:

Maggie Faber, Jeremy Buhler, Frankie Wilson Beyond “will you make me a dashboard?” Meaningful collaboration on visualization and reporting Maggie Faber, Jeremy Buhler, Frankie Wilson

Guiding Questions How do we facilitate collaboration with colleagues and stakeholders on project design, reporting, and visualization? How do we design assessment tools and dashboards to support decision- making and advocacy that actually get used? Can our experiences inform and improve each others’ work?

Collaborative Models Structured Approach Systems Approach Inquiry Approach Collaborative Models

Collaborative Models Structured Approach Systems Approach Inquiry Approach Collaborative Models

User Story Framework As a <user type> When I <context for use of dashboard> I want to <assessment question> in order to <what user is aiming to achieve>

User Story Framework As a library manager When I write a proposal for additional staffing budget to extend opening hours on weekends I want to understand who is using my library on Saturdays and Sundays in order to get support from the heads of these departments for my bid.

Collaborative Models Structured Approach Systems Approach Inquiry Approach Collaborative Models

Consultation process

Task & Audience

Collaborative Models Structured Approach Systems Approach Inquiry Approach Collaborative Models

UBC Library assessment consultation: prompting questions What do you want to assess? Why do you want to assess it? What will you do with the information? How much is the information worth? Adapted from Tanner, Simon. Measuring the impact of digital resources: the balanced value impact model. King’s College London, 2012, p31. www.kdcs.kcl.ac.uk/fileadmin/documents/pubs/BalancedValueImpactModel_SimonTanner_October2012.pdf As simple as these questions appear, it can be difficult for clients to provide clear and specific answers. It is common to begin with an assessment goal that is very broad (e.g. “impact on students”) or ambiguously defined. The discipline of answering these questions often focuses the project (or the dashboard). Clients often struggle to answer how much the information is worth; rephrasing in terms of risk and uncertainty is sometimes helpful. For example, if the information is needed to make a decision ask what’s at stake if they make the wrong decision (risk), and whether other evidence already suggests a particular course of action (uncertainty).

Captivating but overlooked Routing of UBC Library physical items after check-in This information-rich and skillfully designed dashboard was, nevertheless, rarely used. It shows where UBC Library’s physical items are routed after check-in. Each color represents a branch: The left column is where items are checked in, and the right column is the “owning” branch where they end up back on the shelves. It didn’t connect with an audience and fell into disuse in part because the information was not needed for a specific decision or for day-to-day operations. Moral of the story: the best dashboard is often the one that’s right-sized, that’s in tune with its audience’s needs. This requires good listening and interpretation skills on the part of those who build the dashboard. It can also involve managing the expectations of those who request it.

Plain but consulted UBC Library collections expenditures by type (CAD $) This dashboard doesn’t wow the audience with visual elements, but the data about library collections spending is easy to find and always up to date (the report refreshes automatically every week). It is successful because it provides figures that are often required for monitoring, managing, and reporting on collections expenditures. In other words, it provides something its audience needs.

Collaborative Models Structured Approach Systems Approach Inquiry Approach Collaborative Models

Collaborative Models Structured Approach Systems Approach Inquiry Approach Collaborative Models

Thank you! Questions? frankie.wilson@bodleian.ox.ac.uk @qualitydog faberm@uw.edu @visualibrarian jeremy.buhler@ubc.ca