1. Analytics: Planning Considerations
Jacqueline Bichsel, John Fritz, Margie Jantti, and Celeste Schwartz with Sondra Smith
EDUCAUSE 2012 | Preconference Seminar 10P | November 6, 2012

2. Seminar Agenda Introductions Analytics and the maturity index
John Fritz and academic analytics at UMBC
Break
Margie Jantti and the Wollongong library cube
Celeste Schwartz and data informed decisions at MCCC
Closing questions, comments, and evaluations

3. Seminar Speakers Jacqueline Bichsel Senior Research Analyst, EDUCAUSE
John Fritz
Assistant Vice President, Instructional Technology & New Media 
University of Maryland, Baltimore County 
Margie Jantti
University Librarian, University of Wollongong 
Celeste M. Schwartz
Vice President for Information Technology and College Services 
Montgomery County Community College 
Sondra Smith
Director, Analytics Outreach and Education, EDUCAUSE 

4. (Selections from) a Year of Analytics
“SLIDER BAR” TOOL
ANALYTICS SPRINT
ECAR ANALYTICS STUDY

5. Seminar Agenda Introductions Analytics and the maturity index
John Fritz and academic analytics at UMBC
Break
Margie Jantti and the Wollongong library cube
Celeste Schwartz and data informed decisions at MCCC
Closing questions, comments, and evaluations

6. The 2012 Study of Analytics in Higher Education Jacqueline Bichsel, Ph
The 2012 Study of Analytics in Higher Education Jacqueline Bichsel, Ph.D. Senior Research Analyst, ECAR

7. Defining Analytics

8. EDUCAUSE Definition of Analytics
Analytics is the use of data, statistical analysis, and explanatory and predictive models to gain insights and act on complex issues.

9. Analytics Is a Priority

10. But Analytics Use Is Lagging
…particularly in areas of administrative and faculty activities

11. Higher Education’s Progress

12. ECAR Analytics Maturity Index

13. Benchmark Your Institution
Measure your progress on the ECAR Analytics Maturity Index

14. Seminar Agenda Introductions Analytics and the maturity index
John Fritz and academic analytics at UMBC
Break
Margie Jantti and the Wollongong library cube
Celeste Schwartz and data informed decisions at MCCC
Closing Q & A and evaluations

15. About Blackboard @ UMBC
Began using Bb in Spring 2000
Current version: 9.1, SP6
Adoption
95% of all students
75% of all instructors
65% of all courses
350 Communities
Analytics Initiatives
2012 Bb A4L LFT & Implementation
2011 Learning Analytics in Gateway Courses sub-grant
2010 Adopted iStrategy for analysis of all Bb courses
2009 Code release at BbWorld 2009
2008 Check My Activity for Students available
2007
Support Staff:
2 FTE (Admin & Support)
1 Server Admin
John

17. Bb Activity by Grade Distribution

19. New Method: Bb A4L (formerly iStrategy)
BbA4L Data Warehouse
(Queries)
Blackboard
(Activity)
Student Admin.
(Grades, Demographics)
Jeffrey
Check My Activity (Students)
Most Active Bb Courses (Faculty)

20. Accesses by Grade (SP2012)

21. New Reports

22. Evolving CMS Use by Faculty
User & Document Management (Pull)
Password-protected class & group space
Attach or Copy/Paste Documents (expiration)
Communications (Push)
Announcements
, Messages
Discussion & Chat
Assessments (Push & Pull)
Electronic assignment delivery & collection
Quizzing, Surveys, Course Usage
Adaptive release of content based on prior student action.

23. Tim Hardy & ECON 122 Tim attends hybrid course design workshop
One of six faculty funded by the Provost’s FA2009 pilot to deliver a 50/50 hybrid course.
·    Joined the SU2009 cohort of the Alternate Delivery Program (ADP)
·    Attended the January 2009 Hybrid Course Redesign Workshop.
·    His UMBC Blackboard ranking for student-only activity has increased dramatically:
FA2010 (1st overall, 2,970 average hits per student) * See note below *
SP2010 (1st overall, 1,345 average hits per student)
FA2009 (1st overall, 1,666 average hits per student)
SU2009 (4th overall, 842 average hits per student)
SP2009 (36th overall, 366 average hits per student)
FA2008 (532nd overall, 78 average hits per student)
SU2008 (38th overall, 334 average hits per student)
SP2008 (76th overall, 494 average hits per student)
FA2007 (151st overall, 326 average hits per student)
Note: His ECAC329 “Cost Accounting” course registered 5,536 avg. hits per student, 1st among all UMBC Bb courses (not just undergrad) for FA2010.
SP2011 (53.7%)
FA2010 (49.4%)
SP2010 (47.7%)
FA2009 (46%)

24. Grade Center Impact on Activity
Building & Scaling Analytic Capacity
1/30/2012
Grade Center Impact on Activity
We’ve also used student activity to identify and promote effective practices. For example, we’ve seen that courses using the grade center tend to generate more student activity than courses that don’t.
Research by Educause confirms that students value checking grades and access to practice quizzes more than any other function. But this can’t simply mean that faculty should assign and grade more student work. So, how do we scale up the feedback function?
Purdue University & Gardner Institute

29. Personal Analytics in Action

30. Can IT Architect User Choices?
A good book on how "choice architecture" creates default choices that are anything but neutral.
RECAP
Record
Evaluate
Compare
Alternative
Prices – or Performance

31. Demo & Practice
Bb Analytics for Learn (BA4L) Instance

32. Seminar Agenda Introductions Analytics and the maturity index
John Fritz and academic analytics at UMBC
Break
Margie Jantti and the Wollongong library cube
Celeste Schwartz and data informed decisions at MCCC
Closing Q & A and evaluations

33. Impact and effect Margie Jantti, University Librarian
A study on library use and student performance
Margie Jantti, University Librarian
University of Wollongong Library, Australia, 2012

34. There are many approaches to evaluating the value of libraries.
As an academic library, the focus is on the transformative power of information
There are many approaches to evaluating the value of libraries.

35. Limited focus on the transformative power of information
Typical measures
Satisfaction measures
Rankings
Contingency valuation
Information literacy assessment
Usage rates
Limitations
Scale, e.g. samples versus population
Subjective
Often one-dimensional
Limited focus on the transformative power of information

36. Problem statement I:
Does a student’s academic performance improve through the use of library information resources?

37. Problem statement II:
How do we know which students make little or no use of library information resources?

38. Libraries and other units produce lots of discrete data

39. The challenge was to build a relational database (or cube)
books
ereadings
databases
ebooks
student grades

40. Data sources for the Cube
Student data – PIU
Loans – snapshot exported weekly
Electronic resource usage – ezyproxy logs

42. Eresources: >Authentication logs (ezyproxy)
>Monitor use in 10 minute blocks (144 ten minute periods p/day)
>When used – database name captured
Other benefits
logs are updated weekly

43. Figure 1. Correlation between electronic resource usage and student grades

44. Electronic resources: ejournals, ebooks, ereadings etc
Undergraduate student resource usage (Faculty of Commerce) Table 1. Weighted Average Marks (WAM) for Undergraduate Students (Faculty of Commerce)
Electronic resources: ejournals, ebooks, ereadings etc

45. Is it perfect? >Some limitations in correlations
>Arbitary measures; business rules
>Many external factors affect grades, e.g. academic influence

46. But……. A first at UOW Library: >for true integration of data silos
>getting answers to our problem statements
>evaluating our communication and intervention strategies
>for a new way of demonstrating the value of the Library

47. QUESTIONS?

48. Seminar Agenda Introductions Analytics and the maturity index
John Fritz and academic analytics at UMBC
Break
Margie Jantti and the Wollongong library cube
Celeste Schwartz and data informed decisions at MCCC
Closing questions, comments, and evaluations

49. Data-Informed Decision-Making throughout the Institution
Celeste M. Schwartz, Ph. D.
Vice President for Information Technology and College Services

50. What is the Question. What Data can help answer the Question
What is the Question? What Data can help answer the Question? What Actions will be recommended based on the data?

52. Astra – Room Scheduling

53. Student Success
This office uses Dashboards as opposed to the Reports we have reviewed.

55. Enrollment Analysis

56. Help Desk Weekly Lab Usage Monday Tuesday
Central Library – 70% utilization at 12:00pm(noon) Wednesday 10/3/2012
West Library – 70% utilization at 11:00am Monday 10/2/2012
CH Tutoring – 30% utilization at 10:00am Monday 10/1/2012
West Library Laptops – 20% utilization at 7:00pm Monday 10/1/2012
Central Library Laptops – 20% utilization at 12:00pm(noon) Thursday 10/4/2012
Monday
Tuesday

57. Total number of tickets created: 456
Summary of Weekly calls by Status & Assignment.

59. Help Desk Feedback Report
Date
Representative
Problem Resolved?
Service Rating
Comments
10/1/2012 5:02:02 PM
robert gehring
Not provided
Yes
Very Satisfied
Representative was very helpful, and answered all my questions.
10/1/2012 6:43:32 PM
Robert Gehring
He was fast, polite and extremely helpful. Thank you.
10/2/2012 3:07:37 PM
Jasmyne Smith
10/3/2012 4:37:26 PM
Ryan Foster
Satisfied
Ryan did well to resolve my loss of available drives. It did take almost two full weeks to resolve this issue after first being reported on 9/20/12.
10/4/ :08:00 AM
Jasmin
10/4/ :55:31 AM
Jennifer
Great. Thanks
10/4/2012 1:44:50 PM
Jen
This is the third time I've contacted the help desk. On each occasion, you were EXTREMELY patient and helped me resolve the problem. Additionally, you do it WITHOUT making me feel like an idiot! Thank you!
10/5/2012 8:53:06 AM
Kathy Miller
Kathy is always willing to listen and ask questions until everyone is clear on what the difficulty/need is! Many, many thanks to Kathy and Mary Lou!
10/6/ :39:35 AM
Robert
Robert is remarkable and we are very fortunate to have him. Because it was so late at night when my computer crashed, he was able to talk me through re-booting so that I could finish the class and not waste time and inconvenience all my students. Sincere thanks,
Lee

60. What is the Question. What Data can help answer the Question
What is the Question? What Data can help answer the Question? What Actions will be recommended based on the data?

68. Now it’s Your Turn What Actions will you recommend based on the Data?
Read the daily enrollment from the VP for Student Affairs based on the daily updated warehouse data.
Take ten minutes with your team to discuss recommended actions
Each team will report out

69. Data-Informed Decision-Making throughout the Institution
QUESTIONS

70. Seminar Agenda Introductions Analytics and the maturity index
John Fritz and academic analytics at UMBC
Break
Margie Jantti and the Wollongong library cube
Celeste Schwartz and data informed decisions at MCCC
Closing questions, comments, and evaluations

71. Thank You. jbichsel@educause.edu fritz@umbc.edu margie@uow.edu.au