1. Evaluation & MathDL Flora McMartin Broad-based Knowledge flora.mcmartin@gmail.com MathDL Workshop, October 2008

2. What We Know about Math Users National survey of faculty (4,678 respondents, 242 Math) MathDL Workshop, October 2008 Institution Type

3. MathDL Workshop, October 2008 Types of Digital Resources Type of Resource % VF Online Scholarly Resources19 Digital Images18 Teaching/learning Exercises29 Data Sets11 Animations/simulations15 % of respondents reporting Very Frequent (VF) use (top box score)

4. MathDL Workshop, October 2008 Most Popular uses of Digital Resources Type of ResourceMath Online Scholarly Resources Prof. Dev. as Teacher Grants, Scholarship Digital Images Student Study Aid/Review Prof. Dev. as Teacher Lecture Teaching/learning Exercises Student Study Aid/Review Posted on course website Data Sets Student Research/PBL Prof. Dev. as Teacher Lecture Animations/simulation s Student Study Aid/Review Lecture

5. MathDL Workshop, October 2008 Importance of DL Features Type of Feature% CONTENT Supplemented with materials to explain use in teaching17 Supplemented with materials to use for professional development as a teacher 6 PEDAGOGY Peer reviewed & high quality34 Organized to find materials quickly45 Percentage of respondents who ranked feature as most important

6. MathDL Workshop, October 2008 Motivations for Using Digital Resources MotivationMean Improve students’ learning3.83 Stay abreast of professional developments3.79 Helps me keep fresh3.78 Incorporating digital resources in class is fun3.62 Saves me time3.38 Helps me develop my teaching skills3.29 Students expect me to use digital resources in their instruction2.63 Help me better accommodate students with disabilities2.74 Helps students learn difficult concepts2.73 The usefulness of digital resources are overrated2.73 Scale: 1 = strongly disagree, 5 = strongly agree

7. In Sum MathDL Workshop, October 2008 No direct relationship between highly valuing educational digital resources and level of use Faculty prefer search for finding materials Barriers to use cannot be simply described

8. Overcoming Barriers to Use of DLs Activation energy – how much? What is it? Intrinsic motivation is a powerful force – how big a reservoir? Students as extrinsic motivators Tradition is another powerful driver (change is hard) All work practices are different Never underestimate the willingness of a motivated individual to do their own thing (or ignore you) MathDL Workshop, October 2008

9. Application to the MathDL Do these findings apply to MAA members? Study done in 2006, how have changes on Web since then (e.g., growth in popularity of Youtube, iTunes U, blogs, wikis, etc.) affected users? MathDL and MAA sites have changed what does that mean to users? And? MathDL Workshop, October 2008

10. Evaluation, next steps Need to understand user needs, preferences and behaviors Are they aware of it? Can they find it? If they find it, will they use it? What does it allow users to do that they could not do before? Need to understand the environment and competition to Act strategically Develop marketing plans Make strategic partners Measure success and affordability (know when experiments work or fail) MathDL Workshop, October 2008

11. User Panels Panels repre MathDL Workshop, October 2008 WHY? Users becoming more sophisticated and fickle –online survey response rates trending down Ability to track users’ attitudes, behaviors over time Panels carefully selected to match demographics of audience 20 + panelists Respond to 3 – 5 surveys over course of year Potential Questions Usability – e.g., is the site intuitive? How do users move between MAA and MathDL? How does use of Google work for/against use of MathDL?

12. The Plan Selection Criteria Teaching Experience Types of School/institution Type of course (online, traditional) Use of Web/DL resources (professional development, integrated into course) Math Course Topics (Calc, remedial math, etc.) 2 panels Novice Previous users of MathDL Implemented in 2009 MathDL Workshop, October 2008

13. More about Research on Faculty Use of the NSDL For more details visit http://serc.carleton.edu/facultypart We wish to acknowledge the National Science Foundation for their support (DUE-0435398). The findings presented here do not necessarily represent the views of the NSF. Researchers: Alan Wolf, University of Wisconsin, Madison Josh Morrill, Morrill Solutions Research Glenda Morgan, George Mason University Ellen Iverson, Carleton College Cathy Manduca, Carleton College Flora McMartin, Broad-based Knowledge MathDL Workshop, October 2008