1. Just Built It and They Will Come. Right?

2. How do we support teachers and students to use and return to our Digital Library sites? How do we increase the number of students visiting and returning to DL Speakers from mathematics, chemistry and community college digital libraries will offer their opinions. – Meet teachers where they are. Identify their needs and skills. –Engage and support students so they want to return.

3. Customizing NSDL Resources Beverly Park Woolf, Ivon Arroyo University of Massachusetts/Amherst, U.S.A Bev@cs.umass.edu Steve Weimar Math Forum, Drexel University Steve@mathforum.org Supported by National Science Foundation Due 0532776 ‘Customizing Resources for NSDL,’ and HRD 01-20809, “AnimalWorld: Enhancing High School Women’s Mathematical Competence.”

4. Importance of customization in learning Engage and scaffold students so they want to return to the site Students with different cognitive and affective levels should be taught differently. Underrepresented minorities and women need more help in mathematics Students in poorly performing schools need coaching and tutoring The access to and quality of online resources needs to be improved

5. Agenda Importance of customization in learning Research Activities Built three versions of the system Evaluated learning gain and student affect Evaluation results

6. Research Activities Are students learning? Does customization scaffold learning Built three versions of the Math Forum Site 1. Customized Active MF 2. Façade MF 3. Base Level MF

7. Triangle problems are color-coded (easy, just right, difficult) based on student skills and current performance. Intelligent agent monitors student performance. Difficult Easy Just Right 1. Customized Active Math Forum

8. Line and Angle problems are also color coded and updated based on student performance. Customized Active Math Forum Easy Just Right

9. Hints are available for every problem. Customized Active Math Forum Several Hints

10. Related and simple examples are linked to serve as training material. Customized Active Math Forum A B C D Related Examples

11. Training problems have graduated help. These problems are not the same as the original problem. They are simpler. Customized Active Math Forum Training problems have help.

12. Fa ç ade has the same interface as Customized but without the smart customization. This version distinguishes between the effect of shallow changes (e.g., color coding, additional help) and the impact of intelligent customization. 2. Facade Math Forum

13. Base Level is very close to the existing Math Forum website. Students see a large variety of problems with only a gross ranking of difficulty and very little information about the skills covered in an individual problem. 3. Base-Level Math Forum

14. Student learning was measured as pre-and post tests. Student affect (like and dislike of system, confidence in their own ability) was measured with survey questions after use of the tutors. Evaluation Results

15. Learning with Math Forum Three versions of the system were evaluated. Learning gains were high in all three versions.

16. Learning gain with Math Forum

17. Customized version Students attitude changes “Does Math Forum treat people as individuals”

18. “Did you learn with Math Forum?” Customized version

19. Customized version “Math Forum is effective in helping me learn to solve problems”

20. Customized version Would you come back to Math Forum? How many times?

21. How seriously did you work through problems? Customized version

22. Discussion and Future Work We have evidence that customization supports students to: – feel better about their experience with a digital library – want to return to the digital library. Customization can be applied to many digital libraries-- Works for well-defined domains (physics, chemistry) More difficult for social sciences, history, arts We are working on a second evaluation to determine whether customization: –improves student interaction with Digital Libraries –differentially impacts students with different cognitive and affective levels – helps support underrepresented minorities and women