University of Wisconsin-Madison N ATIONAL P ARTNERSHIP FOR A DVANCED C OMPUTATIONAL I NFRASTRUCTURE Assessment & Instructional Technology for College Level STEM Instructors NISE FLAGILT Field-Tested Learning Assessment Guide Eileen Lewis, UC Berkeley Institute on Learning Technology Jean-Pierre Bayard, CSUS National Institute for Science Education Click on the Buttons College Level 1

University of Wisconsin-Madison N ATIONAL P ARTNERSHIP FOR A DVANCED C OMPUTATIONAL I NFRASTRUCTURE What is NISE ? Click here to return: The N ational I nstitute for S cience E ducation was funded in 1995 for five years as a cooperative agreement. NSF directed the Institute to "address the totality of the [science] education enterprise, to assess its effectiveness, and examine what new activities need to be established, what activities are no longer needed, and what new approaches will enhance science education." NISE is based at the University of Wisconsin-Madison and Washington, DC, through its partnership with the National Center for Improving Science Education. The NISE's vision is that all students leave the educational system with an ability to make informed decisions about STEM-related matters that they encounter in their daily lives. In this vision, all stakeholders in STEM education engage in active, contextual learning to acquire both a strong foundation in STEM and the ability to enhance that foundation as lifelong learners. The N ational I nstitute for S cience E ducation was funded in 1995 for five years as a cooperative agreement. NSF directed the Institute to "address the totality of the [science] education enterprise, to assess its effectiveness, and examine what new activities need to be established, what activities are no longer needed, and what new approaches will enhance science education." NISE is based at the University of Wisconsin-Madison and Washington, DC, through its partnership with the National Center for Improving Science Education. The NISE's vision is that all students leave the educational system with an ability to make informed decisions about STEM-related matters that they encounter in their daily lives. In this vision, all stakeholders in STEM education engage in active, contextual learning to acquire both a strong foundation in STEM and the ability to enhance that foundation as lifelong learners. Funded by the National Science Foundation:

University of Wisconsin-Madison N ATIONAL P ARTNERSHIP FOR A DVANCED C OMPUTATIONAL I NFRASTRUCTURE College Level 1 Team Click here to return: Introductory college-level science, technology, engineering and mathematics (STEM) courses act as curriculum "pressure points"—they shape student career trajectories and influence attitudes toward science. The College Level One (CL-1) Team aims to enhance the quality of teaching and learning in these introductory STEM courses. Its research identifies and validates effective practices in these courses, and the team works with the Interacting with Professional Audiences Team to disseminate this information to key audiences. CL-1 activities are based on an annual Institute that focuses on a single key issue affecting college STEM courses. The Institute brings together several NISE Fellows to identify an agenda that helps address critical aspects of the issue of interest and to develop materials of value to college STEM instructors and administrators. Introductory college-level science, technology, engineering and mathematics (STEM) courses act as curriculum "pressure points"—they shape student career trajectories and influence attitudes toward science. The College Level One (CL-1) Team aims to enhance the quality of teaching and learning in these introductory STEM courses. Its research identifies and validates effective practices in these courses, and the team works with the Interacting with Professional Audiences Team to disseminate this information to key audiences. CL-1 activities are based on an annual Institute that focuses on a single key issue affecting college STEM courses. The Institute brings together several NISE Fellows to identify an agenda that helps address critical aspects of the issue of interest and to develop materials of value to college STEM instructors and administrators.

University of Wisconsin-Madison N ATIONAL P ARTNERSHIP FOR A DVANCED C OMPUTATIONAL I NFRASTRUCTURE The Institute on Learning Technology The NISE brought together a group of folks with experience in teaching, computer –based technology and assessment in order to create a resource for instructors teaching in STEM. - Specific mission is to Bridge the Chasm between educational research and STEM instruction Marco Molinaro UC-Berk. Flora McMartin UC-Berk. John Jungck Beloit Coll Steve Ehrmann TLT Susan Millar UW-Mad Jean-Pierre Bayard CSUS NISEFLAGILTBEGIN

University of Wisconsin-Madison N ATIONAL P ARTNERSHIP FOR A DVANCED C OMPUTATIONAL I NFRASTRUCTURE Goals of the ILT To develop a web site and other products that will enable STEM instructors to: Understand why they should use how to use them and how to overcome frequently encountered obstacle. Shape the powerful emerging technologies in learning tools that better prepare college students for the 21st century Provide all their students more experience with, and a better appreciation for STEM as a living enterprise. PROBLEMS ACTIVITIES OUTCOMES Click on Buttons Learning Technologies NISEFLAGILTBEGIN

University of Wisconsin-Madison N ATIONAL P ARTNERSHIP FOR A DVANCED C OMPUTATIONAL I NFRASTRUCTURE Problems Click here to return: The STEM Instructors usually face a set of challenges that motivate them to consider other learning methods.. The ILT is interested in documenting these problems because they represent a unifying platform where instructors visiting our site will recognize specific aspects of their own learning environment. PROBLEMS ACTIVITIES OUTCOMES Click on Buttons Learning Technologies

University of Wisconsin-Madison N ATIONAL P ARTNERSHIP FOR A DVANCED C OMPUTATIONAL I NFRASTRUCTURE Activities Click here to return: Technology for technology's sake is not of interest to us. The ILT seeks to feature activities that are expressly designed to enhance learning, with preference for those that are student-centered and where the use of computer-based technology has a clear and measurable effect on learning. PROBLEMS ACTIVITIES OUTCOMES Click on Buttons Learning Technologies

University of Wisconsin-Madison N ATIONAL P ARTNERSHIP FOR A DVANCED C OMPUTATIONAL I NFRASTRUCTURE Learning Technologies Click here to return: The focus of this research effort is on the instructional use of computer-based technology ranging from to high- performance computing. In all cases, the use of technology has to be incorporated in activities supportive of national principles of good teaching practices. PROBLEMS ACTIVITIES OUTCOMES Click on Buttons Learning Technologies

University of Wisconsin-Madison N ATIONAL P ARTNERSHIP FOR A DVANCED C OMPUTATIONAL I NFRASTRUCTURE Outcomes Click here to return: How do you know it works? The culture of educational institutions is one where traditions are golden and changes are difficult to implement. For learning technologies to appeal to reform-ready STEM instructors, one needs to provide clear evidence of enhanced student learning. PROBLEMS ACTIVITIES OUTCOMES Click on Buttons Learning Technologies

University of Wisconsin-Madison N ATIONAL P ARTNERSHIP FOR A DVANCED C OMPUTATIONAL I NFRASTRUCTURE ILT Research Methodology Data Gathering Phase Analysis of data Dissemination of products  Develop a set of potential cases through phone interviews  Use criteria to select some 10 practices for full case studies  Interview the selected bricoleurs, administrators, students, and support personnel  Develop a set of potential cases through phone interviews  Use criteria to select some 10 practices for full case studies  Interview the selected bricoleurs, administrators, students, and support personnel  Transcribe interviews  Write case studies  Use software to elicit common themes  Use the phone interviews to create short vignettes  Transcribe interviews  Write case studies  Use software to elicit common themes  Use the phone interviews to create short vignettes NISEFLAGILTBEGIN

University of Wisconsin-Madison N ATIONAL P ARTNERSHIP FOR A DVANCED C OMPUTATIONAL I NFRASTRUCTURE Goals of the FLAG NISEFLAGILTBEGIN To offer a readily accessible, up-to-date resource of classroom-tested assessment tools for instructors who have an interest in sharing and implementing new approaches to evaluating student learning, attitudes and performance. Techniques and tools are developed, tested and refined in real classrooms / laboratories at colleges and universities, and reviewed by an experienced Editorial Board of recognized experts. K. Patricia Cross UC- Berkeley Eileen L. Lewis UC- Berkeley Elaine Seymour Univ. of Colorado Dorothy Gabel Indiana Univ. Terry Millar UW-Madison Gloria Rogers Rose Hulman BOARDBOARD

University of Wisconsin-Madison N ATIONAL P ARTNERSHIP FOR A DVANCED C OMPUTATIONAL I NFRASTRUCTURE FLAG Content Matching your Goals to CATS Classroom Assessment Techniques (CATS) Searchable Tools Resources NISEFLAGILTBEGIN GOALS to CATS CATS SEARCH TOOLS RESOURCES Click on Buttons

University of Wisconsin-Madison N ATIONAL P ARTNERSHIP FOR A DVANCED C OMPUTATIONAL I NFRASTRUCTURE Matching Goals to CATS Click here to return: Reflect on your course goals Attitudinal Survey Concept Tests Concept Maps Conceptual Diagnostic Tests Interviews Performance Assessments Portfolios SALG (Student Assessment of Learning Gains) Scoring Rubrics Weekly Reports

University of Wisconsin-Madison N ATIONAL P ARTNERSHIP FOR A DVANCED C OMPUTATIONAL I NFRASTRUCTURE Classroom Assessment Techniques CATS Click here to return: Description Limitations Suggestions for Use Variations Pros & Cons Author’s Story Overview Purpose Teaching Goals Step-by-Step Analysis Theory & Research Links Sources Features of each CAT

University of Wisconsin-Madison N ATIONAL P ARTNERSHIP FOR A DVANCED C OMPUTATIONAL I NFRASTRUCTURE Searchable Tools Click here to return:  Discipline (chemistry, engineering)  Purpose (knowledge, skills, attitude)  Specific Technique (Conceptual Diagnostic)  Download  html  pdf  Word 6/ Mac  Word 97, WordPerfect 7/ PC

University of Wisconsin-Madison N ATIONAL P ARTNERSHIP FOR A DVANCED C OMPUTATIONAL I NFRASTRUCTURE Resources Click here to return:  Annotated bibliography that describes assessment practices in SMET courses  Related web sites on assessment practices  Consulting with someone at the NISE on assessment practices in my own course(s).  A comprehensive list of experts

University of Wisconsin-Madison N ATIONAL P ARTNERSHIP FOR A DVANCED C OMPUTATIONAL I NFRASTRUCTURE Future of ILT & FLAG ILT with a UC Berkeley and CSUS connection Update and generate cases & vignettes as technology changes A national moderated forum on technology ILT partners up with NPACI & NSF Training workshops FLAG has moved to the University of New Mexico under Michael Zeilik – Collaboration with NISE Funding through NSF is pending NISEFLAGILTBEGIN