An iLearn Math Program Evaluation Plan An iLearn Math Program Evaluation Plan Claudia Lopez OLIT 508 Barbra Portzline Fall 2012.

Slides:



Advertisements
Similar presentations
Ventura Community College, California Title V Basic Skills Grant ( ) Title V Transfer Velocity Grant ( ) Kathy Scott, Dean of Institutional.
Advertisements

Introduction to Monitoring and Evaluation
Action Research Not traditional educational research often research tests theory not practical Teacher research in classrooms and/or schools/districts.
M & E for K to 12 BEP in Schools
Team 6 Lesson 3 Gary J Brumbelow Matt DeMonbrun Elias Lopez Rita Martin.
Research & Evaluation Athens Workshop Community Learning Ambassadors 12 th March 2014.
SEM II : Marketing Research
Determining the Validity and Reliability of Key Assessments and Other Questions Regarding Key Assessments.
Teresa Ryerse Overton.  Suburbs of Washington DC  5 Campuses and a separate Medical Education Campus  78,000 Students  2,600 Faculty and Staff  ~8,000.
ISOTURE: A Model for Volunteer Management
Chapter Three: Determining Program Components by David Agnew Arkansas State University.
Title I Needs Assessment and Program Evaluation
Marketing Research Unit 7.
Toolkit Series from the Office of Migrant Education Webinar: CNA Toolkit August 21, 2012.
Strategic Research CHAPTER 6. 2 Strategic Research Strategic research –Information gathering process that enhances the design at a creative strategy level.
Creating Research proposal. What is a Marketing or Business Research Proposal? “A plan that offers ideas for conducting research”. “A marketing research.
The Promise of Developmental Summer Bridge Programs Elisabeth Barnett, Thomas Bailey, and the NCPR Team IES Conference June 2010.
Accountability Assessment Parents & Community Preparing College, Career, & Culturally Ready Graduates Standards Support 1.
Final Client Briefing Call Center Wait Time Project August 27, 2012 – December 11, 2012 State of Florida – Department of Economic Opportunity Florida State.
AET/515 Spanish 101 Instructional Plan SofiaDiaz
E-POSTER PRESENTATION MPhild/Phd
5.04 Discuss the Consumer Research Process. Consumer Research  Consumer research is used to gather information in order to know what consumers want and.
Evaluation 101 Everything You Need to Know to Get Started Evaluating Informal Science Education Media Everything You Need to Know to Get Started Evaluating.
FORMATIVE EVALUATION Intermediate Injury Prevention Course August 23-26, 2011, Billings, MT.
Redesign of Beginning and Intermediate Algebra using ALEKS Lessons Learned Cheryl J. McAllister Laurie W. Overmann Southeast Missouri State University.
Questionnaires and Interviews
Data Analysis for Evaluation Eric Graig, Ph.D.. Slide 2 Innovation Network, Inc. Purpose of this Training To increase your skills in analysis and interpretation.
CENTER FOR NONPROFIT EXCELLENCE EVALUATION WORKSHOP SERIES Session IV: Analyzing and Reporting Your Data Presenters: Patty Emord.
ISTEP: Technology Field Research in Developing Communities Instructor: M. Bernardine Dias CAs: Sarah Belousov and Ermine Teves Spring 2009.
Central Virginia Community College Where your future begins.
ISU Phase 4 – Primary Research Proposal. The Primary Research Proposal Phase will be handed in (along with Phase 3) as part of your final research paper.
MEGAN LOMBARDI STAFF DEVELOPMENT WEEK JANUARY 13, 2015 Why do Students Leave CLC? Results from the Fall 2014 Survey of Withdrawn Students.
Operation STEM Cleveland State University February 22, 2014.
Additional Unit 2 Lecture Notes New Instructional Design Focus School of Education Additional Unit 2 Lecture Notes New Instructional Design Focus School.
Case Study #9 Jackie Adams: Evaluating a Federally Funded Faculty Training Program Melody Murphy AIL 606 The University of Alabama.
Fast Track to Accelerate Student Success 1.What Fast Track is 2.Strategies for Implementation a) Pre-Production b) Implementation c) Post-Production.
Elizabeth Godfrey 1.  Periodic assessment of results Appropriateness, effectiveness, efficiency, impact, sustainability  Identifies intended and unintended.
EVALUATION RESEARCH. Evaluation Research  How do we begin?  What are the different types of evaluation research?  How do these different types fit.
College Algebra: An Overview of Program Change Dr. Laura J. Pyzdrowski Dr. Anthony S. Pyzdrowski Dr. Melanie Butler Vennessa Walker.
 How do we begin?  What are the different types of evaluation research?  How do these different types fit together?  What purpose do they serve?
Assessing Organizational Communication: Strategic Communication Audits Chapter 3 Conducting Team Audits.
Textbook Transformation Grants R1: Final Report Review Lauren Fancher, Director, Affordable Learning Georgia Jeff Gallant, Visiting Program Officer for.
Market Research The key to the customers wallet …..
Instructional Plan | Slide 1 AET/515 Instructional Plan December 17, 2012 Kevin Houser.
MDC Quality Enhancement Plan Mathematics Discipline Meeting Update March 5, 2009.
Promoting Student Success Across the Curriculum Spring 2011 Pilot Program.
EDU 385 CLASSROOM ASSESSMENT Week 1 Introduction and Syllabus.
Focus ABCI Evaluation Possible Threats to Integrity Variations in age of students may result in skewed, invalid, or unreliable data. Agendas of participants.
The Role of the Teacher. Basically, the state believes that teachers, along with the school district, are responsible for students that fail. If a student.
Strategic Research. 6-2 Chapter Outline I.Chapter Key Points II.Research: The Quest for Intelligence and Insight III.The Uses of Research IV.Research.
Danny Pittaway Student Success Coordinator, Coastline Community College Learning Assistance Project, 3CSN 10/12/2015.
MAP the Way to Success in Math: A Hybridization of Tutoring and SI Support Evin Deschamps Northern Arizona University Student Learning Centers.
Gateway to Graduation Dr. Sundar, Dr. Wu, Dr. Jóhannsdóttir October 23, 2015.
Niagara County Community College Sanborn, New York Introduction to Statistics Annual Enrollment Sections/semester ~30 students/section Traditional.
1 The project is financed from the European Union funds within the framework of Erasmus+, Key Action 2: Cooperation for innovation and the exchange of.
Strategic Research. Holiday Inn Express Stays Smart What research results led to an upgrade of all Holiday Inn Express bathrooms? How did their agency,
Fidelity of Implementation A tool designed to provide descriptions of facets of a coherent whole school literacy initiative. A tool designed to provide.
Action Research Not traditional educational research often research tests theory not practical Teacher research in classrooms and/or schools/districts.
Transition Improvement Plan (TIP) Introduction to the Predictor Rubric Transition Improvement.
Sports Market Research. Know Your Customer How do businesses know their customers needs and wants?  Ask them/talking to customers  Surveys  Questionnaires.
SEM II : Marketing Research 1.06 Collect secondary marketing data to ensure accuracy and adequacy of information for decision making.
Presentation on Outcomes Assessment Presentation on Outcomes Assessment toCCED Mohawk Valley Community College October 11, 2004.
Midterm Report Presenter: Eunice Lai Instructor: Patricia Su Date: 19 th April, 2012.
Christopher Jacobson Frederick Community College Cindy Nicodemus Howard Community College.
Russell & Jamieson chapter Evaluation Steps 15. Evaluation Steps Step 1: Preparing an Evaluation Proposal Step 2: Designing the Study Step 3: Selecting.
Training processes for extension education
Bell Ringer List five reasons why you think that some new businesses have almost immediate success while others fail miserably.
Perspective Interview: Sofia Perez
Chapter 4 Instructional Media and Technologies for Learning
Presentation transcript:

An iLearn Math Program Evaluation Plan An iLearn Math Program Evaluation Plan Claudia Lopez OLIT 508 Barbra Portzline Fall 2012

Background The Highway to Success Learning Communities is at the UNM-Valencia Campus. We help students transition into College level English & Math courses. We help all students that tested low in English and/or Math in their Compass or ACT tests.

Rationale New self-paced math program (iLearn) implemented Summer Most of our Math classes now are self-paced iLearn classes. Some students in these classes are not getting through the material in a timely manor therefore leaving them frustrated and not able to pass the final. Instructors in these iLearn classes do not lecture, but are there along with a tutor for support or questions. iLearn program has many glitches, and errors.

Purpose of This Evaluation To find out how effective the iLearn program is. To find out if this program should be continued or discontinued. Results will be used to improve our Math Department.

Primary Stakeholders iLearn Program Designers: Directly impacted by results whether to keep iLearn program or the need to improve it. Highway to Success Program Manager: Makes final decision as to continue or discontinue iLearn.

Secondary Stakeholders Instructors: Have a vested interest to make sure that students pass the final. Highway to Success Tutors: Vested interest in making sure that students understand the concepts in order to pass. Students: Have a vested interest since they have to pass the class in order to keep their scholarships, etc.

Tertiary Stakeholders Other learning institutions: May be interested in implementing iLearn program. Community members: May be considering taking a self-paced Math course.

Key Questions Key questions in this evaluation will focus on finding out if there is a need to continue using the iLearn program. Questions will help us understand the effect that the program has on the students as far as retention and being able to continue on to the next level math class. These questions may also help the instructors in planning supplemental worksheets, and/or study guides to help the students understand concepts.

The Key Questions In what way(s) is the iLearn math program making a difference at the Highway to Success? How do the instructors make sure that the students are learning the concepts that are needed to pass the final? To what extent of computer knowledge should the students have in order to use the iLearn program? Have the pass rates improved or declined since iLearn was implemented in Summer 2011? If pass rates declined, what can be done to improve them? If pass rates improved, how can we keep the rates high?

The Key Questions Continued Students will often minimize the iLearn program by getting on Facebook or other social media. How can we make sure that the time being tracked on iLearn is actually being spent on that program? What learning styles is the iLearn program targeting, and how is this being done? How effective are the tutors in helping to understand the instruction being presented in the iLearn program? Would a special training for math instructors and tutors be effective and in what ways?

Evaluation Design This evaluation is meant to be formative. Conducted for the purpose of either improving the iLearn program or discontinuing it. The evaluation will be conducted by one of the Math instructors along with a Master tutor. Results will shared with all Highway to Success Math instructors as well as the program manager. Existing data will be used from previous semesters’ iLearn classes, and instructor taught classes. Students will fill out surveys using mostly open-ended questions. Evaluations will be given to students in the 10 th -12 th week of the semester. Interviews will be conducted with the instructors & tutors with questions similar to the students.

Evaluation Design Continued Two models that will be used include 2 levels. Level 1 will be used to measure the reactions of the students in regards to the information that is being retained with the iLearn program. A Level 4 evaluation will be used to measure the effectiveness of the iLearn program in relation to how well it works for the department as well as the instructors. This design will include the case study design using Kirkpatrick’s approach. We will be using both qualitative and quantitative data in order to answer the key questions. The main key question that needs to be answered is if iLearn is helping students pass, or are they failing because they are not understanding the curriculum the iLearn program provides? Instrument validity will be important in this evaluation.

Data Analysis – Qualitive Interview Questions: Department Manager will analyze questions carefully & decide how to categorize the results. The results will then be put into different categories according to the key questions. Each category will be coded, and the frequency of the codes will be counted. Data will be displayed using tables, matrices, charts and diagrams. Survey Questions: Instructors and Department Manager will analyze survey questions. Data will be carefully categorized and coded. This data will be compared to the interview questions as well as the archival data. The frequency of these codes will be counted as well. This data will be displayed in tables, charts and diagrams.

Data Analysis – Quantative Archival Data: Pass rates from previous semesters will be compared to the survey and interview questions. Pass rates will help the math department decide if the iLearn program is improving pass rates compared to the traditional instructor led math class Data will be analyzed carefully by the Department Manager, the Administrative assistant, and the math instructors. Data will be recalculated to make sure that there is an accurate pass rate percentage being revealed.

Evaluation Issues & Constraints Monetary: Highway to Success has lost their Title V grant & is now running under UNM-Valencia’s budget. Since Instructor & tutor are doing the evaluation there will not be an extra charge, regular pay will be given. The only charges will be paper, envelopes, and stamps as needed. Time: Evaluation will be performed a few weeks before finals, instructors might not be willing to give students time for filling out surveys. Students might not be willing to fill out surveys due to being behind. Instructor might not have time to do interview. For this reason interviews may be done over the phone or on Skype.

Evaluation Issues & Constraints Continued Attitudes: Some instructors do not want to be bothered with interviews, or evaluations even when it comes to the iLearn program. Students attitudes might come from the frustration with the iLearn program, and might not want to fill out the lengthy survey.

Communication & Reporting of Findings Primary Stakeholders: Will get ongoing reports, either by phone or mail, or ongoing meetings as needed as well as the final report. iLearn Program Designers: Will receive ongoing s as needed, so they may make improvements as needed, they will also receive a final report. Secondary and Tertiary Stakeholders: Will be able to see a final report presented as a PowerPoint presentation with graphs on the Highway to Success website. Instructors will see the final report in a meeting with a PowerPoint presentation.

Evaluation Budget

Bibliography Russ-Eft, D., & Preskill, H. (2009). Evaluation in Organizations: A Systematic Approach to Enhancing Learning, Performance, and Change. New York, NY, USA: Basic Books. Images: