Jason Cole Consultant As presented at the Sakai Summer Conference 12 June 2007 | Amsterdam, Netherlands The public face of eLearning

The public view of higher education

Georgetown University The public perception

Georgetown University The press Federal Study Finds No Edge for Students Using Technology-Based Reading and Math Products

Georgetown University The U.S. Congress … the breach of trust between schools and students. There is an important relationship there that some schools, though certainly not all, have been far too cavalier with. Senator Robert P. Casey, 6 June 2007

Georgetown University The Spellings Commission And some [students] never complete their degrees at all, at least in part because most colleges and universities don’t accept responsibility for making sure that those they admit actually succeed. Many students who do earn degrees have not actually mastered the reading, writing, and thinking skills we expect of college graduates. U.S. Department of Education, 18 September 2006

Georgetown University But the Commission wrote “We recommend that America’s colleges and universities embrace a culture of continuous innovation and quality improvement. We urge these institutions to develop new pedagogies, curricula and technologies to improve learning, particularly in the areas of science and mathematics.”

And eLearning

Georgetown University U.S. Department of Education study of educational software “Congress posed questions about the effectiveness of educational technology and how effectiveness is related to conditions and practices.... On average, after one year, products did not increase or decrease test scores by amounts that were statistically different from zero.” As reported in the press: education technology doesn’t work.

Georgetown University Training: A comment The most important training [for eLearning users] is how to use the technology to achieve lesson objectives, not how to use the software. Training requires assistance during the early use of the software. Debra Sprague, Graduate School of Education, George Mason University, responding to questions about the study at the Blackboard Forum, National Press Club, Washington, DC USA 11 May 2007

Georgetown University Presidents on e-Learning “Based on his work with the University of South Australia and his conversations with presidents and financial officers, [Bill Becker] said there is a general belief that eLearning increases the cost of education. He said the cost of the distance learning courses at the University of South Australia exceed those offered in the classroom because of the amount of time that faculty spend responding to students.” “Access and Persistence Symposium,” September 8, 2005, Washington, DC

Publishers have taken the lead on reporting effectiveness of eLearning

Georgetown University Success in math courses

Georgetown University Use of supplementary material

Georgetown University Faculty believe

And now

Georgetown University Observations Evidence suggests eLearning appears to be most effective for first and second year students. Pedagogy is important to effectiveness of eLearning; “best practices” must be available to faculty. Methods of instruction must be adapted to student preparation, motivation, and constraints.

Georgetown University Evidence being used Improvements when eLearning is used: Retention Satisfactory completion Performance in sequent courses Graduation rates Scope of mastery of the subject Student engagement

Georgetown University More work “What works” - Colleges and universities should produce and document evidence of educational effectiveness of eLearning. Pedagogy - The key to effectiveness is guidance for faculty on the effective use of eLearning resources, not the learning system software

Georgetown University “Further study” Why are faculty turning to publisher’s online services instead of local eLearning services? Which eLearning services are most effective for which courses?

Thank You Jason Cole Cognition and Instruction Associates

Supplementary material

Georgetown University Evaluation of Education Technology: High School Algebra “Effectiveness of Reading and Mathematics Software Products: Findings from the First Student Cohort Report to Congress,” Institute of Education Sciences, U.S. Department of Education, March 2007.

Georgetown University Effectiveness of Reading and Mathematics Software Products U.S. Congressionally mandated report by the Institute of Education Sciences, U.S. Department of Education. Issued March 2007 First and fourth grade reading, sixth grade mathematics and high school algebra. Context: The Department again did not seek funding for educational technology.

Georgetown University Summary findings “Congress posed questions about the effectiveness of educational technology and how effectiveness is related to conditions and practices.... On average, after one year, products did not increase or decrease test scores by amounts that were statistically different from zero.” As reported in the press: education technology doesn’t work.

Georgetown University Study findings “Nearly all teachers received training and believed the training prepared them to use the products.” “Technical difficulties using products mostly were minor.” “When products were being used, students were more likely to engage in individual practice and teachers were more likely to facilitate student learning rather than lecture.”

Georgetown University Warning These data are based on teaching high school algebra and would not be representative of other subjects, levels of instruction, or students with different characteristics.

Georgetown University Training Teachers received about 12 hours of training, including practice using the software. At the end 81% were “confident they were prepared to use the product” By the time of the first classroom observation, only 66% considered themselves prepared to use the software.

Georgetown University Training: A comment The most important training is how to use the technology to achieve lesson objectives, not how to use the software. Training requires assistance during the early use of the software. Debra Sprague, Graduate School of Education, George Mason University, responding to questions about the study at the Blackboard Forum, National Press Club, Washington, DC USA 11 May 2007

Georgetown University Use of the software

Georgetown University Test Results

Georgetown University Cost of software Software provides tutorial, practice, and assessment opportunities. Average licensing fees about $15 per student for the school year; a range of $7 to $30. [Teachers reported] students used the software and average of 118 minutes per week for 23 weeks or 46 hours (of 180 hours).

Georgetown University Impact on classroom activities