1 A new resource for the identification and utilization of E lectronic L earning A ssessment R esources Overview for CTAP Staff September 21, 2005 Presented by John Cradler Ruthmary Cradler Chris York

2 What is an ELAR? A technology application that simplifies delivery, aggregation, and disaggregation of assessment data It provides information that allows teachers and administrators to make informed decisions and to prepare instructional plans based on student data

3 1. Conduct Assessments 2. Obtain Relevant Data 3. Analyze Data 4. Determine Conclusions 5. Plan Instruction 6. Implement Instruction An ELAR should assist educators in conducting data-driven instructional planning

4 Why do we need an ELAR information service? Increased State and Federal instructional accountability requirements for reporting student outcomes. Increased emphasis on using student assessment data to inform district, school, and classroom instructional planning decisions. Technology applications or ELARs are rapidly emerging as a potential tool to enable data-driven decisions Educators need information on how to select and use ELARs that meet their local needs

5 How did the ELAR information service evolve? California Legislature was interested in enabling educators to make more meaningful and effective use of State testing data The legislature recognized that technology has the potential to contribute to more effective use of student testing data to plan instruction State Senator Jack Scott authored SB 1384 which authorized development of criteria, a review process, and a strategy for informing educators about how to select and utilize ELARs

6 How is the ELAR project collaboratively implemented? CLRN: Provides for the online database results of ELAR reviews, coordinates the reviews, and is the host SETS project for the ELAR information service. TICAL: Provided input on the design of the criteria and will assist in designing and conducting professional development and posting information relevant to ELARs on the TICAL website. TechSETS: Assists in the technical review of ELARS and provides input as an ELAR advising entity. Learning Resources: A contractor to CLRN who provides overall leadership and management for the design and implementation of the ELAR information resources.

7 Who are the other important collaborators? CTAP: The 11 CTAP Regions along with regional assessment staff will disseminate information and training on the use of the ELAR information resource. CDE Educational Technology Office: The contracting and overseeing agent for the ELAR component of CLRN is the CDE. CDE Office of Assessment: Played a major role in developing ELAR review criteria and is an active participant in the review of ELARs. ACSA and CUE: Both were supporters of SB 1384 and will be featuring conference workshops on how to use the ELAR information. CARET: The Center for Applied Research in Educational Technology provides research on the use of ELARs.

8 How were the Review Criteria Developed? Research was conducted to inform development of criteria. Draft ELAR Criteria were developed. ELAR Advisors and TICAL Cadre members provided Input. CDE Offices of Assessment and Technology provided Input. Draft criteria were field tested, revised, and finalized. CDE Approved the ELAR Review Criteria and related guidelines by the legislated date of

9 The ELAR Review Criteria: Three categories of minimum requirements for all ELAR applications have been established to cover the range of types of ELARs: Type I: Configured to import, process, and enable analysis of California State testing data. Type II: Not yet configured for California but can be modified to process and analyze California testing data. Type III: Does not use state testing data but can enable locally collected test and other data to inform local instructional planning. All ELARs for each of the three types must meet eight general requirements ranging from having correct spelling to enabling instructional planning.

10 The ELAR Review Criteria (continued): A.Sources of data: this can range from student grades to the CST B.Relevant background: data that could have on impact on student achievement C.Reporting & analysis of data: this ranges from the ELAR being able to enable analysis of data from the district to the classroom level, the same students over time, disaggregating by a variety of sub-groups, customization of reports for particular groups, charting of data, etc. D.Standards to guide ELAR design: this includes attention to both content standards as well as technical standards E.Customization of the ELAR: ranges from local modification of reporting to purchasing only parts of the ELAR needed

11 The ELAR Review Criteria (continued): F.General or other features: this ranges from inclusion of instructional planning templates to links to research-based strategies to address achievement gaps identified by the ELAR G.Support for using the ELAR: this addresses the availability of assistance from the publisher ranging from training to an online ‘help-desk’. H.Technical requirements: this includes bandwidth, hardware, needed technical support, security requirements etc. I.Development and research: a description of any research or evaluation conducted on the cost vs. the benefits or impact of the program on instructional planning

12 The ELAR Application and Review Process: A.The publisher matches the ELAR features and to the ELAR criteria as the major component of the ELAR Application. B.CLRN/ELAR staff arrange a review date, time and location. C.The review process begins with a publisher presentation of the resource to the reviewers. D.Reviewers try out the resource and test the features. E.Reviewers evaluate the publishers match of the ELAR to the criteria and look for areas of agreement/disagreement. F.When the review is completed the resource information is posted on the CLRN/ELAR website.

13 After the review, what is provided? A.ELAR Product Profile: A profile of each of the ELARs that were reviewed describing the target users of the resource, technical requirements, and specific training needs to effectively use the resource. B.Narrative Summary: A summary of the salient features of the program based on the results of the review as well as information on appropriate uses of the resource. C.ELAR Consumer Report Matrix: A matrix allowing for quick and detailed comparison of features identified in the review as a major or minor focus of the ELAR. D.ELAR Complete review: The detailed profile description and analysis of the ELAR including ratings and a narrative summary.

14 The ELAR Consumer Report Matrix: Sample ELAR Comparison Matrix

15 From ELAR criteria development to ELAR utilization 4. Design & conduct ELAR review training 1. Review research & existing ELAR criteria 7. Maintain & Expand the ELAR database 9. Disseminate ELAR information and training 10. Educators access, select & utilizeELARs 11. Evaluate ELAR Program development, implementation & impact & revise as needed 2. Establish & approve review Criteria (Dec 04) 3. Adapt CLRN & TICAL Websites for ELAR access 5. Publishers submitELARs for review 8. Design ELAR Professional Development 6. Conduct reviews of ELARs

16 In summary, the ELAR information resource offers: A one-stop web-based location to review data management systems based on approved criteria. State-approved criteria enabling educators to compare/ evaluate their current instructional management systems. Research and information links on effective use of data management systems in instructional decision making. Professional development through TICAL and CTAP on how to use the ELAR website, to utilize ELARs to inform data- driven decisions, and apply the ELAR review criteria to guide local analysis and selection of ELARs.

17 What are the information features still under construction? As soon as there are sufficient ELARs posted an ELAR ‘search feature’ will allow users to match ELARs with their own profile as is now done with CLRN ELR searches. In collaboration with the Center for Applied Research in Educational Technology (CARET), ELAR related research findings will be linked to the ELAR information site. Online video streamed demonstrations of how to use ELARs is planned for the ELAR component of the CLRN site. User feedback will be collected for each of the ELARs posted and then summarized and edited for inclusion on the ELAR database.

18 ELAR project partner contact information: The ELAR Project is a service of the California Learning Resource Network (CLRN) funded by the CDE through a contract with the Stanislaus County Office of Education Contact: Ellis Vance Contact: Michael Simkins Contact: John Cradler Bob Beuthel Contact Bill Simpson: