Developing items and assembling a test based on curriculum standards EunYoung Lim(P.h.D) Korea Institute for Curriculum and Evaluation

List of Contents Overview of Curriculum Assessment Purposes of educational assessment Test & Item Link between curriculum and assessment Standards Characteristics of achievement standards Structure of standards-Example 1-English Example 2- English & Example 3-Mathmatics Principles of developing achievement standards Procedures of developing achievement standards Roles of achievement standards for assessment Assembling a Test Example 4, Example 5 References

Overview of Curriculum Systematic educational plans including the goals, contents, methods, and evaluation for teaching and learning The process of learning: textbooks, subjects, learning contents, learning order, materials Teacher What to teach? -Knowledge, Contents of subjects etc. Student What to learn and what to experience: all the experiences in school

Assessment Assessment: A systematic process to measure or evaluate the characteristics or performance of individuals, programs, or other entities, for purposes of drawing inference, sometimes used interchageably with test. Educational Assessment: A systematic process to measure and evaluate the learning outcomes of students, the performance of teachers and school, and the quality of curriculum to monitor and assure the quality of education.

Purposes of educational assessment Students Identify student’s strengths and weaknesses Teachers Assess the effectiveness of a particular teaching strategy To improve the Quality of Education School Assess the effectiveness of curriculum and programs Mystery or Offices of Education Provide data that can assist in decision making Monitoring

Test & Item Test: An evaluative device or procedure in which a systematic sample of a test taker’s behavior in a specified domain is obtained and scored using a standardized process. Item: A statement, question, exercise, or task on a test for which the test taker is to select or construct a response, or perform a task.

Link between Curriculum and Assessment Standards Test Knowledge, skills, attitude that students should learn throughout their education Item

What to evaluate of student's’ learning outcomes? Standards Standards What to evaluate of student's’ learning outcomes? What to teach and learn? Teaching Assessment

Characteristics of achievement standards Same as “standards “ Standards of the national curriculum in Korea are named as “Achievement Standards” Evidences of teaching and learning, evaluation Reflect key concepts and skills in each subject Embodies subject-specific competency Demonstrate what students can do as a learning outcome Emphasize application of knowledge and use of processes Becomes a basis for performance-based assessment and teaching for understanding at a school or classroom level

Structure of standards Contents of Curriculum Achievement Standard 1 Achievement Level 1 Achievement Level 2 Achievement Level 3 Achievement Standard 2 Achievement Level 1 Achievement Level 2 Achievement Level 3

Example1: English G3~4: Listening Achievement standards Achievement levels Contents of curriculum Level 1-High Level 2-Mid Level 3-Low

Example2: Grade 5~6-English-Listen Listen to a conversation between two students and select a place and time to meet

Example2: Grade 5~6-English-Listen Content of Curriculum Listen and understand a simple conversation in telephone Achievement Standards EG6132-Can listen a simple conversation in telephone and identify the purposes and contents of a conversation

Example2: Grade 5~6-English-Listen Achievement Standards Achievement Levels EG6132-Can listen a simple conversation in telephone and identify the purposes and contents of a conversation Level1(High)- can identify the purposes and contents of a conversation. Level2(Middle)- can identify the most of the purposes and contents of a conversation after repeated listening. Level3(Low)- can identify a part of the purposes and contents of conversation after repeated listening.

Example: Grade 5~6-English-Listen Information of an item Grade Grade 5-6 Area Listening Achievement standard EG6132-Listen the simple conversation in telephone and identifying the purpose and content of conversation. Type of item Multiple-choice item Answer 3

Example3: Grade 5~6-Math-Geometry Calculate the area of the following rectangle and square

Example3: Grade 5~6-Math-Geometry Achievement standards Contents of curriculum Achievement levels text

Example3: Grade 5~6-Math-Geometry Content of Curriculum Understand how to calculate the area of a rectangle and can calculate the area of a rectangle/a square Achievement Standards MG6301-Can calculate the area of a rectangle/a square

Example3: Grade 5~6-Math-Geometry Achievement Standards Achievement Levels MG6301-Can calculate the area of a rectangle/a square Level1(High)- can calculate the area of a rectangle/ a square and explain how to calculate the area of plane figures Level2(Middle)- can calculate the area of a rectangle/a square and explain how to do it. Level3(Low)- can calculate the area of a rectangle/a square.

Example3: Grade 5~6-Math-Geometry Information of an item Grade Grade 5-6 Area Measurement Sub-area Achievement standards MG6301-Can calculate the area of a rectangle/a square Achievement levels High can calculate the area of a rectangular and square and explain how to calculate the area of plain figures. Mid can calculate the area of a rectangle and a square and explain how to do it. Low can calculate the area of a rectangle and a square .

Principles of developing achievement standards are based on the national curriculum describe specifically knowledge, skills, and attitude in relation to subjects that students should master through learning describe clearly behaviors to be achieved consider the adequacy for amount of study load and level of learning consider feasibility to be implemented for teaching and learning in school become more sophistic and complicated as higher grade

Process of developing achievement standards Setting the direction of achievement standards Analyzing the contents of curriculum Developing achievement standards Studying the feasibility in school For item development and test assembling Confirm achievement standards Set the level of achievement standard

The roles of achievement standards for assessment knowledge, skills, and attitude of subjects Develop Item behaviors to be achieved implemented for teaching and learning Test Evaluate national curriculum

Contents of curriculum Psychometric item criteria Assembling a Test Contents of curriculum Test Specification Psychometric item criteria Number of items / Type of item

Example4: Test-Specification Grade 6 English Scope of contents Grade4~5 and the first semester of Grade6 Time 40 minutes Number of question 30 Area Criteria MC CR Listening 15%~20% 80%~100% 0%~20% Speaking 20%~30% 60%~80% 20%~40% Reading 20%~35% 70%~80% Writing 15%~25% 0% 100% ※ Numbers on the table are dummy, because they are usually classified for the national level test, especially for a sub area or achievement standards

Example4: Test-Specification Area % of items Number of items Total MC CR total Listening 20% 100% 0% 6 Speaking 30% 80% 9 7 2 Reading 25% 70% 7.5 5 3 Writing 15% 10% 4.5 4

Example5: https://www.collegeboard.org/sites/default/files/test_specifications_for_the_redesigned_sat_na3.pdf

References American Educational Research Association, American Psychological Association, National Council on Measurement in Education(2015). Standards for educational and psychological testing, Kim, M. et al (2012) Research report of achievement standards and achievement levels for the 2009 revised curriculum (Research Report CRC 2012-11-1), Seoul, Korea Institute of Curriculum and Evaluation. Ministry of Education and Science (2012), Achievement standards and achievement levels for Grade 5~6 using the 2009 revised curriculum, Daejeon, Korea, Ministry of Education. Kellough, R. D. and Kellough, N. G. (1999). Secondary school teaching: A guide to methods and resources. New Jersey: Prentice Hall.