What Should You Consider When Talking About Technical Assessments? John M. Townsend Tennessee Board of Regents NACTEI 28 th Annual Conference Boise, Idaho

Why are you assessing? Most important question –For Perkins IV? –For Program Improvement? –For student attainment?

If For Student Attainment At what level is appropriate? –Entry level –Mastery level –Industry certification level

When do you assess? End-of-program End-of-course End of Pathway Cluster related

Norm- or Competency-referenced Norm-referenced –Comparison among individuals –Demonstrates changes of students as a group Competency- referenced –Attainment of an individual –Measurable learning outcomes based upon industry standards

Buy or Develop? 1.Standardized instruments 2.Centralized state-developed instruments 3.Local assessments recognized by state

Performance or Cognate Performance –What the student should be able to do –Checklist –Limited number of areas can be assessed in specified time Cognate –What the student should know –Test-items –Many areas can be assessed in a specified time

Building an Assessment Adopt – seek items or assessments that you can adopt as is Adapt – items or assessments that can be modified to fit your standards Build – items or assessments when others cannot be found

Table of Specifications (blueprint) Determine competencies to be assessed Determine competency groups (3 – 10) Determine percentage of each cognitive typology per group

Development Questions Is cost and time worth benefit of test? Who should develop the assessment? Test security issues?

Multiple-choice elements: Stem Correct response Foils or distractors

Fixed-form v Variable-form Fixed-form High reliability Needs high security Limited number of test items needed Variable-form Good reliability Lesser security needed Requires a large number of test items

Assessment Administration Decide the media for administration Identify who should conduct the assessment Determine when the assessment activities should occur

Reporting issues: 1.Grading of the assessment 2.Assessment improvement needs 3.Reports needed

Assessment Review Validity – the degree to which certain inferences can be made from test scores. Reliability – the degree of consistency between two or more measures of the same thing.

Item & Assessment Analysis Find a good statistician, psychometrician or educational psychologist !

Ethical Issues in Testing Code of Fair Testing Practices in Education by the Joint Committee on Testing Practices, American Educational Research Association, American Psychological Association & National Council on Measurement in Education

Legal Issues Accommodations for members of special populations Access to the test-items by parents and others Utilization of assessments in relation to passing or graduation

Postsecondary Issues: “Each eligible agency shall identify in the State plan core indicators of performance for career and technical education students at the postsecondary level that are valid and reliable, and that include, at a minimum measures of each of the following:”

Postsecondary Issues: “(i) Student attainment of challenging career and technical skill proficiencies, including student achievement on technical assessments, that are aligned with industry-recognized standards, if available and appropriate.”

Postsecondary Issues: OVAE OVAE has denied the utilization of GPAs as an indicator OVAE preference is a third-party developed and administered assessment (e.g. NOCTI or centralized state assessment unit) OVAE has not defined what they will accept; burden of proof is on State

Postsecondary Issues: Systemic Local colleges seek to serve local business/industry – diversity of curriculum Cannot require anything beyond what the state requires for graduation – in TN most certification assessments are post- program Accreditation processes often evaluates the assessment process of a program of study

Postsecondary Issues: Challenge to OVAE Goes against Perkins IV Sec. 8 “establishment” clause An unfunded mandate – many postsecondary receive little Perkins IV funds In pedagogy and andragogy, a single assessment is not sound

Postsecondary Issues: Reference Wiseley, Chuck. (August 31, 2007) California Community Colleges NSWG response to the: Proposed DRAFT Guidelines for State Phase-in of Perkins IV Core Indicators 2S1 and 1P1 Technical Skills Assessments.

Contact Information: Dr. John M. Townsend Executive Director, Workforce Development Office of Academic Affairs Tennessee Board of Regents 1415 Murfreesboro Pike, Suite 350 Nashville, TN