1. 2017 National Conference on Student Assessment
Kathy Moore, Policy Advisor
Kentucky Department of Education
Office of Assessment and Accountability

2. Details—Details--Details
Peer Review
KY submitted documentation June 2016
USED letter received January 2017
KY responded February 2017
Details—Details--Details
No further evidence was requested for the security element after the initial submission.
KDE:OAA:ksm: 6/2017

3. Information was provided electronically and hardcopy
Information was provided electronically and hardcopy. An extensive Table of Contents served to organize the large amount of information submitted.
KDE:OAA:ksm: 6/2017

4. The level of detail provided on the summary was down to the page and section number of documents. A few questions were received on non-security pieces. Most were easily answered by simply referencing or clarifying the evidence submitted. Some required further documentation.
KDE:OAA:ksm: 6/2017

5. Peer Review Critical Element 2.5
Prevention of Irregularities
Regulations
Regulation Trainings
Face to face and videos
Manuals
Confidentiality Training
Nondisclosure Signing
Secure Materials Storage
Test Certification Forms
Security audit
Detection of Irregularities
Regulations
Seating Charts
Onsite Monitoring
Forensic Data Analysis
Reported Allegations
Remediation of Incidents
Regulations
Board of Review
Re-training
Lowered Scores
Licensure
Allegation Incident History Record
Investigate Irregularities
Regulations
Allegation Reporting
Personal Interviews
Documentation
Seating Charts
The 4 components of Critical Element 2.5 and how Kentucky addresses each.
The Administration Code regulation drives our administration and security practices.
KDE:OAA:ksm: 6/2017

6. Prevention of Irregularities
Regulations
Regulation Trainings
Face to face and videos
Manuals
Confidentiality Training
Nondisclosure Signing
Secure Materials Storage
Test Certification Forms
Security audit
KDE:OAA:ksm: 6/2017

7. Regulations KAR 703:080 Administration Code
Anyone with the potential to be involved with testing must receive annual training.
KAR 703:070 Inclusion of Special Populations
Everyone involved in providing accommodations must be trained.
Training materials are available online.
inCode.aspx
Two regulations directly guide security around handling materials, administration practices, overall security.
The Administration Code covers acceptable, unacceptable, and expected practices for test administration. The regulation applies to all state-required and state-administered assessments.
The Inclusion of Special Populations covers the acceptable, unacceptable, and expected practices for providing accommodations during test administration. The regulation applies to all state-required and state-administered assessments.
The training materials include PowerPoints, videos, directions for presenting the training, activity handouts. The materials are designed to be presented by District and/or Building Assessment Coordinators, but may also be used as a self-paced training. District Assessment Coordinators are responsible for ensuring that anyone with the potential to be involved with testing is trained prior to working with materials or administration.
KDE:OAA:ksm: 6/2017

8. Regulations Standards for the regulation Professional Ethics
test prep and student information
Educational Defensibility
students and schools right to valid and reliable test scores
test prep activities
Student Ownership
work done entirely by student
student honor code
The Administration Code regulation was developed based on 3 standards which are included in the regulation. These sum up all that needs to be done for a successful administration. Following the details of the regulations and paying attention to details will lead to a clean administration without problems needing to be filed and investigated by the state.
KDE:OAA:ksm: 6/2017

9. Regulations Office of Assessment and Accountability Emphasis
Test Security—checks and processes to monitor appropriate administration of state-required tests including seating charts, room location codes and a student honor code.
Data Quality—collaborative process between the Office of Assessment and Accountability (OAA) and school/d
Throughout the year in a variety of ways, s, face-to-face trainings, phone calls, etc., KDE emphasizes the need for correct data. Reminders about keeping the student information system up-to-date and how the data from the system is used to populate other systems used for accountability are often reiterated. Multiple opportunities to review the data related to the assessments has proven to be a highly successful means to more accurate final data leading to valid and reliable accountability results.
KDE:OAA:ksm: 6/2017

10. Test Administration Manuals
The first thing list in manuals is security.
KDE:OAA:ksm: 6/2017

11. Confidentiality Training
Prepared and conducted by schools
Nondisclosure
Secure Materials Storage
Secure at all times
Limited access
Double locked in classrooms
Students may not transport
Confidentiality is done at KDE as well as in schools.
Signatures on nondisclosure forms are required by everyone dealing in any way with testing and at KDE.
Some new construction in schools included secure test storage areas.
KDE:OAA:ksm: 6/2017

12. Test Certification Forms (BAC and DAC)
Vendor Contractual Requirements
General affidavit nondisclosure
Data security and breach protocols
Including KDE Memorandums of Understanding
Storage and retention of test materials
Signing the form indicates that testing went well without incident or that any problems were addressed and handled appropriately.
KDE:OAA:ksm: 6/2017

13. Security Audit 12/12 13 KDE:OAA:ksm: 6/2017
Standard
High Priority
Dated Started
Date Completed
Notes
Medium Priority
Low Priority
13.) Security Awareness and Training
Revise training materials in a timely manner to reflect changes in policies or procedures (pg. 21).
Currently done in conjunction with Administration Code and Inclusion of Special Populations trainings.
Require field sites to adhere to set procedures for training to rectify variations currently in the field (pg. 21).
8/13
2/14
New langauge in administration code to require interactive training.
Develop a new video to be used to supplement training (pg. 21).
11/12
New videos and materials will be developed once regs are official. Videos updated 2/14
The Kentucky Department of Education contracted with Caveon to conduct a department wide security audit. Caveon staff were on site to interview staff from the Office of Assessment and Accountability regarding the routine practices of not only the assessment office, but the security practices of the department regarding employees the physical location and handling of secure materials. A few District Assessment Coordinators and Building Assessment Coordinators were also interviewed.
Findings and recommendations are categorized and rated high, middle, or low priority.
A spreadsheet internal to KDE/OAA is set up in the same manner with the addition of dates of implementation and notes. 13
KDE:OAA:ksm: 6/2017

14. Detection of Irregularities
Regulations
Seating Charts
Onsite Monitoring
Forensic Data Analysis
Reported Allegations
KDE:OAA:ksm: 6/2017

15. Seating Charts
Basic form provided on KDE webpage and e- mailed directly to DACs.
Forms.aspx
Some vendors provide their version. We recommend to adapt the form so that what is submitted reflects the actual room arrangement and testing scenario.
KDE:OAA:ksm: 6/2017

16. Onsite Monitoring--School Visits
Randomly and purposely chosen
Materials used during visit provided ahead of visit
District Assessment Coordinator notified 30 minutes or less before arrival at school
View process for handling materials and storage areas.
View classrooms, but do not enter while testing is in session
Schools at all levels, demographics, and geographic areas are included in the selection process. One staff member from the communications branch is paired with one staff member from the data branch. All consultants from the communications branch have teaching experience and many from the data branch have education connections—wife teaches, parents were administrators.
KDE:OAA:ksm: 6/2017

17. Forensic Analysis Erasure analysis
Individual review of student response booklets
Student/staff interviews
Primary vendor contracts with Caveon for erasure analysis. KDE is looking to improve understanding and use of the data.
KDE:OAA:ksm: 6/2017

18. Allegations
Reported to KDE by district/school staff, self-reported, students, parents, public
Phone, , letter,
or online at Reporting.aspx
Investigated and processed by KDE
Allegations Coordinator
Staff Attorneys
Other KDE Staff from the legal office and commissioner’s office
Can include data forensic analyses
Allegations can be reported to KDE with full disclosure or anonymously.
KDE staff investigates the allegation through the collection of information such as written notarized statements, interviews (adults and students), state-required testing documentation.
The Board of Review members are all department staff, who may or may not have an education background.
KDE:OAA:ksm: 6/2017

19. Investigate Irregularities
Regulations
Allegation Reporting
Allegation Documentation
Seating Charts
Personal Interviews
KDE:OAA:ksm: 6/2017

20. Allegation Documentation
Steps to reporting an allegation
ts/ %20Steps%20to%20Follow%20When%20a% 20Testing%20Allegation%20or%20Irregularity% 20Occurs.pdf
Forms
egation-Reporting.aspx
Notarized statements are required.
KDE:OAA:ksm: 6/2017

21. Personal Face To Face Interviews
One or more district staff
One or more school staff
One or more students
Depending on the severity of the incident, interviews may be done face-to-face on site or by phone.
Testing is not delayed due to a student refusing to sign or a parent requesting that the student not sign the honor code. The signature serves as a reminder to students of the expectations during testing.
KDE:OAA:ksm: 6/2017

22. Remediation of Incidents
Regulations
Board of Review
Re-training
Lowered Scores
Licensure
Allegation Incident History Record
KDE:OAA:ksm: 6/2017

23. Board of Review Board of Review
Appointed by the Commissioner—6 KDE staff
Trained annually on Administration Code and Inclusion of Special Populations by Office of Assessment & Accountability staff
Makes recommendation to Commissioner of Education
Can result in score change for schools (not students)
Can result in loss of teaching certificate for up to 2 years by the Educational Professional Standards Board (KRS )
Not all members have teaching backgrounds, but all are KDE employees. The work and results is monitored by the Kentucky Legislative Research Commission.
KDE:OAA:ksm: 6/2017

24. Retraining KDE offers KDE offers 6 regulation code trainings per year
2 November
2 December
2 January
Districts/Schools provide
Districts/schools are responsible for the initial training using the provided materials or adapting their own to best suit their needs. When addition training is recommended by the Board of Review the district/school has the option of providing the training or assigning the individual to attend a KDE provided session.
KDE:OAA:ksm: 6/2017

25. Consequences Scores lowered Loss of teaching license
Impacts school and district accountability
Individual student not impacted
Loss of teaching license
Educational Professional Standards Board
Recommendations for lowering of scores or teacher names to be sent to the Educational Professional Standards Board are made by the Board of Review.
KDE:OAA:ksm: 6/2017

26. Incident History Allegation Example
Inappropriate Assistance/Intervention by Staff
Leading students to answers, making corrections to student work
Test Security (Staff)
Discussing item specifics
Test Security (Student)
Removing a test booklet from testing location
Out of Order (Staff)
Not following test schedule
Out of Order (Student)
Working in prior sections or ahead
Special Education
Giving inappropriate accommodations or not providing accommodations
Student Action
Using cell phones
Missing Test Booklets
Misplacing or failing to collect books by test administrator
Other
Giving feedback, allowing students to change answers after testing
Allegations are categorized and the number of incidents reported are logged. Trends are monitored and emphasis is placed on problem areas during spring face to face meetings which target procedures for test administration.
KDE:OAA:ksm: 6/2017

27. Kentucky Department of Education
Office of Assessment and Accountability
300 Sower Blvd.
Frankfort, KY
Kathy Moore, Policy Advisor
Kentucky Department of Education
Office of Legal, Legislative and Communication Services
300 Sower Blvd.
Frankfort, KY
Patsy Kenner, Executive Staff Advisor/Allegations Coordinator
KDE:OAA:ksm: 6/2017