1. ATEAM ATEAM Kenneth Barbour Faye Atkinson, Rhonda Carlyle,
Lori Ivey, Phyllis Price, Jim Rhew,
Kevin Skelton, George Stubblefield
ATEAM

2. Our Purpose
Support the design Assessments of Student Achievement and implementation of Assessment
Collect and Analyze Achievement Data
Report Student Achievement
Design, develop and implement a system to analyze student academic performance outcome data for purposes of professional development, and technical assistance geared to improve instruction and student performance

3. Our Customers Accountability Services Test Development
Testing Policy and Operations
LEA Test Coordinators
Regional Accountability Coordinators (RACs)
Regional Computing Consultants (RCCs)
State Board of Education
Public
Local Education Agencies (LEAs)
Schools
Researchers
Department of Justice
Federal Reporting

4. Data Consumer List State Board of Education
Legislative / Judicial Requests
Federal Submissions
Researchers
LEAs / Schools
Title I & III
Public

5. Our Partners Portfolio Managers in other DPI Areas
Accountability Services Division
Technical Outreach for Public Schools (TOPS)
Regional Accountability Coordinators
Regional Computing Consultants
Test and Accountability Coordinators/Directors

6. Our Governance
Accountability Services Division
Data Management Group (DMG)
US Department of Education
No Child Left Behind (NCLB)
Adequate Yearly Progress (AYP)
Accountability Basics Control (ABCs) of Public Education
Accountability Configuration and Control Board (CCB) - Advisory
State Board of Education (SBE)
Legislature

7. Services of ATEAM Applications Analyses Reports
Accountability Database (ADB)
Technical Support
Customer Support (RITS - External)
Customer Support (RTS - Internal)
Training

8. Products List WinScan ABCTools Request Tracking System (RTS)
Regional Issue Tracking System (RITS)
NC WISE Test Upload System
Online Testing Irregularity Submission System (OTISS)
Individual Student Report (ISR)
Data Collection, Verification and Reporting (DCVR)
Collection Date Submission System
Accountability Database (ADB)
Writing Instruction System (WIS) - TOPS
NC Registration - TOPS
Online Testing Systems - TOPS
Online Ordering System - TOPS
Testing News Network (TNN) – TOPS
Request Tracking System - TOPS

10. Accountability Services Structure
Accountability Division
Lou Fabrizio, Director of Accountability, Policy and Communications
Gary Williamson, Director of Accountability Operations
Tammy Howard, Chief, Test Development Section
Sarah McManus, Chief, Testing Policy and Operations Section
ATEAM
Contractors
TOPS (Technical Outreach for Public Schools)
Pearson
Other Contractors

11. Application: WinScan
Windows application used by the Central Office of LEAs
WinScan is a software program that is used to scan, score and process statewide tests
WinScan identifies and corrects errors
WinScan produces local reports
Class Rosters
Goal Summaries
Label Reports
Many Others

12. Application: ABCTools
Windows application used by the Central Office of LEAs
Data file loading for MYSQL database using managed file transfer
Manages accountability data for NCLB and ABCs with MYSQL data warehousing system
Provides predefined SQL statements to assist users with queries
Provides student level audit of data elements used for NCLB and ABCs
Provides reports to assist school staff in identifying students at risk as result of proficiency levels
Provides classroom scale score averages
Provides access SQL Query Tool
Provides precoding for answer sheets used for ABCs and NCLB
Provides verification reports and processes for test data submitted to NCDPI for NCLB and ABCs
Provides links to accountability information

13. Application: Online Testing Irregularity Submission System (OTISS)
A web based APEX Application
Allows for the collection, approval workflow, tracking and archiving of testing irregularities
Includes school level access

14. Application: Individual Student Report
Windows application used by the Central Office of LEAs
Prints parent/teacher reports
Subject to peer review by USED
NCLB regulates the reports
Includes scale scores, percentiles, exit standards, Lexile measures, Quantile measures, achievement level descriptors, and sub score reporting

15. Application: Data Collection, Verification and Reporting (DCVR) and Accountability Database (ADB)
Manages file transfer, data validation and reporting
Manages the collection of data sets from schools and LEAs, validates and merges the data into an accountability database
Provides interactive error reporting and data collection
Provides for the data cleaning and aggregation of data into the accountability database (ADB)
Insures a unique ID across all the student information collected from various data sources, including NC WISE
Encompasses all business logic to create the source data for the ABCs and AYP reports
Provides the source data for all accountability data collected for the EdFacts requirement

17. Application: Collection Date Submission System
A web based APEX application
Allows the LEAs to enter their start of school, 20th day, First Day of Fall testing, First Day of Spring testing, and last day of school dates
Allows the ATEAM to use these dates to pull the enrollment and historical test data for the schools
Decreases the load on the NC WISE servers and the LEA Testing Coordinators

18. Application: NC WISE Test Upload System
Provides ability to import student test score information into NC WISE
SCO file (from WinScan) is processed during a nightly batch job
Summary / error reports are available through a web interface
Also includes capabilities for import of SAT and CTE test data

19. Analyses: Ad hoc analyses for customers
Growth Model development and validation
Reconciliation of reporting anomalies
Assist standardized test data certification
Apply linking tables to scales
Supply DPI Business Units with analyses results
District School Transformation
Charter Schools
NC Virtual Public Schools
Title I & III
Others

20. Reports: Major Report Systems
NCLB/AYP
ABCs
Disag
NC School Report Card (SRC)
Greenbook - TOPS
EdFacts – (via EDEN)
Considated State Performance Report (CSPR) (EdFacts)

21. Reports: Reports List List of Reports for ABCs/AYP Release
Executive Summary
Low Performing Schools
Secondary Analyses
Number and Percentage of School Level Targets Met by Subgroup
Number and Percent of Schools by Percent of Targets Missed
Number and Percent of Schools by Number of Targets Missed
Number of Schools That Met AYP Using at Least One of the Following Special Flexibilities: Safe Harbor, Confidence Interval, Growth (Schools may be counted on more than one row)
Number of Schools, Number Meeting AYP and Percent Meeting AYP by Number of Targets
Number of Schools that did not make AYP: Number due to one subgroup
Number of Schools that missed AYP due to two subgroups

22. Reports: Reports List Continued -
Title I School Improvement List
LEA Improvement Status
LEAs in Corrective Action
Impact on Performance Using Retesting Results
4-Year Cohort Graduation Rate (Current Year)
4-Year Cohort Graduation Rate (Previous Year)
5-Year Cohort Graduation Rate (Previous Year)
Evolution of the ABCs
AYP Schools Under Review
Possible ABCs Violations of the 95% Rule
Possible Violations of AYP 1% Cap

23. Report Tables List
Preliminary End-of-Grade General Test Multiple-Choice Test Results Statewide Performance Summary in Reading and Mathematics Grades 3-8
Preliminary End-of-Grade NCCLAS Test Results Statewide Performance Summary in Reading and Mathematics Grades 3-8
Preliminary End-of-Grade NCEXTEND2 Test Results Statewide Performance Summary in Reading and Mathematics Grades 3-8
Preliminary End-of-Grade NCEXTEND1 Test Results Statewide Performance Summary in Reading and Mathematics Grades 3-8
End-of-Grade General Test Multiple-Choice Test Results, Statewide Characteristics and Performance of Grade 3-8 Students
Preliminary End-of-Grade General Test Multiple-Choice Test Results Statewide Performance Summary in Science Grades 5 and 8
to End-of-Course General Test Multiple-Choice Test Results, Statewide Performance Summary
Preliminary End-of-Course NCCLAS Test Results Statewide Performance Summary
Preliminary End-of-Course NCEXTEND2 Occupational Course of Study Test Results Statewide Performance Summary

24. Technical Support: Request Tracking System (RTS)
Internal Request Tracking is an integral part of the total IT responsibility of the ATEAM
It serves as the primary method to communicate changes requested to the “owner” of the work with applications and systems and the corresponding resolutions back to the customer
It maintains consistency in logging and reporting requests, utilizing standardized procedures

25. Technical Support: Regional Issue Tracking System (RITS)
External Request Tracking is an integral part of the total IT responsibility of the ATEAM
It serves as the primary method to communicate regional changes requested to the “owner” of the work with applications and systems and the corresponding resolutions back to the customer
It maintains consistency in logging and reporting requests, utilizing standardized procedures

26. Authoritative Sources

27. Technical Outreach for Public Schools (TOPS)
EOC On-line Testing
Paper and Pencil Testing
NCExtend1
Writing Instruction System (WIS)
Data Analysis Group
NC Registration
On-line Ordering System
Test Item Development
Test Item Bank
Testing News Network (TNN)
Logistics
McKimmon Center
Blue Ridge Location
Warehouse (Other)

28. Writing Instruction System (WIS) - TOPS
Application resulting from the Grade 7 Online Writing Project
A pilot using the Moodle freeware system configured and hosted at North Carolina State University (The pilot ended 6/30/09)
Consists of 3 online Professional Development modules for Teachers, 2 on demand writing assignments and 2 content specific writing assignments for students in grades 4 and 7
7th graders were allowed to develop and upload the writing assignments to an online portfolio
Teachers were then able to access and grade the assignments online
Based on the results, the State Board of Education approved the DPI recommendation to extend the pilot for an additional year and to open participation to additional LEAs. The project is extended through June 2010.

29. NC Registration - TOPS
Schools and school systems use the system to indicate students:
who are eligible for testing
to schedule test administrations within predefined testing sessions for groups of students

30. Online Testing System - TOPS
Provide same assessment as paper and pencil online format
Used for the eight EOC tests
Browser based
Need Adobe Flash to use

31. Online Ordering System - TOPS
System developed that allows the LEA Test Coordinator to place, review, and update orders for testing materials
System includes materials such as:
Braille
Read Aloud
One Item per Page
Large Print Test

32. Testing News Network (TNN) - TOPS
TNN is a closed information and collaboration service sponsored by the Accountability Services Division and hosted by TOPS
Users are testing and accountability staff of LEAs in NC Public Schools
Provides secure information dissemination of local school system test coordinators
Provides indexing of critical events, policies, reports, communications, process details and software upgrades for the testing program

33. Training System - TOPS NC Testing Program Online Training
Item Development Basics
Developing EOC Mathematics
Grades 3-5 Items
Grades 6-8 Items
Developing EOC Items
EOC English 10 Items
Additional Courses will be added

34. NC FALCON - TOPS Training
North Carolina Formative Assessment Learning Communities Online Network
Opening as a pilot in January, 2010 to train teachers, educators and administrators on how to do formative assessment

35. Archive System - TOPS
Electronic archive of test materials for the NC Testing Program
EOGs
EOCs
NCEXTEND1
NCEXTEND2
Grade 10 Writing Assessment

36. Help Desk System - TOPS
Provides HelpDesk Support for LEA / School Staff for all TOPS systems

37. Hardware Required for Standardized Testing and Accountability at the LEA Level
In 1992, the NC General Assembly revised the NC Public School Testing Program legislation authorizing NCDPI to implement the end-of-grade and the end-of-course tests (reference Public School Laws of NC, Article 10A, Part 2, 115C )
A key component to the successful implementation of the NC Testing Program is the timely scanning and scoring of the students' answer documents and the printing of the students' reports
To accomplish these tasks, the NCDPI purchases, upgrades, maintains hardware, and develops software for computers, scanners, and printers

38. Software Required for Standardized Testing and Accountability at the LEA Level
ABCTools
Adobe Acrobat
Individual Student Reports
Microsoft Office Professional
Antivirus
SFTP Secure Shell Client
WinScan
WinZip

39. EQUIPMENT RESPONSIBILITIES
EQUIPMENT ROLES
EQUIPMENT RESPONSIBILITIES
Accountability IT Team:
Equipment specifications
Equipment and/or software compatibility
Ghost images
Equipment set up assistance
Accountability Operations Director:
Approve purchases
Accountability Purchasing Agent:
Purchase orders
DPI Fixed Asset Clerk:
Provide asset tags
Maintain official inventory (indicating where equipment is permanently housed)
Provide inventory list for annual audit
State Surplus contact
Accountability Equipment Acquisitions, Maintenance and Inventory Management:
Maintain internal inventory (indicating where equipment is temporarily housed)
Assist with surplus/transfer forms
Liaison between Accountability and Fixed Asset Clerk
Ensure surplus/transfer forms completed and submitted as needed
Approve temporary transfer of equipment
Facilitate annual audit
Lead RAC:
Liaison between RACs and Accountability Inventory Management
Provide RAC input on equipment issues
RCC:
Equipment set up assistance (LEAs)
Equipment maintenance assistance
RAC:
Maintain regional equipment inventory
Reconcile regional and fixed asset inventory annually
Request equipment transfers
LEA Test Coordinator:
Day to day equipment maintenance and upkeep
Familiarity with equipment and its proper use
Report damages/breakdowns to RAC/RCC
TOPS:
Provide shipping of equipment
Maintain equipment pool
Repair equipment

40. Training for: ABCTools
Monthly training for registered participants held at DPI
Demonstration of loading data into ABCTools, if needed. ATEAM may have processes to move data into a set of folders for transformation.
Demonstration of where to find data on web (Disag and others) and what the reports provide
Demonstration on published public reports
Demonstration on Data Security
Demonstration of other data transformation use for school improvement

41. Training for: Precode Plus
Precodes answer sheets for EOC/EOG tests
Designed to assist Testing Coordinators with specialized precoding needs
Allows user to precode from roster listing, precode from a pick list of students or precode using an external file for a specific list of students

42. Training for: Reports
Historical Audit Rosters (Provides Scale Scores and Levels)
Membership/Course Codes (Provides membership rosters by subject)
Student Profile Report
Exit Standards Roster
Graduation Reports
ABC Summary
AYP Reports
Growth Summary Report (Classroom Scale Score Averages)
Students Repeating Courses Report
Student Rosters with Demographics from NC WISE
Classroom Scale Score Averages (Growth Summary from ABCTools)
Rerostering of WinScan Records for EOG/EOC
Goal Summary Reports in WinScan
Growth Target Reports (Why and why not to use these reports)
Detail AYP Reports for Web
Detail ABCs Reports for Web

43. ATEAM (Extra)
Revised 01/21/10 RMC