1. Research, Monitoring, and Evaluation Team Student Achievement Division
Dual Credit Data
Research, Monitoring, and Evaluation Team
Student Achievement Division
May 2017

2. Our Learning Goals During this workshop we will explore:
the data sources that allow us to look at Dual Credit students and programs from various perspectives
provincial level enrolment and achievement highlights for students taking Dual Credits
post-secondary transition rates
Alisha
During this workshop we will explore:
the data sources that allow us to look at dual credit students and programs from various perspectives
Provincial level enrolment and achievement highlights for students taking DCs in the school year
Post-secondary transition rates of students that have taken dual credits

3. Dual Credit and the School College Work Initiative
Since its inception in 1997, the SCWI has contributed to the goals:
increasing the number of students who graduate from secondary school and
providing a seamless transition from secondary school to postsecondary education by supporting collaborative activities and programs.
Alisha
The school college work initiative began in 1997 and dual credits began in the academic year and since then, has contributed to the following two goals:
Increasing the number of students who graduate from secondary school
Providing a seamless transition from secondary school to postsecondary education – through supporting collaborative activities and programs

4. Key Questions When Analysing Data
Purpose
Why did we collect this data? How might this be useful?
Description
What are the patterns/items of interest that you notice from this data?
Limitations
What are the limitations of this data?
Inferences and Questions
What further questions does this data generate for you?
Next Steps
What other data do we need to help frame future action?
Cristina
The way we were able to get insights of how the dual credit students are doing in the dual credit programs and beyond was through data. Since the beginning of the dual credit programs we have systematically collected data, analyzed and used the results for program improvement.
Every year we go through the process of critically thinking about what exactly do we want to find out from the data we are collecting, and further what are the patterns of interest that emerge from the analyses.
We also have to make sure that we understand the limitations of the data , this way we will be able to estimate how close we are to the truth in our results.
The limitations will also point us in the direction we should look next, and what other data we need to help frame future action.

5. Data helps us to… ask “Why?”
see trends and call into question certain patterns
anticipate the future by understanding the past
highlight inequities that need remediation
move from “intuition” to “deduction”
validate or repudiate assumptions
enables us to move closer to causation
Alisha
Data helps us to explore what is happening asking the question “why” do we see what we see in the field
We can monitor trends and establish patterns in the data we collect
Through these patterns, data helps us to anticipate the future and forecast events based on what has happened in the past
Based on data, we can identify existing inequities needing remediation
It allows us to move from decision making based on intuition to decision making based on deduction
Data enables us to validate or repudiate assumptions
And ultimately allows us to move closer to causation and establishing an explanation for what we see

6. Levels of Data Student Course/Class School Board Province Cristina
The education data is collected and aggregated at various levels, serving different purposes.
student-level data refers to data such as personal information (e.g., a student’s age, gender, race, place of residence), enrollment information (e.g., the school a student attends, a student’s current grade level and years of attendance, the number of days a student was absent), academic information (e.g., the courses a student completed, the grades a students earned, the academic requirements a student has fulfilled), and various other forms of data collected and used by educators and educational institutions.
Due to privacy concerns, in the ministry of EDU the data is rarely, almost never, analyzed at student level. Even when the happpends , the data is depersonalized so the individual students can’t be identified. Most often the depersonalized data is aggregated at various levels- course/class, at school, board and province level.
Province 6

7. What data does the ministry look at?
Credit accumulation rates
Pass rates
Mark Distribution
Differences between courses, programs, and subpopulations of students
Provincial and Board graduation rates
Transitions to PSE
Cristina
The standard way of looking at aggregated educational data is through credit accumulation rates : the credit accumulation rates are calculated using a cohort approach and it is calculated as the percentage of students who have earned a certain number of credits for grades 9, 10 and 11 where student-level data is tracked. The number of credits for each grade are :
8 or more credits for grade 9
16 or more credits by the end of grade 10
23 or more credits by the end of grade 11
Pass rates – is the percentage of students who have passed (earned 50% or higher) in a grade
Mark distribution – number/percentage of mark records in each mark rance
Graduation rates – this is also calculated with a cohort approach and it is the percentage of students in a cohort who graduate after 4 years and after 5 years.
Transition to PSE
Transitions to PSE – we will look at this later in the presentation – this is where the students are applying and registering for college and university

8. How does the ministry use data?
We answer questions such as:
Are credit accumulation rates increasing, decreasing or remaining the same?
Which courses are proving most difficult for students to pass?
How are students with special needs doing relative to all students?
Are there regional differences/trends in any performance areas?
What trend data is worth further investigation?
Where are there pockets of excellence to help us improve results?
What do the indicators tell us about the future graduation rates?
How many students are registering in college or university?
Cristina
We answer questions such as:
Are credit accumulation rates increasing, decreasing or remaining the same? –at board, school and provincial level. The schools and boards receive a detailed report of how they are doing in terms of the indicators described above. The schools and boards use this data for program improvement
Other examples of questions are
Which courses are proving most difficult for students to pass?
How are students with special needs doing relative to all students?
Are there regional differences/trends in any performance areas?
What trend data is worth further investigation?
Where are there pockets of excellence to help us improve results?
What do the indicators tell us about the future graduation rates?
How many students are registering in college or university? – and we sometimes can look at that from a program perspective as we will show you toward the end of the slide deck

9. High standards and expectations: Provincial Graduation Rate
6 key levers for secondary reform:
Leadership infrastructure
Engaging and relevant programming
Effective instruction
Focused Interventions for students at risk of not graduating
Legislation and policy development
Research, monitoring and evaluation
Cristina - The graduation rate is now at 86.5% - this is fresh out of the oven as this graduation rate has been announced yesterday. In there were approximately 147,000 graduates from the cohort of students who started grade 9 in   This number also includes those students who graduated within four years. 
The graduation rate is up from the 68% graduation rate in
In alone, approximately 27,500 additional students graduated than would have had the rate remained at the rate of 68%. This is a total of approximately 217,500 more students who have graduated since than would have if the Student Achievement Strategy were not in place.

10. Dual Credit Data Sources
Enhanced Data Collection Solution (EDCS)
Ontario School Information System (OnSIS)
Ontario College Application Service (OCAS)
Ontario Universities’ Application Centre (OUAC)
Student Surveys and Testimonials
Cristina
The data for dual credit students is being collected and retrieved from a number of sources that we will describe in more detail in a second.
EDCS is web-based collection tool whose main purpose is to collect financial and aggregated student data at Regional Planning Team Level. No other tool other than EDCS allow us to look at the dual credit programs at program and RPT level.
OnSIS is the province wide educational data web based tool
OUAC and OUAC collect student application and registration data. These data are linked with the OnSIS data to give us insights on where our students go after they graduate
Additional ad-hoc data collections and surveys are sometimes run. As an example, for a couple of years we administered surveys to measure student satisfaction with the School Within A College programs.

11. Enhanced Data Collection Solution (EDCS)
board level aggregate student data
board level aggregate program data
proposal data
financial data
Alisha
- The Enhanced Data Collection Solution system captures information on:
Board level aggregate student data
Board level aggregate program data
Program proposal data – including detailed program descriptions, student selection processes, operational details
Program financial data – including programs’ board and college benchmark costs, transportation and miscellaneous costs

12. Alisha
Here is a list of extracts from EDCS that can be generated capturing student and program level data as well as financial data.
Information can be retrieved organized by Regional Planning Team, by board or by college
Extracts can be saved as excel documents, allowing for comprehensive data analysis

13. Unique Data Elements to EDCS
Student and financial information by program and regional planning team (RPT)
Program composition
Disengaged and underachieving students
Students who had previously left and returned
Alisha
What is unique about data in EDCS is that student and financial information by program and by Regional Planning Team is captured
Data is also collected on who is involved in each dual credit program: for example, whether the program is primarily designed for students in the primary target group, Specialist High Skills Major (SHSM) students or students from the Ontario Youth Apprenticeship Program (OYAP)
EDCS also captures students who have previously left and returned

14. What is the EDCS data used for?
Approval process
SMART Goal Reports
Dual Credit Student Data Report
Ad Hoc Reporting
Alisha
EDCS data is used in a variety of ways
During the annual dual credit program approval process the review team looks closely at previous year’s data that has been inputted into EDCS by Regional Planning Teams to inform decision making on program approvals. Data is also reviewed in year during the program’s six contract change cycles.
Furthermore, this data is used to prepare RPT SMART Goal Reports in the fall and spring of each school year
EDCS data is used to prepare the annual dual credit student data report – a copy of the report has been distributed here
Lastly and most importantly, as EDCS provides accurate, real time information on dual credit programs and students, this data is used to respond to ad hoc reporting requests at the Ministry and from external stakeholders on the status of dual credit programs, students and funding.

15. Reports for transparency and Accountability
Dual Credit Data Report
provide the dual credit community and general public an overview of dual credit programs and students
posted on the EDU website since
SMART Goal Reports
Supports data informed decisions for program improvement
Alisha
Two reports are prepared at the Ministry using EDCS data to maintain transparency and accountability:
1. Dual Credit Data Report
Provides the dual credit community and the general public an overview of dual credit programs and students
This report has been posted on the EDU website since
2. SMART Goal Reports
Prepared in the fall and spring of each year
Provides RPTs with data that allows them to make data informed decisions for program improvement

16. Cristina
The following few slides are showing a few examples of the information included in both the Annul Dual Credit Report.
This graph shows the student enrolment in Dual Credit programs starting with Student enrolment in dual credit programs increased increased more that 7 times over the last 9 years. In , the enrollment in dual credit programs was 20,264

17. Cristina
This slide shows the difference between the approved number of dual credit students (which is public) and how many enrolled.
Money is allocated for the coming year based on projections.
allocated money that is unspent can be problematic.
EDU uses this as a measure of “are we doing the best that we can, given that we’re projecting for the coming year?”
If the RPT team differs from the provincial % in some significant ways, the gap also stimulates discussion

18. Cristina - This slide demonstrates an increasing trend in participation for the target population which is students who are disengaged and underachieving but with potential to succeed.
Previously, it was challenging to collect data on this indicator because people were reluctant to label students.
There were also data quality issues because the person who submits the report at the end of the year is not the same as the one at the beginning of the year.
The primary focus is on students who face significant challenges in completing the requirements for graduation but have the potential to succeed. (Dual Credit Program: Policy and Program Requirements 2013,

19. Cristina This slide shows how many students who had left school the EDU can get back through a program like DC.
EDCS is the only source for this data, as there is such a variety of scenarios of how and how long these students have left school that it makes it almost impossible to get this data from any other source.

20. Cristina
This slide is showing what is the percentage of students by target group (OYAP, SHSM or/and disengaged and underachieving). These three groups are not mutual exclusive, as students can be OYAP or SHSM and disengaged and underachieving in the same time.
OYAP – 9%
SHSM- 18%
Disengaged and Underachieving – 75%

21. Cristina Retention Rate definition: students who had stayed to the end of the program. Doesn’t mean they had completed the credit.
Clearly, attention is being paid to supporting students to remain in the program. Giving students a chance to see themselves belonging at college is a critical component to the dual credit program.

22. Success Rate (OnSIS) = 87%
Cristina - Success Rate – the percentage of credits that have been earned out of the total number of credits attempted.
This has been steady in the last 2 years at 91%. Matching the right student with the right program and providing the right supports leads to success for more and more students. As numbers of participants have increased, both retention and success have also increased.
If we use the OnSIS data at course level for dual credit students, the success rate is 87%.
And this is where we can see the differences between different collection systems. While EDCS has the advantage of providing timely data, at RPT and program level– it doesn’t have the same accuracy as OnSIS which collects the information at student and course level.
Also, we should add, that all the graphs that we are presenting at province level, are also provided to RPTs at program and RPT levels as part of the SMART Goals reports. The SMART Goals are an excellent tool for RPT s to compare their results against the province averages and have conversations with the Dual Credit team about program improvement.
Success Rate (OnSIS) = 87%

23. Provincial Retention Rate and Success Rates
by Dual Credit Approach,
Approach
Provincial Retention Rate
Provincial Success Rate
Team-taught at secondary school
93%
94%
Team-taught at college
92%
Team-taught level 1 apprenticeship at secondary school
98%
95%
Team-taught level 1 apprenticeship at college
100%
College-delivered course at college
88%
90%
College-delivered course at secondary school
89%
College-delivered level 1 apprenticeship at college location
84%
College-delivered level 1 apprenticeship at secondary school
85%
91%
Cristina –
This slide compares the retention and success rate by program delivery model.
If the audience hadn’t used one of these delivery models, they can still “benchmark” it.
College-delivered college course remains the predominant approach.
Caution: If they’re looking at their own program, it’s important to be mindful if they’re dealing with small numbers when looking at these proportions.

24. OnSIS The Ontario School Information System (OnSIS)
Launched in to collect/manage education data
Secure, web-enabled system
Used by public schools/boards, private schools at elementary and secondary levels.
OnSIS collects data on courses, classes, students and educators three (3) times a year (October, March and June).
There are additional data submissions throughout the year for specific areas.
Suspension/Expulsion
Night School
Summer School
Board Report
Care, Treatment and Correctional Facility Course/Class Enrolment
Technology in Schools
Another source of data for dual credits is the Ontario School Information System.
- It was launched in
It is a secure, web-enabled system
It is used by public schoold and boards and private schools at elementary and secondary levels
It has three submissions – in October, March and June for the course, class, student and educator data.
There are additional data submissions throughout the year for specific areas.
Suspension/Expulsion
Night School
Summer School
Board Report
Care, Treatment and Correctional Facility Course/Class Enrolment
Technology in Schools

25. Ontario School Information System (OnSIS)
Legacy
School
Data
Courses/
Classes
Students
Board Data
Educators
Ontario Education
Number (OEN)
Ministry Educator
Number (MEN)
School
Enrolment
Data
Teacher
Student
Board
Course/
Class
Other Data
Collections

26. This is an overview of how educational data gets into OnSIS
This is an overview of how educational data gets into OnSIS. School boards collect the data through the Student Management Systems and portions of that data is being sent to OnSIS. In OnSIS, the data is validated and depersonalized and it gets transferred in the Ontario Educatio Data Warehouse. This datais analyzed for funding purposes, policy and programs and public reporting. In the Ontario Education Data Warehouse some other information is collected such as EQAO information, Statistics Canada.

27. Data Sources Board Data OnSIS vs Origin of data Historical data
Current data
Personalized
Complete (attendance, full report card information, OSR, teacher knowledge)
Based on what is submitted by boards
Only historical data
Depersonalized
A record of events
Provincial aggregation vs
Cristina
There are a few differences between board data and OnSIS. As we mentioned before, the board data is the source for Onsis data. Also, the boards have both historical and current data , while OnSIS only holds historical data.
The boards have personalized data and when the data gets transferred into OnSIS it gets depersonalized.
The boards collect and hold the complete data (attendance, full report card information, OSR, teacher knowledge) while OnSIS doesn’t. 27

28. Students Identified Through an IPRC, OnSIS 2014-15 = 18%
Alisha
In dual credit students with an IEP as reported on EDCS is 29% and students with an IEP as reported through OnSIS is 30%.
As RPTs and boards are reporting their data more accurately, the OnSIS collection has become very reliable.
In order to reduce reporting requirements , we no longer ask the RPTs to report on IPRC as part of the EDCS collection.
In students identified through an IPRC on OnSIS was 18%
Interestingly, we have suggested to the RPTs to not provide/collect IEP information, but they specifically asked us to keep it, as it is essential information for RPTs to have as they plan their dual credit programs and the amount of special education support they need for those programs.
Students Identified Through an IPRC, OnSIS = 18%
Students With an IEP, OnSIS = 30%

29. Grade 12 Dual Credit Students Who Earned Their OSSD*,
OnSIS (preliminary)
Province
Number
Percentage
Grade 12 Students who earned their OSSD
8,123
75%
* Grade 12 students who took a dual credit in and earned their OSSD by the end of
Source: Data is as reported by schools in the Ontario School Information System (OnSIS), (Preliminary).
Data includes publicly funded schools only.
Cristina - (via Phil) the audience may ask: what about the year later.
Another example of data coming from OnSIS is the number of grade 12 dual credit students who have graduated by the end of the same year. As you can see , that was 75% or 8,123 students

30. Application and Registration
Post-secondary
Application and Registration
Data
What we have done next, or rather my colleagues in ESAB, they linked the Ontario Colleges Application Service (OCAS) data and Ontario Universities Application Centres (OUAC) to see at what PSE institutions the dual credit students are applying after graduation.

31. Dual Credit Students Who Have Applied to go to College
Province
Dual Credit Students
Dual Credit Students
Dual Credit Students
Dual Credit Students
Dual Credit Students
After 1 Year
1,473
2,458
3,358
4,118
4,377
(32%)
After 2 Years
2,347
3,755
5,237
n/a
After 3 Years
2,670
4,246
After 4 Years
2,825
Total Number of Dual Credit Students
13,707
Cristina
In this table, we are looking at the number of dual credit students for the last five cohorts of dual credit students and where they applied in PSE one year , two years …up to 4 years after they took their dual credits.
If we follow the DC cohort, you’ll see that by the fourth year 2,825 students have applied for college. To give you a rough idea – in EDCS (which was at the time the more reliable data source for enrollment, in we had 7,500 students and out of that 2,825 students have applied for college.
was the first year when the quality of the OnSIS data was good enough to use it as the denominator. And we know that 32% of the dual credits applied to college one year after taking a dual credit.

32. Number of Dual Credit Students Who Have Registered for College
Province
Dual Credit Students
Dual Credit Students
Dual Credit Students
Dual Credit Students
Dual Credit Students
After 1 Year
1,050
1,842
2,463
2,664
2,721
(20%)
After 2 Years
1,852
3,088
4,003
4,461
n/a
After 3 Years
2,222
3,612
4,739
After 4 Years
2,428
3,881
Total Number of Dual Credit Students
13,707
Cristina
When following the same logic, we’ll see that 2,428 students from the cohort have registered to a college, and 20% of the dual credit cohort have registered into college in the year following their graduation.

33. Conversion Rate - Percentage of Students Who Applied to and Registered in College by Year
Province
Dual Credit Students
Dual Credit Students
Dual Credit Students
Dual Credit Students
Dual Credit Students
After 1 Year*
71%
75%
73%
65%
62%
Cristina –
When looking at the percentage of students who have registered after graduation, which we called conversion rate – you can see that there was a great decrease in the last two years.
This slide/table allows the audience to ask: “Are these good numbers?” Were there any interventions that we could’ve made.
This is about using the data to stimulate the conversation.
The two goals for the DC program: (1) to complete high school and (2) to promote progression into PSE. It is not good enough to just graduate. RPTs use this data to question…. If this is significant for this RPT team, could we scale the strategies?

34. How do PSE Direct Registration rates compare for the two groups?
Direct Registration Rate for Cohort
Dual Credit
Non-Dual Credit
College
31%
20%
University 7%
35%
Cristina
When we compared the registration rate for DC students with the non-dual credit students – the results are promising – we can see that the dual credit students are much more likely to register to college directly that the general population. This is a positive finding, as one of the two goals of the Dual Credit initiative is to promote successful transition to college.
When looking at university application, the numbers show the reverse, that the dual credit students do not apply to university as much. And that also supports the conclusion that we are reaching the right population – because the main target population for dual credits are students who are disengaged and underachieving with the potential to succeed – and that group is less likely to go to university.
And with that we conclude our presentation – in this presentation we tried to look at the flow of educational data and how it is used to describe the dual credits students as well as their journey in the education system. As you could see , we do have access to a significant amount of information about these students, but we are also missing a few segments that would help us paint a more fulsome picture. It would be nice to know for example, how are these students doing in PSE, or what students apply and get admitted into apprenticeships.
Currently , the OEN has been implemented in PSE data collections and hopefully in the near future we could have a better picture on how are the dual creit program making a difference long term.
We are opening the floor for questions…
Note: Of the full Grade 9 Cohort, 33,022 (20%) students registered directly to college.

35. What does this tell us? For this cohort of students:
The college registration rate is higher among those students who participated in one or more dual credit courses.
This is a positive finding, as one of the two goals of the Dual Credit initiative is to promote successful transition to college.
Cristina

36. Questions?

37. Thank You If you have any questions please feel free to contact us