1. New Technologies, Innovation, and Quality Assurance:
How to Consider QA and its Key Principles in a World of Disruptive Technology and Change in the Academy
Troy Williams
Managing Director, University Ventures
March 2019

2. Technology and Quality Assurance in Higher Education1
Market pressure for faster degree programs Rise of alternative credentials Growth of online and competency-based education New quality assurance = learning verification 2 3 4

3. Roadmap 1
Market pressure for faster degree programs Rise of alternative credentials Growth of online and competency-based education New quality assurance = learning verification 2 3 4

4. 26,000+ colleges and universities worldwide
Over 10,000+ colleges and universities worldwide founded in the past 50 years
From…
…To
Year
1500s
2019
Number of institutions 66
universities, primarily in Europe
26,000+ colleges and universities worldwide
Would estimate 10K+ universities in the last 50 years, mainly because US grew from 1,700 universities to 4,000+. Would conservatively estimate that 40% of universities worldwide have developed since 1950
SOURCE: World Bank; Our World in Data

5. “Peak University” - US university count declining
SOURCE: US National Center for Education Statistics (NCES)

6. A research-oriented, faculty-centered, isomorphic university model
Typical University Model
Objectives
Advance knowledge and research
Prepare students with research and critical thinking skills
Serve society through basic research and innovative thought
Governance
Academic departments organized around same set of canonical disciplines (humanities, social sciences, natural sciences, engineering)
“Guild-like” tenure system rewards faculty conducting research in core disciplines for traditionally recognized journals

7. Employee tenures are decreasing
Percent Decrease ( )
-8%
-9%
-13%
-8%
SOURCE: US Bureau of Labor Statistics

8. Technology is remaking the workplace rapidly
Artificial intelligence/ automation
2007 Smartphone
adoption
2000s Cloud- based enterprise software
2000
Social media
1990s ,
Internet
1980s Personal computer

9. US student count declining
SOURCE: US National Center for Education Statistics (NCES)

10. Global participation in higher education
SOURCE: World Bank; Our World in Data

11. Increasingly, students go to college to get a better job
SOURCE: National Freshmen Survey (UCLA)

12. Two models of education
Employment
Front-loading
Just-in-time
Age
20s
30s
40s
50s
60s

13. Coding “boot camps” address student employment goals
Traditional University
Coding Boot Camp
Length of study
4 years
2-3 months
Total tuition
$200K+
$15-20K
Job placement rate
81%
85%+
Expected salary
$50K
$85K

14. Boot camps will proliferate across many industries
Selected Industries
Illustrative Boot Camps
Software Engineering
Medical Training
Design
Sales

15. New technical skills are necessary

16. The skills gap has widened
SOURCE: US Bureau of Labor Statistics

17. Is the Skills Gap a broad phenomenon?
SOURCE: Trading Economics

18. The mechanics and technology of hiring have changed
75%
Hiring managers that use Applicant Tracking Systems:
<1%
2000:
Today:
73%
Companies that use social media to recruit:
<1%
2000:
Today:
Most popular application mediums:
2000:
Today:

19. Why are employment outcomes worse? Proliferation of technical skills

20. In response, universities are launching boot camps
In the US, more than universities have launched coding boot camps
SOURCE: Course Report

21. How the Skills Gap will impact higher education
Boot camp and shorter degree programs will proliferate
Second and third tier universities must adapt
Regulation needs to support these programs
The traditional 3-4 year degree will decline in popularity
Boot camps will be integrated into higher education

22. Roadmap 1
Market pressure for faster degree programs Rise of alternative credentials Growth of online and competency-based education New quality assurance = learning verification 2 3 4

23. Micro-credentials address the communications gap between educators and employers
LEARNING PROVIDERS
EMPLOYEES
EMPLOYERS
Certificate
Resume
Job Description

24. Digital credentials are at the nexus of key market trends
89% #1
70%
51%
Engagement a Priority
Millennials
Skill-Based Hiring
Skill-based Learning
Demand for machine-readable, actionable data about employees & learners
89% HR pros: a recognition system at work improves the employee experience. Lifts productivity & the bottom line.
Almost 75% of workforce, they rank learning & growth as top priority in job selection.
Reduces cost-to-hire by 70%, time-to-train by 50%; Companies organize around skills, not job titles.
51% employers already moving to competency-based microlearning.
$1.6T
81%
85%
43%
Portable, digital, verified achievements
Debt
Certifications Growing
Lack of Trust
Gigs & Freelancing
$1.56T in US student debt and degree half life is short; other credentials are on the rise.
81% of adults with a non-degree credential say it’s very useful for getting a job.
85% of employers found a lie or misrepresentation on a resume or job application.
43% of workforce by 2020 will be free agents, driving demand for trusted skill indicators.
SOURCE: Credly

25. Micro-credentials will require a marketplace, verification, and collaboration among issuers
Individual Candidates
Academic Institutions
Credential Marketplace
Credential Issuers
Verification Intermediaries
Employers
Recruiting Intermediaries
Training Providers

26. Less selective universities will be impacted more by micro-credentials
Impact of Micro-Credentials
Most selective institutions
Least impacted
Moderately selective institutions
Least selective institutions
Most impacted

27. University degrees will have value based on graduate performance
Driver rating
Seller rating
Restaurant rating
University rating

28. Regulations are tied to time-based measures
Minimum Program Lengths at US Higher Ed Institutions
Associate’s, Bachelor’s, Graduate or Professional Degree
At Least a Two-Year Program that Is Acceptable for Full Credit toward a Bachelor’s Degree
At Least a One-Year (Federal Definition) Program that Leads to a Credential and Prepares Students for Gainful Employment in a Recognized Occupation
Minimum Program Lengths at US Proprietary Institutions
At Least 15-Week Undergraduate Program with 600 Clock Hours or 16 Semester or 24 Quarter Credits, for Students without an Associate’s Degree
At Least 10-Week Program with 300 Clock Hours or 8 Semester or 12 Quarter Credits at Graduate Level, only for Students with at Least an Associate’s Degree
At Least 10-Week Undergraduate Program with Clock Hours that Admits Students without an Associate’s Degree and that Has Verified Completion and Placement Rates of at Least 70%
SOURCE: US Department of Education FSA Handbook

29. The US Department of Education launched an innovative model with EQUIP
EQUIP provides federal financial aid to eligible students enrolled in non-traditional training programs (e.g., boot camps)
The program will improve access to innovative training opportunities that lead to good jobs and strengthen outcomes- based quality assurance processes
Non-Traditional Provider
Institution
Program
1-year certificate in Management and Leadership Fundamentals
Associate of Science in Business Concentration
Associate of Arts in Criminal Justice Concentration
B.S. in Advanced Manufacturing
SOURCE: US Department of Education

30. $5K tuition value 2-7 2-5% of income $5-10K value returned
Income Share Agreements (ISAs): the Next Generation of Student Financing
$5K tuition value
2-7
year obligation while working
2-5% of income
$5-10K value returned
An ISA is a contractual agreement in which a student receives education funding in exchange for a percentage of post-graduation income over a defined number of years
SOURCE: Vemo Education

31. Universities worldwide are adopting income-linked financing programs
Income-share agreements
Income-based loan repayment
Purdue University Income-Share Agreements Could Solve Debt Crisis (2017)
U of U launches financing program to help students finish their degrees (2019)
Clarkson University [and others] have started [ISA] programmes..this could revolutionize higher ed (2019)
SOURCE: The Atlantic; Fox News; Financial Times

32. Credly is a marketplace that houses and verifies credentials
Portable, digital, verified achievements
20M+ credentials documented
10M+ people; 195 countries
400+ providers and schools
Partners include corporations, training providers, and certification associations
SOURCE: Credly

33. No single point of failure
What is blockchain?
Distributed Ledger
System of record with new transactions propagated to entire network
No single point of failure
Centralized data bases prone to tampering or physical damage
Transparent
Immutable record of all transactions
Secure
Economically impractical and physically challenging to hack
Blockchain – a continuously growing list of records, called blocks, which are linked and secured using cryptography

34. Many universities are exploring digital credentialing

35. Credentials will stack to advance an employee’s career
SOURCE: Credly

36. StraighterLine lowers the cost and risk of pursuing a degree
Offers 60+ online, competency- based, general education courses directly to students for a $99/month subscription, plus one-time course fee of ~$59.
Enrolled 30,000 unique students over the past twelve months
Credit transfer agreements with regionally accredited colleges and universities.
Results: 30% yield of students who enroll in the program, with higher retention and faster progress
SOURCE: StraighterLine

37. How the rise of alternative credentials will impact higher education
From…
…To
Certificates and degrees varying from 3 months to 3 years
3 or 4 year degrees
Universities among many providers in a credential marketplace
Universities as the only credential provider
Front-loaded education
Just-in-time delivery as part of lifelong journey

38. Roadmap 1
Market pressure for faster degree programs Rise of alternative credentials Growth of online and competency-based education New quality assurance = learning verification 2 3 4

39. Students are moving online
Students Enrolled in at Least One Distance Course
Ground Students
Percent Students Enrolled in at Least One Distance Course
25.9%
27.9%
29.2%
31.2%
33.1%
Percent Students Enrolled Exclusively in Distance Courses
11.9%
13.6%
14.0%
14.7%
15.4%
SOURCE: US National Center for Education Statistics (NCES)

40. Over half of on-ground students take online courses in Oregon Public Universities
SOURCE: Oregon Higher Education Coordinating Commission; Office of Research and Data

41. Competency-Based Education (CBE) is emerging
Characteristics of CBE
A mastery-based measure of study, not a time-based measure
Theory is integrated with skill practice; knowledge supports performance
Learning is self-paced, flexible, skill-based, personalized, and affordable
Attitudes
Competence
Skills
Knowledge

42. The Carnegie credit hour is a time-based model of mastery
40 x
120 x
1 degree
40 courses
120 credit-hours

43. Western Governor’s University has made competency-based education its cornerstone
Online university launched in 1997 to expand access through online, competency-based programs
Currently has over 100,000 students, or 0.5%+ of all US college students
Uses over 1,800 attributes to flag at-risk students and deliver interventions
Delivers a highly personalized education experience, allowing each student to move at their own pace
Aligns students’ financial incentives with course completion and graduation
SOURCE: WGU Press Releases

44. Students are moving to online and competency-based education
SOURCE: US National Center for Education Statistics (NCES)

45. Roadmap 1
Market pressure for faster degree programs Rise of alternative credentials Growth of online and competency-based education New quality assurance = learning verification 2 3 4

46. Western Governors University’s competency-based model questioned
Education Department's Inspector General probes Western Governors University (2015)
The Education Department's Office of Inspector General is auditing whether a leading institution in competency-based education complied with federal financial aid and other regulations…
The biggest theme , he said, seems to be "regular and substantive interaction," a federal rule that says students in competency-based programs must have a certain amount of contact with instructors.
U.S. Department of Education Issues Final Audit Determination for WGU (2019)
Today, the U.S. Department of Education’s Federal Student Aid (FSA) office determined Western Governors University (WGU) is indeed eligible to participate in federal student aid programs.
FSA determined that, particularly in light of a lack of clear guidance from the Department at the time of the audit period, WGU’s efforts to comply with the governing law and regulations were reasonable and undertaken in good faith.
SOURCE: Politico; US Department of Education

47. How the rise of online, competency-based programs will impact higher education
Regulatory framework must be updated
Universities must focus on the quality of outcome rather than quality of inputs
Universities must adopt new learning verification models
Online, faster, competency-based programs require modern quality assurance standards

48. Degrees are often signals for hiring managers
What is Signaling?
Signaling occurs when the actions of a person with privileged information, an “insider,” triggers selling or buying behavior by those who do not have information
The insider’s behavior is taken as a ‘market signal’ to others about a company’s value – hence, ‘signaling’
[Many economists] believe that college…is mostly signaling. Don’t most people go to college because they think it will get them a job?
SOURCE: Market Business News; The Library of Economics and Liberty

49. Generic college degrees don’t increase performance
Higher Education Is Overrated; Skills Aren’t (2010)
“Knowledge may be power, but “knowledge from college” is neither predictor nor guarantor of success.”
“More and more, employers are going to want to see some proof that a potential employee has actually gained particular skills.”
Degrees don’t matter anymore, skills do (2015)
Even in a robust economy, all graduates are not equal (2018)
“Students…going to less selective schools, maybe getting two-year degrees instead of four-year degrees…might have less specialized skills.”
SOURCE: HBR, Quartz, Marketplace.org

50. No degree, no problem: PwC, Deloitte, and EY hiding college degree information from hiring managers
De-emphasizing formal education in recruitment process, beginning 2015
Deloitte hiding the name of the candidate’s university from recruiters in its search for entry-level talent
EY eliminating the requirement of a college degree
PwC hiding from hiring manager whether applicants have a college degree
Goal is to diversify their talent pool and focus on candidate attributes that more positively correlate with strong job performance
SOURCE: SHRM.org

51. 15 large multinational companies recently announced that no college degree is required to apply
“Companies will hire the candidates whose experience and skills best suit them for the job. Many of those successful applicants will have university degrees. Some of them will not.”
SOURCE: Quartz; Glassdoor

52. Authenticating learning in the online space
Key questions for online learning verification
How does the online program ensure the student of record has done the work?
How are they authenticated? How frequently?
How effective are authentication technologies?

53. Selected Authentication Methods
There is a wide variety of authentication technologies commercially available
Selected Authentication Methods
Live ID
Test-taker presents identification to proctor with video recording
Mobile biometrics authenticate test-taker
Dual log-in technologies (mobile and PC)
Mobile ID
Biometrics including keystroke matching, facial recognition, knuckle recognition, and voice recognition
Auto ID

54. The challenge with authentication in online learning
Most Authentication Systems
Authentication in Online Learning
User cooperates with institution to prevent fraud
User (student) often attempts to commit fraud against institution (cheating)
Example: bank wants person to be able to access account but deny fraudsters
Example: student tries to fool system into thinking that another student is them v. v.
User
System
Fraudster
User
Fraudster
System

55. Online proctoring / remote invigilation is growing rapidly
Estimated US Online Proctoring Market Size
2018
2023
$510M
$136M
28-30%
Annual Growth Rate
Growth of online learning will drive rapid growth in online proctoring / remote invigilation
SOURCE: 2015 Survey of Online Learning; NACE Career Services Survey; Non-profit Form 990s; NCES; IPEDS;
Association of Governing Boards of Universities and Colleges; Council of Graduate Schools; Tyton Partners

56. Online proctoring generally comes in three forms
Proctoring Offering
Description
Most secure offering – test-taker accesses platform and is monitored by remote proctor via audio, video, and screen sharing
Live Proctoring
Record & Review
Audio, video, and screen share feeds are recorded for duration of exam and are subsequently reviewed
Captures audio, motion, and systematic changes to identify inappropriate behaviors with reviewable video recording
Monitors test-taker and alerts instructor of inappropriate behaviors with photo evidence
Automated Proctoring

57. Challenge for quality assurance: Institutional conflicts with enforcement
Online proctoring platform generates flags, with reviewable evidence
Anomalies in behavior
Rule violations
Outright cheating
However, what does the institution, department, or professor actually do with these flags?

58. New technologies and trends driving the need for learning verification
How are institutions reviewing instances of cheating, and how will they hold students accountable?
In today’s market environment, it may not be in the near- term interest of an institution to penalize the student
However, it is in the long-term interest of higher education to make sure that these degrees stand for actual learning

59. With these trends in mind, quality assurance entities must prepare for the future of higher education
Shorter, faster, degrees
Online learning
Credentials marketplace
Learning verification
Competency-based learning

60. University Ventures