1. Amid New, Different, and Traditional Ways of Knowing: Bringing a Networked Improvement Approach to Scale in EdD Curriculum Assessment
Joy C. Phillips, PhD
Kathy D. Geller, PhD
Kenneth J. Mawritz, PhD

2. Session Learning Objectives
Illustration of Improvement Science (IS) Usage
Specifically, Networked Improvement Communities (NIC)
As mechanism to facilitate EdD program curriculum assessment and continuous program improvement.
(Bryk, Gomez, & Grunow, 2011; Bryk, Gomez, Grunow, & Lemahieu, 2015; Russell et al., 2017)
Demonstration of Lewin’s (1951a) force field analysis method
To guide faculty conversations
About curriculum assessment and improvement.
| 2

3. Session Overview Introduction 5” Application of NIC to Our Work 5”
Overview of Our Work ”
Use of force field analysis ”
Table discussions ”
Full group reflections ”

4. Networked Improvement Community (NIC), Defined
“An intentionally designed social organization with a distinctive problem-solving focus; roles, responsibilities and norms for membership; and the maintenance of narratives that detail what they are about and why it is important to affiliate with them” (Bryk, Gomez, & Grunow, 2011)

5. NIC: Essential Elements
1) Focused on a well specified common aim,
2) Guided by deep understanding of the problem, the system that produces it, and a shared working theory on how to improve it,
3) Disciplined by the rigor of improvement science, and
4) Coordinated to accelerate the development, testing, and refinement of interventions, their rapid diffusion out into the field, and effective integration into varied educational contents.
| 4

6. Our Work
Drexel University School of Education EdD faculty have engaged in a comprehensive program assessment process that:
Expands existing program learning outcomes, Keystones, into mastery statements,
Aligns the EdD program with national, regional, and university standards,
Identifies measurable formative and summative outcome data/evidence,
Describes/proposes how measurable data/evidence wil be reviewed,
Describes/proposes reporting and follow-up processes.

7. Description: Our Process
Call for EdD Advisory Committee Faculty, Jan 2017, to conduct Prep work for CAEP visit;
Volunteer work team, 3 Clinical Associate Professors;
Worked via web conferencing and from 3 states (NC, CA, PA) from Jan 2017-to-date;
Worked in an organic fashion to respond to SoE template;
Engaged in rich discussion, critical reflection, questioning, debate, reconsideration, finally agreement;
From tacit to explicit norms: shared understanding that the group would work collaboratively, honor each other’s contributions respectfully, and allow unlimited “push-back” from any member on any point at any time.
SECTION TITLE | 2

8. EdD Program Learning Objectives: Keystones
EdD graduates possess the abilities to create and support communities that are the bases for sustainable change
Leaders develop the habits of mind and competencies to lead complex organizations shaped by global forces
Leaders develop the abilities to sustain their own leadership growth
EdD graduates utilize the full range of emerging technologies to reach across generations, communicate effectively, and engage others in meaningful change
EdD graduates exemplify the curiosity, inquiry skills, and scholarly competencies needed to investigate an idea and transform it into meaningful action.
HEADLINE
SECTION TITLE | 2

9. Program Assessment Planning Tooling Tool
School of Education:
Program Assessment Planning Tooling Tool
Program Learning Outcomes (PLOs)
DSLPs/ Standards/
SoE Themes
Data/Evidence
Review of Evidence
Reporting and
Follow-up
WHAT are the PLOs? PLOs are learning expectations, expressed in active, measurable terms
WHICH [university] Student Learning Priorities (DSLPs), Standards (InTASC, CAEP, SPA) and SoE themes align with each outcome? (See list below)
WHAT data or evidence will be used to determine whether or not the outcome is met and/or level of impact on P-12 learning?
Provide:
--Type of measure
--Type of evidence
--# of students assessed
--Where in the curriculum or program is the outcome assessed?
--When and how often will the outcome be assessed?
HOW will program data/ evidence be reviewed?
Identify:
--the evaluation tool (rubric; evaluation summary form; statistical tool)
--performance target, achievement level, or benchmark
--Who will review the evidence?
--What will be the process?
--When and how often will the evidence be reviewed?
WHAT is the plan for reporting the results and findings (strengths and gaps)? I
dentify:
--To whom will the quantitative and/or qualitative results be reported? How often?
-- Who will be responsible for reporting the results and findings? For tracking the follow-up?
SECTION TITLE | 2

11. Articulated Program Learning Outcome (PLO) for EdD Keystone 2a
Program Learning Outcomes
Standards: CAEP, CPED, DSLPs,
SOE Themes
Data/Evidence
Review of Evidence
Reporting and follow-up
Keystone 2:
Leaders develop the habits of mind and competencies to lead complex organizations shaped by global forces.
CAEP
1 Content & Pedagogical Knowledge
2 Clinical Partnerships & Practice
4 Program Impact
5 Provider Quality Assurance & Continuous Improvement
EdD students participation/performance as evidenced by the following:
Utilize systems thinking to address leadership standards through knowledge base development and application (EDUC 800, 845, 801, 802)
Leadership: individual and team projects
Theory U projects
Create an e-portfolio of student academic and professional accomplishments
(e.g., writing assignments, sample discussion boards, team projects, course activities, IRB approval letter, SOE/GS funding letters, juried conference presentations, community presentations, juried publications, etc. )
Proposed - Create a review process for the e-portfolio.

12. Articulated Program Learning Outcome (PLO) for EdD Keystone 2b
Program Learning Outcomes
Standards: CAEP, CPED, DSLPs, SOE Themes
Data/Evidence
Review of Evidence
Reporting and follow-up
Learning Outcome:
Demonstrate mastery for leading systemic change drawing from a range of theoretical frameworks:
(a) Systems Thinking
(b) Leadership & Adaptive Leadership & Theory U
(c) Creative Problem Solving and Design Thinking
(d) Communication
CPED
1 Ethics/Social justice
2 Positive Difference
3 Collaboration/ communication
Theory w/systemic inquiry
DSLP
1 Communication
2 Creative & Critical Thinking
4 Information Literacy
6 Technology Use
7 Global Competence
8 Leadership
(b) Identify & develop leadership style
(EDUC 800, 845, 801)
Academic papers & team projects
Technology tools
(c) Creative Problem Solving & Design Thinking
(EDUC 800, 804, 845, 801, 802, 803, 810, 818)
Utilize leadership and research to design dissertation
Comprehensive examination rubric
Leadership; Action oriented research and evaluation;
Keystones,
Writing and APA
as scored independently by two professors with a rating of 2.0 (scale 0 – 3) or higher on all categories
CITI Certification
Create a central repository for CITI Certifications (obtained during EDUC 810)
Semi-annual report (Fall/Spring) Comprehensive Examination outcomes by Program Director
Program Manager provides annual listing of CITI certifications for active students

13. Articulated Program Learning Outcome (PLO) for EdD Keystone 2c
Program Learning Outcomes
Standards: CAEP, CPED, DSLPs,
SOE Themes
Data/Evidence
Review of Evidence
Reporting and follow-up
Keystone 2:
Leaders develop the habits of mind and competencies to lead complex organizations shaped by global forces.
SOE 1-Innovation & Creativity
SOE 2-Diversity
SOE 5-Leadership
(d) Effective communication principles (EDUC 800, 845, 801, 802, 802)
Academic papers & team projects
Discussion boards & voice threads
(e) Summative evidence includes the following:
Comprehensive
CITI
Dissertation Proposal
Final Dissertation
Dissertation Proposal Defense Committee Approval
(Forms D-2 & D-2a)
Dissertation Final Defense (Forms D – 3 to D- 5)
Dissertation Approval Form
Completion Form
Work with HRPP to create IRB approvals annual report for EdD research
Create (from D2) Annual report by quarter on doctoral proposal
Create (from D5s) Annual report by quarter on doctoral defenses approved

14. Lewin’s Force Field Analysis
SECTION TITLE | 2

15. Step 1: Working as a Team Identify each driving force and each restraining force in your particular situation
Driving Forces
Restraining Forces
CAEP Accreditation
CPED Alignment
Standards
Leadership Team Effort
Program Excellence
Effective Marketing
Faculty Time
Faculty Knowledge - EdD Program
Difficult Conversations
Resistance to Change
Create New Reporting Systems
Data Analysis Time
Philosophical Differences – EdD and PhD

16. Step 2a: Working as a team
Provide a “brief” definition for each driving force
Driving Forces
CAEP – Council for the Accreditation of Educator Preparation
CPED – Carnegie Project on the Education Doctorate: Alignment Activity
Standards – University, Regional, State, and National Norms
Leadership Team Effort – Steering Committee to Design and Facilitate Process
Program Excellence – Scholarly Commitment to EdD Mission
Effective Marketing – Design a Dynamic Marketing Approach

17. Step 2b: Working as a Team Provide a “brief” definition for each restraining force
Restraining Forces
Faculty Time – Window of Time and Responsibilities
Faculty Knowledge of EdD – Expectation(s) Awareness of the EdD Program Design
Difficult Conversations – Time and Willingness to Discuss the EdD Program
Change – Resistance Associated with a Change in Heart, Mind, & Will
Create New Reporting Systems – Designing New Systems to Provide Evidence
Data Analysis Time – A Staff Window of Time Opportunity to Provide Evidence
Philosophical Differences – a Confluence of PhD, EdD, Personal Beliefs, & 21st C Ideas

18. Step 3: Working as a Team
Identify the priority for each driving force by assigning a percentage to it. Initially, the total percentage may be over 100%.
Identify the priority for each restraining force by assigning a percentage to it. Initially, the total percentage may be over 100%.
Ultimately, ensure that the calculation of each set—driving and restraining forces—equals percentages 100%.

19. Table 1: Sample Percentages Driving and Restraining Forces
Driving Forces %
Restraining Forces
CAEP Accreditation 22
Faculty Time 24
CPED Alignment
Faculty Lack of Knowledge of the EdD Program 16
Standards 6
Difficult Conversations 15
Leadership Team Effort
Resistance to Change 19
Program Excellence 18
Create New Reporting Systems 3
Effective Marketing 8
Time for Data Analysis (Existing Data and Emerging Data) 4
Philosophical Differences in EdD Program Intent
Total
100 %
SECTION TITLE | 2

20. Step 4: Working as a Team
Design a priority pie to symbolize the driving force priorities
Design a priority pie to symbolize the restraining force priorities

21. Priority Pies: Comparison
Driving Forces
Restraining Forces

22. Step 5: Working as a team, “next steps” include the following:
Analyze the priorities established in the driving force column and the restraining force column
Considering this analysis, design a dynamic ongoing plan to review all aspects of the Force Field Priority Protocol
Step 5 provides the launch for effective next steps. Good Luck!

23. Table Talk If attending as a team, work together.
Using the Force Field Analysis Process
Identify your EdD program’s driving and restraining forces.
Discuss at your table various institutional/program response.

24. Reflection Questions? Want to discuss further?
Joy Phillips
Kathy Geller
Ken Mawritz