1. Data Wise Data Presentation
University of Massachusetts Boston
Aimee D’Avignon, Fabian Torres-Ardila, Janna Kellinger, Mike Gilbert, Kevin Ziomek
November, 2014

2. Overview of the Process

3. Step 1: Organize for Collaborative Work
Develop well-functioning, collaborative teams based on the Data Wise Norms:
Take an inquiry stance
Ground statements in evidence
Assume positive intentions
Stick to protocol
Start and end on time
Be here now
-Choose focus area as a department based on data at the dept. level
-Examine program level data to determine action plan

4. Step 2: Build Assessment Literacy
-Understanding what our working conception of assessment literacy is for this project
-Understanding limitations and potential uses for assessment and data
-Understanding affordances and constraints of presenting data in various ways
-Developing a common language around assessment

5. Step 2: Build Assessment Literacy

6. Step 3: Choose a Focus Area
A Focus Area is:
-Related to instruction
-Narrows scope of the inquiry while remaining broad enough to be relevant to many/most of the staff member
Possible Focus Areas:

7. Step 3: Ask a priority question
-Arises from a collaborative process
-Relates to instruction
-Is actionable
-Further narrows scope of inquiry
-Is genuinely intriguing to faculty and staff
Priority Questions:

8. Step 4: Determine a learner-centered problem
-Directly relates to the priority question
-Based on digging into multiple data sources
-Is within the school’s control
-Is a statement about student learning (not a question)
-Is specific and small
Our students struggle with . . .

9. Step 5: Examine Instruction
Reframe the Learner Centered Problem as a Problem of Practice:
-Directly relates to Learner Centered Problem
-Is based on evidence from examining instruction
-Is within our control
Our faculty struggle with . . .

10. Step 6: Develop an Action Plan
-States specifically what teachers will do to address the problem of practice
-Action steps in this plan should be research-based, evidence -based, high-leverage, assigned to specific people, and time-bound

11. Step 7: Plan to Assess Progress
The Plan to Assess Progress:
-Helps us determine whether our action plans are, in fact, helping students make progress towards addressing the LCP
-Should include short-term, medium-term, and long-term assessments
-Should include specific and measurable student learning

12. Step 8: Act and Assess
Make it happen
Evaluate success

13. Our Turn

14. Step 1: Organize for Collaborative Work
Develop well-functioning, collaborative teams based on the Data Wise Norms:
Take an inquiry stance
Ground statements in evidence
Assume positive intentions
Stick to protocol
Start and end on time
Be here now
Two-pronged approach:
1) Datawise Team inquiry projects
2) Dept. wide inquiry project based on our 4 TEAC claims
-Each TEAC claim group examine dept. level data to come up with a learner- centered problem
-Then each program examines program level data to examine instruction and develop and carry out an action plan

15. Step 1: Organize for Collaborative Work
Strengths
Challenges
We have accreditation as motivation
Infrastructure is in place
We have the support of the Dean and the Chair of C&I to work on this project
We are launching a new Undergraduate program which affords us the opportunity to revisit our practices
The structure of the University and College establishes and reinforces data silos. Therefore obtaining rich and illustrative points of data can be time consuming.
scheduling around work responsibilities means that all necessary stakeholders are not able to be present at the same time
Infrastructure (University, College and Department) could be tweaked to improve efficiency
Because we are all stretched for time, the decision was made to use Department meeting time as much as possible to do the department wide Data Wise process work.

16. Step 2: Build Assessment Literacy
One new tool that will be at our disposal is EDWIN:
State data that follows our students from pre-service to in-service
Will include MTELs, PST/PPA data, Teacher Evaluation Scores, Where our students teach, etc.

17. Step 3: Data Overview: Data Inventory
Name of Data Source
Content Area
Dates of Collection
Students Assessed
Who Has Access to these data?
How are these data currently used?
How could these data be more effectively used?
MTEL
C&I, Curric, Content
Varies, prior to Practicum
All
DWT
Exit Survey
Graduation
Christine, faculty
exercise at a retreat
Analyze and share with more persons; programmatic change
GPA
Program entry
Demographics
Program Entry
EDWIN
Exit and Employment Data
Aimee
Brand new
Analyze and share, prepare graduates, support claims
PPA
Lisa, Christine,  GPDs
TEAC, licensure
Identify areas where our students struggle.
Admissions

18. Step 3: Data Overview: Assess Assessments
Exit Survey (Administered Spring 2013 and Spring 2014)
Administration is given twice per year (at graduation): the survey assesses students’ perspectives of the program and their preparation (feelings)
Categories do not add up to 100%
Some prompts are questions whereas others are statements
The data displayed corresponds to a subset of 14 questions. This subset was classified as “Student perception of pedagogical competence” by the data team.
N is small (Spring 2013 = 11; Spring 2014 = 28)

19. Step 3: Data Overview: Choose a Focus Area
The focus area for our first inquiry cycle is: TEAC CLAIMS
Our rationale for choosing this focus area is:
We choose these claims because they are important to our College Mission of preparing urban teachers
Our priority question was:
How well are we preparing our graduates to meet our TEAC Claims?
Our graduates will:
- demonstrate cultural competence and address social justice issues in urban and other diverse contexts. (QP 1.3, 1.4.2)
- demonstrate knowledge of content and pedagogical and assessment practices that promote learning. (QP 1.1, 1.2, 1.4.3)
-incorporate family and community resources into their practice. (Q.P. 1.1, 1.2, 1.4.3)
-demonstrate the skills necessary to engage in professional and life long learning. (QP 1.41, 1.4.3)

20. Step 4: Digging into Student Data
DataWise Team Inquiry Cycle 1

21. Step 4: Digging into Student Data
DataWise Team Inquiry Cycle 1
Student Perception of Pedagogical Competence
Question: My teacher education program prepared me to…
1. UMass-Boston teacher preparation faculty structured their course around real problems of teaching practice.
2. Use the state's curriculum frameworks and standards to plan instruction.
3. Teach content knowledge and skills.
4. Use inquiry methods to create effective learning environments
5. Teach problem solving, conceptual understanding, and other aspects of higher order thinking.
6. Motivate students to participate in academic tasks.
7. Use educational technology as a learning tool.
8. Teach students with different ability levels in the same class
9. Reflect on and improve my teaching performance
10.Teach in a high-stakes testing environment.
11. Use classroom research and inquiry strategies
12. Read and understand Individual Education Plans (IEPs) and provide appropriate accommodations for the individual students in my classroom
13. Use classroom management techniques/procedures.
14. Read and understand 504 plans and provide appropriate accommodations for individual students in my classroom.

22. Step 4: Digging into Student Data
DataWise Team Inquiry Cycle 1
Strongest Aspects:
“faculty brought a wealth of knowledge to the classroom. The minimum amount of experience the adjunct faculty had was 20+ years, that kept my experience focused on the realities that I will experience.”
“Most of the staff actually work in BPS and were great resources to have.”
Suggestions for Program:
“More hands on instruction opportunities.”
“Application of theory into practice”
“Spend more time on practical things like classroom management.”
“More emphasis on day-to-day classroom situations: behavior management, how to deliver assessments, how to interpret test results”
“More attention to practical aspects of teaching; e.g., classroom management, lesson planning”
“More practice less theory.”
“Focus more on the practical aspects of teaching like pedagogy.”

23. Step 4: Digging into Student Data
DataWise Team Inquiry Cycle 1
Qualitative Data Summary: While instructors have a lot of real-world experience and knowledge of teaching in urban schools, students expressed a need for more practice with real-world situations likely to occur in urban schools.
This was reflected in the quantitative data as well. For example, students rated “structuring classes around real teaching problems” high, but some of the practical applications like using standardized assessment results, classroom management, and using IEP and 504 plans rated low.

24. Step 4: Digging into Student Data
DataWise Team Inquiry Cycle 1
Practical Applications
Classes structured around real-world problems

25. Step 4: Digging into Student Data
DataWise Team Inquiry Cycle 1
In addition, another quantitative measure on our exit survey supported the aspect of our conclusion relating to instructor strength:
The UMass Boston Teacher Preparation faculty know a lot about the reality of contemporary schools.
Spring % Spring %

26. Step 4: Digging into Student Data
DataWise Team Inquiry Cycle 1
Learner-centered problem:
While our instructors provide many real life examples from their own experiences, our students struggle to apply them to their own teaching.

27. Step 5: Examine Instruction
DataWise Team
DataWise Team Inquiry Cycle 1
Problem of practice:
Our instructors impart their own real-life experiences but need to build in more ways for students to practice applying this knowledge

28. Step 6: Develop an Action Plan
DataWise Team Inquiry Cycle 1
Potential solutions:
Use instructors real-life experiences to build: in-class role-playing scenarios, case-based teaching, micro-teaching, problem-based learning, analysis and application of real data, simulations, etc.

29. Step 7: Plan to Assess Progress
DataWise Team
DataWise Team Inquiry Cycle 1
-Data from future exit surveys
-Examining syllabi
-Examining student work, e.g. core assignments
-Examining PST scores

30. Step 8: Act and Assess
DataWise Team Inquiry Cycle 1
We are exploring the use of simulations, case studies, role-playing, etc.

31. Our Turn: Inquiry Cycle 2

32. Step 3: Choose a Focus Area
DataWise Team Inquiry Cycle 2
A Focus Area is:
-Related to instruction
-Narrows scope of the inquiry while remaining broad enough to be relevant to many/most of the staff member
Possible Focus Area:
What challenges are our teacher candidates facing that we could better prepare them for?

33. Step 3: Ask a priority question
DataWise Team Inquiry Cycle 2
A Priority Question:
-Arises from a collaborative process
-Relates to instruction
-Is actionable
-Further narrows scope of inquiry
-Is genuinely intriguing to faculty and staff
Priority Question:
How well prepared are our students to meet the expectations of new state standards and assessment?

34. Step 4: Digging into Student Data
DataWise Team Inquiry Cycle 2
This is a second slide to put the oval and text box in

35. Step 4: Digging into Student Data
DataWise Team Inquiry Cycle 2
High Stakes Testing

36. Step 4: Digging into Student Data
DataWise Team Inquiry Cycle 2
Over the two years of data, the area with the lowest overall rating was “Teach in a High Stakes Testing Environment” The two lowest rating items on the 2014 survey were “teach in a high-stakes testing environment” and “interpret and use standardized test results”

37. Step 4: Some Qualitative Quotes
DataWise Team Inquiry Cycle 2
“more emphasis on day-to-day classroom situations: behavior management, how to deliver assessments (TRC, Dibels, etc.), how to interpret test results”
“There were a few teachers (2) who were out of date on the new issues facing teachers such as Race to the Top or the new evaluation system.”

38. Step 4: Digging into Student Data
DataWise Team Inquiry Cycle 2
Learner-centered problem:
Our students struggle with understanding and implementing effective assessment strategies as well as interpreting and applying external assessment results within the context of PARCC.

39. Step 5: Examine Instruction
DataWise Team Inquiry Cycle 2
Problem of practice:
Given that schools and teachers are increasingly held accountable for student achievement, faculty need to better prepare our students for teaching in this high-stakes testing environment.

40. Step 6: Develop an Action Plan
DataWise Team Inquiry Cycle 2
Potential solutions:
-Infusing assessment practices across courses
-professional development around Common Core

41. Step 7: Plan to Assess Progress
DataWise Team Inquiry Cycle 2
-Data from future exit surveys
-Examining syllabi
-Examining student work, e.g. core assignments
-Examining PST scores

42. Step 8: Act and Assess
DataWise Team Inquiry Cycle 2
We have asked someone to talk about how Boston Public Schools analyzes and uses data for improvement at our last department meeting.

43. Your Turn