1. Hilary Caldwell1,2, MSc, PhD Student
Development of the Physical Literacy Environmental Assessment (PLEA) Tool
Hilary Caldwell1,2, MSc, PhD Student
Co-Authors: Alison Bochsler3, MPH & Brian Timmons1,2, PhD
1Department of Kinesiology, McMaster University; 2Child Health & Exercise Medicine Program, McMaster University; 3City of Hamilton Public Health Services
International Physical Literacy Conference
April 13, 2017

2. How do we know if we’re physically literate?
Measure at the individual level or program level
We have to measure it!
© Child Health & Exercise Medicine Program, McMaster University, 2017

3. Individual Evaluation Tools
self
parent
fun
coach
basic
inventory
The PLAY Tools, Passport for Life and CAPL have all become increasingly popular individual assessments of physical literacy. Each tool is appropriate for different settings and situations, but they still all only assess the physical literacy of individual. Don’t get me wrong, this is extremely important and I applaud all the work that has gone into creating and testing these tools. But, to help children and youth develop physical literacy, we need quality programming. And this quality programming needs to be evaluated.
© Child Health & Exercise Medicine Program, McMaster University, 2017

4. Program Evaluation
Systematic collection, analysis and reporting of information about a program to assist in decision making
© Child Health & Exercise Medicine Program, McMaster University, 2017

5. Why conduct program evaluation?
1. Determine program effectiveness
The extent to which a program is meeting its expected outcomes
2. Demonstrate accountability to stakeholders or funders
3. Inform decisions to improve program
© Child Health & Exercise Medicine Program, McMaster University, 2017

6. Why did we make the PLEA Tool?
Rationale:
All existing tools assess physical literacy at the individual level
Need a tool to assess how programs support the development of physical literacy in participants
© Child Health & Exercise Medicine Program, McMaster University, 2017

7. Developing the PLEA Tool
Phase 1:
Pilot Tool
2014
Phase 2:
PLEA Tool Testing & Validations
Phase 3:
National Consultation
2017
© Child Health & Exercise Medicine Program, McMaster University, 2017

8. Tool Development Phase 1: 2014
Step 1: Local Environmental Scan
Sectors were asked to identify any existing tools related to physical literacy that could be used as framework for the development of an audit tool
Examples of 15 Identified Tools:
PLAY Tools (Sport for Life)
Passport for Life (PHE Canada)
Educating for Physical Literacy Checklist (PHE Canada)
CIRA Ontario List of Games by Physical Literacy Skills
Quality Sport Checklist (Sport for Life)
School PE Program Checklist (NASPE)
High Five: Quest 2 Program
School Physical Education Delivery Assessment Tool
CATCH (University of Texas School of Public Health)
© Child Health & Exercise Medicine Program, McMaster University, 2017

9. Tool Development Phase 1
Step 2: Content Development
Contacted 15 content experts
“We would like your input as to what elements, information, or considerations of physical literacy do you feel are important and essential to sport- or physical activity-related programming for children and youth.” 
“In other words, what would a program have to include (that could be measured) to be considered physical literacy ‘friendly’?”
© Child Health & Exercise Medicine Program, McMaster University, 2017

10. Tool Development Phase 1
Thematic Content
ENVIRONMENT (14)
Variety of environments
PROGRAMMING (12)
Emphasis is placed on the development of movement and sports skills rather than competition
LEADERS AND STAFF (8)
Program Leaders are trained and certified by the appropriate governing bodies relevant to the activity
VALUES AND GOALS (8)
Emphasis is on learning and improvement
© Child Health & Exercise Medicine Program, McMaster University, 2017

11. Tool Development Phase 1
In each section below, please check all boxes that apply to your program. In the corresponding comment box, please indicate why boxes were NOT checked, if applicable.
ENVIRONMENT
Access to indoor opportunities for activity
Access to outdoor activities for activity in a variety of environments
Variety of environments available include (check all that apply)
Ice/snow
Land
Water
Air
Appropriate equipment for each participant’s sex, age, size and skill level
Space, facility, and equipment are available for structured and unstructured play
Facilities are accessible to all participants of the community, including those with disabilities
Staff and leaders are trained in safety protocols
Equipment and facilities are maintained and abide by current safety regulations
Total number of boxes checked: /11
Comment Box (If applicable, please explain why boxes were not checked):
© Child Health & Exercise Medicine Program, McMaster University, 2017

12. Tool Development Phase 1
Step 3: Content Relevance (Validity) 5 Content experts contacted
“You may also suggest we combine items or some other arrangement to result in the most logical checklist.”
© Child Health & Exercise Medicine Program, McMaster University, 2017

13. Phase 1 to Phase 2 Wanted more comments and feedback
Added comment boxes after each question
Relevance of all questions?
Added “how relevant is this indicator to your program?”
© Child Health & Exercise Medicine Program, McMaster University, 2017

14. Need additional validation
Phase 1 to Phase 2
Wanted more responses
Distributed to 100 programs across sport, education, physical activity, recreation and not-for-profit sectors
Need additional validation
Added individual participant assessments and parent perceptions questionnaire
© Child Health & Exercise Medicine Program, McMaster University, 2017

15. Phase 2 PLEA Tool
© Child Health & Exercise Medicine Program, McMaster University, 2017

16. Phase 2 PLEA Tool Four Themes: Environment 3. Leaders and Staff
Programming 4. Values & Goals
Example:
© Child Health & Exercise Medicine Program, McMaster University, 2017

17. Phase 2 PLEA Tool
Goal to recruit 100 Hamilton child and youth active programs from many sectors:
Sport
Physical Education classes
School sports & intramurals
Recreation
Dance
After-school programs
Fitness and yoga
Day camps
© Child Health & Exercise Medicine Program, McMaster University, 2017

18. Phase 2 PLEA Tool 100 Programs _____________ 20% Programs Highest 10%
Lowest 10%
PLEA Tool responses
Parent perceptions of program and child’s physical literacy
Individual physical literacy of participants
Perceived and reality of physical literacy
© Child Health & Exercise Medicine Program, McMaster University, 2017

19. Multi-Sector Representation
74 Programs included
15%
© Child Health & Exercise Medicine Program, McMaster University, 2017

20. 30 Activities Represented
After-school programs
Multi-sport
Active Recess
Physical Education
Adventure Running
Rugby
Baseball
Rhythmic Gymnastics
Basketball
Skating Lessons
Climbing
Soccer
Cross-Country
Softball
Cycling
Sports Camp
Dance
Swimming lessons
Drop-in Gym
Synchronized Swimming
Figure Skating
Tennis
Fitness
Track & Field
Flag Football
Volleyball
Gymnastics
Water polo
Ice Hockey
Yoga

21. Results: Environment & Programming
© Child Health & Exercise Medicine Program, McMaster University, 2017

22. Results: Leaders and Staff & Values and Goals
© Child Health & Exercise Medicine Program, McMaster University, 2017

23. PLEA Tool & PLAYfun & PLAYself
Significant differences in PLAYfun scores between high and low groups
Initial Emerging Competent Proficient
Developing Acquired
* p <0.05; models control for age
© Child Health & Exercise Medicine Program, McMaster University, 2017

24. Key Revisions: Environment Domain
Staff are trained in safety protocols designed to minimize risk of injury to participants
Rephrased unstructured play to free play
7  6 indicators ☑
Access to more than one environment for activity (see options below)
  Variety of environments (check all that apply)
  ☑ Indoor (examples: pools, gymnasiums, fields/turfs, arenas)
☑ Outdoor (examples: fields, outdoor rinks, outdoor pools)
☑ Ice/snow (examples: skating, snowshoeing, tobogganing)
☑ Water (examples: canoeing, swimming, snorkeling)
☑ Air (examples: gymnastics, diving, trampolining)
☑ Land (examples: soccer, basketball, track & field)
© Child Health & Exercise Medicine Program, McMaster University, 2017

25. Key Revisions: Programming Domain
No major changes
10 indicators
Allotted time for structured and free play
Focus on developing skills of each participant individually
Groups and levels are organized by individual skills, not solely by age
Participants have some physical literacy or related assessment to monitor improvements or areas of weakness
© Child Health & Exercise Medicine Program, McMaster University, 2017

26. Key Revisions: Leaders and Staff Domain
6 indicators
Further explained development of movement skills ☑
Program Leaders are trained in supporting the development of general movement skills (e.g. throwing, running, jumping)
Program Leaders are trained in supporting the development of specific movement skills relevant to their activity or sport
Program Leaders are given time to develop and execute plans for effective teaching strategies
© Child Health & Exercise Medicine Program, McMaster University, 2017

27. Key Revisions: Values and Goals Domain
No changes
6 indicators
Physical literacy is part of program objectives
Staff, leaders, participants and parents are educated about physical literacy
Program ensures development of long-lasting good habits
© Child Health & Exercise Medicine Program, McMaster University, 2017

28. Phase 3: National Consultation
Circulate PLEA Tool & survey nation-wide
Participants complete PLEA Tool online
Participants provide feedback on PLEA Tool
Please indicate your level of overall satisfaction with the PLEA Tool as a way helping you understand if and how your program is implementing the principles of physical literacy
Is the PLEA Tool important to program evaluation?
Is the PLEA Tool relevant to your area of practice?
How likely are you to use the PLEA Tool?
© Child Health & Exercise Medicine Program, McMaster University, 2017

29. Acknowledgements
Participating sport, recreation, physical education & physical activity programs!
Child Health & Exercise Medicine Program:
Alexis Bullock
Logan Meyers
Bhanu Sharma
Dr. Brian Timmons
Hamilton Public Health:
Alison Bochsler
© Child Health & Exercise Medicine Program, McMaster University, 2017

30. Hilary Caldwell caldweha@mcmaster.ca @CaldwellHilary
Contact Information
Hilary Caldwell
@CaldwellHilary
© Child Health & Exercise Medicine Program, McMaster University, 2017