1. How Helpful Is Your Library?
Peter G. Mohn, LMS
Snohomish Freshman Campus
2006 WLMA Conference
October 13, 2006
Welcome. Today’s presentation will look at the preliminary research I conducted at my campus using the Likert “help” survey developed by Drs. Todd and Kuhlthau for their 2004 Ohio study.

2. State-wide Library Studies
New Mexico, 2002
Michigan, 2003
Minnesota, 2003
Missouri, 2003
North Carolina, 2003
Indiana, 2004
Ohio, 2004
Illinois, 2005
Alaska, 1999
Colorado, 1993; 2000
Pennsylvania, 2000
Texas, 2001
Oregon, 2001
Florida, 2002
Iowa, 2002
Massachusetts, 2002
Since 1999 there have been 16 state-wide library studies conducted to measure the effect school library media programs have on student academic achievement. In 2006, Wisconsin has published its own study, but I haven’t had an opportunity to read as of today.

3. Current Studies in Progress
New Jersey
Delaware
In addition, two more state-wide studies are in the works.

4. Numbers Well over 3,000 schools Well over 3,000,000 students
As far as I know no other education group has conducted such a wide range of large scale studies in such a short period of time. These studies represent a clear measure of the impact libraries have on the academic achievement. The No Child Left Behind Act demands that schools use research based studies to use “best” practices in the classroom. These studies demonstrate what are the “best” practices for libraries in helping students with their academic achievement.

5. Variables Controlled Ethnic background Social Economic Status
Rural, suburban, and urban areas
Technology
Parents education
These studies controlled various variables that are known to impact academic achievement, and with these variables controlled they demonstrate how effective library programs impact the learning of students.

6. Collective Results
% percentage point rise in reading, and literacy scores on state-wide tests
Positive rises in student academic achievement over-all
Results tied to collection size, budget, staffing, collaborative planning, technology, and student use of libraries
Let’s be clear these studies aren’t saying test scores rise 10-15% above other schools without effective library programs. THEY ARE SHOWING THAT EFFECTIVE LIBRARY PROGRAMS HELP IMPROVE TEST SCORES 10-15% HIGHER. So if students of school A score 60% on state literacy exams then students at school B with the same demographics, but with an effective library program will score between 70-75% on the same state-wide exam.

7. How Do Libraries Help?
We have the data of libraries impacting student academic achievement on state-wide tests, and student learning, but what “helps” do libraries and librarians provide to significantly improve student learning?
All but one of these 16 studies look at the relationship between effective library programs and test scores. In addition, the data collected is from only the District staff. However, the Student Learning Through Ohio School Libraries was the first large scale study to not only ask the District staff about their library programs, but to also ask students how school libraries help them.

8. Ohio Study, 2004
Student Learning Through Ohio School Libraries by Todd, Kuhlthau, and OELMA
13,123 students and 879 faculty at 39 schools
Surveyed 48 “helps”
Libraries helped 99.44% of students
Libraries helped 25.54% of students on all 48 statements
Libraries helped 60.3% of students on 43 of the statements
What is amazing about this study is to what degree students perceive how their school libraries help them. For decades we’ve known that school libraries impact student learning, but we really haven’t had a comprehensive study that looks at the dynamics of the “helps” students receive in the educational lives.

9. What is an Effective Library Program?
Adequate & appropriate credentialed staff and support staff
Librarians involvement in collaborative planning
Developing print & digital collections
Motivating students to read
Administering a curriculum-centered library program
Facilitating access to the library & its technology
The Ohio study selected 39 schools to participate in this study based on the criteria of what an effective school library program is. So the results of this study can be used as a benchmark for further study of effective library programs (e.g., the Delaware study), and for schools that don’t meet the effective level for school libraries.

10. Non-Effective School Libraries
What happens to the levels of help in a non-effective school library, and does this help explain why students with non-effective library programs score lower on standardized tests?
Our library program doesn’t qualify as an effective library program. Our program has mix of positive and negative components. We are a credentialed staff, but we are understaffed. Our budget is below average, but not poor. The staff is cooperative, but not collaborative. The print collection is outdated, but we have some digital databases. We have some reading programs, but they could be expanded. The library is curriculum-centered, but lacks adequate resources. Technology is improving, but we’re the only district in the surrounding area without a technology bond.
How do we use the Ohio study surveys to find out what helps need to be improved? Or, is our library program, despite its shortcomings, providing quality service to our staff and students that equals, or exceeds the Ohio study?

11. SFC Study, (in progress)
Doesn’t meet effective library levels
Two phase study
Were there any significant differences
What recommendations should be given to improve the “helps” in the SFC LMC
The results of out two year study are still being analyzed, and a rough draft of the study will be available by December 2006.

12. 2004 SFC Study, part 1
Student and faculty survey based on Ohio faculty survey
Compare SFC students & faculty responses
Compare SFC responses to the Ohio study
Recommend changes to the LMC program
Originally, I conducted a single year study using the Ohio faculty survey as the basis of both the student and faculty surveys. One of the issues I had with the Ohio study was that the faculty survey asked staff what they thought were the helps libraries gave to students, while the student survey asked students to personally talk about how the library helped them. I felt it would be interesting to ask students how they perceived the LMC helped all students in general. Thereby, I could compare faculty and students responses to virtually the same questions. Though by doing this I lost a direct one to one comparison between the Ohio and SFC surveys.

13. 2004 SFC Study Findings
99.1% of the students indicated the LMC and its services help students
50% of the students say LMC helps students on all 48 help statements
63.4% of the students say LMC helps students on 43 of the 48 statements
Faculty & students agreed on rank priority
Faculty means significantly lower than the Ohio faculty
The student survey results were very similar to the Ohio student survey results. There were some differences both positive and negative, which helped in determining the recommendations for part 1. What really helped were the three open-ended questions I asked at the end of the survey for both surveys. They gave me insight to what was working well in the LMC and what needed improvement.
It was curious that the students and faculty rank order of means for the 7 blocks of questions was exactly the same order. The Ohio study’s rank order did not have a one to one parallel. This suggests there may be a connection between staff and student views, but I can’t say which influenced whom. It will take another study or two with the same surveys to see if these results are repeated.
The SFC faculty means were way lower than the Ohio study. The short comings in collaborative planning, etc. demonstrate that many faculty members don’t realize how students use the LMC, and how the LMC helps students.

14. 2004 SFC Study Conclusions Same conclusions as Ohio Study
LMC is a dynamic agent of active learning
Understanding how to research effectively
Students understand how to identify key ideas
Students analyze, synthesize, evaluate, & apply information
Students develop personal conclusions

15. 2004 SFC Study Recommendations
Formalize collaborative process with classroom teachers
Improve computer technology
Create LMC web page to enhance student access to information outside of school
Promote more reading opportunities
Survey the Class of 2008
Despite positive comparisons between the two studies, there were four recommendations for making improvements to the LMC programs, plus a recommendation to survey the next class. Placing computer technology on a five year maintenance replacement cycle. Not only has an LMC web page been developed over the last two years, but I’m helping staff members to build their classroom pages, so students and parents have access to the LMC away from school, and parents can be proactive in helping their children with their assignments. I’ve added a couple more reading programs to the LMC program, and working with the reading teacher to have his kids check out books on a monthly basis. We were hoping this year that the District would implement a shared planning time for all teachers in the District, but parents were against it, so we’re revamping this option for next year.

16. 2005 SFC Study (in progress)
Compare SFC & Ohio student percentages
Compare SFC & Ohio student means
Compare SFC & Ohio student Most Helpful percentages
Compare 04 & 05 SFC student block means
Compare SFC & Ohio student block means
Compare SFC & Ohio faculty block means
The 2005 student survey is the same one used by the Ohio study along with the additional 9 statements recommended by Todd, Kuhlthau, & OELMA. With survey I used six null hypotheses to compare the Ohio, 2004, and 2005 surveys.

17. 2005 SFC Study Findings SFC student percentages lower
SFC student means lower
SFC student Most Helpful percentages are lower
05 SFC student block means significantly lower
SFC & Ohio block means mixed
SFC & Ohio faculty block means significantly lower in every block
The 2005 survey percentage and mean results were lower as compared to the Ohio study and when compared to the 2004 student survey. Despite being lower to the Ohio study, the 2005 student survey was only significantly lower in Blocks 4 and 5. However, compared to the 2004 student survey they were significantly lower in five of the seven blocks.

18. 2005 SFC Study Conclusions
Fewer students perceive the LMC as helping them in their academic & personal pursuits
SFC students perceive the LMC as helpful as Ohio students do, except in reading
A large minority of SFC students do not perceive the LMC as helping them with their reading
A larger percentage of SFC students don’t perceive the LMC as helping them
SFC faculty has a lower perception of how the LMC help their students in all areas
Personally, students in the 2005 survey marked “Most Helpful” as a smaller rate, and there were a greater percentage of students marking “Does Not Apply.” Block 5 which is about reading was about 15% greater in the “Does Not Apply” category, which led to a significance level.

19. 2005 SFC Study Recommendations
Integrate information literacy K-12
Develop K-12 curriculum
Work closely with principals
Shared collaborative planning times
Improve reading opportunities
Bring budget inline with standards
Collaborate with reading teacher
Improve library collection
Provide more reading activities
Continue technology improvements
Pass a technology bond
These are preliminary recommendations, and may change over the few months.

20. Recommendations cont. Conduct state-wide library study
Effective & non-effective schools
Include schools with certified & non-certified librarians
Conduct district elementary, middle school, & upper campus study
Change scale from 0-4 to 1-5
Use 57 help survey
Change N/A to No Help
Compare results to Delaware Study
Take results to legislature for support
I would urge WLMA to petition the state library to fund a state-wide school library study of its own using Todd’s work in Ohio and Delaware.
If you are conducting a study of your own then I suggest making the following changes to your survey. Dr. Todd is using these changes in the Delaware study.

21. Conducting Your Own Study
My rough draft of my study will be online in December. However, you can get a copy of this PowerPoint presentation, and Word copies of the three surveys I used in my study.

22. Ohio Study http://www.oelma.org/StudentLearning/default.asp
The first place I would recommend visiting is OELMA, and look what they have online for their study. You may also want to purchase the CD of this study. It has some very valuable information for setting up your own study.

23. Pre-Survey Planning Who can help you with your study?
Find a good statistic book you can use
Use the KISS principal
Decide online, Excel or paper?
Random sampling
At least 30 students per grade
Time of the year to conduct survey
Use summer to write study
People to help with setting up the study and collating results.
A stat book for setting up your equations
Don’t go over board like I did - I’m using my study for grad work
Try surveymonkey.com, set up your own survey, use Excel to tally resuls, or paper and pencil with a calculator.
Even if you can’t get a true random sample, then go for a stratified random sample
Less than 30 samples means you need to use conservative measurements (N-1).
What is the best time for you? Since I only have my students for one year I needed to conduct my survey in June after all research projects were completed for the year.
It took me about 80 hours to evaluate and write the 2004 year study.

24. Likert Survey Percentages & Means Two-Way t Test for Two Means Total
Grade
Gender
Most Helpful
Standard Deviation
Two-Way t Test for Two Means
Only if comparing Ohio or Delaware studies
You don’t have to do a lot of statisitics for your study. You can calculate just the percentages and means for your study, and you will get good results that will allow you to accurately measure the levels of help your library gives students. Plus, you’ll still be able to compare your study with the Ohio and Delaware studies. You may want to drop the grade and gender portion of the study to keep your study down to a manageable amount of work. If your percentages are greater or lower than 5% from other studies then you can make statements that demonstrate a difference between between studies.
Conducting a Two-Way t Test demands you know the means, number of participants, and standard deviation from the two studies. The t Test allows you to say with certainty that there is significance between the two studies. But it’s not required for a quasi-study.

25. Giving Survey - Paper Get teacher buy-in
Randomly select students (blind)
Either have teacher give survey, or follow a script when you do (neutral voice)
Give faculty the survey yourself
1) I used my history and math departments to help with the survey
2) Or stratified random sample
3) I felt better when I gave the introduction to the survey, then I knew all students heard the same directions

26. Organizing Results Sort survey by grade and gender
Mark one page at a time, less chance
Have people help with scoring
Throw out bad surveys
Create tables before writing
Type up written appendices
Use Todd’s survey as guideline
Don’t be afraid of your own conclusions
State statistical facts
Sorting by grade and gender helps with not repeating work
Less chance of missing a mark while scoring
Look for flippant remarks, picket fences, rows (though look closely at No Helps). Don’t be afraid to throw out surveys, most students will remain honest
Keep tables on separate documents, then import when ready
Having appendices typed up allows you to easily evaluate statistics with personal comments from students and staff
Don’t over state your results or jump to conclusions

27. Writing Paper Parts of your study Title Table of Contents
2 - Executive Summaries
Introduction
Literature Review
Methodology
Results
Conclusions
Recommendations
References
Appendices
The executive summary is attached to the top of the paper, and a second copy is placed after the table of contents. This is the most important part of your study. Many times superintendents don’t have time to read an entire paper, but they’ll trust you did your work, and they’ll accept the conclusions and recommendations you made in your executive summary.

28. Writing Hints You don’t need a large literature section
Concentrate on major themes
Keep paper to around 30 pages
Not counting references & appendices
Combine results & conclusions
You don’t need a large reference section
Don’t write more than 2-3 hours a day
Try to complete a section each day
Your study needs to be between pages not counting the references or the appendices. If it’s under twenty pages then administrators think it’s not important. If it’s over 30 pages then they don’t have time to read it.
The literature review doesn’t have to be long. Just enough to cover major studies on the topic
Sometimes the results and conclusions are merged to keep the paper under 30 pages.
I found writing a little bit a day allowed me to stay fresh, and to continue to refine my ideas before I placed them on paper. I took seven days to complete the writing on the seven blocks for the 2004 study.

29. Questions? Peter.mohn@sno.wednet.edu
Please feel free to contact me if you have any questions about setting up your own study.
Thank you for your time this morning, and a wonderful convention.