1. Hilary Edelstein & Ben Levin
Issues in conducting new research: Challenges and rewards Use of Online Research (UOR)
CSSE, May 31, 2011
By:
Hilary Edelstein & Ben Levin
This presentation is a continuation of the previous presentation on the data from the UOR project. In this presentation, I will go over why we focus on products, where we are at in our project, the product typology and framework, why we use Google Analytics and the issues around working in a collaborative research partnership with many organizations.

2. Why focus on online products?
We became interested in the research products that educational organizations disseminate as part of our work on another project which looked at the overall KM efforts of organizations websites. We found from our literature review that format and content of what is posted for dissemination matters for the uptake of research and that organizations were more likely to post products to disseminate, rather than engaging in events or networks to link research to practice.
We know that people look for research and educational resources online.
In many cases these are products, such as research reports or summaries, literature reviews and sometimes blogs or videos.

3. What we have done so far
Began in 2009 with a literature review on KM and Internet technology, this literature we have collected has now expanded to literature on collaborative partnerships.
We began to pilot the project in October 2009 when the first partner joined and since then have steadily been adding partners, gathering data and more recently, analyzing that data.
On the timeline October to December 2010 stands out. During this time we conducted our first website usage analysis, presented it to the partners and gathered their feedback in helping us understand the impact of the use of online research. The partners feedback has helped us further our analysis tools by pointing out information that they felt was missing or that we should delve deeper. To do that, we have enlisted the help of a Google Analytics certified consulting company whom we work with to help us understand what metrics we should be looking at and how we can best analyze the data coming out of the Use of Online Research project. This consulting company created a new tracking code that specifically tracks online products through Google Analytics. This code is helping us gather and analyze more data.

4. Product analysis typology and framework
Analysis framework:
Typology:
Product typology
This framework includes content, or the type of product posted, the target audience and the form of the product. We found that many products posted fall under the category of a single study, a literature review, a research synthesis, systematic review and sometimes a product does not fit into those content categories so we created a type called other. We found that those content products are often oriented to either an unspecified audience, researchers, teachers or what we call practitioner or school persons; parents; and policy makers.
The product analysis framework is drawn from the product typology although is a bit more specific when it comes to naming the content found. Here we go beyond single study, literature, research synthesis and systematic review and name the type of product we are tracking. These products includes: long and/or short versions of the same product, often of a single study; literature reviews, research summaries or syntheses; systematic reviews and practitioner/school guidance resources; parent resources, teacher specific resources, researcher specific resources and videos.

5. Why the fuss over…
Well,
Most of our partners already had it and those that did not were eager to join.
There is a lot of support for Google Analytics online and with consulting companies.
It is easy to export the data into Excel
Used to analysis data based on specific metrics
Google Analytics is an easy to use web analytics software tool. Most of our partners either already had Google Analytics when entering the project or were looking for a way to analyze their data and found our project as a useful way into the analytic world with their IT team liaising with us.
Although Google Analytics is easy to use at the beginning of the project we were not sure of the best way to export the data. Working with a consulting company, we have found that there are several tools to use that export data from Google Analytics into Excel. We use a tool called Next Analytics and the consulting team helped us create several dashboards and an analysis template that we can use to continually update and analyze our data. We have also worked with the consulting company to create an html code that will track online products and to understand how to customize reports so that we can easily sort and find the information we need.
Google Analtyics lets us look at specific metrics such as page views, visitors, repeat visitors and in Excel we can use the data from Google Analytics to make calculations to find the ratios and percents between metrics to understand website use. This includes for example, repeat visitors and the percent of people who visit the homepage versus the percent of people who visit the research pages.
I am now going to turn to discussing how we collaborate with our partners in the Use of Online Research project

6. What are collaborative partnerships?
Collaborative research, or partnership research is loosely defined as the process of forming a cooperative relationship with the objective that working together will accomplish more than by working alone.

7. Our partners
6 Canadian organizations, 5 organizations from the UK and 1 organization based in New Zealand.
Partner organizations include an: education union, a university research group, not for profit education advocacy group, ministry of education, government funded research organization
We have 12 partners – 6 in canada, 5 in the UK and 1 in New Zealand. We did have more but as I will talk about in the next slide, there are some challenges when working with partners including funding issues which has meant that some partners have had to discontinue their involvement in the project.
Partner orgs include: education union, research group, non profit adocacy group, ministry of education, government funded groups.

8. What does partnering add to the UOR project?
Input and decision-making
Networking between partner organizations to do KM
The challenge of partnering
What does partnering add?
In the UOR project, this collaboration works in two ways: The research team gains access to a data sample, while the partner gains the people who will do the research and analysis for them. Partners have helped make decisions relating to how to raise the response rate of our survey; providing feedback on the product typology; helping us organize the data analysis framework by providing input into the metrics we might examine; and providing feedback on the data.  Partners have also provided input to which products on their website they want us to track.
A positive outcome of collaboration has been networking the different partners with each other. Although there was and is some connections between the individual Canadian and UK partners, they did not know about each other. Because they are similar and working on improving education as well as providing stakeholders with research information their work with our team has included the partners being able to connect and share their work amongst each other.
Challenge to partnering
Although this is a collaborative project that seeks to strengthen the links between researchers and the research/policy/practice divide, there are many barriers to partnering with government, university, community, and independent partners. These include issues around communication, such as how to communicate and how often. How to get work done when the work for the project is what I do, but is a small part of the day or week that the partners do and therefore trusting that the work will get done and being flexible about how or when it will get done is a variable that we have to understand. An issue relating to communication, is that this project is an inductive study. This means that things change when one aspect does not work. Our partners have had to be flexible to adapt to these changes. Other barriers include issues around the technology of Google Analytics. Some of our partners have a dedicated Information Technology unit that addresses technology issues, some do not as a result we have had to struggle with the organizations to add the new code to their analytic accounts and to have some of them provide us with access to their accounts for the analysis. A final barrier is that most of our partners are publically funded organizations. With the recession, many had their funding cut and as a result we have had a partner drop out of our study.
One unanticipated challenge that we encountered with partnering with some organizations was their funding. Many of the organizations we work with are funded, at least in part, by their government. With the worldwide recession, and accompanied budget cuts by governments, several of our partner organizations, especially those in the UK, had their funding cut. As a result, we have had to work with organizations to support their IT use, be flexible and patient with on-going communication and gathering data (Maynard, 2007; Mitton, 2007; Israel et al., 1998). In part due to the financial challenge, one UK partner had to drop out of the project due to restructuring in their organization. Other challenges have related to communication and Internet technology. For example, the Chilean partner dropped out of our study due to incompatibility with the goals of the project; a UK organization is unable to participate in the website tracking aspect of the project due to their current web tracking software but is participating in the survey aspect, and we have intermittent contact with one Canadian partner who although they did participate in an analysis in the fall of 2010, is not participating in our spring 2011 analysis.     
Communication & flexibility
IT departments
When funding is cut

9. Contributions of the UOR project and partnering among educational organizations
Partners network with each other to share knowledge
Expanding the understanding of knowledge mobilization relating to education
Providing a methodological map so that anyone can do this type of work.

10. If you want more information, please contact:
Thank you!
Download our paper at:
If you want more information, please contact:
Hilary Edelstein