Assessing Research Management at Canada’s National Research Council AEA/CES Conference 2005 Crossing Borders, Crossing Boundaries R&D Topical Interest Group – Presentation Toronto, October 28, 2005

Presentation Outline About the National Research Council of Canada (NRC) Challenges in Research Management What is the Research Management Self- Assessment (RMSA) Tool? Development of the Tool Progress to Date, and Moving Forward

The National Research Council of Canada – Science at Work for Canada Economic value Incubators and start-up support Knowledge and technology transfer/diffusion Commercialization and new company creation SME support Government of Canada’s leading resource for scientific research, development and technology-based innovation Research and Development Basic/fundamental to applied Technology development National R&D infrastructure R&D training International collaborations Innovation Systems and Support Regional innovation systems Information and knowledge networks National codes and standards Resources ( ): Full time staff (FTE): 4,100+ Expenditures: $712.4M Appropriations spent: $653.0M Revenues:$75.2 million Infrastructure: 175 buildings, 517,407 m2 Research areas include: Biotechnology Information and Communications Technologies Aerospace Manufacturing Construction Ocean engineering Genomics Fuel cells Bioinformatics High-performance computing Photonics Nanotechnology Environmental and sustainable development technologies 20 institutes across Canada, 2 technology centres, IRAP, CISTI

Our Research Environment

Some Challenges in Managing Research More generic Uncertainty about where most valuable discoveries lie Risks and costs associated with uncertainty Unpredictable results given the pursuit of the unknown Difficulty in assessing contribution/impact of research results Often long timeframes before outcomes or impacts of research become evident Others facing NRC Balancing creativity and accountability (value for Canada) Managing High Quality Personnel within federal framework Planning in a dynamic environment Prioritizing with multiple stakeholder interests Managing multiple collaborations Establishing attribution of results in collaborative efforts Balancing cluster activities, horizontal program requirements and institute research activities

What is RMSA? The Research Management Self-Assessment (RMSA) is a practical, diagnostic tool developed by NRC for use by its institutes in support of continuous improvement and effective management of research and research-related activities. The RMSA: A framework and codified research management practices against which institutes can assess themselves Complementary to NRC’s annual performance reporting, and program evaluation Helps to identify links to other corporate resources and tools Furthermore, RMSA allows: A common language across NRC Knowledge sharing and exchange A non-threatening, honest, open self-examination for institutes Engagement of research and related staff Opportunity for management development

RMSA Framework (August 2005)

RMSA Development

How Does RMSA Work? 1. Assessment Planning 2. Pre-Assessment Session3. Assessment Workshop 4. Follow-up Action Establishing the institute- specific context – anticipated issues / challenges; participants; timing; communications… Awareness building and prioritization of topics for assessment – includes presentation, discussion, and completion of questionnaire Facilitated session to establish current and desired future state – focus on priority areas established in step 2. Diagnostic results are qualitative and quantitative (based on voting) Planning and implementation of follow-up action – follow-up actions to be determined based on prioritized opportunities from assessment results Monitor progress against action plan, and re-assess as needed for other categories over time Prioritized categories for assessment Positioning of RMSA within Institute Key areas identified for improvement

Sample Definition Governance: Within our Institute, roles and responsibilities of management and professional staff have been clearly defined, communicated and are understood by those involved. Managers have authorities commensurate with their responsibilities, and are accountable for their actions. A comprehensive approach to governance is in place, including: use of a defined and understood accountability framework; use of an independent, expert advisory body; documenting and communicating decisions in a transparent manner; and having a fair and independent process to manage conflicts of interest.

The Assessment Scale LevelDescription 1The practices described do not exist within our Institute, and appear to describe an ideal state that is beyond our reach at the present time. 2A few of the defined practices have been adopted at the Institute where I work, however, these are not applied in a uniform fashion. 3Several of the defined practices are established within the Institute where I work, but gaps still exist. 4The defined practices are the norm at the Institute where I work, and part of our culture. 5Not only are the defined practices the norm at the Institute where I work, but we were early-adopters and continue to seek ways to improve and share our practices with others in this area. 9Do not have sufficient knowledge – cannot comment. Discussion to focus on: 1) strengths (what is done well), 2) what can be done better (opportunities); and 3) some challenges/barriers to keep in mind; The qualitative information is key – the numbers provide some scale of the gap that exists (“as is” and “to be”) to help in prioritizing issues.

Progress to Date and Moving Forward Pilot testing of content and process Results to date have been positive –Process has been well-received –Adjustments are expected to refine the content, and framework Pilots to be completed by late Fall 2005 Recommendations to be made to NRC senior executives regarding: –broader rollout of the tool based on pilot experiences –opportunities for corporate follow-up to address cross-cutting issues/gaps (e.g., training, corporate resources and support)

Some Messages to Date Keep tool flexible rather than policy – cannot have a cookie cutter approach Use of tool by institutes in two ways: –As quick checklist (Are we doing this or not?) –As a full assessment (How well are we doing in these various areas?) RMSA should be positioned for use as an internal management tool rather than as a means of measuring institute performance and resource allocation Accountability needs to be built in for follow-through on improvement More support could be provided at the corporate level to institutes for research management – specifics will emerge with assessment activities.

Questions? For further information: Flavia Leung National Research Council of Canada Website: