Multicriteria Methods for the Shaping of Research Priorities - A Case Study for Finnish Forestry and Forest Industries Ahti Salo and Juuso Liesiö Systems Analysis Laboratory Helsinki University of Technology P.O. Box 1100, FIN-02015 HUT Finland

Context Research and technology programmes Earlier engagements mid-term evaluation of national technology programmes in electronics and telecommunication (Salo and Salmenkaita, 2002) ex post evaluation of WoodWisdom, cluster programme for forestry and forest industries (Salo, Gustafsson and Mild, in press) Case study preparation of a new research programme for wood materials science international through Scandinavian collaborations problem context initially characterised by high uncertainties, vague alternatives, inarticulate criteria for the setting of priorities

Research and technology programmes Characteristics provide additional funding to basic and applied research in areas that are deemed vital for long-term industrial competitiveness are the key strategic instrument of Finnish innovation policy a large number of on-going programmes some 40 programmes funded by the National Technology Agency 15 programmes funded by the Academy of Finland usually include a wide range of supporting co-ordination activities workshops, seminars, communication activitites Issues in preparation what specific research topics should be included in the programmes? what proportion of funding should be allocated to the different topics? what horizontal measures are needed to support the uptake of results? need for extensive consultation and validation

Wood Wisdom Background WoodWisdom the Government launched a programme of increased R&D funding in 1996 a total of 100 MEUR allocated to seven cluster programmes influenced by Porter’s work on industrial clusters promotion of collaboration among ministries, funding agencies and researchers WoodWisdom largest of the cluster programmes with a total funding some EUR 33 million National Technology Agency (44%); participating companies and organisations (33%), Academy of Finland (15%); Ministry of Agriculture and Forestry (7%); Ministry of Trade and Industry (2%) wood as raw material in the pulp and paper as well as wood products industry 4 research areas (raw materials, mechanical forest industry, chemical forest industry, and the operating environment of the forest industry) 21 thematic areas, 34 research consortia, 156 projects 53 companies, 67 research units and 789 researchers

Prospective evaluation Approach define the objects of inquiry e.g., projects, research areas, programme-level activities develop an appropriate methodological evaluation framework e.g., multicriteria decision models appoint the workshop participants interest, competence, balance of stakeholders consult the funding agencies make use of advanced ICT tools solicit viewpoints from all the participants allow for anonymous feedback as well synthesize and discuss results “on the spot” Remarks complements but does not replace other forms of evaluation research may be helpful in deriving recommendations

Examination of future research needs A) What objectives should be stressed in this consortium in the future? Assign 100 points to complementary objectives at each level of the hierarchy

Wood quality variations

Feedback on the self-evaluation

Context of decision support Earlier programmes Wood Wisdom cluster programme concluded in February 2002 strong support expressed for continued research in wood materials science desire to initiate a collaborative Scandinavian research programme joint projects with funding from two or more countries means of securing higher visibility vis-à-vis European funded projects sector faced with increased competition from developing countries Challenges the scope of the new programme was unclear in Spring 2002 ’messy’ research topics, approach and objectives, no formally approved criteria integration of national and international aspirations need for to a structured consultation process consultation - how to obtain structured inputs into the planning process? validation - how to validate inputs suggested by researchers and industrialists? shaping - how to generate ideas for the shape of suggested approaches?

Systematic process structuring Development of domain model a taxonomy of research topics three research areas - one Finnish workshop for each sixteen research themes - structured under the research areas research topics - proposed by the research resaerchjers construction of a multi-criteria model for the analysis of research themes Preparatory consultation circulate a survey to some 60 respondents structured around research areas and themes respondents requested to (1) specify the topic and (2) to justify it in detail Participatory workshops examination of survey results assessment of research themes with the help of the multi-criteria model development of suggestions for funding allocations

Five research themes Wood or fibre based composite structures New materials from wood-based polymers or extractives Biotechnical, chemical or physical modification of wood raw material Innovative applications of traditional wood and fibre products Methods of controlling market-oriented utilization of wood raw material Socio-economic aspects treated as a horizontal theme

Workshop objectives Validation of preparatory work do the results of the preparatory work fully reflect future research needs? what further topics should be pursued in the future? how such this collaboration be managed ? Shaping of research priorities how do the research themes relate to the given criteria? what research themes should the most funding be given? what considerations should be accounted for in the shaping of priorities? e.g., dedicated infrastructures International research collaboration which research themes call for collaboration beyond Scandinavia? with whom should internation collaboration be launched? EU (Framework Programme VI), US, Japan, Far East … what measures should be taken to promote such collaboration?

Distributed decision support system Client-server architecture tailored on the basis of the RICH Decisions-software (http://www.rich.hut.fi/) client-nodes solication of evaluations and written comments from the experts server aggregation of evaluative statements and written comments presentation of results using several modes linked via a wireless local area network Weighting of criteria ’correct’ criteria weights difficult to obtain depends in part on the research theme in consideration Rank Inclusion in Criteria Hierarchies (Salo and Punkka, 2003) ”novelty of topics and research competencies equally important” ”industrial relevance and capabilties for exploitation equally important”

Agenda Introduction 30 min presentation of participants, workshop agenda, objectives and tools results from the preparatory work in Finland and Sweden Analysis of research themes (à 30 min) 2 h 30 min initial comments on presentations (10 min) appraisal of research themes (10 min) discussion (10 min) Identification of focal research topics 30 min results from the appraisal of research themes (10 min) proposals for resources allocation (10 min) International collaboration 30 min

Considerations Time horizons Constructive appraisal of research themes roughly about 5-10 years - subject to discussion what should be achieved through the programme in view of the longer term? Constructive appraisal of research themes what measures should be taken to foster innovative capabilities within each of the research themes? what specific research topics would be particularly promising? International dimensions what priorities should be set for Scandinavian research collaboration what kind of research collaboration is needed beyond Scandinavia?

Observations (1/2) Combinationof formal analysis and informal discussions a joint understanding ’units of analysis’ obtained after discussions only survey and discussions formal analysis helps in the pooling of aggragate results positioning of themes does highlight differing viewpoints still a rather subjective presentation complemented by informal discussions Structure of the multi-criteria model relatively small models may work best more time to explain what the criteria are intended to mean at times rather large models have been suggested not all criteria are applicable to all research themes only a limited amount of time can be devoted to each theme less time to generate qualitative insights

Observations (2/2) Process redundancies Decision support system multiplicity of approaches may contribute to validty direct questions vs indirect analysis if allocation of resources is a key question - pose it directly as such! indirect analysis (multi-criteria) may justify conclusions exploration of differences these may reveal hidden assumptions that are not otherwise accounted for Decision support system allows for systematic elicitation of inputs from all participants level of expertise also accounted for in the Finnish workshops anonymous comments also accommodated a written track-record produced for later dissemination less need to write consultancy reports

Publications (2001) Gustafsson, J., A. Salo, T. Gustafsson (2001). PRIME Decisions: An Interactive Tool for Value Tree Analysis, in: M. Köksalan, S. Zionts (eds.), Multiple Criteria Decision Making in the New Millennium, Lecture Notes in Economics and Mathematical Systems 507, Springer-Verlag, Berlin, 2001. Salmenkaita, J.-P, A. Salo (2002): Rationales for Government Intervention in the Commercialization of New Technologies, Technology Analysis & Strategic Management 14/2, 183-200. Salo, A. (2001): Incentives in Technology Foresight, International Journal of Technology Management 21/7-8, 694- 710. Salo, A., K. Cuhls (2003): Technology Foresight - Past and Future, Journal of Forecasting 22/2-3, 79-82. Salo, A., T. Gustafsson (in press): A Group Support System for Foresight Processes, International Journal of Technology Management. Salo, A., T. Gustafsson, R. Ramanathan (2002): Multicriteria Support for Foresight Processes. Journal of Forecasting 22/2-3, 235-256. Salo, A., R.P. Hämäläinen (2001): Preference Ratios in Multiattribute Evaluation (PRIME) – Elicitation and Decision Procedures under Incomplete Information, IEEE Transactions on Systems, Man, and Cybernetics 31/6, 533- 545. Salo, A., O. Kuusi (2001): Developments in Parliamentary Technology Assessment in Finland, Science and Public Policy 28/6, 453-464. Salo, A., T. Käkölä (in press): Groupware Support for Requirements Management in New Product Development, Journal of Organizational Computing and Electronic Commerce. Salo, A., T. Gustafsson, P. Mild (in press): Prospective Evaluation of a Cluster Program for Finnish Forestry and Forest Industries, International Transactions on Operations Research. Salo, A., Salmenkaita, J.-P. (2002): Embedded Foresight into RTD programs, International Journal of Technology, Policy and Management 2/2, 167-193.