1 Helsinki University of Technology Systems Analysis Laboratory Multicriteria Methods for the Shaping of Research Priorities - A Case Study for Finnish Forestry and Forest Industries Ahti Salo and Juuso Liesiö Systems Analysis Laboratory Helsinki University of Technology P.O. Box 1100, FIN HUT Finland

Helsinki University of Technology Systems Analysis Laboratory Context n Research and technology programmes n Earlier engagements –mid-term evaluation of national technology programmes in electronics and telecommunication (Salo and Salmenkaita, 2002) –ex post evaluation of WoodWisdom, cluster programme for forestry and forest industries (Salo, Gustafsson and Mild, in press) n Case study –preparation of a new research programme for wood materials science »international through Scandinavian collaborations –problem context initially characterised by high uncertainties, vague alternatives, inarticulate criteria for the setting of priorities

Helsinki University of Technology Systems Analysis Laboratory Research and technology programmes n Characteristics –provide additional funding to basic and applied research in areas that are deemed vital for long-term industrial competitiveness –are the key strategic instrument of Finnish innovation policy –a large number of on-going programmes »some 40 programmes funded by the National Technology Agency »15 programmes funded by the Academy of Finland –usually include a wide range of supporting co-ordination activities »workshops, seminars, communication activitites n Issues in preparation –what specific research topics should be included in the programmes? –what proportion of funding should be allocated to the different topics? –what horizontal measures are needed to support the uptake of results?  need for extensive consultation and validation

Helsinki University of Technology Systems Analysis Laboratory Wood Wisdom n Background –the Government launched a programme of increased R&D funding in 1996 –a total of 100 MEUR allocated to seven cluster programmes »influenced by Porter’s work on industrial clusters »promotion of collaboration among ministries, funding agencies and researchers n WoodWisdom –largest of the cluster programmes with a total funding some EUR 33 million »National Technology Agency (44%); participating companies and organisations (33%), Academy of Finland (15%); Ministry of Agriculture and Forestry (7%); Ministry of Trade and Industry (2%) –wood as raw material in the pulp and paper as well as wood products industry »4 research areas (raw materials, mechanical forest industry, chemical forest industry, and the operating environment of the forest industry) –21 thematic areas, 34 research consortia, 156 projects –53 companies, 67 research units and 789 researchers

Helsinki University of Technology Systems Analysis Laboratory Prospective evaluation n Approach –define the objects of inquiry »e.g., projects, research areas, programme-level activities –develop an appropriate methodological evaluation framework »e.g., multicriteria decision models –appoint the workshop participants »interest, competence, balance of stakeholders »consult the funding agencies –make use of advanced ICT tools »solicit viewpoints from all the participants »allow for anonymous feedback as well »synthesize and discuss results “on the spot” n Remarks –complements but does not replace other forms of evaluation research –may be helpful in deriving recommendations

Helsinki University of Technology Systems Analysis Laboratory Examination of future research needs A) What objectives should be stressed in this consortium in the future? Assign 100 points to complementary objectives at each level of the hierarchy

Helsinki University of Technology Systems Analysis Laboratory Wood quality variations

Helsinki University of Technology Systems Analysis Laboratory

Helsinki University of Technology Systems Analysis Laboratory Feedback on the self-evaluation

Helsinki University of Technology Systems Analysis Laboratory Context of decision support n Earlier programmes –Wood Wisdom cluster programme concluded in February 2002 »strong support expressed for continued research in wood materials science –desire to initiate a collaborative Scandinavian research programme »joint projects with funding from two or more countries »means of securing higher visibility vis-à-vis European funded projects »sector faced with increased competition from developing countries n Challenges –the scope of the new programme was unclear in Spring 2002 »’messy’ research topics, approach and objectives, no formally approved criteria »integration of national and international aspirations –need for to a structured consultation process Êconsultation - how to obtain structured inputs into the planning process? Ëvalidation - how to validate inputs suggested by researchers and industrialists? Ìshaping - how to generate ideas for the shape of suggested approaches?

Helsinki University of Technology Systems Analysis Laboratory Systematic process structuring n Development of domain model –a taxonomy of research topics »three research areas - one Finnish workshop for each »sixteen research themes - structured under the research areas »research topics - proposed by the research resaerchjers –construction of a multi-criteria model for the analysis of research themes n Preparatory consultation –circulate a survey to some 60 respondents –structured around research areas and themes »respondents requested to (1) specify the topic and (2) to justify it in detail n Participatory workshops –examination of survey results –assessment of research themes with the help of the multi-criteria model –development of suggestions for funding allocations

Helsinki University of Technology Systems Analysis Laboratory ÊWood or fibre based composite structures ËNew materials from wood-based polymers or extractives ÌBiotechnical, chemical or physical modification of wood raw material ÍInnovative applications of traditional wood and fibre products ÎMethods of controlling market-oriented utilization of wood raw material n Socio-economic aspects treated as a horizontal theme Five research themes

Helsinki University of Technology Systems Analysis Laboratory

Helsinki University of Technology Systems Analysis Laboratory Workshop objectives n Validation of preparatory work –do the results of the preparatory work fully reflect future research needs? –what further topics should be pursued in the future? –how such this collaboration be managed ? n Shaping of research priorities –how do the research themes relate to the given criteria? –what research themes should the most funding be given? –what considerations should be accounted for in the shaping of priorities? »e.g., dedicated infrastructures n International research collaboration –which research themes call for collaboration beyond Scandinavia? –with whom should internation collaboration be launched? »EU (Framework Programme VI), US, Japan, Far East … –what measures should be taken to promote such collaboration?

Helsinki University of Technology Systems Analysis Laboratory Distributed decision support system n Client-server architecture –tailored on the basis of the RICH Decisions-software ( –client-nodes »solication of evaluations and written comments from the experts –server »aggregation of evaluative statements and written comments »presentation of results using several modes –linked via a wireless local area network n Weighting of criteria –’correct’ criteria weights difficult to obtain »depends in part on the research theme in consideration »Rank Inclusion in Criteria Hierarchies (Salo and Punkka, 2003) –”novelty of topics and research competencies equally important” –”industrial relevance and capabilties for exploitation equally important”

Helsinki University of Technology Systems Analysis Laboratory Agenda n Introduction 30 min –presentation of participants, workshop agenda, objectives and tools –results from the preparatory work in Finland and Sweden n Analysis of research themes (à 30 min) 2 h 30 min –initial comments on presentations (10 min) –appraisal of research themes (10 min) –discussion (10 min) n Identification of focal research topics 30 min –results from the appraisal of research themes (10 min) –proposals for resources allocation (10 min) –discussion (10 min) n International collaboration 30 min

Helsinki University of Technology Systems Analysis Laboratory Considerations n Time horizons –roughly about 5-10 years - subject to discussion –what should be achieved through the programme in view of the longer term? n Constructive appraisal of research themes –what measures should be taken to foster innovative capabilities within each of the research themes? –what specific research topics would be particularly promising? n International dimensions –what priorities should be set for Scandinavian research collaboration –what kind of research collaboration is needed beyond Scandinavia?

Helsinki University of Technology Systems Analysis Laboratory

Helsinki University of Technology Systems Analysis Laboratory

Helsinki University of Technology Systems Analysis Laboratory

Helsinki University of Technology Systems Analysis Laboratory

Helsinki University of Technology Systems Analysis Laboratory

Helsinki University of Technology Systems Analysis Laboratory

Helsinki University of Technology Systems Analysis Laboratory

Helsinki University of Technology Systems Analysis Laboratory

Helsinki University of Technology Systems Analysis Laboratory

Helsinki University of Technology Systems Analysis Laboratory Observations (1/2) n Combinationof formal analysis and informal discussions –a joint understanding ’units of analysis’ obtained after discussions only »survey and discussions –formal analysis helps in the pooling of aggragate results –positioning of themes does highlight differing viewpoints »still a rather subjective presentation »complemented by informal discussions n Structure of the multi-criteria model –relatively small models may work best »more time to explain what the criteria are intended to mean –at times rather large models have been suggested »not all criteria are applicable to all research themes »only a limited amount of time can be devoted to each theme »less time to generate qualitative insights

Helsinki University of Technology Systems Analysis Laboratory Observations (2/2) n Process redundancies –multiplicity of approaches may contribute to validty –direct questions vs indirect analysis »if allocation of resources is a key question - pose it directly as such! »indirect analysis (multi-criteria) may justify conclusions –exploration of differences »these may reveal hidden assumptions that are not otherwise accounted for n Decision support system –allows for systematic elicitation of inputs from all participants »level of expertise also accounted for in the Finnish workshops »anonymous comments also accommodated –a written track-record produced for later dissemination »less need to write consultancy reports

Helsinki University of Technology Systems Analysis Laboratory Publications (2001  ) Gustafsson, J., A. Salo, T. Gustafsson (2001). PRIME Decisions: An Interactive Tool for Value Tree Analysis, in: M. Köksalan, S. Zionts (eds.), Multiple Criteria Decision Making in the New Millennium, Lecture Notes in Economics and Mathematical Systems 507, Springer-Verlag, Berlin, Salmenkaita, J.-P, A. Salo (2002): Rationales for Government Intervention in the Commercialization of New Technologies, Technology Analysis & Strategic Management 14/2, Salo, A. (2001): Incentives in Technology Foresight, International Journal of Technology Management 21/7-8, Salo, A., K. Cuhls (2003): Technology Foresight - Past and Future, Journal of Forecasting 22/2-3, Salo, A., T. Gustafsson (in press): A Group Support System for Foresight Processes, International Journal of Technology Management. Salo, A., T. Gustafsson, R. Ramanathan (2002): Multicriteria Support for Foresight Processes. Journal of Forecasting 22/2-3, Salo, A., R.P. Hämäläinen (2001): Preference Ratios in Multiattribute Evaluation (PRIME) – Elicitation and Decision Procedures under Incomplete Information, IEEE Transactions on Systems, Man, and Cybernetics 31/6, Salo, A., O. Kuusi (2001): Developments in Parliamentary Technology Assessment in Finland, Science and Public Policy 28/6, Salo, A., T. Käkölä (in press): Groupware Support for Requirements Management in New Product Development, Journal of Organizational Computing and Electronic Commerce. Salo, A., T. Gustafsson, P. Mild (in press): Prospective Evaluation of a Cluster Program for Finnish Forestry and Forest Industries, International Transactions on Operations Research. Salo, A., Salmenkaita, J.-P. (2002): Embedded Foresight into RTD programs, International Journal of Technology, Policy and Management 2/2,