Division Of Early Warning And Assessment MODULE 5: PEER REVIEW

Division Of Early Warning And Assessment Overview Overview This module highlights the importance of peer review in IEA In particular, it emphasizes the need to ensure scientific credibility, policy relevance and legitimacy

Division Of Early Warning And Assessment The module also outlines:  what constitutes peer review in the context of the AEO process  the objectives of peer review  why peer review is critical to IEA  how to select peer reviewers Overview (cont’d) Overview (cont’d)

Division Of Early Warning And Assessment Peer review definition in AEO Peer review definition in AEO  Peer review is a documented critical look at the AEO report, conducted by qualified individuals and organizations.  experts are considered equivalent in technical capacity (thus peers) to the experts who draft the contents of the report.  The peer review is important for ensuring that research and conclusions of the report are technically adequate, competently performed, properly documented, and satisfy established quality requirements.

Division Of Early Warning And Assessment  The AEO peer review is an in-depth assessment of the assumptions, calculations, extrapolations, alternate interpretations, methodology, acceptance criteria, and conclusions of AEO reports and documentation supporting them.  The process is characterized by a limited number of interactions by peer reviewers.  It is undertaken during the whole AEO process, including selection of methods, research and drafting. Peer review… (cont’d) Peer review… (cont’d)

Division Of Early Warning And Assessment Extent of peer review for the AEO  Scientific, technical and policy review has always been a key component of the AEO and other environmental assessment reporting processes  It involves hundreds of stakeholders both inside and outside government structures  The peer review process takes different forms – ranging from informal to formal  activities include sub-regional and regional consultations, collaborating centre network meetings, and targeted expert input.  AEO draft sections are sent to scientists and government experts for review and comprehensive comment

Division Of Early Warning And Assessment Objectives of peer review The objectives of the peer review are to check, among others, for:  adherence by the authors to the TORs provided by the coordinating agency at the start of the assignment  reliability and appropriateness of the scientific basis of the analyses, as well as the data and information used

Division Of Early Warning And Assessment  reliability of the sources of information and citations, and if these are up-to-date  regional and sub-regional coverage of the issues  appropriateness of the conclusions/findings of the analyses  relevance and soundness of the indicators used in the analyses Objectives of peer review (cont’d)

Division Of Early Warning And Assessment The peer review process  comments provided by the expert peer reviewers are documented, with every input being logged in a comprehensive database for follow-up, and assigned to different authors and staff to address  where conflict between/among review comments arises, coordinators contact the experts to discuss and determine an acceptable compromise  peer review builds upon previous activities by widening the scientific input

Division Of Early Warning And Assessment The peer review process (cont’d)  the process aims to harmonize methodologies (and) to improve and expand the peer review system and UNEP networks - in line with the recommendations of the Scientific and Technical Meeting on strengthening the scientific base of UNEP (January 2004)  The peer review process is, therefore, systematic and adapts experiences from other processes

Division Of Early Warning And Assessment Peer reviewers  have technical expertise in the subject matter for which they have been selected; they are expected to participate fully in the process and read all materials  should also protect confidentiality - by not sharing any AEO material with third parties  should respect the set deadlines, be objective and avoid bias  should report any problems encountered in finalizing the peer review process

Division Of Early Warning And Assessment Some peer review activities  Advising on the treatment of the issues across the chapters  Commenting on the key issues as highlighted  Pointing out other key issues which have not been highlighted but would be considered a serious omission if they were left out in the final report.  Where such omissions are highlighted, the reviewer should present. in a paragraph or two, the issues, including full relevant references.

Division Of Early Warning And Assessment  Where such omissions are highlighted, the reviewer should present within a paragraph or two the issues, including the relevant full references  Indicating additional sources of relevant data  Ensuring that trend data have been presented in all chapters.  Ensuring that key indicators on the state of the environment and environmental policy performance are properly tracked Some peer review… (cont’d)

Division Of Early Warning And Assessment  Paying special attention to the interlinkages between trends and policies (both specific to the issue under each chapter and more general policies that impinge on the issue)  demonstrating the environment as a driver of policy and vice versa, and evaluating policy in terms of environmental impact (effective, ineffective, unexpected, etc.)  Providing additional information and data, including boxes and other illustrations to enrich the sections, if necessary Some peer review… (cont’d)

Division Of Early Warning And Assessment  Pointing out inconsistencies and/or contradictions of facts/data within and across the chapters and regional perspectives  Highlighting the major messages from the state of the environment both at the regional and sub- regional levels  Providing a list of the major messages emanating from the state of the environment both at the regional and sub-regional levels  Highlighting the major policy weaknesses clearly evident from both the regional and sub-regional thematic sections Some peer review… (cont’d)

Division Of Early Warning And Assessment  Highlighting the major policy strengths clearly evident from both the regional and sub-regional thematic sections  Providing a list of both the strengths and weaknesses emanating from the state of the environment both at the regional and sub-regional levels  Advising on possible alternative policy initiatives that should be considered to ensure effective environmental management at both regional and sub-regional levels  Indicating any priority and emerging issues, or areas of outstanding vulnerability related to the issue as analysed Some peer review… (cont’d)

Division Of Early Warning And Assessment  Providing an assessment of the treatment of IEA, both between the chapters and the regional sections, as well as across them.  Providing detailed comments on all the points listed above. Reviewers should avoid sending questions on the sections but rather provide substantive comments, which can be considered during revision of the sections. Some peer review… (cont’d)

Division Of Early Warning And Assessment Finding peer reviewers Candidates may be recommended by:  government ministries/departments  scientific institutions  universities, collaborating centres  other stakeholders

Division Of Early Warning And Assessment Determining the Peer Review schedule The Peer Review schedule is a critical feature of the AEO process and should take the following into account:  availability of quality draft material  availability of appropriate experts  time available to use the review comments  deadlines for the AEO report and logistics

Division Of Early Warning And Assessment Documents to send peer reviewers  Most recent and quality copy of the draft environmental assessment report - this could be the whole or section of the report  Clear TORs, including specific activities and deadlines and a clear reporting hierarchy  Clear guidelines on how comments should be provided