Expert Committees R. F. Shangraw, Jr., Ph.D. Chief Executive Officer Project Performance Corporation Methods and Approaches in Screening Proprietary Drug Names Public Meeting Renaissance Washington DC Hotel June 26, 2003

Topics Framing the Problem Identifying Relevant Research Addressing the Key Questions Raising Two Other Concerns

Framing the Problem Using expert committees to review proposed proprietary drug names for potential sound alike and look alike confusion with existing drug names Using expert committees to review multifactoral studies on potential sound alike and look alike confusions Using expert committees to provide input to a single factor study Bound the Decision Identify Alternatives Evaluate Alternatives Consensus Decision

The Key Questions Is an expert committee necessary to review information from studies? How many people should staff an expert committee? What credentials are important for expert committee members? Should expert committees meet in person, via videoconference, teleconference, ?

Identifying Relevant Research Psychology Sociology Law – Jury Deliberations – Expert Witness – Science Court Policy Sciences – Forecasting – Game Theory – Organizational Behavior Medicine – Appropriate and Necessary Care Group Decision-Making Methods – Delphi Method – Nominal Group Technique – NIH Consensus Development Program – RAND/UCLA Appropriateness Method

Necessity of Expert Committees: Research Empirical research suggests group decisions are more consistent – individual judgments are prone to personal bias (McDonnell et al, 1996) There is some empirical evidence that groups make quicker and better decisions (Blinder & Morgan, 2000; Hogarth, 1978) However, some empirical research on group decision making suggests a systematic bias in the outcomes (Janis & Mann, 1977) But, a recent study found that groups are better than individuals in interpreting letters and number substitutions (Laughlin, et al., 2002)

Necessity of Expert Committees: Practice Use an Expert Panel When: Historical data does not exist or is limited Future events are likely to invalidate the results of historical analysis Ethical and moral factors are sufficiently important to the decision

Optimal Size of Expert Committees: Research Assuming specific voting rules, committee size is negatively correlated with decision accuracy (Gabel & Shipan, 2000) Communication quality and complexity is greater in small groups (n=5) than in larger groups (n=10) (Fay, et al., 2000) Limit to 12 – 15 members for effective functioning of the group (Shekelle, et al., 1999)

Optimal Size of Expert Committees: Practice Group Decision-Making Method Recommended Expert Committee Size RAND/UCLA Appropriateness Method 7 – 15 Panelists Nominal Group Technique8 – 12 Participants NIH Consensus Development Program 9 – 16 Panelists

Credentials for Committee Participation: Research Need a baseline level of expertise but additional expertise does not improve accuracy (Armstrong, 1980) Committees should include participation from a variety of relevant disciplines (Fitch et al., 2001) Participants’ status affects group dynamics (Murphy, et al., 1998) Participants must be justifiable as "expert" on the matter under discussion (Jones & Hunter, 1995)

Credentials for Committee Participation: Practice Establish baseline qualifications Review conflicts of interest – No advocacy or promotional position – No financial interests Minimize domineering personalities Increase diversity

Optimal Media for the Expert Committee: Research Recent research focuses on decision quality, satisfaction, and media richness Computer-mediated group decision systems decreases group effectiveness, increases time to complete task, and decreases member satisfaction in the process (Baltes, et al., 2002) However, groups with a history can be just as effective with a computer-mediated system (Alge, et al., 2003) Group decision support systems lead to more complex communications than simple chat systems (Brandy & Young, 2002)

Optimal Media for the Expert Committee: Research Adding video to an audio-only system improves decision quality (Baker, 2002) Participants are less likely to be committed to computer-mediated processes (Shangraw & Bozeman, 1989) After reviewing 200 studies, “no difference” between face-to-face meetings and specific collaborative technologies (Fjermestad & Holtz, 1998)

Optimal Format for the Expert Committee: Practice Combine computer-mediated and face-to-face discussions if economically feasible Match media to the type and structure of the decision

Addressing the Concerns with “Groupthink” Facilitator should be impartial Facilitator should assign the role of “critical evaluator” to all committee participants One or more participants should be rotated through the role of “devil’s advocate” If feasible, sub-divide the group to work under different facilitators Immediately following preliminary consensus, hold a “second chance” meeting to express any residual doubts

Using a Structured Approach: The Nominal Group Technique Briefing provided on topic and method Ideas are silently generated on paper Participants share one idea from his/her list Review and consolidate ideas Voting is done privately Votes are tabulated

Direction of Future Research Intense focus on the value of computer- mediated group decision-making system Specific interest in web-based, distributed computer-mediated group decision support systems Increasing interest in the value of combining expert panels with empirical, data-driven models