1. Assessment of quality decision making in regulatory agencies and pharmaceutical companies with QoDoS (Quality of Decision Making Orientation Scheme)
Not at all
Sometimes
Frequently
Often
Always
Evaluates impact of decisions
Applies a structured approach to decision making
Qualifies probability of success
Quantifies probability of success
Decision making unbiased by external stakeholder demands
Decision making unbiased by internal politics
Decision making unbiased by vested interests of individuals
Decision making unbiased by competitors
Provides training in science of decision making
From: CIRS R&D Briefing 61 Building Quality into Decision-Making Processes in Medicines’ Development, Regulatory Review and Health Technology Assessment. Available at
Introduction
Key Messages
The various decisions made by pharmaceutical companies, regulatory authorities and health technology assessment (HTA) agencies throughout the life cycle of medicines are critical for ensuring that appropriately safe and effective medicines become available in a timely and efficient manner. Despite this, there is a paucity of research into the quality aspect of decision making in this area.
As a consequence, CIRS initiated a programme that aims to address the research gap in quality decision making in the area of medicines’ development, review and HTA assessment.
One of the objectives of the CIRS programme is to utilise the Quality of Decision-Making Orientation Scheme (QoDoS) to assess the quality of decision-making process and evaluate the level of incorporation of the 10 Quality Decision-Making Practices (Bujar et al., 2016a) within companies, regulatory and HTA agencies.
Methodology
QoDoS, consisting of 47 items that measure individual and organisational decision-making approaches and influences (Donelan et al. 2015, 2013) was completed by 76 participants from regulatory agencies and pharmaceutical companies (Bujar et al., 2016a).
All participants were asked to respond to each of the QoDoS 47 statements using a 5-point Likert scale, where 0 = not at all, 1 = sometimes, 2 = frequently, 3 = often and 4 = always.
Results
A total of 38 individuals (27 male, 11 female) from 12 regulatory agencies and 38 individuals (22 male, 16 female) from 23 pharmaceutical companies with varying levels of professional experience (range, 2-40 years) and representing medium-sized and large organisations participated in the study.
Nine selected organisational-level QoDoS items are shown and the results demonstrate key differences between company and agency practices. Both groups considered evaluating the impact of the decisions as an important factor, with agencies using a structured, systematic approach to decision making more frequently than companies. Conversely, there was a general tendency for biases, due to politics, competitors or incentives, to have more impact on decision making for companies compared with agencies. Whilst it was recognised that the science of decision making is important, training in this area was rarely provided. All responders from agencies and 92% from companies felt that they could make better decisions.
Discussion
The study results show that companies perceive their decision making is influenced to a greater extent by certain biases due to politics, competitors, vested interests of individuals, or incentives compared with agencies. This finding may be a result of a mixture of complex factors within the company medicine development process, such as the large number of stakeholders involved, as well as the length of the process and the number of key go/no- go decision points that lead to a final company submission. Other aspects include differences in culture as well as economic pressures, which affect companies to a greater extent.
There was a large difference in responses regarding quantification and qualification of the probability of success, which relates to assigning values and relative importance to decision criteria’, with agencies performing this less routinely than companies. This is consistent with the findings from previous research, and one of the recommendations suggested by companies and agencies to improve the quality of their decision making was to make values, preferences, and uncertainty more explicit. As advised by the responders, such an outcome can be achieved through the incorporation of more formal frameworks, such as a benefit-risk framework by agencies as well as more formal approaches to quality decision making within the organisation and by individuals (Bujar et al., 2016b).
Finally, this study confirmed the need for improvement and training in decision making for both stakeholders.
References
Bujar M, Donelan R, McAuslane N, Salek S, Walker S. Assessing quality of decision making in medicines’ development and the regulatory review: Identifying biases and best practices. Ther Inn Reg Sci. 2016a; doi: /
Bujar M, McAuslane N, Salek S, Walker S. Quality of regulatory decision-making practices: issues facing companies and agencies. Ther Inn Reg Sci. 2016b;DOI: /
Donelan R, Walker S, Salek S. Factors influencing quality decision-making: regulatory and pharmaceutical industry perspectives. Pharmacoepidemiol Drug Saf. 2015;24:
Donelan R, Walker S, Salek S The development and validation of a generic instrument, QoDoS, for assessing the quality of decision making. Frontiers Pharmacol. 2016;7:180.
Worst practice
Needs improvement
Best practice
Average response
Pharmaceutical company (n=38)
Regulatory agency (n=38)