1. Dr Mark Walport Director The Wellcome Trust
Charities, excellence & assessment
Dr Mark Walport
Director
The Wellcome Trust
key messages:
- science is important for improving both health and wealth
- it needs to be managed flexibly - a balance between supporting 'blue skies' research and strategic areas
- measuring success is diffiicult - need to ensure we adopt the right drivers

2. Outline how can research charities meet their mission?
how do we pick winners?
how do we assess outputs?
the future

3. What do universities look like?
the RAE has:
driven excellence
focussed attention on outputs
but has had unintended consequence:
increased volume of research at expense of physical infrastructure
lecturers appointed rather than support staff
pressure for short term research results
damaged clinical research capacity
the work of teams devalued
teaching devalued
RAE 2001
RAE 1996
RAE 1992

4. UK Strength Plurality of funding for biomedical research
*Cancer Research UK was formed in 2002, following the merger of ICRF and Cancer Research Campaign (CRC). Figures before 2002 show the combined spending of ICRF and CRC (Source: Cancer Research, Science and Technology Committee)
The AMRC line shows the total spend on UK medical and health research by AMRC members charities. The figure for 2005 is an estimate based on AMRC subscription data.
Trust funding includes international spend. The Gates Foundation figure only includes grants to institutions in the UK.

5. Charities can meet their missions if…
universities ‘fit for purpose’
a funding model to promote this
we hope to be able to…
fund the best researchers
fund the best ideas
fund in the best research environments
get value for money

6. Picking winners Identifying excellence
are you a good scientist?
what has been discovered?
who has been trained?
how have outputs been communicated?
what is your question?
why is it important?
how will you approach the question?
what resources do you need?

7. Peer review of inputs robust and rigorous
more thorough than RAE methods
international
but can be risk averse

8. Charity Research Support Fund
a metric based award mechanism in relation to quality charitable income
an explicit volume of QR ring-fenced as partnership fund
included in block grant to VCs
merits:
directly rewards success
rigorous peer review
responsive to change between RAEs
enables universities to support infrastructure for research in manner that best suits university

9. The issues Assessing outputs
adoption of true/realistic measures/indicators:
- recognition of the incremental & unpredictability of research
- challenge of identifying ‘impact’ from basic research
avoiding perverse incentives
- weighing papers
- citation/impact factors
managerial vs professional targets
- quantitative vs qualitative indicators
focus on national indicators for an increasingly global pursuit
attribution - multiple funding partners, international collaboration

10. Witness Seminars – obstetric ultrasound
leadership of Professor Ian Donald s
remarkable collaboration between engineers and clinicians
resistance from doctors who had spent years ‘training their hands to see’
demonstration, refinement and results led to huge change in clinical practice

11. ‘PubMed 1000’ project
The PubMed ‘1000’ project aimed to use the first 1000 papers with WT tag in acknowledgement.
3 main aims
describe the papers - the nature of that association.
Review the papers – to find important research & papers classified as ‘landmarks’
Consider the potential role of PubMed as a resource to support evaluation work – clearly as source of access to scientific publication output, and one of the Trust’s core strategic aims is ‘advancing knowledge’ then clearly PubMed is a useful resource.

12. Funds awarded vs. papers produced for the top 15 institutions & trend line

13. Importance rating by nature of research
Demonstrates that the Clinical and Mixed papers tended to get lower ‘importance ratings’ ….
Though again …
pilot project
small numbers involved overall (i.e. clinical papers)
base = 573 original research papers

14. Importance assessments of original research papers vs JIFs (log scale)
For the
record 3
Useful step
Forward 5
Major
addition to
knowledge 7
Landmark
paper

15. Biomedical research in the UK: The future
very strong biomedical research base in UK
reward excellence
drive innovation
build on successful work of UK CRC
get the Health Research Fund right
ensure post-RAE metrics reward national strength in biomedical research