The use and abuse of citations Robert West & Kerstin Stenius

Outline What are citations for? What are citations for? What is good citation practice? What is good citation practice? What are the common abuses? What are the common abuses? Why does it matter? Why does it matter? Conclusions Conclusions

What are citations for? Referential citation Referential citation –to refer to a piece of research output that contributes to the field Critical citation Critical citation –to refer to research to draw attention to its flaws

Citation functions Verification function Verification function –to allow the reader to examine the cited work to check the veracity of a statement that it is being made in the citing paper Acknowledgement function Acknowledgement function –to give credit to the cited authors Documentation function Documentation function –to document how a political debate or historical process or a specific concept has developed and been defined

Good citation practice 1. Every substantive statement that relies on another source should use a citation 2. Citations should not be used in support of obvious common-sense statements 3. The choice of cited sources should be based on scientific considerations not convenience or other biases 4. Cited sources should be checked to make sure they are being accurately reported 5. Where there are conflicting findings this must be reflected in the citations

Bad citation practice 1 Selectively citing sources that one happens to be aware of: e.g. papers from one’s research group Selectively citing sources that one happens to be aware of: e.g. papers from one’s research group Selectively citing oneself or one’s friends or work from your own country: e.g. a tendency for US researchers to cite only US papers Selectively citing oneself or one’s friends or work from your own country: e.g. a tendency for US researchers to cite only US papers Selectively citing sources that support one’s opinions: e.g. failing to acknowledge that evidence is conflicting Selectively citing sources that support one’s opinions: e.g. failing to acknowledge that evidence is conflicting Citing sources that cannot be accessed: e.g. unpublished conference presentations Citing sources that cannot be accessed: e.g. unpublished conference presentations

Bad citation practice 2 Citing sources of evidence that have not been evaluated by peer review and that do not give enough detail to evaluate properly, e.g. conference abstracts Citing sources of evidence that have not been evaluated by peer review and that do not give enough detail to evaluate properly, e.g. conference abstracts Citing without reading e.g. relying on secondary sources Citing without reading e.g. relying on secondary sources Citing sources in support of statements when in fact they do not do so, e.g. misrepresenting the conclusions of the cited source Citing sources in support of statements when in fact they do not do so, e.g. misrepresenting the conclusions of the cited source Over-using citations to increase the apparent erudition of one’s writing, e.g. very long lists of citations for a single point or using a citation in support of some common sense statement) Over-using citations to increase the apparent erudition of one’s writing, e.g. very long lists of citations for a single point or using a citation in support of some common sense statement)

Getting cited Construct a title that includes important key words Construct a title that includes important key words Let your colleagues know about your paper Let your colleagues know about your paper Choose journals that are read by people in your field Choose journals that are read by people in your field Choose journals that are indexed in databases used by people in your field Choose journals that are indexed in databases used by people in your field

Citation counts as quality markers Number of citations is used as a marker of quality and reputation Number of citations is used as a marker of quality and reputation This is misleading because This is misleading because –bad citation practices fail to give due credit to researchers, particularly those outside the US –research in innovative areas will receive few citations –critical citations also get counted –it does not reflect the importance of the research for policy or clinical practice –reviews are typically cited more than empirical articles

Impact factors Journals are judged by their journal impact factor (JIF) Journals are judged by their journal impact factor (JIF) The JIF for 2004 is the number of citations in 2004 to articles in 2002 and 2003 divided by the number of articles in 2002 and 2003 The JIF for 2004 is the number of citations in 2004 to articles in 2002 and 2003 divided by the number of articles in 2002 and 2003 Although, within a research area JIF correlates to some degree with reputation of journals, across areas it is very misleading because of differences in publication lag, amount of research activity and citation practices Although, within a research area JIF correlates to some degree with reputation of journals, across areas it is very misleading because of differences in publication lag, amount of research activity and citation practices

Conclusions Citations provide the vehicle for transmission of information in science Citations provide the vehicle for transmission of information in science Bad citation practice undermines science and at its worst is little better than faking results Bad citation practice undermines science and at its worst is little better than faking results Bad citation practice also distorts science by establishing reputations that are not deserved and failing to credit researchers who deserve it Bad citation practice also distorts science by establishing reputations that are not deserved and failing to credit researchers who deserve it Citation counts are a poor index of quality but we appear to be stuck with them Citation counts are a poor index of quality but we appear to be stuck with them

Summary of things to look for 1. Is there a pro-US bias in the citation list? 2. Is there evidence of self-serving citation bias? 3. Are substantive statements being made that should be supported by citations but are not? 4. Are citations being made to statements that do not need them? 5. Are there long lists of citations for points that do not need them? 6. Are citations being made to derivative studies rather than the pivotal studies that made the discovery? 7. Are there citations to inaccessible sources? 8. Are there citations to empirical work where the work has not been and cannot be properly evaluated? 9. Are sources being cited selectively to support a viewpoint? 10. Are sources being cited accurately?