Volume 88, Issue 1, July (2002), pp. 1-2 © The Author 2002

Medline/PubMed Citation | Related Articles in PubMed | Download to Citation Matcher  

Editorial

Citations and 'impact factor' - the Holy Grail

Paul Trayhurn, Editor-in-Chief
Department of Medicine, University of Liverpool, Liverpool L69 3GA, UK
p.trayhurn@liverpool.ac.uk

The impact of the work of individual scientists and their institutions is increasingly being assessed on the basis of the number of times that their published work, and particularly primary papers, is cited. This quantitative measure, derived from the Science Citation Index database developed by ISI (the Institute for Scientific Information – now part of the Thomson Corporation), is replacing the more informal indices of peer recognition and esteem that have traditionally been used by the scientific community.

Citations are also important for individual journals. Journals have, of course, always been concerned to publish high quality work thought likely to exert a major influence on a field. This has become codified in terms of the ‘impact factor’ of a journal – now established as a critical issue in a highly competitive publishing environment. Impact factor is based on the number of times that articles in a journal are cited in the two years following the year of publication; it is calculated from the number of citations in a given year to articles published in the two previous years divided by the total number of articles published in the same two years. There are, however, some concerns about the validity of the database from which the figures are derived (Adams, 2002).

At the top end of the range of impact factors are the weekly publications, Science and Nature, with figures in the year 2000 of 23·8 and 25·8, respectively (Cell is even higher at 32·4). This reflects their status as prestigious journals with an ethos of publishing groundbreaking work. Impact factors are, however, linked to the particular area of science. Rapidly moving fields, such as molecular and cell biology, where in many cases critical experiments can be performed in a matter of weeks or even days, lead to rapid publication and potentially high levels of citation of preceding papers within the time frame used to calculate impact factors. In some other fields, such as areas of agriculture or nutrition, the timescales can be very different. In animal science, for example, a long-term feeding study may take several years to complete so that citation of earlier work on which it is based will not contribute to the impact factor of the original journals where such work was published. Fortunately, differences between areas of science are increasingly recognised when comparing impact factors of different categories of journal.

The impact factor of the journals in which an author has published is now widely used as a proxy for the quality of the work of that individual or group. This approach is extensively used by tenure and promotion committees, and in the awarding of Fellowships and grants. On a more parochial level, it is also informs the Research Assessment Exercise which so influences the research agenda of British universities. The emphasis on journal impact factor is a reflection of an ‘audit culture’ with its underlying assumption that precise quantitative measures can be derived for complex processes which can then be used for discriminatory and decision making purposes. However, judging an individual primarily on the basis of the journals in which they publish is a somewhat indirect way of assessing the significance of their work. Indeed, it is inappropriate since it has been clearly documented that the impact factor of a journal does not predict the significance or level of citation of an individual article or author within that journal (Seglen, 1997). In practice, article citation rates determine the journal impact factor and not vice versa (Seglen, 1997). It is intriguing, therefore, that the scientific community, particularly at a managerial or bureaucratic level, maintains such a focus on an index which has been shown to be misleading.

Despite the considerable reservations, impact factors are now very much a fact of life. The BJN is fortunate in that our impact factor has risen sharply in recent years. Thus over the last three years for which data is currently available it has risen from 1·71 in 1998 to 1·94 in 1999 and then to 2·42 in 2000. Because of the emphasis given to impact factors, authors increasingly use them in deciding where to submit their work. It is to be hoped, therefore, that the increasing impact factor of the BJN will encourage authors to submit their best work to us. This will, of course, establish a virtuous circle to which all international journals aspire.

There are several reasons why the impact factor of a journal may increase. In the case of our own recent rise, it may partly reflect the nature of the studies being published (and what is submitted to us) with more molecular/cellular-based work and fewer long-term feeding trials. In addition, the inclusion of reviews and supplements (which often provide a collection of ‘mini reviews’) is also likely to be part of the reason for our rising impact factor. Indeed, some journals have made the strategic decision to include reviews in order to increase their impact factor. That this index is influenced by reviews can be illustrated by recent data from the Biochemical Journal which cites a value of 29·4 for the reviews that it publishes as compared with 4·28 for the journal as a whole.

Impact factor has become such a critical issue for journals that some, particularly those where the figures are low, have implemented specific mechanisms for increasing the level of citation, in addition to publishing reviews. For example, authors are positively encouraged to cite articles in the particular journal. Indeed, earlier this year included with the editorial report from one journal to which my colleagues and I had submitted a paper was a request to cite some articles from the same journal, the editorial office enclosing the first page of four recent papers. Great care needs to be exercised with this type of approach and it is necessary to define the boundaries between what is and is not ethical in the drive to augment individual impact factors.

Irrespective of the question of overall impact factor, since it was founded the BJN has published a number of highly cited articles. Interrogation of the Science Citation Index suggests that the most highly cited paper that we have published is that by Durnin and Rahaman which appeared in 1967 – “The assessment of the amount of fat in the human body from measurements of skinfold thickness„ (Durnin & Rahaman, 1967). This paper has been cited some 700 times. A paper by Jackson & Pollack (1978) is the next most highly cited, receiving nearly 500 citations. These figures certainly underestimate the real number of citations to these papers (by definition only those journals included in the database are counted), particularly in the case of the Durnin & Rahaman (1967) study which was published well before citations started to be collected. More recently, several papers published in the late 1980’s and early 1990’s have each been cited well over a hundred times. Such levels of citation clearly indicate a major impact on nutritional science and over 5000 citations are now made each year to papers in the BJN.

Critics of citation analysis use several arguments against the approach: these include the disproportionate effect that a methods paper can have (e.g. the Lowry technique of protein determination); repeated reference to information that has proved to be wrong; and the issue of self-citation. Each of these concerns has some substance, but rather than negate the validity of citation analysis they illustrate the need for intelligent interpretation. It should be remembered that the vast majority of scientific papers are either never cited or are only cited once or twice. In my view, the number of times published work is cited by others is the best single indicator of the contribution made to a field, particularly when assessing the aggregated publications of an individual scientist over a period of years.