Shades of grey in academic citation practice

Published in 2016. Last reviewed in 2023.

Introduction

It is both pointless and probably impossible to credit all our sources of inspiration, but which ones should we include in our list of references? Where are the boundaries between what we may assume is common knowledge not requiring a citation and what is not? How important is it to ensure that readers can easily find their way to verify or scrutinize the sources we rely on when we write?

These are considerations and decisions that are not only made on the basis of ideals and rules spelled out more or less clearly in standardization documents and publishing manuals. Decisions about the use or non-use of citations are also a matter of pragmatic concerns, the use of common sense, and degrees of honesty.

Excessive citations can be overwhelming, sometimes making an academic text virtually unreadable, and large numbers of irrelevant or unnecessary citations can make the really important ones disappear in the crowd. Academics should strive to use primary sources, but how far we should go in order to follow this principle must be weighed against a number of considerations. One of them may simply be the amount of time and resources we have at our disposal in order to ensure the quality of our work.

Academic citation practice thus implies manoeuvring in a landscape where compromises sometimes must be made between ideals and rules on the one hand, and pragmatic and practical considerations on the other, if we aim to produce a fairly readable text, or to be able to complete an academic text at all.

In this landscape, one moves away from clear dichotomies such as 'correct'/'wrong' or 'honest'/'fraudulent' and enters grey zones where the decisions are not immediately obvious, and where, in addition, there might be considerable disagreement about what is the best solution.

It is difficult to convey or communicate claims of what are good and bad citation practices without using concrete examples. In this article I will limit myself to a particular issue that many academics and their students are struggling to find the best solution for: that which arises when we read a text and find an assertion that we want to use, and this claim has already been accompanied by a source reference in the text we are reading. In other words, we are reading a secondary source that refers to another source we assume is a primary source for the assertion that caught our attention. The considerations we have to weigh in such a situation may be well suited to discussing the importance of some central principles for academic citation practices, and the implications of breaking or ignoring them.

Spinach as a good source of iron [1]

I will illustrate with an example I encountered not long ago, in which a scientific article I was reading presented me with some new and outright fascinating knowledge. The following quote, including the reference, is taken from an article published by K. Sune Larsson (1995: 448-449) in the Journal of Internal Medicine:

'The myth from the 1930s that spinach is a rich source of iron was due to misleading information in the original publication: a malpositioned decimal point gave a 10-fold overestimate of iron content (Hamblin, 1981).'[2]

The quote caught my attention for two reasons. First of all, it falsified an idea that I had carried with me since I was a child, that spinach is an excellent source of iron. The most striking thing, however, was that a single decimal point, misplaced 80 years ago, had affected not just myself and my now deceased parents, but a large number of others with regard to what we place on our table.

Truth be told, there is iron in spinach, but not significantly more than in other green vegetables. A larger problem with the idea of spinach as a good source of iron, however, is that it also contains substances that strongly inhibit the intestinal absorption of this mineral (see e.g. Garrison, 2009: 400). In addition, few people can consume spinach in large quantities. Simply put, there are a number of better and more practical ways of absorbing iron than by eating spinach.

Larsson's article made me aware of the remarkable fact that a large number of people in the Western world have been misled for a staggeringly long time. Since many people may still believe that spinach is an excellent source of iron, I have good reason to convey this new-found knowledge to others. The story of this decimal point error is, in addition, a brilliant illustration of how a small stroke may fell a great oak, and a reminder of the importance of accuracy and quality control in the production and distribution of scientific knowledge.

How, then, should I properly pass on the important messages I learned from a single sentence in Larsson's article? The following seems like a fairly appropriate paraphrasing of the original text:

'The idea that spinach is a good source of iron is a myth that was born in the 1930s due to a misplaced decimal point, causing the concentration to appear ten times higher than its real value.'

How should I refer to my source? If I want to include this sentence in an academic publication, what should I place after my sentence? There are several options in this particular case, and I will present the most common among them and discuss what consequences the various alternatives may have.

The first alternative is to leave the sentence as it is, without any reference at all. This is something I can do if I am communicating common knowledge, which obviously is not the case here. Should I choose to omit a reference, I could in the worst case be accused of plagiarism, and the more naive among my readers would perhaps think that I was the one who discovered the decimal point error with the dramatic consequences. A more likely outcome would be that my readers become puzzled, or perhaps irritated, by the fact that I did not provide any form of documentation for how such a remarkable thing could occur. In principle it should be impossible or very difficult to get undocumented statements of this kind published in scientific publications, but as we will see towards the end of this article, it happens from time to time.

The simple truth is too simple

In academia, the following is fortunately a far more common way of passing on such a message:

'The idea that spinach is a good source of iron is a myth that was born in the 1930s due to a misplaced decimal point, causing the concentration to appear ten times higher than its real value (Larsson, 1995: 448-449).'

Here I simply and honestly refer directly to the source where I found the information, and I am even courteous enough to provide exact page numbers for readers who would like to verify it, or who may be interested in exploring whether there is more to learn from Larsson. The problem in this case is that I omit a piece of information: the fact that Larsson's statement is based on an entirely different source, namely Hamblin (1981). In other words, I am referring to an article that I very well know is a secondary source, and thus hide from my readers the fact that Larsson actually just passed on information published by Hamblin 14 years earlier.

A good reason for avoiding the use of secondary sources in academia is that messages that pass through several links have the unfortunate tendency to become modified or altered along the way, as in the game of Chinese whispers. My readers will in this case think that Larsson is the primary source, and my statement will therefore look more solid and trustworthy than it actually is.

Providing this type of reference has other negative consequences. This time it is not me, but Larsson who gets undeserved credit for the discovery of the decimal point error. Another consequence is that readers who try to verify my statement will get an unpleasant surprise when they look up the source (Larsson, 1995: 448-449) I have provided. They will then discover that they have become, quite unwillingly, participants in a kind of treasure hunt. Having reached the first hurdle at page 449 in Larsson's article, their only options would be either to give up the quest for verification, or to proceed to the next destination, Hamblin's 1981 article in the British Medical Journal.

In a case like this, when I am aware of the fact that my citation has weaknesses, it could be tempting to try to make the statement more convincing by adding more references I might have easily available. In our digital age it is not difficult to find other sources that contain the story about the decimal point error and its dramatic consequences. If the supply is as rich as in this case, it is a good idea to select alternatives published fairly recently in respectable journals, such as this article: (Frangoulis, Carlotti, Eisenhauer, & Zervoudaki, 2010: 43), or maybe this book: (Carroll & Vreeman, 2009: 114). If I want, I can add numerous sources like this, getting a long and impressive list of references, full of prestigious journal names and publishers. This would, of course, be an academic confidence trick, but it would not be exposed until readers took the time to look up the sources I listed. Only then will they be able to see that they are all secondary sources, and that they all refer back to the same single sentence in Hamblin's 1981 article.

Honesty is not always the best policy

A third and even more honest alternative would be to refer to my source in this way:

'The idea that spinach is a good source of iron is a myth that was born in the 1930s due to a misplaced decimal point, causing the concentration to appear ten times higher than its real value (Hamblin, 1981, cited in Larsson, 1995: 448-449).'[3]

This is a perfectly legitimate way of referring to sources in cases where it is difficult or impossible to obtain a primary source. The 1981 volume of the British Medical Journal is, however, easily available for anyone with Internet access. Should this type of reference be used in this particular case, it could reflect a case of academic laziness, but coupled with the utmost honesty. Another and perhaps more likely explanation is that we are dealing with an academic who has not understood the importance of the principle of striving to use primary sources in order to minimize the Chinese whispers effect.

This type of citation does not necessarily have to be explained by laziness or lack of knowledge, but rather by an almost touching degree of confidence and trust. In this case I put my trust in Larsson, that he has read and interpreted Hamblin correctly, and that he has good enough reasons for putting his trust in Hamblin. Whatever the explanation, I deserve credit for having made it perfectly clear that I have not consulted the primary source, and that my statement is the last and therefore the least trustworthy link in a chain of sources.

The risk of buying a pig in a poke

A fourth alternative, which unfortunately is far more common than we might wish, is to solve the problem the following way, without consulting Hamblin (1981):

'The idea that spinach is a good source of iron is a myth that was born in the 1930s due to a misplaced decimal point, causing the concentration to appear ten times higher than its real value (Hamblin, 1981).'

In this case I am referring directly to a source that I have not consulted myself, and in doing so I am guilty of an academic lie. The same degree of trust as in the previous alternative is present, but the difference is that the stakes are now much higher. What I hope to achieve with this type of reference is that nobody will discover my laziness. I simply pretend that I have taken the effort to consult Hamblin (1981), without having done so. In short, I have plagiarized the Hamblin reference from Larsson.

An attractive aspect of this academic shortcut is that it is usually impossible to discover and to prove the sin committed. Academics such as Larsson presumably check their sources thoroughly, and double-check that their own text corresponds with the sources it refers to. If Larsson has understood Hamblin correctly, and Hamblin is worthy of his trust, then there would be no negative consequences from this highly dubious type of reference, neither for my readers, nor for the truth and reliability of what I am writing.

Referring to sources that one has not consulted can, however, be a risky business. Academics, like other human beings, do from time to time misinterpret or make errors that are not discovered by peer reviewers or editors, even in respectable journals such as the Journal of Internal Medicine and the British Medical Journal. When several authors independently of each other manage to misrepresent a single source in exactly the same erroneous way, the explanation is either a statistically unlikely coincidence, or a case where authors have plagiarized references. Systematically patterned distributions of errors and misinterpretations are in fact common enough to make it possible to study the prevalence of citation plagiarism and the unfortunate consequences of the practice. Such studies indeed make sad reading for those who are concerned about safeguarding academic principles of honesty and quality control (see e.g. Harzing, 2002; Morrisey, 2004: 152-154; Wetterer, 2006; Wright & Armstrong, 2008).

The narrow path has its temptations

The four alternatives treated so far all represent various types of academic shortcuts, and they all share attributes with various ways in which rumours are spread. The common denominator for all of them is that I do not consult the assumed primary source in this case. I simply end up, in a more or less honest way, passing on what I have read that Larsson has read in Hamblin (1981).

The final and undoubtedly the best alternative is to follow the short and narrow path back to Hamblin's article to see what he wrote on the issue. It is, of course, wise to check the accuracy of what we base ourselves upon when we write and publish, and there is also the possibility that we might learn something even more valuable about the topic.

In this particular case there is in fact a lot more to learn from what we have so far assumed is the primary source. It turns out that Larsson has in fact made several errors when reproducing Hamblin's message, and on top of it all, Hamblin is not at all to be trusted in this particular case.

Hamblin does not provide a reference to support his claim that a decimal point error actually was made; nor any names, dates or other information that could help us verify how the error was made and by whom, or who should be credited for its discovery and correction. In short, there is no documentation that the decimal point error ever occurred.

Despite the complete lack of documentation that characterizes Hamblin's account of the decimal point error, the story has been picked up by numerous authors who have redistributed it through journal articles and books, turning it into a full-blown and still thriving academic urban legend. The decimal point error has in fact become common knowledge within sectors of academia to the extent that it can be used in journal articles and books without any reference at all (see e.g. Adesman, 2009: 39; Coughlin & Zitarelli, 1984: 116; De Beuckelaer, 2002: 194; Gustavii, 2003: 116).

We have, in other words, found an urban legend circulating where urban legends by definition should not occur at all. The reason why this could occur is that a large number of academics have disregarded two important principles for handling of knowledge: striving to consult primary sources, and reading academic texts critically, no matter how well they are published.[4]

The assertion I found in Larsson's article was both concrete and remarkable, and may serve as a good example of a case where it should be natural to consult the primary source for verification. The readers who have done so will have discovered that both Larsson and Hamblin have committed mistakes, and that the statement which caught their attention is dubious or undocumented. In such a situation a temptation may arise: I very well know that the decimal point error story suffers from a lack of documentation, but I cannot resist the temptation to use it, perhaps because it fits so nicely into the argument of an article I am writing.

It is not difficult to find similar examples in academic publications, and it is both strange and worrisome that this is happening at a time when it has never been easier to consult primary sources. The digital revolution has made the procedure of consulting primary sources much more efficient than was the case a few decades ago, but many academics do not grasp this opportunity to ensure or enhance the quality of their works. A part of the explanation for this phenomenon may be that consulting primary sources still implies spending a little extra time and effort. In addition, when we investigate sources we also risk discovering that the matter is considerably more complex and unmanageable than we thought, or as in the spinach case, we find errors and lack of documentation that make our new-found knowledge unsuitable for further dissemination. It may be depressing to realize that what we have spent time and effort investigating is useless, but that is unfortunately both a necessary and important aspect of academic daily life.

In academia the criteria of truth are far more important than conveying a good story. If we do not care about spending time and effort to consult, explore and critically examine sources, we are also contributing to blurring the distinction between science and nonsense.

Footnotes

[1] The following example and discussion is to a large extent identical to the major example in Rekdal (2014).

[2] Larsson's article is written in a style common in medical journals, and the reference to Hamblin's work appears as a numbered note in the text that I have here changed to a citation matching this website's style.

[3] The reference will look different in other styles of referencing, but the principle of communicating clearly to the reader that the message comes from a secondary source, remains the same. In medical journals following the AMA manual of style: a guide for authors and editors (American Medical Association, 2007: 61) this particular reference will appear as a numbered note in the text, and the corresponding entry in the list of references will appear like this:

Hamblin, T. Fake. Br Med J. 1981;283(6307):1671-1674. Cited by: Larsson, K. The dissemination of false data through inadequate citation. J Intern Med. 1995;238(5):448-449.

[4] For a more comprehensive account of the history of this academic urban legend, see Rekdal (2014).

References

Adesman, A. (2009). Babyfacts: the truth about your child's health, from newborn through preschool. Hoboken: John Wiley & Sons. 

American Medical Association. (2007). AMA manual of style: a guide for authors and editors (10. ed.). New York: Oxford University Press. 

Carroll, A. E., & Vreeman, R. C. (2009). Don't swallow your gum!: Myths, half-truths, and outright lies about your body and health. New York: St. Martin's Griffin. 

Coughlin, R., & Zitarelli, D. E. (1984). The ascent of mathematics. New York: MacGraw-Hill Book Company. 

De Beuckelaer, G. M. (2002). It's broken, let's fix it: The Zeitgeist and modern enterprise. Berlin: Springer. 

Frangoulis, C., Carlotti, F., Eisenhauer, L., & Zervoudaki, S. (2010). Converting copepod vital rates into units appropriate for biogeochemical models. Progress in Oceanography, 84(1-2), 43-51.  

Garrison, C. D. (2009). The Iron Disorders Institute guide to anemia (2. ed.). Nashville, TN: Cumberland House. 

Gustavii, B. (2003). How to write & illustrate a scientific paper. Cambridge: Cambridge University Press. 

Hamblin, T. J. (1981). Fake! British Medical Journal, 283(6307), 1671-1674.  

Harzing, A.-W.. (2002). Are our referencing errors undermining our scholarship and credibility? The case of expatriate failure rates. Journal of Organizational Behavior, 23(1), 127-148. 

Larsson, K. S. (1995). The dissemination of false data through inadequate citation. Journal of Internal Medicine, 238(5), 445-450.  

Morrisey, L. J. (2004). Bibliometric and bibliographic analysis in an era of electronic scholarly communication. Science and Technology Libraries, 22(3), 149-160.  

Rekdal, O. B. (2014). Academic urban legends. Social Studies of Science, 44(4), 638–654. (Open access: http://sss.sagepub.com/content/44/4/638.full) 

Wetterer, J. K. (2006). Quotation error, citation copying, and ant extinctions in Madeira. Scientometrics, 67(3), 351-372. 

Wright, M., & Armstrong, J. S. (2008). The ombudsman: Verification of citations: Fawlty Towers of knowledge? Interfaces, 38(2), 125-132.