Tips for Writing Effective Literature Review
A novice researcher can believe that the doing of research is primarily about the investigation—running experiments, developing theory, or doing the analysis. With experience, though, researchers discover the importance of developing an understanding.
It has been argued that many experimental researchers do their best work after they have been in a field for five years or more because it takes time to acquire a deep, thorough appreciation of the area, and of existing knowledge and its limitations.
To acquire this understanding, you need to become an effective reader of research papers. This blog explains the 15 Tips for Writing Effective Literature Review
A successful reader can identify the contributions and value of a paper, while recognizing its flaws; and uses critical scrutiny to identify the extent to which the flaws in a paper are serious. This reading then informs new work, directly as a source of knowledge and indirectly as a guide to how to produce work that will be appreciated.
A particular application of reading, moreover, is to become a reliable referee or thesis examiner. It is an introduction to the elements of effective reading and is a guide to reviewing. Reviewing is a central part of the scientific process.
Criticism and analysis of papers written by other scientists is the main mechanism for identifying good research and eliminating bad and is arguably as important an activity as the research itself.
Many people are intimidated by the task of reviewing, perhaps because it is a kind of intellectual test: you have been asked to demonstrate a thorough understanding of someone else’s work.
It is also intimidating because it brings responsibility; you want to neither wrongly criticize solid work nor recommend that flawed research be published. Unhelpfully, the quality of the review is highly variable—most researchers have stories to tell of good work rejected with only a few hasty words of explanation, or of referees who haven’t read the work at all.
Reviewing can be a chore but deserves the same effort, care, and ethical standards as any other research activity. And it has rewards, beyond the gratitude of editors and authors.
It can lead you to look at your own work from a fresh perspective, and exposes you to different kinds of error or failure in research—the average standard of work submitted for publication is well below that of work that gets published.
And, while you may not be asked to referee a paper as part of your research, your own work will be reviewed or examined, and thus provides a perspective on the standards expected of a submitted paper.
By the time your Research is complete, you need to be confident that you have read and understood all of the scientific literature that has a significant connection to your work. Your reading achieves several aims.
It establishes that your work is indeed novel or innovative; it helps you to understand a current theory, discoveries, and debates.
It can identify new lines of questioning or investigation, and it should provide alternative perspectives on your work. This reading will ultimately be summarized in the background sections and the discussions of related work in your write-ups.
The literature on which your work will rely is usually expected to be papers that have been refereed and published in a reputable venue, theses that have been undertaken and examined at a reputable institution, and books that are based on the information presented in refereed theses and books.
These are the documents that are accepted by the research community as a source of knowledge; indeed, they can be regarded as being the entirety of our scientific knowledge. The literature does not include primary sources such as lab notebooks, responses to a survey, or outputs from an experiment.
What this lack is an interpretation of the contents in light of a specific hypothesis. Other literature—news articles, science magazines, Wikipedia pages, or documentation.
For example—may alert you to the existence of reputable work but is rarely worth citing. That is, your learning may be built on a wider literature, but the arguments in your write-up should be based on the knowledge that is from a refereed source.
A thorough search of the literature can easily lead to the discovery of hundreds of potentially relevant papers. However, papers are not textbooks, and should not be treated as textbooks.
A researcher reading a paper is not studying for an exam; there is rarely a need to understand every line. The number of papers that a researcher working on a particular project has to know well is usually small, even though the number the researcher should have read to establish their relevance is large.
Thus it is important to become an effective reader, by giving each paper neither more nor less time than it deserves. The first time you read a paper, skim through it to identify the extent to which it is relevant—only read it thoroughly if there is likely to be value in doing so.
Make the effort to properly understand the details, but always beware of details that may be wrong, or garbled.
Finding Research Papers
Each research project builds on a body of prior work. Doing and describing research requires a thorough knowledge of the work of others.
However, the number of papers published in major computer science venues each year is at least tens of thousands, a volume that prohibits reading or understanding more than a fraction of the papers appearing in any one field.
A consolation is that, in an active field, other researchers have to a certain extent already explored and digested the older literature.
Their work provides a guide to earlier research—as will your work, once it is published—and thus a complete exploration of the archives is rarely necessary. However, this is one more reason to carefully search for current work.
And beware: reading about a paper that seems relevant is never a substitute for reading the paper itself. If you need to discuss or cite a paper, read it first.
A comprehensive exploration of relevant literature involves following several intertwined paths:
Use obvious search terms to explore the Web. You are likely to find, not just papers, but also home pages for projects and teams concerned with the same research area.
You are also likely to find documents that suggest further valuable search terms. Be exploratory in your search; sometimes the research in an area is divided across separate communities that have different vocabularies.
Some of the major search engines have search tools that are specifically for academic papers. These may index by an individual, by the institution, and by citation. They are today the single most effective method for finding relevant work. Visit the websites of research groups and researchers working in the area.
These sites should give several kinds of links into the wider literature: the names of researchers whose work you should investigate, the names of their co-authors, conferences where relevant work appears, and papers with lists of references to explore.
Follow up the references in promising research papers. These indicate relevant individuals, conferences, and journals. Browse the recent issues of the journals and conferences in the area; search other journals and conferences that might carry relevant papers. Search the publisher-specific digital libraries.
These include publishers such as Wiley and Springer and professional societies such as the ACM and IEEE.
Most conferences have websites that list the program, that is, the papers to appear in the conference that year. Within a conference, papers are often grouped by topic—another hint of relevance.
Consider using the citation indexes. The traditionally printed citation indexes have migrated to the Web, but in practice, their value for computer science is limited, as only a fraction of publications is included.
Go to the library. The simple strategy of having similar material shelved together often leads to unexpected discoveries, without the distractions that arise when browsing the Web.
Discuss your work with as many people as possible. Some of them may well know of relevant work you haven’t encountered. Similar problems often arise in disparate research areas, but the difficulties of keeping up with other fields—the phenomenon sometimes characterized as “working in silos”—mean that people investigating similar problems can be unaware of one another.
The process of search and discovery of useful papers can be thought of as a form of learning.
Each paper or page that you find should refine your understanding of the terminology, help indicate which papers are significant, suggest new directions follow up, and further clarify your criteria for whether a paper is “in” or “out”.
Occasionally there are several versions of the same paper: a preprint in an online archive, a conference version, and a journal version. You should use the version that the author appears to regard as definitive; this will usually be the polished work that has appeared in a journal.
Take a broad definition of “relevant” when searching for papers. It doesn’t just mean those papers that have, say, proposals for competing methods. Does the paper have interesting insights into other research literature?
Does it establish a benchmark? Have the authors found a clever way of proving a theorem that you can apply in your own work? Does the paper justify a decision to not pursue some particular line of investigation? Other people’s research can have many different kinds of effect on your work.
Finding all relevant work is hard; for example, exhaustive professional searches across the medical research literature can take months of full-time work. But finding all significant work is a critical part of doing research.
It is typically the case, too, that the scope of literature that is relevant to you may only be obvious once you have completed the investigative phase of your research and have a good draft of your literature review.
Your topic and interests are likely to shift, focus, or broaden during your research, and your perspective will change as your understanding develops— update your searching as you go.
Searching and reading are separate activities, and it is a mistake to try and do both at once. I recommend that you uncritically gather material and then later critically analyze and categorize.
Save the papers you find into a directory, and go through it later to understand what you have found. In the context of a single search session, it is also helpful to restrict your attention to one or two specific topics.
Having explored the literature, you may discover that your original idea is not so original after all. If so, be honest—review your work to see what aspects may be novel, but don’t fool yourself into working on a problem that is already solved.
Occasionally it happens, for example, that the same problem has been investigated by several other teams over a considerable period. At the same time, the fact that other people have worked on the same problem does not mean that it is impossible to make further contributions in the area.
A key aim of reading is to develop critical thinking skills. Good researchers must demonstrate their ability to objectively analyze the work and claims of others. With experience, you can place each paper in a context of other work that you know, and assess it on a range of characteristics.
In doing so, you will become alert to common mistakes and bogus claims. A challenge of research literature is whether to believe what you read.
Work published in a reputable journal or conference is peer-reviewed; work available online could have any history, from being a prepublication version of an accepted journal paper to being plagiarized work poorly translated from a non-English original.
A cynical but often accurate rule of thumb is that work that is more than one or two years old and has not been published in a significant venue probably has some serious defect.
When you find a version of a paper on the Web, establish whether it has been published somewhere. Use evidence such as the quality of the author’s other publications to establish whether it is part of a serious program of research.
Much research—far too much is just misguided. People investigate problems that are already solved and well understood, or solve problems that technology has made irrelevant, or don’t realize that the proposed improvement actually makes the method worse.
Mathematics may be pointless; the wrong property may be proved, such as complexity instead of correctness; assumptions may be implausible; evaluation strategies may not make sense.
The data set used may be so tiny that the results are meaningless. Some results are just plain wrong. And, while the fact that a paper is referred is an indicator that it is of value, it is not a guarantee. Many people undertake work that did not deserve to be written; sometimes it gets published.
Indeed, few papers are perfect. They are a presentation of new work rather than a considered explanation of well-known results, and the constraints of writing to a deadline mean that mistakes are undiscovered and some issues unexplored.
Some aspects of older papers may be superseded or irrelevant or may rely on limited or technically outdated assumptions. A paper can be seen as a snapshot of a research program at a moment in time—what the researchers knew when they submitted.
For all these reasons, a reader needs to be questioning, balanced, and skeptical. In short, don’t accept something as true just because it was published.
But that does not justify researchers being dismissive of past work; rather, they should respect it and learn from it, because their own work is likely to have similar strengths and weaknesses.
Some inexperienced researchers see other work as either perfect or poor, with nothing in-between. Usually, neither of these extremes is correct.
While many papers may be flawed, they define scientific knowledge. (In contrast, textbooks are usually consolidations of older, established work that is no longer at the frontier.)
If many researchers trust a particular paper, it is still reasonable to be skeptical of its results, but this needs to be balanced against the fact that, if skepticism is justified, these other researchers are all mistaken.
Read papers by asking critical questions of them, such as:
Is there a contribution? Is it significant? Is the contribution of interest? Are the results correct?
Is the appropriate literature discussed?
Does the methodology actually answer the initial question? Are the proposals and results critically analyzed?
Are appropriate conclusions drawn from the results, or are there other possible interpretations?
Are all the technical details correct? Are they sensible?
Could the results be verified?
Are there any serious ambiguities or inconsistencies?
That is, actively attempt to identify the contributions and shortcomings rather than simply reading from one end to the other. This analysis of a paper can be thought of as verifying that each component is robust.
If the paper is important to your work, you should analyze it until you have formed a reasoned opinion about each of its components; and if some components are questionable, this should be reflected in your literature review.
Write down your analysis of the paper —you will read hundreds of papers and in some cases will not formally describe them until months or years later.
However, detailed analysis can be difficult before you have undertaken your own work, and in so doing developed a mature perspective. Thus reviewing of literature should not stop when the investigation begins, but continue alongside the research.
Developing a Literature Review
A literature review is a structured analysis of a body of literature and may cover work from several separate areas of research. This review is not simply a list of these papers.
Rather, the papers should be grouped by topic, and critically discussed in a way that allows the reader to understand their contribution to the field, their limitations, and the questions that they leave open.
The task of writing a literature review for a paper can be challenging, and for a thesis can be more demanding than any other single activity. It, therefore, makes good sense to develop the review progressively.
Begin a rough literature review as soon as you start reading, and, when you read a paper that you think will need to be discussed, add it in. (You should also capture the bibliographic data as you go, and also keep a copy of every paper you read.)
Initially, the literature review will be sketchy and unstructured, but as you add papers you can group them by topic and contribution, and add notes on each paper and how they relate to each other.
Briefly summarize each paper’s contribution and the evidence used to support the claims, and also note any shortcomings or features that are of interest.
You might also want to note, for your own reference, how the work might have been done better: for example, are there obvious experiments that should have been tried, or plausible counter-arguments to the claims? Keep in mind that your understanding of other work helps examiners to judge your abilities as a researcher.
There is no need for these drafts of your literature review to be polished—in all likelihood, no-one but you will ever read these early versions—so think of the writing as a letter to your future self.
That is, at this stage you should focus on organization and content rather than on presentation. The rewriting, editing, and polishing that produce the final literature review will probably be done in a focused way only once your research is complete.
As your review proceeds, it will become easier to decide whether to include each of the papers you read. Many obvious factors will guide your decisions: how close some other work is to yours, or how influential it has been. Some factors may be more subtle.
For example, you may find a survey paper or a recent paper with a thorough literature review of its own, that means that many of the older papers do not need to be discussed;
Some papers that initially seemed important might on reflection seem less relevant, and can be set aside or noted in passing; while a paper that at first seemed too theoretical or abstract may on further investigation be revealed as foundational.
I suggest that in early drafts you be as inclusive as possible. When you do remove discussion of a paper, put the discussion in another file (or comment it out) rather than deleting it altogether, as this text is your record of having read the paper.
Authors, Editors, and Referees
When an author completes a paper, it is submitted to the editor of a journal (or the program chair of a conference) for publication. The editor sends the paper to referees, who evaluate the paper and return assessments.
The editor then uses these assessments to decide whether the paper should be accepted, or, in the case of a journal paper, whether further reviewing or revision is required.
Authors are expected to be honest, ethical, and careful in their preparation of papers. It is ultimately the responsibility of the author—not of the journal, the editor, or the referees—to ensure that the contents of a paper are correct.
It is also the author’s responsibility to ensure that the presentation is at an appropriate standard and that it is their own work unless otherwise stated.
Referees should be fair and objective, maintain confidentiality, and avoid conflict of interest. In addition, they should complete reviews promptly, declare their limitations as referees, take proper care in evaluating the paper, and only recommend acceptance when they are confident that the paper is of an adequate standard.
Although referees can generally assume that authors have behaved ethically, many weak or flawed papers are submitted, and a disproportionate amount of reviewing is spent on such papers—in part because they are often resubmitted after rejection.
The editor’s responsibilities are to choose referees appropriately, ensure that the reviewing is completed promptly and to an adequate standard, arbitrate when the referees’ evaluations differ or when the authors argue that a referee’s evaluation is incorrect, and use the reviews to decide whether the paper should be accepted.
These participants have very different perspectives. At submission, an author may feel that the work is remarkable and perfected, and is likely to be sensitive to criticism.
Even researchers who are not working in the environment of a first-rate research center, or who have never had guidance from a more experienced researcher, may well be convinced that their papers are excellent and that negative comments are misguided.
A consequence is that they may seek ways to address issues raised by referees by making only minimal changes, and need to be persuaded through the convincing argument that the referees’ views are reasonable.
In contrast, the referee and editor are likely to feel unexcited (the majority of submissions are rejected) and disbelieving—in submissions, strong results are usually wrong.
And they too may be sensitive, in this case to typical, frustrating faults, and also to undue criticism of their own work in the author’s literature review. In a sense, the reviewing process can be seen as a mechanism for reconciling these different points of view.
The contribution is the main criterion for judging a paper. In broad terms, a paper is a contribution if it has two properties: originality and validity.
The originality of a paper is the degree to which the ideas presented are significant, new, and interesting. Most papers are to some degree extensions or variations of previously published work; really groundbreaking ideas are rare. Nonetheless, interesting or important ideas are more valuable than trivial increments to existing work.
Deciding whether there is sufficient originality to warrant publication is the main task of the referee. Only a truly excellent presentation, thorough and written well, can save a paper with marginal new ideas, while a revolutionary paper must be appalling in some respect to be rejected.
When evaluating the significance of a contribution, it is helpful to consider its effect or impact: that is, to judge how much change would follow from the paper being published and widely read.
If the only likely effect is passing interest from a few specialists in the area, the paper is minor. If, on the other hand, the likely effect is a widespread change of practice or a flow of interesting new results from other researchers, the paper is indeed groundbreaking.
That some ideas appear obvious does not detract from their originality. Many excellent ideas are obvious in retrospect. Moreover, the ideas in a well-presented paper often seem less sophisticated than those in a poorly presented paper, simply because authors of the former have a better knack for an explanation. Obviousness is not grounds for rejecting a paper.
The real achievement may have been to ask the right question in the first place or to ask it in the right way, that is, to notice that the problem even existed. Organization of existing ideas in a new way or within an alternative framework can also be an original contribution, as can reevaluation of existing ideas or methods.
The validity of a paper is the degree to which the ideas have been shown to be sound. A paper that does no more than a claim from the intuition that the proposal should hold is not valid.
Good science requires a demonstration of correctness, in a form that allows verification by other scientists. such a demonstration is usually by proof or analysis, modeling, simulation, or experiment, or preferably several of these methods together, and is likely to involve some kind of comparison to existing ideas.
In the area of algorithms, proof and analysis are the accepted means of showing that a proposal is worthwhile. The use of theory and mathematical analysis is one of the cornerstones of computer science: computer technology is ephemeral but theoretical results are timeless.
Their very durability, however, creates a need for certainty: an untrustworthy analysis is not valuable.
Evaluation of Papers
The process of evaluating a paper involves asking critical questions such as those listed under “Critical reading”. In addition, there are further questions that should be asked of a paper that is under review, which should not just be correct but should be suitable for the likely audience.
Is the contribution timely or only of historical interest? Is the topic relevant to the venue’s typical readership?
What is missing? What would complete the presentation? Is any of the material unnecessary?
How broad is the likely readership?
Can the paper be understood? Is it clearly written? Is the presentation at an adequate standard?
Does the content justify the length?
Of these, the contribution is the single most significant component and requires a value judgment. The presence of a critical analysis is also important: the authors should correctly identify the strengths, weaknesses, and implications of their work, and not ignore problems or shortcomings. It is easier to trust results when they are described in a balanced way.
Most papers have an explicit or implicit hypothesis—some assertion that is claimed to be true—and a proposal or innovation.
Try to identify what the hypothesis is: if you can’t identify it, there is probably something wrong, and if you can, it will help you to recognize whether all of the paper is pertinent to the hypothesis and whether the important material is missing.
The quality of a paper can be reflected in its bibliography. For example, how many references are there? This is a crude rule-of-thumb, but often effective. For some research problems, there are only a few relevant papers, but such cases are the exception.
The presence of only a few references may be evidence of bad scholarship. Also, some authors cite a reasonable number of papers without actually citing related literature, thus disguising a core bibliography that is far too short.
If any of the references are by the author, it may be that some of them are redundant. If only a couple of the references are recent, the author doesn’t appear to be familiar with other research.
Similarly, be suspicious of papers with no references to the major journals or conferences in the area. Expect author-provided evidence of novelty and innovation, via the right citations.
Occasionally an author submits a paper that is seriously incomplete. No effort has been made to find relevant literature, or the proofs are only sketched, or it is clear that the paper has never been proofread, or, in an extreme case, the paper does little more than outline the basic idea.
Such authors perhaps want to establish that an idea is theirs, without going to the trouble of demonstrating its correctness, or are simply tired of the work and hope referees will supply details they haven’t bothered to obtain themselves.
Such papers don’t deserve a thorough evaluation. However, don’t be too quick to judge a paper as being in this category.
Referees should make an effort to search for errors that don’t affect the quality of the work but should be corrected before going into print. These include spelling and syntax, written expression, errors in the bibliography.
Whether all concepts and terms have been defined or explained, errors in any formulas or mathematics, and inconsistency in just about anything from variable names to table layout to the formatting of the bibliography.
Some of these kinds of errors may be picked up in the typesetting process if the paper is to appear in a journal, but many of them won’t be. In particular, only a referee is likely to find errors in mathematics.
Such errors can become more serious defects that might make the paper unacceptable. A few typographic errors in the mathematics are to be expected.
For example, but if the subscripts are mixed up or the notation keeps changing case then it is quite likely that the author has not checked the results with sufficient care; it may well be reasonable to reject the paper and expect the author to review it before submitting again.
Similar arguments apply to the presentation: to a certain extent poorly written papers should be accepted (however reluctantly), but real incompetence in the presentation is grounds for rejection because a paper is of no value if it cannot be read. But note that the converse does not apply: excellence in presentation does not justify acceptance.
Occasionally a referee receives a paper that is well written and shows real care in the development of the results, but which does no more than reproduce existing work. Such papers must, regrettably, be rejected.
A difficult issue for some papers is whether to recommend outright rejection or to recommend resubmission after major changes. The latter means that, with no more than a reasonable amount of additional work, the paper could be of acceptable standard.
This recommendation should not be used as a form of “soft reject”, to spare the author’s feelings or some such, while asking for changes that are in practice impossible; eventual acceptance, perhaps after several more rounds of reviewing, is the usual final result of such a recommendation.
If getting the work to an acceptable standard will involve substantial additional research and writing, rejection is appropriate. This verdict can be softened in other ways, such as suggesting that the paper is resubmitted once the problems have been addressed.
As a consequence of the peer review system, active researchers should expect to referee about two to three times as many papers as they submit (or somewhat less if their papers are usually co-authored) and only decline to referee a paper with good reason.
For many papers, there may be no potential referee who is truly an expert in the area, so be prepared to referee even when you are not confident in your judgment of the paper.
Always state your limitations as a referee—that you are unfamiliar with the literature in the area, for example, or were not able to check that certain proofs were correct.
That is, you need to admit your ignorance. Ultimately, a referee should not recommend acceptance if the paper is not of adequate standard in some respect—the onus is on the referee to fully evaluate the paper.
[Note: You can free download the complete Office 365 and Office 2019 com setup Guide for here]
Content of Reviews
Reviewing of papers serves two purposes. The explicit purpose is that it is the mechanism used by editors to decide whether papers should be accepted for publication.
The implicit purpose—equally important, and often overlooked—is that it is a means of sharing expertise between scientists, via comments for the authors.
Reviews usually include other things besides written comments (such as scores on certain criteria, which are used to determine whether the paper should be accepted), but it is the comments that authors find valuable.
The review should make some kind of case about the paper: whether it is of an adequate standard and what its flaws are. That is, it is an analysis of the paper, explaining why
There are two main criteria for measuring referees’ reviews.
When recommending that a paper is accepted, the editor must be persuaded that it is of an adequate standard. Brief, superficial comments with no discussion of the detail of the paper provoke the suspicion that the paper has not been carefully refereed.
A positive review should not just be a summary of the paper; it should contain a clear statement of what you believe the contribution to be.
When recommending that a paper is rejected, a clear explanation of the faults should be provided. It is not reasonable, for example, to simply claim without references and explanation that the work is not original or that it has been done before—why should the author believe such a claim if no evidence is given?
Having gone to considerable lengths to conduct and present their work, few authors will be persuaded to discard it by a couple of dismissive comments, and will instead resubmit elsewhere without making changes.
Is there adequate guidance for the authors?
When recommending that a paper is accepted, referees should describe any changes required to fix residual faults or to improve the paper in any way— technically, stylistically, whatever. If the referee doesn’t suggest such changes, they won’t get made.
When recommending that a paper is rejected, a referee should consider what the authors might do next—how they can proceed from the rejection to good research. There are two cases. One is that the paper has some worthwhile core that, with further work, will be acceptable.
A referee should highlight that core and explain at least in general terms how the authors should alter and improve their work. The other case is that nothing of the work is worthwhile, in which event the referee should explain to the author how to come to the same conclusion.
Sometimes the referee just cannot tell whether there is worthwhile material because of defects in the presentation. It is helpful to explain to the authors how they might judge the significance of their work for themselves by, for example, sketching questions the authors should consider.
There are many reasons why these criteria should be observed. The scientific community prides itself on its spirit of collaboration, and it is in that spirit that referees should help others to improve their work.
Poor reviews, although saving the referee effort, make more work for the research community as a whole: if a paper’s shortcomings are not adequately explained, they will still be present if the paper is resubmitted.
Most of all, poor reviewing is self-reinforcing and is bad for scientific standards. It creates a culture of lackluster checking of other people’s work and ultimately saps confidence in published research.
In a review recommending acceptance, there is no further chance to correct mistakes—the referee is the last expert who will carefully examine the paper prior to its going into print.
As noted earlier, only obvious errors such as spelling and punctuation may be caught later, and the referee should check that the paper is substantially correct:
No obvious mathematical errors, no logical errors in proofs, no improbable experimental results, no problems in the bibliography, no bogus or inflated claims, and no serious omission of vital information or inclusion of irrelevant text.
In reviews that recommend rejection or substantial revision, such fine-grain checking is not as important, since (presumably) the paper contains gross errors of some kind. Nonetheless, some level of care is essential, if only to prevent a cycle of correction and resubmission with only a few points addressed each time.
Specific, clear guidance on improving the paper is always welcome. But don’t slip into doing the research for the author. If the work is inept, step back; if the work is strong, your contribution isn’t needed; if simple changes will make a real difference, suggest them, but it is the author’s job to take them to completion.
Drafting a Review
First impressions of papers can be misleading. My reviewing process is to read the paper and make marginal notes, then decide whether the paper should be accepted, then write the comments to the authors. But often, even in that last stage, my opinion of a paper changes, sometimes dramatically.
Perhaps what seemed a minor problem is revealed as a major defect, or perhaps the depth of the paper becomes more evident so that it has greater significance than had seemed to be the case. The lesson is that referees should always be prepared to change their minds, and not commit too soon to a particular point of view.
Another lesson is that positives are as important as negatives: reviews should be constructive. For example, in the reviewing process, it is sometimes possible to strengthen the paper anonymously on behalf of the author.
The reviewing process can all too easily consist of fault-finding, but it is valuable for authors to learn which aspects of their papers are good as well as which aspects are bad. The good aspects will form the basis of any reworking of the material and should thus be highlighted in the review.
Some referees construct flaws in papers where none exist. For example, an assessment may include generic statements that could be made almost regardless of relevance to the paper’s topic, such as “the authors have not considered parallel architectures” on a paper about document processing.
Other examples are vague complaints such as “the problem could have been investigated more deeply” or “aspects of the problem were not considered”. Comments of this kind suggest that the referee is not concerned with making a fair evaluation. If there is a genuine problem, then describe it, preferably with examples; otherwise, say nothing.
Referees should offer obvious or essential references that have been overlooked but should not send authors hunting for papers unnecessarily, especially if they are hard to find.
A referee who recommends acceptance requires at least a passing familiarity with the literature—enough to have reasonable confidence that the work is new and to recommend references as required.
Referees need to be polite. It can be tempting to break this rule (particularly when evaluating an especially frustrating or ill-considered paper) and be patronizing, sarcastic, or downright insulting, but such comments are not acceptable.
Some review processes allow for confidential remarks that are not seen by the author. You can use these remarks to emphasize particular aspects of your review or, if the editor requested a score rather than a recommendation to accept or reject, to state explicitly whether the paper should be accepted.
You can also use this space to tell the editor about your own limitations. However, since authors have no opportunity to defend themselves against comments they cannot see, it is not appropriate to make criticisms in addition to those visible to the author.
Checking Your Review
When you recommend that a paper be accepted, you should: Convince yourself that it has no serious defects. Convince the editor that it is of an acceptable standard, by explaining why it is original, valid, and clear.
List the changes, major and minor, that should be made before it appears in print, and where possible help the author by indicating not just what to change but what to change it to.
But if there are excessive numbers of errors of some kind, you may instead want to give a few examples and recommend that the paper be proofread. Take reasonable care in checking details such as mathematics, formulas, and the bibliography.
When you recommend that a paper be rejected, or recommend that it be resubmitted after major changes, you should Give a clear explanation of the faults and, where possible, discuss how they could be rectified.
WHY RESEARCHERS REVIEW LITERATURE
Literature reviews require the researcher to consider the important scholarly research, a process that serves to frame the research they are preparing to undertake.
Framing research is another way of saying that the researcher is learning about the conversation around the topic of interest, and figuring out where in that conversation the new work will fit.
It is coming to understand what is already known about the topic, what current theories are accepted, what competing theories there may be, and the conditions under which that knowledge was collected.
In framing the research, a review of the literature also helps the researcher narrow the question by identifying overlaps, gaps, and ways questions have been framed already in the literature.
An examination of the theoretical, as well as the research literature, help the researcher identify a specific research purpose, question, or testable hypothesis. In some cases, researchers may choose to replicate questions that have already been asked and try to contribute more information to this conversation.
In a strict replication, researchers try to mimic the original study in all important respects because the purpose of a strict replication is to see if the same types of results as those of the original study will be obtained. If the results of a study cannot be replicated by other researchers, the results of the original study may be viewed as invalid.
Another possibility is to locate an important study and conduct a modified replication. This is a replication with some major modification[s], such as examining a new population or using an improved measurement technique. For instance, in the case of studying the education of bilingual children, a researcher may examine interesting gaps in what previous research studied—say, in which languages the children are bilingual, or the ages of the children, or the educational outcomes being considered—and expand a question in that direction.
For a question of this type, it can be useful to stick fairly closely, both conceptually and methodologically, to prior work in order to be able to more easily compare results and draw conclusions across studies.
Even when replication is not the goal, research questions often build upon or expand existing research. Published research studies typically make specific suggestions for future research in the last section of their research reports, and those planning new research can draw from these suggestions to formulate a good question.
After examining the existing research in a problem area, a researcher may arrive at a creative idea that is not a direct extension of existing research. While there are
Reviewing and Citing Literature
important cases of this in the history of science, they are rare. Thus, it is extremely unlikely that a novice researcher will have such an insight at the beginning of a professional career. Put another way, new research almost always has its origins in existing research.
“New topics” that explore areas of new technology or trending issues are often still directly connected to existing research. Even when it seems there is no literature exactly on point, there will still be relevant literature that discusses the main theories or concepts being used, or that is at a slightly higher level of generality about phenomena of the same type.
Take as an example the topic “How does social media affect the formation of friendships among children in grades 1 to 6?” Social media is new, and specific types of social media change in popularity fairly rapidly. What literature might you review?
The literature on the formation of friendships would be useful, as would any other literature that deals with technology and the formation of friendships. Among the reasons for reviewing literature are identifying important studies that form natural neighbors to your idea, as well as learning about the theories they have used and findings they have reported.
The questions that researchers explored in these studies can help you formulate a research question, as well as identify gaps and overlaps in who has been studied and who is absent from studies.
Because there are many topics on which the results of published research are conflicting, another possibility is to plan a study designed to resolve a conflict. Published reviews of research often point out such conflicts and offer possible explanations for them.
The literature review has many reasons. It frames the research, specifically by connecting the new work to existing research. It allows the researcher to show knowledge of influential work in the field to date, and to argue how the current study furthers prior work or is unique from it—in part answering the important question that every study must answer: so what, or why does this matter?
In addition to framing research, literature reviews provide useful cues for writing a better researchable question. In addition to the content, literature reviews have three additional benefits. First, reviewing the literature helps researchers identify useful measures (also called instruments) that have been used successfully by other researchers.
This may include identifying which standardized educational or psychological tests are commonly used in research on a specific topic or reviewing the surveys that other studies have used to improve one’s own questions or align them for more direct comparison with existing studies. It is also a way to identify seriously flawed measures, which can then be avoided.
Second, a researcher may be able to identify and avoid dead-end topics: ideas for research that may have already been thoroughly investigated and shown to be unsuccessful. An example is a treatment that was tried but failed to produce the desired effects in an experiment.
Third, researchers can get ideas on report structure. Reviewing the literature offers many examples of how to organize and write research reports. By paying careful attention to the style and organization used by previous researchers, researchers can get ideas for how to best structure their own writing.
Literature reviews can stand alone as articles, and some types of journals or annual reviews are dedicated to publishing them. They help newcomers or non-specialists by offering detailed overviews on specific topics, summarizing the history of research on each topic, as well as its current status and controversies.
In addition, standalone literature review articles often provide specific details about how the literature was reviewed—for instance, such a review would describe what journals or databases were searched, for which years, and with what key terms. It would report the number of results returned and explain how those results were then sorted.
These aspects of the standalone literature review serve a purpose similar to documenting how empirical research is carried out—they provide transparency in the methods used to allow the reader to assess the value, comprehensiveness, and limitations of the reported results.
Most often, literature reviews are a part of introducing empirical research. A well-crafted literature review will show the readers the context within which the research was conducted.
For students conducting research, writing reviews of research allows them to demonstrate to instructors, thesis committee members, and others that they are aware of and conversant in the research that is relevant to their study. In the next topic, methods for searching for literature electronically are explored.
Have you begun reading the literature on your problem area? If yes, which description best fits your research approach at present (strict replication, modified replication, newly combined area, resolving a conflict)? How might this shape your review of the literature?
Explain. Was your research purpose or hypothesis explicitly suggested in the literature you have read? Explain.
LOCATING LITERATURE IN ELECTRONIC DATABASES
Most original reporting of empirical research appears as articles published in scholarly journals, and journals are most easily found in databases. This topic explores how to locate journal articles using online research databases. Most researchers and students access journal content through the databases to which their libraries subscribe.
These databases are curated collections of published, scholarly, peer-reviewed work that can be easily sorted and filtered to find specific articles. Without a database subscription, the content is behind a paywall, although there are exceptions. Limited content such as abstracts or excerpts of content is often available for free online.
Database searches work a bit differently than searching the web. Web searches are more flexible, but because they are not limited to scholarly work (unless you select to search from a site that aggregates only scholarly work, such as Google Scholar), finding research articles among the search results can be challenging.
Even if searches in web browsers pull a list of scholarly articles, clicking on a link often leads to a paywall instead of an article, unless the person holds a subscription or is logged into a library that subscribes to that resource.
Databases are curated to a specific set of sources, such as a group of journals, but sometimes contain other items such as newspaper articles, book abstracts, or government documents. Four of the most commonly used databases in the social and behavioral sciences are:
(1) Sociological Abstracts, which covers journal articles published in more than 1,800 journals;
(2) PsycINFO, which contains more than 2.6 million citations;
(3) PsycAR-ARTICLES, which includes over 25,000 searchable full-text journal articles dating as far back as 1894; and
(4) the already mentioned ERIC, which contains abstracts of articles in education found in more than 1,165 journals dating from 1966 to the present. Not all databases are limited to peer-reviewed, scholarly journals. For instance, Sociological Abstracts includes dissertations, PsycINFO includes books, and ERIC includes unpublished documents, such as conference papers.
These are just a few of the many additional databases out there. College libraries usually list their collection on a webpage and often group databases by subject area to help students identify the most relevant listings. Some important electronic databases are described in Appendix A.
Whichever database is used, it is important to first read carefully the instructions for using the particular database being accessed. All databases allow searches based on published information and keywords, but the entry of search information differs, and databases offer different special features to assist users.
Locating literature electronically requires both technique and technical knowledge. First, technique. The main technique to develop is the ability to identify the right databases and search terms or combinations of terms in order to find the most relevant literature.
To generate a useful list of terms, think through the specific parameters of your research, paying special attention to the specific terminology associated with the topic you are studying and the concepts important to your research questions.
For instance, if you are studying bilingual education, you may not only want to look up “bilingual,” but also the specific aspect you are interested in. What outcomes are you reviewing?
Are they related to specific theories of development or intelligence? Identify words or phrases that relate to the research. Consider synonyms for the words or phrases that you have identified. These terms and combinations can help you expand or narrow your search.
Now, the technical. Searching databases work best when you know some basic rules. Databases are designed to help you find what you are looking for. For instance, articles are tagged with standardized subject terms or keywords in order to help searches pull the most relevant results, even when articles use different terminology.
For example, if you look up “adaptive learning,” you may find that the subject listed is actually “adaptive testing.”
After verifying that the two concepts are discussing the same thing, you may find that using this standardized subject term yields much better results. Subject words are usually listed along the side of the results, in the preview, or in the article record, which you see when you click on the article title. You can also narrow results by using advanced search options to limit dates or language.
While databases are ultimately needed, a literature search can be started using a search engine such as Google Scholar. While complete versions of articles may not be accessible, abstracts can be viewed.
Abstracts appear at the beginning of articles and offer a very short, 50- to 300-word summary of the article’s contents: the research question, what information was collected, and the key findings.
Abstracts provide enough information to determine if an article is relevant, and can also help to identify good search terms. To use an article in a literature review, more than the abstract is needed, but relevant articles can be identified and the information can be used to look them up in the library’s databases.
Another helpful way to identify relevant articles is to locate annual review journals, sometimes called “annals,” within your discipline. Annual review articles provide a comprehensive review of the primary research done on specific themes and topics. These articles present an overarching frame, identifying trends from specific eras and presenting the camps of differing opinion on an issue.
Name some keywords you might use in an electronic search of the literature. Try referring to a thesaurus, specialized encyclopedia, or the standardized database terms for your keywords. Now located at least three databases you plan to access and find the contact info of the research librarian you can call or email with questions.
STRUCTURING THE LITERATURE REVIEW
A literature review’s main purpose is to summarize the existing scholarship. It is a standard part of a research report whose role is to introduce themes and topics in previously published articles and books that orient the reader to the conversation about the topic. Often, it presents persuasive information to help make a case about the importance of the topic and research.
Literature reviews start most writing that reports original research, including journal articles, theses, dissertations, and sometimes student papers. When the literature review is a part of a larger work, it comprises about 20% of that piece of work. It may be only a few paragraphs in a shorter length article or consist of a chapter in most dissertations.
As mentioned in Topic 12, literature reviews can be published as standalone articles, but the most common type of literature review is the one found at the outset of an empirical article. We will focus on this example in this topic.
In empirical articles, the literature review often appears at the beginning and may flow directly from the introductory paragraph or follow directly after an introduction. It may or may not be labeled with a heading, depending on journal style and length of the article.
It provides context for the presentation of original research, followed by sections about the study’s hypotheses, methods, and findings, and a concluding section that discusses the broader implications of the findings, as well as their limitations.
A straightforward recipe for starting a literature review is to define key terms and establish the importance of the research topic. There are a few tried and true ways to establish a topic’s importance.
One is to provide a compelling key statistic. Example 1 is the first paragraph of an original research article on fear of cancer recurrence among breast cancer survivors, taken from the journal Health Psychology.
An example comes from the literature review in a peer-reviewed article on cyberbullying. This article had a separate introduction and literature review. The introduction included definitions of bullying and cyberbullying in making the claim (with citations) that bullying is not new, but cyberbullying is on the rise. This excerpt shows the first sentences from the Literature Review section:
Prior to 2002, no research existed on the topic of cyber-bullying. However, researchers have found that two-thirds of students reported that cyber-bullying is just as serious as traditional bullying, if not worse, as people often feel shielded from the ramifications of their actions and, therefore, may state things they would not ordinarily say in person.
Further research conducted on 124 middle school students found that 32% of its participants felt that cyber-bullying was a problem in their school. Interestingly, research suggests that the highest frequency of cyber-bullying is found in public schools, followed by all-girl private schools, with the least cyber-bullying incidents occurring at private charter schools.
In this example, you can see some commonly used elements of the literature review in action. The report claims that there was little research prior to 2002. Statistics claim that many students are affected, and establish that the issue is important to students.
There is also a fact of interest (where bullying most frequently occurs) that might surprise or contradict what someone expects about the trend in cyberbullying. All claims presented are tied to sources, and all relate directly to the specific areas that the research in the report will cover.
CONNECTING THE LITERATURE TO YOUR STUDY
This topic continues to build a literature review by considering how to structure the literature review and connect it to your research.
Author and researcher Gary Klein conducted many research projects on differences between novices and experts across many settings. He found that experts and novices differ in knowing what to pay attention to.
Because novices do not yet know what information is most useful, they often try to pay attention to too many things and do not give more weight to the cues that are most important.
Experts—whether in nursing, firefighting, or playing chess—know what information to pay close attention to and just as importantly, what information to ignore. This ability to filter the environment for important cues makes their performances superior to beginners. What does this have to do with the literature review?
Reading academic literature to prepare a literature review involves filtering through many sources. A part of the job is knowing how to filter articles for relevant information to your topic.
Connecting the literature to your study involves two tasks, and researchers are likely to jump back and forth between them as they learn more from the literature and develop a more targeted research question. The first task is to come up with an outline of topics that the literature review should include.
Depending on the length of the article, these may form the headings and subheadings in the final literature review. Even if there are no headings, these may define the focus of each paragraph. Let’s say your research is on cyberbullying prevention among middle school children.
It can be very helpful to review the major and minor subheadings in original research articles and literature reviews to see what topics others have treated as important. For instance, a standalone literature review by Notar and colleagues (2013) divided their paper into the topics shown in Example 1.
Definition [of Cyberbullying]
Nuances of Electronic Communication
Reasons for Cyberbullying
Roles in Cyberbullying
Gender Differences in Response to Cyberbullying
Is cyberbullying largely a problem for girls?
Traditional v. cyber[bullying]
The second task is to scrutinize the elements of the research question and the research plan that is taking shape. What will the reader need to know about the state of research on your project in order to grasp your research and its findings?
For instance, in the case of cyberbullying prevention among middle school children, the literature review would likely want to include information on what is currently known about preventing cyberbullying. What prevention strategies have been tried?
Has research studied their success? If so, were they successful? It is important to stay on the topic at hand. If there are two types of prevention strategies that are common, and the current research only deals with one, it is worth noting that there are two types, but the literature review need not discuss research on the second strategy in any depth.
Consider ways to take notes from articles in your literature review that will help you accomplish the two tasks above. What would information about each article you need in your notes in order to be able to combine findings and summarize them by topic or theme? What details might you want to have on hand about the design of each study to compare or critique the research design?
Name some of the headings and subheadings you anticipate using to organize your literature review.
PREPARING TO WRITE A CRITICAL REVIEW
A literature review should be a critical assessment of the literature on a topic. Novice writers often make two common mistakes that lead to uncritical reviews.
First, they often take the results of each study to be “facts” that have been proven. As indicated below, all studies should be presumed flawed and, therefore, to offer only degrees of evidence, not “truths” to be accepted uncritically.
Second, novice writers often discuss all studies as though they were equal in quality when some studies are methodologically superior to others.
To prepare to write a critical review, the writer should assess the quality of each research article that will be cited. The first important area that requires critical assessment is sampling. More often than not, researchers work with samples that are less than ideal (such as volunteers instead of random samples of the populations of interest).
Weaknesses in sampling limit the generalizability of the results. For instance, the results of a study on learning theory using a sample of psychology students who volunteered to serve as participants would have limited generalizability to nonvolunteers.
The second important area that requires critical assessment is measurement (sometimes referred to as instrumentation). It is safe to presume that all measures are flawed to some extent. Furthermore, it is safe to presume that various methods of measuring a given variable might lead to somewhat different results.
For instance, one researcher on nutrition might measure by asking overweight adolescents to keep diaries of what they eat throughout a day. Fearing that participants might not be completely truthful in diaries, another researcher on nutrition might observe the food choices of overweight adolescents in the school cafeteria.
The second method has the limitation of being conducted in a highly structured environment, where the food choices might be limited. Neither method is right or wrong. Instead, both methods have limitations, which should be considered when critically assessing the results of the studies.
The third important area to consider when critically assessing studies to cite in a literature review relates to issues with validity, which, as discussed, applies especially to experiments (i.e., studies in which treatments are administered to participants so that one can estimate the effects of the treatments on the participants’ subsequent behaviors).
Experiments are often flawed by having inappropriate control conditions (such as comparing a treatment group consisting of volunteers with a control group from the general population).
Also, laboratory experiments (e.g., receiving rewards for making appropriate food choices in a laboratory setting) may have limited generalizability to receiving rewards in a natural environment (i.e., the rewards may have effects in the natural environment different from those in the laboratory setting).
Fortunately, researchers often discuss the limitations of their studies, usually in the last section of their research reports. Examining researchers’ self-assessments of the limitations of their research can help writers of literature reviews when citing the studies. Example 1 shows a portion of such a discussion of limitations.
However, this study does have limitations, including the small sample size and the nonrandomized controlled design. Consequently, observed changes in study endpoints may be explained by the natural course of postoperative recovery. Also, no biochemical verification was used to confirm the self-report of smoking status (p. 185).
Because flaws are widespread in research, it is important to assess the seriousness of the flaws and use appropriate terms when describing the results. For instance, Example 2 shows some terms that might be used to describe the results of research with serious weaknesses.
Doe’s (2013) study provides some evidence that. . .
Recent research by Doe (2013) raises the possibility that. . .
Preliminary evidence suggests that. . . (Doe, 2013).
Of course, some studies provide strong evidence, such as a national survey using a large, representative sample. Sometimes, there is a series of studies on the same topic, all of which have similar results, making the overall findings of the series highly reliable. Terms that might be used to refer to strong evidence are shown in Example 3.
Overall, this set of studies clearly indicates that. . .
In summary, the five studies described providing nearly conclusive evidence that. . .
Doe’s (2013) national survey demonstrates that. . .
It is not necessary to use terms such as those in Examples 2 and 3 to qualify every comment about each study cited in a literature review. However, keep in mind that when a writer presents a finding or statement from the literature without qualification, readers are likely to assume that the writer believes that the underlying methodology and logic are reasonably sound.
According to the topic, novice writers often make two common mistakes. What is the first one that is mentioned in this topic?
“More often than not, researchers work with samples that are less than ideal.” According to this topic, is this statement true or false?
“It is safe to assume that flawed measures are rare in research.” According to this topic, is this statement true or false?
Do researchers often discuss the limitations of their own studies?
When a writer presents a finding or statement from the literature without qualification, readers are likely to assume what?
CREATING A SYNTHESIS
As indicated in the previous topic, a literature review should be based on a critical assessment of the previously published work on a topic. However, it should consist of more than just critical summaries of individual studies.
Instead, a literature review should be a synthesis—providing a whole picture of what is known and what is not known, as well as an attempt to show how diverse pieces of information fit together and make sense.
A key to creating a synthesis is to write a review that moves from subtopic to subtopic (not from one study to another), while citing whatever studies are relevant for each topic.
To further the creation of a synthesis, the writer of a review should explicitly point out major trends and commonalities in the results of previous research. Example 1 illustrates how this article summarized studies in one paragraph of the literature review under the heading “Maternal Depression and Children’s Health.”
There are several examinations linking maternal depression and child health. Angel and Worobey, in their cross-sectional study of Mexican children in the His-panic Health and Nutrition Examination Survey, found that maternal depression is one of the largest predictors of poor health among children ages six months to 11 years.
Regional studies come to similar conclusions. One study finds that maternal depression is associated with fair or poor health among children under the age of three, though this study uses a cross-sectional, convenience sample of children visiting hospital clinics or emergency departments.
Evidence from a sample of 150 mostly white, middle- to upper-class children with a family history of asthma shows that asthma diagnoses by age eight are higher in children of depressed mothers.
Greater depressive symptoms in mothers are associated with more visits to hospital emergency departments, more hospitalizations, and fewer well-child visits.
In addition, when there are major discrepancies in results, they should be pointed out, and possible explanations for the discrepancies should be noted. Example 2 points out an important difference between the samples used in the studies.
While the studies described previously support the prediction that X is greater than Y in samples of college students, a recent study of young adolescents found no difference in X and Y. It may be that X and Y operate differently within different age groups.
A whole picture of the literature on a research topic should also explicitly point out gaps in what is known, as illustrated in Example 3. Note that the purpose of research is often to fill a gap found in the literature. Example 3 completes the paragraph started in Example 1, pointing out a limitation of the research:
Thus, though there is evidence that maternal depression may lead to health problems in children, the existing literature is often based on small, cross-sectional, or homogenous samples of children and does not examine the mechanisms linking maternal depression and children’s health.
To facilitate providing readers with an overview, writers of reviews should avoid giving extensive, detailed descriptions of the research methodology used in each study.
Notice how, in few words, the writer in Example 1 was able to provide detail on findings and methods within the review of the literature to arrive at the point made in Example 3.
It is usually not necessary to provide in-depth details on how studies were conducted. The main exceptions are when the writer of the review wants to document a particular weakness in a study being cited (e.g., use of a small, unrepresentative sample) or when the details of the methodology might help to explain why two studies on the same topic arrived at substantially different results.
In addition, one or more paragraphs (or even whole sections) might be devoted to describing a particular author’s written work when the work is central to one or more points being made in the literature review.
It can be very useful to keep a spreadsheet of studies with main topics, sample sizes, methodological details, and main findings in order to capture and compare the literature and ensure that the notes from original research articles have concentrated on the right information to collect in order to compose a literature review.
It can also be useful to collect or paraphrase the points you may like to include in your own work from the findings of the article, along with the page number.
Because direct quotations break the flow of a presentation, they should be used very sparingly.
Quotations normally should be used only for (1) presenting especially well-crafted definitions of key terms, (2) presenting concepts that are explained especially well with a particular set of words, and (3) clarifying differences of opinion in the literature when seeing that differences in wording (such as how theories are presented) might help readers to understand the issues involved.
The flow of presentation can be facilitated through the use of appropriate transitional terms and phrases (e.g., however, as a consequence, and indeed) both within and at the beginning of paragraphs. Finally, a brief summary placed at the end of a review can help readers to grasp the whole of the literature review.
Have you started writing a literature review on your topic? If yes, what types of changes will you make to it based on what you learned in this topic? If not, how do you plan to approach it after reading this topic?
WHY ACADEMICS USE CITATION
Academic writing is a conversation in progress, where scientific theories and knowledge are being built, corroborated, debated, and debunked, often based on new empirical evidence or fresh analysis that has been added to the conversation.
All claims that are made in an academic article must be backed up with evidence, and that evidence comes in two types: from the literature, or from your own data. The literature review is made up of the first type.
It relies on things other people have said in the literature before turning to the data as evidence for claims. In both cases, the writer must connect the claims made to the evidence that supports it. The primary way this is done when claims are based on other sources is to acknowledge the source through citation.
The citation is the system that academic writing uses to acknowledge sources. It typically consists of two parts. First, the author provides a shorthand version of the source information within the flow of the writing.
This may be incorporated into the sentence itself by referring to the author or study directly, or more commonly, it appears in parentheses or as a footnote following the information from the source.
Second, the bibliography, which appears at the end, allows any reader to connect the shorthand information in the citation to the complete source information.
For this reason, entries in the bibliography must include complete, specific information about a source, including its author names, year, title, publication name, and publisher information.
Both steps are required for nearly all types of citation. Placing a source in the bibliography is rarely enough—the citation also indicates where ideas from a specific work appear in the text.
Each citation in the text should match to a corresponding entry in the bibliography. Occasionally, and depending on the style used, authors list works in the bibliography that is not directly cited in the text because the work informed or influenced the overall discussion but no specific concept or quote from the work was incorporated into the text.
Sources are cited when information from them is used. This is true regardless of whether the source information is quoted directly or paraphrased. To not attribute the idea to the source from which it was borrowed is plagiarism. This is a form of theft. Not giving credit where it is due is the same as claiming the ideas are original to the author.
If sources are not identified, how can readers distinguish between ideas that are the author’s and ideas the author borrowed from others? The citation is also an essential ingredient in intellectual integrity and academic honesty because it distinguishes the author’s original ideas and contributions from those that he or she is including, responding to, or building upon.
Different academic disciplines use different styles of citation, and each has a style guide that serves as a rule book for the style. You will need to learn the rules for the style that is used for your discipline.
These technical differences are discussed in more detail in Topic 19. Regardless of which citation style is used, the important general rules of when to cite apply across all disciplines.
When you take the exact words from a source, even just a phrase, the words are to be placed in quotation marks. Direct quotes typically require more precise identification of the source.
For instance, some styles require a page number. When quotations take up more than four lines (Chicago) or 40 words (APA and ASA style), they are set off from the text in an indented block, and no quotation marks are used.
It is best to use long quotations only sparingly and only when there is a clear benefit or aesthetic reason for keeping the complete original wording of a passage.
If there is not, it may be better to paraphrase as much as possible and only keep what is unique to the source and necessary to quote. Sometimes, even a single word, if it is key and unique to the text, should be quoted, but typically a quote is a full phrase. Check the rules in your discipline for any specific guidelines on handling quotations.
Paraphrasing is when you restate the idea or information from a source in your own words. These also must be attributed to the source from which the idea was borrowed. The important thing for a citation is that the idea was someone else’s intellectual contribution.
The use of paraphrase allows the writer to recombine ideas from multiple sources into paragraphs that emphasize or synthesize specific points in a way that extensive quoting does not allow. Paraphrasing also allows the author to maintain a consistent tone or style of presentation while presenting information from sources that sound quite different.
Sometimes, paraphrasing can also restate an idea in shorter or simpler terms, or it can unpack an idea to elaborate further on the meaning. For these reasons, good writers paraphrase when possible. Paraphrasing absolutely requires acknowledgment of the source through citation.
Let’s look at an example. In the first passage, an assertion is made without attributing sources. In the second, attribution is added. Look for the key differences in how the information is presented.
People who live in poor neighborhoods with disorderly or degraded physical environments are more likely to show signs and symptoms of depression, and their children are more likely to exhibit problem behavior. When families in high poverty neighborhoods move to better neighborhoods, these symptoms improve.
Example 1 is not a quote but takes ideas from a 2007 article that is directly quoted in Example 2. This is from the literature review of an empirical article on depression and the urban built environment.
Recent studies have shown that neighborhood social disorganization is associated with depressive symptoms and that living in socioeconomically deprived areas is associated with depression, with higher levels of child problem behavior, and with a higher incidence of non-psychotic disorders.
A randomized controlled trial that moved families from high poverty neighborhoods to non-poor neighborhoods showed that both parents and children who moved reported fewer psychological distress symptoms than did control families who did not move.
Example 1 makes the statements seem like the author’s claims to fact, whereas Example 2 makes it clear that the authors are summarizing ideas from the work of others.
Facts and data also require that a source is acknowledged, unless the information is so general and well known that is it considered common knowledge and need not be verified through a source.
Examples of common knowledge include dictionary definitions and statements such as “The United States is in North America” and “Barack Obama was President of the United States.” However, in many statements of fact, there is a gray area because some knowledge that is “common” for one group is not commonly known by others.
If there is doubt, it is better to cite a source that corroborates your information than to assume everyone knows it. Most statistical information, while often commonly available, is tied to a specific time and population.
To determine if a statistic needs a citation, it can help to ask if the fact would still be true in 25 or 50 years. If something corresponds to a specific time period (how many people graduated from college in 2016), it probably needs a source.
Sometimes, citations are a judgment call. The best rule of thumb is: when in doubt, cite. It creates fewer potential problems to cite when you do not need to than to fail to cite when you should have acknowledged the source.
Consider what you have learned in this topic, paying particular attention to the examples above. Consider incorporating a review of your own writing to be sure the presentation is more like Example 2 than Example 1.
INTRODUCTION TO STYLE GUIDES
Organizations as diverse as the American Medical Association (AMA), International Business Machines Corporation (IBM), the Associated Press (AP), and the American Sociological Association (ASA) have published style guides to assist their authors and editors.
Most academic nonfiction books follow the Chicago Manual of Style, so named because it was developed at the University of Chicago. Individual publishers, journals, and magazines also typically have in-house style guides.
Style guides differ from grammar handbooks. While grammar deals with the rules of a specific language, a style guide helps to create consistency in how certain elements and special words, terms, numbers, and abbreviations will be treated. For instance, consider this question: Should academic degrees be capitalized?
The AMA Manual of Style provides this answer: “Capitalize academic degrees when abbreviated but not when written out”. Thus, when writing for the AMA, an author should write the following two sentences as shown here:
Jane Doe earned her master’s degree at Stanford University.
Jane Doe, MA, will be the keynote speaker at this year’s senior seminar.
The two sentences immediately above raise another problematic question: Should periods be used in abbreviations for academic degrees? As you can see, the answer is “no” if you are writing for the AMA.
However, the answer is “yes” if you are writing for a newspaper using The Associated Press Stylebook. Neither style guide is right or wrong, but rather each of the two guides has a different style on this issue.
In fact, the Chicago Manual recently made a change to its style, moving from using periods to no periods in academic degrees.2 What matters is consistency.
A second important function of style guides is to specify the contents and organization of manuscripts. For example, the American Sociological Association’s ASA Style Guide describes the contents of title pages for articles submitted for publication. If you guessed that the title page should contain the names of the authors and their institutions, you would be correct.
However, relying just on your intuition, you might fail to include other required elements: the word count of the document including footnotes and references and a footnote with contact information for the author who will respond to any correspondence, plus any acknowledgments, credits, or ID numbers for grants or funds used to support a study (although for student papers, these are typically not needed).
Third, style guides address issues of citation. They describe the exact format in which the citations should be listed within the text as well as how they should appear in the bibliography.
There are two common citation styles. The first uses a superscript number at the reference point in the text that is connected to the same number at the foot of the page or in the endnotes where the source information appears.
The second uses author last name and date of publication in parentheses at the end of the sentence that refers to the source’s information. Either approach will also include a page number when the information is directly quoted from the original source.
The American Sociological Association’s ASA Style Guide and the Publication Manual of the American Psychological Association (APA Manual) are two styles that are commonly found in social science disciplines.
Each is published by the major association for professionals and academics working in those fields in the United States. The APA Manual covers the same general topics as the ASA Style Guide, but in greater detail. There are important similarities between the two styles.
Both styles use the parenthetical citation style, meaning that sources are identified by author last name and date of publication at the end of a sentence, and include a page number if there is a direct quote. The styles do differ in small ways. For instance, ASA style puts no comma after the author names but APA style does.
When a page number is used, ASA puts a colon, then the page number with no space, but APA style uses a comma. ASA style lists the page number with no abbreviation to indicate the word “pages,” but APA does include an abbreviation (“p.” or “pp.”) when a page number is included. Example 1 shows the same citation used in APA or ASA style with these small differences.
According to multiple sources, students often experience difficulty remembering small differences between styles. The top recommendation is “acquiring a style manual”.
According to multiple sources, students often experience difficulty remembering small differences between styles. The top recommendation is “acquiring a style manual”.
While these differences in style are sometimes slight and do not greatly interfere with readers understanding citation information, consistency of style, including its use of punctuation, can be meaningful across the span of the document, including its relationship to the bibliography.
For this reason, it is important to be consistent in adhering to the style you use. Basic rules and resources for using APA and ASA styles are described in Appendices E and F, respectively.
There are now many programs called “reference managers” or “citation managers” that automate much of the tedious citation and bibliography formatting that academic writing requires.
Examples include Zotero, Endnote, RefWorks, and Mendeley. Some of these programs are web-based; some are installed as software on a computer.
Some are free, and others cost money. University libraries may have an institutional license that allows students an account with a specific reference management solution.
Most electronic research resources now allow users to download article citation information directly into these programs, and some support PDF attachment, sorting and filtering, and notes fields where article notes can be organized.
These programs typically integrate with standard word processing programs to format citations and bibliographic entries properly. Using a reference citation manager can help writers implement style guides appropriately when it comes to citations and bibliography entries, but they will not implement the capitalization or abbreviation rules for a specific style.
Which style guide should you use while you write your research proposal and research report? If it is APA or ASA, consider reviewing Appendices E and F for basic information.
If not, look up and review the basic requirements of the style you are expected to use. Does it affect the type of information you might need to track in your notes about sources?
Consider how you might use a reference citation manager. If you are a student or affiliated with a college or university, check with your library to see if they have an institutional membership with a specific program that you can use.
Look up some options and see whether a citation manager works for you. Whether you use a citation manager or not, make a plan for how to track citation information to make it easy to cite when you write from your notes and quotes.