17.2 Evaluating Your Sources

Article links:

CRAAP Test: “CRAAP Test” provided by UTA Libraries

“Secondary Sources in Their Natural Habitats” provided by Lumen Learning

“Listening to Sources, Talking to Sources” provided by Lumen Learning

“Understanding Bias” provided by Lumen Learning

“Questions to Evaluate the Authority of the Researcher’s Methods” by Joe Moxley

Chapter Preview

  • List the reasons educators are fond of research papers.
  • Identify three categories that academic books fall into.
  • Recall the uses for Google Scholar.
  • Descibe the They Say/I Say process.

CRAAP Test: CRAAP Test

provided by UTA Libraries

Click CRAAP test image to open.
PDF Document: Evaluating Information – CRAPP Test. Click document image to open.

Secondary Sources in Their Natural Habitats

provided by Lumen Learning

Ah, The Research Paper

Such exhilaration! Such consternation! Educators are fond of research papers because they require you to find your own sources, confront conflicting evidence, and synthesize diverse information and ideas—all skills required in any professional leadership role. Research papers also allow students to pursue their own topic of interest; your professors have to assume that you are genuinely interested in at least some major part of the course.1 The open-endedness of research papers sets you up to do your best work as a self-motivated scholar.

Research papers are, by far, the best kind of papers! If you have an original twist to an old idea and about five good sources, you pretty much have a research paper. Most of the hard work is done for you already! If I can give you one piece of advice for research papers, it would be to know what you’re looking for in an article. If you want statistics, skim for statistics. Knowing what you want will cut down the time it takes you to find sources.

Kaethe Leonard

This chapter is about secondary sources: what they are, where to find them, and how to choose them.2 Recall the distinction between primary and secondary sources. Primary sources are original documents, data, or images: the law code of the Le Dynasty in Vietnam, the letters of Kurt Vonnegut, data gathered from an experiment on color perception, an interview, or Farm Service Administration photographs from the 1930s.3 Secondary sources are produced by analyzing primary sources. They include news articles, scholarly articles, reviews of films or art exhibitions, documentary films, and other pieces that have some descriptive or analytical purpose. Some things may be primary sources in one context but secondary sources in another. For example, if you’re using news articles to inform an analysis of a historical event, they’re serving as secondary sources. If you’re counting the number of times a particular newspaper reported on different types of events, then the news articles are serving as primary sources because they’re more akin to raw data.

Some Sources Are Better Than Others

You probably know by now that if you cite Wikipedia as an authoritative source, the wrath of your professor shall be visited upon you. Why is it that even the most informative Wikipedia articles are still often considered illegitimate? And what are good sources to use? The table below summarizes types of secondary sources in four tiers. All sources have their legitimate uses, but the top-tier ones are preferable for citation.

Tier Type Content Uses How to find them
1 Peer-reviewed academic publications Rigorous research and analysis Provide strong evidence for claims and references to other high-quality sources Google Scholar, library catalogs, and academic article databases
2 Reports, articles, and books from credible non-academic sources Well researched and even-handed descriptions of an event or state of the world Initial research on events or trends not yet analyzed in the academic literature; may reference important Tier 1 sources Websites of relevant agencies, Google searches using (site: *.gov or site: *.org), academic article databases
3 Short pieces from newspapers or credible websites Simple reporting of events, research findings, or policy changes Often point to useful Tier 2 or Tier 1 sources, may provide a factoid or two not found anywhere else Strategic Google searches or article databases including newspapers and magazines
4 Agenda-driven or uncertain pieces Mostly opinion, varying in thoughtfulness and credibility May represent a particular position within a debate; more often provide keywords and clues about higher quality sources Non-specific Google searches

Tier 1: Peer-reviewed academic publications

These are sources from the mainstream academic literature: books and scholarly articles. Academic books generally fall into three categories: (1) textbooks written with students in mind, (2) monographs which give an extended report on a large research project, and (3) edited volumes in which each chapter is authored by different people. Scholarly articles appear in academic journals, which are published multiple times a year in order to share the latest research findings with scholars in the field. They’re usually sponsored by some academic society. To get published, these articles and books had to earn favorable anonymous evaluations by qualified scholars. Who are the experts writing, reviewing, and editing these scholarly publications? Your professors. I describe this process below. Learning how to read and use these sources is a fundamental part of being a college student.

Tier 2: Reports, articles and books from credible non-academic sources

Some events and trends are too recent to appear in Tier 1 sources. Also, Tier 1 sources tend to be highly specific, and sometimes you need a more general perspective on a topic. Thus, Tier 2 sources can provide quality information that is more accessible to non-academics. There are three main categories. First, official reports from government agencies or major international institutions like the World Bank or the United Nations; these institutions generally have research departments staffed with qualified experts who seek to provide rigorous, even-handed information to decision-makers. Second, feature articles from major newspapers and magazines like the New York TimesWall Street JournalLondon Times, or The Economist are based on original reporting by experienced journalists (not press releases) and are typically 1500+ words in length. Third, there are some great books from non-academic presses that cite their sources; they’re often written by journalists. All three of these sources are generally well researched descriptions of an event or state of the world, undertaken by credentialed experts who generally seek to be even-handed. It is still up to you to judge their credibility. Your instructors and campus librarians can advise you on which sources in this category have the most credibility.

Tier 3. Short pieces from periodicals or credible websites

A step below the well-developed reports and feature articles that make up Tier 2 are the short tidbits that one finds in newspapers and magazines or credible websites. How short is a short news article? Usually, they’re just a couple paragraphs or less, and they’re often reporting on just one thing: an event, an interesting research finding, or a policy change. They don’t take extensive research and analysis to write, and many just summarize a press release written and distributed by an organization or business. They may describe things like corporate mergers, newly discovered diet-health links, or important school-funding legislation. You may want to cite Tier 3 sources in your paper if they provide an important factoid or two that isn’t provided by a higher-tier piece, but if the Tier 3 article describes a particular study or academic expert, your best bet is to find the journal article or book it is reporting on and use that Tier 1 source instead. If the article mentions which journal the study was published in, you can go right to that journal through your library website. Sometimes you can find the original journal article by putting the scholar’s name and some keywords into Google Scholar.

What counts as a credible website in this tier? You may need some guidance from instructors or librarians, but you can learn a lot by examining the person or organization providing the information (look for an “About” link). For example, if the organization is clearly agenda-driven or not up-front about its aims and/or funding sources, then it definitely isn’t something you want to cite as a neutral authority. Also look for signs of expertise. A tidbit about a medical research finding written by someone with a science background carries more weight than the same topic written by a policy analyst. These sources are sometimes uncertain, which is all the more reason to follow the trail to a Tier 1 or Tier 2 source whenever possible.

Personally, research papers are my thing! They give me a chance to further explore a topic that I usually am genuinely interested in, and it gives me the opportunity to write down everything I know. Sources are easy to find; they’re everywhere. Unfortunately, the useful ones you have to put in a little more effort to find. As much as I love Wikipedia, if I’m going to take the time to write a paper, I want it to be taken seriously. There are so many resources out there to help students find scholarly information. The better the source, the more supported your paper will be. But it doesn’t matter how well supported or amazing your paper is if you don’t cite your sources! A citing mistake could definitely get you a big fat zero on the paper you worked so hard on, and maybe even kicked out of school. Utilize resources like www.easybib.com for a quick works cited, and Purdue’s OWL (english.purdue.edu/owl) for a complete and easy explanation on APA and MLA citing formats.

Aly Button

Tier 4. Agenda-driven or pieces from unknown sources

This tier is essentially everything else, including Wikipedia.4 These types of sources—especially Wikipedia—can be hugely helpful in identifying interesting topics, positions within a debate, keywords to search on, and, sometimes, higher-tier sources on the topic. They often play a critically important role in the early part of the research process, but they generally aren’t (and shouldn’t be) cited in the final paper. Throwing some keywords into Google and seeing what you get is a fine way to get started, but don’t stop there. Start a list of the people, organizations, sources, and keywords that seem most relevant to your topic. For example, suppose you’ve been assigned a research paper about the impact of linen production and trade on the ancient world. A quick Google search reveals that (1) linen comes from the flax plant, (2) the scientific name for flax is Linum usitatissimum, (3) Egypt dominated linen production at the height of its empire, and (4) Alex J. Warden published a book about ancient linen trade in 1867. Similarly, you found some useful search terms to try instead of “ancient world” (antiquity, Egyptian empire, ancient Egypt, ancient Mediterranean) and some generalizations for linen (fabric, textiles, or weaving). Now you’ve got a lot to work with as you tap into the library catalog and academic article databases.

Origins And Anatomy Of A Journal Article

Most of the Tier 1 sources available are academic articles, also called scholarly articles, scholarly papers, journal articles, academic papers, or peer-reviewed articles. They all mean the same thing: a paper published in an academic periodical after being scrutinized anonymously and judged to be sound by other experts in the subfield. Their origin explains both their basic structure and the high esteem they have in the eyes of your professors.

Many journals are sponsored by academic associations. Most of your professors belong to some big, general one (such as the Modern Language Association5, the American Psychological Association6, the National Association for Sport and Physical Education, or the American Physical Society) and one or more smaller ones organized around particular areas of interest and expertise (such as the Association for the Study of Food and Society, the International Association for Statistical Computing, or the Slavic and East European Folklore Association). There are also generalist organizations organized by region of the country or state, such as the Eastern Sociological Society or the Southern Management Association. Each of these associations exists to promote the exchange of research findings and collaboration in their disciplines. Towards this end, they organize conferences, sponsor working groups, and publish one or more academic journals. These journals are meant to both publicize and archive the most interesting and important findings of the field.

Academic papers are essentially reports that scholars write to their peers—present and future—about what they’ve done in their research, what they’ve found, and why they think it’s important. Thus, in a lot of fields they often have a structure reminiscent of the lab reports you’ve written for science classes:

  1. Abstract: A one-paragraph summary of the article: its purpose, methods, findings, and significance.
  2. Introduction: An overview of the key question or problem that the paper addresses, why it is important, and the key conclusion(s) (i.e., thesis or theses) of the paper.
  3. Literature review: A synthesis of all the relevant prior research (the so-called “academic literature” on the subject) that explains why the paper makes an original and important contribution to the body of knowledge.
  4. Data and methods: An explanation of what data or information the author(s) used and what they did with it.
  5. Results: A full explanation of the key findings of the study.
  6. Conclusion/discussion: Puts the key findings or insights from the paper into their broader context; explains why they matter.

Not all papers are so “sciencey.” For example, a historical or literary analysis doesn’t necessarily have a “data and methods” section; but they do explain and justify the research question, describe how the authors’ own points relate to those made in other relevant articles and books, develop the key insights yielded by the analysis, and conclude by explaining their significance. Some academic papers are review articles, in which the “data” are published papers and the “findings” are key insights, enduring lines of debate, and/or remaining unanswered questions.

Scholarly journals use a peer-review process to decide which articles merit publication. First, hopeful authors send their article manuscript to the journal editor, a role filled by some prominent scholar in the field. The editor reads over the manuscript and decides whether it seems worthy of peer-review. If it’s outside the interests of the journal or is clearly inadequate, the editor will reject it outright. If it looks appropriate and sufficiently high quality, the editor will recruit a few other experts in the field to act as anonymous peer reviewers. The editor will send the manuscript (scrubbed of identifying information) to the reviewers who will read it closely and provide a thorough critique. Is the research question driving the paper timely and important? Does the paper sufficiently and accurately review all of the relevant prior research? Are the information sources believable and the research methods rigorous? Are the stated results fully justified by the findings? Is the significance of the research clear? Is it well written? Overall, does the paper add new, trustworthy, and important knowledge to the field? Reviewers send their comments to the editor who then decides whether to (1) reject the manuscript, (2) ask the author(s) to revise and resubmit the manuscript7, or (3) accept it for publication. Editors send the reviewers’ comments (again, with no identifying information) to authors along with their decisions. A manuscript that has been revised and resubmitted usually goes out for peer-review again; editors often try to get reviews from one or two first-round reviewers as well as a new reviewer. The whole process, from start to finish, can easily take a year, and it is often another year before the paper appears in print.

Understanding the academic publication process and the structure of scholarly articles tells you a lot about how to find, read and use these sources:

  1. Find them quickly. Instead of paging through mountains of dubious web content, go right to the relevant scholarly article databases in order to quickly find the highest quality sources.
  2. Use the abstracts. Abstracts tell you immediately whether or not the article you’re holding is relevant or useful to the paper you’re assigned to write. You shouldn’t ever have the experience of reading the whole paper just to discover it’s not useful.
  3. Read strategically. Knowing the anatomy of a scholarly article tells you what you should be reading for in each section. For example, you don’t necessarily need to understand every nuance of the literature review. You can just focus on why the authors claim that their own study is distinct from the ones that came before.
  4. Don’t sweat the technical stuff. Not every social scientist understands the intricacies of log-linear modeling of quantitative survey data; however, the reviewers definitely do, and they found the analysis to be well constructed. Thus, you can accept the findings as legitimate and just focus on the passages that explain the findings and their significance in plainer language.
  5. Use one article to find others. If you have one really good article that’s a few years old, you can use article databases to find newer articles that cited it in their own literature reviews. That immediately tells you which ones are on the same topic and offer newer findings. On the other hand, if your first source is very recent, the literature review section will describe the other papers in the same line of research. You can look them up directly.

Research papers, amongst others, are the most common papers a college student will ever write, and as difficult as it may sound, it is not impossible to complete. Research papers are my favorite kind of papers because of sourcing, paraphrasing, and quoting. Naturally as you would in other papers, your own paper should come from yourself, but when you are proving a point about a specific area of your topic, it is always ok to have a credible source explain further. In college, sources are very important for most, if not all papers you will have, and citing those sources is important as well. After you are able to familiarize yourself with citations, it will come natural like it has for many students.

Timothée Pizarro

Students sometimes grumble when they’re ordered to use scholarly articles in their research. It seems a lot easier to just Google some terms and find stuff that way. However, academic articles are the most efficient resource out there. They are vetted by experts and structured specifically to help readers zero in on the most important passages.

Finding Tier 1 Sources: Article Databases

Your campus library pays big money to subscribe to databases for Tier 1 articles. Some are general purpose databases that include the most prominent journals across disciplines8, and some are specific to a particular discipline.9 Often they have the full-text of the articles right there for you to save or print. We won’t go over particular databases here because every campus has different offerings. If you haven’t already attended a workshop on using the resources provided by your library, you should. A one-hour workshop will save you many, many hours in the future. If there aren’t any workshops, you can always seek advice from librarians and other library staff on the best databases for your topic. Many libraries also have online research guides that point you to the best databases for the specific discipline and, perhaps, the specific course. Librarians are eager to help you succeed with your research—it’s their job and they love it!—so don’t be shy about asking.

An increasingly popular article database is Google Scholar. It looks like a regular Google search, and it aspires to include the vast majority of published scholarship. Google doesn’t share a list of which journals they include or how Google Scholar works, which limits its utility for scholars. Also, because it’s so wide-ranging, it can be harder to find the most appropriate sources. However, if you want to cast a wide net, it’s a very useful tool.

Here are three tips for using Google Scholar effectively:

  1. Add your field (economics, psychology, French, etc.) as one of your keywords. If you just put in “crime,” for example, Google Scholar will return all sorts of stuff from sociology, psychology, geography, and history. If your paper is on crime in French literature, your best sources may be buried under thousands of papers from other disciplines. A set of search terms like “crime French literature modern” will get you to relevant sources much faster.
  2. Don’t ever pay for an article. When you click on links to articles in Google Scholar, you may end up on a publisher’s site that tells you that you can download the article for $20 or $30. Don’t do it! You probably have access to virtually all the published academic literature through your library resources. Write down the key information (authors’ names, title, journal title, volume, issue number, year, page numbers) and go find the article through your library website. If you don’t have immediate full-text access, you may be able to get it through inter-library loan.
  3. Use the “cited by” feature. If you get one great hit on Google Scholar, you can quickly see a list of other papers that cited it. For example, the search terms “crime economics” yielded this hit for a 1988 paper that appeared in a journal called Kyklos:
image

Figure 4.1, Google Scholar

1988 is nearly 30 years ago; for a social-science paper you probably want more recent sources. You can see that, according to Google, this paper was cited by 392 other sources. You can click on that “Cited by 392” to see that list. You can even search within that list of 392 if you’re trying to narrow down the topic. For example, you could search on the term “cities” to see which of those 392 articles are most likely to be about the economic impact of crime on cities.

Library Research As Problem-Solving

You’ll probably engage the subscription article databases at different points in the process. For example, imagine you’ve been assigned a research paper that can focus on any topic relevant to the course. Imagine further that you don’t have a clue about where to start and aren’t entirely sure what counts as an appropriate topic in this discipline. A great approach is to find the top journals in the specific field of your course and browse through recent issues to see what people are publishing on. For example, when I assign an open-topic research paper in my Introduction to Sociology course, I suggest that students looking for a topic browse recent issues of Social Problems or American Journal of Sociology and find an article that looks interesting. They’ll have a topic and—booyah!—their first source. An instructor of a class on kinesiology might recommend browsing Human Movement Science, the Journal of Strength and Conditioning Research, or Perceptual and Motor Skills.

When you have a topic and are looking for a set of sources, your biggest challenge is finding the right keywords. You’ll never find the right sources without them. You’ll obviously start with words and phrases from the assignment prompt, but you can’t stop there. As explained above, lower tier sources (such as Wikipedia) or the top-tier sources you already have are great for identifying alternative keywords, and librarians and other library staff are also well practiced at finding new approaches to try. Librarians can also point you to the best databases for your topic as well.

As you assess your evidence and further develop your thesis through the writing process, you may need to seek additional sources. For example, imagine you’re writing a paper about the added risks adolescents face when they have experienced their parents’ divorce. As you synthesize the evidence about negative impacts, you begin to wonder if scholars have documented some positive impacts as well.10 Thus you delve back into the literature to look for more articles, find some more concepts and keywords (such as “resiliency”), assess new evidence, and revise your thinking to account for these broader perspectives. Your instructor may have asked you to turn in a bibliography weeks before the final paper draft. You can check with your professor, but he or she is probably perfectly fine with you seeking additional sources as your thinking evolves. That’s how scholars write.

Finding good sources is a much more creative task than it seems on the face of it. It’s an extended problem-solving exercise, an iterative cycle of questions and answers. Go ahead and use Wikipedia to get broadly informed if you want. It won’t corrupt your brain. But use it, and all other sources, strategically. You should eventually arrive at a core set of Tier 1 sources that will enable you to make a well informed and thoughtful argument in support of your thesis. It’s also a good sign when you find yourself deciding that some of the first sources you found are no longer relevant to your thesis; that likely means that you have revised and specified your thinking and are well on your way to constructing the kind of self-driven in-depth analysis that your professor is looking for.

OTHER RESOURCES

  1. The Online Writing Laboratory (OWL) at Purdue University provides this list of links to freely available article databases.
  2. Google provides some great tips for getting the most out of Google Scholar.
  3. This resource from Bowling Green State University explains how searching subject headings in a database (compared to key words) can more quickly bring you to relevant sources.

Notes


1 If you aren’t actually interested in anything relating to the course, you’d do well to keep that information to yourself.

provided by Lumen Learning

Ah, The Research Paper

2 Obviously, not all writing assignments require you to find and use secondary sources. This chapter is relevant to those that do.

3 Bored? Browse these images and other collections of the Library of Congress’ American Memory Project: memory.loc.gov. Fascinating!

4 Wikipedia is a conundrum. There are a lot of excellent articles on there, and I, like many other professors, embrace the open-access values that embody things like Wikipedia and this very textbook. It’s not that Wikipedia is crap; it’s just that there are much more solid alternatives.

5 Where MLA citation style comes from.

6 Where APA citation style comes from.

7 From an author’s perspective, a verdict of “revise and resubmit”—colloquially called an “R & R”—is a cause for celebration. In many fields, most papers are revised and resubmitted at least once before being published.

8 Examples include Academic Search Premier (by EBSCO), Academic Search Complete (by EBSCO), Academic OneFile (by Cengage), General OneFile (by Cengage), ArticleFirst (by OCLC), and JSTOR (by ITHAKA).

9 Some examples: PsycINFO (for psychology), CINAHL (for nursing), Environment Complete (for environmental science), Historical Abstracts (for history).

10 One fairly recent article is Ilana Sever, Joseph Gutmann, and Amnon Lazar, “Positive Consequences of Parental Divorce Among Israeli Young Adults”, Marriage and Family Review 42, no. 4 (2007): 7-28

Listening to Sources, Talking to Sources

provided by Lumen Learning

Theses And Sources

Everyone knows that a thorough analysis and persuasive argument needs strong evidence. The credibility of sources, is one key element of strong evidence, but it also matters how sources are used in the text of the paper. Many students are accustomed to thinking of sources simply as expert corroboration for their own points. As a result, they tend to comb texts to find statements that closely parallel what they want to say and then incorporate quotes as evidence that a published author agrees with them. That’s one way to use sources, but there is a lot more to it.

Recall from prior chapters that writing academic papers is about joining a conversation. You’re contributing your own original thinking to some complex problem, be it interpretive, theoretical, or practical. Citing sources helps situate your ideas within that ongoing conversation. Sometimes you’re citing a research finding that provides strong evidence for your point; at other times you’re summarizing someone else’s ideas in order to explain how your own opinion differs or to note how someone else’s concept applies to a new situation. Graff and Birkenstein1 encourage you to think about writing with sources is a “They Say/I Say” process. You first report what “they” say; “they” being published authors, prevalent ideas in society at large, or maybe participants in some kind of political or social debate. Then you respond by explaining what you think: Do you agree? Disagree? A little of both?

This “They Say/I Say” approach can help student writers find balance in their use of sources. On one extreme, some students think that they aren’t allowed to make any claims without citing one or more expert authors saying the same thing. When their instructors encourage them to bring more original thinking into their writing, they’re confused about how to do it. On the other extreme, some students tend to describe, more or less accurately, what sources say about a topic but then go on to state opinions that seem unrelated to the claims they just summarized. For example, a student writer may draw on expert sources to explain how the prevention and early detection of cancer has saved lives2 but then argue for more funding for curing advanced cancer without making any explicit link to the points about prevention and screening. On one extreme, the sources are allowed to crowd out original thinking; on the other, they have seemingly no impact on the author’s conclusions.

How can you know when you’re avoiding both of these extremes? In other words, what kinds of theses (“I Say”) can count as an original claim and still be grounded in the sources (“They Say”)? Here are five common strategies:

  1. Combine research findings from multiple sources to make a larger summary argument. You might find that none of the sources you’re working with specifically claim that early 20th century British literature was preoccupied with changing gender roles but that, together, their findings all point to that broader conclusion.
  2. Combine research findings from multiple sources to make a claim about their implications. You might review papers that explore various factors shaping voting behavior to argue that a particular voting-reform proposal will likely have positive impacts.
  3. Identify underlying areas of agreement. You may argue that the literature on cancer and the literature on violence both describe the unrecognized importance of prevention and early intervention in order to claim that insights about one set of problems may be useful for the other.
  4. Identify underlying areas of disagreement. You may find that the controversies surrounding educational reform—and its debates about accountability, curricula, school funding—ultimately stem from different assumptions about the role of schools in society.
  5. Identify unanswered questions. Perhaps you review studies of the genetic and behavioral contributors to diabetes in order to highlight unknown factors and argue for more in-depth research on the role of the environment.

There are certainly other ways authors use sources to build theses, but these examples illustrate how original thinking in academic writing involves making connections with and between a strategically chosen set of sources.

Incorporating Sources

Here’s a passage of academic writing (an excerpt, not a complete paper) that illustrates several ways that sources can figure into a “They Say/I Say” approach3:

Willingham (2011) draws on cognitive science to explain that students must be able to regulate their emotions in order to learn. Emotional self-regulation enables students to ignore distractions and channel their attention and behaviors in appropriate ways. Other research findings confirm that anxiety interferes with learning and academic performance because it makes distractions harder to resist (Perkins and Graham-Bermann, 2012Putwain and Best, 2011). Other cognitive scientists point out that deep learning is itself stressful because it requires people to think hard about complex, unfamiliar material instead of relying on cognitive short-cuts. Kahneman (2011) describes this difference in terms of two systems for thinking: one fast and one slow. Fast thinking is based on assumptions and habits and doesn’t require a lot of effort. For example, driving a familiar route or a routine grocery-shopping trip are not usually intellectually taxing activities. Slow thinking, on the other hand, is what we do when we encounter novel problems and situations. It’s effortful, and it usually feels tedious and confusing. It is emotionally challenging as well because we are, by definition, incompetent while we’re doing it, which provokes some anxiety. Solving a tough problem is rewarding, but the path itself is often unpleasant.

These insights from cognitive science enable us to critically assess the claims made on both sides of the education reform debate. On one hand, they cast doubt on the claims of education reformers that measuring teachers’ performance by student test scores is the best way to improve education. For example, the Center for Education Reform promotes “the implementation of strong, data-driven, performance-based accountability systems that ensure teachers are rewarded, retained and advanced based on how they perform in adding value to the students who they teach, measured predominantly by student achievement” (http://www.edreform.com/issues/teacher-quality/#what-we-believe). The research that Willingham (2011) and Kahneman (2011) describe suggests that frequent high-stakes testing may actually work against learning by introducing greater anxiety into the school environment.
At the same time, opponents of education reform should acknowledge that these research findings should prompt us to take a fresh look at how we educate our children. While Stan Karp of Rethinking Schools is correct when he argues that “data-driven formulas [based on standardized testing] lack both statistical credibility and a basic understanding of the human motivations and relationships that make good schooling possible” (http://www.rethinkingschools.org/archive/26_03/26_03_karp.shtm), it doesn’t necessarily follow that all education reform proposals lack merit. Challenging standards, together with specific training in emotional self-regulation, will likely enable more students to succeed.4

In that example, the ideas of Willingham and Kahneman are summarized approvingly, bolstered with additional research findings, and then applied to a new realm: the current debate surrounding education reform. Voices in that debate were portrayed as accurately as possible, sometimes with representative quotes. Most importantly, all references were tied directly to the author’s own interpretative point, which relies on the quoted claims.

I think the most important lesson for me to learn about sources was that the best way to use them is to create a new point. What I mean by this is instead of using them only to back up your points, create your own conclusion from what your sources say. As a psychology major, I look at a lot of data from researchers who have created a conclusion from a meta-analysis (a combination of many studies about the same thing). So that’s how I like to think of using sources, I will look at many articles about the same subject and then come up with my own opinion. After using your sources, it is very important to cite them correctly. Personally, I want to be a respected and trustworthy scholar. However, if any of my papers were to be found without proper citations, all of my hard work would be for nothing and people would be wary about the rest of my work.

Aly Button

As you can see, there are times when you should quote or paraphrase sources that you don’t agree with or do not find particularly compelling. They may convey ideas and opinions that help explain and justify your own argument. Similarly, when you cite sources that you agree with, you should choose quotes or paraphrases that serve as building blocks within your own argument. Regardless of the role each source plays in your writing, you certainly don’t need to find whole sentences or passages that express your thinking. Rather, focus on what each of those sources is claiming, why, and how exactly their claims relate to your own points.

The remainder of this chapter explains some key principles for incorporating sources, principles which follow from the general point that academic writing is about entering an ongoing conversation.

Principle 1: Listen to your sources

Have you ever had the maddening experience of arguing with someone who twisted your words to make it seem like you were saying something you weren’t? Novice writers sometimes inadvertently misrepresent their sources when they quote very minor points from an article or even positions that the authors of an article disagree with. It often happens when students approach their sources with the goal of finding snippets that align with their own opinion. For example, the passage above contains the phrase “measuring teachers’ performance by student test scores is the best way to improve education.” An inexperienced writer might include that quote in a paper without making it clear that the author(s) of the source actually dispute that very claim. Doing so is not intentionally fraudulent, but it reveals that the paper-writer isn’t really thinking about and responding to claims and arguments made by others. In that way, it harms his or her credibility.

Academic journal articles are especially likely to be misrepresented by student writers because their literature review sections often summarize a number of contrasting viewpoints. For example, sociologists Jennifer C. Lee and Jeremy Staff wrote a paper in which they note that high-schoolers who spend more hours at a job are more likely to drop out of school.5 However, Lee and Staff’s analysis finds that working more hours doesn’t actually make a student more likely to drop out. Instead, the students who express less interest in school are both more likely to work a lot of hours and more likely to drop out. In short, Lee and Staff argue that disaffection with school causes students to drop-out, not working at a job. In reviewing prior research about the impact of work on dropping out, Lee and Staff write “Paid work, especially when it is considered intensive, reduces grade point averages, time spent on homework, educational aspirations, and the likelihood of completing high school”6. If you included that quote without explaining how it fits into Lee and Staff’s actual argument, you would be misrepresenting that source.

Principle 2: Provide context

Another error beginners often make is to drop in a quote without any context. If you simply quote, “Students begin preschool with a set of self-regulation skills that are a product of their genetic inheritance and their family environment” (Willingham, 2011, p.24), your reader is left wondering who Willingham is, why he or she is included here, and where this statement fits into his or her larger work. The whole point of incorporating sources is to situate your own insights in the conversation. As part of that, you should provide some kind of context the first time you use that source. Some examples:

Willingham, a cognitive scientist, claims that …
Research in cognitive science has found that … (Willingham, 2011).
Willingham argues that “Students begin preschool with a set of self-regulation skills that are a product of their genetic inheritance and their family environment” (Willingham, 2011, p.24). Drawing on findings in cognitive science, he explains “…”

As the second example above shows, providing a context doesn’t mean writing a brief biography of every author in your bibliography—it just means including some signal about why that source is included in your text.

Even more baffling to your reader is when quoted material does not fit into the flow of the text. For example, a novice student might write,

Schools and parents shouldn’t set limits on how much teenagers are allowed to work at jobs. “We conclude that intensive work does not affect the likelihood of high school dropout among youths who have a high propensity to spend long hours on the job” (Lee and Staff, 2007, p. 171). Teens should be trusted to learn how to manage their time.

The reader is thinking, who is this sudden, ghostly “we”? Why should this source be believed? If you find that passages with quotes in your draft are awkward to read out loud, that’s a sign that you need to contextualize the quote more effectively. Here’s a version that puts the quote in context:

Schools and parents shouldn’t set limits on how much teenagers are allowed to work at jobs. Lee and Staff’s carefully designed study found that “intensive work does not affect the likelihood of high school dropout among youths who have a high propensity to spend long hours on the job” (2007, p. 171). Teens should be trusted to learn how to manage their time.

In this latter example, it’s now clear that Lee and Staff are scholars and that their empirical study is being used as evidence for this argumentative point. Using a source in this way invites the reader to check out Lee and Staff’s work for themselves if they doubt this claim.

Many writing instructors encourage their students to contextualize their use of sources by making a “quotation sandwich”; that is, introduce the quote in some way and then follow it up with your own words. If you’ve made a bad habit of dropping in unintroduced quotes, the quotation sandwich idea may help you improve your skills, but in general you don’t need to approach every quote or paraphrase as a three-part structure to have well integrated sources. You should, however, avoid ending a paragraph with a quotation. If you’re struggling to figure out what to write after a quote or close paraphrase, it may be that you haven’t yet figured out what role the quote is playing in your own analysis. If that happens to you a lot, try writing the whole first draft in your own words and then incorporate material from sources as you revise with “They Say/I Say” in mind.

Principle 3: Use sources efficiently

Some student writers are in a rut of only quoting whole sentences. Some others, like myself as a student, get overly enamored of extended block quotes and the scholarly look they give to the page.7 These aren’t the worst sins of academic writing, but they get in the way of one of the key principles of writing with sources: shaping quotes and paraphrases efficiently. Efficiency follows from the second principle, because when you fully incorporate sources into your own explicit argument, you zero in on the phrases, passages, and ideas that are relevant to your points. It’s a very good sign for your paper when most quotes are short (key terms, phrases, or parts of sentences) and the longer quotes (whole sentences and passages) are clearly justified by the discussion in which they’re embedded. Every bit of every quote should feel indispensable to the paper. An overabundance of long quotes usually means that your own argument is undeveloped. The most incandescent quotes will not hide that fact from your professor.

Also, some student writers forget that quoting is not the only way to incorporate sources. Paraphasing and summarizing are sophisticated skills that are often more appropriate to use than direct quoting. The first two paragraphs of the example passage above do not include any quotations, even though they are both clearly focused on presenting the work of others. Student writers may avoid paraphrasing out of fear of plagiarizing, and it’s true that a poorly executed paraphrase will make it seem like the student writer is fraudulently claiming the wordsmithing work of others as his or her own. Sticking to direct quotes seems safer. However, it is worth your time to master paraphasing because it often helps you be more clear and concise, drawing out only those elements that are relevant to the thread of your analysis.

For example, here’s a passage from a hypothetical paper with a block quote that is fully relevant to the argument but, nevertheless, inefficient:

Drawing on a lifetime of research, Kahneman concludes our brains are prone to error:8

System 1 registers the cognitive ease with which it processes information, but it does not generate a warning signal when it becomes unreliable. Intuitive answers come to mind quickly and confidently, whether they originate from skills or from heuristics. There is no simple way for System 2 to distinguish between a skilled and a heuristic response. Its only recourse is to slow down and attempt to construct an answer on its own, which it is reluctant to do because it is indolent. Many suggestions of System 1 are casually endorsed with minimal checking, as in the bat-and-ball problem.

While people can get better at recognizing and avoiding these errors, Kahneman suggests, the more robust solutions involve developing procedures within organizations to promote careful, effortful thinking in making important decisions and judgments.

Even a passage that is important to reference and is well contextualized in the flow of the paper will be inefficient if it introduces terms and ideas that aren’t central to the analysis within the paper. Imagine, for example, that other parts of this hypothetical paper use Kahneman’s other terms for System 1 (fast thinking) and System 2 (slow thinking); the sudden encounter of “System 1” and “System 2” would be confusing and tedious for your reader. Similarly, the terms “heuristics” and “bat-and-ball problem” might be unfamiliar to your reader. Their presence in the block quote just muddies the waters. In this case, a paraphrase is a much better choice. Here’s an example passage that uses a paraphrase to establish the same points more clearly and efficiently:

Drawing on a lifetime of research, Kahneman summarizes that our brains are prone to error because they necessarily rely on cognitive shortcuts that may or may not yield valid judgments.9 We have the capacity to stop and examine our assumptions, Kahneman points out, but we often want to avoid that hard work. As a result, we tend to accept our quick, intuitive responses. While people can get better at recognizing and avoiding these errors, Kahneman suggests that the more robust solutions involve developing procedures within organizations to promote careful, effortful thinking in making important decisions and judgments.

Not only is the paraphrased version shorter (97 words versus 151), it is clearer and more efficient because it highlights the key ideas, avoiding specific terms and examples that aren’t used in the rest of the paper. If other parts of your paper did refer to Kahneman’s System 1 and System 2, then you might choose to include some quoted phrases to make use of some of Kahneman’s great language. Perhaps something like this:

Drawing on a lifetime of research, Kahneman summarizes that our brains are prone to error because they necessarily rely on cognitive shortcuts that may or may not yield valid judgments.10 System 1, Kahneman explains, “does not generate a warning signal when it becomes unreliable.” 11 System 2 can stop and examine these assumptions, but it usually wants to avoid that hard work. As a result, our quick, intuitive responses are “casually endorsed with minimal checking.” 12 While people can get better at recognizing and avoiding these errors, Kahneman suggests, the more robust solutions involve developing procedures within organizations to promote careful, effortful thinking in making important decisions and judgments.

Whether you choose a long quote, short quote, paraphrase or summary depends on the role that the source in playing in your analysis. The trick is to make deliberate, thoughtful decisions about how to incorporate ideas and words from others.

Paraphrasing, summarizing, and the mechanical conventions of quoting take a lot of practice to master. Numerous other resources (like those listed at the end of this chapter) explain these practices clearly and succinctly. Bookmark some good sources and refer to them as needed. If you suspect that you’re in a quoting rut, try out some new ways of incorporating sources.

Principle 4: Choose precise verbs of attribution

It’s time to get beyond the all-purpose “says.” And please don’t look up “says” in the thesaurus and substitute verbs like “proclaim” (unless there was actually a proclamation) or “pronounce” (unless there was actually a pronouncement). Here’s a list of 15 useful alternatives:13

  • Claims
  • Asserts
  • Relates
  • Recounts
  • Complains
  • Reasons
  • Proposes
  • Suggests (if the author is speculating or hypothesizing)
  • Contests (disagrees)
  • Concludes
  • Shows
  • Argues
  • Explains
  • Indicates
  • Points out
  • Offers

More precise choices like these carry a lot more information than “says”, enabling you to relate more with fewer words. For one thing, they can quickly convey what kind of idea you’re citing: a speculative one (“postulates”)? A conclusive one (“determines”)? A controversial one (“counters”)? You can further show how you’re incorporating these sources into your own narrative. For example, if you write that an author “claims” something, you’re presenting yourself as fairly neutral about that claim. If you instead write that the author “shows” something, then you signal to your reader that you find that evidence more convincing. “Suggests” on the other hand is a much weaker endorsement. As I’ll discuss in Chapter 8, saying more with less makes your writing much more engaging.

Sources are your best friend. They either help you reaffirm your thesis or offer a differing opinion that you can challenge in your paper. The biggest thing to worry about, when it comes to sources, is citing. However, there are a multitude of resources to help you cite properly. My personal favorite is called Knightcite.com. You just pick the type of resource, fill in the information on it and voila, you have a perfectly cited resource!

Kaethe Leonard

Conclusion

Like so many things in adult life, writing in college is often both more liberating and burdensome than writing in high school and before. On the one hand, I’ve had students tell me that their high-school experiences made it seem that their own opinions didn’t matter in academic writing, and that they can’t make any claims that aren’t exactly paralleled by a pedigreed quotation. Writing papers based on their own insights and opinions can seem freeing in contrast. At the same time, a college student attending full time may be expected to have original and well considered ideas about pre-Columbian Latin American history, congressional redistricting, sports in society, post-colonial literatures, and nano-technology, all in about two weeks. Under these conditions, it’s easy to see why some would long for the days when simple, competent reporting did the job. You probably won’t have an authentic intellectual engagement with every college writing assignment, but approaching your written work as an opportunity to dialogue with the material can help you find the momentum you need to succeed with this work.

OTHER RESOURCES

  1. Graff and Birkenstein’s little book, They Say/I Say: The Moves that Matter in Academic Writing 2nd ed. (New York: Norton, 2009) is a gem and well worth reading. They offer a series of templates that can help you visualize new ways of relating to sources and constructing arguments.
  2. Another excellent resource is Gordon Harvey’s Writing with Sources: A Guide for Students 2nd ed. (Indianapolis: Hackett, 2008), In it, he discusses the key principles for incorporating sources, the stylistic conventions for quoting and paraphrasing, and the basics of common citation styles. That’s all information you want to have at the ready.
  3. Many university writing centers have nicely concise on-line guides to summarizing, paraphrasing, and quoting. I found some especially good ones at the University of Wisconsin, the University of Washington, and, as always, the Purdue Online Writing Laboratory.

                                                              

Understanding Bias

provided by Lumen Learning

Bias

Photo of graffiti on wall spelling out "Bias"

Bias means presenting facts and arguments in a way that consciously favors one side or other in an argument. Is bias bad or wrong?

No! Everyone who argues strongly for something is biased. So it’s not enough, when you are doing a language analysis, to merely spot some bias and say…”This writer is biased” or “This speaker is biased.”

Let’s begin by reading a biased text.

Hypocrites gather to feed off Daniel’s tragic death

The death of two-year-old Daniel Valerio at the hands of his step-father brought outrage from the media.

Daniel suffered repeated beatings before the final attack by Paul Alton, who was sentenced in Melbourne in February to 22 years jail.

Rupert Murdoch’s Herald-Sun launched a campaign which included a public meeting of hundreds of readers. Time magazine put Daniel on its front cover. The Herald-Sun summed up their message:

The community has a duty to protect our children from abuse – if necessary by laws that some people regard as possibly harsh or unnecessary.

But laws – like making it compulsory for doctors and others to report suspected abuse – cannot stop the violence.

Last year, 30 children were murdered across Australia. Babies under one are more likely to be killed than any other social group.

Daniel’s murder was not a horrific exception but the product of a society that sends some of its members over the edge into despairing violence.

The origin of these tragedies lies in the enormous pressures on families, especially working class families.

The media and politicians wring their hands over a million unemployed. But they ignore the impact that having no job, or a stressful poorly paid job, can have.

Child abuse can happen in wealthy families. But generally it is linked to poverty.

A survey in 1980 of “maltreating families” showed that 56.5 per cent were living in poverty and debt. A further 20 per cent expressed extreme anxiety about finances.

A study in Queensland found that all the children who died from abuse came from working class families.

Police records show that school holidays – especially Xmas – are peak times for family violence. “The sad fact is that when families are together for longer than usual, there tends to be more violence”, said one Victorian police officer.

Most people get by. Family life may get tense, but not violent.

But a minority cannot cope and lash out at the nearest vulnerable person to hand – an elderly person, a woman, or a child.

Compulsory reporting of child abuse puts the blame on the individual parents rather than the system that drives them to this kind of despair.

Neither is it a solution. Daniel was seen by 21 professionals before he was killed. Nonetheless, the Victorian Liberal government has agreed to bring it in.

Their hypocrisy is breathtaking.

This is the same government that is sacking 250 fire-fighters, a move that will lead to more deaths.

A real challenge to the basis of domestic violence means a challenge to poverty.

Yet which side were the media on when Labor cut the under-18 dole, or when Jeff Kennett[1] added $30 a week to the cost of sending a child to kindergarten?

To really minimise family violence, we need a fight for every job and against every cutback.

– by David Glanz, The Socialist, April 1993

There are good and bad aspects of bias.

  1. It is good to be open about one’s bias. For example, the article about Daniel Valerio’s tragic death is written for The Socialist newspaper. Clearly socialists will have a bias against arguments that blame only the individual for a crime when it could be argued that many other factors in society contributed to the crime and need to be changed. Focusing on the individual, from the socialist’s point of view, gets “the system” off the hook when crimes happen. The socialist’s main reason for writing is to criticise the capitalist system. So David Glanz is not pretending to not be biased, because he has published his article in a partisan[2] newspaper.
    • Here are some ways to be open about your bias, but still be naughty.
      • (a) Deliberately avoid mentioning any of the opposing arguments.
      • (b) Deliberately avoid mentioning relevant facts or information that would undermine your own case.
      • (c) Get into hyperbole.
      • (d) Make too much use of emotive language.
      • (e) Misuse or distort statistics.
      • (f) Use negative adjectives when talking about people you disagree with, but use positive adjectives when talking about people you agree with.
      • Can you find examples of any of these “naughty” ways to be biased in Glaz’s article?
  2. You mustn’t assume that because a person writes with a particular bias he/she is not being sincere, or that he/she has not really thought the issue through. The person is not just stating what he/she thinks, he/she is trying to persuade you about something.

Bias can result from the way you have organised your experiences in your own mind. You have lumped some experiences into the ‘good’ box and some experiences into the ‘bad’ box. Just about everybody does this[3]. The way you have assembled and valued experiences in your mind is called your Weltanschauung (Veltarn-shao-oong). If through your own experience, plus good thinking about those experiences, you have a better understanding of something, your bias is indeed a good thing.

For example, if you have been a traffic policeman, and have seen lots of disasters due to speed and alcohol, it is not ‘wrong’ for you to biased against fast cars and drinking at parties and pubs. Your bias is due to your better understanding of the issue, but you still have to argue logically.

Really naughty bias

4. If you pretend to be objective, to not take sides, but actually use techniques that tend to support one side of an argument, in that case you are being naughty. There are subtle ways to do this.

(a) If the support for one side of the argument is mainly at the top of the article, and the reasons to support the opposite side of the issue are mainly at the bottom end of the article; that might be subtle bias – especially if it was written by a journalist. Journalists are taught that many readers only read the first few paragraphs of an article before moving on to reading another article, so whatever is in the first few paragraphs will be what sticks in the reader’s mind.

(b) Quotes from real people are stronger emotionally than just statements by the writer. This is especially true if the person being quoted is an ‘authority’ on the subject, or a ‘celebrity’. So if one side of the issue is being supported by lots of quotes, and the other side isn’t, that is a subtle form of bias.

(c) If when one person is quoted as saying X, but the very next sentence makes that quote sound silly or irrational, that is a subtle form of bias too.

Common sense tells us that if someone is making money out of something, he/she will be biased in favour of it.

For example, a person who makes money out of building nuclear reactors in Europe or China could be expected to support a change in policy in Australia towards developing nuclear energy.

A manufacturer of cigarettes is unlikely to be in favor of health warnings on cigarette packets or bans on smoking in pubs. Photo of papers on a desk, with a bumper sticker reading "Assume Nothing Check Everything"

Nonetheless, logically speaking, we cannot just assume a person who is making money out of something will always take sides with whomever or whatever will make him/her more money.

We have to listen to the arguments as they come up. Assuming someone is biased is not logically okay.  You have to show that someone is biased and use evidence to support your assertion that he/she is biased.

[1] Jeff Kennett was the leader of the Liberal party in Victoria at that time.

[2] When you are a partisan you have taken sides in an argument, or a battle, or a war.

[3] Learning critical thinking (which is what you are learning in Year 11 and 12 English) is aimed at getting you to do more, and better, thinking than that.

Important Concepts

educators are fond of research papers

primary sources

secondary sources

academic books generally fall into three categories

tier 1 sources

historical or literary analysis

google scholar

thorough analysis and persuasive argument

they say/i say process

bias

socialists

weltanschauung (Velt-arn–shao–oong)

Works Cited

1 If you aren’t actually interested in anything relating to the course, you’d do well to keep that information to yourself.

2 Obviously, not all writing assignments require you to find and use secondary sources. This chapter is relevant to those that do.

3 Bored? Browse these images and other collections of the Library of Congress’ American Memory Project: memory.loc.gov. Fascinating!

4 Wikipedia is a conundrum. There are a lot of excellent articles on there, and I, like many other professors, embrace the open-access values that embody things like Wikipedia and this very textbook. It’s not that Wikipedia is crap; it’s just that there are much more solid alternatives.

5 Where MLA citation style comes from.

6 Where APA citation style comes from.

7 From an author’s perspective, a verdict of “revise and resubmit”—colloquially called an “R & R”—is a cause for celebration. In many fields, most papers are revised and resubmitted at least once before being published.

8 Examples include Academic Search Premier (by EBSCO), Academic Search Complete (by EBSCO), Academic OneFile (by Cengage), General OneFile (by Cengage), ArticleFirst (by OCLC), and JSTOR (by ITHAKA).

9 Some examples: PsycINFO (for psychology), CINAHL (for nursing), Environment Complete (for environmental science), Historical Abstracts (for history).

10 One fairly recent article is Ilana Sever, Joseph Gutmann, and Amnon Lazar, “Positive Consequences of Parental Divorce Among Israeli Young Adults”, Marriage and Family Review 42, no. 4 (2007): 7-28.

Notes

4 A side note: You may have noticed that the verbs used in referencing tend to be in present tense: so-and-so “writes” or “claims” or “argues”. That’s what academic writers do, even if the piece and author are from far in the past. It’s called “the historical present” and it’s just one convention of academic writing.

6Ibid., 159.

7 It took me a long time to stop abusing block quotes. They made me feel like my paper was an unassailable fortress of citation! With the friendly but pointed feedback of my professors, I gradually came to see how they took too much space away from my own argument.

11Ibid, 416.

12Ibid, 417.

13 Google “verbs of attribution” to find other suggestions.

License

Icon for the Creative Commons Attribution 4.0 International License

Composing Ourselves and Our World by Elizabeth Burrows; Angela Fowler; Heath Fowler; and Amy Locklear is licensed under a Creative Commons Attribution 4.0 International License, except where otherwise noted.

Share This Book