The Seventeenth Century Meets the Internet: Using a Historian’s Approach to Evaluating Documents as a Guide to Twenty-First Century Online Legal Research
- Download PDF file (1,584 KB)
The PDF version includes font examples not available in the html version.
Over the last ten years, one of the most important themes of the scholarship focused on legal research has been the central role of new technologies. This rich body of scholarship has addressed topics ranging from the challenging pedagogy of teaching legal research in a constantly changing technological landscape1 to the evolving ethical obligation of practicing attorneys to master new technical sources in order to maintain their “research competencies.”2 The role of the Internet has, of course, been central to this discussion. Not only has the Internet resulted in an exponential growth of available legal data, but is has also changed the very manner in which lawyers and judges debate the question of what comprises “the law.”3 The increasingly central role of the Internet is even more important in light of the most recent ABA statistics on current research among practicing lawyers. The ABA data indicate that more than half of practicing attorneys use a free Internet search engine, such as Google, as the first step in their legal research.4 And well over ninety percent use free Internet search engines frequently during the life of a case.5 Internet-based information, then, has a tremendous potential to influence legal decisions, especially since one’s first impression of a legal question can have an important impact on future decisions in the case.
Despite the fundamental role that the Internet now plays in the lives of practicing lawyers, judges, law students, and law professors, relatively little attention has been devoted to the question of how information on the Internet should be evaluated. We have developed detailed guides that outline the available online materials,6 systematic approaches for finding online information,7 and thoughtful analyses that evaluate the pros and cons of having so much information on the web.8 Yet, although lawyers and students are advised to check the “author, date, and publisher” of online information,9 a more substantive approach for evaluating online legal materials has yet to be developed.10 In sum, the tide of the legal research landscape is increasingly carrying lawyers to the Internet—but without the necessary tools to evaluate the information they find once they get there.
This article offers a substantive approach for evaluating information on the Internet that borrows from the general approach used by historians11 in their evaluation of primary- and secondary-source material. This approach assumes that law students and practitioners are able to evaluate the basics—the author (an individual, an institution, anonymous, etc.), the date of publication (including republication, reissuance, etc.), and the publisher of the document (an institution, an individual, the government, a private enterprise, etc.)12—and focuses instead on the more sophisticated level of scrutiny that must be applied when analyzing Internet sources and documents.
I have divided this approach into five basic areas:
(1) The epistemology of the source—how the source might support an argument found in another document, or what kinds of information the document tells the reader “without actually telling the reader.”
(2) The purpose of the document—what is at stake for the author in the particular document, why the document was written, and what evidence in the document provides this information.
(3) The argument presented by the source—how a document makes its case, what its strategy is for accomplishing its goal, how it carries out this strategy, who the intended audience of the document is, and how the intended audience might influence its rhetorical strategy.
(4) The presuppositions of the reader—what biases, presumptions, and preconceptions the reader brings to bear on the interpretation of the source.
(5) The necessity of relating one source to another—the patterns or ideas that are repeated throughout the documents the reader has seen, and what major differences are present.13
Section II of this article provides necessary context for this approach by offering a critical description of the current legal-research environment. This description includes an analysis of the 2011 ABA Legal Technology Survey; a review of recent empirical studies demonstrating the role of Internet research for students; and a discussion of the slowly developing body of case law that suggests that attorneys may actually have a duty to “Google,”14 and, at a minimum, to use Internet sources.15 Understanding the fundamental role of Internet research for the practicing attorney underscores the necessity of developing a rigorous approach to evaluating the information. Section III presents the five-part approach to critically evaluating sources in the context of my law students’ assessment of several historical documents. That section includes examples from the Internet illustrating how this approach provides a fuller understanding of the information an online document contains. Finally, concluding in section IV, I argue that this approach should be adopted by law schools in teaching legal research and also by practicing lawyers, as the legal field becomes increasingly dependent on Internet-based information.
II. Current Practices and Trends in Legal Research
If a lawyer’s research were limited to finding good case law or even, more broadly, finding good primary authority, then perhaps this type of article— advocating a rigorous approach to evaluating the Internet—would not be necessary. In a search for primary authority, a lawyer can reliably turn to a fee-based service, such as Lexis or Westlaw,16 or to one of the many new “free” or low-cost17 sources, such as Google Scholar, Casemaker, or Loislaw. Although all Internet-based information must be evaluated at some level, these fee-based and free sources are generally considered reliable enough that problems of authenticity are not common.18 As all lawyers know, however, the legal research that spans the life of a case is much, much broader than simply locating on-point primary authority. For example, practicing attorneys conduct research on local rules, profiles of opposing counsel, and information on adverse parties; an attorney may need to research background medical or technical information for a medical-malpractice case, or statistics to help prepare for a deposition; a transactional attorney may need to find real-estate data; a litigator may need to find an expert—or research the biography of an opposing expert; a trial lawyer may need to research potential juror biographies.19 In other words, the research a lawyer must conduct in order to competently perform her job is much more expansive than finding the “law.”
The challenge with much of this research is, moreover, that it requires a proficiency in navigating the Internet. It is no longer novel for information to be available on the Internet20—in fact, researchers in any profession are often surprised when information is not available on the Internet. What is novel for lawyers and others is that an increasingly large amount of information is now available only on the Internet (or in some other electronic format.)21 Over the last five years, the number of public and private entities that have “gone paperless” has increased dramatically. 22
Although the availability of Internet-based information is increasing, the availability of reputable, online legal secondary sources is not.23 Virtually all lawyers and law professors would agree that good secondary sources play a fundamental role in legal research. As the ABA survey and other data show, however, secondary sources are among the least-available electronic sources and among the most expensive to access when they are available online.24 And, although free legal-resource sites such as LII and nolo.com can offer some useful legal background information, it is debatable (at best) whether such cites could ever fill the role of traditional legal secondary sources.25 The secondary-source problem is further exacerbated by the fact that many law firms are reducing or even cancelling subscriptions to secondary print sources.26 For good or for ill, if secondary sources are difficult to access online or are unavailable at work, in print form, the data suggest that their usage will decline.27
As a final preliminary matter, I am not advocating that we substitute “Internet surfing” for good secondary research or that pages on the World Wide Web can replace case law. Rather, this article reckons with the current legal environment and advocates that when attorneys do turn to free online legal research, a more rigorous approach to evaluating the results is necessary.
A. THE 2011 ABA LEGAL TECHNOLOGY SURVEY
Since 1990, the American Bar Association has conducted an annual survey of law-firm technology use, the results of which are published in the yearly ABA Legal Technology Survey Report.28 That report, which concentrates on issues relating to technology use rather than specific product use,29 provides data on topics ranging from litigation and courtroom technology, to law-office technology, to online research.30 In order to provide a broadly representative viewpoint, the survey includes responses from attorneys across the spectrums of practice area, geographic area, firm size, and individual years in practice.31
According to the 2011 ABA Report, (the Report), 84.8% of attorneys turn to online sources as the first step in their legal research.32 Within that group, a significant majority begin their research using a free online source rather than a fee-based online source.33 Indeed, in response to the question, “Where do you go first when starting a research project?,” the survey data indicate that of the attorneys who begin legal research online, 56.1% use a free online source compared to 43.9% who use a fee-based online source.34 And, although attorneys have access to many free, legalspecific sites (such as the Legal Information Institute maintained by Cornell35), over half (55.3%) of all free-online-source users turn first to a general search engine such as Google or Yahoo.36 Advocates of print material will not find much comfort in the fact that only 12.8% of the total number of respondents reported using print materials as their first step in legal research.37
These percentages reflect the total responses from all survey respondents: from solo practitioners to firms that employ 500 or more attorneys. The Report also provides data for specific groups; thus, the reader can ascertain, for example, how many solo practitioners use free sources as their “first step” in legal research versus attorneys working at large firms.38 When the data are broken down in this way, an attorney’s financial resources surprisingly do not correlate with the choice to use fee-based or free sources. For example, respondents from the largest law firms, which often have the greatest financial resources, actually reported the largest percentage of attorneys who start a research project using free online research—64.6% of respondents who work for firms of 500 or more attorneys begin research projects using free online sources as opposed to only 25.6% who begin with a fee-based source.39 In contrast, only 39.3% of lawyers working for law firms of just two to nine lawyers reported using free online sources as the first step in their legal research, while 45% of such lawyers used a fee-based source as their first step.
Though this data focuses on where attorneys turn first for their legal research, the Report also includes data on the percentages of attorneys who use free online sources at some point in their legal research: 97.7% of all attorneys report such use. The bottom line is that the percentage of lawyers using free online sources is substantial—and growing.40
As might be expected, the Report demonstrates that the extent to which attorneys use free online sources depends significantly on the type of source they seek. To highlight just a few examples: (1) when researching companies or corporations, 71.5% use free online research, 11.3% use feebased; and only 1% uses print;41 (2) when researching experts, 46.7% use free online sources, 9.1% use fee-based sources, and 3.1% use print;42 (3) for federal administrative–regulatory–executive research, 45.7% use free online sources, 26.4% use fee-based sources, and 3.7% use print;43 and (4) for researching public records, 70% use free sources, 18.9% use fee-based, and 2.6% use print.44
Again, as might be expected, certain categories surveyed in the Report revealed a larger percentage of attorneys using fee-based sources. For example, (1) when researching federal case law, 61.7% use fee-based sources, 27.2% use free sources, and 3.2% use print;45 (2) for federal legislation and statutes, 43.8% use fee-based sources, 41.5% use free sources, and 6.5% use print;46 (3) for case-law research within the attorney’s home state, 60.9% use fee-based sources, 29.8% use free sources, and 4.6% use print;47 (4) for research involving case law in a state other than the attorney’s home state, 57.3% use fee-based sources, 31% use free sources, and 2.2% use print;48 (5) for research involving legislation within the attorney’s home state, 49.2% use free sources, 38.6% use fee-based sources, and 7.6% use print;49 and (6) for research involving legislation in a state other than the attorney’s home state, 44.2% use fee-based sources, 43.1% use free sources, and 2.6% use print.50
As is always true with data, the numbers in the ABA Report could be used to demonstrate any number of trends. For example, the data strongly suggest that attorneys prefer fee-based sources for case-law research, but use free sources for legislative research.51 The data also present more- concrete evidence that print sources are playing an ever-smaller role in the research habits of practicing attorneys.52 Each of these trends could be the basis of a new scholarly article, or provide more data to support existing scholarship.53 The focus of this article, however, is the need for stricter standards vis-à-vis Internet research and, regardless of how the numbers in the Report are “crunched,” it is evident that all attorneys—young and old, solo and large-firm, Maine and California, women and men—are conducting ever-growing percentages of essential research on the wide open Internet.
B. ONLINE LEGAL RESEARCH AND LAW STUDENTS
Recent empirical studies demonstrate that law students are similarly inclined to use Internet sources in their research.54 And one such study, based on data from 2009, reveals that a substantial percentage of law students fails to critically assess online information.55 On a related note, other research data suggest that the Internet has contributed to inherently passive, rather than active, research skills.56
In one study, law librarians at Stanford Law School conducted a survey of the online research habits of Stanford law students.57 The survey, conducted over a three-year period, predictably demonstrated that an increasing number of students conduct research online.58 Given the tremendous growth of LEXIS and Westlaw during the period of the survey (2002–2004), the results were hardly surprising. One aspect of the survey that was surprising, however, involved an assignment in which the students were divided into four research groups and instructed to find a particular California code that governed the statute of limitations for fraud.59 One group was instructed to use LEXIS, one to use WestLaw, a third to use free Internet resources (not specified by the authors), and the last group was simply told to go to the library.60 The librarians assumed the students who were told to go to the library would use print books, but, instead, they “made a beeline for the library computers, and ‘Googled’ their way to the answer.”61 In today’s legal environment, it is certainly likely that a young attorney might be asked to find a specific statute without using a fee-based source. One of the best options is to find a copy of the code online. Careful evaluation of the online source would be crucial, however, to ensure that the young attorney had the official version of the code, as well as its most up-to-date version. As professors of legal research well know, online versions of state statutes range from the “official” versions to the unofficial and from the annotated versions to the unannotated; problems frequently exist with the currency of information in the unofficial sites.62
Another recent study surveyed upper-level Brooklyn University law students’ legal-research habits including, among many other things,63 their evaluation of Internet sources.64 Of the students surveyed, nearly 80% reported using free websites to conduct legal research, with Google being the most popular free website.65 Other students reported using Wikipedia, and a slightly smaller percentage reported using legal-specific free websites, such as Cornell’s LII.66 The study’s authors identified one of the key problems of online research in noting that “[a]lthough having free websites certainly makes information more accessible, it does not necessarily mean that such information is accurate or reliable.”67 The authors further noted that their students were aware of this problem.68 But although 88% of the students reported checking the reliability of their Internet sources,69 the criteria used to check reliability were not rigorous: 57% simply looked for information about the author of the site; 52% looked for the date of the article; 49% checked to see if the source cited other sources, 35% looked to identify the website’s publisher and 40% validated the website with another official or reliable source.70 These statistics suggest that a large percentage of students are evaluating Internet sources using only author–publisher–date information and at least 60% are not validating the information “with another official or reliable source.”71
C. ORDER IN THE COURT! OR SHOULD WE SAY, “GOOGLING” IN THE COURT?
In addition to the evidence of the ABA data for practicing lawyers and the empirical studies on law students’ Internet use, the Internet is also playing an increasingly important role in the courts. For example, materials acquired directly from the Internet, such as printouts of web pages, are now regularly held to be admissible evidence at trial.72 As a result, a lawyer who has an ethical duty to “zealously advocate” cannot simply ignore the Internet in the representation of a trial client.73
Additionally, the trend is growing for courts to use Internet citations in their opinions on issues ranging from criminal procedure to civil rights to judicial power.74 And, although many of the Internet citations in court opinions link to government websites of some kind, courts have also cited Internet sources relating to nonprofit research, commercial information, news, and popular culture.75 It is worth recalling the ABA data, indicating that these same types of sources (i.e., those relating to commercial or company information, government information, and news) are among the sources for which attorneys frequently rely on free Internet research.76
A discussion of the relationship between Internet citations in court opinions and attorney use of the Internet may be more relevant than the literature has thus far recognized. If a substantial percentage of citations in a court opinion originate in motions or briefs, then attorneys’ use of Internet sources will have a direct impact on Internet citations in court opinions. To illustrate, one of the cases used by Margolis to show the Supreme Court’s growing trend of citing to the Internet is Christian Legal Society v. Martinez, 130 S. Ct. 2971, 2980 (2010), which included a citation to an online law-school handbook.77 The citation for the handbook is first mentioned in the Petitioner’s Brief78 and the Court certainly appears to have used the Internet citation because Petitioner brought it to the Court’s attention.79 Practicing attorneys and law students should be aware of this trend and consider it an additional factor supporting the need for careful scrutiny of their Internet usage.80
Not only will lawyers need to be increasingly aware of the use of Internet citations by the Court, but they must also be aware that the time may come when lawyers actually have a “duty to Google.”81 In Davis v. Department of Justice,82 the United States Circuit Court of Appeals for the District of Columbia considered whether the FBI had properly complied with a Freedom of Information Act (FOIA) request. The FBI had released numerous audiotapes associated with a corruption investigation but relied on one of FOIA’s privacy exemptions in refusing to release four additional tapes.83 The FBI argued that, because it was unable to determine whether the speakers on the four tapes were dead or alive, the privacy exemption protected the remaining tapes from disclosure.84 On appeal, the court held that the FBI had failed to make reasonable efforts to ascertain whether the two speakers were still, in fact, alive. The FBI admitted that it had limited its search to two print books, including a published compilation of noteworthy recent deaths entitled, Who Was Who?85 Holding that the FBI had failed to make a reasonable search, the court’s reasoning is most noteworthy for its specific recognition of “Googling” as a reasonable search method that the FBI had failed to use. The court stated,
[O]ne has to ask why—in the age of the Internet—the FBI restricts itself to a dead-tree source with a considerable time lag between death and publication, with limited utility for the FBI’s purpose, and with entries restricted to a small fraction of even the “prominent and noteworthy?” Why, in short, doesn’t the FBI just Google the two names? Surely, in the Internet age, a “reasonable alternative” for finding out whether a prominent person is dead is to use Google (or any other search engine) to find a report of that person’s death. Moreover, while finding a death notice for the second speaker—the informant—may be harder (assuming that he was not prominent), Googling also provides ready access to hundreds of websites collecting obituaries from all over the country, any one of which might resolve that speaker’s status as well.86
The court reversed the lower court’s grant of summary judgment and remanded the case with specific directions for the FBI to evaluate alternative methods for determining whether the speakers on the requested audiotapes were dead. Other courts have reached similar conclusions, applying similar reasoning.87
In another example, an appellate court upheld an attorney’s right to use Google during a trial, despite the presiding judge’s explicit decision that “googling potential jurors” was not acceptable in his courtroom.88 In Carino v. Muenzen, the Court reviewed the conversation that occurred between the Court and Plaintiff’s Counsel during jury selection:
The Court: Are you Googling these [potential jurors]?
Plaintiff ’s Counsel: Your Honor, there’s no code law that says I’m not allowed to do that. I—any courtroom—
The Court: Is that what you’re doing?
Plaintiff ’s Counsel: I’m getting information on jurors—we’ve done it all the time, everyone does it. It’s not unusual. It’s not. There’s no rule, no case or any suggestion in any case that says—
* * * * *
The Court: No, no, here is the rule. The rule is it’s my courtroom and I control it.
Plaintiff ’s Counsel: I understand.
The Court: I believe in a fair and even playing field. I believe that everyone should have an equal opportunity. Now, with that said there was no advance indication that you would be using it. The only reason you’re doing that is because we happen to have a [Wi- Fi] connection in this courtroom at this point which allows you to have wireless internet access.
Plaintiff’s Counsel: Correct, Judge.
The Court: And that is fine provided there was a notice. There is no notice. Therefore, you have an inherent advantage regarding the jury selection process, which I don’t particularly feel is appropriate. So, therefore, my ruling is close the laptop for the jury selection process. You want to—I can’t control what goes on outside of this courtroom, but I can control what goes on inside the courtroom.89
The court ruled in favor of Plaintiff’s Counsel, holding that the trial judge had acted unreasonably in preventing an attorney from using the Internet to research potential jurors.90
Despite the deference we normally show a judge’s discretion in controlling the courtroom, we are constrained in this case to conclude that the judge acted unreasonably in preventing use of the internet by [plaintiff’s] counsel. There was no suggestion that counsel’s use of the computer was in any way disruptive. That he had the foresight to bring his laptop computer to court, and defense counsel did not, simply cannot serve as a basis for judicial intervention in the name of “fairness” or maintaining “a level playing field.” The “playing field” was, in fact, already “level” because internet access was open to both counsel, even if only one of them chose to utilize it.91
Again, the important issue for this article is the court’s explicit recognition of the role of the Internet in the life of a lawyer, even trumping a trial judge’s discretion over attorney conduct in his own courtroom.
So legal research is facing an important trend and its necessary corollary: the Internet is no longer merely an alternative source for information, and lawyers and law students must critically evaluate Internet information. In sum, the Internet is increasingly playing a central role in the legal environments in which lawyers (and, sooner than we think, our law students) are required to practice. Recognizing that much room exists for important debate on whether attorneys should or should not be relying on Internet sources, the reality is certainly something that needs to be reckoned with. The problem for many law students, and young attorneys, is that they have not been adequately trained to critically evaluate Internet information beyond a cursory review.
III. A Substantive Approach to Evaluating Information on the Internet.
Virtually every lawyer and law student recognizes the need to identify basic information such as author, publisher, and date when using sources on the Internet.92 In addition, many legal-research scholars recognize the need to apply scrutiny to other information obtained from free Internet sources. Unfortunately, the level of scrutiny is too often described in vague terms, which does not provide meaningful assistance to the researcher.93 Given the growing role of the Internet in the legal landscape, lawyers or law students who use the Internet must develop a thorough approach to evaluate the sources upon which they rely. The process used by historians in evaluating both primary- and secondary-source material offers a model for a substantive approach to this scrutiny.
Although historians, like legal researchers, employ a methodology that includes the basic identification of author, publisher, and date of publication, historians are careful to critically examine documents by further evaluating the following areas: (1) the document’s epistemology— information the document tells the reader without actually doing so, (2) the document’s purpose, (3) the document’s argument (4) the reader’s presuppositions, and (5) the necessity of relating one document to another.94
Each of these five steps is presented below in detail, accompanied by images (when available) of historical documents to illustrate the application of these steps, as well as examples of current information on the Internet.
A. THE EPISTEMOLOGY OF A DOCUMENT
The “epistemology of a document” refers to the document’s basis of knowledge, including the relation to other documents and implied information. In order to evaluate a document’s epistemology, the researcher must ask questions such as, How might the document support an argument found in another source? and What kinds of information does the document tell the reader without actually telling the reader?
The historical document I have used to illustrate this principle is the seal of the Massachusetts Bay Colony, which was used for the colony’s official documents and records.95 The seal, pictured below, is a “document” with which most law students are already familiar; the image is included in numerous high-school and college U.S. history textbooks and appears frequently in introductory classes on colonial America.96 Based on a cursory evaluation of this document, students usually notice things such as the “officialness” of the border, created by the Latin text and engraved trim, and the relatively “flat” appearance of the depicted Native American man.97
When asked specifically about the epistemology of the document—for example, what does it tell us without actually telling us—students look more critically, and usually notice the bow and arrow in the man’s hands. Students might posit, for example, that whoever created the document was trying to communicate something about the violent nature of Native Americans.
Interestingly, students seldom notice the words coming out of the man’s mouth in the form of a small banner. Because the words are spelled backward, the reader often misses them altogether—usually presuming that the banner simply contains more Latin text. Students’ evaluation of this document abruptly changes when they realize (or I tell them) that the man is saying “COME OVER HERE AND HELP US.” Rather than see simply a neutral document, students begin to perceive something else. As one historian of colonial American has written,
[The seal] was a very useful tool for the colonial enterprise. Colonial epistemology began with Europeans’ production of cultural, historical, and political representations about the Indians of North America as “inferior,” ahistorical, and elemental beings who were deserving of, and in the case of the Bay Colony seal, even pleading for, the domination of Europe. This production of knowledge began not only with written accounts of New World exploration and settlement . . . but also on visual markers—such as the Bay Colony Seal—that legitimized these New World ventures and attempted to fix Native and colonial identity.98
The phrase “come over here and help us” is critical to understanding what the document is really communicating. Students theorize that if English settlers were convinced that the Native Americans wanted the English to “come over and help them,” then any guilt associated with the European treatment of Native Americans was assuaged. After all, if the Native Americans asked “us” to “come over and help them”—then “we” had every right to colonize.99 Of course, many interpretations exist—but the crucial difference is that students become much more sophisticated in their critique by simply asking a few questions that they had previously not thought to ask.
Following our evaluation of the seal, I immediately turn to a twenty-first century example to illustrate how this same principle might be encountered in a real-life legal-research project.100 The “document” that I have used in the past is a web page containing information related to class actions brought under a specific California statute known as the UCL (California’s Unfair Competition Law).101 The web page was designed to look very much like a government web page; containing links to cases, statutes, local rules, local court dockets, etc.102 One subheading on the “Useful Links for Lawyers” page is captioned “Class Actions” and contains links to items such as Federal Rule of Civil Procedure 23, California Civil Code 382, and the “Class Action ‘Fairness’ Act.”103 The Class Action Fairness Act of 2005 (28 U.S.C. §§ 1332(d), 1453, and 1711–1715), otherwise known as CAFA, does not have quotes around the word “Fairness.” Emphatic quotes, akin to air quotes, are typically used to denote sarcasm, satire, or irony. After pointing out the “air quotes” around the word “Fairness,” I ask the students to figure out what the document is telling us “without actually telling us.” Almost everyone speculates (correctly) that the author of the web page does not consider CAFA to be “fair.” Indeed, a click on the CAFA subheading (designated Class Action “Fairness” Act) reveals links to articles and blogs with titles such as, “The Unfairness of the Class Action Fairness Act.”104 The editorial comments that accompany the articles, blogs and case excerpts clearly highlight what the author considers to be the “unfair” aspects of the federal legislation.105 In relatively short order, through the application of a more critical approach to the document’s message, students discover a crucial component of the document’s agenda.
The evaluation of the Massachusetts Bay Colony seal and the California web page takes no more than a few minutes. The familiar historical document serves to get students’ interest and demonstrates how a historian would approach the evaluation. Once exposed to the idea that a document’s epistemology can (and should) be evaluated, students are quick to ask similar questions in future research.
B. THE PURPOSE OF THE DOCUMENT
In assessing the purpose of a document, students must pause to consider questions such as, What is at stake for the author of the document? Why was the document written? and, What evidence in the document provides this information?106
For this component, I use two censuses taken in a Mexican village during 1690, when Mexico was a colony of Spain; one census was recorded by the ruling Spanish government, one was taken by the local indigenous authorities.107 The Spanish census indicated that the village contained 1249 “souls,” but the local census indicated only 711. Obviously, the answer to the question, “Why was this document written?”, is crucial to understanding the data. I inform the students that the purpose of the census was to determine the amount of tax owed by the village—the more adult citizens of the village, the more tax the village owed.108 As soon as students have this information, their critical evaluation of the documents changes dramatically. Without any additional information from me, students can intelligently speculate as to “what was at stake for the author”—for the Spanish it was more tax revenue, for the local people it was less.
Returning to the California class-action web page, I ask the students to see if they can determine what might be at stake for the author. A little further searching allows them to locate an “about the author” link, which states, “I am a plaintiff’s class action lawyer.”109 If the analysis ended with the traditional “Who is the author?”, the students’ work would be done. But by focusing on the question, “What is at stake for the author?”, the students keep reading. They find text that states, “I am actively accepting new co-counseling arrangements in pending and prospective contingency-fee class action cases. If you’re interested in working together on a case, or working up a case together, give me a call at _____.”110 What is at stake for the author of this document? Money!
Again, this exercise takes a relatively short time and is something that students readily grasp. It is not a case of something that is too difficult to learn—it is simply teaching students to evaluate a source critically by asking a set of questions that they may not have considered before.111
C. THE ARGUMENT PRESENTED BY A DOCUMENT
In assessing the argument presented by a document, students must consider questions such as, How does a document make its case?, What is its strategy for accomplishing its goal?, How does it carry out this strategy?, What is the intended audience of the document?, and, How might the intended audience influence its rhetorical strategy?112 Many of these questions point to the bias of a document, which is perhaps one of the most important factors a lawyer must consider when evaluating Internet documents.113 Similar to the Massachusetts Bay Colony seal, the document I use with my students is another well-known image from American History, the 1787 “Slave Medallion,” designed by Josiah Wedgewood, with the inscription, “Am I Not a Man and A Brother.”114
In their first efforts, students evaluate the “argument” presented by this document in a straightforward manner—they speculate that the document was created by or for abolitionists and that it appears to “make its case” by depicting an enslaved man who is thankful for the abolitionists’ support.115 I briefly provide some of the historical critiques of the medallion, including the argument that the image embodies the ideals of the abolitionists, i.e., a grateful slave on bended knee, rather than embodying the enslaved man’s reality of torture, oppression, and resistance.116 Students can then speculate as to whether the document was intended to illustrate solidarity with enslaved people—or solidarity with other abolitionists.117 In the process, they get a feel for why questions such as, “Who is the intended audience of the document?” and, “How does the intended audience influence the document’s strategy?” are important.118
Following our review of the medallion, I show the students a number of current web pages that illustrate bias. For example, I use two law-firm web pages from Wisconsin law firms that specialize in employee non-compete agreements.119 Each of the web pages analyses the same, recent Wisconsin cases, yet the message of one web page is decidedly pro-employer while the message of the other is decidedly pro-employee.120 On closer scrutiny, the first web page is indeed a pro-employer web page, promising “One More Victory for Employers in The Non-Compete Agreement Realm,”121 while the second webpage is maintained by “Attorneys for Employee Rights.”122 For any practicing attorney, the concept that a law-firm webpage has a bias or biases in favor of certain groups is not big news—but this is not so obvious to law students.123 When the students ask the right questions, however, the ability to detect bias is relatively straightforward. In the case of the employee–employer web-sites, establishing the identity of the intended audience—and recognizing how the intended audience influences the content of the site’s material—substantially improves a student’s ability to evaluate the information. After all, the cases listed on each of these web pages are, in fact, the most recent, important cases pertaining to Wisconsin non-compete law. Using the cases as a starting point for further, comprehensive research on this area of law is not unreasonable and may even save a client money. And we know from the ABA survey that lawyers are increasingly using case updates from law-firm web pages as part of their legal research. Ultimately, the appropriate warning to students (or young lawyers) that a law-firm web page is not the ideal place to begin a research project should not prevent institutions (and law firms) from providing students with some evaluative tools—such as sensitizing them to a document’s subtext or bias—in order to better using such web pages when appropriate.
What I have observed with this and other exercises is that students are not resistant to learning good evaluative techniques—in fact, many seem to enjoy the detective work. The most-frequent comments I hear while doing these exercises are not “I don’t get this” or “I’m confused”— instead, I hear “I had never thought of that” or “That makes sense.” Once they see that the use of strategic questions results in a much fuller appreciation of the document, they are eager to continue asking.
D. THE PRESUPPOSITIONS OF THE READER
If the argument presented by the document, even implicitly, is one way the researcher can encounter bias, the presuppositions, or biases, of the reader, are its flip-side.124 In this part of the analysis, the legal researcher must learn to identify the biases, presumptions, and preconceptions that they themselves bring to bear on the interpretation of a document. At the outset, it is important for students to understand that the word “bias” in this context simply means the identities, experiences, and interests that they posses.125 Just as students can learn to identify the internal biases in a document, so they can learn to recognize their own internal biases and, more importantly, how their identities and interests inevitably shape the way they search for and interpret information on the Internet.126
To demonstrate reader bias, I use an old social-security pamphlet published by the government in 1941.127 The pamphlet is divided into eight frames, which briefly summarize the process through which a fictitious worker would earn social-security credits and then receive social-security payments after retirement.
I first ask students for their reactions to the pamphlet’s message. The fact that the worker is a man and that his wife, rather than have her own job, is merely a “widow” often leads the students to comment on gender stereotypes in the workplace in that period.128 The statement that tends to reveal the greatest student bias, however, is typically the last frame of the pamphlet, which states, “Checks will come as a matter of right. He and his employer paid for them (emphasis added).” The variety of student presuppositions leads naturally to a variety of comments on this frame. Some students suggest that this pamphlet must have been written by an extreme “left-winger” because only a “liberal” would state that social security is a matter of right.129 Along these same lines, another student commented that the document was likely some kind of propaganda because social security is an “entitlement,” not something that a person receives as a “matter of right.” Opposite biases are also revealed; for example, one student commented that this pamphlet could effectively be used in this year’s election to inform people that security in old age “is a right, not a privilege.” The student added that society should have an obligation to take care of its “old people.”
Of course, no “correct” reading of the pamphlet exists—the student comments are instructive in showing them in an accessible way how their current presuppositions inform their interpretation of a source. In this instance, individual biases with regard to government payments (social security) affected the students’ beliefs regarding who published the document (for example, a “left-winger”) and the purpose of the document (for example, propaganda).
Moving from 1941 to 2010, I use the issues raised in the Morgan Hill Unified School District litigation to discuss the role of student presuppositions in relation to a recent legal case.130 The Morgan Hill case involved a group of students at Live Oak High School in Morgan Hill, California, who wore American-themed t-shirts to school on Cinco de Mayo 2010.131 Stating that they believed the students’ safety was jeopardized by their clothing, school administrators directed the students to remove (or turn inside out) their shirts, which depicted the American flag, or risk suspension.132 The students brought suit against the district and school administrators, claiming various constitutional violations, including violation of the First Amendment, Equal Protection Clause, and Due Process Clause.133 On November 8, 2011, United States District Court Judge James Ware of the Northern District of California held that the School District had not violated the students’ First Amendment rights. Applying the substantial-disruption test set forth in Tinker v. Des Moines Independent Community School District,134 the court found that school officials had sufficient evidence upon which to reasonably forecast that allowing the students to wear the t-shirts on Cinco de Mayo Day would result in violence causing substantial disruption to school operations and endangering the safety of the students.135
Engaging in a brief advocacy exercise, I divide the students in two groups—asking one group to research online news sources for information in support of the students and one group to do the same in support of the school district. Not surprisingly, many online news sources use language that easily identifies the bias of the author.136 But, more importantly, students themselves quickly realize how their own biases affect their interpretation of the sources. Students must reckon with their own presuppositions involving free speech, the American Flag, and a public high school (Live Oak) with a large Hispanic population of which twenty percent are identified as current English-language learners.
One student searching for sources supporting the plaintiff students found a Google hit apparently authored by the ACLU.137 The student initially did not consult the source, admitting a personal bias against the ACLU (“too liberal”). After seeing the ACLU source appear in several different search results, the student finally opened the link and found an ACLU-sponsored letter in support of the plaintiff students.138 In other words, her original presupposition precluded her from accessing a source that later turned out to be helpful.139 Other student statements revealed conflicts among their own biases. For example, several students stated that although they were inclined to support free speech in virtually every setting, other, equally important personal beliefs led them away from supporting the student plaintiffs in the Morgan Hill case. Students with this perspective viewed the following statement from an online CNN article as quite credible: “The students who brought the lawsuit against school officials claim to be proud of the American flag. But it’s obvious they don’t have the foggiest idea what it represents.”140 In brief, these students believed that wearing the American Flag t-shirts was disrespectful of Hispanic culture—and therefore, regardless of their presuppositions (or not) in favor of free speech, they were biased in favor of sources that criticized the plaintiff students.
Overall, students recognized that their biases influenced their evaluation of the credibility of online sources, as well as their inclination to access a particular source in the first place. This short class discussion was also useful to introduce students to the subject of managing their own presuppositions when representing actual clients in real life.141 As the Internet (and Internet sources) continues to play an increasing role in the practice of law,142 dealing with one’s presuppositions is a skill that all lawyers must develop.
As with each of these exercises, the most important skill learned by students is simply learning to ask new questions of the sources that they encounter. Biases are not static—student experiences and identities will change over time—but remembering to query how one’s biases affect one’s interpretation of source material is a skill that requires only a little selfreflection.
E. THE NECESSITY OF RELATING ONE DOCUMENT TO ANOTHER
The final component of the five-part evaluation requires that students relate one document to another in the course of doing their research. Students must evaluate the patterns or ideas that are repeated throughout the documents, as well as take stock of the major differences that are present.143 Although this step is important in any research project, it is even more crucial when conducting Internet research primarily because, as other scholars have discussed at length, Internet sources are often characterized by several “physical” flaws. First, Internet sources are often substantively incomplete;144 as a result, other sources must be consulted for a more complete and accurate picture. Second, computer screens typically “do not provide the context of surrounding material.”145 And, finally, different types of documents often appear the same on a computer screen—so an op-ed from a local newspaper may appear very similar to a Supreme Court decision.146 Coupled with these physical flaws, online sources must necessarily be linked to the larger legal picture. For example, relating one document to another is essentially the process a researcher uses when an Internet source leads to “the law”—the cases, statutes, administrative regulations, etc.—that governs the legal question being researched. In situations where a lawyer or law student is not seeking a case or statute (such as researching data to discredit an opposing expert, or researching facts regarding a potential juror147), information located on the Internet must be placed in a larger context.
Although I list this step as the last of the five, it is actually interwoven with the four other steps.148 As I point out to my students, this step requires that students evaluate the epistemology, the purpose, and the argument for each of their related sources. By focusing on the patterns and ideas, and the similarities and differences of a group of documents, the problem of “incomplete” Internet sources that lack the context of surrounding materials is mitigated.
Legal research is one of the fundamental components of good lawyering. The majority of legal scholars and teachers, including this author, would agree that there is no substitute for critical analysis of a problem, followed by a carefully drafted research plan, which is then implemented using a variety of primary and secondary authority from reputable sources.
The Internet, however, is changing the game for all practicing attorneys. Factors such as cost, accessibility, and efficiency have made it the “first step” in legal research for many lawyers. As a result, we cannot wish away the fact that our students and new lawyers will begin to use (or continue to use) the Internet for substantive legal research. What we can do is implement a more rigorous approach to teaching our students how to evaluate this information—and do so as one component of a larger legal research curriculum.
In a recent article about teaching research to “millennials,” Kaplan and Darvil argue, “Research instructors must encourage their students to think critically about these resources and to evaluate the diversity of information types. By doing so, they will teach their students to be information literate.”149 In order to achieve information literacy, law students must possess the ability to (1) “recognize when information is needed and [(2)] have the ability to locate, evaluate, and use effectively the needed information”150 in order to practice law in a competent manner.151
The approach outlined in this article picks up directly on this call to action and can be applied as a complement to almost any method of teaching legal research. For example, Kaplan and Darvil suggest that one method of exposing students to the types of materials that are increasingly used by practicing lawyers is “to add course page links to reliable and free legal websites commonly used by practitioners.”152 Students, in turn, would be required to vet the website and argue as to the document’s validity.153 The approach outlined in this article provides the necessary tools for this type of vetting. Determining a document’s validity is more complex than simply identifying the author, publisher, and date of an article. Students must think more broadly about issues such as the purpose of a document, the bias of a document, and the argument a document makes.
Students should of course learn good research strategies with both primary and traditional secondary materials. But free Internet sources are a reality in the current legal environment. We have an obligation to teach students to be more rigorous in their approach to evaluating online sources. William Butler Yeats is reported to have said that “Education is not filling a bucket, but lighting a fire.”154 The approach suggested in this article is akin to lighting a fire, in the form of teaching our students to ask new questions of the documents they encounter every day as lawyers.
* ©Assistant Professor of Practice at the University of Cincinnati, College of Law.. Prior to my career as a lawyer and law professor, I taught Latin American history at Indiana University, Bloomington and the University of Cincinnati. I would like to thank Dean Lou Bilionis and the College of Law for awarding me a Schott Summer Research Grant, which helped fund the research and writing of this article. I would also like to thank my colleagues at Cincinnati for their feedback on my presentation of these ideas during the 2011 Summer Faculty Research Workshop. I especially thank Ian Gallacher and Susan Bay for their feedback and support at the 2011 Southeast Legal Writing Conference at Mercer College of Law. Ian and Susan listened to my first presentation of the ideas that form the core of this article—their suggestions and encouragement convinced me to move forward with the project. And finally, thank you to Jessica Clark and Ian Gallacher for their tremendous editing assistance—this article is better in every way for their excellent suggestions and editing pens.
1 See e.g. Aliza B. Kaplan & Kathleen Darvil, Think [and Practice] Like a Lawyer: Legal Research for the New Millennials, 8 J. ALWD 153 (2011); Ellie Margolis, Surfin’ Safari—Why Competent Lawyers Should Research on the Web, 10 Yale J.L. & Tech. 82 (2007).
2 See e.g. Deborah Hackerson, Access to Justice Starts in the Library: The Importance of Competent Research Skills and Free/Low-cost Research Resources, 62 Me. L. Rev. 473 (2010); Patrick Meyer, Law Firm Legal Research Requirements for New Attorneys, 101 L. Lib. J. 297 (2009); Theresa Gabaldon, Virtual Virtuous Living: How Can the I-Generation of Lawyers Best Love and Serve its Neighbors? 43 Val. U. L. Rev. 1045 (2009).
3 For an excellent summary of how “electronic research is changing the nature of the law and legal reasoning itself,” see Ellie Margolis, Authority without Borders: The World Wide Web and the Delegalization of Law, 41 Seton Hall L. Rev. 909, 911 (2011); Ian Gallacher, “Aux Armes, Citoyens!”: Time for Law Schools to Lead the Movement for Free and Open Access to the Law, 40 U. Toledo L. Rev. 1 (2008); see also generally Amy E. Sloan, If You Can’t Beat ‘Em, Join ‘Em: A Pragmatic Approach to Nonprecedential Opinions in the Federal Appellate Courts, 86 Neb. L. Rev. 895 (2008); Sarah E. Ricks, A Modest Proposal for Regulating Unpublished, Non-Precedential Federal Appellate Opinions While Courts and Litigants Adapt to Federal Rule of Appellate Procedure 32.1, 9 J. App. Prac. & Process 17 (2007).
4 ABA Leg. Tech. Resource Ctr., 2011 Legal Technology Survey Report, Online Research, Volume 5 xv–xvi (ABA 2011). [hereinafter ABA Report]
5 Id. at xiv (98% conduct legal research online).
6 See e.g. Timothy L. Coggins, Legal, Factual and Other Internet Sites for Attorneys and Legal Professionals, 15 Rich. J.L. & Tech. 13 (2009).
7 See e.g. Rachel Fisher, Research Right Using Books and Bytes, 45 Tenn. B.J. 25 (2009).
8 Ian Gallacher, Forty-Two: The Hitchhiker’s Guide to Teaching Legal Research to the Google Generation, 39 Akron L. Rev. 151 (2006); Anne Klinefelter, When to Research is to Reveal: The Growing Threat to Attorney and Client Confidentiality from Online Tracking, 16 Va. J.L. & Tech. 1 (2011). This issue is of equal concern to other countries. See e.g. Natasha Choolhun, Google: To Use or Not to Use: What Is the Question? 9 Leg. Info. Mgt. 168–72 (2009).
9 The author, date, and publisher are the three categories most often referred to under the category of “credibility” when authors discuss using Internet sources. See e.g. Amy E. Sloan, Basic Legal Research: Tools and Strategies 295–96 (4th ed., Aspen Publishers 2009) (cautioning readers to assess the credibility of Internet sources through steps such as identifying the “history and mission” of web-page publishers as well as the biographical information of the author. In addition, the ABA Model Rules of Professional Conduct clearly embody a lawyer’s duty to conduct research in a diligent fashion. See generally Model R. Prof. Conduct (ABA 2010). Among other rules on point, Rule 1.3 (addressing the duty of general diligence), id. at R. 3.1, and Rule 3.3 (addressing the duty of candor to the tribunal) are relevant to this discussion, id. at R. 3.3.
10 See e.g. John W. Fritch & Robert L. Cromwell, Evaluating Internet Resources: Identity, Affiliation, and Cognitive Authority in a Networked World, 52 J. of the Am. Soc’y for Info. Science & Tech., 499–507 (2001); Leah Graham & Panagiotis Takis Metaxas, “Of Course It’s True; I Saw It on the Internet!”: Critical Thinking in the Internet Era, 46 Communs. of the ACM 71–75 (May 2003).
11 Historians certainly do not have a monopoly on a rigorous approach to evaluating source material; in fact, many academic disciplines depend on similar rigorous approaches. History was simply the natural point of comparison for me based on my experiences as a doctoral student and teacher.
12 Most legal-research textbooks used by first-year law students include this type of basic information for evaluating Internet sources. See e.g. Sloan, supra n. 9, at 295–96.
13 Mark A. Kishlansky, How to Read a Document, in Sources of the West: Readings in Western Civilization, Volume II: From 1600 to the Present xiii-xxii (7th ed., Longman Publ’rs 2008); Am. Historical Ass’n, Statement on Standards of Professional Conduct, http://www.historians.org/pubs/free/ProfessionalStandards.cfm (last accessed Mar. 10, 2012); Ref. & User Services Ass’n, Am. Lib. Ass’n, Using Primary Sources on the Web, http://www.ala.org/rusa/sections/history/resources/pubs/usingprimarysources (last accessed Mar. 4, 2012); Anthony Brundage, Going to the Sources: A Guide to Historical Research and Writing 1–89 (3d ed., Harlan Davidson 2002); W.H. McDowell, Historical research: A Guide 109–26 (Longman Publ’rs 2002).; Nat’l Archives, Teachers’ Resources, History in the Raw, http://www.archives.gov/education/history-in-the-raw.html (last accessed Mar. 4, 2012); Kelly Schrum, Surfing for the Past: How to Separate the Good from the Bad, Am. Historical Ass’n Persps. (May 2003), http://www.historians.org/perspectives/issues/2003/0305/0305for3.cfm (last accessed Mar. 10, 2012); Robert C. Williams, The Historian’s Toolbox: A Student’s Guide to the Theory and Craft of History, 56–73, 177–81 (2d ed., M. E. Sharpe 2007).
14 See Carole Levitt & Mark Rosch, A Cybersleuth’s Guide to the Internet: Conducting Effective Investigative & Legal Research on the Internet 47–48 (11th ed., IFL Press 2011).
15 See e.g. Davis v. Dep’t of Just., 460 F.3d 92, 95 (D.C. Cir. 2006) (holding that Dep’t of Justice failed to make reasonable efforts to comply with the Freedom of Information Act request when it did not, among other things, consult readily available search engines such as Google). On a more general level, these issues also dovetail with a lawyer’s duty of professional responsibility, such as the duty to “zealously advocate” for one’s client. For a discussion of this duty, see infra sec. II.C.
16 Although outside the scope of this article, the generally “uncritical” acceptance of LEXIS and Westlaw is an issue deserving additional scholarly attention. For a good discussion of this issue, see William R. Mills, The Decline and Fall of the Dominant Paradigm: Trustworthiness of Case Reports in the Digital Age, 53 N.Y. L. Sch. L. Rev. 917, 927–28 (2008/2009).
17 Even “free” sources, of course, come with a cost, whether through the hidden advertising costs of Google or the membership fees of state bar associations.
18 For an excellent discussion of the authenticity and preservation of internet sources, see generally Matt Novak, Legal Research in the Digital Age: Authentication and Preservation of Primary Material, Neb. Law. 19–25 (July/August 2010, (available at http://digitalcommons.unl.edu/cgi/viewcontent.cgi?article=1003&context=lawlibrary). Issues of accuracy and reliability must, however, be considered. See generally Mills, supra n. 16, for a comprehensive discussion of the problems of accurate case reports in the “digital age.”
19 Carino v. Muenzen, 2010 N.J. Super. Unpub. LEXIS 2154 at **24–27 (App. Div. Aug. 30, 2010) (addressing the issue of permissibility of Internet access by counsel during jury selection when no prior notice is given to the court and opposing counsel, and when opposing counsel opts not to use similar technology). Recognizing the ever-increasing role of the Internet in the life of a litigator, the ABA recently offered a course entitled iPad for Litigators, focusing on issues ranging from conducting legal research and depositions to selecting juries and presenting trial evidence. The course sold out for its January 19, 2012, debut. See Am. L. Inst., ABA, Course Details, iPad for Litigators, http://www.ali-aba.org/index.cfm?fuseaction= courses.course&course_code=RSTP13 (last accessed Mar. 4, 2012).
20 A striking example of the move toward internet-based information was the announcement in June 2011 that the state of Wyoming “had gone Google.” The Casper Journal reported: “Commemorating a Wyoming first, Gov. Matt Mead, Wyoming’s Chief Information Officer, Flint Waters and the President of Enterprise at Google, David Girouard, announced on June 22 that all 10,000 of Wyoming’s state employees have now migrated to Google Apps for Government.” Dale Bohren, Wyoming Goes “Google,” Casper Journal.com, http://www.casperjournal.com/news/article_a51684fb-f167-58b4-bc0c- 2343446e3884.html (June 26, 2011).
21 Id.; see also Richard J. Matthews & Mary Alice Baish, State–by–State Report on Authentication of Online Legal Resources, 10–11 (2007), http://www.aallnet.org/Documents/Government-relations/authen_rprt/authenfinalreport.pdf (including detailed discussion of the growing trend among states to offer online sources as the sole versions of official legal resources such as cases and state statutes); see generally Carole Levitt & Mark Rosch, Internet for Lawyers, www.netforlawyers.com (last accessed Mar. 4, 2012).
22 See e.g. Nicole Perlroth, Digital Data on Patients Raises Risk of Breaches, N.Y. Times (Dec. 18, 2011) (available at http://www.nytimes.com/2011/12/19/technology/as-patient-records-are-digitized-data-breaches-are-on-the-rise.html); Patricia Salkin & Howard F. Gross, Paperless Governments Moving Towards Sustainability, Alb. L. Sch. Leg. Stud. Research Paper Series No. 27 (2011–2012), 1–6 (available at http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1932288). On a slightly different but related note, see also the Berkeley Electronic Press (www.bepress.com), established in 1999, which publishes high-quality, peer-reviewed journals only in electronic format, and familiar Social Science Research Network (SSRN) (www.ssrn.com), a worldwide collaborative of over 178,000 authors and more than 1.3 million users that distributes abstracts and full text papers from scholars around the world.
23 Margolis, Surfin’ Safari, supra n. 1, at 117–18; Leslie A. Street & Amanda M. Runyon, Finding the Middle Ground in Collection Development: How Academic Law Libraries Can Shape Their Collections in Response to the Call for More Practice- Oriented Legal Education 102 L. Lib. J. 399, 413–14 (2010).
24 Margolis, Surfin’ Safari, supra n. 1, at 114–15; ABA Report, supra n. 4, at 31, 35.
25 See generally Coggins, supra n. 6.
26 See e.g. Street & Runyon, supra n. 23, at 411–16; Gallacher, Aux Armes, Citoyens, supra n. 3, at 9–13; Margolis, Surfin’ Safari, supra n. 1 at 114.
27 Margolis, Surfin’ Safari, supra n. 1, at 114–15.
28 ABA Report, supra n. 5, at iv–vi.
29 Although individual respondents often include product names in their survey answers, the ABA Report focuses on usage rather than on product name. Id. at vi.
30 See generally id.
31 Id. at vi–viii. The survey also includes demographic data on topics ranging from billing practices, to office IT support, to gender and age. Id. As a result, the survey provides useful data on topics well outside the scope of this article.
32 Id. at 21 (These and other 2011 statistics regarding legal research practices are from vol. 5 of the Report). The 2010 survey reported that 80.4% of respondents turned to online research as the first step. Id.
34 Id. 84.8% of attorneys begin their research online; of that 84.8%, 56.1% use a free online source. Id.
35 Leg. Info. Inst., www.law.cornell.edu (last accessed Mar. 13, 2012).
36 ABA Report, supra n. 4, at 21.
37 Id. at 21.
40 The 2010 survey reported that 80.4% of respondents turned to online research as the first step. Id. Compare with the 2006 ABA results reported by Margolis, Surfin’ Safari, supra n. 1 at 108, in which only 24% of respondents started their research with a free, general search engine.
41 Id. at 24. The choices were (a) online free, (b) online feebased, (c) print, (d) CD-ROM, and (e) do not use.
44 Id.at 31.
45 Id. at.26.
47 Id. at.34.
48 Id. at 32.
49 Id. at 34.
50 Id. at 33.
51 See id. at 26, 32, 34.
52 See id. at 24–33.
53 For several excellent examples see supra n. 2.
54 For two detailed examples, see generally Erika V. Wayne & J. Paul Lomio, Book Lovers Beware: A Survey of Online Research Habits of Stanford Law Students, Robert Crown L. Lib. Leg. Research Paper Series (June 2005) and Kaplan & Darvil, supra n. 1.
55 Kaplan & Darvil, supra n. 1, at 167.
56 Ian Gallacher, “Who Are Those Guys?”: The Results of a Survey Studying the Information Literacy of Incoming Law Students, 44 Cal. W. L. Rev. 151, 192 (2007).
57 Wayne & Lomio, supra n. 54, at 4–5.
58 Id. at 3.
59 Id. at 14–15.
61 Id. at 15. For additional commentary on this study, see Gallacher, “Who are Those Guys?” supra n. 56 at 189–92.
62 Consider the Ohio Revised Code as an example. The official online location of the Ohio revised code (unnannotated) is http://codes.ohio.gov/orc; the site is maintained by Lawriter. Westlaw’s version of the “official” code is called Baldwin’s Ohio Revised Code Annotated. (While these Revised Codes may be certified by the Ohio Secretary of State, they are actually not the official statutes of Ohio. The session laws are the official statutes. See ORC 1.53, General Assembly’s Web Page, http://www.legislature.state.oh.us/laws.cfm (last accessed Mar. 27, 2012). In addition, both official and unofficial pages are often accessed at http://www.megalaw.com/oh/ohcode.php. And, as a final note, the Ohio Revised Code also has a facebook page, http://www.facebook.com/pages/Ohio-Revised-Code/132623190107987 (last accessed Mar. 13, 2012).
63 Kaplan & Darvil’s exhaustive study looks at the research habits of upper-level students in both school and work environments. The study provides data on students’ use of print sources, written research plans, and free and fee-based sources, a consideration of research costs, an evaluation of internet sources, and first steps in a research project. Kaplan & Darvil, supra n. 1, at 165. The authors noted that many of their findings mirror law-firm studies, such as the ABA Report. Id.
65 Id. at 167.
67 Id. On a related note, the ABA reports that between 25 and 38% of Internet researchers are not satisfied with their ability to ascertain the credentials of a given source’s author or publisher. ABA Report, supra n. 4, at 41. I disagree with Kaplan & Darvil’s assertion that “[i]n a professional environment, lawyers do not rely on websites like Wikipedia to find and interpret the law.” Kaplan & Darvil, supra n. 1, at 167. I agree that lawyers are not using sources such as Wikipedia to find and interpret the law if we define the “law” simply as primary authority. But I would argue, and the ABA statistics would support, that lawyers must and do rely on a range of Internet sources to perform their jobs competently. As the cases in section II.B below demonstrate, if relevant material is available on the Internet, an attorney may very well have a duty to research via the Internet.
68 Kaplan & Darvil, supra n. 1, at 167–68.
69 Id. at 167.
70 Id. at 167; id. at 167 n. 84.
71 Id. at 167.
72 See e.g. Van Westrienen v. Americontinenetal Collection Corp., 94 F. Supp. 2d 1087, 1109 (D. Or. 2000) (holding that a website may be considered admissible evidence of a party-opponent, and not barred by the hearsay rule); Perfect 10, Inc., v. Cybernet Ventures, Inc., 213 F. Supp. 2d 1146, 1155 (C.D. Cal. 2002) (holding that printouts from a website are admissible pursuant to the best evidence rule); see also Telewizja Polska USA, Inc. v. Echostar Satellite Corp., 2004 U.S. Dist. LEXIS 20845 (N. D. Ill., 2004) (same).
73 See generally Van Westrienen, 94 F. Supp. 2d 1087; Perfect 10, 213 F. Supp. 2d 1146; Telewizja Polska, 2004 U.S. Dist. LEXIS 20845.
74 Margolis, Authority without Borders, supra n. 3, at 912; Frederick Schauer & Virginia J. Wise, Nonlegal Information and the Delegalization of Law, 29 J. Leg. Stud. 495, 500–03, 501 tbl.1 (2000); John J. Hasko, Persuasion in the Court: Nonlegal Materials in U.S. Supreme Court Opinions, 94 L. Lib. J. 427, 429–31, 431 tbl.1 (2002); William R. Wilkerson, The Emergence of Internet Citations in U.S. Supreme Court Opinions, 27 Just. Sys. J. 323, 325 (2006); Michael Whiteman, The Death of Twentieth-Century Authority, 58 UCLA L. Rev. Discourse 27, 42–47 (2010).
75 Margolis, Authority Without Borders, supra n. 3, at 941.
76 See supra text accompanying nn. 40–43.
77 Margolis, Authority Without Borders, supra n. 3, at 188.
78 Br. in Support of Petr., Christian Leg. Soc’y Ch. of U. of Cal., Hastings College of the L. v. Nell Newton, 2009 WL 1265294 at *35 (9th Cir. May 5, 2009).
80 See Margolis, Surfin’ Safari, supra n. 1, at 117–18.
81 Carole Levitt and Mark Rosch are two attorneys who have written and presented widely on the increasingly relevant issue of whether an attorney actually has a “duty to Google.” Levitt & Rosch, A Cybersleuth’s Guide, supra n. 15, at About the Authors page; Carole Levitt & Mark Rosch, Computer Counselor: Making Internet Searches Part of Due Diligence, 29 L.A. Law. 46 (Feb. 2007).
82 460 F.3d at 95.
83 Id. at 95.
85 Id. at 98–99.
86 Id. at 102–03.
87 SeeMunster v. Groce, 829 N.E.2d 52 (Ind. Ct. App. 2005). The court held that plaintiff had not perfected service in having failed to use due diligence to find an address for defendant. In reaching its decision, the court indicated that it had found information pertaining to the defendant’s whereabouts by simply entering the name into “Google.” Id. at 61–62, n. 3.
88 Carino v. Muenzen, 2010 N.J. Super. Unpub. LEXIS 2154 at *9, **26–27 (App. Div. Aug. 30, 2010).
89 Id. at **9–10.
90 Id. at **26–27.
92 See supra n. 9.
93 A striking example of this is a recent article that advocates for a new type of literacy—labeled “neteracy” (short for “Internet literacy”) and is described as a “skill set and a mentality appropriate to our new technological environment.” Bernard J. Hibbitts, The Technology of Law, 102 L. Lib. J. 101, 105 (2010). Hibbitts notes that “neteracy” has caught on among scholars of English composition, one of whom, Aaron Barlow, “described neteracy as being” comfortable with the idea that, one way or another, we can handle most anything we find on our screens. We can judge data and websites at the flick of an eye, picking up subliminal clues that tell us the level of expertise involved. We can tell at a glance what links to follow, whether we are being lured into a commercial morass or might be heading towards a new gem. Id. at 106 (quoting Aaron Barlow, The Medium Is the Process: The Process Is the Message, ePluribus Media, http:// www.epluribusmedia.org/columns/2007/20070711_process_medium.html (posted July 12, 2007)). For teachers of legal research, Barlow’s views would almost surely appear overly optimistic—experience suggests that law students are not able to evaluate the Internet with mere “glances.” And although Hibbitts himself acknowledges that evaluation of online sources is critical, he stops short of offering a substantive process for evaluating them. Instead, he offers the following questions: Does the comprehended information seem accurate? Should it be trusted? Does it—or do its authors—need to be queried or tested before materials can be believed? Again the key here is for students, or anyone else for that matter, to take responsibility for critical analysis of sources instead of leaving that largely to others. Id. at 108. Hibbitts argues that students must take responsibility for critical analysis of sources, but does not offer a meaningful process to do so. Id.
94 See supra n. 13.
95 See Sec’y of the Commonwealth, Pub. Recs. Div., The History of the Arms and Great Seal of the Commonwealth of Massachusetts, http://www.sec.state.ma.us/pre/presea/sealhis.htm (last accessed Mar. 13, 2012).
96 When I show the image to my students, many indicate that they have seen it before and, when I remind them that it was the seal of the Massachusetts Bay Colony, many more express some familiarity with it.
97 Throughout this part of the article, I draw on comments made during classroom discussion in four separate classes: Legal Research and Writing (a first semester legal writing course for first-year law students), Advocacy (a second-semester persuasive writing class for first-year law students), Intensive Practical Lawyering (a general lawyering skills class for upperlevel law students), and Pre-Trial Litigation.
98 Cathy Rex, Indians and Images: The Massachusetts Bay Colony Seal, James Printer; and the Anxiety of Colonial Identity, 63 Am. Q. 62–63 (Mar. 2011) (emphasis added).
99 See supra n. 98.
100 The web page I use actually figured into a case on which I worked while in private practice. A younger associate had gathered information from this particular page and could not figure out why a “state operated” web page could be so biased. Remembering this experience, I decided to use this document in my class.
101 Cal. Bus. & Prof. Code §§ 17200–17210 (Deering 2010).
102 The version of the web page I have used is from 2008. A recent check of the webpage reveals that the formatting has changed in several ways; for example, the author’s identifying information is now located on the top left-hand corner of the front page. Kimberly A. Kralowec, The UCL Practitioner, http://www.uclpractitioner.com (last accessed Mar. 14, 2012).
103 Id. at Useful Links for Lawyers, http://www.uclpractitioner.com/links.html (last accessed Mar. 14, 2012).
104 Id. at Posts categorized “Class Action ‘Fairness’ Act,” http://www.uclpractitioner.com/class_action_fairness_act/page/2/ (last accessed Mar. 14, 2012). When I first started using the web page in class, the CAFA link included language that stated: “The legislation often deprives Americans of legal recourse when they are wronged by powerful corporations. The bill makes it far more difficult to bring class action suits, and may prolong such litigation, clogging the federal courts’ dockets.” Although this language no longer appears on the webpage, it is available (almost verbatim) on Wikipedia’s CAFA page. Wikipedia, Class Action Fairness Act of 2005, http://en.wikipedia.org/wiki/Class_Action_Fairness_Act_of_2005 (last modified Dec. 8, 2011).
105 This article is in no way intended to be a judgment about the validity of the web-page author’s opinions with regard to CAFA. Since its passage in 2005, the merits of CAFA have been debated by scholars and practitioners with strong arguments both for and against the fairness of its provision.
106 See supra n. 13.
107 Herbert S. Klein & Jacques A. Barbier, Recent Trends in the Study of Spanish American Colonial Public Finance, 23 Latin Am. Research Rev. 35, 40–41 (1988) (census used for tax collection); W. George Lovell, Christopher H. Lutz & William R. Swezey, The Indian Population of Southern Guatemala, 1549–1551: An Analysis of López de Cerrato’s Tasaciones de Tributos, 40 The Ams. 459, 466 (Apr. 1984).
108 Lovell, Lutz & Swezey, supra n. 108, at 466.
109 See supra n. 103.
110 Id. I inform my students (and want to be clear in this article) that I am not criticizing the author of the webpage for advertising her legal services. The purpose of the exercise is to get students to recognize how a web site’s goals are connected to the information being offered.
111 I also use the somewhat notorious example of the “fake” World Trade Organization website (www.gatt.com), which has been confused by lawyers and even law schools with the real WTO website (located at www.wto.org). The students were able to distinguish the fake by searching for the document’s hidden agenda and purpose.
112 See supra n. 13.
113 For an interesting discussion of how extremely biased web pages hide their messages in “official” webpage packaging, see Michel Marriott, Rising Tide: Sites Born of Hate, N.Y. Times, http://www.nytimes.com/1999/03/18/technology/rising-tidesites- born-of-hate.html?pagewanted=all&src=pm (Mar. 18, 1999).
114 Mary Guyatt, The Wedgewood Slave Medallion, 13 J. of Design History 93, 93 (2000). Variations on this medallion were numerous during the 1700s and 1800s; even though students may not have seen the specific example that I show in class, many have seen a variation in a high-school or college history textbook.
115 See supra n. 98.
116 Guyatt, supra n.115, at 99–100.
117 Guyatt writes, [L]argely unaware of the brutal plantation system, the plight of the enslaved individual was sold to the public through this image, quite literally in-fact, as the figure was reproduced upon a variety of items of jewelry and pottery. This consumption is significant for understanding the image, as in the eighteenth century, the nascent age of consumerism, to possess the fashionable marked the individual as part of society, as possessing its social mores and ethics. Those who wore the medallions, cufflinks or broaches, and ate off or displayed the plates, were therefore not expressing solidarity with those who were enslaved; they were expressing support and association with the largesse of the abolitionists. Guyatt, supra n. 115, at 99–100.
118 See supra n. 98.
119 Alan Olson, Attorneys for Employee Rights, Post-Employment Competition, http://www.employee-advocates.com/ PracticeAreas/Post-Employment-Competition.asp (last accessed Mar. 14, 2012); Godfrey & Kahn, Publications, “One More Victory for Employers in the Non-Compete Agreement Realm,” http://www.gklaw.com/news.cfm?action=pub_detail&publication_ id=1005 (last accessed Mar. 14, 2012).
120 Olson, supra n. 121; Godfrey & Kahn, supra n. 120.
121 Godfrey & Kahn, supra n. 120.
122 Olson, supra n. 120.
123 Generating a simple discussion about the need to evaluate the bias of law-firm web pages and blogs (their audience, their strategies, etc.) is particularly important since these sources are proliferating at a rapid rate. See Lisa Smith-Butler, Cost Effective Legal Research Redux: How to Avoid Becoming the Accidental Tourist, Lost in Cyberspace, 9 Fla. Coastal L. Rev. 293, 338–42 (2008); Margolis, Surfin’ Safari, supra n. 1, at 116–17.
124 See supra n. 13.
125 For example, a student raised during the 1970s will have a different perspective, or bias, than a student raised during the 1990s. Students who paid for their own college tuition may have a different bias than students who did not. A student from the rural Midwest may have a different bias than a student from the urban Northeast. Moreover, one’s political, social, and religious beliefs may have an impact on one’s evaluation of documents and other source material. Once a student identifies the biases through which information is inevitably filtered, it is possible to use almost any source for the particular legal purpose at hand.
126 For a wonderful discussion of how a person’s identity and experiences influence his or her understanding of seemingly “neutral” historical documents, see generally Sam Wineburg, Historical Thinking and Other Unnatural Acts, 80 Phi Delta Kappan 488–99 (1999).
127 Wis. Historical Soc’y, Turning Points in Wisconsin History, http://content.wisconsinhistory.org/cdm4/document.php? CISOROOT=/tp&CISOPTR=51382&CISOSHOW=51380&REC=8 (last accessed Mar. 27, 2012).
128 See supra n. 98.
130 Dariano v. Morgan Hill Unified Sch. Dist., 2011 U.S. Dist. LEXIS 130834 (N.D. Cal. Nov. 8, 2011).
131 Id. at **3–5
133 Id. at *5.
134 393 U.S. 503 (1969).
135 Dariano, 2011 U.S. Dist. LEXIS 130834 at **21–23. The Court held that the Tinker analysis also provided officials with a legitimate nondiscriminatory reason (preventing disruption) for banning only the American-flag t-shirts, negating the students’ equal-protection claim. Id. at **23–26.
136 See e.g. Todd Starnes, Judge: School Can Ban American Flag Shirts, http://radio.foxnews.com/toddstarnes/topstories/ judge-school-can-ban-american-flag-shirts.html (Nov. 10, 2011) (largely criticizing the decision); Ruben Navarrette, When Wearing a U.S. Flag T-shirt is Wrong, http://www.cnn.com/2011/11/18/opinion/navarrette-tshirt- controversy/index.html (Nov. 18, 2011) (largely supporting the decision).
137 Ltr. from ACLU to Dr. Wesley Smith, Superintendent, Morgan Hill Unified Sch. Dist., Live Oak High School Censorship on May 5, 2010, (May 10, 2010) (available at http://www.aclunc.org/cases/other_legal_advocacy/ asset_upload_file776_ 9139.pdf).
139 Other student statements revealed a conflict between their own biases. For example, several students stated that they were inclined to support free speech in every context, but, based on other, equally important, personal beliefs, they were not inclined to support the student plaintiffs in the Morgan Hill case. In brief, these students believed that wearing the American Flag t-shirts was disrespectful of Hispanic culture—and therefore, regardless of their presuppositions (or not) in favor of free speech, they were biased in favor of sources that criticized the Plaintiff students.
140 Navarrette, supra n. 137.
141 Lawyer misconduct clearly results if one’s biases result in “actions that are prejudicial to the administration of justice.” See comment 3 to Rule 8.4 of the Model Rules of Professional Conduct, which states, A lawyer who, in the course of representing a client, knowingly manifests by words or conduct, bias or prejudice based upon race, sex, religion, national origin, disability, age, sexual orientation or socioeconomic status, violates paragraph (d) when such actions are prejudicial to the administration of justice. Legitimate advocacy respecting the foregoing factors does not violate paragraph (d). A trial judge’s finding that peremptory challenges were exercised on a discriminatory basis does not alone establish a violation of this rule. Model R. Prof. Conduct 8.4 (comment 3). The purpose of the exercise described in this article is to prevent this type of bias by helping students to recognize their own internal presuppositions.
The American Historical Association’s professional standards explicitly recognize the need to recognize one’s biases. “Professional integrity in the practice of history requires awareness of one’s own biases.” Am. Historical Ass’n, supra n. 13, at 3. Scholarship, http://www.historians.org/pubs/free/ProfessionalStandards.cfm. “Everyone who comes to the study of history brings with them a host of identities, experiences, and interests that cannot help but affect the questions they ask of the past and the answers they wish to know. When applied with integrity and self-critical fair-mindedness, the political, social, and religious beliefs of historians can appropriately inform their historical practice.” Id. at 2. Shared Values of Historians.
142 See supra nn. 74–77.
143 See supra n. 13.
144 Gallacher, Aux Armes, Citoyens, supra n. 3, at 19–22.
145 Suzanne E. Rowe, Legal Research, Legal Writing, and Legal Analysis: Putting Law School into Practice 29 Stetson L. Rev. 1193, 1201 (2000); see also Margolis, Authority Without Borders, supra n. 3, at 925–27.
146 Margolis, Authority Without Borders, supra n. 3, at 926–28.
147 See supra nn. 18–19.
148 I do find it useful to provide a few sources that allow students to trace the connections between online sources. For example, www.alexa.com allows the user to type in a URL and see “traffic” details for the selected web page, contact and ownership information for the domain name, other sites linking in to the designated web page, and “related” links to other sites visited by users who visited the selected webpage. Archived information about a particular webpage can be found at sites such as the Wayback Machine, now referred to as the Internet Archive (http://www.archive.org/web/web.php). Finally, Google and other search engines provide web page links by typing “link:” into the search box followed immediately by the URL of the selected site.
149 Kaplan & Darvil, supra n. 1, at 177.
150 Id. (quoting Ass’n of College & Research Lib., Presidential Committee on Information Literacy: Final Report (Jan. 10, 1989) (available at http://www.ala.org/acrl/publications/whitepapers/presidential).
152 Id. at 178.
154 This quote, often attributed to Yeats, is actually a perfect example for the necessity of the approach outlined in this article. Numerous (arguably reputable) Internet sources attribute the quote to Yeats—though no source for the quote is provided. See e.g. Nat’l Educ. Ass’n, Sample Quotes, Education Quotes, http://www.nea.org/grants/35593.htm (last accessed Apr. 4, 2012). Other sources suggest that the quote has been misattributed to Yeats. See e.g. i202 Fall 2009 Sch. Of Info., UC Berkeley, “Education is not filling a bucket, but lighting a fire,” http://courses.ischool.berkeley.edu/i202/f09/blog/educationnot- filling-a-bucket-lighting-a-fire (last accessed Apr. 4, 2012). Rather than simply accept the attribution, and thereby possibly contributing to the quote’s potential misattribution, adopting the critical approach to Internet sources described here would give students a more nuanced sense of whether the quote is one they might want to use in a document and, if so, how they might treat it.