dissertation citation generator

Plagiarism Checker

Compare your paper to billions of pages and articles with Scribbr’s Turnitin-powered plagiarism checker.

Run a free check

AI Detector

Detect AI-generated content like ChatGPT3.5, GPT4 and Gemini in seconds

Try for free

Check your Citations

Improve your in-text citations and references for errors and inconsistencies using Scribbr's AI technology or human experts.

Proofreading & Editing

Have a human editor polish your writing to ensure your arguments are judged on merit, not grammar errors.

Get expert writing help

universalSourceForm.defaults.intro.title

universalSourceForm.overwrites.thesis.intro.text,universalSourceForm.defaults.intro.text

Home / Guides / Citation Guides / APA Format / How to Cite a Thesis or Dissertation in APA

How to Cite a Thesis or Dissertation in APA

In this citation guide, you will learn how to reference and cite an undergraduate thesis, master’s thesis, or doctoral dissertation. This guide will also review the differences between a thesis or dissertation that is published and one that has remained unpublished. The guidelines below come from the 7th edition of the Publication Manual of the American Psychological Association (2020a), pages 333 and 334. Please note that the association is not affiliated with this guide.

Alternatively, you can visit EasyBib.com for helpful citation tools to cite your thesis or dissertation .

Guide Overview

Citing an unpublished thesis or dissertation, citing a published dissertation or thesis from a database, citing a thesis or dissertation published online but not from a database, citing a thesis or dissertation: reference overview, what you need.

Since unpublished theses can usually only be sourced in print form from a university library, the correct citation structure includes the university name where the publisher element usually goes.

Author’s last name, F. M. (Year published). Title in sentence case [Unpublished degree type thesis or dissertation]. Name of institution.

Ames, J. H., & Doughty, L. H. (1911). The proposed plans for the Iowa State College athletic field including the design of a reinforced concrete grandstand and wall [Unpublished bachelor’s thesis]. Iowa State University.

In-text citation example:

• Parenthetical :  (Ames & Doughty, 1911)
• Narrative :  Ames & Doughty (1911)

If a thesis or dissertation has been published and is found on a database, then follow the structure below. It’s similar to the format for an unpublished dissertation/thesis, but with a few differences:

• The institution is presented in brackets after the title
• The archive or database name is included

Author’s last name, F. M. (Year published). Title in sentence case (Publication or Document No.) [Degree type thesis or dissertation, Name of institution]. Database name.

Examples 1:

Knight, K. A. (2011). Media epidemics: Viral structures in literature and new media (Accession No. 2013420395) [Doctoral dissertation, University of California, Santa Barbara]. ProQuest Dissertations Publishing.

Trotman, J.B. (2018). New insights into the biochemistry and cell biology of RNA recapping (Document No. osu1523896565730483) [Doctoral dissertation, Ohio State University]. OhioLINK Electronic Theses & Dissertations Center.

In the example given above, the dissertation is presented with a Document Number (Document No.). Sometimes called a database number or publication number, this is the identifier that is used by the database’s indexing system. If the database you are using provides you with such a number, then include it directly after the work’s title in parentheses.

If you are interested in learning more about how to handle works that were accessed via academic research databases, see Section 9.3 of the Publication Manual.

In-text citation examples :

• Parenthetical citation : (Trotman, 2018)
• Narrative citation : Trotman (2018)

Author’s last name, F. M. (Year Published). Title in sentence case [Degree type thesis or dissertation, Name of institution]. Name of archive or collection. URL

Kim, O. (2019). Soviet tableau: cinema and history under late socialism [Doctoral dissertation, University of Pittsburgh]. Institutional Repository at the University of Pittsburgh. https://d-scholarship.pitt.edu/37669/7/Olga%20Kim%20Final%20ETD.pdf

Stiles, T. W. (2001). Doing science: Teachers’ authentic experiences at the Lone Star Dinosaur Field Institute [Master’s thesis, Texas A&M University]. OAKTrust. https://hdl.handle.net/1969.1/ETD-TAMU-2001-THESIS-S745

It is important to note that not every thesis or dissertation published online will be associated with a specific archive or collection. If the work is published on a private website, provide only the URL as the source element.

In-text citation examples:

• Parenthetical citation : (Kim, 2019)
• Narrative citation : Kim (2019)
• Parenthetical citation : (Stiles, 2001)
• Narrative citation : Stiles (2001)

We hope that the information provided here will serve as an effective guide for your research. If you’re looking for even more citation info, visit EasyBib.com for a comprehensive collection of educational materials covering multiple source types.

If you’re citing a variety of different sources, consider taking the EasyBib citation generator for a spin. It can help you cite easily and offers citation forms for several different kinds of sources.

To start things off, let’s take a look at the different types of literature that are classified under Chapter 10.6 of the Publication Manual :

• Undergraduate thesis
• Master’s thesis
• Doctoral dissertation

You will need to know which type you are citing. You’ll also need to know if it is published or unpublished .

When you decide to cite a dissertation or thesis, you’ll need to look for the following information to use in your citation:

• Author’s last name, and first and middle initials
• Year published
• Title of thesis or dissertation
• If it is unpublished
• Publication or document number (if applicable; for published work)
• Degree type (bachelor’s, master’s, doctoral)
• Thesis or dissertation
• Name of institution awarding degree
• DOI (https://doi.org/xxxxx) or URL (if applicable)

Since theses and dissertations are directly linked to educational degrees, it is necessary to list the name of the associated institution; i.e., the college, university, or school that is awarding the associated degree.

To get an idea of the proper form, take a look at the examples below. There are three outlined scenarios:

• Unpublished thesis or dissertation
• Published thesis or dissertation from a database
• Thesis or dissertation published online but not from a database

American Psychological Association. (2020a). Publication manual of the American Psychological Association (7th ed.). https://doi.org/10.1037/0000165-000

American Psychological Association. (2020b). Style-Grammar-Guidelines. https://apastyle.apa.org/style-grammar-guidelines/citations/basic-principles/parenthetical-versus-narrative

Published August 10, 2012. Updated March 24, 2020.

Written and edited by Michele Kirschenbaum and Elise Barbeau. Michele Kirschenbaum is a school library media specialist and the in-house librarian at EasyBib.com. Elise Barbeau is the Citation Specialist at Chegg. She has worked in digital marketing, libraries, and publishing.

APA Formatting Guide

APA Formatting

• Annotated Bibliography
• Block Quotes
• et al Usage
• In-text Citations
• Multiple Authors
• Paraphrasing
• Page Numbers
• Parenthetical Citations
• Reference Page
• Sample Paper
• APA 7 Updates
• View APA Guide

Citation Examples

• Book Chapter
• Journal Article
• Magazine Article
• Newspaper Article
• Website (no author)
• View all APA Examples

How useful was this post?

Click on a star to rate it!

We are sorry that this post was not useful for you!

Let us improve this post!

Tell us how we can improve this post?

To cite a published thesis in APA style, it is important that you know some basic information such as the author, publication year, title of the thesis, institute name, archive name, and URL (uniform resource locator). The templates for an in-text citation and reference list entry of a thesis, along with examples, are given below:

In-text citation template and example:

Use the author surname and the publication year in the in-text citation.

Author Surname (Publication Year)

Cartmel (2007)

Parenthetical:

(Author Surname, Publication Year)

(Cartmel, 2007)

Reference list entry template and example:

The title of the thesis is set in sentence case and italicized. Enclose the thesis and the institute awarding the degree inside brackets following the publication year. Then add the name of the database followed by the URL.

Author Surname, F. M. (Publication Year). Title of the thesis [Master’s thesis, Institute Name]. Name of the Database. URL

Cartmel, J. (2007). Outside school hours care and schools [Master’s thesis, Queensland University of Technology]. EPrints. http://eprints.qut.edu.au/17810/1/Jennifer_Cartmel_Thesis.pdf

To cite an unpublished dissertation in APA style, it is important that you know some basic information such as the author, year, title of the dissertation, and institute name. The templates for in-text citation and reference list entry of an online thesis, along with examples, are given below:

Author Surname (Year)

Averill (2009)

(Author Surname, Year)

(Averill, 2009)

The title of the dissertation is set in sentence case and italicized. Enclose “Unpublished doctoral dissertation” inside brackets following the year. Then add the name of the institution awarding the degree.

Author Surname, F. M. (Publication Year). Title of the dissertation [Unpublished doctoral dissertation]. Name of the Institute.

Averill, R. (2009). Teacher–student relationships in diverse New Zealand year 10 mathematics classrooms: Teacher care [Unpublished doctoral dissertation]. Victoria University of Wellington.

APA Citation Examples

Writing Tools

Citation Generators

Other Citation Styles

Plagiarism Checker

Upload a paper to check for plagiarism against billions of sources and get advanced writing suggestions for clarity and style.

Get Started

Fast and free citation generator APA 6th and 7th ed. • MLA 8th ed. • Chicago 16th ed.

• Create Title Page
• Style Guide
• Manage Bibliographies

Mindfullness & COVID-19

JavaScript is off. Please enable to use site.

Note: The default citation style is now APA 7. To use APA 6 ensure that the APA 6 button is selected.

Citefast had an sql injection attack on May 29, 2021 which deleted all saved citations and accounts. We apologize for the inconvenience this might cause you. We are working on ensuring does not happen again.

• Social media
• Online video
• Dissertation/Thesis
• Encyclopedia/Dictionary
• Government publication
• Audio/Podcast
• Television/Radio

Current citation (not saved)

You have no saved citations in this bibliography .

Want to keep your citations for longer and access them from other computers? It's easy and free! Sign up.

Copy the information below in your paper according to the Guide on the right. Use your own page numbers.

APA 7 In-text citation guide

In-text citations are required when you use someone else's ideas, theories or research in your paper.

Quick Guide

Examples: (choose depending if author and/or date is mentioned in text)

Quotation :

• "The bones were very fragile" (Cole, 2019, p. 13).
• Cole (2019) found that "The bones were very fragile" (p. 33).
• In 2019, Cole found that "The bones were very fragile" (p. 33).

Paraphrase :

• The bones broke easily because they were porous (Cole, 2011).
• Cole (2011) discovered that the bones broke easily.
• In 2011, Cole found that the bones were easily broken (p. 33).

Note: APA style encourages the inclusion of page numbers for paraphrases, but it is not mandatory. Include page or paragraph numbers if it will help reader find the information.

No authors : Use the title in place of author. Shorten title if needed. Use double quotation marks for title of an article, a chapter, or a web page. Use italics for title of a periodical, a book, a brochure or a report.

• the observations found ("Arctic Voyage," 2014)
• the book Vitamin Discoveries (2013)

Two authors : Within the text use the word  and .  If the authors' names are within parentheses use the  &  symbol.

• Cole and Dough (1998) argued ...
• ...if they were left to their own devices.(Cole & Dough, 1998)

Three or more authors:   Include only the last name of the first author followed by "et al."

(Wasserstein et al., 2017)

Spell out the name in full the first time and abbreviate subsequent times only if abbreviation is well known.

• First time: American Psychological Association (2020) explained...
• Second time: APA (2020) proved ...

When quoting always provide author, year and specific page citation or paragraph number for nonpaginated material.

If the quotation is less than 40 words incorporate it into the text and enclose the quotation with quotation marks. Cite the source immediately after the close of the quotation marks.

If the authors are named in the text, they do not have to be used in the citation.

In fact, "a neurosis is characterized by anxiety" (Kristen & Warb, 2012, p. 157).

"A neurosis is characterized by anxiety," according to Kristen and Warb's (2012, p. 157) longitudinal study.

If the quotation is over 40 words, you must indent the entire quotation and start the quotation on a new line. No quotation marks are required. Cite the quoted source after the final punctuation mark.

Alberta is occasionally divided into two regions, Northern Alberta and Southern Alberta. The majority of Alberta's population is located in large urban cities, mostly located in the South. Alberta is Canada's most populous province of all three Canadian Prairie provinces. Edmonton is the Capital of Alberta. (Hern, 1996, p. 22)

Paraphrasing

APA style encourages the inclusion of page numbers, but it is not mandatory. Include page or paragraph numbers if it will help reader find the information.

• (Reiton, 2003, para. 3)

If the document does not contain page numbers, include paragraph numbers.

• (Reiton, 2003, para. 3).   

If neither is available omit page and paragraph numbers.  Do not count paragraph numbers.  

When paraphrasing from multiple sources, include all authors name in parentheses in alphabetical order.

• (Cole, 2006; Mann & Arthur, 2011; Zigmung, 2000).

APA In-Text Citation Guide

• "The bones were very fragile" (Cole, 2011, p. 13).
• Cole (2011) found that "The bones were very fragile" (p. 33).
• In 2011, Cole found that "The bones were very fragile" (p. 33).

Note: APA style encourages the inclusion of page numbers for paraphrases, but it is not mandatory. Include page or paragraph numbers if it will help reader find the information.)

Two or more authors : Within the text use the word  and .  If the authors' names are within parentheses use the  &  symbol.

Three to five authors : Include all authors' last names the first time the citation is used.  If you use the same citation again within the same paragraph, use only the first last name followed by 'et al'.  If you used the citation again omit the year. 

• First time:   Cole, Dough and Ferris (1998) explained...
• Second time:  Cole et al. (1998) proved ...
• Third time:  Cole et al. demonstrated...

Six or more authors:   Include only the last name of the first author followed by "et al."

(Wasserstein et al., 2010)

• First time: American Psychological Association (1998) explained...
• Second time: APA (1998) proved ...

Alberta is occasionally divided into two regions, Northern Alberta and Southern Alberta. The majority of Alberta's population is located in large urban cities, mostly located in the South. Alberta is Canada's most populous Province of all three Canadian prairie provinces. Edmonton is the Capital of Alberta. (Hern, 1996, p. 22)

In-Text Citations Parenthetical Citations

In-text citations are called parenthetical references in MLA. This involves placing information about the source in parentheses after a quote or a paraphrase. The information in the parenthetical references must match the corresponding information in the list of works cited.

The purpose of parenthetical references is to indicate to readers not only what works you used, but what you used from each source and where in the source you found the material. This can be done by inserting a parenthetical reference in your text at the spot where you have used the source's ideas or words.

You should keep parenthetical references as brief and as few as clarity and accuracy permit.

General Guidelines

• The Soviets were surrounded by enemies (Waters 119).
• Waters argues that the Soviets were surrounded by enemies (119).

Authors – Identification of source

• (Natl. Research Council 15)
• Do not use abbreviations such as ed. or trans.
• ("The evolving internet")
• (Black and Mondoux 123)
• (Eddison, Zhu, and Lalonde)
• (Becker et al. 13)
• (Becker, Lafontaine, Robins, Given, and Rush 13)
• (Feder, The Birth of a Nation 124)

Location of passage within source

• give relevant page number if available
• give volume and page number in a multivolume work
• if citing entire work omit page numbers
• (Louis par. 20)
• film, television, broadcasts cannot be cited by numbers

Placement of parenthetical reference in text

• Cole found that "The bones were very fragile" (33-34).

Alberta is occasionally divided into two regions, Northern Alberta and Southern Alberta. The majority of Alberta's population is located in large urban cities, mostly located in the South. Alberta is Canada's most populous Province of all three Canadian prairie provinces. Edmonton is the Capital of Alberta. (Herick 22)

• In Chicago style, footnotes or endnotes are used to reference pieces of work in the text.
• To cite from a source a superscript number is placed after a quote or a paraphrase.
• Citation numbers should appear in sequential order.
• Each number then corresponds to a citation, a footnote or to an endnote.
• Endnotes must appear on an endnotes page. The page should be titled Notes (centered at top). This page should appear immediately before the bibliography page.
• Footnotes must appear at the bottom of the page that they are referred to.  

Example:   Cole found that "The bones were very fragile" (33-34). 1

Each superscript then refers to a numbered citation in the footnotes or endnotes.

Footnotes/endnotes:

The first time the in-text reference is cited you must include, author's first name, author's last name, title, place of publication, publisher name, year and referenced pages. e.g.

1. James Smith, The first and last war , (New York, Hamilton, 2003), 2.

If the citation has already been cited it may be shortened to author's last name, shortened title, and page referenced number. e.g.

2. Smith, The first , 220-221.

If the citation has been referenced immediately prior, the note may be shortened even further to ibid with the page number. e.g.

3. Ibid., 786.

For each author-date citation in the text, there must be a corresponding entry in the reference list under the same name and date.

An author-date citation in running text or at the end of a block quotation consists of the last (family) name of the author, followed by the year of publication of the work in question. In this context, author may refer not only to one or more authors or an institution but also to one or more editors, translators, or compilers. No punctuation appears between author and date. Abbreviations such as ed. or trans. are omitted.

(Woodward 1987)

(Schuman and Scott 1987)

When a specific page, section, equation, or other division of the work is cited, it follows the date, preceded by a comma. When a volume as a whole is referred to, without a page number, vol. is used. For volume plus page, only a colon is needed. The n in the Fischer and Siple example below indicates "note" (see 14.164 ). The last example shows how one might cite a section of a work that contains no page or section numbers or other numerical signposts—the case for some electronic documents (see 15.8 ).

(Piaget 1980, 74)

(LaFree 2010, 413, 417–18)

(Johnson 1979, sec. 24)

Fowler and Hoyle 1965, eq. 87)

(García 1987, vol. 2)

(García 1987, 2:345)

(Barnes 1998, 2:354–55, 3:29)

(Fischer and Siple 1990, 212n3)

(Hellman 1998, under "The Battleground")

The following features have been recently added to Citefast:

• Edit and delete citations
• Copy and paste functionality
• Citations will be saved for 24 hours
• A short wizard to guide you through the site the first time you use it
• Updated interface for inputting citation information
• Ability to create In-text citations
• In-text guide for APA, MLA and Chicago citations
• Export your bibliography to Word
• Add editor and chapter information to bibliography for book
• More tool tips to make entering data easier

Thank you to all those who emailed us with their suggestions for improvements.

Login to Citefast

Not a member? Sign Up

Create a Citefast account

Already have an account? Login

Login with citation in progress

Do you want to save your citation? It will be saved to the account you login to .

Password assistance

Verifying that it's you.

For your security, we need to verify your identity. We've sent a code to the email . Please enter it below. Remember to check your junk mail folder if you do not see it in your inbox.

Do you want to save the citation you are working on?

Change style?

Change bibliographies.

You will lose the citation you are entering.

Change Account Settings

Reset password, go to citefast for schools - no ads.

• Plagiarism and grammar
• Citation guides

Citation Machine® — Write Smarter

Start a new citation or manage your existing bibliographies.

Scan your paper for plagiarism and grammar errors.

Check your paper for grammar and plagiarism

Catch plagiarism and grammar mistakes with our paper checker

Use Citation Machine® Plus to get smart recommendations!

Scan your paper for unintentional plagiarism and get advanced recommendations for sentence structure, writing style, grammar and more!

• expert check

Know you're citing correctly

No matter what citation style you're using (APA, MLA, Chicago, etc.) we'll help you create the right bibliography

Check for unintentional plagiarism

Scan your paper the way your teacher would to catch unintentional plagiarism. Then, easily add the right citation

Strengthen your writing

Give your paper an in-depth check. Receive feedback within 24 hours from writing experts on your paper's main idea, structure, conclusion, and more.

Find and fix grammar errors

Don't give up sweet paper points for small mistakes. Our algorithms flag grammar and writing issues and provide smart suggestions

See what's included with Citation Machine® Plus

Citation machine® guides & resources, mla format: everything you need to know and more.

Filled with a wide variety of examples and visuals, our Citation Machine® MLA guide will help you master the citation process. Learn how to cite websites, books, journal articles, magazines, newspapers, films, social media, and more!

MLA Citation Generator | Website | Books | Journal Articles | YouTube | Images | Movies | Interview | PDFs

Comprehensive Guide to APA Format

Our Citation Machine® APA guide is a one-stop shop for learning how to cite in APA format. Read up on what APA is, or use our citing tools and APA examples to create citations for websites, books, journals, and more!

APA Citation Generator | Website | Books | Journal Articles | YouTube | Images | Movies | Interview | PDFs

Everything You Need to Know About Chicago Style

Creating citations in Chicago style has never been easier thanks to our extensive Citation Machine® Chicago style guide and tools. Learn about footnotes, endnotes, and everything in between, or easily create citations for websites, books, journal articles, and more!

Chicago Citation Generator | Website | Books | Journal Articles | YouTube | Images | Movies | Interview | PDFs

Citation Machine®’s Ultimate Writing Guides

Whether you’re a student, writer, foreign language learner, or simply looking to brush up on your grammar skills, our comprehensive grammar guides provide an extensive overview on over 50 grammar-related topics. Confused about reflexive verbs, demonstrative adjectives, or conjunctive adverbs? Look no further! Learn about these grammar topics and many, many more in our thorough and easy to understand reference guides!

Citing Sources Guide | Grammar Guide | Plagiarism Guide | Writing Tips

Student Blog

Stay up to date! Get research tips and citation information or just enjoy some fun posts from our student blog.

• Citation Machine® Plus
• Citation Guides
• Chicago Style
• Harvard Referencing
• Terms of Use
• Global Privacy Policy
• Cookie Notice
• DO NOT SELL MY INFO

• Free Tools for Students
• MLA Citation Generator

Free MLA Citation Generator

Generate accurate citations in MLA format automatically, with MyBib!

😕 What is an MLA Citation Generator?

An MLA citation generator is a software tool designed to automatically create academic citations in the Modern Language Association (MLA) citation format. The generator will take information such as document titles, author, and URLs as in input, and output fully formatted citations that can be inserted into the Works Cited page of an MLA-compliant academic paper.

The citations on a Works Cited page show the external sources that were used to write the main body of the academic paper, either directly as references and quotes, or indirectly as ideas.

👩‍🎓 Who uses an MLA Citation Generator?

MLA style is most often used by middle school and high school students in preparation for transition to college and further education. Ironically, MLA style is not actually used all that often beyond middle and high school, with APA (American Psychological Association) style being the favored style at colleges across the country.

It is also important at this level to learn why it's critical to cite sources, not just how to cite them.

🙌 Why should I use a Citation Generator?

Writing citations manually is time consuming and error prone. Automating this process with a citation generator is easy, straightforward, and gives accurate results. It's also easier to keep citations organized and in the correct order.

The Works Cited page contributes to the overall grade of a paper, so it is important to produce accurately formatted citations that follow the guidelines in the official MLA Handbook .

⚙️ How do I use MyBib's MLA Citation Generator?

It's super easy to create MLA style citations with our MLA Citation Generator. Scroll back up to the generator at the top of the page and select the type of source you're citing. Books, journal articles, and webpages are all examples of the types of sources our generator can cite automatically. Then either search for the source, or enter the details manually in the citation form.

The generator will produce a formatted MLA citation that can be copied and pasted directly into your document, or saved to MyBib as part of your overall Works Cited page (which can be downloaded fully later!).

MyBib supports the following for MLA style:

Daniel is a qualified librarian, former teacher, and citation expert. He has been contributing to MyBib since 2018.

Purdue Online Writing Lab Purdue OWL® College of Liberal Arts

Research and Citation Resources

Welcome to the Purdue OWL

This page is brought to you by the OWL at Purdue University. When printing this page, you must include the entire legal notice.

Copyright ©1995-2018 by The Writing Lab & The OWL at Purdue and Purdue University. All rights reserved. This material may not be published, reproduced, broadcast, rewritten, or redistributed without permission. Use of this site constitutes acceptance of our terms and conditions of fair use.

If you are having trouble locating a specific resource please visit the  search page  or the  Site Map . The Citation Chart  provides a detailed overview of MLA Style, APA Style, and Chicago Manual of Style source documentation by category.

Conducting Research

These OWL resources will help you conduct research using primary source methods, such as interviews and observations, and secondary source methods, such as books, journals, and the Internet. This area also includes materials on evaluating research sources.

Using Research

These OWL resources will help you use the research you have conducted in your documents. This area includes material on quoting and paraphrasing your research sources, as well as material on how to avoid plagiarism.

APA Style (7th Edition)

These OWL resources will help you learn how to use the American Psychological Association (APA) citation and format style. This section contains resources on in-text citation and the References page, as well as APA sample papers, slide presentations, and the APA classroom poster.

These OWL resources will help you learn how to use the Modern Language Association (MLA) citation and format style. This section contains resources on in-text citation and the Works Cited page, as well as MLA sample papers, slide presentations, and the MLA classroom poster

Chicago Manual of Style

This section contains information on the Chicago Manual of Style method of document formatting and citation. These resources follow the 17th edition of the Chicago Manual of Style, which was issued in 2017.

Institute of Electrical and Electronics Engineers (IEEE) Style

These resources describe how to structure papers, cite sources, format references, and handle the complexities of tables and figures according to the latest Institute of Electrical and Electronics Engineers (IEEE) guidelines.

American Medical Association (AMA) Style

These resources provide guidance on how to cite sources using American Medical Association (AMA) Style, 10th Ed., including examples for print and electronic sources.

Research Overview

We live in an age overflowing with sources of information. With so many information sources at our fingertips, knowing where to start, sorting through it all and finding what we want can be overwhelming! This handout provides answers to the following research-related questions: Where do I begin? Where should I look for information? What types of sources are available?

Conducting Primary Research

Primary research involves collecting data about a given subject directly from the real world. This section includes information on what primary research is, how to get started, ethics involved with primary research and different types of research you can do. It includes details about interviews, surveys, observations, and analysis.

Evaluating Sources of Information

Evaluating sources of information is an important step in any research activity. This section provides information on evaluating bibliographic citations, aspects of evaluation, reading evaluation, print vs. online sources, and evaluating Internet sources.

Searching Online

This section covers finding information online. It includes information about search engines, Boolean operators, Web directories, and the invisible Web. It also includes an extensive, annotated links section.

Internet References

This page contains links and short descriptions of writing resources including dictionaries, style manuals, grammar handbooks, and editing resources. It also contains a list of online reference sites, indexes for writers, online libraries, books and e-texts, as well as links to newspapers, news services, journals, and online magazines.

Archival Research

This resource discusses conducting research in a variety of archives. It also discusses a number of considerations and best practices for conducting archival research.

This resources was developed in consultation with Purdue University Virginia Kelly Karnes Archives and Special Collections staff.

Navigation Menu

Search code, repositories, users, issues, pull requests..., provide feedback.

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly.

To see all available qualifiers, see our documentation .

• Notifications

An LLM-powered knowledge curation system that researches a topic and generates a full-length report with citations.

stanford-oval/storm

Folders and files, repository files navigation, storm: synthesis of topic outlines through retrieval and multi-perspective question asking.

| Research preview | Paper | Documentation (WIP) |

Latest News 🔥

• [2024/04] We release refactored version of STORM codebase! We define interface for STORM pipeline and reimplement STORM-wiki (check out src/storm_wiki ) to demonstrate how to instantiate the pipeline. We provide API to support customization of different language models and retrieval/search integration.

Overview (Try STORM now!)

While the system cannot produce publication-ready articles that often require a significant number of edits, experienced Wikipedia editors have found it helpful in their pre-writing stage.

Try out our live research preview to see how STORM can help your knowledge exploration journey and please provide feedback to help us improve the system 🙏!

How STORM works

STORM breaks down generating long articles with citations into two steps:

• Pre-writing stage : The system conducts Internet-based research to collect references and generates an outline.
• Writing stage : The system uses the outline and references to generate the full-length article with citations.

STORM identifies the core of automating the research process as automatically coming up with good questions to ask. Directly prompting the language model to ask questions does not work well. To improve the depth and breadth of the questions, STORM adopts two strategies:

• Perspective-Guided Question Asking : Given the input topic, STORM discovers different perspectives by surveying existing articles from similar topics and uses them to control the question-asking process.
• Simulated Conversation : STORM simulates a conversation between a Wikipedia writer and a topic expert grounded in Internet sources to enable the language model to update its understanding of the topic and ask follow-up questions.

Based on the separation of the two stages, STORM is implemented in a highly modular way using dspy .

Getting started

Below, we provide a quick start guide to run STORM locally.

• Install the required packages. conda create -n storm python=3.11 conda activate storm pip install -r requirements.txt
• Set up OpenAI API key (if you want to use OpenAI models to power STORM) and You.com search API key. Create a file secrets.toml under the root directory and add the following content: # Set up OpenAI API key. OPENAI_API_KEY= " your_openai_api_key " # If you are using the API service provided by OpenAI, include the following line: OPENAI_API_TYPE= " openai " # If you are using the API service provided by Microsoft Azure, include the following lines: OPENAI_API_TYPE= " azure " AZURE_API_BASE= " your_azure_api_base_url " AZURE_API_VERSION= " your_azure_api_version " # Set up You.com search API key. YDC_API_KEY= " your_youcom_api_key "

2. Running STORM-wiki locally

Currently, we provide example scripts under examples to demonstrate how you can run STORM using different models.

To run STORM with gpt family models : Make sure you have set up the OpenAI API key and run the following command.

• --do-research : if True, simulate conversation to research the topic; otherwise, load the results.
• --do-generate-outline : If True, generate an outline for the topic; otherwise, load the results.
• --do-generate-article : If True, generate an article for the topic; otherwise, load the results.
• --do-polish-article : If True, polish the article by adding a summarization section and (optionally) removing duplicate content.

To run STORM with mistral family models on local VLLM server : have a VLLM server running with the Mistral-7B-Instruct-v0.2 model and run the following command.

• --url URL of the VLLM server.
• --port Port of the VLLM server.

Customize STORM

Customization of the pipeline.

STORM is a knowledge curation engine consisting of 4 modules:

• Knowledge Curation Module: Collects a broad coverage of information about the given topic.
• Outline Generation Module: Organizes the collected information by generating a hierarchical outline for the curated knowledge.
• Article Generation Module: Populates the generated outline with the collected information.
• Article Polishing Module: Refines and enhances the written article for better presentation.

The interface for each module is defined in src/interface.py , while their implementations are instantiated in src/storm_wiki/modules/* . These modules can be customized according to your specific requirements (e.g., generating sections in bullet point format instead of full paragraphs).

🌟 You can share your customization of Engine by making PRs to this repo!

Customization of Retriever Module

As a knowledge curation engine, STORM grabs information from the Retriever module. The interface for the Retriever module is defined in src/interface.py . Please consult the interface documentation if you plan to create a new instance or replace the default search engine API. By default, STORM utilizes the You.com search engine API (see YouRM in src/rm.py ).

🌟 PRs for integrating more search engines/retrievers are highly appreciated!

Customization of Language Models

STORM provides the following language model implementations in src/lm.py :

• OpenAIModel
• ClaudeModel
• TogetherClient

🌟 PRs for integrating more language model clients are highly appreciated!

💡 For a good practice,

• choose a cheaper/faster model for conv_simulator_lm which is used to split queries, synthesize answers in the conversation.
• if you need to conduct the actual writing step, choose a more powerful model for article_gen_lm . Based on our experiments, weak models are bad at generating text with citations.
• for open models, adding one-shot example can help it better follow instructions.

Please refer to the scripts in the examples directory for concrete guidance on customizing the language model used in the pipeline.

Replicate NAACL2024 result

Please switch to the branch NAACL-2024-code-backup

Paper Experiments

The FreshWiki dataset used in our experiments can be found in ./FreshWiki .

Run the following commands under ./src .

Pre-writing Stage

For batch experiment on FreshWiki dataset:

• --engine (choices=[ gpt-4 , gpt-35-turbo ]): the LLM engine used for generating the outline
• --max-conv-turn : the maximum number of questions for each information-seeking conversation
• STORM also uses a general conversation to collect basic information about the topic. So, the maximum number of QA pairs is max_turn * (max_perspective + 1) . 💡 Reducing max_turn or max_perspective can speed up the process and reduce the cost but may result in less comprehensive outline.
• The parameter will not have any effect if --disable-perspective is set (the perspective-driven question asking is disabled).

To run the experiment on a single topic:

• The script will ask you to enter the Topic and the Ground truth url that will be excluded. If you do not have any url to exclude, leave that field empty.

The generated outline will be saved in {output_dir}/{topic}/storm_gen_outline.txt and the collected references will be saved in {output_dir}/{topic}/raw_search_results.json .

Writing Stage

• --do-polish-article : if True, polish the article by adding a summarization section and removing duplicate content if --remove-duplicate is set True.
• The script will ask you to enter the Topic . Please enter the same topic as the one used in the pre-writing stage.

The generated article will be saved in {output_dir}/{topic}/storm_gen_article.txt and the references corresponding to citation index will be saved in {output_dir}/{topic}/url_to_info.json . If --do-polish-article is set, the polished article will be saved in {output_dir}/{topic}/storm_gen_article_polished.txt .

Customize the STORM Configurations

We set up the default LLM configuration in LLMConfigs in src/modules/utils.py . You can use set_conv_simulator_lm() , set_question_asker_lm() , set_outline_gen_lm() , set_article_gen_lm() , set_article_polish_lm() to override the default configuration. These functions take in an instance from dspy.dsp.LM or dspy.dsp.HFModel .

Automatic Evaluation

In our paper, we break down the evaluation into two parts: outline quality and full-length article quality.

Outline Quality

We introduce heading soft recall and heading entity recall to evaluate the outline quality. This makes it easier to prototype methods for pre-writing.

Run the following command under ./eval to compute the metrics on FreshWiki dataset:

Full-length Article Quality

eval/eval_article_quality.py provides the entry point of evaluating full-length article quality using ROUGE, entity recall, and rubric grading. Run the following command under eval to compute the metrics:

Use the Metric Yourself

The similarity-based metrics (i.e., ROUGE, entity recall, and heading entity recall) are implemented in eval/metrics.py .

For rubric grading, we use the prometheus-13b-v1.0 introduced in this paper . eval/evaluation_prometheus.py provides the entry point of using the metric.

Contributions

If you have any questions or suggestions, please feel free to open an issue or pull request. We welcome contributions to improve the system and the codebase!

Contact person: Yijia Shao and Yucheng Jiang

Please cite our paper if you use this code or part of it in your work:

Contributors 6

• Python 100.0%