extended literature review methodology

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, automatically generate references for free.

• Knowledge Base
• Dissertation
• What is a Literature Review? | Guide, Template, & Examples

What is a Literature Review? | Guide, Template, & Examples

Published on 22 February 2022 by Shona McCombes . Revised on 7 June 2022.

What is a literature review? A literature review is a survey of scholarly sources on a specific topic. It provides an overview of current knowledge, allowing you to identify relevant theories, methods, and gaps in the existing research.

There are five key steps to writing a literature review:

• Search for relevant literature
• Evaluate sources
• Identify themes, debates and gaps
• Outline the structure
• Write your literature review

A good literature review doesn’t just summarise sources – it analyses, synthesises, and critically evaluates to give a clear picture of the state of knowledge on the subject.

Instantly correct all language mistakes in your text

Be assured that you'll submit flawless writing. Upload your document to correct all your mistakes.

Table of contents

Why write a literature review, examples of literature reviews, step 1: search for relevant literature, step 2: evaluate and select sources, step 3: identify themes, debates and gaps, step 4: outline your literature review’s structure, step 5: write your literature review, frequently asked questions about literature reviews, introduction.

• Quick Run-through
• Step 1 & 2

When you write a dissertation or thesis, you will have to conduct a literature review to situate your research within existing knowledge. The literature review gives you a chance to:

• Demonstrate your familiarity with the topic and scholarly context
• Develop a theoretical framework and methodology for your research
• Position yourself in relation to other researchers and theorists
• Show how your dissertation addresses a gap or contributes to a debate

You might also have to write a literature review as a stand-alone assignment. In this case, the purpose is to evaluate the current state of research and demonstrate your knowledge of scholarly debates around a topic.

The content will look slightly different in each case, but the process of conducting a literature review follows the same steps. We’ve written a step-by-step guide that you can follow below.

The only proofreading tool specialized in correcting academic writing

The academic proofreading tool has been trained on 1000s of academic texts and by native English editors. Making it the most accurate and reliable proofreading tool for students.

Correct my document today

Writing literature reviews can be quite challenging! A good starting point could be to look at some examples, depending on what kind of literature review you’d like to write.

• Example literature review #1: “Why Do People Migrate? A Review of the Theoretical Literature” ( Theoretical literature review about the development of economic migration theory from the 1950s to today.)
• Example literature review #2: “Literature review as a research methodology: An overview and guidelines” ( Methodological literature review about interdisciplinary knowledge acquisition and production.)
• Example literature review #3: “The Use of Technology in English Language Learning: A Literature Review” ( Thematic literature review about the effects of technology on language acquisition.)
• Example literature review #4: “Learners’ Listening Comprehension Difficulties in English Language Learning: A Literature Review” ( Chronological literature review about how the concept of listening skills has changed over time.)

You can also check out our templates with literature review examples and sample outlines at the links below.

Download Word doc Download Google doc

Before you begin searching for literature, you need a clearly defined topic .

If you are writing the literature review section of a dissertation or research paper, you will search for literature related to your research objectives and questions .

If you are writing a literature review as a stand-alone assignment, you will have to choose a focus and develop a central question to direct your search. Unlike a dissertation research question, this question has to be answerable without collecting original data. You should be able to answer it based only on a review of existing publications.

Make a list of keywords

Start by creating a list of keywords related to your research topic. Include each of the key concepts or variables you’re interested in, and list any synonyms and related terms. You can add to this list if you discover new keywords in the process of your literature search.

• Social media, Facebook, Instagram, Twitter, Snapchat, TikTok
• Body image, self-perception, self-esteem, mental health
• Generation Z, teenagers, adolescents, youth

Search for relevant sources

Use your keywords to begin searching for sources. Some databases to search for journals and articles include:

• Your university’s library catalogue
• Google Scholar
• Project Muse (humanities and social sciences)
• Medline (life sciences and biomedicine)
• EconLit (economics)
• Inspec (physics, engineering and computer science)

You can use boolean operators to help narrow down your search:

Read the abstract to find out whether an article is relevant to your question. When you find a useful book or article, you can check the bibliography to find other relevant sources.

To identify the most important publications on your topic, take note of recurring citations. If the same authors, books or articles keep appearing in your reading, make sure to seek them out.

You probably won’t be able to read absolutely everything that has been written on the topic – you’ll have to evaluate which sources are most relevant to your questions.

For each publication, ask yourself:

• What question or problem is the author addressing?
• What are the key concepts and how are they defined?
• What are the key theories, models and methods? Does the research use established frameworks or take an innovative approach?
• What are the results and conclusions of the study?
• How does the publication relate to other literature in the field? Does it confirm, add to, or challenge established knowledge?
• How does the publication contribute to your understanding of the topic? What are its key insights and arguments?
• What are the strengths and weaknesses of the research?

Make sure the sources you use are credible, and make sure you read any landmark studies and major theories in your field of research.

You can find out how many times an article has been cited on Google Scholar – a high citation count means the article has been influential in the field, and should certainly be included in your literature review.

The scope of your review will depend on your topic and discipline: in the sciences you usually only review recent literature, but in the humanities you might take a long historical perspective (for example, to trace how a concept has changed in meaning over time).

Remember that you can use our template to summarise and evaluate sources you’re thinking about using!

Take notes and cite your sources

As you read, you should also begin the writing process. Take notes that you can later incorporate into the text of your literature review.

It’s important to keep track of your sources with references to avoid plagiarism . It can be helpful to make an annotated bibliography, where you compile full reference information and write a paragraph of summary and analysis for each source. This helps you remember what you read and saves time later in the process.

You can use our free APA Reference Generator for quick, correct, consistent citations.

To begin organising your literature review’s argument and structure, you need to understand the connections and relationships between the sources you’ve read. Based on your reading and notes, you can look for:

• Trends and patterns (in theory, method or results): do certain approaches become more or less popular over time?
• Themes: what questions or concepts recur across the literature?
• Debates, conflicts and contradictions: where do sources disagree?
• Pivotal publications: are there any influential theories or studies that changed the direction of the field?
• Gaps: what is missing from the literature? Are there weaknesses that need to be addressed?

This step will help you work out the structure of your literature review and (if applicable) show how your own research will contribute to existing knowledge.

• Most research has focused on young women.
• There is an increasing interest in the visual aspects of social media.
• But there is still a lack of robust research on highly-visual platforms like Instagram and Snapchat – this is a gap that you could address in your own research.

There are various approaches to organising the body of a literature review. You should have a rough idea of your strategy before you start writing.

Depending on the length of your literature review, you can combine several of these strategies (for example, your overall structure might be thematic, but each theme is discussed chronologically).

Chronological

The simplest approach is to trace the development of the topic over time. However, if you choose this strategy, be careful to avoid simply listing and summarising sources in order.

Try to analyse patterns, turning points and key debates that have shaped the direction of the field. Give your interpretation of how and why certain developments occurred.

If you have found some recurring central themes, you can organise your literature review into subsections that address different aspects of the topic.

For example, if you are reviewing literature about inequalities in migrant health outcomes, key themes might include healthcare policy, language barriers, cultural attitudes, legal status, and economic access.

Methodological

If you draw your sources from different disciplines or fields that use a variety of research methods , you might want to compare the results and conclusions that emerge from different approaches. For example:

• Look at what results have emerged in qualitative versus quantitative research
• Discuss how the topic has been approached by empirical versus theoretical scholarship
• Divide the literature into sociological, historical, and cultural sources

Theoretical

A literature review is often the foundation for a theoretical framework . You can use it to discuss various theories, models, and definitions of key concepts.

You might argue for the relevance of a specific theoretical approach, or combine various theoretical concepts to create a framework for your research.

Like any other academic text, your literature review should have an introduction , a main body, and a conclusion . What you include in each depends on the objective of your literature review.

The introduction should clearly establish the focus and purpose of the literature review.

If you are writing the literature review as part of your dissertation or thesis, reiterate your central problem or research question and give a brief summary of the scholarly context. You can emphasise the timeliness of the topic (“many recent studies have focused on the problem of x”) or highlight a gap in the literature (“while there has been much research on x, few researchers have taken y into consideration”).

Depending on the length of your literature review, you might want to divide the body into subsections. You can use a subheading for each theme, time period, or methodological approach.

As you write, make sure to follow these tips:

• Summarise and synthesise: give an overview of the main points of each source and combine them into a coherent whole.
• Analyse and interpret: don’t just paraphrase other researchers – add your own interpretations, discussing the significance of findings in relation to the literature as a whole.
• Critically evaluate: mention the strengths and weaknesses of your sources.
• Write in well-structured paragraphs: use transitions and topic sentences to draw connections, comparisons and contrasts.

In the conclusion, you should summarise the key findings you have taken from the literature and emphasise their significance.

If the literature review is part of your dissertation or thesis, reiterate how your research addresses gaps and contributes new knowledge, or discuss how you have drawn on existing theories and methods to build a framework for your research. This can lead directly into your methodology section.

A literature review is a survey of scholarly sources (such as books, journal articles, and theses) related to a specific topic or research question .

It is often written as part of a dissertation , thesis, research paper , or proposal .

There are several reasons to conduct a literature review at the beginning of a research project:

• To familiarise yourself with the current state of knowledge on your topic
• To ensure that you’re not just repeating what others have already done
• To identify gaps in knowledge and unresolved problems that your research can address
• To develop your theoretical framework and methodology
• To provide an overview of the key findings and debates on the topic

Writing the literature review shows your reader how your work relates to existing research and what new insights it will contribute.

The literature review usually comes near the beginning of your  dissertation . After the introduction , it grounds your research in a scholarly field and leads directly to your theoretical framework or methodology .

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the ‘Cite this Scribbr article’ button to automatically add the citation to our free Reference Generator.

McCombes, S. (2022, June 07). What is a Literature Review? | Guide, Template, & Examples. Scribbr. Retrieved 14 May 2024, from https://www.scribbr.co.uk/thesis-dissertation/literature-review/

Is this article helpful?

Shona McCombes

Other students also liked, how to write a dissertation proposal | a step-by-step guide, what is a theoretical framework | a step-by-step guide, what is a research methodology | steps & tips.

State-of-the-art literature review methodology: A six-step approach for knowledge synthesis

• Original Article
• Open access
• Published: 05 September 2022
• Volume 11 , pages 281–288, ( 2022 )

Cite this article

You have full access to this open access article

• Erin S. Barry   ORCID: orcid.org/0000-0003-0788-7153 1 , 2 ,
• Jerusalem Merkebu   ORCID: orcid.org/0000-0003-3707-8920 3 &
• Lara Varpio   ORCID: orcid.org/0000-0002-1412-4341 3  

28k Accesses

8 Citations

18 Altmetric

Explore all metrics

Introduction

Researchers and practitioners rely on literature reviews to synthesize large bodies of knowledge. Many types of literature reviews have been developed, each targeting a specific purpose. However, these syntheses are hampered if the review type’s paradigmatic roots, methods, and markers of rigor are only vaguely understood. One literature review type whose methodology has yet to be elucidated is the state-of-the-art (SotA) review. If medical educators are to harness SotA reviews to generate knowledge syntheses, we must understand and articulate the paradigmatic roots of, and methods for, conducting SotA reviews.

We reviewed 940 articles published between 2014–2021 labeled as SotA reviews. We (a) identified all SotA methods-related resources, (b) examined the foundational principles and techniques underpinning the reviews, and (c) combined our findings to inductively analyze and articulate the philosophical foundations, process steps, and markers of rigor.

In the 940 articles reviewed, nearly all manuscripts (98%) lacked citations for how to conduct a SotA review. The term “state of the art” was used in 4 different ways. Analysis revealed that SotA articles are grounded in relativism and subjectivism.

This article provides a 6-step approach for conducting SotA reviews. SotA reviews offer an interpretive synthesis that describes: This is where we are now. This is how we got here. This is where we could be going. This chronologically rooted narrative synthesis provides a methodology for reviewing large bodies of literature to explore why and how our current knowledge has developed and to offer new research directions.

Similar content being viewed by others

An analysis of current practices in undertaking literature reviews in nursing: findings from a focused mapping review and synthesis

Reviewing the literature, how systematic is systematic.

Reading and interpreting reviews for health professionals: a practical review

Avoid common mistakes on your manuscript.

Literature reviews play a foundational role in scientific research; they support knowledge advancement by collecting, describing, analyzing, and integrating large bodies of information and data [ 1 , 2 ]. Indeed, as Snyder [ 3 ] argues, all scientific disciplines require literature reviews grounded in a methodology that is accurate and clearly reported. Many types of literature reviews have been developed, each with a unique purpose, distinct methods, and distinguishing characteristics of quality and rigor [ 4 , 5 ].

Each review type offers valuable insights if rigorously conducted [ 3 , 6 ]. Problematically, this is not consistently the case, and the consequences can be dire. Medical education’s policy makers and institutional leaders rely on knowledge syntheses to inform decision making [ 7 ]. Medical education curricula are shaped by these syntheses. Our accreditation standards are informed by these integrations. Our patient care is guided by these knowledge consolidations [ 8 ]. Clearly, it is important for knowledge syntheses to be held to the highest standards of rigor. And yet, that standard is not always maintained. Sometimes scholars fail to meet the review’s specified standards of rigor; other times the markers of rigor have never been explicitly articulated. While we can do little about the former, we can address the latter. One popular literature review type whose methodology has yet to be fully described, vetted, and justified is the state-of-the-art (SotA) review.

While many types of literature reviews amalgamate bodies of literature, SotA reviews offer something unique. By looking across the historical development of a body of knowledge, SotA reviews delves into questions like: Why did our knowledge evolve in this way? What other directions might our investigations have taken? What turning points in our thinking should we revisit to gain new insights? A SotA review—a form of narrative knowledge synthesis [ 5 , 9 ]—acknowledges that history reflects a series of decisions and then asks what different decisions might have been made.

SotA reviews are frequently used in many fields including the biomedical sciences [ 10 , 11 ], medicine [ 12 , 13 , 14 ], and engineering [ 15 , 16 ]. However, SotA reviews are rarely seen in medical education; indeed, a bibliometrics analysis of literature reviews published in 14 core medical education journals between 1999 and 2019 reported only 5 SotA reviews out of the 963 knowledge syntheses identified [ 17 ]. This is not to say that SotA reviews are absent; we suggest that they are often unlabeled. For instance, Schuwirth and van der Vleuten’s article “A history of assessment in medical education” [ 14 ] offers a temporally organized overview of the field’s evolving thinking about assessment. Similarly, McGaghie et al. published a chronologically structured review of simulation-based medical education research that “reviews and critically evaluates historical and contemporary research on simulation-based medical education” [ 18 , p. 50]. SotA reviews certainly have a place in medical education, even if that place is not explicitly signaled.

This lack of labeling is problematic since it conceals the purpose of, and work involved in, the SotA review synthesis. In a SotA review, the author(s) collects and analyzes the historical development of a field’s knowledge about a phenomenon, deconstructs how that understanding evolved, questions why it unfolded in specific ways, and posits new directions for research. Senior medical education scholars use SotA reviews to share their insights based on decades of work on a topic [ 14 , 18 ]; their junior counterparts use them to critique that history and propose new directions [ 19 ]. And yet, SotA reviews are generally not explicitly signaled in medical education. We suggest that at least two factors contribute to this problem. First, it may be that medical education scholars have yet to fully grasp the unique contributions SotA reviews provide. Second, the methodology and methods of SotA reviews are poorly reported making this form of knowledge synthesis appear to lack rigor. Both factors are rooted in the same foundational problem: insufficient clarity about SotA reviews. In this study, we describe SotA review methodology so that medical educators can explicitly use this form of knowledge synthesis to further advance the field.

We developed a four-step research design to meet this goal, illustrated in Fig.  1 .

Four-step research design process used for developing a State-of-the-Art literature review methodology

Step 1: Collect SotA articles

To build our initial corpus of articles reporting SotA reviews, we searched PubMed using the strategy (″state of the art review″[ti] OR ″state of the art review*″) and limiting our search to English articles published between 2014 and 2021. We strategically focused on PubMed, which includes MEDLINE, and is considered the National Library of Medicine’s premier database of biomedical literature and indexes health professions education and practice literature [ 20 ]. We limited our search to 2014–2021 to capture modern use of SotA reviews. Of the 960 articles identified, nine were excluded because they were duplicates, erratum, or corrigendum records; full text copies were unavailable for 11 records. All articles identified ( n  = 940) constituted the corpus for analysis.

Step 2: Compile all methods-related resources

EB, JM, or LV independently reviewed the 940 full-text articles to identify all references to resources that explained, informed, described, or otherwise supported the methods used for conducting the SotA review. Articles that met our criteria were obtained for analysis.

To ensure comprehensive retrieval, we also searched Scopus and Web of Science. Additionally, to find resources not indexed by these academic databases, we searched Google (see Electronic Supplementary Material [ESM] for the search strategies used for each database). EB also reviewed the first 50 items retrieved from each search looking for additional relevant resources. None were identified. Via these strategies, nine articles were identified and added to the collection of methods-related resources for analysis.

Step 3: Extract data for analysis

In Step 3, we extracted three kinds of information from the 940 articles papers identified in Step 1. First, descriptive data on each article were compiled (i.e., year of publication and the academic domain targeted by the journal). Second, each article was examined and excerpts collected about how the term state-of-the-art review was used (i.e., as a label for a methodology in-and-of itself; as an adjective qualifying another type of literature review; as a term included in the paper’s title only; or in some other way). Finally, we extracted excerpts describing: the purposes and/or aims of the SotA review; the methodology informing and methods processes used to carry out the SotA review; outcomes of analyses; and markers of rigor for the SotA review.

Two researchers (EB and JM) coded 69 articles and an interrater reliability of 94.2% was achieved. Any discrepancies were discussed. Given the high interrater reliability, the two authors split the remaining articles and coded independently.

Step 4: Construct the SotA review methodology

The methods-related resources identified in Step 2 and the data extractions from Step 3 were inductively analyzed by LV and EB to identify statements and research processes that revealed the ontology (i.e., the nature of reality that was reflected) and the epistemology (i.e., the nature of knowledge) underpinning the descriptions of the reviews. These authors studied these data to determine if the synthesis adhered to an objectivist or a subjectivist orientation, and to synthesize the purposes realized in these papers.

To confirm these interpretations, LV and EB compared their ontology, epistemology, and purpose determinations against two expectations commonly required of objectivist synthesis methods (e.g., systematic reviews): an exhaustive search strategy and an appraisal of the quality of the research data. These expectations were considered indicators of a realist ontology and objectivist epistemology [ 21 ] (i.e., that a single correct understanding of the topic can be sought through objective data collection {e.g., systematic reviews [ 22 ]}). Conversely, the inverse of these expectations were considered indicators of a relativist ontology and subjectivist epistemology [ 21 ] (i.e., that no single correct understanding of the topic is available; there are multiple valid understandings that can be generated and so a subjective interpretation of the literature is sought {e.g., narrative reviews [ 9 ]}).

Once these interpretations were confirmed, LV and EB reviewed and consolidated the methods steps described in these data. Markers of rigor were then developed that aligned with the ontology, epistemology, and methods of SotA reviews.

Of the 940 articles identified in Step 1, 98% ( n  = 923) lacked citations or other references to resources that explained, informed, or otherwise supported the SotA review process. Of the 17 articles that included supporting information, 16 cited Grant and Booth’s description [ 4 ] consisting of five sentences describing the overall purpose of SotA reviews, three sentences noting perceived strengths, and four sentences articulating perceived weaknesses. This resource provides no guidance on how to conduct a SotA review methodology nor markers of rigor. The one article not referencing Grant and Booth used “an adapted comparative effectiveness research search strategy that was adapted by a health sciences librarian” [ 23 , p. 381]. One website citation was listed in support of this strategy; however, the page was no longer available in summer 2021. We determined that the corpus was uninformed by a cardinal resource or a publicly available methodology description.

In Step 2 we identified nine resources [ 4 , 5 , 24 , 25 , 26 , 27 , 28 ]; none described the methodology and/or processes of carrying out SotA reviews. Nor did they offer explicit descriptions of the ontology or epistemology underpinning SotA reviews. Instead, these resources provided short overview statements (none longer than one paragraph) about the review type [ 4 , 5 , 24 , 25 , 26 , 27 , 28 ]. Thus, we determined that, to date, there are no available methodology papers describing how to conduct a SotA review.

Step 3 revealed that “state of the art” was used in 4 different ways across the 940 articles (see Fig.  2 for the frequency with which each was used). In 71% ( n  = 665 articles), the phrase was used only in the title, abstract, and/or purpose statement of the article; the phrase did not appear elsewhere in the paper and no SotA methodology was discussed. Nine percent ( n  = 84) used the phrase as an adjective to qualify another literature review type and so relied entirely on the methodology of a different knowledge synthesis approach (e.g., “a state of the art systematic review [ 29 ]”). In 5% ( n  = 52) of the articles, the phrase was not used anywhere within the article; instead, “state of the art” was the type of article within a journal. In the remaining 15% ( n  = 139), the phrase denoted a specific methodology (see ESM for all methodology articles). Via Step 4’s inductive analysis, the following foundational principles of SotA reviews were developed: (1) the ontology, (2) epistemology, and (3) purpose of SotA reviews.

Four ways the term “state of the art” is used in the corpus and how frequently each is used

Ontology of SotA reviews: Relativism

SotA reviews rest on four propositions:

The literature addressing a phenomenon offers multiple perspectives on that topic (i.e., different groups of researchers may hold differing opinions and/or interpretations of data about a phenomenon).

The reality of the phenomenon itself cannot be completely perceived or understood (i.e., due to limitations [e.g., the capabilities of current technologies, a research team’s disciplinary orientation] we can only perceive a limited part of the phenomenon).

The reality of the phenomenon is a subjective and inter-subjective construction (i.e., what we understand about a phenomenon is built by individuals and so their individual subjectivities shape that understanding).

The context in which the review was conducted informs the review (e.g., a SotA review of literature about gender identity and sexual function will be synthesized differently by researchers in the domain of gender studies than by scholars working in sex reassignment surgery).

As these propositions suggest, SotA scholars bring their experiences, expectations, research purposes, and social (including academic) orientations to bear on the synthesis work. In other words, a SotA review synthesizes the literature based on a specific orientation to the topic being addressed. For instance, a SotA review written by senior scholars who are experts in the field of medical education may reflect on the turning points that have shaped the way our field has evolved the modern practices of learner assessment, noting how the nature of the problem of assessment has moved: it was first a measurement problem, then a problem that embraced human judgment but needed assessment expertise, and now a whole system problem that is to be addressed from an integrated—not a reductionist—perspective [ 12 ]. However, if other scholars were to examine this same history from a technological orientation, learner assessment could be framed as historically constricted by the media available through which to conduct assessment, pointing to how artificial intelligence is laying the foundation for the next wave of assessment in medical education [ 30 ].

Given these foundational propositions, SotA reviews are steeped in a relativist ontology—i.e., reality is socially and experientially informed and constructed, and so no single objective truth exists. Researchers’ interpretations reflect their conceptualization of the literature—a conceptualization that could change over time and that could conflict with the understandings of others.

Epistemology of SotA reviews: Subjectivism

SotA reviews embrace subjectivism. The knowledge generated through the review is value-dependent, growing out of the subjective interpretations of the researcher(s) who conducted the synthesis. The SotA review generates an interpretation of the data that is informed by the expertise, experiences, and social contexts of the researcher(s). Furthermore, the knowledge developed through SotA reviews is shaped by the historical point in time when the review was conducted. SotA reviews are thus steeped in the perspective that knowledge is shaped by individuals and their community, and is a synthesis that will change over time.

Purpose of SotA reviews

SotA reviews create a subjectively informed summary of modern thinking about a topic. As a chronologically ordered synthesis, SotA reviews describe the history of turning points in researchers’ understanding of a phenomenon to contextualize a description of modern scientific thinking on the topic. The review presents an argument about how the literature could be interpreted; it is not a definitive statement about how the literature should or must be interpreted. A SotA review explores: the pivotal points shaping the historical development of a topic, the factors that informed those changes in understanding, and the ways of thinking about and studying the topic that could inform the generation of further insights. In other words, the purpose of SotA reviews is to create a three-part argument: This is where we are now in our understanding of this topic. This is how we got here. This is where we could go next.

The SotA methodology

Based on study findings and analyses, we constructed a six-stage SotA review methodology. This six-stage approach is summarized and guiding questions are offered in Tab.  1 .

Stage 1: Determine initial research question and field of inquiry

In Stage 1, the researcher(s) creates an initial description of the topic to be summarized and so must determine what field of knowledge (and/or practice) the search will address. Knowledge developed through the SotA review process is shaped by the context informing it; thus, knowing the domain in which the review will be conducted is part of the review’s foundational work.

Stage 2: Determine timeframe

This stage involves determining the period of time that will be defined as SotA for the topic being summarized. The researcher(s) should engage in a broad-scope overview of the literature, reading across the range of literature available to develop insights into the historical development of knowledge on the topic, including the turning points that shape the current ways of thinking about a topic. Understanding the full body of literature is required to decide the dates or events that demarcate the timeframe of now in the first of the SotA’s three-part argument: where we are now . Stage 2 is complete when the researcher(s) can explicitly justify why a specific year or event is the right moment to mark the beginning of state-of-the-art thinking about the topic being summarized.

Stage 3: Finalize research question(s) to reflect timeframe

Based on the insights developed in Stage 2, the researcher(s) will likely need to revise their initial description of the topic to be summarized. The formal research question(s) framing the SotA review are finalized in Stage 3. The revised description of the topic, the research question(s), and the justification for the timeline start year must be reported in the review article. These are markers of rigor and prerequisites for moving to Stage 4.

Stage 4: Develop search strategy to find relevant articles

In Stage 4, the researcher(s) develops a search strategy to identify the literature that will be included in the SotA review. The researcher(s) needs to determine which literature databases contain articles from the domain of interest. Because the review describes how we got here , the review must include literature that predates the state-of-the-art timeframe, determined in Stage 2, to offer this historical perspective.

Developing the search strategy will be an iterative process of testing and revising the search strategy to enable the researcher(s) to capture the breadth of literature required to meet the SotA review purposes. A librarian should be consulted since their expertise can expedite the search processes and ensure that relevant resources are identified. The search strategy must be reported (e.g., in the manuscript itself or in a supplemental file) so that others may replicate the process if they so choose (e.g., to construct a different SotA review [and possible different interpretations] of the same literature). This too is a marker of rigor for SotA reviews: the search strategies informing the identification of literature must be reported.

Stage 5: Analyses

The literature analysis undertaken will reflect the subjective insights of the researcher(s); however, the foundational premises of inductive research should inform the analysis process. Therefore, the researcher(s) should begin by reading the articles in the corpus to become familiar with the literature. This familiarization work includes: noting similarities across articles, observing ways-of-thinking that have shaped current understandings of the topic, remarking on assumptions underpinning changes in understandings, identifying important decision points in the evolution of understanding, and taking notice of gaps and assumptions in current knowledge.

The researcher(s) can then generate premises for the state-of-the-art understanding of the history that gave rise to modern thinking, of the current body of knowledge, and of potential future directions for research. In this stage of the analysis, the researcher(s) should document the articles that support or contradict their premises, noting any collections of authors or schools of thinking that have dominated the literature, searching for marginalized points of view, and studying the factors that contributed to the dominance of particular ways of thinking. The researcher(s) should also observe historical decision points that could be revisited. Theory can be incorporated at this stage to help shape insights and understandings. It should be highlighted that not all corpus articles will be used in the SotA review; instead, the researcher(s) will sample across the corpus to construct a timeline that represents the seminal moments of the historical development of knowledge.

Next, the researcher(s) should verify the thoroughness and strength of their interpretations. To do this, the researcher(s) can select different articles included in the corpus and examine if those articles reflect the premises the researcher(s) set out. The researcher(s) may also seek out contradictory interpretations in the literature to be sure their summary refutes these positions. The goal of this verification work is not to engage in a triangulation process to ensure objectivity; instead, this process helps the researcher(s) ensure the interpretations made in the SotA review represent the articles being synthesized and respond to the interpretations offered by others. This is another marker of rigor for SotA reviews: the authors should engage in and report how they considered and accounted for differing interpretations of the literature, and how they verified the thoroughness of their interpretations.

Stage 6: Reflexivity

Given the relativist subjectivism of a SotA review, it is important that the manuscript offer insights into the subjectivity of the researcher(s). This reflexivity description should articulate how the subjectivity of the researcher(s) informed interpretations of the data. These reflections will also influence the suggested directions offered in the last part of the SotA three-part argument: where we could go next. This is the last marker of rigor for SotA reviews: researcher reflexivity must be considered and reported.

SotA reviews have much to offer our field since they provide information on the historical progression of medical education’s understanding of a topic, the turning points that guided that understanding, and the potential next directions for future research. Those future directions may question the soundness of turning points and prior decisions, and thereby offer new paths of investigation. Since we were unable to find a description of the SotA review methodology, we inductively developed a description of the methodology—including its paradigmatic roots, the processes to be followed, and the markers of rigor—so that scholars can harness the unique affordances of this type of knowledge synthesis.

Given their chronology- and turning point-based orientation, SotA reviews are inherently different from other types of knowledge synthesis. For example, systematic reviews focus on specific research questions that are narrow in scope [ 32 , 33 ]; in contrast, SotA reviews present a broader historical overview of knowledge development and the decisions that gave rise to our modern understandings. Scoping reviews focus on mapping the present state of knowledge about a phenomenon including, for example, the data that are currently available, the nature of that data, and the gaps in knowledge [ 34 , 35 ]; conversely, SotA reviews offer interpretations of the historical progression of knowledge relating to a phenomenon centered on significant shifts that occurred during that history. SotA reviews focus on the turning points in the history of knowledge development to suggest how different decisions could give rise to new insights. Critical reviews draw on literature outside of the domain of focus to see if external literature can offer new ways of thinking about the phenomenon of interest (e.g., drawing on insights from insects’ swarm intelligence to better understand healthcare team adaptation [ 36 ]). SotA reviews focus on one domain’s body of literature to construct a timeline of knowledge development, demarcating where we are now, demonstrating how this understanding came to be via different turning points, and offering new research directions. Certainly, SotA reviews offer a unique kind of knowledge synthesis.

Our six-stage process for conducting these reviews reflects the subjectivist relativism that underpins the methodology. It aligns with the requirements proposed by others [ 24 , 25 , 26 , 27 ], what has been written about SotA reviews [ 4 , 5 ], and the current body of published SotA reviews. In contrast to existing guidance [ 4 , 5 , 20 , 21 , 22 , 23 ], our description offers a detailed reporting of the ontology, epistemology, and methodology processes for conducting the SotA review.

This explicit methodology description is essential since many academic journals list SotA reviews as an accepted type of literature review. For instance, Educational Research Review [ 24 ], the American Academy of Pediatrics [ 25 ], and Thorax all lists SotA reviews as one of the types of knowledge syntheses they accept [ 27 ]. However, while SotA reviews are valued by academia, guidelines or specific methodology descriptions for researchers to follow when conducting this type of knowledge synthesis are conspicuously absent. If academics in general, and medical education more specifically, are to take advantage of the insights that SotA reviews can offer, we need to rigorously engage in this synthesis work; to do that, we need clear descriptions of the methodology underpinning this review. This article offers such a description. We hope that more medical educators will conduct SotA reviews to generate insights that will contribute to further advancing our field’s research and scholarship.

Cooper HM. Organizing knowledge syntheses: a taxonomy of literature reviews. Knowl Soc. 1988;1:104.

Google Scholar  

Badger D, Nursten J, Williams P, Woodward M. Should all literature reviews be systematic? Eval Res Educ. 2000;14:220–30.

Article   Google Scholar  

Snyder H. Literature review as a research methodology: an overview and guidelines. J Bus Res. 2019;104:333–9.

Grant MJ, Booth A. A typology of reviews: an analysis of 14 review types and associated methodologies. Health Info Libr J. 2009;26:91–108.

Sutton A, Clowes M, Preston L, Booth A. Meeting the review family: exploring review types and associated information retrieval requirements. Health Info Libr J. 2019;36:202–22.

Moher D, Liberati A, Tetzlaff J, Altman DG, Prisma Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 2009;6:e1000097.

Tricco AC, Langlois E, Straus SE, World Health Organization, Alliance for Health Policy and Systems Research. Rapid reviews to strengthen health policy and systems: a practical guide. Geneva: World Health Organization; 2017.

Jackson R, Feder G. Guidelines for clinical guidelines: a simple, pragmatic strategy for guideline development. Br Med J. 1998;317:427–8.

Greenhalgh T, Thorne S, Malterud K. Time to challenge the spurious hierarchy of systematic over narrative reviews? Eur J Clin Invest. 2018;48:e12931.

Bach QV, Chen WH. Pyrolysis characteristics and kinetics of microalgae via thermogravimetric analysis (TGA): a state-of-the-art review. Bioresour Technol. 2017;246:88–100.

Garofalo C, Milanović V, Cardinali F, Aquilanti L, Clementi F, Osimani A. Current knowledge on the microbiota of edible insects intended for human consumption: a state-of-the-art review. Food Res Int. 2019;125:108527.

Carbone S, Dixon DL, Buckley LF, Abbate A. Glucose-lowering therapies for cardiovascular risk reduction in type 2 diabetes mellitus: state-of-the-art review. Mayo Clin Proc. 2018;93:1629–47.

Hofkens PJ, Verrijcken A, Merveille K, et al. Common pitfalls and tips and tricks to get the most out of your transpulmonary thermodilution device: results of a survey and state-of-the-art review. Anaesthesiol Intensive Ther. 2015;47:89–116.

Schuwirth LW, van der Vleuten CP. A history of assessment in medical education. Adv Health Sci Educ Theory Pract. 2020;25:1045–56.

Arena A, Prete F, Rambaldi E, et al. Nanostructured zirconia-based ceramics and composites in dentistry: a state-of-the-art review. Nanomaterials. 2019;9:1393.

Bahraminasab M, Farahmand F. State of the art review on design and manufacture of hybrid biomedical materials: hip and knee prostheses. Proc Inst Mech Eng H. 2017;231:785–813.

Maggio LA, Costello JA, Norton C, Driessen EW, Artino AR Jr. Knowledge syntheses in medical education: a bibliometric analysis. Perspect Med Educ. 2021;10:79–87.

McGaghie WC, Issenberg SB, Petrusa ER, Scalese RJ. A critical review of simulation-based medical education research: 2003–2009. Med Educ. 2010;44:50–63.

Krishnan DG, Keloth AV, Ubedulla S. Pros and cons of simulation in medical education: a review. Education. 2017;3:84–7.

National Library of Medicine. MEDLINE: overview. 2021. https://www.nlm.nih.gov/medline/medline_overview.html . Accessed 17 Dec 2021.

Bergman E, de Feijter J, Frambach J, et al. AM last page: a guide to research paradigms relevant to medical education. Acad Med. 2012;87:545.

Maggio LA, Samuel A, Stellrecht E. Systematic reviews in medical education. J Grad Med Educ. 2022;14:171–5.

Bandari J, Wessel CB, Jacobs BL. Comparative effectiveness in urology: a state of the art review utilizing a systematic approach. Curr Opin Urol. 2017;27:380–94.

Elsevier. A guide for writing scholarly articles or reviews for the educational research review. 2010. https://www.elsevier.com/__data/promis_misc/edurevReviewPaperWriting.pdf . Accessed 3 Mar 2020.

American Academy of Pediatrics. Pediatrics author guidelines. 2020. https://pediatrics.aappublications.org/page/author-guidelines . Accessed 3 Mar 2020.

Journal of the American College of Cardiology. JACC instructions for authors. 2020. https://www.jacc.org/pb-assets/documents/author-instructions-jacc-1598995793940.pdf . Accessed 3 Mar 2020.

Thorax. Authors. 2020. https://thorax.bmj.com/pages/authors/ . Accessed 3 Mar 2020.

Berven S, Carl A. State of the art review. Spine Deform. 2019;7:381.

Ilardi CR, Chieffi S, Iachini T, Iavarone A. Neuropsychology of posteromedial parietal cortex and conversion factors from mild cognitive impairment to Alzheimer’s disease: systematic search and state-of-the-art review. Aging Clin Exp Res. 2022;34:289–307.

Chan KS, Zary N. Applications and challenges of implementing artificial intelligence in medical education: integrative review. JMIR Med Educ. 2019;5:e13930.

World Health Organization. Framework for action on interprofessional education and collaborative practice. 2010. https://www.who.int/publications/i/item/framework-for-action-on-interprofessional-education-collaborative-practice . Accessed July 1 2021.

Hammersley M. On ‘systematic’ reviews of research literatures: a ‘narrative’ response to Evans & Benefield. Br Educ Res J. 2001;27:543–54.

Chen F, Lui AM, Martinelli SM. A systematic review of the effectiveness of flipped classrooms in medical education. Med Educ. 2017;51:585–97.

Arksey H, O’Malley L. Scoping studies: towards a methodological framework. Int J Soc Res Methodol. 2005;8:19–32.

Matsas B, Goralnick E, Bass M, Barnett E, Nagle B, Sullivan E. Leadership development in US undergraduate medical education: a scoping review of curricular content and competency frameworks. Acad Med. 2022;97:899–908.

Cristancho SM. On collective self-healing and traces: How can swarm intelligence help us think differently about team adaptation? Med Educ. 2021;55:441–7.

Download references

Acknowledgements

We thank Rhonda Allard for her help with the literature review and compiling all available articles. We also want to thank the PME editors who offered excellent development and refinement suggestions that greatly improved this manuscript.

Author information

Authors and affiliations.

Department of Anesthesiology, F. Edward Hébert School of Medicine, Uniformed Services University, Bethesda, MD, USA

Erin S. Barry

School of Health Professions Education (SHE), Maastricht University, Maastricht, The Netherlands

Department of Medicine, F. Edward Hébert School of Medicine, Uniformed Services University, Bethesda, MD, USA

Jerusalem Merkebu & Lara Varpio

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Erin S. Barry .

Ethics declarations

Conflict of interest.

E.S. Barry, J. Merkebu and L. Varpio declare that they have no competing interests.

Additional information

The opinions and assertions contained in this article are solely those of the authors and are not to be construed as reflecting the views of the Uniformed Services University of the Health Sciences, the Department of Defense, or the Henry M. Jackson Foundation for the Advancement of Military Medicine.

Supplementary Information

40037_2022_725_moesm1_esm.docx.

For information regarding the search strategy to develop the corpus and search strategy for confirming capture of any available State of the Art review methodology descriptions. Additionally, a list of the methodology articles found through the search strategy/corpus is included

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Barry, E.S., Merkebu, J. & Varpio, L. State-of-the-art literature review methodology: A six-step approach for knowledge synthesis. Perspect Med Educ 11 , 281–288 (2022). https://doi.org/10.1007/s40037-022-00725-9

Download citation

Received : 03 December 2021

Revised : 25 July 2022

Accepted : 27 July 2022

Published : 05 September 2022

Issue Date : October 2022

DOI : https://doi.org/10.1007/s40037-022-00725-9

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• State-of-the-art literature review
• Literature review
• Literature review methodology
• Find a journal
• Publish with us
• Track your research

Dissertations and research projects

• Book a session
• Planning your research

Developing a theoretical framework

Reflecting on your position, extended literature reviews, presenting qualitative data.

• Quantitative research
• Writing up your research project
• e-learning and books
• SkillsCheck This link opens in a new window
• ⬅ Back to Skills Centre This link opens in a new window
• Review this resource

What is a theoretical framework?

Developing a theoretical framework for your dissertation is one of the key elements of a qualitative research project. Through writing your literature review, you are likely to have identified either a problem that need ‘fixing’ or a gap that your research may begin to fill.

The theoretical framework is your toolbox . In the toolbox are your handy tools: a set of theories, concepts, ideas and hypotheses that you will use to build a solution to the research problem or gap you have identified.

The methodology is the instruction manual: the procedure and steps you have taken, using your chosen tools, to tackle the research problem.

Why do I need a theoretical framework?

Developing a theoretical framework shows that you have thought critically about the different ways to approach your topic, and that you have made a well-reasoned and evidenced decision about which approach will work best. theoretical frameworks are also necessary for solving complex problems or issues from the literature, showing that you have the skills to think creatively and improvise to answer your research questions. they also allow researchers to establish new theories and approaches, that future research may go on to develop., how do i create a theoretical framework for my dissertation.

First, select your tools. You are likely to need a variety of tools in qualitative research – different theories, models or concepts – to help you tackle different parts of your research question.  

When deciding what tools would be best for the job of answering your research questions or problem, explore what existing research in your area has used. You may find that there is a ‘standard toolbox’ for qualitative research in your field that you can borrow from or apply to your own research.

You will need to justify why your chosen tools are best for the job of answering your research questions, at what stage they are most relevant, and how they relate to each other. Some theories or models will neatly fit together and appear in the toolboxes of other researchers. However, you may wish to incorporate a model or idea that is not typical for your research area – the ‘odd one out’ in your toolbox. If this is the case, make sure you justify and account for why it is useful to you, and look for ways that it can be used in partnership with the other tools you are using.

You should also be honest about limitations, or where you need to improvise (for example, if the ‘right’ tool or approach doesn’t exist in your area).

This video from the Skills Centre includes an overview and example of how you might create a theoretical framework for your dissertation:

How do I choose the 'right' approach?

When designing your framework and choosing what to include, it can often be difficult to know if you’ve chosen the ‘right’ approach for your research questions. One way to check this is to look for consistency between your objectives, the literature in your framework, and your overall ethos for the research. This means ensuring that the literature you have used not only contributes to answering your research objectives, but that you also use theories and models that are true to your beliefs as a researcher.

Reflecting on your values and your overall ambition for the project can be a helpful step in making these decisions, as it can help you to fully connect your methodology and methods to your research aims.

Should I reflect on my position as a researcher?

If you feel your position as a researcher has influenced your choice of methods or procedure in any way, the methodology is a good place to reflect on this.  Positionality  acknowledges that no researcher is entirely objective: we are all, to some extent, influenced by prior learning, experiences, knowledge, and personal biases. This is particularly true in qualitative research or practice-based research, where the student is acting as a researcher in their own workplace, where they are otherwise considered a practitioner/professional. It's also important to reflect on your positionality if you belong to the same community as your participants where this is the grounds for their involvement in the research (ie. you are a mature student interviewing other mature learners about their experences in higher education). 

The following questions can help you to reflect on your positionality and gauge whether this is an important section to include in your dissertation (for some people, this section isn’t necessary or relevant):

• How might my personal history influence how I approach the topic?
• How am I positioned in relation to this knowledge? Am I being influenced by prior learning or knowledge from outside of this course?
• How does my gender/social class/ ethnicity/ culture influence my positioning in relation to this topic?
• Do I share any attributes with my participants? Are we part of a s hared community? How might this have influenced our relationship and my role in interviews/observations?
• Am I invested in the outcomes on a personal level? Who is this research for and who will feel the benefits?

One option for qualitative projects is to write an extended literature review. This type of project does not require you to collect any new data. Instead, you should focus on synthesising a broad range of literature to offer a new perspective on a research problem or question.  

The main difference between an extended literature review and a dissertation where primary data is collected, is in the presentation of the methodology, results and discussion sections. This is because extended literature reviews do not actively involve participants or primary data collection, so there is no need to outline a procedure for data collection (the methodology) or to present and interpret ‘data’ (in the form of interview transcripts, numerical data, observations etc.) You will have much more freedom to decide which sections of the dissertation should be combined, and whether new chapters or sections should be added.

Here is an overview of a common structure for an extended literature review:

Introduction

• Provide background information and context to set the ‘backdrop’ for your project.
• Explain the value and relevance of your research in this context. Outline what do you hope to contribute with your dissertation.
• Clarify a specific area of focus.
• Introduce your research aims (or problem) and objectives.

Literature review

You will need to write a short, overview literature review to introduce the main theories, concepts and key research areas that you will explore in your dissertation. This set of texts – which may be theoretical, research-based, practice-based or policies – form your theoretical framework. In other words, by bringing these texts together in the literature review, you are creating a lens that you can then apply to more focused examples or scenarios in your discussion chapters.

Methodology

As you will not be collecting primary data, your methodology will be quite different from a typical dissertation. You will need to set out the process and procedure you used to find and narrow down your literature. This is also known as a search strategy.

Including your search strategy

A search strategy explains how you have narrowed down your literature to identify key studies and areas of focus. This often takes the form of a search strategy table, included as an appendix at the end of the dissertation. If included, this section takes the place of the traditional 'methodology' section.

If you choose to include a search strategy table, you should also give an overview of your reading process in the main body of the dissertation.  Think of this as a chronology of the practical steps you took and your justification for doing so at each stage, such as:

• Your key terms, alternatives and synonyms, and any terms that you chose to exclude.
• Your choice and combination of databases;
• Your inclusion/exclusion criteria, when they were applied and why. This includes filters such as language of publication, date, and country of origin;
• You should also explain which terms you combined to form search phrases and your use of Boolean searching (AND, OR, NOT);
• Your use of citation searching (selecting articles from the bibliography of a chosen journal article to further your search).
• Your use of any search models, such as PICO and SPIDER to help shape your approach.
• Search strategy template A simple template for recording your literature searching. This can be included as an appendix to show your search strategy.

The discussion section of an extended literature review is the most flexible in terms of structure. Think of this section as a series of short case studies or ‘windows’ on your research. In this section you will apply the theoretical framework you formed in the literature review – a combination of theories, models and ideas that explain your approach to the topic – to a series of different examples and scenarios. These are usually presented as separate discussion ‘chapters’ in the dissertation, in an order that you feel best fits your argument.

Think about an order for these discussion sections or chapters that helps to tell the story of your research. One common approach is to structure these sections by common themes or concepts that help to draw your sources together. You might also opt for a chronological structure if your dissertation aims to show change or development over time. Another option is to deliberately show where there is a lack of chronology or narrative across your case studies, by ordering them in a fragmentary order! You will be able to reflect upon the structure of these chapters elsewhere in the dissertation, explaining and defending your decision in the methodology and conclusion.

A summary of your key findings – what you have concluded from your research, and how far you have been able to successfully answer your research questions.

• Recommendations – for improvements to your own study, for future research in the area, and for your field more widely.
• Emphasise your contributions to knowledge and what you have achieved.

Alternative structure

Depending on your research aims, and whether you are working with a case-study type approach (where each section of the dissertation considers a different example or concept through the lens established in your literature review), you might opt for one of the following structures:

Splitting the literature review across different chapters:

This structure allows you to pull apart the traditional literature review, introducing it little by little with each of your themed chapters. This approach works well for dissertations that attempt to show change or difference over time, as the relevant literature for that section or period can be introduced gradually to the reader.

Whichever structure you opt for, remember to explain and justify your approach. A marker will be interested in why you decided on your chosen structure, what it allows you to achieve/brings to the project and what alternatives you considered and rejected in the planning process. Here are some example sentence starters:

In qualitative studies, your results are often presented alongside the discussion, as it is difficult to include this data in a meaningful way without explanation and interpretation. In the dsicussion section, aim to structure your work thematically, moving through the key concepts or ideas that have emerged from your qualitative data. Use extracts from your data collection - interviews, focus groups, observations - to illustrate where these themes are most prominent, and refer back to the sources from your literature review to help draw conclusions. 

Here's an example of how your data could be presented in paragraph format in this section:

Example from  'Reporting and discussing your findings ', Monash University .

• << Previous: Planning your research
• Next: Quantitative research >>
• Last Updated: Apr 17, 2024 1:52 PM
• URL: https://libguides.shu.ac.uk/researchprojects

Literature Reviews

• Getting started

What is a literature review?

Why conduct a literature review, stages of a literature review, lit reviews: an overview (video), check out these books.

• Types of reviews
• 1. Define your research question
• 2. Plan your search
• 3. Search the literature
• 4. Organize your results
• 5. Synthesize your findings
• 6. Write the review
• Artificial intelligence (AI) tools
• Thompson Writing Studio This link opens in a new window
• Need to write a systematic review? This link opens in a new window

Contact a Librarian

Ask a Librarian

Definition: A literature review is a systematic examination and synthesis of existing scholarly research on a specific topic or subject.

Purpose: It serves to provide a comprehensive overview of the current state of knowledge within a particular field.

Analysis: Involves critically evaluating and summarizing key findings, methodologies, and debates found in academic literature.

Identifying Gaps: Aims to pinpoint areas where there is a lack of research or unresolved questions, highlighting opportunities for further investigation.

Contextualization: Enables researchers to understand how their work fits into the broader academic conversation and contributes to the existing body of knowledge.

tl;dr  A literature review critically examines and synthesizes existing scholarly research and publications on a specific topic to provide a comprehensive understanding of the current state of knowledge in the field.

What is a literature review NOT?

❌ An annotated bibliography

❌ Original research

❌ A summary

❌ Something to be conducted at the end of your research

❌ An opinion piece

❌ A chronological compilation of studies

The reason for conducting a literature review is to:

Literature Reviews: An Overview for Graduate Students

While this 9-minute video from NCSU is geared toward graduate students, it is useful for anyone conducting a literature review.

Writing the literature review: A practical guide

Available 3rd floor of Perkins

Writing literature reviews: A guide for students of the social and behavioral sciences

Available online!

So, you have to write a literature review: A guided workbook for engineers

Telling a research story: Writing a literature review

The literature review: Six steps to success

Systematic approaches to a successful literature review

Request from Duke Medical Center Library

Doing a systematic review: A student's guide

• Next: Types of reviews >>
• Last Updated: May 17, 2024 8:42 AM
• URL: https://guides.library.duke.edu/litreviews

Services for...

• Faculty & Instructors
• Graduate Students
• Undergraduate Students
• International Students
• Patrons with Disabilities

• Harmful Language Statement
• Re-use & Attribution / Privacy
• Support the Libraries

• Research article
• Open access
• Published: 16 May 2019

An analysis of current practices in undertaking literature reviews in nursing: findings from a focused mapping review and synthesis

• Helen Aveyard   ORCID: orcid.org/0000-0001-5133-3356 1 &
• Caroline Bradbury-Jones 2  

BMC Medical Research Methodology volume  19 , Article number:  105 ( 2019 ) Cite this article

40k Accesses

25 Citations

10 Altmetric

Metrics details

In this paper we discuss the emergence of many different methods for doing a literature review. Referring back to the early days, when there were essentially two types of review; a Cochrane systematic review and a narrative review, we identify how the term systematic review is now widely used to describe a variety of review types and how the number of available methods for doing a literature review has increased dramatically. This led us to undertake a review of current practice of those doing a literature review and the terms used to describe them.

We undertook a focused mapping review and synthesis. Literature reviews; defined as papers with the terms review or synthesis in the title, published in five nursing journals between January 2017–June 2018 were identified. We recorded the type of review and how these were undertaken.

We identified more than 35 terms used to describe a literature review. Some terms reflected established methods for doing a review whilst others could not be traced to established methods and/or the description of method in the paper was limited. We also found inconsistency in how the terms were used.

We have identified a proliferation of terms used to describe doing a literature review; although it is not clear how many distinct methods are being used. Our review indicates a move from an era when the term narrative review was used to describe all ‘non Cochrane’ reviews; to a time of expansion when alternative systematic approaches were developed to enhance rigour of such narrative reviews; to the current situation in which these approaches have proliferated to the extent so that the academic discipline of doing a literature review has become muddled and confusing. We argue that an ‘era of consolidation’ is needed in which those undertaking reviews are explicit about the method used and ensure that their processes can be traced back to a well described, original primary source.

Peer Review reports

Over the past twenty years in nursing, literature reviews have become an increasingly popular form of synthesising evidence and information relevant to the profession. Along with this there has been a proliferation of publications regarding the processes and practicalities of reviewing [ 1 , 2 , 3 , 4 ], This increase in activity and enthusiasm for undertaking literature reviews is paralleled by the foundation of the Cochrane Collaboration in 1993. Developed in response to the need for up-to-date reviews of evidence of the effectiveness of health care interventions, the Cochrane Collaboration introduced a rigorous method of searching, appraisal and analysis in the form of a ‘handbook’ for doing a systematic review [ 5 ] .Subsequently, similar procedural guidance has been produced, for example by the Centre for Reviews and Dissemination (CRD) [ 6 ] and The Joanna Briggs Institute [ 7 ]. Further guidance has been published to assist researchers with clarity in the reporting of published reviews [ 8 ].

In the early days of the literature review era, the methodological toolkit for those undertaking a literature was polarised, in a way that mirrored the paradigm wars of the time within mixed-methods research [ 9 ]. We refer to this as the ‘dichotomy era’ (i.e. the 1990s), The prominent methods of literature reviewing fell into one of two camps: The highly rigorous and systematic, mostly quantitative ‘Cochrane style’ review on one hand and a ‘narrative style’ review on the other hand, whereby a body of literature was summarised qualitatively, but the methods were often not articulated. Narrative reviews were particularly popular in dissertations and other student work (and they continue to be so in many cases) but have been criticised for a lack of systematic approach and consequently significant potential for bias in the findings [ 10 , 11 ].

The latter 1990s and early 2000, saw the emergence of other forms of review, developed as a response to the Cochrane/Narrative dichotomy. These alternative approaches to the Cochrane review provided researchers with reference points for performing reviews that drew on different study types, not just randomised controlled trials. They promoted a systematic and robust approach for all reviews, not just those concerned with effectiveness of interventions and treatments. One of the first published description of methods was Noblet and Hare’s (1998) ‘Meta-ethnography’ [ 12 ]. This method, although its name suggests otherwise, could incorporate and synthesise all types of qualitative research, not just ethnographies. The potential confusion regarding the inclusion of studies that were not ethnographies within a meta-ethnography, promoted the description of other similar methods, for example, the meta-synthesis of Walsh and Downe (2005) [ 13 ] and the thematic synthesis of Thomas and Harden (2008) [ 14 ]. Also, to overcome the dichotomy of the quantitative/qualitative reviews, the integrative review was described according to Whitemore and Knafl (2005) [ 15 ]. These reviews can be considered to be literature reviews that have been done in a systematic way but not necessarily adhering to guidelines established by the Cochrane Collaboration. We conceptualise this as the ‘expansion era’. Some of the methods are summarised in Table  1 .

Over the past two decades there has been a proliferation of review types, with corresponding explosion of terms used to describe them. A review of evidence synthesis methodologies by Grant and Booth in 2009 [ 20 ] identified 14 different approaches to reviewing the literature and similarly, Booth and colleagues [ 21 ] detailed 19 different review types, highlighting the range of review types currently available. We might consider this the ‘proliferation era’. This is however, somewhat a double-edged sword, because although researchers now have far more review methods at their disposal, there is risk of confusion in the field. As Sabatino and colleagues (2014) [ 22 ] have argued, review methods are not always consistently applied by researchers.

Aware of such potential inconsistency and also our own confusion at times regarding the range of review methods available, we questioned what was happening within our own discipline of nursing. We undertook a snap-shot, contemporary analysis to explore the range of terms used to describe reviews, the methods currently described in nursing and the underlying trends and patterns in searching, appraisal and analysis adopted by those doing a literature review. The aim was to gain some clarity on what is happening within the field, in order to understand, explain and critique what is happening within the proliferation era.

In order to explore current practices in doing a literature review, we undertook a ‘Focused Mapping Review and Synthesis’ (FMRS) – an approach that has been described only recently. This form of review [ 19 ] is a method of investigating trends in academic publications and has been used in a range of issues relevant to nursing and healthcare, for example, theory in qualitative research [ 23 ] and vicarious trauma in child protection research [ 24 ].

A FMRS seeks to identify what is happening within a particular subject or field of inquiry; hence the search is restricted to a particular time period and to pre-identified journals. The review has four distinct features: It: 1) focuses on identifying trends in an area rather than a body of evidence; 2) creates a descriptive map or topography of key features of research within the field rather than a synthesis of findings; 3) comments on the overall approach to knowledge production rather than the state of the evidence; 4) examines this within a broader epistemological context. These are translated into three specific focused activities: 1) targeted journals; 2) a specific subject; 3) a defined time period. The FMRS therefore, is distinct from other forms of review because it responds to questions concerned with ‘what is happening in this field?’ It was thus an ideal method to investigate current practices in literature reviews in nursing.

Using the international Scopus (2016) SCImago Journal and Country Rank, we identified the five highest ranked journals in nursing at that time of undertaking the review. There was no defined method for determining the number of journals to include in a review; the aim was to identify a sample and we identified five journals in order to search from a range of high ranking journals. We discuss the limitations of this later. Journals had to have ‘nursing’ or ‘nurse’ in the title and we did not include journals with a specialist focus, such as nutrition, cancer etcetera. The included journals are shown in Table  2 and are in order according to their ranking. We recognise that our journal choice meant that only articles published in English made it into the review.

A key decision in a FMRS is the time-period within which to retrieve relevant articles. Like many other forms of review, we undertook an initial scoping to determine the feasibility and parameters of the project [ 19 ]. In our previous reviews, the timeframe has varied from three months [ 23 ] to 6 years [ 24 ]. The main criterion is the likelihood for the timespan to contain sufficient articles to answer the review questions. We set the time parameter from January 2017–June 2018. We each took responsibility for two and three journals each from which to retrieve articles. We reviewed the content page of each issue of each journal. For our purposes, in order to reflect the diverse range of terms for describing a literature review, as described earlier in this paper, any paper that contained the term ‘review’ or ‘synthesis’ in the title was included in the review. This was done by each author individually but to enhance rigour, we worked in pairs to check each other’s retrieval processes to confirm inter-rater consistency. This process allowed any areas of uncertainty to be discussed and agreed and we found this form of calibration crucial to the process. The inclusion and exclusion criteria are shown in Table  3 .

Articles meeting the inclusion criteria, papers were read in full and data was extracted and recorded as per the proforma developed for the study (Table 4 ). The proforma was piloted on two papers to check for usability prior to data extraction. Data extraction was done independently but we discussed a selection of papers to enhance rigour of the process. No computer software was used in the analysis of the data. We did not critically appraise the included studies for quality because our purpose was to profile what is happening in the field rather than to draw conclusions from the included studies’ findings.

Once the details from all the papers had been extracted onto the tables, we undertook an analysis to identify common themes in the included articles. Because our aim was to produce a snap-shot profile, our analysis was thematic and conceptual. Although we undertook some tabulation and numerical analysis, our primary focus was on capturing patterns and trends characterised by the proliferation era. In line with the FMRS method, in the findings section we have used illustrative examples from the included articles that reflect and demonstrate the point or claim being made. These serve as useful sources of information and reference for readers seeking concrete examples.

Between January 2017 and June 2018 in the five journals we surveyed, a total of 222 papers with either ‘review’ or ‘synthesis’ in the title were retrieved and included in our analysis. We identified three primary themes: 1) Proliferation in names for doing a review; 2) Allegiance to an established review method; 3) Clarity about review processes. The results section is organised around these themes.

Proliferation in names for doing a review

We identified more than 35 terms used by authors to describe a literature review. Because we amalgamated terms such as ‘qualitative literature review’ and ‘qualitative review’ the exact number is actually slightly higher. It was clear from reading the reviews that many different terms were used to describe the same processes. For example qualitative systematic review, qualitative review and meta-synthesis, qualitative meta-synthesis, meta-ethnography all refer to a systematic review of qualitative studies. We have therefore grouped together the review types that refer to a particular type of review as described by the authors of the publications used in this study (Table  5 ).

In many reviews, the specific type of review was indicated in the title as seen for example in Table  5 . A striking feature was that all but two of the systematic reviews that contained a meta-analysis were labelled as such in the title; providing clarity and ease of retrieval. Where a literature review did not contain a meta-analysis, the title of the paper was typically referred to a ‘systematic review’; the implication being that a systematic review is not necessarily synonymous with a meta-analysis. However as discussed in the following section, this introduced some muddying of water, with different interpretations of what systematic review means and how broadly this term is applied. Some authors used the methodological type of included papers to describe their review. For example, a Cochrane-style systematic review was undertaken [ 25 ] but the reviewers did not undertake a meta-analysis and thus referred to their review as a ‘quantitative systematic review’.

Allegiance to an established literature review method

Many literature reviews demonstrated allegiance to a defined method and this was clearly and consistently described by the authors. For example, one team of reviewers [ 26 ] articulately described the process of a ‘meta-ethnography’ and gave a detailed description of their study and reference to the origins of the method by Noblet and Hare (1988) [ 12 ]. Another popular method was the ‘integrative review’ where most authors referred to the work of one or two seminal papers where the method was originally described (for example, Whitemore & Knafl 2005 [ 15 ]).

For many authors the term systematic review was used to mean a review of quantitative research, but some authors [ 27 , 28 , 29 ],used the term systematic review to describe reviews containing both qualitative and quantitative data.

However in many reviews, commitment to a method for doing a literature review appeared superficial, undeveloped and at times muddled. For example, three reviews [ 30 , 31 , 32 ] , indicate an integrative review in the title of their review, but this is the only reference to the method; there is no further reference to how the components of an integrative review are addressed within the paper. Other authors do not state allegiance to any particular method except to state a ‘literature review’ [ 33 ] but without an outline of a particular method for doing so. Anther review [ 34 ] reports a ‘narrative review’ but does not give further information about how this was done, possibly indicative of the lack of methods associated with the traditional narrative review. Three other reviewers documented how they searched, appraised and analysed their literature but do not reference an over-riding approach for their review [ 35 , 36 , 37 ]. In these examples, the review can be assumed to be a literature review, but the exact approach is not clear.

In other reviews, the methods for doing a literature review appear to be used interchangeably. For example in one review [ 38 ] the term narrative review was used in the title but in the main text an integrative review was described. In another review [ 39 ] two different and distinct methods were combined in a ‘meta-ethnographic meta-synthesis’.

Some authors [ 40 , 41 ] referred to a method used to undertake their review, for example a systematic review, but did not reference the primary source from where the method originated. Instead a secondary source, such as a textbook is used to reference the approach taken [ 20 , 42 ].

Clarity about review processes

Under this theme we discerned two principal issues: searching and appraisal. The majority of literature reviews contain three components- searching, appraisal and analysis, details of which are usually reported in the methods section of the papers. However, this is not always the case and for example, one review [ 43 ] provides only a search strategy with no information about the overall method or how critical appraisal or analysis were undertaken. Despite the importance of the process of analysis, we found little discussion of this in the papers reviewed.

The overwhelming trend for those doing a literature review was to describe a comprehensive search; although for many in our sample, a comprehensive search appeared to be limited to a database search; authors did not describe additional search strategies that would enable them to find studies that might be missed through electronic searching. Furthermore, authors did not define what a comprehensive search entailed, for example whether this included grey literature. We identified a very small number of studies where authors had undertaken a purposive sample [ 26 , 44 ]; in these reviews authors clearly stated that their search was for ‘seminal papers’ rather than all papers.

We reviewed the approaches to critical appraisal described in the papers and there were varying interpretations of what this means and which aspect of the included articles were to be subject to appraisal. Some authors [ 36 , 45 , 46 ] used the term ‘critical appraisal’ to refer to relevance of the paper to the review, rather than quality criteria. In that sense critical appraisal was used more as an inclusion criterion regarding relevance, rather than quality in the methods used. Mostly though, the term was used to describe the process of critical analysis of the methodological quality of included papers included in a review. When the term was used in this way to refer to quality criteria, appraisal tools were often used; for example, one review [ 47 ] provides a helpful example when they explain how a particular critical appraisal tool was used to asses the quality of papers in their review. Formal critical appraisal was undertaken by the vast majority reviewers, however the role of critical appraisal in the paper was often not explained [ 33 , 48 ]. It was common for a lot of detail to be provided about the approach to appraisal, including how papers were assessed and how disagreements between reviewers about the quality of individual papers were resolved, with no further mention of the subsequent role of the appraisal in the review. The reason for doing the critical appraisal in the review was often unclear and furthermore, in many cases, researchers included all papers within their review regardless of quality. For example, one team of reviewers [ 49 ] explained how the process, in their view, is not to exclude studies but to highlight the quality of evidence available. Another team of reviewers described how they did not exclude studies on the basis of quality because of the limited amount of research available on the topic [ 50 ].

Our review has identified a multiplicity of similar terms and approaches used by authors when doing a literature review, that we suggests marks the ‘proliferation era’. The expansion of terms used to describe a literature review has been observed previously [ 19 , 21 ]. We have identified an even wider range of terms, indicating that this trend may be increasing. This is likely to give the impression of an incoherent and potentially confusing approach to the scholarly undertaking of doing a literature review and is likely to be particularly problematic for novice researchers and students when attempting to grapple with the array of approaches available to them. The range of terms used in the title of papers to describe a literature review may cause both those new to research to wonder what the difference is between a qualitative evidence synthesis and a qualitative systematic review and which method is most suitable for their enquiry.

The clearest articles in our review were those that reported a systematic review with or without a meta-analysis. For example, one team of reviewers [ 25 ] undertook a Cochrane-style systematic review but did not undertake a meta-analysis and thus referred to their review as a ‘quantitative systematic review’. We found this form of labelling clear and helpful and is indeed in line with current recommendations [ 8 ]. While guidelines exist for the publication of systematic reviews [ 8 , 51 ], given the range of terms that are used by authors, some may be unclear when these guidelines should apply and this adds some confusion to the field. Of course, authors are at liberty to call their review processes whatever they deem appropriate, but our analysis has unearthed some inconsistencies that are confusing to the field of literature reviewing.

There is current debate about the status of literature reviews that are not ‘Cochrane’ style reviews [ 52 ]. Classification can be complex and whilst it might be tempting to refer to all non Cochrane-style reviews as ‘narrative reviews’ [ 52 ], literature reviews that conform to a recognised method would generally not be considered as such [ 53 ] and indeed the Cochrane Collaboration handbook refers to the principles of systematic review as applicable to different types of evidence, not just randomised controlled trials [ 5 ] .This raises the question as to whether the term systematic review should be an umbrella term referring to any review with an explicit method; which is implicit in the definition of a systematic review, but which raises the question as to how rigorous a method has to be to meet these standards, a thorny issue which we have identified in this study.

This review has identified a lack of detail in the reporting of the methods used by those doing a review. In 2017, Thorne raised the rhetorical question: ‘What kind of monster have we created?‘ [ 54 ]. Critiquing the growing investment in qualitative metasyntheses, she observed that many reviews were being undertaken that position themselves as qualitative metasyntheses, yet are theoretically and methodologically superficial. Thorne called for greater clarity and sense of purpose as the ‘trend in synthesis research marches forward’ [ 54 ]. Our review covered many review types, not just the qualitative meta-synthesis and its derivatives. However, we concur with Thorne’s conclusion that research methods are not extensively covered or debated in many of the published papers which might explain the confusion of terms and mixing of methods.

Despite the proliferation in terms for doing a literature review, and corresponding associated different methods and a lack of consistency in their application, our review has identified how the methods used (or indeed the reporting of the methods) appear to be remarkably similar in most publications. This may be due to limitations in the word count available to authors. However for example, the vast majority of papers describe a comprehensive search, critical appraisal and analysis. The approach to searching is of particular note; whilst comprehensive searching is the cornerstone of the Cochrane approach, other aproaches advocate that a sample of literature is sufficient [ 15 , 20 ]. Yet in our review we found only two examples where reviewers had used this approach, despite many other reviews claiming to be undertaking a meta-ethnography or meta-synthesis. This indicates that many of those doing a literature review have defaulted to the ‘comprehensive search’ irrespective of the approach to searching suggested in any particular method which is again indicative of confusion in the field.

Differences are reported in the approach to searching and critical appraisal and these appear not to be linked to different methods, but seem to be undertaken on the judgement and discretion of the reviewers without rationale or justification within the published paper. It is not for us to question researchers’ decisions as regards managing the flow of articles through their reviews, but when it comes to the issue of both searching and lack of clarity about the role of critical appraisal there is evidence of inconsistency by those doing a literature review. This reflects current observations in the literature where the lack of clarity about the role of critical appraisal within a literature review is debated . [ 55 , 56 ].

Our review indicates that many researchers follow a very similar process, regardless of their chosen method and the real differences that do exist between published methods are not apparent in many of the published reviews. This concurs with previously mentioned concerns [ 54 ] about the superficial manner in which methods are explored within literature reviews. The overriding tendency is to undertake a comprehensive review, critical appraisal and analysis, following the formula prescribed by Cochrane, even if this is not required by the literature review method stated in the paper. Other researchers [ 52 ] have questioned whether the dominance of the Cochrane review should be questioned. We argue that emergence of different methods for doing a literature review in a systematic way has indeed challenged the perceived dominance of the Cochrane approach that characterised the dichotomy era, where the only alternative was a less rigourous and often poorly described process of dealing with literature. It is positive that there is widespread acknowledgement of the validity of other approaches. But we argue that the expansion era, whereby robust processes were put forward as alternatives that filled the gap left by polarisation, has gone too far. The magnitude in the number of different approaches identified in this review has led to a confused field. Thorne [ 54 ] refers to a ‘meta-madness’; with the proliferation of methods leading to the oversimplification of complex literature and ideas. We would extend this to describe a ‘meta-muddle’ in which, not only are the methods and results oversimplified, but the existence of so many terms used to describe a literature review, many of them used interchangeably, has added a confusion to the field and prevented the in-depth exploration and development of specific methods. Table  6 shows the issues associated with the proliferation era and importantly, it also highlights the recommendations that might lead to a more coherent reviewing community in nursing.

The terms used for doing a literature review are often used both interchangeably and inconsistently, with minimal description of the methods undertaken. It is not surprising therefore that some journal editors do not index these consistently within the journal. For example, in one edition of one journal included in the review, there are two published integrative reviews. One is indexed in the section entitled as a ‘systematic review’, while the other is indexed in a separate section entitled ‘literature review’. In another edition of a journal, two systematic reviews with meta-analysis are published. One is listed as a research article and the other as a review and discussion paper. It seems to us then, that editors and publishers might sometimes also be confused and bewildered themselves.

Whilst guidance does exist for the publication of some types of systematic reviews in academic journals; for example the PRISMA statement [ 8 ] and Entreq guidelines [ 51 ], which are specific to particular qualitative synthesis, guidelines do not exist for each approach. As a result, for those doing an alternative approach to their literature review, for example an integrative review [ 15 ], there is only general publication guidance to assist. In the current reviewing environment, there are so many terms, that more specific guidance would be impractical anyway. However, greater clarity about the methods used and halting the introduction of different terms to mean the same thing will be helpful.

Limitations

This study provides a snapshot of the way in which literature reviews have been described within a short publication timeframe. We were limited for practical reasons to a small section of high impact journals. Including a wider range of journals would have enhanced the transferability of the findings. Our discussion is, of course, limited to the review types that were published in the timeframe, in the identified journals and which had the term ‘review’ or ‘synthesis’ in the title. This would have excluded papers that were entitled ‘meta-analysis’. However as we were interested in the range of reviews that fall outside the scope of a meta-analysis, we did not consider that this limited the scope of the paper. Our review is further limited by the lack of detail of the methods undertaken provided in many of the papers reviewed which, although providing evidence for our arguments, also meant that we had to assume meaning that was unclear from the text provided.

The development of rigorous methods for doing a literature review is to be welcomed; not all review questions can be answered by Cochrane style reviews and robust methods are needed to answer review questions of all types. Therefore whilst we welcome the expansion in methods for doing a literature review, the proliferation in the number of named approaches should be, in our view, a cause for reflection. The increase in methods could be indicative of an emerging variation in possible approaches; alternatively, the increase could be due to a lack of conceptual clarity where, on closer inspection, the methods do not differ greatly and could indeed be merged. Further scrutiny of the methods described within many papers support the latter situation but we would welcome further discussion about this. Meanwhile, we urge researchers to make careful consideration of the method they adopt for doing a literature review, to justify this approach carefully and to adhere closely to its method. Having witnessed an era of dichotomy, expansion and proliferation of methods for doing a literature review, we now seek a new era of consolidation.

Bettany-Saltikov J. How to do a systematic literature review in nursing: a step-by-step guide. Maidenhead: Open University Press; 2012.

Google Scholar  

Coughlan M, Cronin P, Ryan F. Doing a literature review in nursing, Health and social care. London: Sage; 2013.

Aveyard H. Doing a literature review in health and social care. Maidenhead: Open University Press; 2018.

Davis D. A practical overview of how to conduct a systematic review. Nurs Stand. 2016;31(12):60–70.

Article   Google Scholar  

Higgins and Green. Cochrane handbook for systematic reviews of interventions version 5.1.0: Cochrane Collaboration; 2011.

Centre for Reviews and Dissemination. Systematic Reviews: CRD's guidance for undertaking reviews in health care. York: University of York; 2008.

Aromataris E, Munn Z, editors. Joanna Briggs Institute Reviewer's Manual: The Joanna Briggs Institute; 2017. Available from https://reviewersmanual.joannabriggs.org/

Shamseer L, Moher D, Clarke M, Ghersi D, Liberati A, Petticrew M, Shekelle P, Stewart L. Preferred reporting items for systematic review and meta-analysis protocols. Br Med J. 2015;2:349 (Jan 02).

Creswell JW, Plano-Clark VL. Designing and conducting mixed methods research. 2nd ed. Thousand Oaks: Sage Publications; 2011.

Greenhalgh T. How to read a paper. 4th ed: Wiley; 2010.

Booth A, Papaioannou D, Sutton A. Systematic appproaches to a successful literature review Sage London; 2012.

Noblit GW, Hare RD. Meta-ethnography, synthesising qualitative studies, qualitative research methods, volume 11. London: SAGE Publications; 1988.

Walsh D, Downe S. Meta-synthesis method for qualitative research: a literature review. J Adv Nurs. 2005;50(2):204–11.

Thomas J, Harden A. Methods for the thematic synthesis of qualitative research in systematic reviews. BMC Med Res Methodol. 2008;8:45.

Whittemore R, Knafl K. The integrative review: updated methodology. J Adv Nurs. 2005;52:546–53.

Scoping Peters MDJ, Godfrey CM, McInerney P, Soares CB, Khalil H, Parker D. Methodology for Joanna Briggs institute scoping review. Joanna Briggs institute reviewers manual: Australia; 2015.

Wong G, Greenhalgh T, Westhorp G, Buckingham J, Pawson R. RAMESES publication standards: realist synthesis BMC medicine, vol. 11; 2013. p. 21.

Plüddemann A, Aronson JK, Onakpoya I, Heneghan C, Mahtani KR. Redefining rapid reviews: a flexible framework for restricted systematic reviews. BMJ Evidence-Based Medicine Epub ahead of print: 27 June 2018. https://doi.org/10.1136/bmjemb-2018-110990 .

Bradbury-Jones C, Breckenridge J, Clark MT, Herber OR, Jones C, Taylor J. Advancing the science of literature reviewing: the focused mapping review and synthesis as a novel approach. Int J Soc Res Methodol. https://doi.org/10.1080/13645579.2019.1576328 .

Grant MJ, Booth A. A typology of review- an analysis of 14 review types and associated methodologies. Health Info Libr J. 2009;26(2):91–108.

Booth, A., Noyes, J., Flemming, K., Gerhardus, A., Wahlster, P., van der Wilt G.J., Mozygemba K, Refolo P, Sacchini D, Tummers, M, Rehfuess, E. (2016) Guidance on choosing qualitative evidence synthesis methods for use in health technology assessments of complex interventions. Available: http://www.integrate-hta.eu/wp-content/uploads/2016/02/Guidance-on-choosing-qualitative-evidence-synthesis-methods-for-use-in-HTA-of-complex-interventions.pdf

Sabatino L, Stievano A, Rocco G, Kallio H, Pietila A, KAngasniemi M. The dignity of the nursing profession: a meta-synthesis of qualitative research. Nurs Ethics. 2014;2(6):659–72.

Bradbury-Jones C, Taylor J, Herber O. How theory is used and articulated in qualitative research: development of a new typology. Soc Sci Med. 2014;120:135–41.

Taylor J, Bradbury-Jones C, Breckenridge J, Jones C, Herber OR. Risk of vicarious trauma in nursing research: a focused mapping review and synthesis. J Clin Nurs. 2016. https://doi.org/10.1111/jocn.13235 .

Haggman Laitila A, Rompannen J. Outcomes of interventions for nurse leaders’ well being at work. A quantitative systematic review. J Adv Nurs. 2018;74:34–44.

Strandos M, Bondas T. The nurse patient relationship as a story of health enhancement in community care- a meta-ethnography. J Adv Nurs. 2018;74:11–8.

Gilissen J, Pivodic L, Smets T, Gastmans C, Stichels RV, Delieus L, Van den Black L. Preconditions for successful advanced care planning in nursing homes: a systematic review. Int J Nurs Stud. 2017;66:47–59.

Walczak A, Mcdonald F, Patterson P, Dobinson K, Kimberley A. How does parental cancer affect adolescent and young adult offspring. A systematic review. Int J Nurs Stud. 2018;77:54–80.

Leyva-Moral JM, Palmoero PA, Feijoo-Cid M, Edwards JE. Reproductive decision making in women living with HIV: systematic review. Int J Nurs Stud. 2018;77:207–21.

Pires S, Monteiro S, Pereira A, Chaló D, Melo E, Rodriguese A. Non technical skills assessment for pre-licensure nursing students: an integrative literature review nurse education today, vol. 58; 2017. p. 19–24.

Wilkinson A, Meilkle N, Law P, Yong A, Butler P, Kim J, Mulligan H, Hale L. How older adults and their informal carers prevent falls: an integrative review. Int J Nurs Stud. 2018;82:13–8.

Drewniak D, Krones T, Wild V. Do attitudes and behaviours of health care professionals exacerbate health care disparities among immigrant and ethnic minority groups? An integrative literature review. Int J Nurs Stud. 2017;70:89–98.

Garone A, Craen Van de P. The role of language skills abd internationalisation in nursing degree programmes: a literature review. Nurse Educ Today. 2017;49:140–4.

Casey M, O’Connor L, Cashin A (et al) An overview of the outcomes and impact of specialist and advanced nursing and midwifery practice on quality of care, cost and access to services: A narrative review. Nurse Educ Today 2017;56:35-40.

Adib-Hajbaghery M, Sharifi N. Effect of simulation training on the development of nurses and nursing students critical thinking: a systematic review. Nurse Educ Today. 2017;50:17–24.

Irwin C, Bliss J, Poole J. Does preceptorship improve confidence and competence in newly qualified nurses: a systematic literature review. Nurse educ Today. 2018;60:35–46.

Jefferies D, McNallya S, Roberts K, Wallace A, Stunden A, D'Souzaa S, Glew P. The importance of academic literacy for undergraduate nursing students and its relationship to future professional clinical practice: a systematic review. Nurse educ Today. 2018;60:84–91.

Lewis ML, Neville C, Ashkanasy NM. Emotional intelligence and affective events in nurse education- a narrative review. Nurse Educ Today. 2017;53:34–40.

Jensen D, Sorensen A. Nurses experiences of working in organisations undergoing restructuring: a meta-synthesis of qualitative research studies. Int J Nurs Stud. 2017;66:7–14.

Milligan F, Wareing M, Preston-Shoot M, Pappas Y, Randhawa G, Bhandol J. Supporting nursing, midwifery and allied health professional students to raise concerns with the quality of care: a review of the research literature. Nurse Educ Today. 2017;57:29–39.

Rozendo CA, Salas AS. A critical review of social and health inequalities in the nursing curriculum. Nurse Educ Today. 2017;50:62–71.

Kiteley R, Stogdon C. Literature reviews in social work. London: Sage; 2014.

Book   Google Scholar  

Rebeiro G, Evans A, Edward K, Chapman R. Registered nurse buddies. Educators by proxy? Nurse Educ Today. 2017;55:1–4.

Sinclair S, Raffin Bouchal S, Venturato L, Milsonic-Kondejewski J, Smith Macdonald L. Compassion fatigue: a meta-narrative review of the health care literature. Int J Nurs Stud. 2017;69:9–24.

Hovey S, Dyck MJ, Reese C, Myoung JK. Nursing students’ attitudes towards persons who are aged: an integrative review. J Adv Nurs. 2017;49:145–52.

Granheim B, Shaw J, Mansah M. Use of interprofessional learning and simulation in undergraduate nursing programmes to address interprofessional communication and collaboration- an integrative review. Nurse Educ Today. 2018;62:118–27.

Philips P, Lumley E, Duncan R, Aber A, Buckley Woods H, Jones GC. A systematic review of qualitative research into people’s experiences of living with venous leg ulcers. J Adv Nurs. 2018;74:550–63.

Slater CE, Cusick A. Factors relating to self directed learning readiness of students in health professional programmes a scoping review. Nurse Educ Today. 2017;52:22–33.

Vanderspank-Wright B, Efstathiou N, Vandjk A. Critical care nurses experience of withdrawing treatment: a systematic review of qualitative evidence. Int J Nurs Stud. 2018;77:15–26.

Kelly M, Wills J, Sykes S. Do nurses’ personal health behaviours impact their health promotion practice? A systematic review. Int J Nurs Stud. 2017;76:62–77.

Tong A, Flemming K, McInnes E, Oliver S, Craig J. Enhancing transparency in the reporting in the synthesis of qualitative research. BMC Med Res Methodol. 2012;12(1):181.

Greenhalgh T, Thorne S, Malterud K. Time to challenge the spurious hierarchy of systematic over narrative reviews? Eur J Clin Investig. 2018. https://doi.org/10.1111/eci12931 .

Aveyard H, Payne S, Preston N. A postgraduate’s guide to doing a literature review. Maidenhead: Open University Press; 2016.

Thorne S. Metasynthetic madness: what kind of monster have we created? Qual Health Res. 2017;27(1):3–12.

Sibeoni J, Orri M, Colin S, Valentin M, Pradere J, Revah-Levy A. The lived experience of anorexia nervosa in adolescence, comparison of parents’ view of adolescence, parents and professionals: a meta-synthesis. Int J Nurs Stud. 2017;65:25–34.

Nightingale S, Spiby H, Sheen K, Slade P. The impact of emotional intelligence in long term health care professionals on caring behaviours towards patients in clinical and long term settings. Integrative review. Int J Nurs Stud. 2018;80:106–17.

Download references

Acknowledgements

Availability of data and materials.

The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.

Author information

Authors and affiliations.

Faculty of Health and Life Sciences, Oxford Brookes University, Jack Straw’s Lane, Oxford, OX3 0FL, England, UK

Helen Aveyard

University of Birmingham, Birmingham, England, UK

Caroline Bradbury-Jones

You can also search for this author in PubMed   Google Scholar

Contributions

Both HA and CB-J contributed to the data collection and analysis. HA wrote the paper and CB-J commented on the drafts. HA revised the paper according to the reviewers’ comments and CB-J commented on these revisions 30/4/19. Both authors read and approved the final manuscript.

Corresponding author

Correspondence to Helen Aveyard .

Ethics declarations

Ethics approval and consent to participate.

not applicable.

Consent for publication

Competing interests, publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License ( http://creativecommons.org/licenses/by/4.0/ ), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Cite this article.

Aveyard, H., Bradbury-Jones, C. An analysis of current practices in undertaking literature reviews in nursing: findings from a focused mapping review and synthesis. BMC Med Res Methodol 19 , 105 (2019). https://doi.org/10.1186/s12874-019-0751-7

Download citation

Received : 16 December 2018

Accepted : 07 May 2019

Published : 16 May 2019

DOI : https://doi.org/10.1186/s12874-019-0751-7

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Evidence synthesis
• Literature review
• Meta-ethnography
• Systematic review

BMC Medical Research Methodology

ISSN: 1471-2288

• General enquiries: [email protected]

Purdue Online Writing Lab Purdue OWL® College of Liberal Arts

Writing a Literature Review

Welcome to the Purdue OWL

This page is brought to you by the OWL at Purdue University. When printing this page, you must include the entire legal notice.

Copyright ©1995-2018 by The Writing Lab & The OWL at Purdue and Purdue University. All rights reserved. This material may not be published, reproduced, broadcast, rewritten, or redistributed without permission. Use of this site constitutes acceptance of our terms and conditions of fair use.

A literature review is a document or section of a document that collects key sources on a topic and discusses those sources in conversation with each other (also called synthesis ). The lit review is an important genre in many disciplines, not just literature (i.e., the study of works of literature such as novels and plays). When we say “literature review” or refer to “the literature,” we are talking about the research ( scholarship ) in a given field. You will often see the terms “the research,” “the scholarship,” and “the literature” used mostly interchangeably.

Where, when, and why would I write a lit review?

There are a number of different situations where you might write a literature review, each with slightly different expectations; different disciplines, too, have field-specific expectations for what a literature review is and does. For instance, in the humanities, authors might include more overt argumentation and interpretation of source material in their literature reviews, whereas in the sciences, authors are more likely to report study designs and results in their literature reviews; these differences reflect these disciplines’ purposes and conventions in scholarship. You should always look at examples from your own discipline and talk to professors or mentors in your field to be sure you understand your discipline’s conventions, for literature reviews as well as for any other genre.

A literature review can be a part of a research paper or scholarly article, usually falling after the introduction and before the research methods sections. In these cases, the lit review just needs to cover scholarship that is important to the issue you are writing about; sometimes it will also cover key sources that informed your research methodology.

Lit reviews can also be standalone pieces, either as assignments in a class or as publications. In a class, a lit review may be assigned to help students familiarize themselves with a topic and with scholarship in their field, get an idea of the other researchers working on the topic they’re interested in, find gaps in existing research in order to propose new projects, and/or develop a theoretical framework and methodology for later research. As a publication, a lit review usually is meant to help make other scholars’ lives easier by collecting and summarizing, synthesizing, and analyzing existing research on a topic. This can be especially helpful for students or scholars getting into a new research area, or for directing an entire community of scholars toward questions that have not yet been answered.

What are the parts of a lit review?

Most lit reviews use a basic introduction-body-conclusion structure; if your lit review is part of a larger paper, the introduction and conclusion pieces may be just a few sentences while you focus most of your attention on the body. If your lit review is a standalone piece, the introduction and conclusion take up more space and give you a place to discuss your goals, research methods, and conclusions separately from where you discuss the literature itself.

Introduction:

• An introductory paragraph that explains what your working topic and thesis is
• A forecast of key topics or texts that will appear in the review
• Potentially, a description of how you found sources and how you analyzed them for inclusion and discussion in the review (more often found in published, standalone literature reviews than in lit review sections in an article or research paper)
• Summarize and synthesize: Give an overview of the main points of each source and combine them into a coherent whole
• Analyze and interpret: Don’t just paraphrase other researchers – add your own interpretations where possible, discussing the significance of findings in relation to the literature as a whole
• Critically Evaluate: Mention the strengths and weaknesses of your sources
• Write in well-structured paragraphs: Use transition words and topic sentence to draw connections, comparisons, and contrasts.

Conclusion:

• Summarize the key findings you have taken from the literature and emphasize their significance
• Connect it back to your primary research question

How should I organize my lit review?

Lit reviews can take many different organizational patterns depending on what you are trying to accomplish with the review. Here are some examples:

• Chronological : The simplest approach is to trace the development of the topic over time, which helps familiarize the audience with the topic (for instance if you are introducing something that is not commonly known in your field). If you choose this strategy, be careful to avoid simply listing and summarizing sources in order. Try to analyze the patterns, turning points, and key debates that have shaped the direction of the field. Give your interpretation of how and why certain developments occurred (as mentioned previously, this may not be appropriate in your discipline — check with a teacher or mentor if you’re unsure).
• Thematic : If you have found some recurring central themes that you will continue working with throughout your piece, you can organize your literature review into subsections that address different aspects of the topic. For example, if you are reviewing literature about women and religion, key themes can include the role of women in churches and the religious attitude towards women.
• Qualitative versus quantitative research
• Empirical versus theoretical scholarship
• Divide the research by sociological, historical, or cultural sources
• Theoretical : In many humanities articles, the literature review is the foundation for the theoretical framework. You can use it to discuss various theories, models, and definitions of key concepts. You can argue for the relevance of a specific theoretical approach or combine various theorical concepts to create a framework for your research.

What are some strategies or tips I can use while writing my lit review?

Any lit review is only as good as the research it discusses; make sure your sources are well-chosen and your research is thorough. Don’t be afraid to do more research if you discover a new thread as you’re writing. More info on the research process is available in our "Conducting Research" resources .

As you’re doing your research, create an annotated bibliography ( see our page on the this type of document ). Much of the information used in an annotated bibliography can be used also in a literature review, so you’ll be not only partially drafting your lit review as you research, but also developing your sense of the larger conversation going on among scholars, professionals, and any other stakeholders in your topic.

Usually you will need to synthesize research rather than just summarizing it. This means drawing connections between sources to create a picture of the scholarly conversation on a topic over time. Many student writers struggle to synthesize because they feel they don’t have anything to add to the scholars they are citing; here are some strategies to help you:

• It often helps to remember that the point of these kinds of syntheses is to show your readers how you understand your research, to help them read the rest of your paper.
• Writing teachers often say synthesis is like hosting a dinner party: imagine all your sources are together in a room, discussing your topic. What are they saying to each other?
• Look at the in-text citations in each paragraph. Are you citing just one source for each paragraph? This usually indicates summary only. When you have multiple sources cited in a paragraph, you are more likely to be synthesizing them (not always, but often
• Read more about synthesis here.

The most interesting literature reviews are often written as arguments (again, as mentioned at the beginning of the page, this is discipline-specific and doesn’t work for all situations). Often, the literature review is where you can establish your research as filling a particular gap or as relevant in a particular way. You have some chance to do this in your introduction in an article, but the literature review section gives a more extended opportunity to establish the conversation in the way you would like your readers to see it. You can choose the intellectual lineage you would like to be part of and whose definitions matter most to your thinking (mostly humanities-specific, but this goes for sciences as well). In addressing these points, you argue for your place in the conversation, which tends to make the lit review more compelling than a simple reporting of other sources.

Log in using your username and password

• Search More Search for this keyword Advanced search
• Latest content
• Current issue
• Write for Us
• BMJ Journals More You are viewing from: Google Indexer

You are here

• Volume 19, Issue 1
• Reviewing the literature
• Article Text
• Article info
• Citation Tools
• Rapid Responses
• Article metrics

• Joanna Smith 1 ,
• Helen Noble 2
• 1 School of Healthcare, University of Leeds , Leeds , UK
• 2 School of Nursing and Midwifery, Queens's University Belfast , Belfast , UK
• Correspondence to Dr Joanna Smith , School of Healthcare, University of Leeds, Leeds LS2 9JT, UK; j.e.smith1{at}leeds.ac.uk

https://doi.org/10.1136/eb-2015-102252

Statistics from Altmetric.com

Request permissions.

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Implementing evidence into practice requires nurses to identify, critically appraise and synthesise research. This may require a comprehensive literature review: this article aims to outline the approaches and stages required and provides a working example of a published review.

Are there different approaches to undertaking a literature review?

What stages are required to undertake a literature review.

The rationale for the review should be established; consider why the review is important and relevant to patient care/safety or service delivery. For example, Noble et al 's 4 review sought to understand and make recommendations for practice and research in relation to dialysis refusal and withdrawal in patients with end-stage renal disease, an area of care previously poorly described. If appropriate, highlight relevant policies and theoretical perspectives that might guide the review. Once the key issues related to the topic, including the challenges encountered in clinical practice, have been identified formulate a clear question, and/or develop an aim and specific objectives. The type of review undertaken is influenced by the purpose of the review and resources available. However, the stages or methods used to undertake a review are similar across approaches and include:

Formulating clear inclusion and exclusion criteria, for example, patient groups, ages, conditions/treatments, sources of evidence/research designs;

Justifying data bases and years searched, and whether strategies including hand searching of journals, conference proceedings and research not indexed in data bases (grey literature) will be undertaken;

Developing search terms, the PICU (P: patient, problem or population; I: intervention; C: comparison; O: outcome) framework is a useful guide when developing search terms;

Developing search skills (eg, understanding Boolean Operators, in particular the use of AND/OR) and knowledge of how data bases index topics (eg, MeSH headings). Working with a librarian experienced in undertaking health searches is invaluable when developing a search.

Once studies are selected, the quality of the research/evidence requires evaluation. Using a quality appraisal tool, such as the Critical Appraisal Skills Programme (CASP) tools, 5 results in a structured approach to assessing the rigour of studies being reviewed. 3 Approaches to data synthesis for quantitative studies may include a meta-analysis (statistical analysis of data from multiple studies of similar designs that have addressed the same question), or findings can be reported descriptively. 6 Methods applicable for synthesising qualitative studies include meta-ethnography (themes and concepts from different studies are explored and brought together using approaches similar to qualitative data analysis methods), narrative summary, thematic analysis and content analysis. 7 Table 1 outlines the stages undertaken for a published review that summarised research about parents’ experiences of living with a child with a long-term condition. 8

• View inline

An example of rapid evidence assessment review

In summary, the type of literature review depends on the review purpose. For the novice reviewer undertaking a review can be a daunting and complex process; by following the stages outlined and being systematic a robust review is achievable. The importance of literature reviews should not be underestimated—they help summarise and make sense of an increasingly vast body of research promoting best evidence-based practice.

• ↵ Centre for Reviews and Dissemination . Guidance for undertaking reviews in health care . 3rd edn . York : CRD, York University , 2009 .
• ↵ Canadian Best Practices Portal. http://cbpp-pcpe.phac-aspc.gc.ca/interventions/selected-systematic-review-sites / ( accessed 7.8.2015 ).
• Bridges J , et al
• ↵ Critical Appraisal Skills Programme (CASP). http://www.casp-uk.net / ( accessed 7.8.2015 ).
• Dixon-Woods M ,
• Shaw R , et al
• Agarwal S ,
• Jones D , et al
• Cheater F ,

Twitter Follow Joanna Smith at @josmith175

Competing interests None declared.

Read the full text or download the PDF:

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• J Grad Med Educ
• v.8(3); 2016 Jul

The Literature Review: A Foundation for High-Quality Medical Education Research

a  These are subscription resources. Researchers should check with their librarian to determine their access rights.

Despite a surge in published scholarship in medical education 1 and rapid growth in journals that publish educational research, manuscript acceptance rates continue to fall. 2 Failure to conduct a thorough, accurate, and up-to-date literature review identifying an important problem and placing the study in context is consistently identified as one of the top reasons for rejection. 3 , 4 The purpose of this editorial is to provide a road map and practical recommendations for planning a literature review. By understanding the goals of a literature review and following a few basic processes, authors can enhance both the quality of their educational research and the likelihood of publication in the Journal of Graduate Medical Education ( JGME ) and in other journals.

The Literature Review Defined

In medical education, no organization has articulated a formal definition of a literature review for a research paper; thus, a literature review can take a number of forms. Depending on the type of article, target journal, and specific topic, these forms will vary in methodology, rigor, and depth. Several organizations have published guidelines for conducting an intensive literature search intended for formal systematic reviews, both broadly (eg, PRISMA) 5 and within medical education, 6 and there are excellent commentaries to guide authors of systematic reviews. 7 , 8

• A literature review forms the basis for high-quality medical education research and helps maximize relevance, originality, generalizability, and impact.
• A literature review provides context, informs methodology, maximizes innovation, avoids duplicative research, and ensures that professional standards are met.
• Literature reviews take time, are iterative, and should continue throughout the research process.
• Researchers should maximize the use of human resources (librarians, colleagues), search tools (databases/search engines), and existing literature (related articles).
• Keeping organized is critical.

Such work is outside the scope of this article, which focuses on literature reviews to inform reports of original medical education research. We define such a literature review as a synthetic review and summary of what is known and unknown regarding the topic of a scholarly body of work, including the current work's place within the existing knowledge . While this type of literature review may not require the intensive search processes mandated by systematic reviews, it merits a thoughtful and rigorous approach.

Purpose and Importance of the Literature Review

An understanding of the current literature is critical for all phases of a research study. Lingard 9 recently invoked the “journal-as-conversation” metaphor as a way of understanding how one's research fits into the larger medical education conversation. As she described it: “Imagine yourself joining a conversation at a social event. After you hang about eavesdropping to get the drift of what's being said (the conversational equivalent of the literature review), you join the conversation with a contribution that signals your shared interest in the topic, your knowledge of what's already been said, and your intention.” 9

The literature review helps any researcher “join the conversation” by providing context, informing methodology, identifying innovation, minimizing duplicative research, and ensuring that professional standards are met. Understanding the current literature also promotes scholarship, as proposed by Boyer, 10 by contributing to 5 of the 6 standards by which scholarly work should be evaluated. 11 Specifically, the review helps the researcher (1) articulate clear goals, (2) show evidence of adequate preparation, (3) select appropriate methods, (4) communicate relevant results, and (5) engage in reflective critique.

Failure to conduct a high-quality literature review is associated with several problems identified in the medical education literature, including studies that are repetitive, not grounded in theory, methodologically weak, and fail to expand knowledge beyond a single setting. 12 Indeed, medical education scholars complain that many studies repeat work already published and contribute little new knowledge—a likely cause of which is failure to conduct a proper literature review. 3 , 4

Likewise, studies that lack theoretical grounding or a conceptual framework make study design and interpretation difficult. 13 When theory is used in medical education studies, it is often invoked at a superficial level. As Norman 14 noted, when theory is used appropriately, it helps articulate variables that might be linked together and why, and it allows the researcher to make hypotheses and define a study's context and scope. Ultimately, a proper literature review is a first critical step toward identifying relevant conceptual frameworks.

Another problem is that many medical education studies are methodologically weak. 12 Good research requires trained investigators who can articulate relevant research questions, operationally define variables of interest, and choose the best method for specific research questions. Conducting a proper literature review helps both novice and experienced researchers select rigorous research methodologies.

Finally, many studies in medical education are “one-offs,” that is, single studies undertaken because the opportunity presented itself locally. Such studies frequently are not oriented toward progressive knowledge building and generalization to other settings. A firm grasp of the literature can encourage a programmatic approach to research.

Approaching the Literature Review

Considering these issues, journals have a responsibility to demand from authors a thoughtful synthesis of their study's position within the field, and it is the authors' responsibility to provide such a synthesis, based on a literature review. The aforementioned purposes of the literature review mandate that the review occurs throughout all phases of a study, from conception and design, to implementation and analysis, to manuscript preparation and submission.

Planning the literature review requires understanding of journal requirements, which vary greatly by journal ( table 1 ). Authors are advised to take note of common problems with reporting results of the literature review. Table 2 lists the most common problems that we have encountered as authors, reviewers, and editors.

Sample of Journals' Author Instructions for Literature Reviews Conducted as Part of Original Research Article a

Common Problem Areas for Reporting Literature Reviews in the Context of Scholarly Articles

Locating and Organizing the Literature

Three resources may facilitate identifying relevant literature: human resources, search tools, and related literature. As the process requires time, it is important to begin searching for literature early in the process (ie, the study design phase). Identifying and understanding relevant studies will increase the likelihood of designing a relevant, adaptable, generalizable, and novel study that is based on educational or learning theory and can maximize impact.

Human Resources

A medical librarian can help translate research interests into an effective search strategy, familiarize researchers with available information resources, provide information on organizing information, and introduce strategies for keeping current with emerging research. Often, librarians are also aware of research across their institutions and may be able to connect researchers with similar interests. Reaching out to colleagues for suggestions may help researchers quickly locate resources that would not otherwise be on their radar.

During this process, researchers will likely identify other researchers writing on aspects of their topic. Researchers should consider searching for the publications of these relevant researchers (see table 3 for search strategies). Additionally, institutional websites may include curriculum vitae of such relevant faculty with access to their entire publication record, including difficult to locate publications, such as book chapters, dissertations, and technical reports.

Strategies for Finding Related Researcher Publications in Databases and Search Engines

Search Tools and Related Literature

Researchers will locate the majority of needed information using databases and search engines. Excellent resources are available to guide researchers in the mechanics of literature searches. 15 , 16

Because medical education research draws on a variety of disciplines, researchers should include search tools with coverage beyond medicine (eg, psychology, nursing, education, and anthropology) and that cover several publication types, such as reports, standards, conference abstracts, and book chapters (see the box for several information resources). Many search tools include options for viewing citations of selected articles. Examining cited references provides additional articles for review and a sense of the influence of the selected article on its field.

Box Information Resources

• Web of Science a
• Education Resource Information Center (ERIC)
• Cumulative Index of Nursing & Allied Health (CINAHL) a
• Google Scholar

Once relevant articles are located, it is useful to mine those articles for additional citations. One strategy is to examine references of key articles, especially review articles, for relevant citations.

Getting Organized

As the aforementioned resources will likely provide a tremendous amount of information, organization is crucial. Researchers should determine which details are most important to their study (eg, participants, setting, methods, and outcomes) and generate a strategy for keeping those details organized and accessible. Increasingly, researchers utilize digital tools, such as Evernote, to capture such information, which enables accessibility across digital workspaces and search capabilities. Use of citation managers can also be helpful as they store citations and, in some cases, can generate bibliographies ( table 4 ).

Citation Managers

Knowing When to Say When

Researchers often ask how to know when they have located enough citations. Unfortunately, there is no magic or ideal number of citations to collect. One strategy for checking coverage of the literature is to inspect references of relevant articles. As researchers review references they will start noticing a repetition of the same articles with few new articles appearing. This can indicate that the researcher has covered the literature base on a particular topic.

Putting It All Together

In preparing to write a research paper, it is important to consider which citations to include and how they will inform the introduction and discussion sections. The “Instructions to Authors” for the targeted journal will often provide guidance on structuring the literature review (or introduction) and the number of total citations permitted for each article category. Reviewing articles of similar type published in the targeted journal can also provide guidance regarding structure and average lengths of the introduction and discussion sections.

When selecting references for the introduction consider those that illustrate core background theoretical and methodological concepts, as well as recent relevant studies. The introduction should be brief and present references not as a laundry list or narrative of available literature, but rather as a synthesized summary to provide context for the current study and to identify the gap in the literature that the study intends to fill. For the discussion, citations should be thoughtfully selected to compare and contrast the present study's findings with the current literature and to indicate how the present study moves the field forward.

To facilitate writing a literature review, journals are increasingly providing helpful features to guide authors. For example, the resources available through JGME include several articles on writing. 17 The journal Perspectives on Medical Education recently launched “The Writer's Craft,” which is intended to help medical educators improve their writing. Additionally, many institutions have writing centers that provide web-based materials on writing a literature review, and some even have writing coaches.

The literature review is a vital part of medical education research and should occur throughout the research process to help researchers design a strong study and effectively communicate study results and importance. To achieve these goals, researchers are advised to plan and execute the literature review carefully. The guidance in this editorial provides considerations and recommendations that may improve the quality of literature reviews.

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here .

Loading metrics

Open Access

Peer-reviewed

Research Article

Users’ continuance intention towards an AI painting application: An extended expectation confirmation model

Roles Conceptualization, Data curation, Formal analysis, Funding acquisition, Investigation, Methodology, Project administration, Resources, Software, Supervision, Validation, Visualization, Writing – original draft

Affiliation Postdoctoral Research Station, Central Academy of Fine Arts, Beijing, China

Roles Data curation, Investigation, Software, Supervision, Validation, Writing – original draft, Writing – review & editing

Affiliation School of Art, Soochow University, Suzhou, China

Roles Data curation, Funding acquisition, Project administration, Software, Supervision, Writing – review & editing

* E-mail: [email protected]

• Xiaofan Yu, 
• Yi Yang, 

• Published: May 15, 2024
• https://doi.org/10.1371/journal.pone.0301821
• Reader Comments

With the rapid advancement of technology, Artificial Intelligence (AI) painting has emerged as a leading intelligence service. This study aims to empirically investigate users’ continuance intention toward AI painting applications by utilizing and expanding the Expectation Confirmation Model (ECM), Technology Acceptance Model (TAM), Unified Theory of Acceptance and Use of Technology (UTAUT), and the Flow Theory. A comprehensive research model is proposed. A total of 443 questionnaires were distributed to users with AI painting experiences for data collection. The hypotheses were tested through structural equation modeling. The primary conclusions drawn from this research include: 1) Confirmation plays a crucial role, significantly and positively predicting satisfaction and social impact. 2) Personal innovativeness has a significant effect on confirmation. 3) Satisfaction, flow experience, and social influence directly and positively predict intention, with social influence showing the most significant impact, while perceived usefulness, perceived enjoyment, and performance expectancy show no significant impact on intention. 4) Habit plays a negative moderating role in the association between social influence and continued intention to use. These findings offer valuable insights and inspiration for users seeking to understand the appropriate utilization of AI painting and provide actionable directions for the development of AI painting.

Citation: Yu X, Yang Y, Li S (2024) Users’ continuance intention towards an AI painting application: An extended expectation confirmation model. PLoS ONE 19(5): e0301821. https://doi.org/10.1371/journal.pone.0301821

Editor: Alejandro Vega-Muñoz, Universidad Central de Chile, CHILE

Received: October 29, 2023; Accepted: March 17, 2024; Published: May 15, 2024

Copyright: © 2024 Yu et al. This is an open access article distributed under the terms of the Creative Commons Attribution License , which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Data Availability: All relevant data are within the manuscript and its Supporting information files.

Funding: The current study represents one of the periodic achievements of the 2023 China Postdoctoral Science Foundation, Project ID: 2023M734077. The fund provides research funding support. Sponsors or funders played role in: Investigation; Preparation of the manuscript.

Competing interests: The authors have declared that no competing interests exist.

1 Introduction

Today, artificial intelligence (AI) has permeated every aspect of our lives, including communication, innovation, social activities, and even painting. AI painting gains widespread popularity among internet users worldwide. The integration of AI technology into the realm of art can provide users with new opportunities for artistic creation [ 1 ]. The advantages of using AI painting tools are undeniable, offering users convenience, openness, sharing capabilities, efficiency, and time-saving benefits. In 2022, the popularity of AI painting soard. Platforms like TikTok, Little Red Book, AI Art, and Assisted Creation Platform have made neural style transfer, a technique that combines AI technology and art, common practice [ 2 ]. The topic of "AI painting" on TikTok alone garnered nearly 1.3 billion views, with over 30 million people actively engaging with it. As the number of AI painting users continues to grow, its potential for promoting artistic activity becomes increasingly promising.

Existing research on AI painting uses different theories, including the technology acceptance model [ 3 ], and the unified theory of acceptance and use of technology [ 4 ], to explain users’ adoption of AI painting. However, the findings are still fragmented. Previous studies on AI painting primarily focused on the generation systems [ 5 , 6 ] and art education [ 7 ]. Although general-purpose AI painting tools found more applications in fields like art education and design, it is essential to take the continuing intention to use into consideration when promoting this new technology. The majority of prior studies on AI painting devices aimed to investigate users’ initial utilization of the devices, while only a few studies have examined their behavior and intention to continue using them.

Previous studies provided empirical evidence that demonstrates the moderating effects of users’ habits on artificial intelligence (AI) technology products and services, leading to increased intentions to repurchase [ 8 ], this effect also extends to AI Chatbots [ 9 ]. However, these findings are not investigated in the context of AI painting. Additionally, social media habits reflect automatic expressions of past behavior, aligning with social psychology and occurring within a stable environment, as noted in previous studies [ 10 ]. Habit plays a crucial role in predicting future purchasing and usage behaviors. When individuals use AI painting in a stable environment, they develop refined usage patterns through ongoing interactions, fostering an emotional connection and affirmation towards AI painting. Usage habits result from repeated patterns and goal-oriented behaviors over an extended period. Habit influences usage behavior in an interactive relationship [ 11 ]. Therefore, this study recognizes habits as a potential moderating variable.

Thus, this study aims to examine users’ post-continuance behavior concerning AI painting. To address this issue, the present study incorporates the Expectation Confirmation Model (ECM), Technology Acceptance Model (TAM), Unified Theory of Acceptance and Use of Technology (UTAUT), and the Flow Theory into an integrated user adoption model of AI painting. Previously, the Expectation Confirmation Model has been utilized to explore users’ perspectives on various technologies such as sharing economy platforms [ 12 ], mobile instant messaging, and Internet protocol television [ 13 ]. However, there are few attempts to apply this theory to understand users’ perspectives on AI painting. By applying the ECM to the integrated research model of AI painting, we can gain a more comprehensive understanding of users’ behaviors and usage patterns in the context of AI painting. Furthermore, the findings of this study can demonstrate the potential and feasibility of combining multiple theories in proposing an integrated research model for users’ continuance intention to use behavior.

The remainder of the paper was structured as follows: Section II provides a brief overview of AI painting and reviews related literature and the development of hypotheses. Section III describes the methodology employed to examine the proposed model. Section IV describes the results of the proposed model. Finally, Section V presents and discusses the research results and their implications.

2 Literature review and theoretical models

2.1 ai painting.

AI painting has gained significant popularity in China, with many users embracing this tool for various purposes [ 14 ]. It serves as a helpful assistant in people’s daily lives, enabling them to enhance their painting experiences with just a click on their phones. The convenience of generating AI paintings and the ability to save and share the created images permanently have contributed to its widespread use [ 15 ].

The increasing utilization of AI painting has attracted academic attention, leading to numerous studies exploring its significance and users’ attitudes towards it. For instance, Sun et al. (2022) compared human and AI paintings using a deep neural network (DNN) [ 16 ]. Yu et al. (2022) analyzed paintings generated by word-to-image systems and investigated users’ creative actions and attitudes toward AI through experiments [ 2 ]. Gu. et al. (2022) examined how AI art has influenced the valuation, purchase, and collection intentions of Chinese and Western paintings [ 17 ]. Furthermore, researchers have focused on exploring the adoption intentions of AI painting software and understanding the factors that drive user behavior. Previous studies applied models such as ETAM (Expectation Confirmation Model) and AIBPS (Artificial Intelligence-based Behavior Prediction System) to examine the application of AI painting and identify positive and negative factors associated with perceived usefulness [ 18 ]. Another study by Du et al. in 2023 examined the factors influencing the attitudes and behavioral intentions of designers from different fields in using AI painting tools [ 4 ].

2.2 Expectation confirmation model

The Expectation Confirmation Model (ECM) is a widely used theory in the field of marketing that focuses on evaluating customer satisfaction and understanding post-purchase behavior [ 19 ]. It also applies to assess users’ continued intention to use information systems [ 20 ]. The theory aims to provide a comprehensive analysis of the consumer consumption process [ 21 ]. In the context of AI technology, several researchers have utilized the ECM framework to understand user behavior and satisfaction [ 4 , 20 ].

In the expectation confirmation model, the confirmation of expectations plays a crucial role in shaping user satisfaction [ 22 ]. By comparing their perceived satisfaction with initial expectations, users can assess the alignment or divergence between the two [ 23 , 24 ]. User satisfaction is determined by the discrepancy between pre-purchase expectations and confirmed expectations through post-purchase evaluation [ 25 ]. However, when it comes to the continued use of AI painting, user behavior becomes an important factor influencing decision-making. Therefore, theoretical extensions may be necessary to explain the factors affecting the ongoing use of AI painting tools.

2.2.1 Satisfaction.

Satisfaction (SAT) refers to the psychological structure that corresponds to the user’s response to the level of satisfaction, indicating whether the satisfaction is positive or negative [ 21 ]. Satisfaction has long been used to study the continuance intention to use different types of online products or services [ 26 – 28 ]. For instance, Prathap (2020) found that users’ perceived satisfaction in using mobile payments plays a determining role in their intention to continue using them. In the context of AI painting [ 29 ], if users’ experience exceeds their expectations, they are likely to feel satisfied, leading to a positive inclination for continued usage of AI painting. Consequently, the following assumptions were made:

• H1: Satisfaction is positively related to user continuance intention with the AI painting.

2.2.2 Confirmation.

Confirmation (CON) refers to the alignment between an individual’s perception of the outcome and their established expectations [ 30 – 32 ]. Confirmation was recognized as an influential factor in various aspects, such as perceived usefulness, perceived ease of use, perceived enjoyment, effort expectancy, and social influence, in previous studies [ 25 , 26 ]. Cheng (2021) studied the concept of confirmed and satisfaction in robo-advisor services [ 33 ]. The results show that a higher level of positive certainty means higher satisfaction. However, interestingly, Park (2020) found that users’ confirmation had a non-significant impact on satisfaction with smart wearable devices [ 34 ]. Furthermore, the positive correlation between confirmation, perceived usefulness, and perceived ease of use is observed in the context of smartphones being used as smart pedagogical tools [ 35 ]. Therefore, it is expected that confirmation will influence the post-use experience of AI painting, including users’ satisfaction, perceived usefulness, perceived ease of use, perceived enjoyment, effort expectancy, and social influence. Based on this, the following six hypotheses are proposed:

• H2: Confirmation is positively related to satisfaction with AI painting.
• H3: Confirmation is positively related to the perceived usefulness of AI painting.
• H4: Confirmation is positively related to the perceived ease of use of AI painting.
• H5: Confirmation is positively related to perceived enjoyment of AI painting.
• H6: Confirmation is positively related to the effort expectancy of AI painting.
• H7: Confirmation is positively related to the social influence of AI painting.

2.2.3 Personal innovativeness.

Personal innovativeness (PI) is defined as the consistent and determined use of a specific new information technology innovation by an individual [ 36 ]. Within the realm of information technology, the personal innovativeness exhibited by a user plays a significant role in influencing innovative behavior. Some researchers have identified personal innovativeness as a crucial antecedent [ 37 , 38 ], and others have recognized it as a significant moderating factor [ 39 , 40 ].

Numerous studies established a connection between personal innovativeness and user satisfaction, showing that an individual’s level of innovativeness greatly impacts his or her willingness to adopt new systems or technologies. Huang et al. (2007) expanded the expectation confirmation model by incorporating personal innovativeness as one of the individual factors [ 41 ]. The study by Lee et al. (2021) also found that personal innovativeness is positively correlated with use confirmation in the context of artificial intelligence-based voice assistant systems [ 42 ]. Consequently, for the use of AI painting applications, it is expected that individuals with high levels of personal innovativeness will actively interact with various features, explore newly introduced applications and services, and thereby strengthen their confirmation to a greater extent compared to those with low personal innovativeness. Therefore, the present study puts forward the following hypothesis:

• H8: Personal innovativeness is positively related to the confirmation of AI painting.
• H9: Personal innovativeness is positively related to the satisfaction of AI painting.

2.3 Technology acceptance theory

The Technology Acceptance Model (TAM), initially proposed by Davis in 1986 [ 43 ], has become a widely used framework for understanding and predicting user behavior in the adoption of information technology and systems [ 44 , 45 ]. The theory aims to analyze the influence of external factors on personal beliefs, attitudes, and intentions [ 46 ]. TAM has been applied in various fields such as technology, marketing, AI robotics, and psychology to examine online consumer behavior and technology adoption [ 3 ]. The key concept in TAM is that perceived ease of use and perceived usefulness significantly impact users’ attitudes and intentions to continue using a particular technology, thereby influencing their actual use behavior.

The Unified Theory of Acceptance and Use of Technology (UTAUT) is an extension of TAM that integrates relevant models from the 1990s [ 47 ]. It consists of four components: effort expectancy, performance expectancy, social factors, and facilitating conditions, along with four moderating factors: age, gender, education, and voluntariness of use. The combination of these constructs and moderating factors directly influences the assessment of behavioral intention. This study adopts the UTAUT model, focusing specifically on the factors of perceived enjoyment, effort expectancy, and social influence.

Both the TAM and UTAUT models are widely used to assess technology acceptance [ 48 ]. With the emergence of AI in recent years, new products, services, applications, and systems have been introduced to the market, providing customers with access to a wide range of AI-powered technologies. In the context of AI painting, this study focused on users’ intentions to continue using AI painting tools, which represent their decision to continue using the technology after initial adoption. Understanding users’ continued usage behavior is an essential aspect of technology acceptance and adoption [ 20 ].

2.3.1 Perceived usefulness.

Perceived usefulness (PU) refers to users’ subjective assessment of the utility of AI painting experiences [ 49 ]. Numerous studies have identified perceived usefulness as a key factor influencing users’ satisfaction with technology and their intentions to persist in its use [ 50 , 51 ]. Oyman et al. (2022) have suggested that users’ perceived usefulness plays a significant role in the willingness to use augmented reality technologies [ 52 ]. This study focuses on AI painting continuance intentions to use, defined as users’ decision to continue using AI painting technology after accepting it. We therefore propose the following assumptions:

• H10: Perceived usefulness is positively related to user satisfaction with AI painting.
• H11: Perceived usefulness is positively related to users’ intention to continue using AI painting.

2.3.2 Perceived ease of use.

Perceived ease of use (PEOU) refers to an individual’s subjective assessment of the simplicity and effortlessness involved in utilizing a particular system or technology, taking into account aspects like the ease of learning and the minimal physical and mental exertion needed [ 39 ]. A multitude of research have validated that perceived ease of use affects satisfaction with using new technologies and is a determinant of users’ overall satisfaction. Previous studies have consistently shown a positive correlation between perceived ease of use and consumer satisfaction regarding the use of information systems and services [ 41 ]. The findings of Ngubelanga and Duffett (2021) suggested that perceived ease of use has a reinforcing effect on consumers’ satisfaction with mobile commerce applications [ 50 ]. Thus, we hypothesize that:

• H12: Perceived ease of use is positively related to user satisfaction with AI painting.

2.3.3 Perceived enjoyment.

Perceived enjoyment (PE) refers to the pleasure or enjoyment experienced by individuals when using AI painting [ 53 ]. In a study by Davis et al. (1992), enjoyment variables were added to the TAM model to explain the influence of intrinsic motivation on the adoption of new technologies [ 54 ]. Previous research has affirmed the pivotal role of perceived enjoyment in shaping behavioral intentions toward the acceptance and utilization of diverse information systems and technologies [ 52 , 55 ]. The concept of perceived enjoyment has been extensively applied in the realm of users’ adoption of AI, notably within network environments [ 42 ]. In their research, Yang and Lee (2019) establish that perceived enjoyment is a crucial determinant in the users’ willingness to use virtual personal assistant devices [ 56 ]. Drawing from the available literature, the following hypothesis was formulated:

• H13: Perceived enjoyment is positively related to users’ intention to continue using AI painting.

2.3.4 Effort expectancy.

Effort expectancy (EE) is defined as the perception of the degree of ease associated with using a system or technology [ 47 ]. Numerous empirical studies have proven that effort expectancy is found to influence behavioral intention [ 26 , 57 ]. Adjei et al. (2021) found a positive relationship between effort expectancy and the intention to use mobile phone-based Interactive Voice Response in a rural area of Ghana [ 58 ]. In contrast, Qin and Yu (2023) identified that effort expectancy negatively influenced the intention to use Tencent Meeting for online courses [ 59 ]. Hence, additional research is needed to these inconsistencies. As a result, the following assumption was proposed:

• H14: Effort expectancy is positively related to users’ intention to continue using AI painting.

2.3.5 Social influence.

Social influence (SI) pertains to the degree to which a user believes that others, particularly their acquaintances and friends, significantly impact their decision to continue using new technologies [ 47 ]. Studies have shown that social influence has a significant effect on people’s readiness to adopt new technologies [ 60 ]. Additionally, social influence is a key factor in behavioral intention across various artificial intelligence studies involving AI tools [ 61 ], AI Integrated CRM Systems [ 62 ], and AI-assisted learning environments [ 63 ]. Zeebaree et al. (2022) observed a positive correlation between social influence and the intention to adopt E-Government services in northern Iraq [ 64 ]. Conversely, Kašparová (2023) discovered that social influence hurt the behavioral intention towards business intelligence tools [ 65 ]. Therefore, further research is necessary to address these discrepancies. Consequently, the following assumptions were made:

• H15: Social influence positively related to user continuance intention of AI painting.

2.4 The flow theory

Flow theory suggests that a positive subjective experience is a key driver for engaging in an activity [ 66 ]. When individuals "feel good" about an activity, it has an intrinsic motivational effect, leading to increased involvement. In a state of flow, individuals are motivated by internal factors, experience a sense of control, and maintain focused concentration [ 67 ].

Flow experience (FLO) refers to the mental state of complete immersion and engagement in an activity [ 68 ]. People in a state of flow are fully absorbed in an activity, focused on a specific goal, and disregard the presence of other factors, resulting in feelings of happiness and fulfillment [ 69 ]. Flow experience is considered the most optimal state of experience for individuals and is used as a reliable measure of satisfaction in various domains, including travel and tourism, where it serves as a significant indicator of destination loyalty [ 70 ].

In the context of AI painting, it is important to consider how to maintain an optimal level of engagement to keep users in a state of flow. Based on this understanding, the following assumption was made:

• H16: Flow experience is positively related to users’ intention to continue using AI painting.

Habits (HAB) are memory-based tendencies that automatically respond to cues and lead to repeated past behaviors [ 71 ]. Habits are part of the array of non-conscious processes that can influence behavior [ 72 ]. These tendencies develop through the associations formed between cues and responses in memory, which are solidified through repeated actions in consistent contexts [ 73 ]. Habits play a foundational role in shaping behavior, impacting decision-making processes, and reducing the reliance on conscious thought [ 74 ].

This study delves into how habits moderate the link between a social influence and the intention to continue, drawing on previous research [ 8 , 75 ], that has highlighted the direct connection between habits and the intention to continue. Despite the crucial role of habits as moderators, they have been relatively overlooked in the artificial intelligence literature. Prior studies have shown that habits moderating effect influences user satisfaction, Perceived Usefulness, flow experience, and the intention to continue use [ 76 ]. Some research also indicates that habits have a slight negative moderating impact on the relationship between customer satisfaction and the intention to continue [ 77 ]. However, there is a scarcity of research that integrates habits with the Extended Expectation Confirmation Model (ECM) and TAM in the context of AI painting, implying that habits could potentially regulate the relationships among these constructs. Building on this insight, the following assumption is put forward:

• H17: Habits play a moderating role in the relationship between users’ social influence and their intention to continue using AI painting.

2.6 Research model

Building on prior research, this study expanded the existing ECM (Extended Conceptual Model) by integrating multiple user-oriented theories such as TAM (Technology Acceptance Model), UTAUT (Unified Theory of Acceptance and Use of Technology), and the Flow model (refer to Fig 1 ). Moreover, the authors have introduced habits as a moderating factor within the model. Through the integration of these theories and the exploration of habits’ moderating influence, the study seeks to develop a thorough insight into users’ continuance intentions to use AI-assisted painting.

• PPT PowerPoint slide
• PNG larger image
• TIFF original image

https://doi.org/10.1371/journal.pone.0301821.g001

3 Methodology

3.1 questionnaire design and measurements.

The questionnaire design process proposed by Park (1993) serves as the foundation for this study [ 78 ]. The data collection started on May 18, 2023, and ended on September 29, 2023. We obtained written consent from participants in questionnaires, and we also got ethics approval from the ethics committee of the first author’s local institution (Effective Approval Date: 5 May 2023). To ensure the accuracy and comprehensiveness of the questionnaire, we followed the three-indicator rule, which required each structure to have at least three items [ 79 ]. The 34 questions representing the 11 structures were extracted primarily from professional journals. To accurately convey the content from the initial query posed in English, a dual translation approach (English to Chinese) was employed. The translation process was conducted by two professional translators, and the questionnaire was further revised by three relevant professors.

The questionnaire was divided into two sections. The first section captures participants’ background information, such as age, gender, and profession. The second section comprises a series of questions related to the study’s subject. Before answering the questionnaire, all participants had experience using AI for painting. Previous studies have recommended using a seven-point Likert scale as an interval scale, with scores ranging from 1 (strongly disagree) to 7 (strongly agree). This scale was employed to effectively assess all potential variables. After revising the questionnaire based on feedback, a pilot test was conducted with 30 participants, resulting in the elimination of four items and retaining the valid questions. The questionnaire’s structure, including the remaining items, is displayed in Table 1 .

https://doi.org/10.1371/journal.pone.0301821.t001

3.2 Participants

The study was primarily conducted in Chinese cities, where there is a higher prevalence of AI painting within the TikTok app compared to other regions. Participation in the study required participants to have prior experience using AI painting. To ensure participants’ understanding of the subject matter, an overview of the definition of AI painting was provided before they answered the questionnaire. All participants used AI drawings to ensure their familiarity with the topic.

Before data collection, the researchers provided participants with a summary of the study, instructions for information gathering, and obtained informed consent. It is important to ensure that participants were fully informed about the purpose and procedures of the study, as well as their rights and privacy protection.

3.3 Collection and analysis

The online questionnaire was conducted in China using popular social media platforms such as Tencent QQ, WeChat, and Weibo. This approach was chosen to target individuals who frequently engage with the internet and technology, as they are more likely to exhibit an interest in technological items like AI painting. The web-based questionnaire was available for 30 days and aimed to assess the attitudes of 610 individuals towards AI painting.

After removing incomplete and unsuccessful responses from the attention tests, a total of 443 valid questionnaires remained for analysis. The demographic information of the research participants is presented in Table 2 , providing an overview of the characteristics of the sample.

https://doi.org/10.1371/journal.pone.0301821.t002

To examine the interrelationships among variables, particularly in intricate models, Structural Equation Modeling (SEM) was employed. SEM requires a recommended ratio of 1:10 between the number of observations and the number of indicators. However, Bentler (1996) proposes an alternative guideline where the sample size should be five times larger than the number of free parameters [ 82 ]. Given the information provided, the sample size of 443 valid responses satisfies the prescribed threshold, as it falls within the recommended range of 170 to 340.

Before conducting the data analysis, the demographic information was summarized, and the statistical validity was verified using SPSS 26.0 software [ 83 ]. Additionally, to test the proposed hypotheses and examine the correlations between each variable, a path analysis was performed using AMOS 24.0 software. The researchers utilized the PROCESS plug-in to explore the impact of the moderator on the conditional effect, providing further insights into the relationships between variables.

4.1 Preliminary analysis

Table 3 presents the initial analytical data within the study model. The table indicates that there are 443 valid samples for each item, with no missing data observed. Additionally, the standard deviation and mean values for all items fall within the normal range, implying that there are no data entry errors or anomalies.

https://doi.org/10.1371/journal.pone.0301821.t003

To assess the normal distribution of the variables, it is recommended that the magnitudes of skewness and kurtosis remain below 1.5 [ 84 ]. The absolute values of skewness and kurtosis in the data have been found to meet this conventional criterion, suggesting a normal distribution of the constructs. This finding provides a solid foundation for future research endeavors, as it ensures the appropriateness of statistical analyses and interpretations.

4.2 Analysis of exploratory factors

To assess the validity and reliability of the measurement scales used in the study, certain criteria are commonly employed. The standard factor loading for each item should ideally be higher than 0.6, and at least above 0.5 to be considered acceptable [ 85 ]. In Table 4 the factor loading results range from 0.693 to 0.929, which are all above the 0.5 threshold. This indicates that the factor structure utilized in this study is suitable for examining the research objectives.

https://doi.org/10.1371/journal.pone.0301821.t004

For validity analysis, reliability is typically indicated by Cronbach’s alpha coefficient values greater than 0.7 [ 86 ]. However, it is also considered acceptable to consider a Cronbach’s alpha coefficient value of 0.6 as a "criterion-in-use" [ 87 ]. In Table 4 , the Cronbach’s alpha values presented for each construct demonstrate internal consistency above 0.6, indicating acceptable reliability.

4.3 Validity test

To assess the convergent and discriminant validity of the measurement scales used in the study, Confirmatory Factor Analysis (CFA) provides useful indices. Comprehensive Reliability (CR) and Average Variance Extraction (AVE) should typically exceed 0.7 and 0.5, respectively [ 64 ]. Furthermore, to evaluate discriminant validity, the square root of the AVE for each construct should be greater than the correlations between constructs [ 88 ].

Based on the previous study, a total of 443 samples were analyzed, and the mean and standard deviation values for each construct are presented in Table 4 . The values of CR and AVE for all constructs are higher than the respective thresholds of 0.7 and 0.5, indicating satisfactory convergent validity.

In Table 5 , the diagonal entries represent the square root of the AVE, which is displayed in bold. These values are found to be higher than the correlations between the constructs, indicating discriminant validity. This outcome provides evidence that the measurement scales used in the research have achieved both convergent and discriminant reliability.

https://doi.org/10.1371/journal.pone.0301821.t005

4.4 Fit indices

Assessing the goodness of fit of a model to the collected data is a common research practice. Various fit indices, including the Chi-square (X2) statistic, degrees of freedom (DF), and other fit indices, are commonly used to evaluate model fitness [ 89 ].

According to previous studies, it is suggested that the Comparative Fit Index (CFI) and Tucker-Lewis Index (TLI) should ideally exceed a threshold of 0.9. Simultaneously, the values of the Goodness of Fit Index (GFI) exceed the threshold of 0.8, indicating an acceptable level of fit [ 67 ]. The Root-mean-square error of approximation (RMSEA) is recommended to be less than 0.08 to indicate a satisfactory fit for the study model [ 90 ].

The parameters related to the current study model have been evaluated against these criteria and are presented in Table 6 . By comparing the values of the fit indices to the established thresholds, the model’s goodness of fit can be assessed.

https://doi.org/10.1371/journal.pone.0301821.t006

4.5 Hypotheses tests

The results of the structural equation modeling (SEM) analysis conducted for the present study are summarized in Table 7 , Figs 1 and 2 .

https://doi.org/10.1371/journal.pone.0301821.g002

https://doi.org/10.1371/journal.pone.0301821.t007

Analysis of Fig 2 reveals that confirmation has a positive effect on various factors, including satisfaction to use (H2), perceived usefulness (H3), perceived ease of use (H4), perceived enjoyment (H5), effort expectancy (H6), and social influence (H7). These effects are supported by statistically significant t-values and beta coefficients, with specific values reported as follows (t = 10.843, β = 0.970, P<0.05 for H2; t = 14.678, β = 0.811, P<0.05 for H3; t = 6.984, β = 0.406, P<0.05 for H4; t = 15.890, β = 0.808, P<0.05 for H5; t = 15.237, β = 0.735, P<0.05 for H6; and t = 8.890, β = 0.671, P<0.05 for H7).

Additionally, personal innovativeness (H8) demonstrates a positive influence on confirmation, supported by a statistically significant t-value (t = 11.111, β = 0.611, P<0.05). However, personal innovativeness (H9), perceived usefulness (H10), and perceived ease of use (H12) do not show any significant influences on satisfaction with use.

Furthermore, satisfaction to use (H1), perceived usefulness (H11), perceived enjoyment (H13), social influence (H15), and flow experience (H16) positively impact continuance intention to use. These effects are supported by statistically significant t-values and beta coefficients (t = 3.616, β = 0.245, P<0.05 for H1; t = 3.227, β = 0.215, P<0.05 for H11; t = 2.005, β = 0.132, P<0.05 for H13; t = 5.402, β = 0.317, P<0.05 for H15; t = 4.809, β = 0.226, P<0.05 for H16).

However, it should be noted that effort expectancy (H14) does not exert any significant effect on continuance intention to use, as indicated by a non-significant t-value (t = -1.751, P>0.05).

4.6 The moderating effect analysis

In the present study, the PROCESS plug-in in SPSS was utilized to analyze H17, which examines the impact of habit on the correlation between social influence and continuance intention to use. This plugin, developed in 2013, facilitated the implementation of hierarchical regression analysis with bootstrapping, using a sample size exceeding 5000 iterations [ 83 ].

The P-value was used as an indicator to estimate the presence of a conditional effect. The results presented in Table 8 indicate that there is a moderation factor of habit in the conditional effect of social influence on continuance intention to use. The statistical analysis shows that the null hypothesis (p<0.05) can be rejected, suggesting a significant correlation between social influence and continuance intention to use. Additionally, the analysis reveals that this relationship is negatively moderated by the moderating variable, habit, as depicted in Fig 3 . This implies that with the increasing effect of habit, social influence has a stronger influence on continuance intention to use. Conversely, when the effect of habit decreases, the association will be weaker. Thus, hypothesis H17 is supported. Furthermore, the study employed the Johnson-Neyman (J-N) technique to discern the superior effect from the inferior effect while conducting tests on basic slopes [ 91 ]. Fig 4 illustrates that there is no significant moderation between social influence and continuance intention to use for customers with habit scores above 4.0 (p>0.05). However, a notable conditional effect is observed when habit scores fall below a threshold of 4.0.

https://doi.org/10.1371/journal.pone.0301821.g003

https://doi.org/10.1371/journal.pone.0301821.g004

https://doi.org/10.1371/journal.pone.0301821.t008

5 Discussion and conclusions

This study aimed to develop a theoretical framework and investigate users’ intention to continue using an AI painting application by combining the Expectation Confirmation Model (ECM) with elements from TAM, UTAUT, and Flow Theory. The SEM analysis supported 11 out of 17 hypotheses, providing empirical evidence for most of the assumptions in the paper. The results of each hypothesis are discussed in detail below.

Firstly, the study unveiled that the majority of hypotheses concerning the influence of antecedents on satisfaction were supported. Users’ confirmation had a positive impact on perceived ease of use, perceived usefulness, perceived enjoyment, effort expectancy, social influence, and satisfaction. Confirmation plays a pivotal role, significantly and positively predicting satisfaction and social impact. Moreover, individuals’ innovativeness has a significant impact on confirmation. However, perceived usefulness, perceived ease of use, and personal innovativeness did not affect the intention of being satisfied with AI painting. This indicated that users’ expectations regarding utility, ease of use, and innovative potential might not have been met within the context of AI painting. The lack of impact on satisfaction could be attributed to the misalignment between users’ perceived expectations and their experience with AI painting tools. Additionally, for some individuals, the disparity between traditional artistic processes and the automated nature of AI painting tools could result in a feeling of dissatisfaction or disconnection [ 17 ]. Furthermore, the limitations and constraints imposed by AI technology in the creative process could also contribute to a lack of satisfaction among users who appreciate the depth and personal touch offered by traditional art forms [ 92 ].

In essence, the dissatisfaction could stem from a mismatch between users’ preconceptions about technology and their actual engagement with AI painting, as well as a desire for a more personalized and emotionally engaging experience [ 4 ]. The research findings indicate a lack of correlation between personal innovativeness and satisfaction with AI painting. Personal innovativeness typically involves individuals’ unique and forward-thinking creative abilities in a creative domain [ 40 , 93 ]. When using AI painting tools, personal innovativeness may not directly impact satisfaction as these tools often incorporate algorithms and data to generate art pieces, and users primarily interact with the tool rather than create the entire piece. In this scenario, the scope for personal innovativeness to influence satisfaction is limited. Furthermore, the design and functionality of AI painting tools may constrain the expression of personal innovativeness. Some users may expect greater flexibility in incorporating their creativity and ideas, but the algorithms and operational methods of AI painting tools may limit the individual’s space for innovation, thereby weakening the correlation between personal innovativeness and satisfaction [ 61 , 94 ].

Secondly, the findings indicated that users’ willingness to continue using AI painting was influenced by satisfaction, perceived usefulness, perceived enjoyment, effort expectancy, social influence, and flow experience. Particularly, satisfaction, flow experience, and social influence directly and positively predicted intention, with social influence demonstrating the most substantial impact. Surprisingly, it was found that effort expectancy had a relatively limited effect on continuance intention, in contrast to earlier research highlighting its significance in the success of app services [ 33 , 34 ].

The research findings indicated that perceived usefulness, perceived enjoyment, and effort expectancy did not impact the intention to continue using AI painting. These results are not similar to those of Xu (2023) and Gupta (2021) [ 18 , 31 ]. One possible explanation within the realm of artistic creation and innovative expression is that individual cognition and emotional experiences might play a more significant role. While perceived usefulness is typically a key factor influencing the acceptance and use of technology, in the context of art and creativity, individuals’ aesthetic experiences of the artwork itself could be more influential. The complexity of AI painting tools and the constraints they may impose on the creative process could also contribute to the lack of correlation with perceived enjoyment. Users may perceive AI-generated artwork as lacking the emotional depth or personal touch that traditional art forms provide, affecting their overall enjoyment of the AI painting activity. Additionally, effort expectancy may lose its impacting power on continuation intention in this context, as the pleasure and satisfaction derived from the artistic creation process for many individuals may outweigh the associated effort costs [ 95 , 96 ]. Therefore, the study results indicate that in the realm of artistic creation, individuals value personal experiences and emotions more than practicality, enjoyment, or effort expectancy. This perspective could offer new insight into understanding people’s attitudes towards using AI painting technology, suggesting that in designing and promoting similar technologies, a greater emphasis on user experience and emotional needs, over and above the functional utility of the technology itself, may be required.

Finally, the study delved into the moderating role of habits in the relationship between social influence and the intention to continue using the AI painting application. The results indicated that habitual factors could interfere with the equilibrium between social influence and sustained usage. Put simply, as behavior becomes more ingrained as a habit, it is more inclined to have a negative impact on this association, consistent with previous research [ 8 , 77 ].

5.1 Contributions

The main contribution of this study is the adaptation of the ECM framework for the context of AI painting, particularly within social media platforms. By expanding the applicability of ECM, we have enhanced our understanding of predictors of user satisfaction and intention to continue using AI painting. We have redefined the concept of usability to align with the unique features of AI painting technology and its intended goals on social platforms. Specifically, we introduced dimensions of effort expectancy, social influence, and perceived enjoyment to address these aspects. While previous studies have explored the role of effort expectancy and social influence on continuance intention to use [ 70 , 74 , 75 ], we extended this research to the context of AI painting. Our findings indicate that positive social influence can significantly influence a user’s intention to continue using AI painting.

Another significant aspect of expanding the ECM framework is the inclusion of personal innovativeness as a factor influencing confirmation and satisfaction. The original ECM assumes a passive user who either accepts or rejects the available technology, without considering alternatives. However, in the AI painting market, users have multiple options to choose from, and switching between applications incurs minimal costs [ 97 ]. Many users utilize multiple AI painting applications to engage with different groups of people. Additionally, recent developments have introduced features in AI painting applications that enable users to send and receive money, intensifying the competition among AI painting providers.

5.2 Implications

This study provides several implications for scholars and practitioners in the field of AI painting. Firstly, the study proposed AI painting continuance use model delineates a specific set of factors that service providers and application developers could make to enhance the continuous adoption and utilization of AI painting. It suggests that AI painting providers should explore interventions that increase the likelihood of positive confirmation and user satisfaction, as both factors significantly influence one’s intention to continue using AI painting. With the widespread use of AI painting tools, service providers should be more flexible and responsive to changes in the AI painting landscape, focusing on social influence, satisfaction, and flow experience within their AI painting applications.

Secondly, the results of the research model can be utilized by AI painting app service providers to enhance the user experience. They can apply the findings to improve perceived ease of use, perceived usefulness, perceived enjoyment, social influence, effort expectancy, and flow experience. Closing the gap between user expectations and actual usage experience by enhancing personal innovativeness is crucial for app service providers.

Thirdly, apart from continuously refining and improving the design of AI painting tools, technical developers should acknowledge that users may not be fully prepared for the capabilities offered by advanced AI tools, especially when lacking proper guidance. It is important to educate users about AI technologies through platforms like TikTok to increase their comfort and knowledge about AI tools and boost their overall AI painting experience.

Finally, the research findings indicate that users alter their decision-making patterns as they develop a habit of online AI painting.

5.3 Limitations and future work

While the current research has provided valuable insights into AI painting, it is important to acknowledge certain boundaries that should be addressed in future investigations. Firstly, the application of AI painting may vary among different age groups, genders, and backgrounds. However, the current study did not explore the potential differences resulting from age or gender. Therefore, future research could delve into this aspect and examine how disparities between different groups may lead to distinct outcomes.

Secondly, it is recommended for future studies to investigate the moderating effect of additional factors, such as experience and trust, on the relationship between user behavior and AI painting. These factors could potentially influence users’ perceptions and continuance intentions in using AI painting applications.

Lastly, the current research was confined to a single AI painting application within one nation, without considering other applications in the market. Therefore, further research is necessary to validate and generalize the findings by including multiple applications and comparing the results with the present study.

Supporting information

S1 dataset..

https://doi.org/10.1371/journal.pone.0301821.s001

Acknowledgments

The authors would like to thank all the participants of this study.

• View Article
• Google Scholar
• PubMed/NCBI
• 23. Brill T M, Munoz L, Miller R J. Siri, Alexa, and other digital assistants: a study of customer satisfaction with artificial intelligence applications. The Role of Smart Technologies in Decision Making. Routledge, 2022; 35–70. https://doi.org/10.1080/0267257x.2019.1687571
• 84. Tabachnick B G, Fidell L S, Ullman J B. Using multivariate statistics. Boston, MA: Pearson, 2013. https://lccn.loc.gov/2017040173 .
• 90. Kenny D A. Measuring model fit. 2015. https://davidakenny.net/cm/fit.htm .