case study method methodology

The Ultimate Guide to Qualitative Research - Part 1: The Basics

• Introduction and overview
• What is qualitative research?
• What is qualitative data?
• Examples of qualitative data
• Qualitative vs. quantitative research
• Mixed methods
• Qualitative research preparation
• Theoretical perspective
• Theoretical framework
• Literature reviews

Research question

• Conceptual framework
• Conceptual vs. theoretical framework

Data collection

• Qualitative research methods
• Focus groups
• Observational research

What is a case study?

Applications for case study research, what is a good case study, process of case study design, benefits and limitations of case studies.

• Ethnographical research
• Ethical considerations
• Confidentiality and privacy
• Power dynamics
• Reflexivity

Case studies

Case studies are essential to qualitative research , offering a lens through which researchers can investigate complex phenomena within their real-life contexts. This chapter explores the concept, purpose, applications, examples, and types of case studies and provides guidance on how to conduct case study research effectively.

Whereas quantitative methods look at phenomena at scale, case study research looks at a concept or phenomenon in considerable detail. While analyzing a single case can help understand one perspective regarding the object of research inquiry, analyzing multiple cases can help obtain a more holistic sense of the topic or issue. Let's provide a basic definition of a case study, then explore its characteristics and role in the qualitative research process.

Definition of a case study

A case study in qualitative research is a strategy of inquiry that involves an in-depth investigation of a phenomenon within its real-world context. It provides researchers with the opportunity to acquire an in-depth understanding of intricate details that might not be as apparent or accessible through other methods of research. The specific case or cases being studied can be a single person, group, or organization – demarcating what constitutes a relevant case worth studying depends on the researcher and their research question .

Among qualitative research methods , a case study relies on multiple sources of evidence, such as documents, artifacts, interviews , or observations , to present a complete and nuanced understanding of the phenomenon under investigation. The objective is to illuminate the readers' understanding of the phenomenon beyond its abstract statistical or theoretical explanations.

Characteristics of case studies

Case studies typically possess a number of distinct characteristics that set them apart from other research methods. These characteristics include a focus on holistic description and explanation, flexibility in the design and data collection methods, reliance on multiple sources of evidence, and emphasis on the context in which the phenomenon occurs.

Furthermore, case studies can often involve a longitudinal examination of the case, meaning they study the case over a period of time. These characteristics allow case studies to yield comprehensive, in-depth, and richly contextualized insights about the phenomenon of interest.

The role of case studies in research

Case studies hold a unique position in the broader landscape of research methods aimed at theory development. They are instrumental when the primary research interest is to gain an intensive, detailed understanding of a phenomenon in its real-life context.

In addition, case studies can serve different purposes within research - they can be used for exploratory, descriptive, or explanatory purposes, depending on the research question and objectives. This flexibility and depth make case studies a valuable tool in the toolkit of qualitative researchers.

Remember, a well-conducted case study can offer a rich, insightful contribution to both academic and practical knowledge through theory development or theory verification, thus enhancing our understanding of complex phenomena in their real-world contexts.

What is the purpose of a case study?

Case study research aims for a more comprehensive understanding of phenomena, requiring various research methods to gather information for qualitative analysis . Ultimately, a case study can allow the researcher to gain insight into a particular object of inquiry and develop a theoretical framework relevant to the research inquiry.

Why use case studies in qualitative research?

Using case studies as a research strategy depends mainly on the nature of the research question and the researcher's access to the data.

Conducting case study research provides a level of detail and contextual richness that other research methods might not offer. They are beneficial when there's a need to understand complex social phenomena within their natural contexts.

The explanatory, exploratory, and descriptive roles of case studies

Case studies can take on various roles depending on the research objectives. They can be exploratory when the research aims to discover new phenomena or define new research questions; they are descriptive when the objective is to depict a phenomenon within its context in a detailed manner; and they can be explanatory if the goal is to understand specific relationships within the studied context. Thus, the versatility of case studies allows researchers to approach their topic from different angles, offering multiple ways to uncover and interpret the data .

The impact of case studies on knowledge development

Case studies play a significant role in knowledge development across various disciplines. Analysis of cases provides an avenue for researchers to explore phenomena within their context based on the collected data.

This can result in the production of rich, practical insights that can be instrumental in both theory-building and practice. Case studies allow researchers to delve into the intricacies and complexities of real-life situations, uncovering insights that might otherwise remain hidden.

Types of case studies

In qualitative research , a case study is not a one-size-fits-all approach. Depending on the nature of the research question and the specific objectives of the study, researchers might choose to use different types of case studies. These types differ in their focus, methodology, and the level of detail they provide about the phenomenon under investigation.

Understanding these types is crucial for selecting the most appropriate approach for your research project and effectively achieving your research goals. Let's briefly look at the main types of case studies.

Exploratory case studies

Exploratory case studies are typically conducted to develop a theory or framework around an understudied phenomenon. They can also serve as a precursor to a larger-scale research project. Exploratory case studies are useful when a researcher wants to identify the key issues or questions which can spur more extensive study or be used to develop propositions for further research. These case studies are characterized by flexibility, allowing researchers to explore various aspects of a phenomenon as they emerge, which can also form the foundation for subsequent studies.

Descriptive case studies

Descriptive case studies aim to provide a complete and accurate representation of a phenomenon or event within its context. These case studies are often based on an established theoretical framework, which guides how data is collected and analyzed. The researcher is concerned with describing the phenomenon in detail, as it occurs naturally, without trying to influence or manipulate it.

Explanatory case studies

Explanatory case studies are focused on explanation - they seek to clarify how or why certain phenomena occur. Often used in complex, real-life situations, they can be particularly valuable in clarifying causal relationships among concepts and understanding the interplay between different factors within a specific context.

Intrinsic, instrumental, and collective case studies

These three categories of case studies focus on the nature and purpose of the study. An intrinsic case study is conducted when a researcher has an inherent interest in the case itself. Instrumental case studies are employed when the case is used to provide insight into a particular issue or phenomenon. A collective case study, on the other hand, involves studying multiple cases simultaneously to investigate some general phenomena.

Each type of case study serves a different purpose and has its own strengths and challenges. The selection of the type should be guided by the research question and objectives, as well as the context and constraints of the research.

The flexibility, depth, and contextual richness offered by case studies make this approach an excellent research method for various fields of study. They enable researchers to investigate real-world phenomena within their specific contexts, capturing nuances that other research methods might miss. Across numerous fields, case studies provide valuable insights into complex issues.

Critical information systems research

Case studies provide a detailed understanding of the role and impact of information systems in different contexts. They offer a platform to explore how information systems are designed, implemented, and used and how they interact with various social, economic, and political factors. Case studies in this field often focus on examining the intricate relationship between technology, organizational processes, and user behavior, helping to uncover insights that can inform better system design and implementation.

Health research

Health research is another field where case studies are highly valuable. They offer a way to explore patient experiences, healthcare delivery processes, and the impact of various interventions in a real-world context.

Case studies can provide a deep understanding of a patient's journey, giving insights into the intricacies of disease progression, treatment effects, and the psychosocial aspects of health and illness.

Asthma research studies

Specifically within medical research, studies on asthma often employ case studies to explore the individual and environmental factors that influence asthma development, management, and outcomes. A case study can provide rich, detailed data about individual patients' experiences, from the triggers and symptoms they experience to the effectiveness of various management strategies. This can be crucial for developing patient-centered asthma care approaches.

Other fields

Apart from the fields mentioned, case studies are also extensively used in business and management research, education research, and political sciences, among many others. They provide an opportunity to delve into the intricacies of real-world situations, allowing for a comprehensive understanding of various phenomena.

Case studies, with their depth and contextual focus, offer unique insights across these varied fields. They allow researchers to illuminate the complexities of real-life situations, contributing to both theory and practice.

Whatever field you're in, ATLAS.ti puts your data to work for you

Download a free trial of ATLAS.ti to turn your data into insights.

Understanding the key elements of case study design is crucial for conducting rigorous and impactful case study research. A well-structured design guides the researcher through the process, ensuring that the study is methodologically sound and its findings are reliable and valid. The main elements of case study design include the research question , propositions, units of analysis, and the logic linking the data to the propositions.

The research question is the foundation of any research study. A good research question guides the direction of the study and informs the selection of the case, the methods of collecting data, and the analysis techniques. A well-formulated research question in case study research is typically clear, focused, and complex enough to merit further detailed examination of the relevant case(s).

Propositions

Propositions, though not necessary in every case study, provide a direction by stating what we might expect to find in the data collected. They guide how data is collected and analyzed by helping researchers focus on specific aspects of the case. They are particularly important in explanatory case studies, which seek to understand the relationships among concepts within the studied phenomenon.

Units of analysis

The unit of analysis refers to the case, or the main entity or entities that are being analyzed in the study. In case study research, the unit of analysis can be an individual, a group, an organization, a decision, an event, or even a time period. It's crucial to clearly define the unit of analysis, as it shapes the qualitative data analysis process by allowing the researcher to analyze a particular case and synthesize analysis across multiple case studies to draw conclusions.

Argumentation

This refers to the inferential model that allows researchers to draw conclusions from the data. The researcher needs to ensure that there is a clear link between the data, the propositions (if any), and the conclusions drawn. This argumentation is what enables the researcher to make valid and credible inferences about the phenomenon under study.

Understanding and carefully considering these elements in the design phase of a case study can significantly enhance the quality of the research. It can help ensure that the study is methodologically sound and its findings contribute meaningful insights about the case.

Ready to jumpstart your research with ATLAS.ti?

Conceptualize your research project with our intuitive data analysis interface. Download a free trial today.

Conducting a case study involves several steps, from defining the research question and selecting the case to collecting and analyzing data . This section outlines these key stages, providing a practical guide on how to conduct case study research.

Defining the research question

The first step in case study research is defining a clear, focused research question. This question should guide the entire research process, from case selection to analysis. It's crucial to ensure that the research question is suitable for a case study approach. Typically, such questions are exploratory or descriptive in nature and focus on understanding a phenomenon within its real-life context.

Selecting and defining the case

The selection of the case should be based on the research question and the objectives of the study. It involves choosing a unique example or a set of examples that provide rich, in-depth data about the phenomenon under investigation. After selecting the case, it's crucial to define it clearly, setting the boundaries of the case, including the time period and the specific context.

Previous research can help guide the case study design. When considering a case study, an example of a case could be taken from previous case study research and used to define cases in a new research inquiry. Considering recently published examples can help understand how to select and define cases effectively.

Developing a detailed case study protocol

A case study protocol outlines the procedures and general rules to be followed during the case study. This includes the data collection methods to be used, the sources of data, and the procedures for analysis. Having a detailed case study protocol ensures consistency and reliability in the study.

The protocol should also consider how to work with the people involved in the research context to grant the research team access to collecting data. As mentioned in previous sections of this guide, establishing rapport is an essential component of qualitative research as it shapes the overall potential for collecting and analyzing data.

Collecting data

Gathering data in case study research often involves multiple sources of evidence, including documents, archival records, interviews, observations, and physical artifacts. This allows for a comprehensive understanding of the case. The process for gathering data should be systematic and carefully documented to ensure the reliability and validity of the study.

Analyzing and interpreting data

The next step is analyzing the data. This involves organizing the data , categorizing it into themes or patterns , and interpreting these patterns to answer the research question. The analysis might also involve comparing the findings with prior research or theoretical propositions.

Writing the case study report

The final step is writing the case study report . This should provide a detailed description of the case, the data, the analysis process, and the findings. The report should be clear, organized, and carefully written to ensure that the reader can understand the case and the conclusions drawn from it.

Each of these steps is crucial in ensuring that the case study research is rigorous, reliable, and provides valuable insights about the case.

The type, depth, and quality of data in your study can significantly influence the validity and utility of the study. In case study research, data is usually collected from multiple sources to provide a comprehensive and nuanced understanding of the case. This section will outline the various methods of collecting data used in case study research and discuss considerations for ensuring the quality of the data.

Interviews are a common method of gathering data in case study research. They can provide rich, in-depth data about the perspectives, experiences, and interpretations of the individuals involved in the case. Interviews can be structured , semi-structured , or unstructured , depending on the research question and the degree of flexibility needed.

Observations

Observations involve the researcher observing the case in its natural setting, providing first-hand information about the case and its context. Observations can provide data that might not be revealed in interviews or documents, such as non-verbal cues or contextual information.

Documents and artifacts

Documents and archival records provide a valuable source of data in case study research. They can include reports, letters, memos, meeting minutes, email correspondence, and various public and private documents related to the case.

These records can provide historical context, corroborate evidence from other sources, and offer insights into the case that might not be apparent from interviews or observations.

Physical artifacts refer to any physical evidence related to the case, such as tools, products, or physical environments. These artifacts can provide tangible insights into the case, complementing the data gathered from other sources.

Ensuring the quality of data collection

Determining the quality of data in case study research requires careful planning and execution. It's crucial to ensure that the data is reliable, accurate, and relevant to the research question. This involves selecting appropriate methods of collecting data, properly training interviewers or observers, and systematically recording and storing the data. It also includes considering ethical issues related to collecting and handling data, such as obtaining informed consent and ensuring the privacy and confidentiality of the participants.

Data analysis

Analyzing case study research involves making sense of the rich, detailed data to answer the research question. This process can be challenging due to the volume and complexity of case study data. However, a systematic and rigorous approach to analysis can ensure that the findings are credible and meaningful. This section outlines the main steps and considerations in analyzing data in case study research.

Organizing the data

The first step in the analysis is organizing the data. This involves sorting the data into manageable sections, often according to the data source or the theme. This step can also involve transcribing interviews, digitizing physical artifacts, or organizing observational data.

Categorizing and coding the data

Once the data is organized, the next step is to categorize or code the data. This involves identifying common themes, patterns, or concepts in the data and assigning codes to relevant data segments. Coding can be done manually or with the help of software tools, and in either case, qualitative analysis software can greatly facilitate the entire coding process. Coding helps to reduce the data to a set of themes or categories that can be more easily analyzed.

Identifying patterns and themes

After coding the data, the researcher looks for patterns or themes in the coded data. This involves comparing and contrasting the codes and looking for relationships or patterns among them. The identified patterns and themes should help answer the research question.

Interpreting the data

Once patterns and themes have been identified, the next step is to interpret these findings. This involves explaining what the patterns or themes mean in the context of the research question and the case. This interpretation should be grounded in the data, but it can also involve drawing on theoretical concepts or prior research.

Verification of the data

The last step in the analysis is verification. This involves checking the accuracy and consistency of the analysis process and confirming that the findings are supported by the data. This can involve re-checking the original data, checking the consistency of codes, or seeking feedback from research participants or peers.

Like any research method , case study research has its strengths and limitations. Researchers must be aware of these, as they can influence the design, conduct, and interpretation of the study.

Understanding the strengths and limitations of case study research can also guide researchers in deciding whether this approach is suitable for their research question . This section outlines some of the key strengths and limitations of case study research.

Benefits include the following:

• Rich, detailed data: One of the main strengths of case study research is that it can generate rich, detailed data about the case. This can provide a deep understanding of the case and its context, which can be valuable in exploring complex phenomena.
• Flexibility: Case study research is flexible in terms of design , data collection , and analysis . A sufficient degree of flexibility allows the researcher to adapt the study according to the case and the emerging findings.
• Real-world context: Case study research involves studying the case in its real-world context, which can provide valuable insights into the interplay between the case and its context.
• Multiple sources of evidence: Case study research often involves collecting data from multiple sources , which can enhance the robustness and validity of the findings.

On the other hand, researchers should consider the following limitations:

• Generalizability: A common criticism of case study research is that its findings might not be generalizable to other cases due to the specificity and uniqueness of each case.
• Time and resource intensive: Case study research can be time and resource intensive due to the depth of the investigation and the amount of collected data.
• Complexity of analysis: The rich, detailed data generated in case study research can make analyzing the data challenging.
• Subjectivity: Given the nature of case study research, there may be a higher degree of subjectivity in interpreting the data , so researchers need to reflect on this and transparently convey to audiences how the research was conducted.

Being aware of these strengths and limitations can help researchers design and conduct case study research effectively and interpret and report the findings appropriately.

Ready to analyze your data with ATLAS.ti?

See how our intuitive software can draw key insights from your data with a free trial today.

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base

Methodology

• What Is a Case Study? | Definition, Examples & Methods

What Is a Case Study? | Definition, Examples & Methods

Published on May 8, 2019 by Shona McCombes . Revised on November 20, 2023.

A case study is a detailed study of a specific subject, such as a person, group, place, event, organization, or phenomenon. Case studies are commonly used in social, educational, clinical, and business research.

A case study research design usually involves qualitative methods , but quantitative methods are sometimes also used. Case studies are good for describing , comparing, evaluating and understanding different aspects of a research problem .

Table of contents

When to do a case study, step 1: select a case, step 2: build a theoretical framework, step 3: collect your data, step 4: describe and analyze the case, other interesting articles.

A case study is an appropriate research design when you want to gain concrete, contextual, in-depth knowledge about a specific real-world subject. It allows you to explore the key characteristics, meanings, and implications of the case.

Case studies are often a good choice in a thesis or dissertation . They keep your project focused and manageable when you don’t have the time or resources to do large-scale research.

You might use just one complex case study where you explore a single subject in depth, or conduct multiple case studies to compare and illuminate different aspects of your research problem.

Receive feedback on language, structure, and formatting

Professional editors proofread and edit your paper by focusing on:

• Academic style
• Vague sentences
• Style consistency

See an example

Once you have developed your problem statement and research questions , you should be ready to choose the specific case that you want to focus on. A good case study should have the potential to:

• Provide new or unexpected insights into the subject
• Challenge or complicate existing assumptions and theories
• Propose practical courses of action to resolve a problem
• Open up new directions for future research

TipIf your research is more practical in nature and aims to simultaneously investigate an issue as you solve it, consider conducting action research instead.

Unlike quantitative or experimental research , a strong case study does not require a random or representative sample. In fact, case studies often deliberately focus on unusual, neglected, or outlying cases which may shed new light on the research problem.

Example of an outlying case studyIn the 1960s the town of Roseto, Pennsylvania was discovered to have extremely low rates of heart disease compared to the US average. It became an important case study for understanding previously neglected causes of heart disease.

However, you can also choose a more common or representative case to exemplify a particular category, experience or phenomenon.

Example of a representative case studyIn the 1920s, two sociologists used Muncie, Indiana as a case study of a typical American city that supposedly exemplified the changing culture of the US at the time.

While case studies focus more on concrete details than general theories, they should usually have some connection with theory in the field. This way the case study is not just an isolated description, but is integrated into existing knowledge about the topic. It might aim to:

• Exemplify a theory by showing how it explains the case under investigation
• Expand on a theory by uncovering new concepts and ideas that need to be incorporated
• Challenge a theory by exploring an outlier case that doesn’t fit with established assumptions

To ensure that your analysis of the case has a solid academic grounding, you should conduct a literature review of sources related to the topic and develop a theoretical framework . This means identifying key concepts and theories to guide your analysis and interpretation.

There are many different research methods you can use to collect data on your subject. Case studies tend to focus on qualitative data using methods such as interviews , observations , and analysis of primary and secondary sources (e.g., newspaper articles, photographs, official records). Sometimes a case study will also collect quantitative data.

Example of a mixed methods case studyFor a case study of a wind farm development in a rural area, you could collect quantitative data on employment rates and business revenue, collect qualitative data on local people’s perceptions and experiences, and analyze local and national media coverage of the development.

The aim is to gain as thorough an understanding as possible of the case and its context.

Here's why students love Scribbr's proofreading services

Discover proofreading & editing

In writing up the case study, you need to bring together all the relevant aspects to give as complete a picture as possible of the subject.

How you report your findings depends on the type of research you are doing. Some case studies are structured like a standard scientific paper or thesis , with separate sections or chapters for the methods , results and discussion .

Others are written in a more narrative style, aiming to explore the case from various angles and analyze its meanings and implications (for example, by using textual analysis or discourse analysis ).

In all cases, though, make sure to give contextual details about the case, connect it back to the literature and theory, and discuss how it fits into wider patterns or debates.

If you want to know more about statistics , methodology , or research bias , make sure to check out some of our other articles with explanations and examples.

• Normal distribution
• Degrees of freedom
• Null hypothesis
• Discourse analysis
• Control groups
• Mixed methods research
• Non-probability sampling
• Quantitative research
• Ecological validity

Research bias

• Rosenthal effect
• Implicit bias
• Cognitive bias
• Selection bias
• Negativity bias
• Status quo bias

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

McCombes, S. (2023, November 20). What Is a Case Study? | Definition, Examples & Methods. Scribbr. Retrieved July 6, 2024, from https://www.scribbr.com/methodology/case-study/

Is this article helpful?

Shona McCombes

Other students also liked, primary vs. secondary sources | difference & examples, what is a theoretical framework | guide to organizing, what is action research | definition & examples, get unlimited documents corrected.

✔ Free APA citation check included ✔ Unlimited document corrections ✔ Specialized in correcting academic texts

• Privacy Policy

Home » Case Study – Methods, Examples and Guide

Case Study – Methods, Examples and Guide

Table of Contents

A case study is a research method that involves an in-depth examination and analysis of a particular phenomenon or case, such as an individual, organization, community, event, or situation.

It is a qualitative research approach that aims to provide a detailed and comprehensive understanding of the case being studied. Case studies typically involve multiple sources of data, including interviews, observations, documents, and artifacts, which are analyzed using various techniques, such as content analysis, thematic analysis, and grounded theory. The findings of a case study are often used to develop theories, inform policy or practice, or generate new research questions.

Types of Case Study

Types and Methods of Case Study are as follows:

Single-Case Study

A single-case study is an in-depth analysis of a single case. This type of case study is useful when the researcher wants to understand a specific phenomenon in detail.

For Example , A researcher might conduct a single-case study on a particular individual to understand their experiences with a particular health condition or a specific organization to explore their management practices. The researcher collects data from multiple sources, such as interviews, observations, and documents, and uses various techniques to analyze the data, such as content analysis or thematic analysis. The findings of a single-case study are often used to generate new research questions, develop theories, or inform policy or practice.

Multiple-Case Study

A multiple-case study involves the analysis of several cases that are similar in nature. This type of case study is useful when the researcher wants to identify similarities and differences between the cases.

For Example, a researcher might conduct a multiple-case study on several companies to explore the factors that contribute to their success or failure. The researcher collects data from each case, compares and contrasts the findings, and uses various techniques to analyze the data, such as comparative analysis or pattern-matching. The findings of a multiple-case study can be used to develop theories, inform policy or practice, or generate new research questions.

Exploratory Case Study

An exploratory case study is used to explore a new or understudied phenomenon. This type of case study is useful when the researcher wants to generate hypotheses or theories about the phenomenon.

For Example, a researcher might conduct an exploratory case study on a new technology to understand its potential impact on society. The researcher collects data from multiple sources, such as interviews, observations, and documents, and uses various techniques to analyze the data, such as grounded theory or content analysis. The findings of an exploratory case study can be used to generate new research questions, develop theories, or inform policy or practice.

Descriptive Case Study

A descriptive case study is used to describe a particular phenomenon in detail. This type of case study is useful when the researcher wants to provide a comprehensive account of the phenomenon.

For Example, a researcher might conduct a descriptive case study on a particular community to understand its social and economic characteristics. The researcher collects data from multiple sources, such as interviews, observations, and documents, and uses various techniques to analyze the data, such as content analysis or thematic analysis. The findings of a descriptive case study can be used to inform policy or practice or generate new research questions.

Instrumental Case Study

An instrumental case study is used to understand a particular phenomenon that is instrumental in achieving a particular goal. This type of case study is useful when the researcher wants to understand the role of the phenomenon in achieving the goal.

For Example, a researcher might conduct an instrumental case study on a particular policy to understand its impact on achieving a particular goal, such as reducing poverty. The researcher collects data from multiple sources, such as interviews, observations, and documents, and uses various techniques to analyze the data, such as content analysis or thematic analysis. The findings of an instrumental case study can be used to inform policy or practice or generate new research questions.

Case Study Data Collection Methods

Here are some common data collection methods for case studies:

Interviews involve asking questions to individuals who have knowledge or experience relevant to the case study. Interviews can be structured (where the same questions are asked to all participants) or unstructured (where the interviewer follows up on the responses with further questions). Interviews can be conducted in person, over the phone, or through video conferencing.

Observations

Observations involve watching and recording the behavior and activities of individuals or groups relevant to the case study. Observations can be participant (where the researcher actively participates in the activities) or non-participant (where the researcher observes from a distance). Observations can be recorded using notes, audio or video recordings, or photographs.

Documents can be used as a source of information for case studies. Documents can include reports, memos, emails, letters, and other written materials related to the case study. Documents can be collected from the case study participants or from public sources.

Surveys involve asking a set of questions to a sample of individuals relevant to the case study. Surveys can be administered in person, over the phone, through mail or email, or online. Surveys can be used to gather information on attitudes, opinions, or behaviors related to the case study.

Artifacts are physical objects relevant to the case study. Artifacts can include tools, equipment, products, or other objects that provide insights into the case study phenomenon.

How to conduct Case Study Research

Conducting a case study research involves several steps that need to be followed to ensure the quality and rigor of the study. Here are the steps to conduct case study research:

• Define the research questions: The first step in conducting a case study research is to define the research questions. The research questions should be specific, measurable, and relevant to the case study phenomenon under investigation.
• Select the case: The next step is to select the case or cases to be studied. The case should be relevant to the research questions and should provide rich and diverse data that can be used to answer the research questions.
• Collect data: Data can be collected using various methods, such as interviews, observations, documents, surveys, and artifacts. The data collection method should be selected based on the research questions and the nature of the case study phenomenon.
• Analyze the data: The data collected from the case study should be analyzed using various techniques, such as content analysis, thematic analysis, or grounded theory. The analysis should be guided by the research questions and should aim to provide insights and conclusions relevant to the research questions.
• Draw conclusions: The conclusions drawn from the case study should be based on the data analysis and should be relevant to the research questions. The conclusions should be supported by evidence and should be clearly stated.
• Validate the findings: The findings of the case study should be validated by reviewing the data and the analysis with participants or other experts in the field. This helps to ensure the validity and reliability of the findings.
• Write the report: The final step is to write the report of the case study research. The report should provide a clear description of the case study phenomenon, the research questions, the data collection methods, the data analysis, the findings, and the conclusions. The report should be written in a clear and concise manner and should follow the guidelines for academic writing.

Examples of Case Study

Here are some examples of case study research:

• The Hawthorne Studies : Conducted between 1924 and 1932, the Hawthorne Studies were a series of case studies conducted by Elton Mayo and his colleagues to examine the impact of work environment on employee productivity. The studies were conducted at the Hawthorne Works plant of the Western Electric Company in Chicago and included interviews, observations, and experiments.
• The Stanford Prison Experiment: Conducted in 1971, the Stanford Prison Experiment was a case study conducted by Philip Zimbardo to examine the psychological effects of power and authority. The study involved simulating a prison environment and assigning participants to the role of guards or prisoners. The study was controversial due to the ethical issues it raised.
• The Challenger Disaster: The Challenger Disaster was a case study conducted to examine the causes of the Space Shuttle Challenger explosion in 1986. The study included interviews, observations, and analysis of data to identify the technical, organizational, and cultural factors that contributed to the disaster.
• The Enron Scandal: The Enron Scandal was a case study conducted to examine the causes of the Enron Corporation’s bankruptcy in 2001. The study included interviews, analysis of financial data, and review of documents to identify the accounting practices, corporate culture, and ethical issues that led to the company’s downfall.
• The Fukushima Nuclear Disaster : The Fukushima Nuclear Disaster was a case study conducted to examine the causes of the nuclear accident that occurred at the Fukushima Daiichi Nuclear Power Plant in Japan in 2011. The study included interviews, analysis of data, and review of documents to identify the technical, organizational, and cultural factors that contributed to the disaster.

Application of Case Study

Case studies have a wide range of applications across various fields and industries. Here are some examples:

Business and Management

Case studies are widely used in business and management to examine real-life situations and develop problem-solving skills. Case studies can help students and professionals to develop a deep understanding of business concepts, theories, and best practices.

Case studies are used in healthcare to examine patient care, treatment options, and outcomes. Case studies can help healthcare professionals to develop critical thinking skills, diagnose complex medical conditions, and develop effective treatment plans.

Case studies are used in education to examine teaching and learning practices. Case studies can help educators to develop effective teaching strategies, evaluate student progress, and identify areas for improvement.

Social Sciences

Case studies are widely used in social sciences to examine human behavior, social phenomena, and cultural practices. Case studies can help researchers to develop theories, test hypotheses, and gain insights into complex social issues.

Law and Ethics

Case studies are used in law and ethics to examine legal and ethical dilemmas. Case studies can help lawyers, policymakers, and ethical professionals to develop critical thinking skills, analyze complex cases, and make informed decisions.

Purpose of Case Study

The purpose of a case study is to provide a detailed analysis of a specific phenomenon, issue, or problem in its real-life context. A case study is a qualitative research method that involves the in-depth exploration and analysis of a particular case, which can be an individual, group, organization, event, or community.

The primary purpose of a case study is to generate a comprehensive and nuanced understanding of the case, including its history, context, and dynamics. Case studies can help researchers to identify and examine the underlying factors, processes, and mechanisms that contribute to the case and its outcomes. This can help to develop a more accurate and detailed understanding of the case, which can inform future research, practice, or policy.

Case studies can also serve other purposes, including:

• Illustrating a theory or concept: Case studies can be used to illustrate and explain theoretical concepts and frameworks, providing concrete examples of how they can be applied in real-life situations.
• Developing hypotheses: Case studies can help to generate hypotheses about the causal relationships between different factors and outcomes, which can be tested through further research.
• Providing insight into complex issues: Case studies can provide insights into complex and multifaceted issues, which may be difficult to understand through other research methods.
• Informing practice or policy: Case studies can be used to inform practice or policy by identifying best practices, lessons learned, or areas for improvement.

Advantages of Case Study Research

There are several advantages of case study research, including:

• In-depth exploration: Case study research allows for a detailed exploration and analysis of a specific phenomenon, issue, or problem in its real-life context. This can provide a comprehensive understanding of the case and its dynamics, which may not be possible through other research methods.
• Rich data: Case study research can generate rich and detailed data, including qualitative data such as interviews, observations, and documents. This can provide a nuanced understanding of the case and its complexity.
• Holistic perspective: Case study research allows for a holistic perspective of the case, taking into account the various factors, processes, and mechanisms that contribute to the case and its outcomes. This can help to develop a more accurate and comprehensive understanding of the case.
• Theory development: Case study research can help to develop and refine theories and concepts by providing empirical evidence and concrete examples of how they can be applied in real-life situations.
• Practical application: Case study research can inform practice or policy by identifying best practices, lessons learned, or areas for improvement.
• Contextualization: Case study research takes into account the specific context in which the case is situated, which can help to understand how the case is influenced by the social, cultural, and historical factors of its environment.

Limitations of Case Study Research

There are several limitations of case study research, including:

• Limited generalizability : Case studies are typically focused on a single case or a small number of cases, which limits the generalizability of the findings. The unique characteristics of the case may not be applicable to other contexts or populations, which may limit the external validity of the research.
• Biased sampling: Case studies may rely on purposive or convenience sampling, which can introduce bias into the sample selection process. This may limit the representativeness of the sample and the generalizability of the findings.
• Subjectivity: Case studies rely on the interpretation of the researcher, which can introduce subjectivity into the analysis. The researcher’s own biases, assumptions, and perspectives may influence the findings, which may limit the objectivity of the research.
• Limited control: Case studies are typically conducted in naturalistic settings, which limits the control that the researcher has over the environment and the variables being studied. This may limit the ability to establish causal relationships between variables.
• Time-consuming: Case studies can be time-consuming to conduct, as they typically involve a detailed exploration and analysis of a specific case. This may limit the feasibility of conducting multiple case studies or conducting case studies in a timely manner.
• Resource-intensive: Case studies may require significant resources, including time, funding, and expertise. This may limit the ability of researchers to conduct case studies in resource-constrained settings.

About the author

Muhammad Hassan

Researcher, Academic Writer, Web developer

You may also like

One-to-One Interview – Methods and Guide

Basic Research – Types, Methods and Examples

Ethnographic Research -Types, Methods and Guide

Quasi-Experimental Research Design – Types...

Descriptive Research Design – Types, Methods and...

Focus Groups – Steps, Examples and Guide

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Sage Choice

Continuing to enhance the quality of case study methodology in health services research

Shannon l. sibbald.

1 Faculty of Health Sciences, Western University, London, Ontario, Canada.

2 Department of Family Medicine, Schulich School of Medicine and Dentistry, Western University, London, Ontario, Canada.

3 The Schulich Interfaculty Program in Public Health, Schulich School of Medicine and Dentistry, Western University, London, Ontario, Canada.

Stefan Paciocco

Meghan fournie, rachelle van asseldonk, tiffany scurr.

Case study methodology has grown in popularity within Health Services Research (HSR). However, its use and merit as a methodology are frequently criticized due to its flexible approach and inconsistent application. Nevertheless, case study methodology is well suited to HSR because it can track and examine complex relationships, contexts, and systems as they evolve. Applied appropriately, it can help generate information on how multiple forms of knowledge come together to inform decision-making within healthcare contexts. In this article, we aim to demystify case study methodology by outlining its philosophical underpinnings and three foundational approaches. We provide literature-based guidance to decision-makers, policy-makers, and health leaders on how to engage in and critically appraise case study design. We advocate that researchers work in collaboration with health leaders to detail their research process with an aim of strengthening the validity and integrity of case study for its continued and advanced use in HSR.

Introduction

The popularity of case study research methodology in Health Services Research (HSR) has grown over the past 40 years. 1 This may be attributed to a shift towards the use of implementation research and a newfound appreciation of contextual factors affecting the uptake of evidence-based interventions within diverse settings. 2 Incorporating context-specific information on the delivery and implementation of programs can increase the likelihood of success. 3 , 4 Case study methodology is particularly well suited for implementation research in health services because it can provide insight into the nuances of diverse contexts. 5 , 6 In 1999, Yin 7 published a paper on how to enhance the quality of case study in HSR, which was foundational for the emergence of case study in this field. Yin 7 maintains case study is an appropriate methodology in HSR because health systems are constantly evolving, and the multiple affiliations and diverse motivations are difficult to track and understand with traditional linear methodologies.

Despite its increased popularity, there is debate whether a case study is a methodology (ie, a principle or process that guides research) or a method (ie, a tool to answer research questions). Some criticize case study for its high level of flexibility, perceiving it as less rigorous, and maintain that it generates inadequate results. 8 Others have noted issues with quality and consistency in how case studies are conducted and reported. 9 Reporting is often varied and inconsistent, using a mix of approaches such as case reports, case findings, and/or case study. Authors sometimes use incongruent methods of data collection and analysis or use the case study as a default when other methodologies do not fit. 9 , 10 Despite these criticisms, case study methodology is becoming more common as a viable approach for HSR. 11 An abundance of articles and textbooks are available to guide researchers through case study research, including field-specific resources for business, 12 , 13 nursing, 14 and family medicine. 15 However, there remains confusion and a lack of clarity on the key tenets of case study methodology.

Several common philosophical underpinnings have contributed to the development of case study research 1 which has led to different approaches to planning, data collection, and analysis. This presents challenges in assessing quality and rigour for researchers conducting case studies and stakeholders reading results.

This article discusses the various approaches and philosophical underpinnings to case study methodology. Our goal is to explain it in a way that provides guidance for decision-makers, policy-makers, and health leaders on how to understand, critically appraise, and engage in case study research and design, as such guidance is largely absent in the literature. This article is by no means exhaustive or authoritative. Instead, we aim to provide guidance and encourage dialogue around case study methodology, facilitating critical thinking around the variety of approaches and ways quality and rigour can be bolstered for its use within HSR.

Purpose of case study methodology

Case study methodology is often used to develop an in-depth, holistic understanding of a specific phenomenon within a specified context. 11 It focuses on studying one or multiple cases over time and uses an in-depth analysis of multiple information sources. 16 , 17 It is ideal for situations including, but not limited to, exploring under-researched and real-life phenomena, 18 especially when the contexts are complex and the researcher has little control over the phenomena. 19 , 20 Case studies can be useful when researchers want to understand how interventions are implemented in different contexts, and how context shapes the phenomenon of interest.

In addition to demonstrating coherency with the type of questions case study is suited to answer, there are four key tenets to case study methodologies: (1) be transparent in the paradigmatic and theoretical perspectives influencing study design; (2) clearly define the case and phenomenon of interest; (3) clearly define and justify the type of case study design; and (4) use multiple data collection sources and analysis methods to present the findings in ways that are consistent with the methodology and the study’s paradigmatic base. 9 , 16 The goal is to appropriately match the methods to empirical questions and issues and not to universally advocate any single approach for all problems. 21

Approaches to case study methodology

Three authors propose distinct foundational approaches to case study methodology positioned within different paradigms: Yin, 19 , 22 Stake, 5 , 23 and Merriam 24 , 25 ( Table 1 ). Yin is strongly post-positivist whereas Stake and Merriam are grounded in a constructivist paradigm. Researchers should locate their research within a paradigm that explains the philosophies guiding their research 26 and adhere to the underlying paradigmatic assumptions and key tenets of the appropriate author’s methodology. This will enhance the consistency and coherency of the methods and findings. However, researchers often do not report their paradigmatic position, nor do they adhere to one approach. 9 Although deliberately blending methodologies may be defensible and methodologically appropriate, more often it is done in an ad hoc and haphazard way, without consideration for limitations.

Cross-analysis of three case study approaches, adapted from Yazan 2015

The post-positive paradigm postulates there is one reality that can be objectively described and understood by “bracketing” oneself from the research to remove prejudice or bias. 27 Yin focuses on general explanation and prediction, emphasizing the formulation of propositions, akin to hypothesis testing. This approach is best suited for structured and objective data collection 9 , 11 and is often used for mixed-method studies.

Constructivism assumes that the phenomenon of interest is constructed and influenced by local contexts, including the interaction between researchers, individuals, and their environment. 27 It acknowledges multiple interpretations of reality 24 constructed within the context by the researcher and participants which are unlikely to be replicated, should either change. 5 , 20 Stake and Merriam’s constructivist approaches emphasize a story-like rendering of a problem and an iterative process of constructing the case study. 7 This stance values researcher reflexivity and transparency, 28 acknowledging how researchers’ experiences and disciplinary lenses influence their assumptions and beliefs about the nature of the phenomenon and development of the findings.

Defining a case

A key tenet of case study methodology often underemphasized in literature is the importance of defining the case and phenomenon. Researches should clearly describe the case with sufficient detail to allow readers to fully understand the setting and context and determine applicability. Trying to answer a question that is too broad often leads to an unclear definition of the case and phenomenon. 20 Cases should therefore be bound by time and place to ensure rigor and feasibility. 6

Yin 22 defines a case as “a contemporary phenomenon within its real-life context,” (p13) which may contain a single unit of analysis, including individuals, programs, corporations, or clinics 29 (holistic), or be broken into sub-units of analysis, such as projects, meetings, roles, or locations within the case (embedded). 30 Merriam 24 and Stake 5 similarly define a case as a single unit studied within a bounded system. Stake 5 , 23 suggests bounding cases by contexts and experiences where the phenomenon of interest can be a program, process, or experience. However, the line between the case and phenomenon can become muddy. For guidance, Stake 5 , 23 describes the case as the noun or entity and the phenomenon of interest as the verb, functioning, or activity of the case.

Designing the case study approach

Yin’s approach to a case study is rooted in a formal proposition or theory which guides the case and is used to test the outcome. 1 Stake 5 advocates for a flexible design and explicitly states that data collection and analysis may commence at any point. Merriam’s 24 approach blends both Yin and Stake’s, allowing the necessary flexibility in data collection and analysis to meet the needs.

Yin 30 proposed three types of case study approaches—descriptive, explanatory, and exploratory. Each can be designed around single or multiple cases, creating six basic case study methodologies. Descriptive studies provide a rich description of the phenomenon within its context, which can be helpful in developing theories. To test a theory or determine cause and effect relationships, researchers can use an explanatory design. An exploratory model is typically used in the pilot-test phase to develop propositions (eg, Sibbald et al. 31 used this approach to explore interprofessional network complexity). Despite having distinct characteristics, the boundaries between case study types are flexible with significant overlap. 30 Each has five key components: (1) research question; (2) proposition; (3) unit of analysis; (4) logical linking that connects the theory with proposition; and (5) criteria for analyzing findings.

Contrary to Yin, Stake 5 believes the research process cannot be planned in its entirety because research evolves as it is performed. Consequently, researchers can adjust the design of their methods even after data collection has begun. Stake 5 classifies case studies into three categories: intrinsic, instrumental, and collective/multiple. Intrinsic case studies focus on gaining a better understanding of the case. These are often undertaken when the researcher has an interest in a specific case. Instrumental case study is used when the case itself is not of the utmost importance, and the issue or phenomenon (ie, the research question) being explored becomes the focus instead (eg, Paciocco 32 used an instrumental case study to evaluate the implementation of a chronic disease management program). 5 Collective designs are rooted in an instrumental case study and include multiple cases to gain an in-depth understanding of the complexity and particularity of a phenomenon across diverse contexts. 5 , 23 In collective designs, studying similarities and differences between the cases allows the phenomenon to be understood more intimately (for examples of this in the field, see van Zelm et al. 33 and Burrows et al. 34 In addition, Sibbald et al. 35 present an example where a cross-case analysis method is used to compare instrumental cases).

Merriam’s approach is flexible (similar to Stake) as well as stepwise and linear (similar to Yin). She advocates for conducting a literature review before designing the study to better understand the theoretical underpinnings. 24 , 25 Unlike Stake or Yin, Merriam proposes a step-by-step guide for researchers to design a case study. These steps include performing a literature review, creating a theoretical framework, identifying the problem, creating and refining the research question(s), and selecting a study sample that fits the question(s). 24 , 25 , 36

Data collection and analysis

Using multiple data collection methods is a key characteristic of all case study methodology; it enhances the credibility of the findings by allowing different facets and views of the phenomenon to be explored. 23 Common methods include interviews, focus groups, observation, and document analysis. 5 , 37 By seeking patterns within and across data sources, a thick description of the case can be generated to support a greater understanding and interpretation of the whole phenomenon. 5 , 17 , 20 , 23 This technique is called triangulation and is used to explore cases with greater accuracy. 5 Although Stake 5 maintains case study is most often used in qualitative research, Yin 17 supports a mix of both quantitative and qualitative methods to triangulate data. This deliberate convergence of data sources (or mixed methods) allows researchers to find greater depth in their analysis and develop converging lines of inquiry. For example, case studies evaluating interventions commonly use qualitative interviews to describe the implementation process, barriers, and facilitators paired with a quantitative survey of comparative outcomes and effectiveness. 33 , 38 , 39

Yin 30 describes analysis as dependent on the chosen approach, whether it be (1) deductive and rely on theoretical propositions; (2) inductive and analyze data from the “ground up”; (3) organized to create a case description; or (4) used to examine plausible rival explanations. According to Yin’s 40 approach to descriptive case studies, carefully considering theory development is an important part of study design. “Theory” refers to field-relevant propositions, commonly agreed upon assumptions, or fully developed theories. 40 Stake 5 advocates for using the researcher’s intuition and impression to guide analysis through a categorical aggregation and direct interpretation. Merriam 24 uses six different methods to guide the “process of making meaning” (p178) : (1) ethnographic analysis; (2) narrative analysis; (3) phenomenological analysis; (4) constant comparative method; (5) content analysis; and (6) analytic induction.

Drawing upon a theoretical or conceptual framework to inform analysis improves the quality of case study and avoids the risk of description without meaning. 18 Using Stake’s 5 approach, researchers rely on protocols and previous knowledge to help make sense of new ideas; theory can guide the research and assist researchers in understanding how new information fits into existing knowledge.

Practical applications of case study research

Columbia University has recently demonstrated how case studies can help train future health leaders. 41 Case studies encompass components of systems thinking—considering connections and interactions between components of a system, alongside the implications and consequences of those relationships—to equip health leaders with tools to tackle global health issues. 41 Greenwood 42 evaluated Indigenous peoples’ relationship with the healthcare system in British Columbia and used a case study to challenge and educate health leaders across the country to enhance culturally sensitive health service environments.

An important but often omitted step in case study research is an assessment of quality and rigour. We recommend using a framework or set of criteria to assess the rigour of the qualitative research. Suitable resources include Caelli et al., 43 Houghten et al., 44 Ravenek and Rudman, 45 and Tracy. 46

New directions in case study

Although “pragmatic” case studies (ie, utilizing practical and applicable methods) have existed within psychotherapy for some time, 47 , 48 only recently has the applicability of pragmatism as an underlying paradigmatic perspective been considered in HSR. 49 This is marked by uptake of pragmatism in Randomized Control Trials, recognizing that “gold standard” testing conditions do not reflect the reality of clinical settings 50 , 51 nor do a handful of epistemologically guided methodologies suit every research inquiry.

Pragmatism positions the research question as the basis for methodological choices, rather than a theory or epistemology, allowing researchers to pursue the most practical approach to understanding a problem or discovering an actionable solution. 52 Mixed methods are commonly used to create a deeper understanding of the case through converging qualitative and quantitative data. 52 Pragmatic case study is suited to HSR because its flexibility throughout the research process accommodates complexity, ever-changing systems, and disruptions to research plans. 49 , 50 Much like case study, pragmatism has been criticized for its flexibility and use when other approaches are seemingly ill-fit. 53 , 54 Similarly, authors argue that this results from a lack of investigation and proper application rather than a reflection of validity, legitimizing the need for more exploration and conversation among researchers and practitioners. 55

Although occasionally misunderstood as a less rigourous research methodology, 8 case study research is highly flexible and allows for contextual nuances. 5 , 6 Its use is valuable when the researcher desires a thorough understanding of a phenomenon or case bound by context. 11 If needed, multiple similar cases can be studied simultaneously, or one case within another. 16 , 17 There are currently three main approaches to case study, 5 , 17 , 24 each with their own definitions of a case, ontological and epistemological paradigms, methodologies, and data collection and analysis procedures. 37

Individuals’ experiences within health systems are influenced heavily by contextual factors, participant experience, and intricate relationships between different organizations and actors. 55 Case study research is well suited for HSR because it can track and examine these complex relationships and systems as they evolve over time. 6 , 7 It is important that researchers and health leaders using this methodology understand its key tenets and how to conduct a proper case study. Although there are many examples of case study in action, they are often under-reported and, when reported, not rigorously conducted. 9 Thus, decision-makers and health leaders should use these examples with caution. The proper reporting of case studies is necessary to bolster their credibility in HSR literature and provide readers sufficient information to critically assess the methodology. We also call on health leaders who frequently use case studies 56 – 58 to report them in the primary research literature.

The purpose of this article is to advocate for the continued and advanced use of case study in HSR and to provide literature-based guidance for decision-makers, policy-makers, and health leaders on how to engage in, read, and interpret findings from case study research. As health systems progress and evolve, the application of case study research will continue to increase as researchers and health leaders aim to capture the inherent complexities, nuances, and contextual factors. 7

• SUGGESTED TOPICS
• The Magazine
• Newsletters
• Managing Yourself
• Managing Teams
• Work-life Balance
• The Big Idea
• Data & Visuals
• Reading Lists
• Case Selections
• HBR Learning
• Topic Feeds
• Account Settings
• Email Preferences

What the Case Study Method Really Teaches

• Nitin Nohria

Seven meta-skills that stick even if the cases fade from memory.

It’s been 100 years since Harvard Business School began using the case study method. Beyond teaching specific subject matter, the case study method excels in instilling meta-skills in students. This article explains the importance of seven such skills: preparation, discernment, bias recognition, judgement, collaboration, curiosity, and self-confidence.

During my decade as dean of Harvard Business School, I spent hundreds of hours talking with our alumni. To enliven these conversations, I relied on a favorite question: “What was the most important thing you learned from your time in our MBA program?”

• Nitin Nohria is the George F. Baker Jr. and Distinguished Service University Professor. He served as the 10th dean of Harvard Business School, from 2010 to 2020.

Partner Center

Case Study Research Method in Psychology

Saul Mcleod, PhD

Editor-in-Chief for Simply Psychology

BSc (Hons) Psychology, MRes, PhD, University of Manchester

Saul Mcleod, PhD., is a qualified psychology teacher with over 18 years of experience in further and higher education. He has been published in peer-reviewed journals, including the Journal of Clinical Psychology.

Learn about our Editorial Process

Olivia Guy-Evans, MSc

Associate Editor for Simply Psychology

BSc (Hons) Psychology, MSc Psychology of Education

Olivia Guy-Evans is a writer and associate editor for Simply Psychology. She has previously worked in healthcare and educational sectors.

On This Page:

Case studies are in-depth investigations of a person, group, event, or community. Typically, data is gathered from various sources using several methods (e.g., observations & interviews).

The case study research method originated in clinical medicine (the case history, i.e., the patient’s personal history). In psychology, case studies are often confined to the study of a particular individual.

The information is mainly biographical and relates to events in the individual’s past (i.e., retrospective), as well as to significant events that are currently occurring in his or her everyday life.

The case study is not a research method, but researchers select methods of data collection and analysis that will generate material suitable for case studies.

Freud (1909a, 1909b) conducted very detailed investigations into the private lives of his patients in an attempt to both understand and help them overcome their illnesses.

This makes it clear that the case study is a method that should only be used by a psychologist, therapist, or psychiatrist, i.e., someone with a professional qualification.

There is an ethical issue of competence. Only someone qualified to diagnose and treat a person can conduct a formal case study relating to atypical (i.e., abnormal) behavior or atypical development.

 Famous Case Studies

• Anna O – One of the most famous case studies, documenting psychoanalyst Josef Breuer’s treatment of “Anna O” (real name Bertha Pappenheim) for hysteria in the late 1800s using early psychoanalytic theory.
• Little Hans – A child psychoanalysis case study published by Sigmund Freud in 1909 analyzing his five-year-old patient Herbert Graf’s house phobia as related to the Oedipus complex.
• Bruce/Brenda – Gender identity case of the boy (Bruce) whose botched circumcision led psychologist John Money to advise gender reassignment and raise him as a girl (Brenda) in the 1960s.
• Genie Wiley – Linguistics/psychological development case of the victim of extreme isolation abuse who was studied in 1970s California for effects of early language deprivation on acquiring speech later in life.
• Phineas Gage – One of the most famous neuropsychology case studies analyzes personality changes in railroad worker Phineas Gage after an 1848 brain injury involving a tamping iron piercing his skull.

Clinical Case Studies

• Studying the effectiveness of psychotherapy approaches with an individual patient
• Assessing and treating mental illnesses like depression, anxiety disorders, PTSD
• Neuropsychological cases investigating brain injuries or disorders

Child Psychology Case Studies

• Studying psychological development from birth through adolescence
• Cases of learning disabilities, autism spectrum disorders, ADHD
• Effects of trauma, abuse, deprivation on development

Types of Case Studies

• Explanatory case studies : Used to explore causation in order to find underlying principles. Helpful for doing qualitative analysis to explain presumed causal links.
• Exploratory case studies : Used to explore situations where an intervention being evaluated has no clear set of outcomes. It helps define questions and hypotheses for future research.
• Descriptive case studies : Describe an intervention or phenomenon and the real-life context in which it occurred. It is helpful for illustrating certain topics within an evaluation.
• Multiple-case studies : Used to explore differences between cases and replicate findings across cases. Helpful for comparing and contrasting specific cases.
• Intrinsic : Used to gain a better understanding of a particular case. Helpful for capturing the complexity of a single case.
• Collective : Used to explore a general phenomenon using multiple case studies. Helpful for jointly studying a group of cases in order to inquire into the phenomenon.

Where Do You Find Data for a Case Study?

There are several places to find data for a case study. The key is to gather data from multiple sources to get a complete picture of the case and corroborate facts or findings through triangulation of evidence. Most of this information is likely qualitative (i.e., verbal description rather than measurement), but the psychologist might also collect numerical data.

1. Primary sources

• Interviews – Interviewing key people related to the case to get their perspectives and insights. The interview is an extremely effective procedure for obtaining information about an individual, and it may be used to collect comments from the person’s friends, parents, employer, workmates, and others who have a good knowledge of the person, as well as to obtain facts from the person him or herself.
• Observations – Observing behaviors, interactions, processes, etc., related to the case as they unfold in real-time.
• Documents & Records – Reviewing private documents, diaries, public records, correspondence, meeting minutes, etc., relevant to the case.

2. Secondary sources

• News/Media – News coverage of events related to the case study.
• Academic articles – Journal articles, dissertations etc. that discuss the case.
• Government reports – Official data and records related to the case context.
• Books/films – Books, documentaries or films discussing the case.

3. Archival records

Searching historical archives, museum collections and databases to find relevant documents, visual/audio records related to the case history and context.

Public archives like newspapers, organizational records, photographic collections could all include potentially relevant pieces of information to shed light on attitudes, cultural perspectives, common practices and historical contexts related to psychology.

4. Organizational records

Organizational records offer the advantage of often having large datasets collected over time that can reveal or confirm psychological insights.

Of course, privacy and ethical concerns regarding confidential data must be navigated carefully.

However, with proper protocols, organizational records can provide invaluable context and empirical depth to qualitative case studies exploring the intersection of psychology and organizations.

• Organizational/industrial psychology research : Organizational records like employee surveys, turnover/retention data, policies, incident reports etc. may provide insight into topics like job satisfaction, workplace culture and dynamics, leadership issues, employee behaviors etc.
• Clinical psychology : Therapists/hospitals may grant access to anonymized medical records to study aspects like assessments, diagnoses, treatment plans etc. This could shed light on clinical practices.
• School psychology : Studies could utilize anonymized student records like test scores, grades, disciplinary issues, and counseling referrals to study child development, learning barriers, effectiveness of support programs, and more.

How do I Write a Case Study in Psychology?

Follow specified case study guidelines provided by a journal or your psychology tutor. General components of clinical case studies include: background, symptoms, assessments, diagnosis, treatment, and outcomes. Interpreting the information means the researcher decides what to include or leave out. A good case study should always clarify which information is the factual description and which is an inference or the researcher’s opinion.

1. Introduction

• Provide background on the case context and why it is of interest, presenting background information like demographics, relevant history, and presenting problem.
• Compare briefly to similar published cases if applicable. Clearly state the focus/importance of the case.

2. Case Presentation

• Describe the presenting problem in detail, including symptoms, duration,and impact on daily life.
• Include client demographics like age and gender, information about social relationships, and mental health history.
• Describe all physical, emotional, and/or sensory symptoms reported by the client.
• Use patient quotes to describe the initial complaint verbatim. Follow with full-sentence summaries of relevant history details gathered, including key components that led to a working diagnosis.
• Summarize clinical exam results, namely orthopedic/neurological tests, imaging, lab tests, etc. Note actual results rather than subjective conclusions. Provide images if clearly reproducible/anonymized.
• Clearly state the working diagnosis or clinical impression before transitioning to management.

3. Management and Outcome

• Indicate the total duration of care and number of treatments given over what timeframe. Use specific names/descriptions for any therapies/interventions applied.
• Present the results of the intervention,including any quantitative or qualitative data collected.
• For outcomes, utilize visual analog scales for pain, medication usage logs, etc., if possible. Include patient self-reports of improvement/worsening of symptoms. Note the reason for discharge/end of care.

4. Discussion

• Analyze the case, exploring contributing factors, limitations of the study, and connections to existing research.
• Analyze the effectiveness of the intervention,considering factors like participant adherence, limitations of the study, and potential alternative explanations for the results.
• Identify any questions raised in the case analysis and relate insights to established theories and current research if applicable. Avoid definitive claims about physiological explanations.
• Offer clinical implications, and suggest future research directions.

5. Additional Items

• Thank specific assistants for writing support only. No patient acknowledgments.
• References should directly support any key claims or quotes included.
• Use tables/figures/images only if substantially informative. Include permissions and legends/explanatory notes.
• Provides detailed (rich qualitative) information.
• Provides insight for further research.
• Permitting investigation of otherwise impractical (or unethical) situations.

Case studies allow a researcher to investigate a topic in far more detail than might be possible if they were trying to deal with a large number of research participants (nomothetic approach) with the aim of ‘averaging’.

Because of their in-depth, multi-sided approach, case studies often shed light on aspects of human thinking and behavior that would be unethical or impractical to study in other ways.

Research that only looks into the measurable aspects of human behavior is not likely to give us insights into the subjective dimension of experience, which is important to psychoanalytic and humanistic psychologists.

Case studies are often used in exploratory research. They can help us generate new ideas (that might be tested by other methods). They are an important way of illustrating theories and can help show how different aspects of a person’s life are related to each other.

The method is, therefore, important for psychologists who adopt a holistic point of view (i.e., humanistic psychologists ).

Limitations

• Lacking scientific rigor and providing little basis for generalization of results to the wider population.
• Researchers’ own subjective feelings may influence the case study (researcher bias).
• Difficult to replicate.
• Time-consuming and expensive.
• The volume of data, together with the time restrictions in place, impacted the depth of analysis that was possible within the available resources.

Because a case study deals with only one person/event/group, we can never be sure if the case study investigated is representative of the wider body of “similar” instances. This means the conclusions drawn from a particular case may not be transferable to other settings.

Because case studies are based on the analysis of qualitative (i.e., descriptive) data , a lot depends on the psychologist’s interpretation of the information she has acquired.

This means that there is a lot of scope for Anna O , and it could be that the subjective opinions of the psychologist intrude in the assessment of what the data means.

For example, Freud has been criticized for producing case studies in which the information was sometimes distorted to fit particular behavioral theories (e.g., Little Hans ).

This is also true of Money’s interpretation of the Bruce/Brenda case study (Diamond, 1997) when he ignored evidence that went against his theory.

Breuer, J., & Freud, S. (1895).  Studies on hysteria . Standard Edition 2: London.

Curtiss, S. (1981). Genie: The case of a modern wild child .

Diamond, M., & Sigmundson, K. (1997). Sex Reassignment at Birth: Long-term Review and Clinical Implications. Archives of Pediatrics & Adolescent Medicine , 151(3), 298-304

Freud, S. (1909a). Analysis of a phobia of a five year old boy. In The Pelican Freud Library (1977), Vol 8, Case Histories 1, pages 169-306

Freud, S. (1909b). Bemerkungen über einen Fall von Zwangsneurose (Der “Rattenmann”). Jb. psychoanal. psychopathol. Forsch ., I, p. 357-421; GW, VII, p. 379-463; Notes upon a case of obsessional neurosis, SE , 10: 151-318.

Harlow J. M. (1848). Passage of an iron rod through the head.  Boston Medical and Surgical Journal, 39 , 389–393.

Harlow, J. M. (1868).  Recovery from the Passage of an Iron Bar through the Head .  Publications of the Massachusetts Medical Society. 2  (3), 327-347.

Money, J., & Ehrhardt, A. A. (1972).  Man & Woman, Boy & Girl : The Differentiation and Dimorphism of Gender Identity from Conception to Maturity. Baltimore, Maryland: Johns Hopkins University Press.

Money, J., & Tucker, P. (1975). Sexual signatures: On being a man or a woman.

Further Information

• Case Study Approach
• Case Study Method
• Enhancing the Quality of Case Studies in Health Services Research
• “We do things together” A case study of “couplehood” in dementia
• Using mixed methods for evaluating an integrative approach to cancer care: a case study

Case Studies

Case studies are a popular research method in business area. Case studies aim to analyze specific issues within the boundaries of a specific environment, situation or organization.

According to its design, case studies in business research can be divided into three categories: explanatory, descriptive and exploratory.

Explanatory case studies aim to answer ‘how’ or ’why’ questions with little control on behalf of researcher over occurrence of events. This type of case studies focus on phenomena within the contexts of real-life situations. Example: “An investigation into the reasons of the global financial and economic crisis of 2008 – 2010.”

Descriptive case studies aim to analyze the sequence of interpersonal events after a certain amount of time has passed. Studies in business research belonging to this category usually describe culture or sub-culture, and they attempt to discover the key phenomena. Example: “Impact of increasing levels of multiculturalism on marketing practices: A case study of McDonald’s Indonesia.”

Exploratory case studies aim to find answers to the questions of ‘what’ or ‘who’. Exploratory case study data collection method is often accompanied by additional data collection method(s) such as interviews, questionnaires, experiments etc. Example: “A study into differences of leadership practices between private and public sector organizations in Atlanta, USA.”

Advantages of case study method include data collection and analysis within the context of phenomenon, integration of qualitative and quantitative data in data analysis, and the ability to capture complexities of real-life situations so that the phenomenon can be studied in greater levels of depth. Case studies do have certain disadvantages that may include lack of rigor, challenges associated with data analysis and very little basis for generalizations of findings and conclusions.

John Dudovskiy

Case Study Research: Methods and Designs

Case study research is a type of qualitative research design. It’s often used in the social sciences because it involves…

Case study research is a type of qualitative research design. It’s often used in the social sciences because it involves observing subjects, or cases, in their natural setting, with minimal interference from the researcher.

In the case study method , researchers pose a specific question about an individual or group to test their theories or hypothesis. This can be done by gathering data from interviews with key informants.

Here’s what you need to know about case study research design .

What Is The Case Study Method?

Main approaches to data collection, case study research methods, how case studies are used, case study model.

Case study research is a great way to understand the nuances of a matter that can get lost in quantitative research methods. A case study is distinct from other qualitative studies in the following ways:

• It’s interested in the effect of a set of circumstances on an individual or group.
• It begins with a specific question about one or more cases.
• It focuses on individual accounts and experiences.

Here are the primary features of case study research:

• Case study research methods typically involve the researcher asking a few questions of one person or a small number of people—known as respondents—to test one hypothesis.
• Case study in research methodology may apply triangulation to collect data, in which the researcher uses several sources, including documents and field data. This is then analyzed and interpreted to form a hypothesis that can be tested through further research or validated by other researchers.
• The case study method requires clear concepts and theories to guide its methods. A well-defined research question is crucial when conducting a case study because the results of the study depend on it. The best approach to answering a research question is to challenge the existing theories, hypotheses or assumptions.
• Concepts are defined using objective language with no reference to preconceived notions that individuals might have about them. The researcher sets out to discover by asking specific questions on how people think or perceive things in their given situation.

They commonly use the case study method in business, management, psychology, sociology, political science and other related fields.

A fundamental requirement of qualitative research is recording observations that provide an understanding of reality. When it comes to the case study method, there are two major approaches that can be used to collect data: document review and fieldwork.

A case study in research methodology also includes literature review, the process by which the researcher collects all data available through historical documents. These might include books, newspapers, journals, videos, photographs and other written material. The researcher may also record information using video cameras to capture events as they occur. The researcher can also go through materials produced by people involved in the case study to gain an insight into their lives and experiences.

Field research involves participating in interviews and observations directly. Observation can be done during telephone interviews, events or public meetings, visits to homes or workplaces, or by shadowing someone for a period of time. The researcher can conduct one-on-one interviews with individuals or group interviews where several people are interviewed at once.

Let’s look now at case study methodology.

The case study method can be divided into three stages: formulation of objectives; collection of data; and analysis and interpretation. The researcher first makes a judgment about what should be studied based on their knowledge. Next, they gather data through observations and interviews. Here are some of the common case study research methods:

One of the most basic methods is the survey. Respondents are asked to complete a questionnaire with open-ended and predetermined questions. It usually takes place through face-to-face interviews, mailed questionnaires or telephone interviews. It can even be done by an online survey.

2. Semi-structured Interview

For case study research a more complex method is the semi-structured interview. This involves the researcher learning about the topic by listening to what others have to say. This usually occurs through one-on-one interviews with the sample. Semi-structured interviews allow for greater flexibility and can obtain information that structured questionnaires can’t.

3. Focus Group Interview

Another method is the focus group interview, where the researcher asks a few people to take part in an open-ended discussion on certain themes or topics. The typical group size is 5–15 people. This method allows researchers to delve deeper into people’s opinions, views and experiences.

4. Participant Observation

Participant observation is another method that involves the researcher gaining insight into an experience by joining in and taking part in normal events. The people involved don’t always know they’re being studied, but the researcher observes and records what happens through field notes.

Case study research design can use one or several of these methods depending on the context.

Case studies are widely used in the social sciences. To understand the impact of socio-economic forces, interpersonal dynamics and other human conditions, sometimes there’s no other way than to study one case at a time and look for patterns and data afterward.

It’s for the same reasons that case studies are used in business. Here are a few uses:

• Case studies can be used as tools to educate and give examples of situations and problems that might occur and how they were resolved. They can also be used for strategy development and implementation.
• Case studies can evaluate the success of a program or project. They can help teams improve their collaboration by identifying areas that need improvements, such as team dynamics, communication, roles and responsibilities and leadership styles.
• Case studies can explore how people’s experiences affect the working environment. Because the study involves observing and analyzing concrete details of life, they can inform theories on how an individual or group interacts with their environment.
• Case studies can evaluate the sustainability of businesses. They’re useful for social, environmental and economic impact studies because they look at all aspects of a business or organization. This gives researchers a holistic view of the dynamics within an organization.
• We can use case studies to identify problems in organizations or businesses. They can help spot problems that are invisible to customers, investors, managers and employees.
• Case studies are used in education to show students how real-world issues or events can be sorted out. This enables students to identify and deal with similar situations in their lives.

And that’s not all. Case studies are incredibly versatile, which is why they’re used so widely.

Human beings are complex and they interact with each other in their everyday life in various ways. The researcher observes a case and tries to find out how the patterns of behavior are created, including their causal relations. Case studies help understand one or more specific events that have been observed. Here are some common methods:

1. Illustrative case study

This is where the researcher observes a group of people doing something. Studying an event or phenomenon this way can show cause-and-effect relationships between various variables.

2. Cumulative case study

A cumulative case study is one that involves observing the same set of phenomena over a period. Cumulative case studies can be very helpful in understanding processes, which are things that happen over time. For example, if there are behavioral changes in people who move from one place to another, the researcher might want to know why these changes occurred.

3. Exploratory case study

An exploratory case study collects information that will answer a question. It can help researchers better understand social, economic, political or other social phenomena.

There are several other ways to categorize case studies. They may be chronological case studies, where a researcher observes events over time. In the comparative case study, the researcher compares one or more groups of people, places, or things to draw conclusions about them. In an intervention case study, the researcher intervenes to change the behavior of the subjects. The study method depends on the needs of the research team.

Deciding how to analyze the information at our disposal is an important part of effective management. An understanding of the case study model can help. With Harappa’s Thinking Critically course, managers and young professionals receive input and training on how to level up their analytic skills. Knowledge of frameworks, reading real-life examples and lived wisdom of faculty come together to create a dynamic and exciting course that helps teams leap to the next level.

Explore Harappa Diaries to learn more about topics such as Objectives Of Research , What are Qualitative Research Methods , How To Make A Problem Statement and How To Improve your Cognitive Skills to upgrade your knowledge and skills.

• Open access
• Published: 03 July 2024

The impact of evidence-based nursing leadership in healthcare settings: a mixed methods systematic review

• Maritta Välimäki 1 , 2 ,
• Shuang Hu 3 ,
• Tella Lantta 1 ,
• Kirsi Hipp 1 , 4 ,
• Jaakko Varpula 1 ,
• Jiarui Chen 3 ,
• Gaoming Liu 5 ,
• Yao Tang 3 ,
• Wenjun Chen 3 &
• Xianhong Li 3  

BMC Nursing volume  23 , Article number:  452 ( 2024 ) Cite this article

135 Accesses

Metrics details

The central component in impactful healthcare decisions is evidence. Understanding how nurse leaders use evidence in their own managerial decision making is still limited. This mixed methods systematic review aimed to examine how evidence is used to solve leadership problems and to describe the measured and perceived effects of evidence-based leadership on nurse leaders and their performance, organizational, and clinical outcomes.

We included articles using any type of research design. We referred nurses, nurse managers or other nursing staff working in a healthcare context when they attempt to influence the behavior of individuals or a group in an organization using an evidence-based approach. Seven databases were searched until 11 November 2021. JBI Critical Appraisal Checklist for Quasi-experimental studies, JBI Critical Appraisal Checklist for Case Series, Mixed Methods Appraisal Tool were used to evaluate the Risk of bias in quasi-experimental studies, case series, mixed methods studies, respectively. The JBI approach to mixed methods systematic reviews was followed, and a parallel-results convergent approach to synthesis and integration was adopted.

Thirty-one publications were eligible for the analysis: case series ( n  = 27), mixed methods studies ( n  = 3) and quasi-experimental studies ( n  = 1). All studies were included regardless of methodological quality. Leadership problems were related to the implementation of knowledge into practice, the quality of nursing care and the resource availability. Organizational data was used in 27 studies to understand leadership problems, scientific evidence from literature was sought in 26 studies, and stakeholders’ views were explored in 24 studies. Perceived and measured effects of evidence-based leadership focused on nurses’ performance, organizational outcomes, and clinical outcomes. Economic data were not available.

Conclusions

This is the first systematic review to examine how evidence is used to solve leadership problems and to describe its measured and perceived effects from different sites. Although a variety of perceptions and effects were identified on nurses’ performance as well as on organizational and clinical outcomes, available knowledge concerning evidence-based leadership is currently insufficient. Therefore, more high-quality research and clinical trial designs are still needed.

Trail registration

The study was registered (PROSPERO CRD42021259624).

Peer Review reports

Global health demands have set new roles for nurse leaders [ 1 ].Nurse leaders are referred to as nurses, nurse managers, or other nursing staff working in a healthcare context who attempt to influence the behavior of individuals or a group based on goals that are congruent with organizational goals [ 2 ]. They are seen as professionals “armed with data and evidence, and a commitment to mentorship and education”, and as a group in which “leaders innovate, transform, and achieve quality outcomes for patients, health care professionals, organizations, and communities” [ 3 ]. Effective leadership occurs when team members critically follow leaders and are motivated by a leader’s decisions based on the organization’s requests and targets [ 4 ]. On the other hand, problems caused by poor leadership may also occur, regarding staff relations, stress, sickness, or retention [ 5 ]. Therefore, leadership requires an understanding of different problems to be solved using synthesizing evidence from research, clinical expertise, and stakeholders’ preferences [ 6 , 7 ]. If based on evidence, leadership decisions, also referred as leadership decision making [ 8 ], could ensure adequate staffing [ 7 , 9 ] and to produce sufficient and cost-effective care [ 10 ]. However, nurse leaders still rely on their decision making on their personal [ 11 ] and professional experience [ 10 ] over research evidence, which can lead to deficiencies in the quality and safety of care delivery [ 12 , 13 , 14 ]. As all nurses should demonstrate leadership in their profession, their leadership competencies should be strengthened [ 15 ].

Evidence-informed decision-making, referred to as evidence appraisal and application, and evaluation of decisions [ 16 ], has been recognized as one of the core competencies for leaders [ 17 , 18 ]. The role of evidence in nurse leaders’ managerial decision making has been promoted by public authorities [ 19 , 20 , 21 ]. Evidence-based management, another concept related to evidence-based leadership, has been used as the potential to improve healthcare services [ 22 ]. It can guide nursing leaders, in developing working conditions, staff retention, implementation practices, strategic planning, patient care, and success of leadership [ 13 ]. Collins and Holton [ 23 ] in their systematic review and meta-analysis examined 83 studies regarding leadership development interventions. They found that leadership training can result in significant improvement in participants’ skills, especially in knowledge level, although the training effects varied across studies. Cummings et al. [ 24 ] reviewed 100 papers (93 studies) and concluded that participation in leadership interventions had a positive impact on the development of a variety of leadership styles. Clavijo-Chamorro et al. [ 25 ] in their review of 11 studies focused on leadership-related factors that facilitate evidence implementation: teamwork, organizational structures, and transformational leadership. The role of nurse managers was to facilitate evidence-based practices by transforming contexts to motivate the staff and move toward a shared vision of change.

As far as we are aware, however, only a few systematic reviews have focused on evidence-based leadership or related concepts in the healthcare context aiming to analyse how nurse leaders themselves uses evidence in the decision-making process. Young [ 26 ] targeted definitions and acceptance of evidence-based management (EBMgt) in healthcare while Hasanpoor et al. [ 22 ] identified facilitators and barriers, sources of evidence used, and the role of evidence in the process of decision making. Both these reviews concluded that EBMgt was of great importance but used limitedly in healthcare settings due to a lack of time, a lack of research management activities, and policy constraints. A review by Williams [ 27 ] showed that the usage of evidence to support management in decision making is marginal due to a shortage of relevant evidence. Fraser [ 28 ] in their review further indicated that the potential evidence-based knowledge is not used in decision making by leaders as effectively as it could be. Non-use of evidence occurs and leaders base their decisions mainly on single studies, real-world evidence, and experts’ opinions [ 29 ]. Systematic reviews and meta-analyses rarely provide evidence of management-related interventions [ 30 ]. Tate et al. [ 31 ] concluded based on their systematic review and meta-analysis that the ability of nurse leaders to use and critically appraise research evidence may influence the way policy is enacted and how resources and staff are used to meet certain objectives set by policy. This can further influence staff and workforce outcomes. It is therefore important that nurse leaders have the capacity and motivation to use the strongest evidence available to effect change and guide their decision making [ 27 ].

Despite of a growing body of evidence, we found only one review focusing on the impact of evidence-based knowledge. Geert et al. [ 32 ] reviewed literature from 2007 to 2016 to understand the elements of design, delivery, and evaluation of leadership development interventions that are the most reliably linked to outcomes at the level of the individual and the organization, and that are of most benefit to patients. The authors concluded that it is possible to improve individual-level outcomes among leaders, such as knowledge, motivation, skills, and behavior change using evidence-based approaches. Some of the most effective interventions included, for example, interactive workshops, coaching, action learning, and mentoring. However, these authors found limited research evidence describing how nurse leaders themselves use evidence to support their managerial decisions in nursing and what the outcomes are.

To fill the knowledge gap and compliment to existing knowledgebase, in this mixed methods review we aimed to (1) examine what leadership problems nurse leaders solve using an evidence-based approach and (2) how they use evidence to solve these problems. We also explored (3) the measured and (4) perceived effects of the evidence-based leadership approach in healthcare settings. Both qualitative and quantitative components of the effects of evidence-based leadership were examined to provide greater insights into the available literature [ 33 ]. Together with the evidence-based leadership approach, and its impact on nursing [ 34 , 35 ], this knowledge gained in this review can be used to inform clinical policy or organizational decisions [ 33 ]. The study is registered (PROSPERO CRD42021259624). The methods used in this review were specified in advance and documented in a priori in a published protocol [ 36 ]. Key terms of the review and the search terms are defined in Table  1 (population, intervention, comparison, outcomes, context, other).

In this review, we used a mixed methods approach [ 37 ]. A mixed methods systematic review was selected as this approach has the potential to produce direct relevance to policy makers and practitioners [ 38 ]. Johnson and Onwuegbuzie [ 39 ] have defined mixed methods research as “the class of research in which the researcher mixes or combines quantitative and qualitative research techniques, methods, approaches, concepts or language into a single study.” Therefore, we combined quantitative and narrative analysis to appraise and synthesize empirical evidence, and we held them as equally important in informing clinical policy or organizational decisions [ 34 ]. In this review, a comprehensive synthesis of quantitative and qualitative data was performed first and then discussed in discussion part (parallel-results convergent design) [ 40 ]. We hoped that different type of analysis approaches could complement each other and deeper picture of the topic in line with our research questions could be gained [ 34 ].

Inclusion and exclusion criteria

Inclusion and exclusion criteria of the study are described in Table  1 .

Search strategy

A three-step search strategy was utilized. First, an initial limited search with #MEDLINE was undertaken, followed by analysis of the words used in the title, abstract, and the article’s key index terms. Second, the search strategy, including identified keywords and index terms, was adapted for each included data base and a second search was undertaken on 11 November 2021. The full search strategy for each database is described in Additional file 1 . Third, the reference list of all studies included in the review were screened for additional studies. No year limits or language restrictions were used.

Information sources

The database search included the following: CINAHL (EBSCO), Cochrane Library (academic database for medicine and health science and nursing), Embase (Elsevier), PsycINFO (EBSCO), PubMed (MEDLINE), Scopus (Elsevier) and Web of Science (academic database across all scientific and technical disciplines, ranging from medicine and social sciences to arts and humanities). These databases were selected as they represent typical databases in health care context. Subject headings from each of the databases were included in the search strategies. Boolean operators ‘AND’ and ‘OR’ were used to combine the search terms. An information specialist from the University of Turku Library was consulted in the formation of the search strategies.

Study selection

All identified citations were collated and uploaded into Covidence software (Covidence systematic review software, Veritas Health Innovation, Melbourne, Australia www.covidence.org ), and duplicates were removed by the software. Titles and abstracts were screened and assessed against the inclusion criteria independently by two reviewers out of four, and any discrepancies were resolved by the third reviewer (MV, KH, TL, WC). Studies meeting the inclusion criteria were retrieved in full and archived in Covidence. Access to one full-text article was lacking: the authors for one study were contacted about the missing full text, but no full text was received. All remaining hits of the included studies were retrieved and assessed independently against the inclusion criteria by two independent reviewers of four (MV, KH, TL, WC). Studies that did not meet the inclusion criteria were excluded, and the reasons for exclusion were recorded in Covidence. Any disagreements that arose between the reviewers were resolved through discussions with XL.

Assessment of methodological quality

Eligible studies were critically appraised by two independent reviewers (YT, SH). Standardized critical appraisal instruments based on the study design were used. First, quasi-experimental studies were assessed using the JBI Critical Appraisal Checklist for Quasi-experimental studies [ 44 ]. Second, case series were assessed using the JBI Critical Appraisal Checklist for Case Series [ 45 ]. Third, mixed methods studies were appraised using the Mixed Methods Appraisal Tool [ 46 ].

To increase inter-reviewer reliability, the review agreement was calculated (SH) [ 47 ]. A kappa greater than 0.8 was considered to represent a high level of agreement (0–0.1). In our data, the agreement was 0.75. Discrepancies raised between two reviewers were resolved through discussion and modifications and confirmed by XL. As an outcome, studies that met the inclusion criteria were proceeded to critical appraisal and assessed as suitable for inclusion in the review. The scores for each item and overall critical appraisal scores were presented.

Data extraction

For data extraction, specific tables were created. First, study characteristics (author(s), year, country, design, number of participants, setting) were extracted by two authors independently (JC, MV) and reviewed by TL. Second, descriptions of the interventions were extracted by two reviewers (JV, JC) using the structure of the TIDIeR (Template for Intervention Description and Replication) checklist (brief name, the goal of the intervention, material and procedure, models of delivery and location, dose, modification, adherence and fidelity) [ 48 ]. The extractions were confirmed (MV).

Third, due to a lack of effectiveness data and a wide heterogeneity between study designs and presentation of outcomes, no attempt was made to pool the quantitative data statistically; the findings of the quantitative data were presented in narrative form only [ 44 ]. The separate data extraction tables for each research question were designed specifically for this study. For both qualitative (and a qualitative component of mixed-method studies) and quantitative studies, the data were extracted and tabulated into text format according to preplanned research questions [ 36 ]. To test the quality of the tables and the data extraction process, three authors independently extracted the data from the first five studies (in alphabetical order). After that, the authors came together to share and determine whether their approaches of the data extraction were consistent with each other’s output and whether the content of each table was in line with research question. No reason was found to modify the data extraction tables or planned process. After a consensus of the data extraction process was reached, the data were extracted in pairs by independent reviewers (WC, TY, SH, GL). Any disagreements that arose between the reviewers were resolved through discussion and with a third reviewer (MV).

Data analysis

We were not able to conduct a meta-analysis due to a lack of effectiveness data based on clinical trials. Instead, we used inductive thematic analysis with constant comparison to answer the research question [ 46 , 49 ] using tabulated primary data from qualitative and quantitative studies as reported by the original authors in narrative form only [ 47 ]. In addition, the qualitizing process was used to transform quantitative data to qualitative data; this helped us to convert the whole data into themes and categories. After that we used the thematic analysis for the narrative data as follows. First, the text was carefully read, line by line, to reveal topics answering each specific review question (MV). Second, the data coding was conducted, and the themes in the data were formed by data categorization. The process of deriving the themes was inductive based on constant comparison [ 49 ]. The results of thematic analysis and data categorization was first described in narrative format and then the total number of studies was calculated where the specific category was identified (%).

Stakeholder involvement

The method of reporting stakeholders’ involvement follows the key components by [ 50 ]: (1) people involved, (2) geographical location, (3) how people were recruited, (4) format of involvement, (5) amount of involvement, (6) ethical approval, (7) financial compensation, and (8) methods for reporting involvement.

In our review, stakeholder involvement targeted nurses and nurse leader in China. Nurse Directors of two hospitals recommended potential participants who received a personal invitation letter from researchers to participate in a discussion meeting. Stakeholders’ participation was based on their own free will. Due to COVID-19, one online meeting (1 h) was organized (25 May 2022). Eleven participants joined the meeting. Ethical approval was not applied and no financial compensation was offered. At the end of the meeting, experiences of stakeholders’ involvement were explored.

The meeting started with an introductory presentation with power points. The rationale, methods, and preliminary review results were shared with the participants [ 51 ].The meeting continued with general questions for the participants: (1) Are you aware of the concepts of evidence-based practice or evidence-based leadership?; (2) How important is it to use evidence to support decisions among nurse leaders?; (3) How is the evidence-based approach used in hospital settings?; and (4) What type of evidence is currently used to support nurse leaders’ decision making (e.g. scientific literature, organizational data, stakeholder views)?

Two people took notes on the course and content of the conversation. The notes were later transcripted in verbatim, and the key points of the discussions were summarised. Although answers offered by the stakeholders were very short, the information was useful to validate the preliminary content of the results, add the rigorousness of the review, and obtain additional perspectives. A recommendation of the stakeholders was combined in the Discussion part of this review increasing the applicability of the review in the real world [ 50 ]. At the end of the discussion, the value of stakeholders’ involvement was asked. Participants shared that the experience of participating was unique and the topic of discussion was challenging. Two authors of the review group further represented stakeholders by working together with the research team throughout the review study.

Search results

From seven different electronic databases, 6053 citations were identified as being potentially relevant to the review. Then, 3133 duplicates were removed by an automation tool (Covidence: www.covidence.org ), and one was removed manually. The titles and abstracts of 3040 of citations were reviewed, and a total of 110 full texts were included (one extra citation was found on the reference list but later excluded). Based on the eligibility criteria, 31 studies (32 hits) were critically appraised and deemed suitable for inclusion in the review. The search results and selection process are presented in the PRISMA [ 52 ] flow diagram Fig.  1 . The full list of references for included studies can be find in Additional file 2 . To avoid confusion between articles of the reference list and studies included in the analysis, the studies included in the review are referred inside the article using the reference number of each study (e.g. ref 1, ref 2).

Search results and study selection and inclusion process [ 52 ]

Characteristics of included studies

The studies had multiple purposes, aiming to develop practice, implement a new approach, improve quality, or to develop a model. The 31 studies (across 32 hits) were case series studies ( n  = 27), mixed methods studies ( n  = 3) and a quasi-experimental study ( n  = 1). All studies were published between the years 2004 and 2021. The highest number of papers was published in year 2020.

Table  2 describes the characteristics of included studies and Additional file 3 offers a narrative description of the studies.

Methodological quality assessment

Quasi-experimental studies.

We had one quasi-experimental study (ref 31). All questions in the critical appraisal tool were applicable. The total score of the study was 8 (out of a possible 9). Only one response of the tool was ‘no’ because no control group was used in the study (see Additional file 4 for the critical appraisal of included studies).

Case series studies . A case series study is typically defined as a collection of subjects with common characteristics. The studies do not include a comparison group and are often based on prevalent cases and on a sample of convenience [ 53 ]. Munn et al. [ 45 ] further claim that case series are best described as observational studies, lacking experimental and randomized characteristics, being descriptive studies, without a control or comparator group. Out of 27 case series studies included in our review, the critical appraisal scores varied from 1 to 9. Five references were conference abstracts with empirical study results, which were scored from 1 to 3. Full reports of these studies were searched in electronic databases but not found. Critical appraisal scores for the remaining 22 studies ranged from 1 to 9 out of a possible score of 10. One question (Q3) was not applicable to 13 studies: “Were valid methods used for identification of the condition for all participants included in the case series?” Only two studies had clearly reported the demographic of the participants in the study (Q6). Twenty studies met Criteria 8 (“Were the outcomes or follow-up results of cases clearly reported?”) and 18 studies met Criteria 7 (“Q7: Was there clear reporting of clinical information of the participants?”) (see Additional file 4 for the critical appraisal of included studies).

Mixed-methods studies

Mixed-methods studies involve a combination of qualitative and quantitative methods. This is a common design and includes convergent design, sequential explanatory design, and sequential exploratory design [ 46 ]. There were three mixed-methods studies. The critical appraisal scores for the three studies ranged from 60 to 100% out of a possible 100%. Two studies met all the criteria, while one study fulfilled 60% of the scored criteria due to a lack of information to understand the relevance of the sampling strategy well enough to address the research question (Q4.1) or to determine whether the risk of nonresponse bias was low (Q4.4) (see Additional file 4 for the critical appraisal of included studies).

Intervention or program components

The intervention of program components were categorized and described using the TiDier checklist: name and goal, theory or background, material, procedure, provider, models of delivery, location, dose, modification, and adherence and fidelity [ 48 ]. A description of intervention in each study is described in Additional file 5 and a narrative description in Additional file 6 .

Leadership problems

In line with the inclusion criteria, data for the leadership problems were categorized in all 31 included studies (see Additional file 7 for leadership problems). Three types of leadership problems were identified: implementation of knowledge into practice, the quality of clinical care, and resources in nursing care. A narrative summary of the results is reported below.

Implementing knowledge into practice

Eleven studies (35%) aimed to solve leadership problems related to implementation of knowledge into practice. Studies showed how to support nurses in evidence-based implementation (EBP) (ref 3, ref 5), how to engage nurses in using evidence in practice (ref 4), how to convey the importance of EBP (ref 22) or how to change practice (ref 4). Other problems were how to facilitate nurses to use guideline recommendations (ref 7) and how nurses can make evidence-informed decisions (ref 8). General concerns also included the linkage between theory and practice (ref 1) as well as how to implement the EBP model in practice (ref 6). In addition, studies were motivated by the need for revisions or updates of protocols to improve clinical practice (ref 10) as well as the need to standardize nursing activities (ref 11, ref 14).

The quality of the care

Thirteen (42%) focused on solving problems related to the quality of clinical care. In these studies, a high number of catheter infections led a lack of achievement of organizational goals (ref 2, ref 9). A need to reduce patient symptoms in stem cell transplant patients undergoing high-dose chemotherapy (ref 24) was also one of the problems to be solved. In addition, the projects focused on how to prevent pressure ulcers (ref 26, ref 29), how to enhance the quality of cancer treatment (ref 25) and how to reduce the need for invasive constipation treatment (ref 30). Concerns about patient safety (ref 15), high fall rates (ref 16, ref 19), dissatisfaction of patients (ref 16, ref 18) and nurses (ref 16, ref 30) were also problems that had initiated the projects. Studies addressed concerns about how to promote good contingency care in residential aged care homes (ref 20) and about how to increase recognition of human trafficking problems in healthcare (ref 21).

Resources in nursing care

Nurse leaders identified problems in their resources, especially in staffing problems. These problems were identified in seven studies (23%), which involved concerns about how to prevent nurses from leaving the job (ref 31), how to ensure appropriate recruitment, staffing and retaining of nurses (ref 13) and how to decrease nurses’ burden and time spent on nursing activities (ref 12). Leadership turnover was also reported as a source of dissatisfaction (ref 17); studies addressed a lack of structured transition and training programs, which led to turnover (ref 23), as well as how to improve intershift handoff among nurses (ref 28). Optimal design for new hospitals was also examined (ref 27).

Main features of evidence-based leadership

Out of 31 studies, 17 (55%) included all four domains of an evidence-based leadership approach, and four studies (13%) included evidence of critical appraisal of the results (see Additional file 8 for the main features of evidence-based Leadership) (ref 11, ref 14, ref 23, ref 27).

Organizational evidence

Twenty-seven studies (87%) reported how organizational evidence was collected and used to solve leadership problems (ref 2). Retrospective chart reviews (ref 5), a review of the extent of specific incidents (ref 19), and chart auditing (ref 7, ref 25) were conducted. A gap between guideline recommendations and actual care was identified using organizational data (ref 7) while the percentage of nurses’ working time spent on patient care was analyzed using an electronic charting system (ref 12). Internal data (ref 22), institutional data, and programming metrics were also analyzed to understand the development of the nurse workforce (ref 13).

Surveys (ref 3, ref 25), interviews (ref 3, ref 25) and group reviews (ref 18) were used to better understand the leadership problem to be solved. Employee opinion surveys on leadership (ref 17), a nurse satisfaction survey (ref 30) and a variety of reporting templates were used for the data collection (ref 28) reported. Sometimes, leadership problems were identified by evidence facilitators or a PI’s team who worked with staff members (ref 15, ref 17). Problems in clinical practice were also identified by the Nursing Professional Council (ref 14), managers (ref 26) or nurses themselves (ref 24). Current practices were reviewed (ref 29) and a gap analysis was conducted (ref 4, ref 16, ref 23) together with SWOT analysis (ref 16). In addition, hospital mission and vision statements, research culture established and the proportion of nursing alumni with formal EBP training were analyzed (ref 5). On the other hand, it was stated that no systematic hospital-specific sources of data regarding job satisfaction or organizational commitment were used (ref 31). In addition, statements of organizational analysis were used on a general level only (ref 1).

Scientific evidence identified

Twenty-six studies (84%) reported the use of scientific evidence in their evidence-based leadership processes. A literature search was conducted (ref 21) and questions, PICO, and keywords were identified (ref 4) in collaboration with a librarian. Electronic databases, including PubMed (ref 14, ref 31), Cochrane, and EMBASE (ref 31) were searched. Galiano (ref 6) used Wiley Online Library, Elsevier, CINAHL, Health Source: Nursing/Academic Edition, PubMed, and the Cochrane Library while Hoke (ref 11) conducted an electronic search using CINAHL and PubMed to retrieve articles.

Identified journals were reviewed manually (ref 31). The findings were summarized using ‘elevator speech’ (ref 4). In a study by Gifford et al. (ref 9) evidence facilitators worked with participants to access, appraise, and adapt the research evidence to the organizational context. Ostaszkiewicz (ref 20) conducted a scoping review of literature and identified and reviewed frameworks and policy documents about the topic and the quality standards. Further, a team of nursing administrators, directors, staff nurses, and a patient representative reviewed the literature and made recommendations for practice changes.

Clinical practice guidelines were also used to offer scientific evidence (ref 7, ref 19). Evidence was further retrieved from a combination of nursing policies, guidelines, journal articles, and textbooks (ref 12) as well as from published guidelines and literature (ref 13). Internal evidence, professional practice knowledge, relevant theories and models were synthesized (ref 24) while other study (ref 25) reviewed individual studies, synthesized with systematic reviews or clinical practice guidelines. The team reviewed the research evidence (ref 3, ref 15) or conducted a literature review (ref 22, ref 28, ref 29), a literature search (ref 27), a systematic review (ref 23), a review of the literature (ref 30) or ‘the scholarly literature was reviewed’ (ref 18). In addition, ‘an extensive literature review of evidence-based best practices was carried out’ (ref 10). However, detailed description how the review was conducted was lacking.

Views of stakeholders

A total of 24 studies (77%) reported methods for how the views of stakeholders, i.e., professionals or experts, were considered. Support to run this study was received from nursing leadership and multidisciplinary teams (ref 29). Experts and stakeholders joined the study team in some cases (ref 25, ref 30), and in other studies, their opinions were sought to facilitate project success (ref 3). Sometimes a steering committee was formed by a Chief Nursing Officer and Clinical Practice Specialists (ref 2). More specifically, stakeholders’ views were considered using interviews, workshops and follow-up teleconferences (ref 7). The literature review was discussed with colleagues (ref 11), and feedback and support from physicians as well as the consensus of staff were sought (ref 16).

A summary of the project findings and suggestions for the studies were discussed at 90-minute weekly meetings by 11 charge nurses. Nurse executive directors were consulted over a 10-week period (ref 31). An implementation team (nurse, dietician, physiotherapist, occupational therapist) was formed to support the implementation of evidence-based prevention measures (ref 26). Stakeholders volunteered to join in the pilot implementation (ref 28) or a stakeholder team met to determine the best strategy for change management, shortcomings in evidence-based criteria were discussed, and strategies to address those areas were planned (ref 5). Nursing leaders, staff members (ref 22), ‘process owners (ref 18) and program team members (ref 18, ref 19, ref 24) met regularly to discuss the problems. Critical input was sought from clinical educators, physicians, nutritionists, pharmacists, and nurse managers (ref 24). The unit director and senior nursing staff reviewed the contents of the product, and the final version of clinical pathways were reviewed and approved by the Quality Control Commission of the Nursing Department (ref 12). In addition, two co-design workshops with 18 residential aged care stakeholders were organized to explore their perspectives about factors to include in a model prototype (ref 20). Further, an agreement of stakeholders in implementing continuous quality services within an open relationship was conducted (ref 1).

Critical appraisal

In five studies (16%), a critical appraisal targeting the literature search was carried out. The appraisals were conducted by interns and teams who critiqued the evidence (ref 4). In Hoke’s study, four areas that had emerged in the literature were critically reviewed (ref 11). Other methods were to ‘critically appraise the search results’ (ref 14). Journal club team meetings (ref 23) were organized to grade the level and quality of evidence and the team ‘critically appraised relevant evidence’ (ref 27). On the other hand, the studies lacked details of how the appraisals were done in each study.

The perceived effects of evidence-based leadership

Perceived effects of evidence-based leadership on nurses’ performance.

Eleven studies (35%) described perceived effects of evidence-based leadership on nurses’ performance (see Additional file 9 for perceived effects of evidence-based leadership), which were categorized in four groups: awareness and knowledge, competence, ability to understand patients’ needs, and engagement. First, regarding ‘awareness and knowledge’, different projects provided nurses with new learning opportunities (ref 3). Staff’s knowledge (ref 20, ref 28), skills, and education levels improved (ref 20), as did nurses’ knowledge comprehension (ref 21). Second, interventions and approaches focusing on management and leadership positively influenced participants’ competence level to improve the quality of services. Their confidence level (ref 1) and motivation to change practice increased, self-esteem improved, and they were more positive and enthusiastic in their work (ref 22). Third, some nurses were relieved that they had learned to better handle patients’ needs (ref 25). For example, a systematic work approach increased nurses’ awareness of the patients who were at risk of developing health problems (ref 26). And last, nurse leaders were more engaged with staff, encouraging them to adopt the new practices and recognizing their efforts to change (ref 8).

Perceived effects on organizational outcomes

Nine studies (29%) described the perceived effects of evidence-based leadership on organizational outcomes (see Additional file 9 for perceived effects of evidence-based leadership). These were categorized into three groups: use of resources, staff commitment, and team effort. First, more appropriate use of resources was reported (ref 15, ref 20), and working time was more efficiently used (ref 16). In generally, a structured approach made implementing change more manageable (ref 1). On the other hand, in the beginning of the change process, the feedback from nurses was unfavorable, and they experienced discomfort in the new work style (ref 29). New approaches were also perceived as time consuming (ref 3). Second, nurse leaders believed that fewer nursing staff than expected left the organization over the course of the study (ref 31). Third, the project helped staff in their efforts to make changes, and it validated the importance of working as a team (ref 7). Collaboration and support between the nurses increased (ref 26). On the other hand, new work style caused challenges in teamwork (ref 3).

Perceived effects on clinical outcomes

Five studies (16%) reported the perceived effects of evidence-based leadership on clinical outcomes (see Additional file 9 for perceived effects of evidence-based leadership), which were categorized in two groups: general patient outcomes and specific clinical outcomes. First, in general, the project assisted in connecting the guideline recommendations and patient outcomes (ref 7). The project was good for the patients in general, and especially to improve patient safety (ref 16). On the other hand, some nurses thought that the new working style did not work at all for patients (ref 28). Second, the new approach used assisted in optimizing patients’ clinical problems and person-centered care (ref 20). Bowel management, for example, received very good feedback (ref 30).

The measured effects of evidence-based leadership

The measured effects on nurses’ performance.

Data were obtained from 20 studies (65%) (see Additional file 10 for measured effects of evidence-based leadership) and categorized nurse performance outcomes for three groups: awareness and knowledge, engagement, and satisfaction. First, six studies (19%) measured the awareness and knowledge levels of participants. Internship for staff nurses was beneficial to help participants to understand the process for using evidence-based practice and to grow professionally, to stimulate for innovative thinking, to give knowledge needed to use evidence-based practice to answer clinical questions, and to make possible to complete an evidence-based practice project (ref 3). Regarding implementation program of evidence-based practice, those with formal EBP training showed an improvement in knowledge, attitude, confidence, awareness and application after intervention (ref 3, ref 11, ref 20, ref 23, ref 25). On the contrary, in other study, attitude towards EBP remained stable ( p  = 0.543). and those who applied EBP decreased although no significant differences over the years ( p  = 0.879) (ref 6).

Second, 10 studies (35%) described nurses’ engagement to new practices (ref 5, ref 6, ref 7, ref 10, ref 16, ref 17, ref 18, ref 21, ref 25, ref 27). 9 studies (29%) studies reported that there was an improvement of compliance level of participants (ref 6, ref 7, ref 10, ref 16, ref 17, ref 18, ref 21, ref 25, ref 27). On the contrary, in DeLeskey’s (ref 5) study, although improvement was found in post-operative nausea and vomiting’s (PONV) risk factors documented’ (2.5–63%), and ’risk factors communicated among anaesthesia and surgical staff’ (0–62%), the improvement did not achieve the goal. The reason was a limited improvement was analysed. It was noted that only those patients who had been seen by the pre-admission testing nurse had risk assessments completed. Appropriate treatment/prophylaxis increased from 69 to 77%, and from 30 to 49%; routine assessment for PONV/rescue treatment 97% and 100% was both at 100% following the project. The results were discussed with staff but further reasons for a lack of engagement in nursing care was not reported.

And third, six studies (19%) reported nurses’ satisfaction with project outcomes. The study results showed that using evidence in managerial decisions improved nurses’ satisfaction and attitudes toward their organization ( P  < 0.05) (ref 31). Nurses’ overall job satisfaction improved as well (ref 17). Nurses’ satisfaction with usability of the electronic charting system significantly improved after introduction of the intervention (ref 12). In handoff project in seven hospitals, improvement was reported in all satisfaction indicators used in the study although improvement level varied in different units (ref 28). In addition, positive changes were reported in nurses’ ability to autonomously perform their job (“How satisfied are you with the tools and resources available for you treat and prevent patient constipation?” (54%, n  = 17 vs. 92%, n  = 35, p  < 0.001) (ref 30).

The measured effects on organizational outcomes

Thirteen studies (42%) described the effects of a project on organizational outcomes (see Additional file 10 for measured effects of evidence-based leadership), which were categorized in two groups: staff compliance, and changes in practices. First, studies reported improved organizational outcomes due to staff better compliance in care (ref 4, ref 13, ref 17, ref 23, ref 27, ref 31). Second, changes in organization practices were also described (ref 11) like changes in patient documentation (ref 12, ref 21). Van Orne (ref 30) found a statistically significant reduction in the average rate of invasive medication administration between pre-intervention and post-intervention ( p  = 0.01). Salvador (ref 24) also reported an improvement in a proactive approach to mucositis prevention with an evidence-based oral care guide. On the contrary, concerns were also raised such as not enough time for new bedside report (ref 16) or a lack of improvement of assessment of diabetic ulcer (ref 8).

The measured effects on clinical outcomes

A variety of improvements in clinical outcomes were reported (see Additional file 10 for measured effects of evidence-based leadership): improvement in patient clinical status and satisfaction level. First, a variety of improvement in patient clinical status was reported. improvement in Incidence of CAUTI decreased 27.8% between 2015 and 2019 (ref 2) while a patient-centered quality improvement project reduced CAUTI rates to 0 (ref 10). A significant decrease in transmission rate of MRSA transmission was also reported (ref 27) and in other study incidences of CLABSIs dropped following of CHG bathing (ref 14). Further, it was possible to decrease patient nausea from 18 to 5% and vomiting to 0% (ref 5) while the percentage of patients who left the hospital without being seen was below 2% after the project (ref 17). In addition, a significant reduction in the prevalence of pressure ulcers was found (ref 26, ref 29) and a significant reduction of mucositis severity/distress was achieved (ref 24). Patient falls rate decreased (ref 15, ref 16, ref 19, ref 27).

Second, patient satisfaction level after project implementation improved (ref 28). The scale assessing healthcare providers by consumers showed improvement, but the changes were not statistically significant. Improvement in an emergency department leadership model and in methods of communication with patients improved patient satisfaction scores by 600% (ref 17). In addition, new evidence-based unit improved patient experiences about the unit although not all items improved significantly (ref 18).

Stakeholder involvement in the mixed-method review

To ensure stakeholders’ involvement in the review, the real-world relevance of our research [ 53 ], achieve a higher level of meaning in our review results, and gain new perspectives on our preliminary findings [ 50 ], a meeting with 11 stakeholders was organized. First, we asked if participants were aware of the concepts of evidence-based practice or evidence-based leadership. Responses revealed that participants were familiar with the concept of evidence-based practice, but the topic of evidence-based leadership was totally new. Examples of nurses and nurse leaders’ responses are as follows: “I have heard a concept of evidence-based practice but never a concept of evidence-based leadership.” Another participant described: “I have heard it [evidence-based leadership] but I do not understand what it means.”

Second, as stakeholder involvement is beneficial to the relevance and impact of health research [ 54 ], we asked how important evidence is to them in supporting decisions in health care services. One participant described as follows: “Using evidence in decisions is crucial to the wards and also to the entire hospital.” Third, we asked how the evidence-based approach is used in hospital settings. Participants expressed that literature is commonly used to solve clinical problems in patient care but not to solve leadership problems. “In [patient] medication and care, clinical guidelines are regularly used. However, I am aware only a few cases where evidence has been sought to solve leadership problems.”

And last, we asked what type of evidence is currently used to support nurse leaders’ decision making (e.g. scientific literature, organizational data, stakeholder views)? The participants were aware that different types of information were collected in their organization on a daily basis (e.g. patient satisfaction surveys). However, the information was seldom used to support decision making because nurse leaders did not know how to access this information. Even so, the participants agreed that the use of evidence from different sources was important in approaching any leadership or managerial problems in the organization. Participants also suggested that all nurse leaders should receive systematic training related to the topic; this could support the daily use of the evidence-based approach.

To our knowledge, this article represents the first mixed-methods systematic review to examine leadership problems, how evidence is used to solve these problems and what the perceived and measured effects of evidence-based leadership are on nurse leaders and their performance, organizational, and clinical outcomes. This review has two key findings. First, the available research data suggests that evidence-based leadership has potential in the healthcare context, not only to improve knowledge and skills among nurses, but also to improve organizational outcomes and the quality of patient care. Second, remarkably little published research was found to explore the effects of evidence-based leadership with an efficient trial design. We validated the preliminary results with nurse stakeholders, and confirmed that nursing staff, especially nurse leaders, were not familiar with the concept of evidence-based leadership, nor were they used to implementing evidence into their leadership decisions. Our data was based on many databases, and we screened a large number of studies. We also checked existing registers and databases and found no registered or ongoing similar reviews being conducted. Therefore, our results may not change in the near future.

We found that after identifying the leadership problems, 26 (84%) studies out of 31 used organizational data, 25 (81%) studies used scientific evidence from the literature, and 21 (68%) studies considered the views of stakeholders in attempting to understand specific leadership problems more deeply. However, only four studies critically appraised any of these findings. Considering previous critical statements of nurse leaders’ use of evidence in their decision making [ 14 , 30 , 31 , 34 , 55 ], our results are still quite promising.

Our results support a previous systematic review by Geert et al. [ 32 ], which concluded that it is possible to improve leaders’ individual-level outcomes, such as knowledge, motivation, skills, and behavior change using evidence-based approaches. Collins and Holton [ 23 ] particularly found that leadership training resulted in significant knowledge and skill improvements, although the effects varied widely across studies. In our study, evidence-based leadership was seen to enable changes in clinical practice, especially in patient care. On the other hand, we understand that not all efforts to changes were successful [ 56 , 57 , 58 ]. An evidence-based approach causes negative attitudes and feelings. Negative emotions in participants have also been reported due to changes, such as discomfort with a new working style [ 59 ]. Another study reported inconvenience in using a new intervention and its potential risks for patient confidentiality. Sometimes making changes is more time consuming than continuing with current practice [ 60 ]. These findings may partially explain why new interventions or program do not always fully achieve their goals. On the other hand, Dubose et al. [ 61 ] state that, if prepared with knowledge of resistance, nurse leaders could minimize the potential negative consequences and capitalize on a powerful impact of change adaptation.

We found that only six studies used a specific model or theory to understand the mechanism of change that could guide leadership practices. Participants’ reactions to new approaches may be an important factor in predicting how a new intervention will be implemented into clinical practice. Therefore, stronger effort should be put to better understanding the use of evidence, how participants’ reactions and emotions or practice changes could be predicted or supported using appropriate models or theories, and how using these models are linked with leadership outcomes. In this task, nurse leaders have an important role. At the same time, more responsibilities in developing health services have been put on the shoulders of nurse leaders who may already be suffering under pressure and increased burden at work. Working in a leadership position may also lead to role conflict. A study by Lalleman et al. [ 62 ] found that nurses were used to helping other people, often in ad hoc situations. The helping attitude of nurses combined with structured managerial role may cause dilemmas, which may lead to stress. Many nurse leaders opt to leave their positions less than 5 years [ 63 ].To better fulfill the requirements of health services in the future, the role of nurse leaders in evidence-based leadership needs to be developed further to avoid ethical and practical dilemmas in their leadership practices.

It is worth noting that the perceived and measured effects did not offer strong support to each other but rather opened a new venue to understand the evidence-based leadership. Specifically, the perceived effects did not support to measured effects (competence, ability to understand patients’ needs, use of resources, team effort, and specific clinical outcomes) while the measured effects could not support to perceived effects (nurse’s performance satisfaction, changes in practices, and clinical outcomes satisfaction). These findings may indicate that different outcomes appear if the effects of evidence-based leadership are looked at using different methodological approach. Future study is encouraged using well-designed study method including mixed-method study to examine the consistency between perceived and measured effects of evidence-based leadership in health care.

There is a potential in nursing to support change by demonstrating conceptual and operational commitment to research-based practices [ 64 ]. Nurse leaders are well positioned to influence and lead professional governance, quality improvement, service transformation, change and shared governance [ 65 ]. In this task, evidence-based leadership could be a key in solving deficiencies in the quality, safety of care [ 14 ] and inefficiencies in healthcare delivery [ 12 , 13 ]. As WHO has revealed, there are about 28 million nurses worldwide, and the demand of nurses will put nurse resources into the specific spotlight [ 1 ]. Indeed, evidence could be used to find solutions for how to solve economic deficits or other problems using leadership skills. This is important as, when nurses are able to show leadership and control in their own work, they are less likely to leave their jobs [ 66 ]. On the other hand, based on our discussions with stakeholders, nurse leaders are not used to using evidence in their own work. Further, evidence-based leadership is not possible if nurse leaders do not have access to a relevant, robust body of evidence, adequate funding, resources, and organizational support, and evidence-informed decision making may only offer short-term solutions [ 55 ]. We still believe that implementing evidence-based strategies into the work of nurse leaders may create opportunities to protect this critical workforce from burnout or leaving the field [ 67 ]. However, the role of the evidence-based approach for nurse leaders in solving these problems is still a key question.

Limitations

This study aimed to use a broad search strategy to ensure a comprehensive review but, nevertheless, limitations exist: we may have missed studies not included in the major international databases. To keep search results manageable, we did not use specific databases to systematically search grey literature although it is a rich source of evidence used in systematic reviews and meta-analysis [ 68 ]. We still included published conference abstract/proceedings, which appeared in our scientific databases. It has been stated that conference abstracts and proceedings with empirical study results make up a great part of studies cited in systematic reviews [ 69 ]. At the same time, a limited space reserved for published conference publications can lead to methodological issues reducing the validity of the review results [ 68 ]. We also found that the great number of studies were carried out in western countries, restricting the generalizability of the results outside of English language countries. The study interventions and outcomes were too different across studies to be meaningfully pooled using statistical methods. Thus, our narrative synthesis could hypothetically be biased. To increase transparency of the data and all decisions made, the data, its categorization and conclusions are based on original studies and presented in separate tables and can be found in Additional files. Regarding a methodological approach [ 34 ], we used a mixed methods systematic review, with the core intention of combining quantitative and qualitative data from primary studies. The aim was to create a breadth and depth of understanding that could confirm to or dispute evidence and ultimately answer the review question posed [ 34 , 70 ]. Although the method is gaining traction due to its usefulness and practicality, guidance in combining quantitative and qualitative data in mixed methods systematic reviews is still limited at the theoretical stage [ 40 ]. As an outcome, it could be argued that other methodologies, for example, an integrative review, could have been used in our review to combine diverse methodologies [ 71 ]. We still believe that the results of this mixed method review may have an added value when compared with previous systematic reviews concerning leadership and an evidence-based approach.

Our mixed methods review fills the gap regarding how nurse leaders themselves use evidence to guide their leadership role and what the measured and perceived impact of evidence-based leadership is in nursing. Although the scarcity of controlled studies on this topic is concerning, the available research data suggest that evidence-based leadership intervention can improve nurse performance, organizational outcomes, and patient outcomes. Leadership problems are also well recognized in healthcare settings. More knowledge and a deeper understanding of the role of nurse leaders, and how they can use evidence in their own managerial leadership decisions, is still needed. Despite the limited number of studies, we assume that this narrative synthesis can provide a good foundation for how to develop evidence-based leadership in the future.

Implications

Based on our review results, several implications can be recommended. First, the future of nursing success depends on knowledgeable, capable, and strong leaders. Therefore, nurse leaders worldwide need to be educated about the best ways to manage challenging situations in healthcare contexts using an evidence-based approach in their decisions. This recommendation was also proposed by nurses and nurse leaders during our discussion meeting with stakeholders.

Second, curriculums in educational organizations and on-the-job training for nurse leaders should be updated to support general understanding how to use evidence in leadership decisions. And third, patients and family members should be more involved in the evidence-based approach. It is therefore important that nurse leaders learn how patients’ and family members’ views as stakeholders are better considered as part of the evidence-based leadership approach.

Future studies should be prioritized as follows: establishment of clear parameters for what constitutes and measures evidence-based leadership; use of theories or models in research to inform mechanisms how to effectively change the practice; conducting robust effectiveness studies using trial designs to evaluate the impact of evidence-based leadership; studying the role of patient and family members in improving the quality of clinical care; and investigating the financial impact of the use of evidence-based leadership approach within respective healthcare systems.

Data availability

The authors obtained all data for this review from published manuscripts.

World Health Organization. State of the world’s nursing 2020: investing in education, jobs and leadership. 2020. https://www.who.int/publications/i/item/9789240003279 . Accessed 29 June 2024.

Hersey P, Campbell R. Leadership: a behavioral science approach. The Center for; 2004.

Cline D, Crenshaw JT, Woods S. Nurse leader: a definition for the 21st century. Nurse Lead. 2022;20(4):381–4. https://doi.org/10.1016/j.mnl.2021.12.017 .

Article   Google Scholar  

Chen SS. Leadership styles and organization structural configurations. J Hum Resource Adult Learn. 2006;2(2):39–46.

Google Scholar  

McKibben L. Conflict management: importance and implications. Br J Nurs. 2017;26(2):100–3.

Article   PubMed   Google Scholar  

Haghgoshayie E, Hasanpoor E. Evidence-based nursing management: basing Organizational practices on the best available evidence. Creat Nurs. 2021;27(2):94–7. https://doi.org/10.1891/CRNR-D-19-00080 .

Majers JS, Warshawsky N. Evidence-based decision-making for nurse leaders. Nurse Lead. 2020;18(5):471–5.

Tichy NM, Bennis WG. Making judgment calls. Harvard Business Rev. 2007;85(10):94.

Sousa MJ, Pesqueira AM, Lemos C, Sousa M, Rocha Á. Decision-making based on big data analytics for people management in healthcare organizations. J Med Syst. 2019;43(9):1–10.

Guo R, Berkshire SD, Fulton LV, Hermanson PM. %J L in HS. Use of evidence-based management in healthcare administration decision-making. 2017;30(3): 330–42.

Liang Z, Howard P, Rasa J. Evidence-informed managerial decision-making: what evidence counts?(part one). Asia Pac J Health Manage. 2011;6(1):23–9.

Hasanpoor E, Janati A, Arab-Zozani M, Haghgoshayie E. Using the evidence-based medicine and evidence-based management to minimise overuse and maximise quality in healthcare: a hybrid perspective. BMJ evidence-based Med. 2020;25(1):3–5.

Shingler NA, Gonzalez JZ. Ebm: a pathway to evidence-based nursing management. Nurs 2022. 2017;47(2):43–6.

Farokhzadian J, Nayeri ND, Borhani F, Zare MR. Nurse leaders’ attitudes, self-efficacy and training needs for implementing evidence-based practice: is it time for a change toward safe care? Br J Med Med Res. 2015;7(8):662.

Article   PubMed   PubMed Central   Google Scholar  

American Nurses Association. ANA leadership competency model. Silver Spring, MD; 2018.

Royal College of Nursing. Leadership skills. 2022. https://www.rcn.org.uk/professional-development/your-career/nurse/leadership-skills . Accessed 29 June 2024.

Kakemam E, Liang Z, Janati A, Arab-Zozani M, Mohaghegh B, Gholizadeh M. Leadership and management competencies for hospital managers: a systematic review and best-fit framework synthesis. J Healthc Leadersh. 2020;12:59.

Liang Z, Howard PF, Leggat S, Bartram T. Development and validation of health service management competencies. J Health Organ Manag. 2018;32(2):157–75.

World Health Organization. Global Strategic Directions for Nursing and Midwifery. 2021. https://apps.who.int/iris/bitstream/handle/10665/344562/9789240033863-eng.pdf . Accessed 29 June 2024.

NHS Leadership Academy. The nine leadership dimensions. 2022. https://www.leadershipacademy.nhs.uk/resources/healthcare-leadership-model/nine-leadership-dimensions/ . Accessed 29 June 2024.

Canadian Nurses Association. Evidence-informed decision-making and nursing practice: Position statement. 2018. https://hl-prod-ca-oc-download.s3-ca-central-1.amazonaws.com/CNA/2f975e7e-4a40-45ca-863c-5ebf0a138d5e/UploadedImages/documents/Evidence_informed_Decision_making_and_Nursing_Practice_position_statement_Dec_2018.pdf . Accessed 29 June 2024.

Hasanpoor E, Hajebrahimi S, Janati A, Abedini Z, Haghgoshayie E. Barriers, facilitators, process and sources of evidence for evidence-based management among health care managers: a qualitative systematic review. Ethiop J Health Sci. 2018;28(5):665–80.

PubMed   PubMed Central   Google Scholar  

Collins DB, Holton EF III. The effectiveness of managerial leadership development programs: a meta-analysis of studies from 1982 to 2001. Hum Res Dev Q. 2004;15(2):217–48.

Cummings GG, Lee S, Tate K, Penconek T, Micaroni SP, Paananen T, et al. The essentials of nursing leadership: a systematic review of factors and educational interventions influencing nursing leadership. Int J Nurs Stud. 2021;115:103842.

Clavijo-Chamorro MZ, Romero-Zarallo G, Gómez-Luque A, López-Espuela F, Sanz-Martos S, López-Medina IM. Leadership as a facilitator of evidence implementation by nurse managers: a metasynthesis. West J Nurs Res. 2022;44(6):567–81.

Young SK. Evidence-based management: a literature review. J Nurs Adm Manag. 2002;10(3):145–51.

Williams LL. What goes around comes around: evidence-based management. Nurs Adm Q. 2006;30(3):243–51.

Fraser I. Organizational research with impact: working backwards. Worldviews Evidence-Based Nurs. 2004;1:S52–9.

Roshanghalb A, Lettieri E, Aloini D, Cannavacciuolo L, Gitto S, Visintin F. What evidence on evidence-based management in healthcare? Manag Decis. 2018;56(10):2069–84.

Jaana M, Vartak S, Ward MM. Evidence-based health care management: what is the research evidence available for health care managers? Eval Health Prof. 2014;37(3):314–34.

Tate K, Hewko S, McLane P, Baxter P, Perry K, Armijo-Olivo S, et al. Learning to lead: a review and synthesis of literature examining health care managers’ use of knowledge. J Health Serv Res Policy. 2019;24(1):57–70.

Geerts JM, Goodall AH, Agius S, %J SS. Medicine. Evidence-based leadership development for physicians: a systematic literature review. 2020;246: 112709.

Barends E, Rousseau DM, Briner RB. Evidence-based management: The basic principles. Amsterdam; 2014. https://research.vu.nl/ws/portalfiles/portal/42141986/complete+dissertation.pdf#page=203 . Accessed 29 June 2024.

Stern C, Lizarondo L, Carrier J, Godfrey C, Rieger K, Salmond S, et al. Methodological guidance for the conduct of mixed methods systematic reviews. JBI Evid Synthesis. 2020;18(10):2108–18. https://doi.org/10.11124/JBISRIR-D-19-00169 .

Lancet T. 2020: unleashing the full potential of nursing. Lancet (London, England). 2019. p. 1879.

Välimäki MA, Lantta T, Hipp K, Varpula J, Liu G, Tang Y, et al. Measured and perceived impacts of evidence-based leadership in nursing: a mixed-methods systematic review protocol. BMJ Open. 2021;11(10):e055356. https://doi.org/10.1136/bmjopen-2021-055356 .

The Joanna Briggs Institute. Joanna Briggs Institute reviewers’ manual: 2014 edition. Joanna Briggs Inst. 2014; 88–91.

Pearson A, White H, Bath-Hextall F, Salmond S, Apostolo J, Kirkpatrick P. A mixed-methods approach to systematic reviews. JBI Evid Implement. 2015;13(3):121–31.

Johnson RB, Onwuegbuzie AJ. Mixed methods research: a research paradigm whose time has come. Educational Researcher. 2004;33(7):14–26.

Hong, Pluye P, Bujold M, Wassef M. Convergent and sequential synthesis designs: implications for conducting and reporting systematic reviews of qualitative and quantitative evidence. Syst Reviews. 2017;6(1):61. https://doi.org/10.1186/s13643-017-0454-2 .

Ramis MA, Chang A, Conway A, Lim D, Munday J, Nissen L. Theory-based strategies for teaching evidence-based practice to undergraduate health students: a systematic review. BMC Med Educ. 2019;19(1):1–13.

Sackett DL, Rosenberg WM, Gray JM, Haynes RB, Richardson WS. Evidence based medicine: what it is and what it isn’t. Bmj. British Medical Journal Publishing Group; 1996. pp. 71–2.

Goodman JS, Gary MS, Wood RE. Bibliographic search training for evidence-based management education: a review of relevant literatures. Acad Manage Learn Educ. 2014;13(3):322–53.

Aromataris E, Munn Z. Chapter 3: Systematic reviews of effectiveness. JBI Manual for Evidence Synthesis. 2020; https://synthesismanual.jbi.global .

Munn Z, Barker TH, Moola S, Tufanaru C, Stern C, McArthur A et al. Methodological quality of case series studies: an introduction to the JBI critical appraisal tool. 2020;18(10): 2127–33.

Hong Q, Pluye P, Fàbregues S, Bartlett G, Boardman F, Cargo M, et al. Mixed methods Appraisal Tool (MMAT) Version 2018: user guide. Montreal: McGill University; 2018.

McKenna J, Jeske D. Ethical leadership and decision authority effects on nurses’ engagement, exhaustion, and turnover intention. J Adv Nurs. 2021;77(1):198–206.

Maxwell M, Hibberd C, Aitchison P, Calveley E, Pratt R, Dougall N, et al. The TIDieR (template for intervention description and replication) checklist. The patient Centred Assessment Method for improving nurse-led biopsychosocial assessment of patients with long-term conditions: a feasibility RCT. NIHR Journals Library; 2018.

Braun V, Clarke V. Using thematic analysis in psychology. Qualitative Res Psychol. 2006;3(2):77–101.

Pollock A, Campbell P, Struthers C, Synnot A, Nunn J, Hill S, et al. Stakeholder involvement in systematic reviews: a scoping review. Syst Reviews. 2018;7:1–26.

Braye S, Preston-Shoot M. Emerging from out of the shadows? Service user and carer involvement in systematic reviews. Evid Policy. 2005;1(2):173–93.

Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. Syst Reviews. 2021;10(1):1–11.

Porta M. Pilot investigation, study. A dictionary of epidemiology. Oxford University Press Oxford; 2014. p. 215.

Kreis J, Puhan MA, Schünemann HJ, Dickersin K. Consumer involvement in systematic reviews of comparative effectiveness research. Health Expect. 2013;16(4):323–37.

Joseph ML, Nelson-Brantley HV, Caramanica L, Lyman B, Frank B, Hand MW, et al. Building the science to guide nursing administration and leadership decision making. JONA: J Nurs Adm. 2022;52(1):19–26.

Gifford W, Davies BL, Graham ID, Tourangeau A, Woodend AK, Lefebre N. Developing Leadership Capacity for Guideline Use: a pilot cluster Randomized Control Trial: Leadership Pilot Study. Worldviews Evidence-Based Nurs. 2013;10(1):51–65. https://doi.org/10.1111/j.1741-6787.2012.00254.x .

Hsieh HY, Henker R, Ren D, Chien WY, Chang JP, Chen L, et al. Improving effectiveness and satisfaction of an electronic charting system in Taiwan. Clin Nurse Specialist. 2016;30(6):E1–6. https://doi.org/10.1097/NUR.0000000000000250 .

McAllen E, Stephens K, Swanson-Biearman B, Kerr K, Whiteman K. Moving Shift Report to the Bedside: an evidence-based Quality Improvement Project. OJIN: Online J Issues Nurs. 2018;23(2). https://doi.org/10.3912/OJIN.Vol23No02PPT22 .

Thomas M, Autencio K, Cesario K. Positive outcomes of an evidence-based pressure injury prevention program. J Wound Ostomy Cont Nurs. 2020;47:S24.

Cullen L, Titler MG. Promoting evidence-based practice: an internship for Staff nurses. Worldviews Evidence-Based Nurs. 2004;1(4):215–23. https://doi.org/10.1111/j.1524-475X.2004.04027.x .

DuBose BM, Mayo AM. Resistance to change: a concept analysis. Nursing forum. Wiley Online Library; 2020. pp. 631–6.

Lalleman PCB, Smid GAC, Lagerwey MD, Shortridge-Baggett LM, Schuurmans MJ. Curbing the urge to care: a bourdieusian analysis of the effect of the caring disposition on nurse middle managers’ clinical leadership in patient safety practices. Int J Nurs Stud. 2016;63:179–88.

Article   CAS   PubMed   Google Scholar  

Martin E, Warshawsky N. Guiding principles for creating value and meaning for the next generation of nurse leaders. JONA: J Nurs Adm. 2017;47(9):418–20.

Griffiths P, Recio-Saucedo A, Dall’Ora C, Briggs J, Maruotti A, Meredith P, et al. The association between nurse staffing and omissions in nursing care: a systematic review. J Adv Nurs. 2018;74(7):1474–87. https://doi.org/10.1111/jan.13564 .

Lúanaigh PÓ, Hughes F. The nurse executive role in quality and high performing health services. J Nurs Adm Manag. 2016;24(1):132–6.

de Kok E, Weggelaar-Jansen AM, Schoonhoven L, Lalleman P. A scoping review of rebel nurse leadership: descriptions, competences and stimulating/hindering factors. J Clin Nurs. 2021;30(17–18):2563–83.

Warshawsky NE. Building nurse manager well-being by reducing healthcare system demands. JONA: J Nurs Adm. 2022;52(4):189–91.

Paez A. Gray literature: an important resource in systematic reviews. J Evidence-Based Med. 2017;10(3):233–40.

McAuley L, Tugwell P, Moher D. Does the inclusion of grey literature influence estimates of intervention effectiveness reported in meta-analyses? Lancet. 2000;356(9237):1228–31.

Sarah S. Introduction to mixed methods systematic reviews. https://jbi-global-wiki.refined.site/space/MANUAL/4689215/8.1+Introduction+to+mixed+methods+systematic+reviews . Accessed 29 June 2024.

Whittemore R, Knafl K. The integrative review: updated methodology. J Adv Nurs. 2005;52(5):546–53.

Download references

Acknowledgements

We want to thank the funding bodies, the Finnish National Agency of Education, Asia Programme, the Department of Nursing Science at the University of Turku, and Xiangya School of Nursing at the Central South University. We also would like to thank the nurses and nurse leaders for their valuable opinions on the topic.

The work was supported by the Finnish National Agency of Education, Asia Programme (grant number 26/270/2020) and the University of Turku (internal fund 26003424). The funders had no role in the study design and will not have any role during its execution, analysis, interpretation of the data, decision to publish, or preparation of the manuscript.

Author information

Authors and affiliations.

Department of Nursing Science, University of Turku, Turku, FI-20014, Finland

Maritta Välimäki, Tella Lantta, Kirsi Hipp & Jaakko Varpula

School of Public Health, University of Helsinki, Helsinki, FI-00014, Finland

Maritta Välimäki

Xiangya Nursing, School of Central South University, Changsha, 410013, China

Shuang Hu, Jiarui Chen, Yao Tang, Wenjun Chen & Xianhong Li

School of Health and Social Services, Häme University of Applied Sciences, Hämeenlinna, Finland

Hunan Cancer Hospital, Changsha, 410008, China

Gaoming Liu

You can also search for this author in PubMed   Google Scholar

Contributions

Study design: MV, XL. Literature search and study selection: MV, KH, TL, WC, XL. Quality assessment: YT, SH, XL. Data extraction: JC, MV, JV, WC, YT, SH, GL. Analysis and interpretation: MV, SH. Manuscript writing: MV. Critical revisions for important intellectual content: MV, XL. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Xianhong Li .

Ethics declarations

Ethics approval and consent to participate.

No ethical approval was required for this study.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Differences between the original protocol

We modified criteria for the included studies: we included published conference abstracts/proceedings, which form a relatively broad knowledge base in scientific knowledge. We originally planned to conduct a survey with open-ended questions followed by a face-to-face meeting to discuss the preliminary results of the review. However, to avoid extra burden in nurses due to COVID-19, we decided to limit the validation process to the online discussion only.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary Material 1

Supplementary material 2, supplementary material 3, supplementary material 4, supplementary material 5, supplementary material 6, supplementary material 7, supplementary material 8, supplementary material 9, supplementary material 10, rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Välimäki, M., Hu, S., Lantta, T. et al. The impact of evidence-based nursing leadership in healthcare settings: a mixed methods systematic review. BMC Nurs 23 , 452 (2024). https://doi.org/10.1186/s12912-024-02096-4

Download citation

Received : 28 April 2023

Accepted : 13 June 2024

Published : 03 July 2024

DOI : https://doi.org/10.1186/s12912-024-02096-4

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Evidence-based leadership
• Health services administration
• Organizational development
• Quality in healthcare

BMC Nursing

ISSN: 1472-6955

• General enquiries: [email protected]

• Open access
• Published: 03 July 2024

Exploring medical students' experience of the learning environment: a mixed methods study in Saudi medical college

• Mohammed Almansour 1 ,
• Noura Abouammoh 2 ,
• Reem Bin Idris 3 ,
• Omar Abdullatif Alsuliman 3 ,
• Renad Abdulrahman Alhomaidi 3 ,
• Mohammed Hamad Alhumud 3 &
• Hani A. Alghamdi 2  

BMC Medical Education volume  24 , Article number:  723 ( 2024 ) Cite this article

288 Accesses

5 Altmetric

Metrics details

In medical education, the learning environment (LE) significantly impacts students' professionalism and academic performance. Positive LE perceptions are linked to better academic outcomes. Our study, which was conducted 15 years after curriculum reform at King Saud University's College of Medicine, aimed to explore students' perspectives on their LE and identify areas for improvement. By understanding their experiences, we strive to enhance LE and promote academic success.

This mixed-method study employed an explanatory sequential approach in which a cross-sectional analytical survey phase was collected first using the Johns Hopkins Learning Environment Scale (JHLES), followed by qualitative focus groups. Findings from quantitative and qualitative methods were integrated using joint display.

A total of 653 medical students completed the JHLES. The total average score was 81 out of 140 (16.8), and the average subscale scores ranged from 2.27 (0.95) for inclusion and safety to 3.37 (0.91) for community of peers. The qualitative approach encompasses both inductive and deductive analyses, identifying overarching themes comprising proudness, high expectations and competition, and views about the curriculum. The integration of results emphasizes the need for continued efforts to create a supportive and inclusive LE that positively influences students' experiences and academic success.

This research offers valuable insights for educational institutions seeking to enhance medical education quality and support systems. Recommendations include faculty development, the cultivation of supportive environments, curriculum revision, improved mentorship programs, and initiatives to promote inclusivity and gender equity. Future research should explore longitudinal and comparative studies, innovative mixed methods approaches, and interventions to further optimize medical education experiences. Overall, this study contributes to the ongoing dialog on medical education, offering a nuanced understanding of the complex factors influencing students' perceptions and suggesting actionable strategies for improvement.

Peer Review reports

The learning environment of medical students plays a significant role in shaping qualified, well-rounded physicians. It can also impact students' professionalism, ethics, and morals. As these students graduate and begin their professional practice, their competency can be a direct reflection of the medical institutes from which they graduated. The learning environment (LE) is a term used to describe the physical, cultural, and psychosocial climate in which learning takes place [ 1 ]. Students' skills, knowledge, and attitudes are influenced by the teaching and learning environment of their educational institutes. The interactions they have with their peers, faculty members, and administrators play a role in their learning environment. The curriculum that is taught to students is part of this environment, and the curriculum's design is a vital component [ 2 ].

The impact of LE on the academic performance of medical students is significant. Therefore, it is crucial to provide a supportive environment that positively influences students' perceptions of their LE. Research has consistently shown that students who perceive their LE to be positive and supportive are more likely to perform well academically [ 3 ]. Conversely, students who perceive their LE to be negative may experience adverse effects on their academic performance [ 3 ].

A student-centered curriculum of outstanding standards must be provided, and evaluation of the educational setting at both academic and clinical sites is essential [ 4 ]. King Saud University's College of Medicine program is seven years long, starting with a preparatory year, followed by two basic sciences (preclinical) years, then three clinical-practice years, and a one-year internship. The program employs a combination of problem-based learning and interactive lecturing to teach medical and healthcare-related sciences, emphasizing critical thinking and self-directed learning. Clinical training programs provide hands-on experience, with the goal of producing skilled and compassionate healthcare professionals.

Two studies were conducted at the College of Medicine at King Saud University (COM-KSU). The first study was conducted in 2008, prior to the college's curriculum reform in 2009, which transitioned from a traditional to a system-oriented hybrid curriculum [ 5 ]. Researchers utilized the Dundee Ready Educational Environment Measure (DREEM) scale to evaluate the learning environment (LE), and the results indicated that first-year students had significantly higher scores than other students [ 5 ]. Additionally, preclinical students had significantly greater scores than did clinical students, and gender was not a statistically significant factor [ 5 ].

The second study was conducted in 2014, where fifth-year medical students were evaluated using the DREEM scale to assess their perception of the LE [ 6 ]. The study revealed that the students' perception of the educational environment was satisfactory [ 6 ].

The Johns Hopkins Learning Environment Scale (JHLES) was created by the Johns Hopkins University School of Medicine to evaluate the quality of the learning environment for residents and medical students [ 7 ]. The 28-item scale helps medical educators identify areas of improvement by assessing seven factors or subscales, comprising community of peers, faculty relationships, academic climate, meaningful engagement, mentoring, inclusion and safety, and physical space [ 7 ].

The aim of our study was to investigate the perceptions of medical students regarding their LE at the COM-KSU 15 years after the curriculum was reformed. We seek to understand the experiences of students in this particular LE and gain insights into the factors that influence their perceptions of the LE. By exploring the students' perspectives, we aim to identify areas where improvements can be made to enhance LE and ensure that it is conducive to learning and promotes academic success.

Aim, design, and setting

This mixed-method study aimed to investigate students’ perceptions of the LE at COM-KSU 15 year proceeding a curriculum change, followed by an exploration of their perspectives aiming to identify areas of improvement of the LE. This study employed an explanatory sequential approach in which a cross-sectional analytical survey phase collected first, followed by qualitative focus groups. The research was carried out between November 2022 and March 2023 within the College of Medicine at King Saud University (COM-KSU), which is the pioneering medical education institution in the Kingdom of Saudi Arabia and is located in the capital city of Riyadh.

Participants and sampling

All the COM-KSU undergraduate students and interns were invited to participate in the study, with a total of 1471 students and 268 interns. The total number of enumeration techniques over the period of the study was used. Convenient sampling was employed in this study. The decision to employ convenient sampling was based on practical considerations of the accessibility and availability of participants. Consequently, a total of 653 individuals voluntarily participated in the first phase of the study, and the research team initiated the participant recruitment process by extending invitations to all undergraduate students and interns enrolled in the COM-KSU. The invitations were disseminated via multiple channels, including email, WhatsApp groups, and personal visits to each classroom within the college.

The data collection process comprised two distinct online surveys, each serving a specific purpose. The first survey focused on the quantitative phase and included questions related to demographic information and the Johns Hopkins Learning Environment Scale (JHLES). The second survey, designed for registration in the qualitative phase, included demographic inquiries along with a means of contact and the provision of available time slots. Subsequently, the research team communicated with the registered participants and arranged for focused group discussions (FGDs) to be conducted. Two FGDs were needed (5 and 7 participants) based on the theory of data saturation. Each FGD lasted approximately 70 min and was held at the College of Medicine. The discussions were facilitated by one of the authors, who is a qualitative methodologist and a faculty member at the same college, and the participants were comfortable discussing negative views as they were discussing positive views.

In the quantitative study phase, an online survey encompassing various components was developed. This survey collected demographic data, including information on gender, age, academic year, GPA, employment status, marital status, and residence type. Additionally, the Johns Hopkins Learning Environment Scale (JHLES), a validated tool used for assessing undergraduate medical school learning environments, was used. The JHLES consists of 28 items distributed across seven domains, and its use for this study was conducted without the need for direct permission, as it is publicly available.

In the qualitative study phase, students and interns were actively engaged in Focus Group Discussions (FGDs), aimed at eliciting their perspectives on the learning environment (LE). The FGDs employed a topic guide comprising open-ended questions aligned with the LE domains delineated by the JHLES. These questions included inquiries such as "How would you characterize your relationships with your peers?" and "To what extent does the college environment support collaboration with fellow students from the same college?" Furthermore, participants were asked to share their opinions regarding the faculty and provide insights into their perceptions of the curriculum. The FGDs were complemented with probing questions and follow-up queries to delve deeper into participants' experiences and perspectives.

Statistical analysis

For the first phase in this study, sociodemographic data were presented using descriptive statistics. The mean and standard deviation (SD) for the total score and the seven domains of the JHLES were calculated. Cross-tabulation was used to explore the relationships between the JHLES scores and the sociodemographic variables, and tests of significance through chi-square tests and ANOVA were performed. All analyses were performed using R (version 4.2.2), [ 8 ].

Qualitative data collection

The questions in the topic guide included probing questions and encompassed domains and questions from the JHLES. As open-ended questions were used to collect data, themes included deductive and inductive analysis. Inductive analysis was based on a priori themes based on the JHLES domains.

Qualitative analysis

Thematic analysis was adopted for qualitative analysis. This approach was proposed by Ritchie and Spencer (1994) to be helpful in providing a sequential structure for data analysis. This was conducted using NVivo software version 11.4.2. Using software increases the efficiency of data organization and retrieval. Familiarization, descriptive coding, basic analysis, and interpretation are the steps followed in the data analysis, and quotes from the participants were used to support the themes. Analyzing the data and identifying common descriptive themes were tasks shared with the team. The team agreed on a coding frame. The analysis was conducted independently, and the results are presented in comparison to the quantitative findings in Table  5 .

Mixed methods integration

Findings from quantitative and qualitative methods were integrated using joint display. The outcomes of the JHLES and FGDs were compared side-by-side. Integrating findings can create a holistic understanding of the learning environment of the College of Medicine, leading to a conclusion where the whole is greater than the sum of its parts.

Joint display of the data provided visual means of presenting qualitative and quantitative findings granting the ability to associate reasoning with different item score. Qualitative findings complement the quantitative findings in providing meaning to the score and explored in students’ perspective reasons for these scores. The qualitative findings also explained how students’ pride and perception about their own status reflect on the JHLES score. Students’ needs and preferences were expressed explicitly during the FGDs.

Ethical considerations

This study was approved by King Saud University’s Institutional Review Board (KSU IRB) with the approval number E-22–7298. Electronic informed consent was obtained from all participants in the quantitative arm, and written informed consent was obtained from all participants in the qualitative arm prior to their participation in the study.

Quantitative results

Sociodemographic characteristics.

Table 1 presents the sociodemographic characteristics of all participants. The total number of medical students and interns included in this phase of the study was 653. Of those studied, there was an almost equal gender distribution, with males making up slightly more than half (59%). There were relatively varied numbers of academic years, with less than average representation coming from the intern level at a participation rate of only 4%, while the highest engagement occurred during fourth-year studies at approximately 26%. Most individuals boasted high academic records, achieving an above-average GPA of 4.50–5.00 (65.7%). Of those who participated, a small fraction had lower grades below a GPA of 4 (11.5%). The majority of the participants were unemployed (96.2%), while less than 4% were either employed (full- or part-time) or freelancers (1.5%). Regarding personal life traits, most of the participants were single (98.5%) and lived with their families residing in Riyadh (93%).

As shown in Table  2 , the overall mean score for student experience was 81 ( SD  = 16.76). Among the specific subscales, the highest mean score was observed for physical space (3.52; SD  = 0.95), and the lowest mean score was found for inclusion and safety (2.27; SD  = 0.95).

Sociodemographic variables and overall and domain scores of the JHLES (mean and SD)

Associations between sociodemographic variables and the different domains of the JHLES as well as the overall score are represented in Table  3 . Male students reported a higher mean overall score than females did (83.4 ± 17.1 and 77.5 ± 15.7, respectively). As the number of academic years increased, the first-year students reported a greater average score than did the senior-year students, with a mean overall score for first-year medical students of 87.6 ( SD  = 16.9), whereas the average score for senior-year students (fifth-year) was 74.8 ± 18.2. Students who possessed higher GPAs (4.50–5.00) achieved the highest mean score of 82.2 ± 16, while those with GPAs less than 4.00 reported the lowest average score of 73.3 ( SD  = 15). Employment status was another variable impacting students' individual perceptions of this survey total score, where employed students generally outperformed unemployed students, with higher scores (88.6 ± 18.6) than unemployed students (80.7 ± 16.7). A significant association was observed between the overall JHELS score and gender, academic year, and GPA at the 0.05 level.

There was a notable difference in scores between males and females, with males reporting higher scores for all domains except “inclusion and safety”. Intriguingly, both genders reflected a similar pattern for reporting the highest score for physical space and the lowest for inclusion and safety. However, four domains showed statistically significant associations at the 0.05 level: peer community, faculty relationships, academic climate, and meaningful engagement.

Among the different academic year levels, first-year students reported the highest score for community of peers (3.64 ± 0.88) and the lowest for inclusion and safety (1.82 ± 0.90). Similarly, second- to fourth-year students reported the highest scores for physical space and the lowest scores for inclusion and safety. As the academic year progressed, fifth-year students and interns obtained the lowest scores in meaningful engagement (2.26 ± 0.94 and 2.18 ± 0.95, respectively), but the highest score was given for physical space among 5th-year students (3.42 ± 0.93) and communities of peers for interns (3.43 ± 0.87). There were statistically significant associations with all subscales except physical space ( P value = 0.33).

Students with high GPA (4.50–5.0) recorded higher results across all domains than did their peers who earned a lower GPA (i.e., less than 4.00), with the exception of inclusion and safety. It is interesting to note that the physical space domain stood out as the highest scorer for all groups, while the scores for inclusion and safety fell short among all groups according to GPA. There was a statistically significant association with the first three domains only, community of peers, faculty relationships, and academic climate.

Employed students reported higher scores on measures related to community engagement reflected in the community of peers (3.70 ± 0.63), while unemployed and freelance students had the highest scores for physical space (3.52 ± 0.95 and 3.65 ± 1.13, respectively). The inclusion and safety subscale scores were the lowest for unemployed and employed students (2.25 ± 0.95 and 2.49 ± 1.20, respectively), while freelancers reported the lowest score for the academic climate subscale (2.36 ± 0.61). Employment status was significantly associated with only the mentoring subscale ( P value = 0.02).

Students who were single attained the highest average score of 3.37 ± 0.91 on the physical space domain, while inclusion and safety presented a challenging component (2.26 ± 0.95). Conversely, those who were married or engaged garnered the highest community of peer ratings, averaging 3.53 ± 0.78, and the lowest for faculty relationships, with a mean value of 2.72 ± 0.92. Students residing with family or in private accommodations, as well as those with families living in Riyadh or outside Riyadh, reported the highest scores in the physical space domain and the lowest scores in inclusion and safety. However, the association was not statistically significant between all groups or across all subscales ( P value > 0.05) .

Qualitative results

Participants of both genders, senior and junior years, represented the FGDs (Table  4 ). One participant was employed, and all were living with their families.

As open-ended questions were used to collect data, themes were derived from deductive and inductive analysis. Inductive analysis was based on a priori themes based on the JHLES domains. Table 5 shows the domains in which participants’ perceptions were compared with the quantitative findings. Some qualitative findings aligned with the quantitative findings, while others contradicted or explained them.

Evaluating the learning environment for medical students is essential for improving their professional standards, knowledge, and skills. This mixed methods study explored medical students' perceptions about the learning environment at the College of Medicine, a well-known university in Saudi Arabia, King Saud University. This study is two-pronged, first, to quantitatively assess students’ perceptions of the COM-KSU learning environment and, second, to qualitatively explore their experience in the same medical school.

Our study yielded an overall average score of 81 out of 140 on the JHLES. Notably, there was no predefined threshold for a passing or positive score on this tool. Compared to the original study where the scale was first used and validated, the average score in our study was lower (107 vs 81, respectively) [ 7 ]. This discrepancy might be related to the original study's single-institute design affects the generalizability of its results, and the differences in student characteristics due to the U.S. requiring a bachelor’s degree for medical school admission, unlike KSA, where students enter directly after high school, play a role. Additionally, the original study did not focus on the "hidden curriculum" influenced by organizational culture and structure, which may explain the discrepancy given the distinct social, organizational, and learning cultures between our context and the American one. However, our results were consistent with those of other studies that were conducted in other medical schools in different countries, including Malaysia, India and Pakistan, ranging from 81.1 to 86 [ 9 , 10 , 11 ].

Two previously published studies in the same setting, COM-KSU (2008 and 2017), utilized the DREEM survey and revealed that medical students reported different average scores (89.9 out of 200 and 171.57 out of 250, respectively) [ 5 , 6 ]. Compared to the current study utilizing the JHLES, we may compare the findings based on a significant correlation between the two measures that support the use of the JHLES in the assessment of the same construct [ 11 ]. This comparison yielded reassuring results that the perceptions of medical students are still positive, with variations in the domains of LE, as described below. The added value of the qualitative component of the current study elicits more depth in understanding LE in the COM-KSU.

Although there was no difference among male and female students in the DREEM overall average score in a previous study that was conducted at the same college in 2017, our study revealed a higher overall average score among males (83.4) than females (77.5). The lower recorded score among females might be explained by their tendency to have higher expectations of a learning environment that was not achieved as their counterpart expected [ 12 , 13 ]. For explanations, male students had higher scores in different domains related to their relationships with the faculty and peers, including mentorship, peer support, and the academic climate. Nevertheless, both genders perceived a negative view where they expressed potential gender discrimination in the focus group interviews. Male students felt that they were treated differently than females, while their counterparts believed that males had more opportunities to build relationships with the faculty and gain more experience accordingly.

In terms of academic years, the domains and overall average scores decreased as the students progressed from the first year to their internships, with an exceptional decrease in the third year followed by the recovery of scores afterward. Nevertheless, students in the first year had higher average scores than interns, possibly due to the new environment and the support provided during their first year. Qualitative group interviews elaborated more on this variation, where medical students in the first year felt a sense of pride and honor upon being accepted in the COM-KSU. They believe that this was a validation of their social status.

Although the relationship between medical students’ feelings of pride in belonging to their college and the learning environment is complex and multifaceted [ 14 ], a positive and supportive learning environment that fosters a sense of belonging can enhance medical students’ feelings of pride and affiliation with their college [ 15 , 16 ], which is evident among first-year medical students. In contrast, a negative learning environment that lacks support and inclusivity can detrimentally impact medical students’ feelings of pride and belonging [ 17 ]. Nevertheless, first-year students still experienced negative emotional effects that were not captured by the quantitative questionnaire due to the lack of professional identification they encountered when they moved from the preparatory year to medical school.

However, the decrease in the average score during the third year could be explained by engagement in clinical rotations and practical applications instead of merely learning basic science. This perception was explained during focus group interviews where students explained the third year as the most challenging due to the preparation for their actual medical practice. This included starting to see patients, taking medical history, and performing physical examinations. Interestingly, this result was consistent with other studies that were conducted in different medical schools, although different assessment tools were used, including MSLES, DREEM, and the same tool used in this study (i.e., JHLES) [ 3 , 7 , 10 ]. In contrast, other studies have shown that medical students feel more satisfied with clinical practice than with basic science during the first and second years [ 12 , 18 , 19 ].

This paradox might be explained by the difficulty students faced at the beginning of the clinical year, after which it decreased or diminished after they gained confidence in their practice under the supervision of well-trained faculty [ 20 , 21 , 22 ]. Hence, higher average scores in the following years could be explained by the maturity of the medical students and their ability to overcome early difficulties after they have more experience during clinical rotations. In the COM-KSU, medical students in their fifth year are prepared to experience life as physicians where they have pure clinical experience joining medical teams, attending rounds, clinics and doing procedures under the supervision of trained faculty and senior doctors. Hence, when mentoring was assessed among medical students, their perception reflected by the average score given to this domain increased as the number of academic years increased, with the highest score occurring during the internship. Mentorship plays an important role in the learning environment, as described in other studies [ 23 , 24 , 25 ]. The importance of the student‒faculty relationship and the enhancement of faculty influence on students are supported by the qualitative findings, which demonstrate that students' perceptions of faculty support vary, which is congruent with other studies [ 26 , 27 , 28 ]. However, a study revealed that the majority of faculty members are not prepared to provide the kind of support that has been shown to be most effective for students [ 29 ].

Furthermore, the meaningful engagement of students declines as the academic year progresses, as expressed by students’ responses to this domain in the JHLES. The qualitative approach elaborated more when students complained about the lack of support provided by the student council, which the COM-KSU perceived as the hub where medical students can engage and obtain the required support. From the students’ perspective, the student council was not able to provide effective support or bring about significant changes for students facing challenges related to their medical study needs. The qualitative study participants agreed with the findings of other local studies, highlighting the absence of a supportive environment for students in our local colleges [ 12 , 30 , 31 ]. On the other hand, the majority of students reflected positively on peer support, where they found it to have a positive impact on them. They identified college friends and colleagues as the main sources of support, which was congruent with other studies that explained the same attitude [ 32 , 33 , 34 ].

According to the students’ performance measured by their GPA, students with higher GPA had higher JHLES scores, both overall and domain average scores. High-achieving students tend to have more positive perceptions of the learning environment than do students with lower GPAs [ 10 , 11 , 21 , 22 , 35 ]. This could suggest a positive association between academic achievement and students’ perceptions of the educational setting [ 18 , 19 , 36 , 37 , 38 ]. However, students experienced positive consequences from high competition in the learning environment due to family and physician expectations that were captured during the focus group discussion. Similar results were found in another study that was conducted in the medical school of the University of Valladolid [ 39 ].

Inclusion and safety were negatively perceived in this study among medical students at all levels, regardless of their gender, academic year, or performance, which was reflected in their GPAs. This finding was consistent with other studies measuring the same domain average score of Cyberjaya University College of Medical Sciences (CUCMS), Nil Ratan Sircar Medical College (NRSMC), and College of Medicine and Sagore Dutta Hospital (CMSDH) [ 9 , 11 ]; however, this finding was in contrast to that of PUGSOM [ 40 ]. A possible explanation might be related to the aforementioned reasons, which were associated with students’ perceptions of gender discrimination, stress in the first year due to the new environment and in the third year due to engagement in clinical practice, and their achievements, which elevated stress when they had lower GPAs. Previous studies have shown that the prevalence of stress is greater during the first three years of medical education, which is consistent with our findings [ 35 ].

In contrast, the physical space domain in our study received the highest score, where we believe that physical space has improved as a result of the college's 2018 expansion [ 41 ].

Strengths and limitations

One key strength of this study is the employment of a comprehensive mixed methods approach to gain an understanding of how students perceive their learning environment. This approach collects numerical data, delves deeply into the students’ experiences and feelings, and provides valuable insights through the integration of findings from both approaches. Another strength of this study is the large number of participants from different academic years, which allows for a diverse range of perspectives from both new and experienced students.

Nevertheless, convenience sampling may not fully represent the student population and limits the generalizability of the findings. Additionally, focusing on one institution may not capture the experiences of students across different settings, cultures, or cities, potentially limiting the applicability of any recommendations to other medical colleges or regions. However, the large sample size, the diversity of data and the integration of results may enhance the transferability of the findings.

Recommendations for educational institutions

Enhance faculty development: Address the issues of perceived neutrality and reported negative interactions with faculty by investing in faculty development programs. These programs should focus on improving communication skills and mentoring abilities and cultivating more supportive and encouraging faculty‒student relationships. Creating opportunities for regular feedback from students can also aid in faculty improvement. This is important as students showed high tendency to be influenced by advice from faculty member.

Cultivate Supportive Environments: Foster a less stressful academic climate by promoting a culture of mutual respect and collaboration within the institution. Encourage open dialog between students and faculty, where questions and concerns can be raised without judgment. Stress management and well-being programs should be implemented to help students cope with academic pressures.

Revise Curriculum and Mentorship Programs: Address curriculum concerns by engaging students in the curriculum development process. Consider their suggestions for better organization, logical flow, and references. Additionally, structured mentorship programs that connect students with experienced doctors who can provide guidance, share experiences, and serve as positive role models should be established.

Evaluate and Improve Support Services: Reevaluate the effectiveness of support services such as the students' council and academic support departments. These services are responsive to students' needs and have the authority to enact meaningful changes. Regularly solicit feedback from students to gauge the impact of these services.

Promote Inclusivity and Gender Equity: as FGDs showed that both genders feel discriminated against, creating initiatives to address perceptions of discrimination and gender bias within the learning environment is important. This may involve raising awareness, offering training on gender sensitivity, and implementing policies that promote inclusivity and equal opportunities for all students, regardless of gender.

Recommendations for further research:

Longitudinal studies should be conducted to track the changes in students’ perceptions and experiences. This will help us identify emerging trends and understand the long-term effects of interventions and policy changes.

This research can be expanded by including studies with medical schools or institutions to validate our findings and assess how applicable they are in diverse educational settings.

The use of mixed methods research in the field of education should be further explored. Investigate approaches that combine qualitative and deductive methods to gain deeper insights into students’ educational experiences.

Dive deeper into specific areas highlighted in this research, such as mentoring programs and concerns related to the curriculum. Explore ways to enhance mentoring effectiveness and develop strategies for improving the curriculum to create a learning environment.

Interventions targeted at addressing identified areas should be implemented for improvement while thoroughly evaluating their impact. This will enable institutions to assess the effectiveness of these interventions based on data-driven decisions leading to the enhancement of education.

This study was the first to assess the learning environment of medical students at COM-KSU through quantitative and qualitative approaches. The overall average JHLES score indicated room for improvement, in line with global trends. Gender disparities, challenges in different academic years, and the critical role of mentorship were identified. Academic performance correlated positively with students' perceptions, while inclusion and safety were areas of concern. The physical space domain received the highest score, reflecting investments in infrastructure. These findings underscore the need for targeted interventions to address gender disparities, enhance mentorship, improve student engagement, and ensure inclusivity and safety, ultimately enhancing the educational experience of COM-KSU medical students.

Availability of data and materials

The datasets used and analyzed during the current study are available for request from the corresponding author.

Abbreviations

• Learning environment

Johns Hopkins Learning Environment Scale

College of Medicine at King Saud University

Dundee Ready Educational Environment Measure

Focused group discussions

Medical School Learning Environment Survey

Cyberjaya University College of Medical Sciences

Nil Ratan Sircar Medical College

College of Medicine and Sagore Dutta Hospital

Rusticus SA, Wilson D, Casiro O, Lovato C. Evaluating the quality of health professions learning environments: development and validation of the Health Education Learning Environment Survey (HELES). Eval Health Prof. 2020;43:162–8.

Article   Google Scholar  

Dunham L, Dekhtyar M, Gruener G, CichoskiKelly E, Deitz J, Elliott D, Stuber ML, Skochelak SE. Medical student perceptions of the learning environment in medical school change as students transition to clinical training in undergraduate medical school. Teach Learn Med. 2017;29:383–91.

Wayne SJ, Fortner SA, Kitzes JA, Timm C, Kalishman S. Cause or effect? The relationship between student perception of the medical school learning environment and academic performance on USMLE Step 1. Med Teach. 2013;35:376–80.

Harden RM. The learning environment and the curriculum. Med Teach. 2001;23:335–6.

Al Ayed IH, Sheik SA. Assessment of the educational environment at the college of medicine of king Saud University, Riyadh. EMHJ - East Mediterr Health J. 2008;14(4):953–9.

Google Scholar  

Soliman MM, Sattar K, Alnassar S, Alsaif F, Alswat K, Alghonaim M, Alhaizan M, Al-Furaih N. Medical students’ perception of the learning environment at King Saud University Medical College, Saudi Arabia, using DREEM inventory. Adv Med Educ Pract. 2017;8:221–7.

Shochet RB, Colbert-Getz JM, Wright SM. The Johns Hopkins learning environment scale: measuring medical students’ perceptions of the processes supporting professional formation. Acad Med J Assoc Am Med Coll. 2015;90:810–8.

R Core Team (2022) R: The R Project for Statistical Computing. https://www.r-project.org/ . Accessed 13 Aug 2023

Sengupta P, Sharma A, Das N. Perception of Learning environment among undergraduate medical students in two different medical schools through DREEM and JHLES Questionnaire. J Clin Diagn Res JCDR. 2017;11:JC01–4.

Shahzad S, Wajid G. How different is the learning environment of public and private sector medical colleges in Lahore, Pakistan? Pak J Med Sci. 2023;39:757–63.

Tackett S, Shochet R, Shilkofski NA, Colbert-Getz J, Rampal K, Abu Bakar H, Wright S. Learning environment assessments of a single curriculum being taught at two medical schools 10,000 miles apart. BMC Med Educ. 2015;15:105.

Al-Hazimi A, Zaini R, Al-Hyiani A, Hassan N, Gunaid A, Ponnamperuma G, Karunathilake I, Roff S, McAleer S, Davis M. Educational environment in traditional and innovative medical schools: a study in four undergraduate medical schools. Educ Health Abingdon Engl. 2004;17:192–203.

Schultz-Jones B, Ledbetter C, Bishop J (2014) Investigating Differences Between Male and Female Students’ Perceptions of School Library Learning Environments. IASL Annu Conf Proc. https://doi.org/10.29173/iasl7867 .

Tanya Albert Henry (2016) Med students’ perceptions of learning environment matter. In: Am. Med. Assoc. https://www.ama-assn.org/education/accelerating-change-medical-education/med-students-perceptions-learning-environment . Accessed 13 Aug 2023.

Mahmood H, Chaudhry MA, Rashid H, Farani TM, Ali AR, Batool Z, Amin B. Medical student’s perception of educational environment. Proceedings. 2020;34:47–51.

Xie Y, Cao D, Sun T, Yang LB. The effects of academic adaptability on academic burnout, immersion in learning, and academic performance among Chinese medical students: a cross-sectional study. BMC Med Educ. 2019. https://doi.org/10.1186/s12909-019-1640-9 .

Verbree A-R, Isik U, Janssen J, Dilaver G. Inclusion and diversity within medical education: a focus group study of students’ experiences. BMC Med Educ. 2023;23:61.

Sarwar S, Tarique S. Perception of educational environment: does it impact academic performance of medical students? JPMA J Pak Med Assoc. 2016;66:1210–4.

Yoo D-M, Kim D-H. The relationship between students’ perception of the educational environment and their subjective happiness. BMC Med Educ. 2019;19:409.

Almansour M, AlMehmadi BA, Zeb NG, Matbouly G, Sami W, Badahdah A-M. Promising score for teaching and learning environment: an experience of a fledgling medical college in Saudi Arabia. BMC Med Educ. 2023;23:479.

Eyal L, Cohen R. Preparation for clinical practice: a survey of medical students’ and graduates’ perceptions of the effectiveness of their medical school curriculum. Med Teach. 2006;28:e162-170.

Fleit HB, Lu W-H, Olvet DM, Chandran L. Medical students’ perception of behaviors in the clinical learning environment evolve with increasing clinical exposure as measured with situational video vignettes. Med Teach. 2020;42:822–7.

Dobie S, Smith S, Robins L. How assigned faculty mentors view their mentoring relationships: an interview study of mentors in medical education. Mentor Tutoring Partnersh Learn. 2010;18:337–59.

Rose GL, Rukstalis MR, Schuckit MA. Informal mentoring between faculty and medical students. Acad Med J Assoc Am Med Coll. 2005;80:344–8.

Sonawane T, Meshram R, Jagia G, Gajbhiye R, Adhikari S. Effects of Mentoring in First Year Medical Undergraduate Students using DASS-21. J Clin Diagn Res. 2021. https://doi.org/10.7860/JCDR/2021/50102.15682 .

Cress CM. Creating inclusive learning communities: the role of student–faculty relationships in mitigating negative campus climate. Learn Inq. 2008;2:95–111.

Hendelman W, Byszewski A. Formation of medical student professional identity: categorizing lapses of professionalism, and the learning environment. BMC Med Educ. 2014;14:139.

Neufeld A, Malin G. How medical students’ perceptions of instructor autonomy-support mediate their motivation and psychological well-being. Med Teach. 2020;42:650–6.

Costino K. Equity-minded faculty development: an intersectional identity-conscious community of practice model for faculty learning. Metrop Univ. 2018;29:117–36.

Al-Mohaimeed A. Perceptions of the educational environment of a new medical school, Saudi Arabia. Int J Health Sci. 2013;7:150–9.

Altemani AH, Merghani TH. The quality of the educational environment in a medical college in Saudi Arabia. Int J Med Educ. 2017;8:128–32.

Lerchenfeldt S, Mi M, Eng M. The utilization of peer feedback during collaborative learning in undergraduate medical education: a systematic review. BMC Med Educ. 2019;19:321.

Nofziger AC, Naumburg EH, Davis BJ, Mooney CJ, Epstein RM. Impact of peer assessment on the professional development of medical students: a qualitative study. Acad Med J Assoc Am Med Coll. 2010;85:140–7.

Yamada Y, Klugar M, Ivanova K, Oborna I. Psychological distress and academic self-perception among international medical students: the role of peer social support. BMC Med Educ. 2014;14:256.

Abdulghani HM, AlKanhal AA, Mahmoud ES, Ponnamperuma GG, Alfaris EA. Stress and its effects on medical students: a cross-sectional study at a college of medicine in Saudi Arabia. J Health Popul Nutr. 2011;29:516–22.

Khursheed I, Baig L. Students’ perceptions of educational environment of a private medical school in Pakistan. JPMA J Pak Med Assoc. 2014;64:1244–9.

Mayya S, Roff S. Students’ perceptions of educational environment: a comparison of academic achievers and under-achievers at kasturba medical college, India. Educ Health Abingdon Engl. 2004;17:280–91.

Ahmed Y, Taha MH, Al-Neel S, Gaffar AM. Students’ perception of the learning environment and its relation to their study year and performance in Sudan. Int J Med Educ. 2018. https://doi.org/10.5116/ijme.5af0.1fee .

Corell A, Regueras LM, Verdú E, Verdú MJ, de Castro JP. Effects of competitive learning tools on medical students: a case study. PLoS ONE. 2018;13.

Tackett S, Bakar HA, Shilkofski NA, Coady N, Rampal K, Wright S. Profiling medical school learning environments in Malaysia: a validation study of the Johns Hopkins Learning Environment Scale. J Educ Eval Health Prof. 2015;12.

King Saud University Medical City. المدينة الطبية بجامعة الملك سعود | news. 2018. https://medicalcity.ksu.edu.sa/ar/news/details/26-10-2018 . Accessed 21 Oct 2023

Download references

Acknowledgements

Not applicable.

This research is non-funded.

Author information

Authors and affiliations.

Department of Medical Education, College of Medicine, King Saud University, Riyadh, Saudi Arabia

Mohammed Almansour

Department of Family and Community Medicine, College of Medicine, King Saud University, Riyadh, Saudi Arabia

Noura Abouammoh & Hani A. Alghamdi

College of Medicine, King Saud University, Riyadh, Saudi Arabia

Reem Bin Idris, Omar Abdullatif Alsuliman, Renad Abdulrahman Alhomaidi & Mohammed Hamad Alhumud

You can also search for this author in PubMed   Google Scholar

Contributions

All authors made a significant contribution to the work reported, whether that is in the conception, study design, execution, acquisition of data, analysis and interpretation, or in all these areas; took part in drafting, revising or critically reviewing the article; gave final approval of the version to be published; have agreed on the journal to which the article has been submitted; and agree to be accountable for all aspects of the work.

Corresponding author

Correspondence to Hani A. Alghamdi .

Ethics declarations

Ethics approval and consent to participate.

This study was approved by King Saud University’s Institutional Review Board (KSU IRB) with the approval number E-22–7298.

Consent to participate

Electronic informed consent was obtained from all participants in the quantitative arm, and written informed consent was obtained from all participants in the qualitative arm before they participated in the study.

Consent for publication

Competing interests.

The authors declare that they have no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Almansour, M., Abouammoh, N., Idris, R.B. et al. Exploring medical students' experience of the learning environment: a mixed methods study in Saudi medical college. BMC Med Educ 24 , 723 (2024). https://doi.org/10.1186/s12909-024-05716-4

Download citation

Received : 14 March 2024

Accepted : 26 June 2024

Published : 03 July 2024

DOI : https://doi.org/10.1186/s12909-024-05716-4

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Medical education
• Comparative studies
• Mixed methods

BMC Medical Education

ISSN: 1472-6920

• Submission enquiries: [email protected]
• General enquiries: [email protected]

2024 Theses Doctoral

Religious Routes to Conflict Mitigation: Three Papers on Buddhism, Nationalism, and Violence

Dorjee, Tenzin

The notion that religion intensifies nationalism and escalates conflict is widely accepted. In spite of its frequent association with violence, however, religious doctrines and institutions sometimes appear to have the radical power to deescalate conflict and reroute the expression of political grievances away from bloodshed. How, and under what conditions, might religion lend itself to the mitigation of ethnic conflict? Focusing on Buddhist nationalisms in East Asia and Southeast Asia, the three papers in this dissertation study the influence of religious beliefs on political attitudes and conflict behavior at various levels of analysis. Using ethnographic approaches, case study methods, and original field data collected from nearly a hundred interviews among Tibetan subjects in India and Sinhalese monastics in Sri Lanka, these essays seek to deepen the nuances and complexity in our understanding of the relationship between Buddhism, nationalism, and violence.Paper #1 studies the relationship between Buddhism and suicide protest, focusing on the puzzle of self-immolation: Why do high-commitment protesters in some conflicts choose this method over conventional tactics of nonviolent resistance or suicide terrorism? Taking the wave of Tibetan self-immolations between 2009 and 2018 as a case study, this paper probes the causal importance of strategic considerations, structural constraints, and normative restraints that may have influenced the protesters’ choice of method. I develop a theoretical framework proposing that suicide protesters evaluate potential tactics based on three criteria: disruptive capability, operational feasibility, and ethical permissibility. Leveraging in-depth interviews and a close reading of the self-immolators’ last words, I conclude that the Buddhist clergy’s broad conception of violence, interacting with international norms, constrains the protesters’ tactical latitude by narrowing the parameters of what qualifies as nonviolent action, thereby eliminating many of the standard repertoires of contention from the movement’s arsenal while sanctioning self-immolation as a legitimate form of dissent. I argue that a fundamental paradox in the self-immolators’ theory of change, namely the tension between a tactic’s disruptive capability and ethical permissibility, ends up restricting their freedom of action. Paper #2 zooms out to examine the relationship between religion, nationalism, and violence. It starts with a broad question: How, and under what conditions, might religion lend itself to the mitigation –– or the escalation –– of ethnonational conflict? To what extent do religious ideas travel from scripture to political preferences and conflict behavior? I develop two hypotheses predicting the influence of scriptural ideas on nationalist commitment and suggestibility to violence –– devoting special attention to how a group’s conception of its own national interest might be affected when the religious identity of its members supersedes their political identity. The paper finds that the Buddhist belief in rebirth can undermine the strength of one’s nationalist commitment by injecting a dose of ambiguity into one’s conception of identity. This suggests that a religious belief such as rebirth can be mobilized to deescalate ethnonational conflict by highlighting the fluidity of ethnic identity and thus lowering the stakes of conflict. Moreover, it also finds that Mahayana Buddhism’s emphasis on altruism, while rooted in compassion toward others, can end up increasing an individual’s suggestibility to violence and therefore should not be assumed to be a pacifying force in conflict. Mahayana doctrines, though built on more inclusivist founding principles than the Theravada tradition and therefore more resistant to exclusivist ideologies like nationalism, are nevertheless susceptible to utilitarian reasoning and lend themselves readily to the justification of violence. In our interviews, Tibetan monastics, educated under a uniform Mahayana curriculum, turned out to be far more suggestible to violence than their Theravada counterparts in Sri Lanka, an observation that supports our counterintuitive hypothesis linking an altruism-oriented curriculum with suggestibility to violence. Paper #3 takes a historical case study approach to examine how Buddhist religious ideas may have, in interaction with liberal international norms, influenced the Tibetan leadership’s de-escalation politics in the Sino-Tibetan conflict. While paper #2 of this dissertation explored Buddhism’s relationship with nationalism and violence at the level of rank-and-file citizens, this paper shifts the focus from group-level preferences to elite-level decision-making. It relies on document analysis and process tracing methods to answer a particular historical question: How did the independence-seeking Tibetan nationalist leadership of the 1960s evolve into compromise-seeking pacifists in the 1980s and subsequent decades? I seek to illuminate the pathways by which religious beliefs and charismatic leadership structure, in interaction with the normative constraints of liberal internationalism, may have facilitated the Tibetan leadership’s de-escalation politics in the Sino-Tibetan conflict. To do so, I leverage counterfactual history (Belkin & Tetlock, 1996), biographical data of key leaders (Creswell, 1998), and document analysis of their speeches and writings –– including a close examination of the Dalai Lama’s annual March 10 speeches from 1960 to 2011. While the other two papers explore the multifaceted relationship between Buddhism, nationalism, and violence by studying the political attitudes and conflict behavior of ordinary people and rank-and-file monastics, this paper delves into the political and psychological evolution of two Tibetan leaders, the Dalai Lama and former Tibetan prime minister Samdhong Rinpoche, to examine the ways in which private religious beliefs can interact with global norms to guide and constrain the high-level foreign policy decision-making of political elites.

Geographic Areas

• China--Tibet Autonomous Region
• Political science
• International relations
• Self-immolation--Religious aspects--Buddhism
• Political violence--Religious aspects--Buddhism
• Nationalism
• Mahayana Buddhism
• Theravāda Buddhism
• Religion and politics
• Bstan-ʼdzin-rgya-mtsho, Dalai Lama XIV, 1935-

This item is currently under embargo. It will be available starting 2029-06-24.

More About This Work

• DOI Copy DOI to clipboard

High-Entropy Materials Design by Integrating the First-Principles Calculations and Machine Learning: A Case Study in the Al-Co-Cr-Fe-Ni System

• Original Paper
• Published: 04 July 2024

Cite this article

• Guangchen Liu 1 ,
• Songge Yang 1 &
• Yu Zhong   ORCID: orcid.org/0000-0002-2363-7093 1  

High-entropy alloys (HEAs) have drawn significant attention within the materials science community due to their exceptional properties. However, optimizing compositions in these alloys is challenging due to their vast compositional space. Conventional computational methods are constrained by high computational costs, limiting rapid database expansion. In this study, we propose a novel approach that integrates first-principles calculations with machine learning to efficiently explore the Al-Co-Cr-Fe-Ni system, encompassing both face-centered cubic (FCC) and body-centered cubic (BCC) phases. Our approach establishes comprehensive relationships among compositions, phase stability, and elastic properties at 0 K. A comprehensive database and software for this system are developed. Analysis of this database reveals the presence of FCC and BCC phases under specific valence electron concentration (VEC) criteria. Through screening and statistical analysis using a performance index, we quantitatively investigate the brittleness and ductility, facilitating the identification of optimal HEAs candidates. This work enhances our understanding of the HEA properties, offering a promising pathway for materials design and optimization.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save.

• Get 10 units per month
• Download Article/Chapter or Ebook
• 1 Unit = 1 Article or 1 Chapter
• Cancel anytime

Price includes VAT (Russian Federation)

Instant access to the full article PDF.

Rent this article via DeepDyve

Institutional subscriptions

Data Availability

All data generated or analyzed during this study are included in this published article and its supplementary information files.

B. Cantor, I.T.H. Chang, P. Knight, A.J.B. Vincent, Microstructural development in equiatomic multicomponent alloys. Mater. Sci. Eng. A 375 , 213–218 (2004)

Article   Google Scholar  

T.K. Chen, T.T. Shun, J.W. Yeh, M.S. Wong, Nanostructured nitride films of multi-element high-entropy alloys by reactive DC sputtering. Surf. Coat. Technol. 188 , 193–200 (2004)

C.-Y. Hsu, J.-W. Yeh, S.-K. Chen, T.-T. Shun, Wear resistance and high-temperature compression strength of Fcc CuCoNiCrAl 0.5 Fe alloy with boron addition. Metall. Mater. Trans. A 35 (5), 1465–1469 (2004)

P.K. Huang, J.W. Yeh, T.T. Shun, S.K. Chen, Multi-principal-element alloys with improved oxidation and wear resistance for thermal spray coating. Adv. Eng. Mater. 6 (1–2), 74–78 (2004)

Article   CAS   Google Scholar  

J.-W. Yeh, S.-J. Lin, T.-S. Chin, J.-Y. Gan, S.-K. Chen, T.-T. Shun, C.-H. Tsau, S.-Y. Chou, Formation of simple crystal structures in Cu-Co-Ni-Cr-Al-Fe-Ti-V alloys with multiprincipal metallic elements. Metall. Mater. Trans. A 35 (8), 2533–2536 (2004)

J.W. Yeh, S.K. Chen, S.J. Lin, J.Y. Gan, T.S. Chin, T.T. Shun, C.H. Tsau, S.Y. Chang, Nanostructured high-entropy alloys with multiple principal elements: novel alloy design concepts and outcomes. Adv. Eng. Mater. 6 (5), 299–303 (2004)

D.B. Miracle, J.D. Miller, O.N. Senkov, C. Woodward, M.D. Uchic, J. Tiley, Exploration and development of high entropy alloys for structural applications. Entropy 16 (1), 494–525 (2014)

J.-W. Yeh, Alloy design strategies and future trends in high-entropy alloys. Jom 65 , 1759–1771 (2013)

M.-H. Tsai, J.-W. Yeh, High-entropy alloys: a critical review. Mater. Res. Lett. 2 (3), 107–123 (2014)

Y. Zhang, Y.J. Zhou, Solid solution formation criteria for high entropy alloys, in Materials Science Forum . (Trans Tech Publications, Stafa, 2007), pp.1337–1339

Google Scholar  

C. Li, J.C. Li, M. Zhao, Q. Jiang, Effect of alloying elements on microstructure and properties of multiprincipal elements high-entropy alloys. J. Alloy. Compd. 475 (1–2), 752–757 (2009)

Y. Zhang, Mechanical properties and structures of high entropy alloys and bulk metallic glasses composites, in Materials Science Forum . (Stafa, Trans Tech Publications, 2010), pp.1058–1061

H.Y. Diao, R. Feng, K.A. Dahmen, P.K. Liaw, Fundamental deformation behavior in high-entropy alloys: an overview. Curr. Opin. Solid State Mater. Sci. 21 (5), 252–266 (2017)

L.H. Wen, H.C. Kou, J.S. Li, H. Chang, X.Y. Xue, L. Zhou, Effect of aging temperature on microstructure and properties of AlCoCrCuFeNi high-entropy alloy. Intermetallics 17 (4), 266–269 (2009)

C.-M. Lin, H.-L. Tsai, Evolution of microstructure, hardness, and corrosion properties of high-entropy Al 0.5 CoCrFeNi alloy. Intermetallics 19 (3), 288–294 (2011)

M.-H. Tsai, Physical properties of high entropy alloys. Entropy 15 (12), 5338–5345 (2013)

C.-J. Tong, M.-R. Chen, J.-W. Yeh, S.-J. Lin, S.-K. Chen, T.-T. Shun, S.-Y. Chang, Mechanical performance of the Al x CoCrCuFeNi high-entropy alloy system with multi-principal elements. Metall. Mater. Trans. A 36 (5), 1263–1271 (2005)

Y.J. Zhou, Y. Zhang, Y.L. Wang, G.L. Chen, Solid solution alloys of Al-Co-Cr-Fe-Ni-Ti x with excellent room-temperature mechanical properties. Appl. Phys. Lett. 90 (18), 181904 (2007)

L. Cao, X. Wang, Y. Wang, L. Zhang, Y. Yang, F. Liu, Y. Cui, Microstructural evolution, phase formation and mechanical properties of multi-component AlCoCrFeNi x alloys. Appl. Phys. A 125 , 1–11 (2019)

O.N. Senkov, J.D. Miller, D.B. Miracle, C. Woodward, Accelerated exploration of multi-principal element alloys with solid solution phases. Nat. Commun. 6 (1), 1–10 (2015)

W.P. Huhn, M. Widom, Prediction of A2 to B2 phase transition in the high-entropy alloy Mo-Nb-Ta-W. JOM 65 (12), 1772–1779 (2013)

A.J.S.F. Tapia, D. Yim, H.S. Kim, B.-J. Lee, An approach for screening single phase high-entropy alloys using an in-house thermodynamic database. Intermetallics 101 , 56–63 (2018)

J.M. Sanchez, I. Vicario, J. Albizuri, T. Guraya, J.C. Garcia, Phase prediction, microstructure and high hardness of novel light-weight high entropy alloys. J. Mater. Res. Technol. 8 (1), 795–803 (2019)

Y. Lederer, C. Toher, K.S. Vecchio, S. Curtarolo, The search for high entropy alloys: a high-throughput ab-initio approach. Acta Mater. 159 , 364–383 (2018)

J.E. Saal, I.S. Berglund, J.T. Sebastian, P.K. Liaw, G.B. Olson, Equilibrium high entropy alloy phase stability from experiments and thermodynamic modeling. Scr. Mater. 146 , 5–8 (2018)

C. Jiang, B.P. Uberuaga, Efficient ab initio modeling of random multicomponent alloys. Phys. Rev. Lett. 116 (10), 105501 (2016)

Article   PubMed   Google Scholar  

C. Zhang, F. Zhang, S. Chen, W. Cao, Computational thermodynamics aided high-entropy alloy design. JOM 64 (7), 839–845 (2012)

D. Ma, B. Grabowski, F. Körmann, J. Neugebauer, D. Raabe, Ab initio thermodynamics of the CoCrFeMnNi high entropy alloy: Importance of entropy contributions beyond the configurational one. Acta Mater. 100 , 90–97 (2015)

M. Asadikiya, Y. Zhang, L. Wang, D. Apelian, Y. Zhong, Design of ternary high-entropy aluminum alloys (HEAls). J. Alloy. Compd. 891 , 161836 (2022)

S. Yang, Y. Wang, Z.-K. Liu, B. Mishra, Y. Zhong, Ab initio modeling on the thermodynamic and temperature-dependent elastic properties of subsystems of the FCC FeNiCoCr medium entropy alloys (MEAs). Acta Mater. 260 , 119341 (2023)

J. Zhang, C. Gadelmeier, S. Sen, R. Wang, X. Zhang, Y. Zhong, U. Glatzel, B. Grabowski, G. Wilde, S.V. Divinski, Zr diffusion in BCC refractory high entropy alloys: a case of ‘non-sluggish’ diffusion behavior. Acta Mater. 233 , 117970 (2022)

J. Zhang, R. Wang, Y. Zhong, Identification of the eutectic points in the multicomponent systems with the high-throughput CALPHAD approach. J. Phase Equilib. Diffus. 43 (6), 844–857 (2022)

J.E. Saal, S. Kirklin, M. Aykol, B. Meredig, C. Wolverton, Materials design and discovery with high-throughput density functional theory: the open quantum materials database (OQMD). Jom 65 , 1501–1509 (2013)

G.R. Schleder, A.C. Padilha, C.M. Acosta, M. Costa, A. Fazzio, From DFT to machine learning: recent approaches to materials science–a review. J. Phys. 2 (3), 032001 (2019)

CAS   Google Scholar  

J. Gao, J. Zhong, G. Liu, S. Yang, B. Song, L. Zhang, Z. Liu, A machine learning accelerated distributed task management system (Malac-Distmas) and its application in high-throughput CALPHAD computations aiming at efficient alloy design. Adv. Powder Mater. 1 , 100005 (2022)

W. Yi, G. Liu, J. Gao, L. Zhang, Boosting for concept design of casting aluminum alloys driven by combining computational thermodynamics and machine learning techniques. J. Mater. Inform. 1 (2), 11 (2021)

W. Yi, G. Liu, Z. Lu, J. Gao, L. Zhang, Efficient alloy design of Sr-modified A356 alloys driven by computational thermodynamics and machine learning. J. Mater. Sci. Technol. 112 , 277–290 (2021)

T. Mueller, A.G. Kusne, R. Ramprasad, Machine learning in materials science: recent progress and emerging applications. Rev. Comput. Chem. 29 , 186–273 (2016)

R. Ramprasad, R. Batra, G. Pilania, A. Mannodi-Kanakkithodi, C. Kim, Machine learning in materials informatics: recent applications and prospects. npj Comput. Mater. 3 (1), 1–13 (2017)

N. Islam, W. Huang, H.L. Zhuang, Machine learning for phase selection in multi-principal element alloys. Comput. Mater. Sci. 150 , 230–235 (2018)

W. Huang, P. Martin, H.L. Zhuang, Machine-learning phase prediction of high-entropy alloys. Acta Mater. 169 , 225–236 (2019)

C. Wen, Y. Zhang, C. Wang, D. Xue, Y. Bai, S. Antonov, L. Dai, T. Lookman, Y. Su, Machine learning assisted design of high entropy alloys with desired property. Acta Mater. 170 , 109–117 (2019)

T. Akiba, S. Sano, T. Yanase, T. Ohta, M. Koyama, in Optuna: a next-generation hyperparameter optimization framework, Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 2019, pp. 2623–2631.

S. Guo, C. Ng, J. Lu, C. Liu, Effect of valence electron concentration on stability of fcc or bcc phase in high entropy alloys. J. Appl. Phys. (2011). https://doi.org/10.1063/1.3587228

S. Yang, J. Lu, F. Xing, L. Zhang, Y. Zhong, Revisit the VEC rule in high entropy alloys (HEAs) with high-throughput CALPHAD approach and its applications for material design—a case study with Al-Co-Cr-Fe-Ni system. Acta Mater. 192 , 11–19 (2020)

G. Kresse, J. Furthmüller, Efficient iterative schemes for ab initio total-energy calculations using a plane-wave basis set. Phys. Rev. B 54 (16), 11169 (1996)

G. Kresse, J. Furthmüller, Efficiency of ab-initio total energy calculations for metals and semiconductors using a plane-wave basis set. Comput. Mater. Sci. 6 (1), 15–50 (1996)

J.P. Perdew, J.A. Chevary, S.H. Vosko, K.A. Jackson, M.R. Pederson, D.J. Singh, C. Fiolhais, Atoms, molecules, solids, and surfaces: Applications of the generalized gradient approximation for exchange and correlation. Phys. Rev. B 46 (11), 6671 (1992)

J.P. Perdew, K. Burke, M. Ernzerhof, Generalized gradient approximation made simple. Phys. Rev. Lett. 77 (18), 3865 (1996)

Article   CAS   PubMed   Google Scholar  

A. Zunger, S.-H. Wei, L.G. Ferreira, J.E. Bernard, Special quasirandom structures. Phys. Rev. Lett. 65 (3), 353 (1990)

S. Yang, G. Liu, Y. Zhong, Revisit the VEC criterion in high entropy alloys (HEAs) with high-throughput ab initio calculations: a case study with Al-Co-Cr-Fe-Ni system. J. Alloys Compd. 916 , 165477 (2022)

A. van de Walle, M. Asta, G. Ceder, The alloy theoretic automated toolkit: a user guide. Calphad 26 (4), 539–553 (2002)

A. van de Walle, P. Tiwary, M. de Jong, D.L. Olmsted, M. Asta, A. Dick, D. Shin, Y. Wang, L.-Q. Chen, Z.-K. Liu, Efficient stochastic generation of special quasirandom structures. Calphad 42 , 13–18 (2013)

F.D. Murnaghan, Finite deformations of an elastic solid. Am. J. Math. 59 (2), 235–260 (1937)

S. Yang, Y. Zhong, Ab initio modeling of fcc Fe-Co-Cr-Ni high entropy alloys with full composition range. J. Phase Equilib. Diffus. 42 (5), 656–672 (2021)

G. Hinton, N. Srivastava, K. Swersky, Neural networks for machine learning lecture 6a overview of mini-batch gradient descent. Cited on 14 (8), 2 (2012)

L. Bottou, Online algorithms and stochastic approximations, in Online Learning and Neural Networks . (Cambridge University Press, Cambridge, 1998)

D.P. Kingma, J. Ba, Adam: a method for stochastic optimization (2014), Preprint at https://arxiv.org/abs/1412.6980

K. Jin, Y. Gao, H. Bei, Intrinsic properties and strengthening mechanism of monocrystalline Ni-containing ternary concentrated solid solutions. Mater. Sci. Eng. A 695 , 74–79 (2017)

Y.-L. Liu, Y. Zhang, H.-B. Zhou, G.-H. Lu, M. Kohyama, Theoretical strength and charge redistribution of fcc Ni in tension and shear. J. Phys. 20 (33), 335216 (2008)

J. Zarestky, C. Stassis, Lattice dynamics of γ-Fe. Phys. Rev. B 35 (9), 4500 (1987)

A. Kanrar, U. Ghosh, The elastic stiffness coefficients of nickel-iron single-crystal alloys at room temperature. J. Appl. Phys. 52 (9), 5851–5852 (1981)

H. Ledbetter, Comment on “The elastic stiffness coefficients of nickel-iron single-crystal alloys at room temperature.” J. Appl. Phys. 57 (11), 5069–5070 (1985)

J. Lenkkeri, Measurements of elastic moduli of face-centered cubic alloys of transition metals. J. Phys. F 11 (10), 1991 (1981)

J. Gump, H. Xia, M. Chirita, R. Sooryakumar, M. Tomaz, G. Harp, Elastic constants of face-centered-cubic cobalt. J. Appl. Phys. 86 (11), 6005–6009 (1999)

S. Guo, C. Ng, J. Lu, C.T. Liu, Effect of valence electron concentration on stability of fcc or bcc phase in high entropy alloys. J. Appl. Phys. 109 (10), 103505 (2011)

G. Sheng, C.T. Liu, Phase stability in high entropy alloys: formation of solid-solution phase or amorphous phase. Prog. Nat. Sci. 21 (6), 433–446 (2011)

Y. Mu, H. Liu, Y. Liu, X. Zhang, Y. Jiang, T. Dong, An ab initio and experimental studies of the structure, mechanical parameters and state density on the refractory high-entropy alloy systems. J. Alloy. Compd. 714 , 668–680 (2017)

L. Kaufman, H. Bernstein, Computer Calculation of Phase Diagrams. With Special Reference to Refractory Metals (Academic Press, New York, 1970)

O.H. Nielsen, R.M. Martin, First-principles calculation of stress. Phys. Rev. Lett. 50 (9), 697 (1983)

O.H. Nielsen, R.M. Martin, Stresses in semiconductors: Ab initio calculations on Si, Ge, and GaAs. Phys. Rev. B 32 (6), 3792 (1985)

W. Voigt, Lehrbuch der Kristallphysik, Verlag und Druck, Von BG Teubner. Leipz. Berl. 962 , 252 (1928)

S. Pugh, XCII. Relations between the elastic moduli and the plastic properties of polycrystalline pure metals. Lond. Edinb. Dublin Philos. Mag. J. Sci. 45 (367), 823–843 (1954)

H. Niu, X.-Q. Chen, P. Liu, W. Xing, X. Cheng, D. Li, Y. Li, Extra-electron induced covalent strengthening and generalization of intrinsic ductile-to-brittle criterion. Sci. Rep. 2 (1), 718 (2012)

Article   PubMed   PubMed Central   Google Scholar  

O. Senkov, D. Miracle, Generalization of intrinsic ductile-to-brittle criteria by Pugh and Pettifor for materials with a cubic crystal structure. Sci. Rep. 11 (1), 4531 (2021)

Article   CAS   PubMed   PubMed Central   Google Scholar  

M.C. Gao, J.-W. Yeh, P.K. Liaw, Y. Zhang, High-Entropy Alloys: Fundamentals and Applications (Springer, Cham, 2016)

Book   Google Scholar  

Download references

Acknowledgements

Part of the results were produced with funding support from the Advanced Cyberinfrastructure Coordination Ecosystem: Services & Support (ACCESS) program Award Numbers TG-DMR190004. The authors would like to thank the support and guidance from the WPI Academic & Research Computing program manager, Siamak Mohammed Z. Najafi, and his team member, James Kingsley, for their consulting and programming support for high-performance computing.

Author information

Authors and affiliations.

Mechanical and Materials Engineering Department, Worcester Polytechnic Institute, 100 Institute Rd, Worcester, MA, 01609, USA

Guangchen Liu, Songge Yang & Yu Zhong

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Yu Zhong .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file1 (DOCX 988 KB)

Rights and permissions.

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Liu, G., Yang, S. & Zhong, Y. High-Entropy Materials Design by Integrating the First-Principles Calculations and Machine Learning: A Case Study in the Al-Co-Cr-Fe-Ni System. High Entropy Alloys & Materials (2024). https://doi.org/10.1007/s44210-024-00041-3

Download citation

Received : 31 March 2024

Accepted : 26 June 2024

Published : 04 July 2024

DOI : https://doi.org/10.1007/s44210-024-00041-3

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• First-principles calculation
• Machine learning
• High entropy alloys
• Elastic properties
• Phase stability
• Find a journal
• Publish with us
• Track your research

• Search Menu
• Sign in through your institution
• Advance articles
• JALM Talk Podcasts
• Special Issues & Special Collections
• ADLM Guidance Documents
• Author Guidelines
• Submission Site
• Call for Papers
• Self-Archiving Policy
• Why Publish?
• Open Access
• About The Journal of Applied Laboratory Medicine
• Editorial Board
• Advertising & Corporate Services
• Journals on Oxford Academic
• Books on Oxford Academic

Article Contents

Introduction, materials and methods, supplemental material.

• < Previous

Screening for Primary Aldosteronism by Mass Spectrometry Versus Immunoassay Measurements of Aldosterone: A Prospective Within-Patient Study

• Article contents
• Figures & tables
• Supplementary Data

Sybille Fuld, Georgiana Constantinescu, Christina Pamporaki, Mirko Peitzsch, Manuel Schulze, Jun Yang, Lisa Müller, Aleksander Prejbisz, Andrzej Januszewicz, Hanna Remde, Lydia Kürzinger, Ulrich Dischinger, Matthias Ernst, Sven Gruber, Martin Reincke, Felix Beuschlein, Jacques W M Lenders, Graeme Eisenhofer, Screening for Primary Aldosteronism by Mass Spectrometry Versus Immunoassay Measurements of Aldosterone: A Prospective Within-Patient Study, The Journal of Applied Laboratory Medicine , Volume 9, Issue 4, July 2024, Pages 752–766, https://doi.org/10.1093/jalm/jfae017

• Permissions Icon Permissions

Measurements of aldosterone by mass spectrometry are more accurate and less prone to interferences than immunoassay measurements, and may produce a more accurate aldosterone:renin ratio (ARR) when screening for primary aldosteronism (PA).

Differences in diagnostic performance of the ARR using mass spectrometry vs immunoassay measurements of aldosterone were examined in 710 patients screened for PA. PA was confirmed in 153 patients and excluded in 451 others. Disease classifications were not achieved in 106 patients. Areas under receiver-operating characteristic curves (AUROC) and other measures were used to compare diagnostic performance.

Mass spectrometry-based measurements yielded lower plasma aldosterone concentrations than immunoassay measurements. For the ARR based on immunoassay measurements of aldosterone, AUROCs were slightly lower ( P = 0.018) than those using mass spectrometry measurements (0.895 vs 0.906). The cutoff for the ARR to reach a sensitivity of 95% was 30 and 21.5 pmol/mU by respective immunoassay and mass spectrometry-based measurements, which corresponded to specificities of 57% for both. With data restricted to patients with unilateral PA, diagnostic sensitivities of 94% with specificities >81% could be achieved at cutoffs of 68 and 52 pmol/mU for respective immunoassay and mass spectrometry measurements.

Mass spectrometry-based measurements of aldosterone for the ARR provide no clear diagnostic advantage over immunoassay-based measurements. Both approaches offer limited diagnostic accuracy for the ARR as a screening test. One solution is to employ the higher cutoffs to triage patients likely to have unilateral PA for further tests and possible adrenalectomy, while using the lower cutoffs to identify others for targeted medical therapy.

German Clinical Trials Register ID: DRKS00017084.

The aldosterone:renin ratio (ARR) is recommended for screening for primary aldosteronism (PA). We compared use of the ARR derived from immunoassay and LC–MS/MS measurements of aldosterone for screening in a prospective cohort study involving 604 patients (total n = 710, excluded n = 106). There were no clinically relevant differences in diagnostic accuracy between methods. Cutoffs that offered 95% sensitivity provided suboptimal low diagnostic specificities for both methods. Improved specificity was achieved with higher cutoffs to specifically identify patients with surgically curable PA and minimize unnecessary further investigations in others. Such patients may thereby benefit from improved efficiency and effectiveness of the diagnostic process.

Primary aldosteronism (PA) is defined by an inappropriately high adrenal aldosterone production that escapes physiological feedback control. This over-production is either lateralized to one adrenal gland (unilateral) or is nonlateralized. For the latter situation, patients have been referred to as having idiopathic hyperaldosteronism or bilateral PA. Unilateral disease can potentially be cured by adrenalectomy while nonlateralized PA is treated with mineralocorticoid receptor antagonists.

Screening for PA is recommended for patients with a blood pressure >150/100 mmHg or with any stage of hypertension accompanied by one or more features: hypokalemia, adrenal incidentaloma, resistance to first line antihypertensive medications, obstructive sleep apnea syndrome, or a family history of PA ( 1 ). Screening involves measurements of plasma aldosterone and renin, which are then reformulated into the aldosterone:renin ratio (ARR) as originally described by Hiramatsu et al. in 1981 ( 2 ). In the case of an abnormally elevated ARR, a confirmatory test is indicated that aims to induce a physiological suppression of aldosterone. If the confirmatory test fails to achieve this, PA is confirmed.

Screening for PA demands strict sampling conditions, including the need to withdraw antihypertensive medications that impact the renin–angiotensin–aldosterone–system ( 3 ). Most commonly, the ARR is calculated using immunoassay measurements of aldosterone and renin, now often facilitated by automated chemiluminescence immunoassays (CLIA) of both aldosterone and renin. Measurements of plasma renin activity, the rate of enzymatic conversion of angiotensinogen to angiotensin I, have largely been replaced by less labor-intensive measurements of direct renin concentrations. Apart from obviating problems associated with handling and disposal of radioactive material, CLIAs offer advantages over radioimmunoassay and mass spectrometry-based measurements of high sample throughput. However, comparability of different laboratory methods is limited, contributing to discordant recommendations on cutoff values ( 3–6 ).

CLIAs are prone to highly variable interferences from several substances in sample matrices ( 7 ). Such interferences for aldosterone are manifest by higher concentrations measured by immunoassays than by mass spectrometry ( 5 , 8–11 ). Interferences with 2 commonly used CLIAs for aldosterone from DiaSorin and Immunodiagnostic Systems are particularly prominent at low plasma concentrations of aldosterone ( 10 , 12 ). Such interferences can lead to a false-positive diagnosis in a substantial proportion of patients who undergo confirmatory tests for PA, many of whom then proceed to highly invasive and potentially unnecessary adrenal venous sampling (AVS) procedures ( 10 ). Use of liquid chromatography with tandem mass spectrometry (LC–MS/MS) avoids such interferences ( 5 , 8 , 13 , 14 ).

Although use of LC–MS/MS is clearly important for confirmatory tests, these methods are currently not widely available. It remains unclear whether immunoassay interference at higher plasma concentrations of aldosterone has any significant impact on the use of the ARR to screen for PA. The present study, involving a prospective within-patient design, therefore assessed whether mass spectrometry-based measurements of aldosterone provide a superior measure for the ARR during screening compared to immunoassay measurements. Optimal cutoffs for both methods were also assessed.

We conducted a multicenter study according to a prospective design (German Clinical Trials Register ID DRKS00017084), which as clarified in the Supplemental Appendix (in the online Data Supplement) offers numerous advantages over more common retrospective designs and represents a novel feature of the study. Patients aged 18–70 years were consecutively enrolled between April 2019 and April 2023 at 6 tertiary care centers based in Germany, Australia, Poland, and Switzerland. Inclusion of patients in the study demanded suspicion of PA based on one or more criteria outlined in the Supplemental Appendix and according to the currently accepted clinical practice guideline ( 1 ). All patients provided written informed consent under the protocol, which was approved by the ethics committees at the 6 participating centers.

Clinical Test Procedures

The specific design features of the protocol are detailed in the Supplemental Appendix, including patient flow through the protocol (see Supplemental Fig. 1 ) and the visit plan (see Supplemental Table 1 ). For interpretation of test results, the protocol requires withdrawal of antihypertensive medications known to affect the renin–angiotensin system and replacement by medications devoid of known interferences (e.g., diltiazem, verapamil, doxazosin, prazosin, hydralazine). Patients first undergo screening for PA according to measurements of plasma aldosterone and direct renin concentrations performed locally on 2 separate occasions with blood samples (10 mL in EDTA collection tubes) collected in the morning hours in the seated position. Positive test results for the ARR are based on local cutoffs as in routine use at study centers ( Supplemental Table 2 ). Plasma concentrations of aldosterone and other steroids are also measured by LC–MS/MS. Patients undergo a saline infusion test (SIT) carried out in the seated position if screening returns a positive result for the ARR. In keeping with the prospective within-patient design, patients also undergo a SIT if steroid profiles support a diagnosis of PA independent of any negative result of the ARR. This represents an important study design feature since it allows for identification of patients with false-negative results for the ARR. Anatomical imaging to search for an adenoma and AVS to determine lateralized adrenal aldosterone secretion are reserved for patients willing to undergo adrenalectomy.

Disease Classifications and Outcome Assessments

As detailed in the online Data Supplement , patients undergo clinical and biochemical outcome assessments at 6 months or more after adrenalectomy, after initial exclusion of PA or after AVS findings of nonlateralized PA. Outcomes in patients who undergo adrenalectomy are assessed based on clinical and biochemical therapeutic success, according to the PA surgical outcome criteria ( 15 ) and also outlined in Supplemental Table 3 . Among these patients, a diagnosis of unilateral PA is established according to postoperative biochemical cure, whereas PA is excluded when initially negative screening or confirmation test results are reproduced at outcome assessment.

Because outcome assessments were not possible in many patients, including those diagnosed with PA who did not undergo adrenalectomy, an extended combination of different criteria is also used for disease classification. For these extended criteria, exclusion of PA requires both negative results for the ARR and steroid profiles, or a baseline plasma aldosterone ≤162 pmol/L. For all other patients, confirmation and exclusion of PA depends on the results of the SIT according to a previously established and subsequently validated cutoff of >162 pmol/L for disease confirmation ( 10 , 16 ). All patients in whom these criteria apply are described as the “extended criteria group.”

Classifications of disease according to either outcome assessments or the extended criteria were not possible for several reasons ( Supplemental Table 4 ).

Laboratory Measurements of Aldosterone and Renin

Routine measurements of plasma aldosterone and renin were by CLIAs, including the Liaison immunoassay (DiaSorin) at 4 centers, and the iSYS immunoassay (Immunodiagnostic Systems) at 2 centers ( Supplemental Table 2 ). Where measurements of renin were recorded in metric units, they were converted to mU/L, using 1.67 as the conversion factor. Separate samples of plasma from the same blood collection tubes used for routine measurements were used to measure plasma aldosterone and other steroids at a single center according to a previously established LC–MS/MS procedure ( 17 ).

Statistical Analysis

The JMP Pro statistics software package v.17 (SAS Institute Inc.) was used for data analysis. Continuous parameters are shown as medians with interquartile ranges. For comparison of patient characteristics and biochemical parameters, χ 2 - or Kruskal–Wallis tests for nonparametric data were used. Passing–Bablok regression and Bland–Altman analysis of agreement were used to determine correlations between LC–MS/MS and immunoassay measurements. Receiver-operating characteristic (ROC) curves were generated using nominal logistic regression. Areas under ROC curves (AUROCs) were compared to assess diagnostic test performance according to confidence intervals (CIs), which for paired analyses involved the CIs for differences in AUROCs. Measurements for aldosterone, renin, and ARR were averaged when determined more than once when patients were on adjusted antihypertensive medications. Cutoffs for the ARR, associated diagnostic specificities and overall test accuracies were determined using ROC curve tables at a diagnostic sensitivity of 95% considered appropriate for a screening test. Reported P values are 2-sided unless otherwise stated.

Patient Inclusion Criteria, Exclusions, and Outcomes

The most frequent criterion for inclusion was an average blood pressure >150/100 mmHg on at least 2 visits, followed by hypertension with hypokalemia or an incidentaloma ( Supplemental Table 5 ). Among the 710 patients enrolled into the study, 106 were excluded from the analysis according to the reasons detailed in Supplemental Table 4 and clarified in Fig. 1 . According to the extended criteria, PA was confirmed in 153 patients and excluded in 451. According to outcome assessments available in 198 patients, unilateral PA was confirmed in 53 patients and PA was excluded in 119. For the other 26 patients, unilateral PA could not be confirmed, nor could PA be excluded.

Flowchart of patients who were included in the study and in the data analysis. Patients diagnosed by extended criteria included all those in whom valid screening and confirmatory testing (if indicated) were available. Diagnosis of unilateral PA as well as definitive exclusion of PA required an outcome assessment.

Patient Characteristics and Laboratory Measurements

There was a smaller ( P < 0.05) proportion of females among patients with than without PA, whereas age, BMI, blood pressure, and heart rate did not differ between groups for classifications based on extended criteria ( Table 1 ). By contrast, for classifications based on outcome assessments, systolic and diastolic blood pressures were higher ( P < 0.05) in patients with unilateral PA than in those without PA. As expected, plasma aldosterone, renin, and ARR were higher ( P < 0.0001) in patients with PA than without PA, regardless of the criteria used for disease classification.

Characteristics and biochemical test results of patients with and without PA.

All data for continuous variables are presented as medians with interquartile ranges in parentheses. BP, blood pressure.

a ARR calculated using immunoassay measurements of aldosterone.

b ARR calculated using LC–MS/MS measurements of aldosterone.

There was considerable overlap in immunoassay- and LC–MS/MS-derived values for the ARR in patients with and without PA, as classified according to the extended criteria ( Fig. 2 ). Overlap was less pronounced for comparisons of patient groups classified according to outcome assessment ( Fig. 2 ).

Dot plots of immunoassay (A, C) and LC–MS/MS (B, D) based measurements of aldosterone for the ARR in patients diagnosed with PA vs patients without PA by extended criteria (A, B) and in patients diagnosed with unilateral PA vs patients without PA diagnosed by outcome assessment (C, D).

Receiver-Operating Characteristic Curves

Independent of the criteria used for disease classification, AUROCs for the ARR showed little difference according to whether the ratio was calculated using immunoassay or LC–MS/MS measurements of aldosterone ( Fig. 3 ). Nevertheless, for classifications based on the extended criteria, the AUROC was slightly although significantly larger for the ARR based on LC–MS/MS than immunoassay measurements of aldosterone (ΔAUROC = −0.011; CI: −0.020 to −0.002; P = 0.018). Comparisons of ROC curves for patients classified with and without disease based on outcome assessments vs those based on the extended criteria indicated larger AUROCs for the former than the latter according to both measurements by LC–MS/MS (0.983, CI: 0.961–0.993 vs 0.906 CI: 0.878–0.929) and immunoassay (0.984, CI: 0.963–0.993 vs 0.895 CI: 0.865–0.919).

ROC curves for the diagnosis of PA based on extended criteria (A) and for the diagnosis of unilateral PA based on outcome assessment (B). ROC curves are displayed for measurements of aldosterone for the ARR by immunoassay (IA) or mass spectrometry (MS).

Cutoffs and Corresponding Specificities at 95% Sensitivity

To provide for a diagnostic sensitivity of 95%, cutoffs for the ARR were 30 and 21.5 pmol/mU for respective immunoassay and LC–MS/MS measurements according to ROC curve tables for patients classified with and without disease by the extended criteria ( Table 2 ). At these cutoffs, diagnostic specificities were 57% (CI: 53–62) for both methods of measurement. To achieve a similar diagnostic sensitivity among patients classified with disease according to outcome assessments, cutoffs for respective immunoassay and LC–MS/MS measurements were 68 and 52 pmol/mU. This corresponded to respective specificities of 88% (CI:77–94)% and 91% (CI:84–95)%. For the extended criteria group, those cutoffs corresponded to specificities of 81% and 84%. From the corresponding ROC curve table, lower sensitivities but higher specificities could be derived for a cutoff of 90 pmol/mU as recommended by the clinical practice guideline.

Diagnostic performance of the ARR based on either CLIA and LC–MS/MS measurements of aldosterone at 95% sensitivity.

a Cutoffs are in pmol/mU. Conversion factors: for pmol/mU to ng/mU, multiply by 0.36044; for pmol/mU to ng/ng, multiply by 0.602: for pmol/mU to pmol/ng, multiply by 1.67. The other data are presented as percentage with 95% CIs in parentheses.

b Cutoff for 95% sensitivity could not be deduced from ROC curve table, therefore, the lowest cutoff for 94% sensitivity is reported.

Differences in Diagnostic Performance According to Immunoassay and Study Center

Passing–Bablok regression and Bland–Altman agreement analyses confirmed higher ( P < 0.0001) measurements of aldosterone by immunoassay than LC–MS/MS ( Fig. 4 ). According to the latter analysis, measurements of aldosterone were 43.8% (CI: 40.3–47.4%) and 29.1% (CI: 25.4–32.8%) higher by respective Liaison and iSYS immunoassays than by LC–MS/MS ( Fig. 4 ). Comparisons of ROC curves for the ARR according to whether measurements were by the Liaison and iSYS immunoassays suggested higher diagnostic performance for the latter than the former assay ( Supplemental Fig. 3 ). For a sensitivity of 95%, the ROC curves indicated higher cutoffs for the iSYS than the Liaison immunoassay ( Supplemental Table 6 ). However, there were also differences in patient characteristics and reasons for study inclusion between the centers using the iSYS and Liaison immunoassays ( Supplemental Table 7 ). Centers using the iSYS immunoassays contributed the most to patients classified with unilateral PA. Underlying these differences, there were also differences in diagnostic performance for the ARR among individual study centers ( Supplemental Fig. 4 ) along with center-to-center differences in patient characteristics and reasons for study inclusion ( Supplemental Table 8 ).

Comparisons of LC–MS/MS and immunoassay measurements of aldosterone by Passing–Bablok regression and Bland–Altman plots among patients classified with disease according to extended criteria and restricted to measurements by Liaison (A, B) and iSYS (C, D) immunoassays. Dashed lines show relative mean differences, dotted lines show relative mean differences ±1.96SD (limits of agreement), shaded areas represent the corresponding 95% CIs. IA, immunoassay; CI, 95% confidence interval.

This study shows that LC–MS/MS and immunoassay measurements of aldosterone for screening are not associated with any clinically relevant differences in diagnostic test performance of derived ARRs. This contrasts with confirmatory testing where diagnostic accuracy is compromised by use of immunoassays ( 10 , 18 , 19 ). The present study also establishes cutoffs for the ARR based on LC–MS/MS measurements of aldosterone and immunoassay measurements of renin. Most importantly, our study establishes that for identification of patients with unilateral PA who may be cured by adrenalectomy, higher cutoffs offer a solution to minimize need for confirmatory tests and subsequent AVS. A major strength of our study involves both the large cohort size and prospective within-patient design, advantages over previous studies that employed smaller cohorts and retrospective designs ( 20–26 ). Furthermore, measurements used to define cutoffs were performed under standardized sampling conditions and after the cessation of potentially interfering medications.

To our knowledge, only 2 other studies with smaller cohorts have directly compared the ARR based on LC–MS/MS and immunoassay derived measurements of aldosterone ( 27 , 28 ). The comparator in both studies was a radioimmunoassay rather than a CLIA, which is now more commonly used in routine practice. In 1 of the 2 studies, positive screening results were not verified by confirmatory tests ( 27 ), whereas in the other there were no ROC curve comparisons ( 28 ). Nevertheless, both studies align with ours in that no notable differences in diagnostic performance of the ARR were apparent for immunoassay and LC–MS/MS measurements of aldosterone.

There have been numerous other studies that focused on the diagnostic performance of the ARR using immunoassay measurements of aldosterone ( 21 , 23–26 , 29–33 ). However, many involved retrospective data and often interfering medications were not considered or there were other inconsistencies in reported conditions of blood sampling and classifications of disease. Perhaps the most relevant and carefully controlled study was that by Jansen et al. ( 31 ). In that prospective study, ARRs with and without medication adjustment were compared, with presence or absence of PA assessed by the SIT. The AUROC for immunoassay measurements of aldosterone on standardized medications was close to that of our study. Also, from the ROC curve, specificity was only approximately 45% at a sensitivity of 95%. Thus, as in the present study, the conclusion was that the ARR was suboptimal as a screening test, stressing the need for alternatives.

Depending on the patient cohort, to reach sensitivities of 90%–100% we observed lower specificities for the ARR than otherwise widely reported. For this range of sensitivities, many previous studies reported specificities between 76% and 99%. In part, these relatively high specificities may reflect verification bias ( 20–22 , 28 , 30 ). Spectrum bias related to inclusion of normotensive volunteers rather than patients in whom PA is suspected and then excluded may also contribute to higher than true diagnostic performance ( 22 ). Avoidance of verification bias can benefit from additional tests beyond the index test to exclude disease, which in the present study was achieved by negative results of machine learning models at screening, use of the SIT for follow-up of positive test results and outcome assessments to exclude PA and verify unilateral disease in a subset of patients. Although these features reduce the likelihood of verification bias, without a gold standard for confirmation and exclusion of disease, incorrect classifications remain a problem. Inclusion of only patients who comply with the currently recommended criteria that qualify for screening for PA reduces spectrum bias; nevertheless, as illustrated in our study, this does not completely avoid this potential problem.

In our study, the potential for spectrum bias is reflected by the subgroup analysis according to immunoassay type and study center, where differences in recruited patient populations and variable management of patients may have contributed to the differences in diagnostic performance and variable cutoffs for the ARR. These findings serve to illustrate the likely basis for divergent results among studies in previous recommendations about optimal cutoffs for the ARR.

In many previous studies on diagnostic accuracy of the ARR, confirmation of PA using the SIT relied on immunoassay measurements of aldosterone ( 22–26 , 31–33 ), which may have led to a high rate of false-positive confirmatory tests and biased estimation of ARR accuracy ( 10 ). To our knowledge, ours is the first study on diagnostic accuracy of the ARR that reduced this bias using LC–MS/MS instead of immunoassay measurements of plasma aldosterone for the SIT.

The aim of a screening test is to identify most patients with a particular disease. On this basis, we considered use of a cutoff with a high sensitivity of 95% to be most appropriate. This high sensitivity, however, is achieved at the cost of a lower specificity. Our proposed cutoffs for the ARR based on either CLIA or LC–MS/MS measurements of aldosterone are in line with other findings of systematically higher plasma aldosterone concentrations by immunoassay than LC–MS/MS ( 9 , 20 , 28 ). Thus, for LC–MS/MS measurements of aldosterone, cutoffs for the ARR are lower than by immunoassay measurements. To yield 95% sensitivity, our findings suggest a cutoff of 30 pmol/mU for immunoassay vs 21–22 pmol/mU for LC–MS/MS measurements. These cutoffs for immunoassay measurements come close to those of 30–33 pmol/mU already in use by most of our participating study centers, and according to the 34 pmol/mU originally proposed by Manolopoulou ( 23 ). The 21–22 pmol/mU for LC–MS/MS measurements is close to the 26 pmol/mU cutoff proposed by Juutilainen et al. ( 27 ).

These cutoffs are, however, associated with unacceptably low specificity. We therefore propose higher cutoffs of 68 pmol/mU for CLIAs and 52 pmol/mU for LC–MS/MS measurements, which allow for identification of 95% of patients with surgically curable PA. Use of these higher cutoffs to triage patients for further testing should reduce the number of confirmatory tests and AVS procedures. These conclusions are supported by previous findings that the extent of elevations in the ARR correlate positively with the likelihood of unilateral disease ( 20 , 21 ). Furthermore, the cutoff for LC–MS/MS measurements corresponds to cutoffs of 55 pmol/mU or 46 pmol/mU proposed by Guo et al. ( 28 ) and Baron et al. ( 20 ).

A problem for all studies on diagnostic accuracy in the context of PA is the lack of a gold standard to confirm disease. A strength of our study is the detailed overview on how patients with and without PA were classified. Our criteria for disease confirmation and exclusion serve as a composite reference standard, which is one of the proposed methods to compensate for the lack of a perfect gold standard ( 34 ). Some previous studies used the SIT as a reference test for confirmation or exclusion of PA for all patients ( 21 , 29–32 ). As there is no confirmatory test available that fulfills the demands of a true gold standard and a reference test ( 35 ), we abstained from applying confirmatory tests to all patients. Low pretest probabilities as indicated by negative screening test results can lead to an elevated number of false-positive test results when the reference test is imperfect. Definitive diagnosis of PA is only possible in the case of adrenalectomy followed by a postoperative outcome assessment. However, this was only possible in a subset of our patients and it must be acknowledged that not using the same reference standard on all patients may have led to differential verification bias as also explained elsewhere ( 36 , 37 ).

To avoid unnecessary confirmatory tests and an unacceptably high demand for AVS, but without missing a significant number of patients with PA during screening, new screening tools with improved accuracy should be developed. Rather than according equal importance to aldosterone and renin as mirrored by the ARR, diagnostic approaches that reflect the pathophysiological relationship between aldosterone and renin might provide higher diagnostic accuracy. Such approaches should also take into account other potential covariates such as sex ( 22 , 38 ) and age ( 39 ).

Diagnostic studies should define clearly what form of PA is to be diagnosed. To date, the focus has been to identify patients with and without lateralized PA using the same screening tool. However, while unilateral PA, whose cure can be assessed by defined criteria ( 15 ), may fit sufficiently into the paradigm of a binary classification of “health” and “disease,” nonlateralized disease can be understood as a continuum of autonomous aldosterone secretion ( 4 ). For this, a cutoff that suggests a binary outcome seems unsuitable. One solution could be to implement screening methods aimed at a specific subtype rather than a one-size-fits-all screening method for 2 pathophysiologically and histologically different diseases, such as unilateral and nonlateralized PA.

In summary, the choice of low cutoffs for the ARR to identify patients with any form of PA is likely to lead to a high number of unnecessary confirmatory tests. A partial solution may be achieved by a shift in the major goal of screening to the identification of those patients who would benefit from surgical therapy. For other patients with nonlateralized PA or low renin hypertension, mineralocorticoid receptor antagonist therapy is effective for personalized blood pressure control.

Supplemental material is available at The Journal of Applied Laboratory Medicine online.

Nonstandard Abbreviations: ARR, aldosterone:renin ratio; PA, primary aldosteronism; AUROC, area under receiver-operating characteristic curve; AVS, adrenal venous sampling; SIT, saline infusion test; ROC, receiver-operating characteristic.

Author Contributions:   The corresponding author takes full responsibility that all authors on this publication have met the following required criteria of eligibility for authorship: (a) significant contributions to the conception and design, acquisition of data, or analysis and interpretation of data; (b) drafting or revising the article for intellectual content; (c) final approval of the published article; and (d) agreement to be accountable for all aspects of the article thus ensuring that questions related to the accuracy or integrity of any part of the article are appropriately investigated and resolved. Nobody who qualifies for authorship has been omitted from the list.

Authors’ Disclosures or Potential Conflicts of Interest:   Upon manuscript submission, all authors completed the author disclosure form .

Research Funding: The Deutsche Forschungsgemeinschaft (314061271-TRR/CRC 205-1/2) to the study and to G. Eisenhofer, G. Constantinescu, C. Pamporaki, M. Peitzsch, M. Schulze, H. Remde, M. Reincke, M. Ernst and F. Beuschlein; the Clinical Research Priority Program of the University of Zurich for the CRPP HYRENE to F. Beuschlein and M. Ernst; the Else Kröner-Fresenius Stiftung (2012_A103, 2015_A228, and 2019_A104; RISE project 060.11681) to the study and M. Reincke and S. Fuld; the Eva Luise and Horst Köhler Stiftung to S. Fuld; the Polish National Institute of Cardiology Grant 5.1/VII/23 to A. Prejbisz; the European Union Horizon 2020 research and innovation program (grant agreement no. 694913) to M. Reincke; and the Australian National Health and Medical Research Council Investigator Grant (APP1194576) to J. Yang.

Disclosures: Scientific support not directly related to this study was provided by DiaSorin Pty Ltd and Australian Medical Research Future Fund (MRFAR000172) to J. Yang, the Kurt and Senta Hermann Foundation to S. Gruber, Novo Nordisk to U. Dischinger, honoraria from Alnylam Pharmaceuticals Inc. and Recordati Rare Diseases to U. Dischinger, Habilitationsförderung für Frauen der TU Dresden (12/2021–11/2022) to C. Pamporaki, travel and meeting support for EASD 2022 from Novo Nordisk Pharma AG to M. Ernst, travel support to meeting of the Endocrine Society of Australia to G. Eisenhofer. G. Eisenhofer, M. Reincke, F. Beuschlein, M. Schulze, and C. Pamporaki are co-holders of submitted patents (WO-2022171680-A1 and German no. 2383/21DE) relevant to this study. J. Yang is Lead of Consumer Engagement Committee of the Primary Aldosteronism Foundation.

Role of Sponsor: The funding organizations played no role in the design of study, choice of enrolled patients, review and interpretation of data, preparation of manuscript, or final approval of manuscript.

Acknowledgments: Thanks are extended to Katharina Langton, Carola Kunath, Jimmy Rusdian Masjkur, Catherine He, Zhong Lu and Ramona Walter for support with patients and to Thomas Baumgartner for coordination of Zurich patient specimens and data handling.

Funder   JW , Carey   RM , Mantero   F , Murad   MH , Reincke   M , Shibata   H , et al.    The management of primary aldosteronism: case detection, diagnosis, and treatment: an endocrine society clinical practice guideline . J Clin Endocrinol Metab   2016 ; 101 : 1889 – 916 .

Google Scholar

Hiramatsu   K , Yamada   T , Yukimura   Y , Komiya   I , Ichikawa   K , Ishihara   M , et al.    A screening test to identify aldosterone-producing adenoma by measuring plasma renin activity: results in hypertensive patients . Arch Intern Med   1981 ; 141 : 1589 – 93 .

Li   X , Goswami   R , Yang   S , Li   Q . Aldosterone/direct renin concentration ratio as a screening test for primary aldosteronism: a meta-analysis . J Renin Angiotensin Aldosterone Syst   2016 ; 17 : 147032031665745 .

Mehdi   A , Rao   P , Thomas   G . Our evolving understanding of primary aldosteronism . Cleve Clin J Med   2021 ; 88 : 221 – 7 .

Ray   JA , Kushnir   MM , Palmer   J , Sadjadi   S , Rockwood   AL , Meikle   AW . Enhancement of specificity of aldosterone measurement in human serum and plasma using 2D-LC–MS/MS and comparison with commercial immunoassays . J Chromatogr B Analyt Technol Biomed Life Sci   2014 ; 970 : 102 – 7 .

Pizzolo   F , Corgnati   A , Guarini   P , Pavan   C , Bassi   A , Corrocher   R , Olivieri   O . Plasma aldosterone assays: comparison between chemiluminescence-based and ria methods . Clin Chem   2006 ; 52 : 1431 – 2 .

Ghazal   K , Brabant   S , Prie   D , Piketty   ML . Hormone immunoassay interference: a 2021 update . Ann Lab Med   2021 ; 42 : 3 – 23 .

Turpeinen   U , Hämäläinen   E , Stenman   UH . Determination of aldosterone in serum by liquid chromatography–tandem mass spectrometry . J Chromatogr B Analyt Technol Biomed Life Sci   2008 ; 862 : 113 – 8 .

Hinchliffe   E , Carter   S , Owen   LJ , Keevil   BG . Quantitation of aldosterone in human plasma by ultra high performance liquid chromatography tandem mass spectrometry . J Chromatogr B Analyt Technol Biomed Life Sci   2013 ; 913–914 : 19 – 23 .

Eisenhofer   G , Kurlbaum   M , Peitzsch   M , Constantinescu   G , Remde   H , Schulze   M , et al.    The saline infusion test for primary aldosteronism: implications of immunoassay inaccuracy . J Clin Endocrinol Metab   2022 ; 107 : e2027 – 36 .

Brown   JM , Auchus   RJ , Honzel   B , Luther   JM , Yozamp   N , Vaidya   A . Recalibrating interpretations of aldosterone assays across the physiologic range: immunoassay and liquid chromatography-tandem mass spectrometry measurements under multiple controlled conditions . J Endocr Soc   2022 ; 6 : bvac049 .

Constantinescu   G , Bidlingmaier   M , Gruber   M , Peitzsch   M , Poitz   DM , van Herwaarden   AE , et al.    Mass spectrometry reveals misdiagnosis of primary aldosteronism with scheduling for adrenalectomy due to immunoassay interference . Clin Chim Acta   2020 ; 507 : 98 – 103 .

Mckenna   TJ , Sequeira   SJ , Heffernan   A , Chambers   J , Cunningham   S . Diagnosis under random conditions of all disorders of the renin-angiotensin-aldosterone axis, including primary hyperaldosteronism . J Clin Endocrinol Metab   1991 ; 73 : 952 – 7 .

van der Gugten   JG , Holmes   DT . Quantitation of aldosterone in serum or plasma using liquid chromatography-tandem mass spectrometry (LC-MS/MS) . Methods Mol Biol   2022 ; 2546 : 45 – 54 .

Burrello   J , Burrello   A , Stowasser   M , Nishikawa   T , Quinkler   M , Prejbisz   A , et al.    The primary aldosteronism surgical outcome score for the prediction of clinical outcomes after adrenalectomy for unilateral primary aldosteronism . Ann Surg   2020 ; 272 : 1125 – 32 .

Stowasser   M , Ahmed   AH , Cowley   D , Wolley   M , Guo   Z , McWhinney   BC , et al.    Comparison of seated with recumbent saline suppression testing for the diagnosis of primary aldosteronism . J Clin Endocrinol Metab   2018 ; 103 : 4113 – 24 .

Peitzsch   M , Dekkers   T , Haase   M , Sweep   FCGJ , Quack   I , Antoch   G , et al.    An LC–MS/MS method for steroid profiling during adrenal venous sampling for investigation of primary aldosteronism . J Steroid Biochem Mol Biol   2015 ; 145 : 75 – 84 .

Fuss   CT , Brohm   K , Kurlbaum   M , Hannemann   A , Kendl   S , Fassnacht   M , et al.    Confirmatory testing of primary aldosteronism with saline infusion test and LC-MS/MS . Eur J Endocrinol   2021 ; 184 : 167 – 78 .

Thuzar   M , Young   K , Ahmed   AH , Ward   G , Wolley   M , Guo   Z , et al.    Diagnosis of primary aldosteronism by seated saline suppression test -variability between immunoassay and HPLC-MS/MS . J Clin Endocrinol Metab   2020 ; 105 : e477 – 83 .

Baron   S , Amar   L , Faucon   AL , Blanchard   A , Baffalie   L , Faucard   C , et al.    Criteria for diagnosing primary aldosteronism on the basis of liquid chromatography-tandem mass spectrometry determinations of plasma aldosterone concentration . J Hypertens   2018 ; 36 : 1592 – 601 .

Morimoto   R , Ono   Y , Tezuka   Y , Kudo   M , Yamamoto   S , Arai   T , et al.    Rapid screening of primary aldosteronism by a novel chemiluminescent immunoassay . Hypertension   2017 ; 70 : 334 – 41 .

O’Shea   PM , Griffin   TP , Browne   GA , Gallagher   N , Brady   JJ , Dennedy   MC , et al.    Screening for primary aldosteronism using the newly developed IDS-iSYS® automated assay system . Pract Lab Med   2017 ; 7 : 6 – 14 .

Manolopoulou   J , Fischer   E , Dietz   A , Diederich   S , Holmes   D , Junnila   R , et al.    Clinical validation for the aldosterone-to-renin ratio and aldosterone suppression testing using simultaneous fully automated chemiluminescence immunoassays . J Hypertens   2015 ; 33 : 2500 – 11 .

Corbin   F , Douville   P , Lebel   M . Active renin mass concentration to determine aldosterone-to-renin ratio in screening for primary aldosteronism . Int J Nephrol Renovasc Dis   2011 ; 4 : 115 – 20 .

Bernini   G , Moretti   A , Orlandini   C , Berti   P , Miccoli   P , Bardini   M , et al.    Plasma and urine aldosterone to plasma renin activity ratio in the diagnosis of primary aldosteronism . J Hypertens   2008 ; 26 : 981 – 8 .

Tiu   SC , Choi   CH , Shek   CC , Ng   YW , Chan   FKW , Ng   CM , Kong   APS . The use of aldosterone-renin ratio as a diagnostic test for primary hyperaldosteronism and its test characteristics under different conditions of blood sampling . J Clin Endocrinol Metab   2005 ; 90 : 72 – 8 .

Juutilainen   A , Savolainen   K , Romppanen   J , Turpeinen   U , Hämäläinen   E , Kemppainen   J , et al.    Combination of LC-MS/MS aldosterone and automated direct renin in screening for primary aldosteronism . Clin Chim Acta   2014 ; 433 : 209 – 15 .

Guo   Z , Poglitsch   M , McWhinney   BC , Ungerer   JPJ , Ahmed   AH , Gordon   RD , et al.    Aldosterone LC-MS/MS assay-specific threshold values in screening and confirmatory testing for primary aldosteronism . J Clin Endocrinol Metab   2018 ; 103 : 3965 – 73 .

O’Shea   PM , Griffin   TP , Denieffe   S , Fitzgibbon   MC . The aldosterone to renin ratio in the diagnosis of primary aldosteronism: promises and challenges . Int J Clin Pract   2019 ; 73 : e13353 .

Burrello   J , Monticone   S , Buffolo   F , Lucchiari   M , Tetti   M , Rabbia   F , et al.    Diagnostic accuracy of aldosterone and renin measurement by chemiluminescent immunoassay and radioimmunoassay in primary aldosteronism . J Hypertens   2016 ; 34 : 920 – 7 .

Jansen   PM , Van Den Born   BJH , Frenkel   WJ , De Bruijne   ELE , Deinum   J , Kerstens   MN , et al.    Test characteristics of the aldosterone-to-renin ratio as a screening test for primary aldosteronism . J Hypertens   2014 ; 32 : 115 – 26 .

Pilz   S , Keppel   MH , Trummer   C , Theiler-Schwetz   V , Pandis   M , Borzan   V , et al.    Diagnostic accuracy of the aldosterone–to–active renin ratio for detecting primary aldosteronism . J Endocr Soc   2019 ; 3 : 1748 – 58 .

Vorselaars   WMCM , Valk   GD , Vriens   MR , Westerink   J , Spiering   W . Case detection in primary aldosteronism: high-diagnostic value of the aldosterone-to-renin ratio when performed under standardized conditions . J Hypertens   2018 ; 36 : 1585 – 91 .

Umemneku Chikere   CM , Wilson   K , Graziadio   S , Vale   L , Allen   AJ . Diagnostic test evaluation methodology: a systematic review of methods employed to evaluate diagnostic tests in the absence of gold standard—an update . PLoS One   2019 ; 14 : e0223832 .

Leung   AA , Symonds   CJ , Hundemer   GL , Ronksley   PE , Lorenzetti   DL , Pasieka   JL , et al.    Performance of confirmatory tests for diagnosing primary aldosteronism: a systematic review and meta-analysis . Hypertension   2022 ; 79 : 1835 – 44 .

de Groot   JAH , Dendukuri   N , Janssen   KJM , Reitsma   JB , Bossuyt   PMM , Moons   KGM . Adjusting for differential-verification bias in diagnostic-accuracy studies: a Bayesian approach . Epidemiology   2011 ; 22 : 234 – 41 .

Alonzo   TA , Brinton   JT , Ringham   BM , Glueck   DH . Bias in estimating accuracy of a binary screening test with differential disease verification . Stat Med   2011 ; 30 : 1852 – 64 .

Burrello   J , Amongero   M , Buffolo   F , Sconfienza   E , Forestiero   V , Burrello   A , et al.    Development of a prediction score to avoid confirmatory testing in patients with suspected primary aldosteronism . J Clin Endocrinol Metab   2021 ; 106 : 1708 – 16 .

Buffolo   F , Burrello   J , Burrello   A , Heinrich   D , Adolf   C , Müller   LM , et al.    Clinical score and machine learning-based model to predict diagnosis of primary aldosteronism in arterial hypertension . Hypertension   2021 ; 78 : 1595 – 604 .

Supplementary data

Email alerts

Companion article.

• Correction to: Screening for Primary Aldosteronism by Mass Spectrometry Versus Immunoassay Measurements of Aldosterone: A Prospective Within-Patient Study

Citing articles via

• Recommend to Your Librarian
• Advertising and Corporate Services
• Journals Career Network

Affiliations

• Online ISSN 2475-7241
• Copyright © 2024 Association for Diagnostics & Laboratory Medicine
• About Oxford Academic
• Publish journals with us
• University press partners
• What we publish
• New features  
• Open access
• Institutional account management
• Rights and permissions
• Get help with access
• Accessibility
• Advertising
• Media enquiries
• Oxford University Press
• Oxford Languages
• University of Oxford

Oxford University Press is a department of the University of Oxford. It furthers the University's objective of excellence in research, scholarship, and education by publishing worldwide

• Copyright © 2024 Oxford University Press
• Cookie settings
• Cookie policy
• Privacy policy
• Legal notice

This Feature Is Available To Subscribers Only

Sign In or Create an Account

This PDF is available to Subscribers Only

For full access to this pdf, sign in to an existing account, or purchase an annual subscription.

J.D. Power 2024 U.S. Initial Quality Study: American car makers fare well in major study

The Big Three automakers fared well in the J.D. Power 2024 U.S. Initial Quality Study released Thursday, with General Motors and Stellantis, which makes Dodge, Chrysler, Jeep, Ram and Fiat vehicles, finishing near the top for quality, along with some Asian and European premium brands.

Ram ranked as the highest brand overall for quality, followed by GM's Chevrolet brand and then Hyundai in third place. Ford vehicles showed a marked improvement over last year, when it did not rank high on quality. For the premium brands, Porsche, Lexus and Genesis came in first, second and third respectively.

The study is respected and its results are often tied to executive performance bonuses because the rankings objectively assess the industry.

Here's what you need to know about the J.D. Power 2024 U.S. Initial Quality Study.

J.D. Power 2024 U.S. Initial Quality Study methods

The study rates quality by "problems per 100 vehicles" in the first 90 days of ownership. This is the first year that the study incorporated franchise dealership repair visits with its "voice of the customer" data to create a more expansive metric.

The industry average this year was 195 problems per 100 vehicles — put another way, it means that the average new vehicle had 1.95 customer-reported problems, Frank Hanley, senior director of auto benchmarking at J.D. Power, told the Detroit Free Press — a part of the USA TODAY Network.

That's a modest decline in quality from a year-ago when the survey found 192 problems per 100 vehicles. The lower the score, the higher the vehicle's quality. Last year, Dodge, Ram and Buick took the top three spots.

J.D. Power 2024 U.S. Initial Quality Study best cars

See what J.D Power named best in its category:

Best vehicle brands and scores (mass market)

• Ram with a score of 149
• Chevrolet with a score of 160
• Hyundai with a score of 162

Best vehicle brands and scores (premium)

• Porsche with a score of 172
• Lexus with a score of 174
• Genesis with a score of 184

GM came in high for quality across the most models

Of GM's four brands, only Chevrolet and Buick ranked above industry average with scores of 160 and 164 problems per 100 vehicles, respectively. GMC and Cadillac came above the industry average. GMC had a score of 201 and Cadillac 214. Nevertheless, J.D. Power said GM won six model-level awards, the most of all the automakers.

GM models ranked highest in their segment :

• Cadillac XT5 midsize SUV (Lexus RX second)
• Cadillac XT6 upper midsize SUV (Porsche Cayenne second)
• Chevrolet Equinox midsize SUV (GMC Terrain second)
• Chevrolet Silverado Heavy Duty pickup (Ford Super Duty second, Ram 2500/3500 third)
• Chevrolet Tahoe large SUV (Chevrolet Suburban second)
• Chevrolet Traverse midsize SUV (Ford Explorer second)

The other automakers to win the most model awards were Hyundai and Toyota, each with four awards. Hyundai models that rank highest in their respective segment are Genesis G80, Hyundai Santa Cruz, Kia Carnival and Kia Forte.

Toyota models that rank highest in their respective segment are Lexus IS, Lexus LC, Lexus UX and Toyota Camry. Toyota has the highest-ranking model overall, the Lexus LC, with 106 PP100

Among brands, Chevrolet received the most segment awards (four), followed by Lexus (three).

Cutting edge cars, cutting edge problems

But the study found that new technology on vehicles actually hindered some automakers' ability to maintain high quality.

The study showed EVs and plug-in hybrids require more repairs than gasoline-powered vehicles in all the repair categories.

Electric carmaker Polestar topped the list with 316 problems per hundred cars. EV automakers Tesla and Rivian rated poorly too, with a score of 266 each.

Tesla recall: Thousands of Cybertrucks recalled for issues with wipers, trunk bed trim

“Owners of cutting edge, tech-filled (fully electric vehicles) and (plug-in hybrids) are experiencing problems that are of a severity level high enough for them to take their new vehicle into the dealership at a rate three times higher than that of gas-powered vehicle owners,” Hanley said.

Gasoline and diesel-powered vehicles average 180 problems per 100 vehicles, while EVs are 86 points higher at 266. In the past, Tesla has performed better, J.D. Power's study noted. But the removal of traditional feature controls, such as turn signals and wiper stalks, has not been well received by Tesla customers, the study said.

Car dealerships down: Restoration underway after auto dealer software supplier hacked

Key complaints that literally stink

J.D. Power said customers key complaints on quality centered on technology. For example, frustration around false warnings as car owners did not understand what some bells and whistles meant.

"For instance, rear seat reminder technology, designed to help vehicle owners avoid inadvertently leaving a child or pet in the rear seat when exiting the vehicle, contributes 1.7 problems per 100 across the industry," the study stated. "Some mistakenly perceive it signals an unbuckled seat belt or cite the warning goes off when no one is present in the rear seat."

Also, advanced driver assistance systems, which are intended to save lives, irritated car owners with inaccurate alerts from rear cross traffic warnings and reverse automatic emergency braking, a newly added feature to the survey this year.

Also, problems with Android Auto and Apple CarPlay persist as the feature remains one of the top 10 problems as customers experience difficulties connecting to their vehicle or losing connection. More than half of Apple users and 42% of Samsung users access their respective feature each time they get behind the wheel, which indicated that customers want their smartphone experience brought into the vehicle and be integrated wirelessly, the study said.

Features, controls and displays were another problematic category in the study, rating slightly better than the issue-prone infotainment category. From activating the windshield wipers to the more intricate operation of an manufacturer's smartphone application, this category is particularly troublesome in EVs, the study noted.

Finally, customers reported unpleasant interior smells. The issue has worsened the most from 2023, with every brand except Kia and Nissan having an increase in unpleasant interior smell problems. Problem odors are described by owners to be emanating from their vehicle’s heating, ventilating and air conditioning systems.