how to write a case study in public health

News alert: UC Berkeley has announced its next university librarian

Secondary menu

• Log in to your Library account
• Hours and Maps
• Connect from Off Campus
• UC Berkeley Home

Search form

Oomph library resources: phw 250/250b epidemiologic methods: epidemiologic case study resources.

• Online Books on Epidemiology and Biostatistics
• R for Public Health
• Stata Resources and Tips
• Epidemiologic Case Study Resources
• Rural Health Resources
• Help/Off-Campus Access

Epidemiologic Case Studies

• Epidemiologic Case Studies (US CDC) These case studies are interactive exercises developed to teach epidemiologic principles and practices. They are based on real-life outbreaks and public health problems and were developed in collaboration with the original investigators and experts from the Centers for Disease Control and Prevention (CDC). The case studies require students to apply their epidemiologic knowledge and skills to problems confronted by public health practitioners at the local, state, and national level every day.
• Case Studies (WHO) From "Strengthening health security by implementing the International Health Regulations," each case has learning objectives and documentation.
• Case Studies in Social Medicine A series of Perspective articles from the New England Journal of Medicine that highlight the importance of social concepts and social context in clinical medicine. The series uses discussions of real clinical cases to translate theories and methods for understanding social processes into terms that can readily be used in medical education, clinical practice, and health system planning.
• African Case Studies in Public Heath Case study exercises based on real events in African contexts and written by experienced Africa-based public health trainers and practitioners. These case studies represent the most up-to-date and context-appropriate case study exercises for African public health training programs. These exercises are designed to reinforce and instill competencies for addressing health threats in the future leaders of public health in Africa.
• Case Consortium @ Columbia University: Public Health Cases The case collection includes "teaching" cases. Nearly all the cases are multimedia and based on original research; a few are written from secondary sources. All cases are offered free of charge.
• Epi Teams Training: Case Studies From the North Carolina Institute for Public Health, this curriculum includes several interactive case studies designed be used by the Epi Team as a group. These case studies are based on actual outbreaks that have occurred in North Carolina and elsewhere.
• National Center for Case Study Teaching in Science The mission of the NCCSTS at the University at Buffalo is to promote the development and dissemination of materials and practices for case teaching in the sciences. Our website provides access to an award-winning collection of peer-reviewed case studies. We offer a five-day summer workshop and a two-day fall conference to train faculty in the case method of teaching science. In addition, we are actively engaged in educational research to assess the impact of the case method on student learning. "Case Collection" includes over 100 public health cases.

Books of Case Studies

• << Previous: Stata Resources and Tips
• Next: Rural Health Resources >>
• Last Updated: Feb 20, 2024 9:33 AM
• URL: https://guides.lib.berkeley.edu/publichealth/PHW250

Welcome To Open Case Studies

Connecting you with real-world public health data.

The Open Case Studies project showcases the possibilities of what can be achieved when working with real-world data.

Housed in a freely accessible GitHub repository, the project’s self-contained and experiential guides demonstrate the data analysis process and the use of various data science methods, tools, and software in the context of messy, real-world data.

These case studies will empower current and future data scientists to leverage real-world data to solve leading public health challenges.

Who Are Open Case Studies For?

Your experiential guide to the power of data analysis.

The Open Case Studies project provides insights about gathering and working with data for students, instructors, and those with experience in data science or statistical methods at nonprofit organizations and public sector agencies.

Each case study in the project focuses on an important public health topic and introduces methods to provide users with the skills and knowledge for greater legibility, reproducibility, rigor, and flexibility in their own data analyses.

Case Study Bank Overview

Real data on ten public health challenges in the U.S.

The following in-depth case studies use real data and focus on five areas of public health that are particularly pressing in the United States.

Vaping Behaviors in American Youth

This case study explores the trends of tobacco product usage among American youths surveyed in the National Youth Tobacco Survey (NYTS) from 2015-2019. It demonstrates how to use survey data and code books and provides an introduction to writing functions to wrangle similar but slightly different data repetitively. The case study introduces packages for using survey weighting and survey design to perform an analysis to compare vaping product usage among different groups, and covers how to use a logistic regression to compare groups for a variable that is binary (such as true or false — in this case it was using vaping products or not). This case study also covers how to make visualizations of multiple groups over time with confidence interval error bars.

Opioids in the United States

This case study examines the number of opioid pills (specifically oxycodone and hydrocodone, as they are the top two misused opioids) shipped to pharmacies and practitioners at the county-level around the United States from 2006 to 2014 using data from the Drug Enforcement Administration (DEA). This case study demonstrates how to get data from a source called an application programming interface (API). It explores why and how to normalize data, as well as why and how to potentially stratify or redefine groups. It also shows how to compare two independent groups when the data is not normally distributed using a test called the Wilcoxon rank sum test (also called the Mann Whitney U test) and how to add confidence intervals to plots (using a method called bootstrapping).

Disparities in Youth Disconnection

This case study focuses on rates of youth (people between 16-24) disconnection (those who are neither working nor in school) among different racial, ethnic and gender subgroups to identify subgroups that may be particularly vulnerable. It demonstrates that deeper inspection of subgroups yields some differences that are not otherwise discernable, how to import data from a PDF using screenshots of sections of the PDF, and how to use the Mann-Kendall trend test to test for the presence of a consistent direction in the relationship of disconnection rates with time. This case study also shows how to make a visualization that stylistically matches that of an existing report, how to add images to plots, and how to create effective bar plots for multiple comparisons across several groups.

Mental Health of American Youth

This case study investigates how the rate of self-reported symptoms of major depressive episodes (MDE) has changed over time among American youth (age 12-17) from 2004-2018. It describes the impact of self-reporting bias in surveys, how to get data directly from a website, as well as how to compare changes in the frequency of a variable between two groups using a chi-squared test to determine if two variables are independent (in this case if the sex of the students influenced the frequency of reported MDE symptoms in 2004 and 2018). This case study also demonstrates how to create direct labels on visualizations with many groups across time, as well as how to create an animated gif.

Exploring CO2 Emissions Across Time

This case study investigates how CO2 emissions have changed since the 1700s and how the level of emissions has compared for different countries around the world. It explores how yearly average temperature and the number of natural disasters in the United States has changed over time and provides an introduction for examining if two sets of data are correlated with one another. This case study also goes into great detail about how to make what are called heatmaps and other plots to visualize multiple groups over time. This includes adding labels directly to lines on plots with multiple lines.

Predicting Annual Air Pollution

This case study uses machine learning methods to predict annual air pollution levels spatially within the United States based on data about population density, urbanization, road density, as well as satellite pollution data and chemical modeling data among other predictors. Machine learning methods are used to predict air pollution levels when traditional monitoring systems are not available in a particular area or when there is not enough spatial granularity with current monitoring systems. The case study also demonstrates how to visualize data using maps.

Exploring Global Patterns of Obesity Across Rural and Urban Regions

This case study compares average Body Mass Index measurements for males and females from rural and urban regions from over 200 countries around the world, with a particular emphasis on the United States. It provides a thorough introduction to wrangling data from a PDF, how to compare two paired groups using the t test and the nonparametric Wilcoxon signed-rank test using R programming, and how to make visualizations of group comparisons that emphasize a particular subset of the data.

Exploring Global Patterns of Dietary Behaviors Associated with Health Risk

This case study investigates the consumption of dietary factors associated with health risk among males and females from over 200 countries around the world, with a particular emphasis on the United States. It demonstrates how to wrangle data from a PDF; how to combine data from two different sources; how to compare two paired groups and multiple paired groups using t-tests, ANOVA, and linear regression; and how to create visualizations of several groups and how to combine plots together with very different scales.

Influence of Multicollinearity on Measured Impact of Right-To-Carry Gun Laws

This case study focuses on two well-known studies that evaluated the influence of right-to-carry gun laws on violent crime rates. It demonstrates a phenomenon called multicollinearity, where explanatory variables that can predict one another can lead to aberrant and unstable findings; how to make visualizations with labels, such as arrows or equations; and how to combine multiple plots together.

School Shootings in the United States

This case study illustrates ways to communicate trends in a dataset about the number and characteristics of school shooting events for students in grades K-12 in the United States since 1970. It demonstrates how to create a dashboard, which is a website that shows patterns in a dataset in a concise manner; how to import data from a Google Sheets document; how to create interactive tables and maps; and how to properly calculate percentages for data when there are missing values.

Which Case Study Is Right For Me?

Connecting with the public health data you need.

The Open Case Studies project approaches data in many different ways. The guide below will help connect you with a case study:

Data science projects often start with a question. Here, you may look for case studies that explore a question that is similar to one you are interested in investigating with your data.

How does something change over time?

Investigating how a variable has changed over time can help identify consistent trends.

How do survey responses compare for different groups over time?

Survey data requires special care and attention to the survey design.

How do groups compare?

Public health researchers are often interested to know if one group is more vulnerable than another or if two or more groups are actually different from one another.

How do groups compare over time?

Comparing several groups over time can provide insight into if the change over time is different for different groups.

How do paired groups compare?

Paired groups are those that are not independent in some way. Perhaps you want to know how data from the same person over time compares with that of another person over time, or perhaps you are interested in how something changed in a city before and after an intervention, or perhaps you want to compare groups using data that has structure where there is coupling or matching of data values across samples.

Are certain groups or possibly subgroups more vulnerable?

Understand how to compare subpopulations at a deeper level.

How does something compare across regions?

Often it is useful to investigate if data differs by region, as many environmental, cultural, and political differences can influence public health outcomes.

How can I predict outcomes for new data?

Learn how the data might look next year or for locations that you don’t have data about.

Does this influence my data?

Analyze how a variable influences another variable.

Are these two variables related to one another?

Understand how two variables are related and how strongly they are related to one another.

How can I display this data for others to find and interpret and use easily?

Make it easy for others to find your data, see the major trends in your data, or search for specific values in your data.

Data can come from many different sources, from the more obvious like an excel file to the less obvious like an image or a website. These case studies demonstrate how to use data from a variety of possible sources.

Using data from a PDF or just parts of a PDF can be challenging. You could type the data into a new excel file, but this can result in mistakes and it is difficult to reproduce.

Data are often in CSV files and it is typically easy to import data and work with data in this form. However, sometimes it can be difficult if, for example, the first few lines are structured differently or if you have unusual missing value indicators.

If you find data on a website that doesn’t allow you to download in a convenient way, you can actually directly import the data into R programming language.

This is one of the most common data forms, and it is typically easy to import data and work with data in this form. However, sometimes it can be challenging, especially if you have many files.

You can extract text from image files. This can be useful if, for example, you want to only use certain parts of a PDF.

It is possible to find the data that you need to use from an application programming interface (API).

Google Sheet

You can download data from a Google Sheet, copy and paste it into Excel, or directly import the data into R programming language.

Survey data/Code books

Working with survey data requires special care and attention, and you can do this directly with R programming language.

Multiple files

If you find that you need to import data from multiple files, there is a more efficient way to do so without importing each one by one.

Data wrangling is the process of organizing your data in a more useful format. These case studies explore how to clean, rearrange, reshape, modify, filter, combine, or join your data.

Extracting data from a PDF

Extracting and organizing data from a PDF will make it easier to use.

Geocoding data

The process of assigning relevant latitude and longitude coordinates to data values is called geocoding. This can be helpful (although not always necessary) to create a map of your data.

Recoding data

If you have data values that are confusing and could be changed to something better, or if you want to convert your data to true or false, you might want to consider recoding these values.

Methods of joining data

Sometimes, you obtain data from multiple sources that need to be combined together.

Filtering data

Perhaps you need to filter your data for only specific values for given variables. In other words, you might want to filter census employment data to only values for females who are also Black and live in Connecticut.

Modifying data (normalizing, transforming, scaling etc.)

Sometimes it is difficult to know when or how to normalize data.

Working with text

You can work with, remove, replace, or change words, phrases, letters, numbers, or punctuation marks in your data.

Reshaping data

Sometimes it is useful to shape your data so that you have many columns (for example, when performing certain analyses), however it can be useful at other times (for example, when creating plots) to collapse multiple columns into fewer columns with more rows.

Repetitive process

Sometimes you need to wrangle multiple datasets from different sources in a similar manner.

A picture is worth a thousand words, particularly when it comes to interpreting data. These case studies demonstrate how to make effective visualizations in various contexts. The first ten represent basic visualizations while 11-22 are more advanced.

A table that is easy to interpret

Adding colors or simple graphics can make tables easier to interpret.

Scatter plot

Scatter plots can be a strong option for evaluating the relationship between variables, and especially for evaluating changes in a variable over time.

Line plots are often useful for evaluating changes over time.

Bar plots are a good choice if you want to compare data to a threshold.

Box plots are particularly useful for comparing groups with many data values. They provide information about the spread of the data.

Pie chart/waffle plot

Pie charts or waffle plots can be a strong option when comparing relative percentages.

It can be difficult to visualize multiple groups at simultaneously. In these situations, heat maps can be a great option.

Correlation plots

If you have many variables and need to know if they are correlated to one another, there are methods to efficiently check this.

Visualize missing data

It can be helpful to quickly identify how much of your data is missing (has NA values).

Create a map of your data

Often the best way to interpret regional differences in data is to make a map.

• Advanced Visualizations

Matching a style

If you are working with collaborators, you can make your visualizations match the style of their figures.

Faceted plots allow you to quickly create multiple plots at once

It can be difficult to visualize multiple groups at the same time, so faceted plots are a great option in this situation.

Adding labels directly to plots with many different groups

If you compare many groups over time, for example, it can be difficult to see which line corresponds to which group. Adding labels directly to these lines can be very helpful and negates the need for an overcomplicated legend.

Emphasize a particular group

Sometimes you will have several different groups and you want to highlight a specific group.

Adding annotations to plots

Adding labels, such as thresholds, arrows, or equations, can make it easier for people to interpret your plot.

Add error bars to your plot

Adding error bars can help convey information about the confidence of the estimates in your plots.

Combine multiple plots together

Sometimes it is useful to put a variety of plots together and add text to explain what the plot shows.

Create an interactive plot when you have too many groups to label

If you compare a very large number of groups, it can be difficult to tell what is happening. Often it can help to make the plot interactive so that the user can hover over points or lines to see what they indicate.

Create an interactive map of your data

Sometimes it is easiest to see regional differences by interacting with and exploring an interactive map.

Create an interactive table of your data

Sometimes you might want to be able to search through your data or allow others to easily do so.

Add images to your figures

Including images to a plot, such as a logo, can be a helpful addition.

Create an interactive dashboard/website for your data

Dashboards can quickly convey major trends in a dataset, and they can also allow users to interact with the data to choose what aspects about the data they wish to explore.

To better understand data, it is helpful to use statistical tests. These case studies demonstrate a variety of statistical tests and concepts.

Are two groups different?

Correlation

Are two variables related to one another?

Are multiple groups different?

Linear regression

Would you like to compare groups?

Chi-squared test of independence

Do the frequencies of two groups suggest that they are independent?

Mann-Kendall Trend test

Is there a consistent change over time?

Machine learning

Would you like to predict data?

Calculate percentages with missing data?

Would you like to calculate percentages, but you are missing some data?

About The Project

Learn about the team behind the Open Case Studies project.

As part of the larger Open Case Studies project (OCS) at opencasestudies.org , these case studies were developed for and funded by the Bloomberg American Health Initiative. The OCS project is made up of a team of researchers at the Johns Hopkins Bloomberg School of Public Health (JHSPH).

Let us know how the Open Case Studies project has enhanced your educational curriculum or ability to tackle tough data-rich research projects.

JHSPH Faculty Contributors

Jessica Fanzo, PhD

Brendan Saloner, PhD

Megan Latshaw, PhD, MHS

Renee M. Johnson, PhD, MPH

Daniel Webster, ScD, MPH

Elizabeth Stuart, PhD

Bloomberg American Health Initiative

Joshua M. Sharfstein, MD – Director, Bloomberg American Health Initiative

Michelle Spencer, MS – Associate Director, Bloomberg American Health Initiative

Paulani Mui, MPH – Special Projects Officer, Bloomberg American Health Initiative

Other Contributors

Aboozar Hadavand, PhD, MA, MS, Minerva University

Roger Peng, PhD, MS, Johns Hopkins Bloomberg School of Public Health

Kirsten Koehler, PhD, MS, Johns Hopkins Bloomberg School of Public Health

Alex McCourt, PhD, JD, MPH, Johns Hopkins Bloomberg School of Public Health

Ashkan Afshin, MD, ScD, MPH, MSc, University of Washington and Institute for Health Metrics and Evaluation (IHME)

Erin Mullany, BA, Institute for Health Metrics and Evaluation (IHME)

External Review Panel

Leslie Myint, PhD, Macalester College

Shannon E. Ellis, PhD, University of California – San Diego

Christina Knudson, PhD, University of St. Thomas

Michael Love, PhD, University of North Carolina

Nicholas Horton, ScD, Amherst College

Mine Çetinkaya-Rundel, PhD, University of Edinburgh, Duke University, RStudio

Let Us Know How You're Using Open Case Studies

As the Open Case Studies project expands, we learn from you. Tell us what data you'd like to see, how you're using the data, or anything we can do to improve the project.

CURRICULUM, INSTRUCTION, AND PEDAGOGY article

Designing an interactive field epidemiology case study training for public health practitioners.

• RTI International, Research Triangle Park, NC, United States

Globally, public health practitioners are called upon to respond quickly and capably to mitigate a variety of immediate and incipient threats to the health of their communities, which often requires additional training in new or updated methodologies or epidemiologic phenomena. Competing public health priorities and limited training resources can present challenges in developed and developing countries alike. Training provided to front-line public health workers by ministries of health, donors and/or partner organizations should be delivered in a way that is effective, adaptable to local conditions and culture, and should be an experience perceived as a job benefit. In this review, we share methods for interactive case-study training methodologies, including the use of problem-based scenarios, role-play activities, and other small-group focused efforts that encourage the learner to discuss and synthesize the concepts taught. We have fine-tuned these methods through years of carrying out training of all levels of public health practitioners in dozens of countries worldwide.

Background and Rationale

In a rapidly changing field marked by frequent turnover, the need to provide continuing education for public health practitioners is constant. Often there is no requirement for prior credentialing in public health for local-level jobs 1 , 2 . Alternatively, training and professional development may be clinically-oriented as opposed to the more relevant population based public health focus ( 1 ). In many parts of the world public health training programs targeting the health professions are minimal ( 2 ). For these reasons, in the US and elsewhere, public health professions tend to be high-responsibility, low-pay jobs, marked by high turnover rates ( 3 ).

In many low- and middle-income countries, keeping educated health professionals in-country is challenging ( 4 ). Public health professionals are discouraged by lack of proper compensation and insufficient opportunities for continuing professional development and education ( 5 ). Results from a 2006 study conducted for the Ugandan Ministry of Health investigating health workforce morale, satisfaction, motivation, and intent to remain in Uganda showed that health workers were dissatisfied with their jobs. About one in four noted a desire to leave the country to improve their outlook ( 6 ). Further complicating matters, those who have learned occupational skills through on-the-job experience are vulnerable to being laid off with changes of government administration ( 7 ).

Our experience carrying out training activities over the last two decades has provided the opportunity to train public health practitioners in-person, online, and via self-directed learning. We have designed and conducted trainings for front-line local responders as well as district and national-level officials both in the US and abroad. For example, in the US, we have trained county and state-level public health workers in surveillance and outbreak investigation through in-person training courses, Web conferencing platforms, and via both synchronous and asynchronous distance-based methods. We have trained national-level officials and laboratorians through specialized in-person trainings as well as via large conference-based and train-the-trainer style settings. Examples include organizing and developing content for a multi-agency tabletop exercise focused on bioterrorism response post-9/11, development of a curriculum focused on rapid response to avian and pandemic influenza, strengthening workforce capacity in sentinel site surveillance for respiratory illness, and, with the US Centers for Disease Control and Prevention (CDC), a full Master's-degree level curricula the Central American Field Epidemiology Training Program (FETP) Master's degree program, as well as front-line public health practitioner trainings at Intermediate and Basic-levels within Central America and topic-specific FETP modules elsewhere. These efforts were supported by CDC, the National Council of State and Territorial Epidemiologists (CSTE), the World Health Organization (WHO), the Pan-American Health Organization (PAHO), and others.

Whether delivered to a non-epidemiologist who finds themselves responsible for outbreak investigation, or to a national epidemiology official who will subsequently oversee training of their staff, the case study or interactive exercise typically serves as a pivotal feature of training for public health professionals, moving them toward actively engaging in learning, knowledge sharing and in the application of previously learned information ( 8 ). We describe key elements in the development and use of case studies as a learning tool for public health practitioners, based on our years of experience in developing, implementing, and evaluating public health trainings. The learning objectives of this article are the following:

• Identify critical teaching points around which to structure a case study training.

• Consider relevant sources of information to provide information and “plot” to a case study.

• Choose a case study format appropriate for the trainees and content.

• Incorporate design elements that encourage interaction and thoughtful discussion among learners into a case study.

Pedagogical Framework

Case-study based learning approaches have proven particularly effective in the medical education context. Comparison studies conducted in the medical sciences have found interactive, case-study focused learning a more effective teaching tool than didactic lectures ( 9 , 10 ). Although the literature focuses on medical education, the success of case studies in public health education is also reflected in the growing number of public health programs with problem-based curricula ( 11 , 12 ). Our goal has been to create effective training materials for public health professionals that present relevant, interactive, and interesting content that teach job skills (not merely general principles) in a way that can be quickly understood and applied while providing an opportunity for peers of different experience levels to learn from one another.

Curriculum development is a multi-layered process involving instructional designers, subject matter experts, and editorial and art personnel ( 13 ). Training format is highly dependent upon the needs of the involved partner(s). Information-driven training presented in the didactic style only can be less effective for information retention and learner motivation. Learning methods allowing the participant to contribute meaningfully from their own experience, develop skills in a supervised setting, and practice skills immediately pursuant to learning them, will result in more retention and ability to apply the skills taught. Learners who are given ample opportunities to analyze past experience through reflection, evaluation and reconstruction fulfill a key element of experience-based learning and are able to draw deeper meaning from prior events in their personal and professional lives ( 14 ). As such, interactive and problem-based approaches to building applied public health skills are critical features of many public health trainings. Interactive and problem-based approaches engender openness toward new experiences, a critical element in facilitating lifelong learning. At the social or group level, these approaches help emphasize critical social action and the importance of adopting a stance shaped by moral and socio-political responsibility ( 14 ).

Dissemination of information or didactic methods may be used as a means of communicating concepts, methods, or the use of new technologies, but should be accompanied by interactive teaching methods. Interactive teaching can occur in a variety of settings, which may be dependent on program, resources, and learner characteristics. The training audience is professional public health practitioners, who may or may not be formally educated in public health.

The size of the training has varied vastly, from fewer than 12 learners to over 100. With larger groups, we often have a lead instructor who is experienced both in the subject area and in carrying out trainings who is accompanied by “facilitators” or small-group leaders who can provide guidance during case-study work in break-out sessions.

Although we have designed trainings in many different formats, the most successful setting for a case study in our experience has been one where learners come together in-person for a scheduled period of instruction and worked activities, ranging in time from half-day trainings to 2-week courses.

The success of a particular training methodology is difficult to assess. For the several dozen trainings we have carried out, we include a participant evaluation that captures information such as whether the participant found the training useful, whether they feel able to use the information learned in their job, how satisfied they were with the training, and open comments. Informal interaction with trainees and feedback from these surveys over the years has shaped the case study format presented here. The success of a public health training should be measurable by participants' ability to perform specific job functions, and more broadly, improvements in components of the public health system such as timeliness and completeness of communicable disease reporting and surveillance.

Blueprints for an Interactive Curriculum

There are several considerations when planning and developing curriculum materials. First, the needs of the intended target audience (i.e., the learners) should form the core of any development process. The curriculum should be designed to both meet training and skill-building needs, and to approach the target audience in a comfortable and accessible way; it should meet them at their current level; the materials should address the fact that—trained or not—professionals who have been working in any field have experience to contribute that adds to the depth of a training. Second, training materials should be developed in a way that ensures consistency in the delivery. Thus, lectures should include a speaker script and built-in questions for the audience with suggestions to both engage learners and continually check knowledge acquisition. Having the script provides particular flexibility for train-the-trainer programs or other circumstances where the person with the most content and training expertise may not be the one to personally deliver the content to all trainees. Case studies should be developed in a standard format that includes an instructor guide complete with suggested answers, worked calculations, and suggested instructional methods for emphasizing critical teaching points. These methods ensure that critical content is taught regardless of the depth of experience of the teacher or facilitator. Finally, training materials should engage the learner in discussion and healthy debate, contribute to their own learning, and provide peer-to-peer learning opportunities. We discuss several types of interactive exercises which are often collectively referred to as “case studies,” but specifically may feature different elements (Table 1 ).

Table 1 . Terminology, definition, and examples of interactive training activities often referred to as “case studies.”

We propose the following general objectives of a group case study, regardless of format or topic area:

• To meet defined learning objectives or competencies.

• To learn in an environment that respects the skills and abilities of the learners, and allows each learner to use and apply their unique skills.

• To have learners interact with each other, thus enhancing their own learning through listening to the ideas and experience of fellow learners.

• To exhibit genuine interest in and be motivated by the content they are learning.

We have found that there are four critical conceptual elements in developing and executing an interactive training:

1. Reference key points or concepts to be learned and applied.

2. Guide learners to work through a problem.

3. Keep learners engaged through active questioning.

4. Draw on learners' experience.

Each of these is described in detail below, with a measles outbreak in a small African village used as a guide for demonstrating their implementation through a case study exercise.

Reference Key Points or Concepts to Be Learned and Applied

The main intent of a case study is to teach predefined content. Often training delivery of a case study is preceded by didactic content, or other information-driven training such as web-based tutorials, workbooks, or reading material. Whether or not information-driven training is included, the key concepts to be learned and applied should be viewed as the framework upon which the rest of the case study will be built. For example, a lecture or reading may put forth a set of 9 steps to investigate an outbreak. The case study format will then be dictated by achieving each of these 9 steps. It is of benefit to clearly and obviously delineate any key steps or phases as one works through the case study. The beginning learner gains more benefit from understanding a firm case study structure than from trying to figure out which step or concept should be applied in which situations, even though events may not occur so neatly on the job. When teaching a set of concepts, it is better to teach them clearly and simply than to allow the learner to become frustrated, struggle, and possibly fumble. More advanced learners can be given the opportunity to struggle with decision-making and unexpected sequences of events, after they have mastered a clear set of tools from which to draw. A few examples of clearly structured concepts that lend themselves to case studies are given in Table 2 , and a scenario using these types of structures within a case study is given in Table 3 .

Table 2 . Examples of concepts that can be given a clear sequence or structure for teaching purposes.

Table 3 . Scenario-based examples of a measles outbreak in a small village case study.

Regardless of topic, a simple introduction for the participant and the instructor as to the point and process of the case study is required. Begin by providing learning objectives (as in Table 3A ) and a short introduction to the case study, including if, and how, learners should interact with others, the resources they should use for case study completion, the product that is expected, and the estimated time required to complete the tasks given.

Guide Learners to Work Through a Problem

A hallmark feature of a case study is working through a problem that parallels or is based on a real-life situation. An appealing format is to present the learner with introductory or background information and update information as the learner works through the problems of the case study.

Basing information and scenarios on real events is the best way to prevent the scenario from feeling contrived. Possible sources of information for scenarios include the experience, internal reports, or data of the curriculum developer or colleagues; published literature; local, national, or international bulletins; health situation updates provided by State health departments, CDC or WHO; and less formal sources such as news reports and ProMed, the Program for Monitoring Emerging Diseases. In some cases, the details of an actual outbreak, surveillance system, or other event can be used directly. Typically, an actual scenario can be used, but the details can be embellished, drawn out, or supplemented by different events in order to meet the learning purposes of the case study. For example, in a scenario where the learner is in the role of someone getting ready to plan a case-control study, the setting from a ProMed report or international bulletin might be ideal, but the details given from a case-control study published in the literature provide the topical details needed to expand upon the case study scenario. Unless teaching about a specific real-event is a learning objective, a case study developer should be free to modify details and provide creative license to developing characters for the case study events. The example case study used in Table 3 of this article was inspired by a health news article about difficulties facing vaccination campaigns in Guinea 3 , and was structured on WHO recommendations for measles surveillance and case definitions ( 15 ). Resource material used to develop the case study should be appropriately cited.

Once the learning objectives, introduction, and plot for the scenario are outlined, the learner or the group is positioned within the scenario through assignment of specific roles or responsibilities, for example:

• “You are the health director for Mlima Province, and you receive a phone call from the local hospital epidemiologist who is concerned about….”

• “You are a surveillance officer with the Forêt Health district, and you have just been placed in charge of developing a new surveillance system for…”

The plot of the case study can then be developed as questions are posed, the learner makes plans, decisions, calculations, and more. An example presentation of the scenario based on the previously established measles scenario is given in Table 3B .

Keep Learners Engaged Through Active Questioning

A key to a successful case study is placing learners in an active role, rather than asking them to reproduce information-driven training concepts in the context of the scenario. Although a case study is designed to be a classroom tool, in field epidemiology the case study should strive to put the learner in the field. For data-centered trainings, this can be accomplished by instructing the learner to carry out data processing and analysis. For example, a learner may be provided with a line listing and asked to construct an epidemic curve, or they may be provided a two-by-two table and asked to calculate relative risk. The effort to place the learner in an active role is particularly important for more routine types of questions that aim to get the learner to repeat concepts (as with an information-driven training). Some examples of inactive questions and how they can be modified to be active are shown in Table 4 .

Table 4 . Comparison of scenarios with inactive vs. active questioning methods.

Questions should cause the learner to use the knowledge and information they have gleaned both within and outside of the training to “act” or make a decision but should not affect the ultimate plot of the case study. See an example from the measles case study in Table 3C .

Draw on Learners' Experience

Adult learners are motivated by experiences that affect them personally. Facilitating contributory participation increases their personal investment ( 16 ). For a group case study, there are multiple advantages in using the case study to draw out learners' experience and personal perspective. First, when learners contribute their own experience to the case study, they gain confidence and enjoyment throughout the activity. Second, a case study that assumes learners have no experience or contribution may feel pedantic or even condescending. Third, encouraging the learner to share their ideas, opinions, and experience creates a synergistic learning opportunity that the questions on the page alone cannot accomplish. This can be done by asking discussion or reflection questions during the progress of the exercise and at its conclusion. Questions that aim to solicit information from the learner regarding past experiences he or she has in making decisions or navigating situations like those encountered in the case study can be effective in enhancing the learning experience. These types of questions can also encourage beneficial self-reflection and allow for recognition of how skills have been or could be applied in reality. Alternatively, questions that treat issues or controversies with no perfect answer fit into the context of the case study while drawing on learners' opinions, often leading to rich dialogue and experience sharing. Table 3D provides a scenario-based example of how to draw upon learner experiences relevant to the measles case study.

Interactive Training Design Options

The specific design used will depend on training content, setting, and the number of learners. Below, we discuss several designs in the context of smaller groups (e.g., 9 or fewer individuals) as well as larger groups (e.g., 10–30 individuals). In many cases, larger groups can be accommodated with small-group case study designs. Using “breakout” groups, where a classroom of a couple dozen or more breaks into smaller groups for case study work, is the easiest method to conduct participatory case-study based training. In many cases, small modifications can be made that also take advantage of the larger class size.

Problem-Based Scenarios

Problem-based scenarios are the mainstay of case study trainings. They guide learners through carrying out activities and can be flexible in length, depending on the depth of information covered and activities or computations required.

Small Groups

In a problem-based scenario, a situation or background information is presented, and the individual or small group must work through a series of questions that address learning objectives in the context presented. Questions ask learners to provide information, conduct a calculation, or come to a decision and move on to the next question. Additional information that adds to the scenario may be provided once or several times throughout the case study.

Large Groups

A couple variations on the problem-based case study scenario make them more interesting in the large group setting. One option is to create slightly different scenarios for breakout groups. For example, in a case study where groups are designing surveillance systems according to set principles taught in class, each small group can focus on a different set of diseases or conditions for their surveillance system. Alternatively, small groups can be provided with updates unique to their group. While this requires additional work in developing case study answer guides, small groups can present their results to each other at the end of the session and can learn more about public health challenges and considerations than just the work they themselves have performed.

Role plays are excellent tools for practicing scenarios in which the learner must think of what to say or do, and work well for interview situations and meeting scenarios. They are best used among learners who have already spent some time together, so that fewer inhibitions exist.

While many case studies ask the learner to assume they have a specific role or identity in order to answer case study questions, a role-play asks a group of learners to go a step further by carrying out interactions with other group members from the perspective of the role they are playing. To prevent learners from feeling uncomfortable in carrying out their role play, each “role” should contain specific guidance with information about their character, including general terms about what they should say and do in the role play.

Acting out an interview can provide inexperienced learners an opportunity to practice and offers experienced learners an opportunity to add improvisational content from their own experience. Scenarios may have learners conduct interviews with food workers, business proprietors, case-patients, research subjects, hospital personnel, and more. Additionally, they may be the interviewee with members of the media or government officials. Both the person conducting the interview and the person being interviewed are given guidance that can include questionnaires or question topic domains for the interviewer, a personality profile of their roles, background information for the person answering questions, top priority concerns for stakeholders, and any other details relevant to the situation. Additionally, role players can be given guidance to cause challenges for each other. For example, an interviewee's information sheet can tell them that they do not like to discuss “personal” problems, and they should do their best to avoid directly answering questions that ask for sensitive information.

Town hall meetings, stakeholder meetings, or media events can be simulated when roles relevant to the situation are assigned to different group members. For example, group members could be in a scenario dealing with student health, and members can be assigned the roles of school officials, parents, health department personnel, or students. Information sheets for each role can encourage the role player to be at odds with others to encourage discussion, bring up concepts taught in the class, and rationalize their actions relevant to the scenario. The value derived from role playing is in practicing personal interactions and the process of considering various viewpoints.

Role plays can be carried out within the context of the large group by dividing the audience into breakout groups, each representing a group of people, such as health spokespeople, media, or community members. The entire group would have the same pre-defined list of character traits or concerns, but individuals within the group voicing their own perspective provide lively interaction. Another option is to have each small group carry out the same role play, with one person per role, and then to follow the role play with a large group discussion. The instructor can ask questions such as, “For those of you who conducted the interview, what was the most challenging aspect of obtaining the information you needed?” Debriefing encourages learners to process what they have learned and allows them to share their own experiences in similar circumstances.

Create a Common Product

This exercise is particularly useful when the skills to complete a large task are being taught, such as skills for writing reports or designing surveillance systems.

Although it is not feasible, in a classroom setting, to have learners complete a large task at their desk, as a group they can efficiently combine forces to cover key concepts from the teaching and produce a common paper, outline, graph, or presentation that addresses key points. Simple example assignments to a small group include:

• Given the scenario, create a flow diagram of a surveillance system that collects population-based pneumonia and influenza data.

• Write the outline of a bulletin article that summarizes the outbreak investigation methods and results that we have worked through today.

• With each group member taking responsibility for one section of a protocol, write an outline for key content to be included in each section. Then share your results with your group members and solicit their feedback.

Breakout groups creating a common product can provide an added dimension and present their product to the entire class. In these cases, it is beneficial to ask the class for constructive criticism on each other's work. Audience members can rate the presenting group against how well they addressed class concepts or met defined criteria. We have used Olympic-style score cards for this task, which lessens the risk of “speaking out” from the viewpoint of the learner offering feedback, but also gives the instructor impetus to ask specific individuals for their rationale in providing that score.

“Go Out” Assignments

Although the ability to use this exercise is highly dependent upon the training setting, “going out” is an interactive way to solidify concepts in questionnaire design, questionnaire execution, and data collection.

Small or Large Groups

If a training is being held in a location where learners have easy and safe access to a public lunch room, university students, or a busy walkway or plaza, the learner can go out of the classroom and engage the general public in practicing basic interviewing skills and piloting questions from a data collection instrument. For example, the learner can collect data to bring back to the classroom, such as whether people are wearing hats, so they can create a collective distribution diagram during a biostatistics lesson. A specific and somewhat tight time limit should be given with the understanding that this is a part of the training, not a break.

Presentations From the Field

Where instruction is being given to individuals representing a variety of expertise levels, presentations by two or three learners who have more experience or expertise can provide a highly educational perspective. For example, in a training we carried out on strengthening population-based influenza surveillance, influenza officers from countries with a strong surveillance system were asked ahead of time to present on the design, site selection, case selection, and limitations or barriers in their surveillance systems. Other learners, on hearing the presentations, asked specific logistical and operational questions and observed how concepts being taught in a lecture really did apply to the “real world.” This helps build professional networks that learners can turn to for expertise or advice post-training.

Much like the practice of field epidemiology itself, there is an art and a science to producing a case study. We lay out a process for structuring a case study around key teaching points, finding elements to include in a case study plot, and incorporating interactive activities and methods throughout a case study, acknowledging that real-world situations may not always follow a predefined sequence.

First, ensure that the goals and content of the training are compatible with the goals of a case study mentioned earlier in this paper. Case studies lend themselves well to situations in which the learners have some experience working in their designated fields to enhance their participation. Clear didactic materials (whether lecture, job aid, or other format) for learners to refer to can keep the discussion focused appropriately. When a group of learners comes together, the discussion can flounder or become derailed by lack of clarity in the underlying concepts. Thus, it is important that any didactic materials also be carefully crafted. An invited or guest speaker for didactic materials can be beneficial but can also confuse learners if concepts are presented differently than in the case study or other training materials. If guest speakers are delivering didactic content, we have found that it is easiest to provide them slides covering the critical points. This ensures the speaker meets the required objectives, and provides a benefit to the course as the speaker can give subject matter feedback to curriculum developers. When teaching complex or involved ideas, we have found that having an additional checklist or conceptual aid handy helps keep the discussion on track (and the training on time).

Second, consider the amount of time for the training, characteristics of the learners and training content, and select a format. For long trainings, change the case study format to prevent learner fatigue. However, do think about contextual factors when selecting a format. In one multi-day training we carried out, learners were to complete a “go-out” assignment. However, this assignment was near the end of the week (learners were tired), and we chose an hour just before lunch to complete the assignment so that students would be able to interact with a larger number of patrons at the cafeteria at the training site. However, most of our learners were distracted by a coffee and a snack. While they eventually completed the assignment, we lost almost an hour of training time. Other considerations for choosing a format include number of participants, level of learner expertise, cultural sensibilities in interacting with one another, and availability of teaching and support staff.

Third, identify key steps and teaching points for inclusion as questions or processes during the case study. If the training content is not easily related to a set of steps or a process, it should be tailored to the identified learning objectives and should ensure that the “action” verbs from learning objectives are carried out. Write questions around these points that coincide with the plot or unfolding of the scenario. Professional experience or published literature can provide a basis for scenario development.

Fourth, ensure that questions are clear and action-oriented. Always provide an answer key or main teaching points that should be derived from each question in an instructor copy. Estimate that it will take about 10 min for a group to read, discuss, and record answers to each question.

In settings where breakout groups are utilized, it is recommended there be a facilitator embedded with each group to help moderate, ensure all have a chance to participate, that appropriate effort is being exerted, and that learners generally arrive at the intended answers.

Participatory case studies are a beneficial way of delivering training for professionals who need to learn, reinforce, and apply specific skills to carry out their job duties. In the field of Public Health, where the responsibilities and hours can be significant, the challenge of recruiting and maintaining dedicated workers must be met in order to protect and promote healthy communities. Training public health professionals to learn new skills and encouraging them to share their experience with others through a network-building group case study is an opportunity that is valuable for our public health work force as well as for our communities.

Author Contributions

AN wrote the manuscript with significant contribution from LB. PM conceptualized the manuscript. All authors provided content expertise, contributed to manuscript revision, read, and approved the submitted version.

This publication was supported, in part, by the Cooperative Agreement Number 1U19GH001591 funded by the U.S. Centers for Disease Control and Prevention (CDC). Its contents are solely the responsibility of the authors and do not necessarily represent the official views of the Centers for Disease Control and Prevention or the Department of Health and Human Services.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

The authors wish to acknowledge the insight and feedback from partners and trainees over the past two decades.

1. ^ Fairfax County Virginia: Public Health Nursing at the Fairfax County Health Department. http://www.fairfaxcounty.gov/hd/careers-in-public-health/public-nursing-jobs.htm (2016) (Accessed July 11, 2016).

2. ^ Summit County Public Health: Careers in Public Health. Careers in Public Health. http://scphoh.org/pages/careers.html (2015) (Accessed July 11, 2016).

3. ^ IRINNews.org Human Rights: Vaccination teams defeat “Ebola effect” in Guinea (2015). http://www.irinnews.org/news/2015/04/29 (Accessed August 13, 2018).

1. Anyangwe SCE, Mtonga C. Inequities in the global health workforce: the greatest impediment to health in sub-saharan Africa. Int J Env Res Pub Health (2007) 4:93–100. doi: 10.3390/ijerph2007040002

PubMed Abstract | CrossRef Full Text | Google Scholar

2. The World Health Report 2006 - Working Together for Health. WHO (2006). Available online at: http://www.who.int/whr/2006/en/ (Accessed September 29, 2015).

3. Pourshaban D, Basurto-Dávila R, Shih M. Building and sustaining strong public health agencies. J Pub Health Manag Pract. (2015) 21 (Suppl. 6):80–90. doi: 10.1097/PHH.0000000000000311

4. Mackey TK, Liang BA. Rebalancing brain drain: exploring resource allocation to address health worker migration and promote global health. Health Policy (2012) 107:66–73. doi: 10.1016/j.healthpol.2012.04.006

CrossRef Full Text | Google Scholar

5. Cancedda C, Farmer PE, Kerry V, Nuthulaganti T, Scott KW, Goosby E, et al. Maximizing the impact of training initiatives for health professionals in low-income countries: frameworks, challenges, and best practices. PLoS Med. (2015) 12:6. doi: 10.1371/journal.pmed.1001840

6. Hagopian A, Zuyderduin A, Kyobutungi N, Yumkella F. Job Satisfaction and morale in the ugandan health workforce. Health Affairs (2009) 28:w863−75. doi: 10.1377/hlthaff.28.5.w863

7. Bradshaw YW, Ndegwa SN. The Uncertain Promise of Southern Africa. Bloomington: Indiana University Press (2000).

Google Scholar

8. Dicker RC. Case studies in applied epidemiology. Pan Afr Med J. (2017) 27 (Suppl. 1):1. doi: 10.11604/pamj.supp.2017.27.1.12886

9. Yunus M, Karim HR, Bhattacharyya P, Ahmed G. Comparison of effectiveness of class lecture versus workshop-based teaching of basic life support on acquiring practice skills among the health care providers. Int J Crit Illn Inj Sci . (2016) 6:61–4. doi: 10.4103/2229-5151.183018

10. Faisal R, Rehman Kur, Bahadur S, Shinwari L. Problem-based learning in comparison with lecture-based learning among medical students. J Pakistan Med Assn . (2016) 66:650–3.

PubMed Abstract | Google Scholar

11. Harvard T.H. Chan School of Public Health: Teaching by Case Method (2018). Available online at: www.hsph.harvard.edu/ecpe/programs/teaching -by-case-method (Accessed Aug 13, 2018).

12. Sibbald SL, Speechley M, Amardeep T. Adapting to the needs of the public health workforce: an integrated case-based training program. Front Public Health (2016) 4:221. doi: 10.3389/fpubh.2016.00221

PubMed Abstract | CrossRef Full Text

13. Baseman JG, Marsden-Haug N, Holt VL, Stergachis A, Goldoft M, Gale JL. Epidemiology competency development and application to training for local and regional public health practitioners. Pub Health Rep . (2008) 123 (Suppl 1):44–52. doi: 10.1177/00333549081230S111

14. Andresen L, Boud D, Cohen R. Experience based learning. In: Foley, G editor. Understanding Adult Education and Training , 2nd ed. Sydney, NSW: Allen and Unwin (2001). p. 225–39.

15. PAHO. Measles Elimination Field Guide. 2nd ed. Washington, DC: PAHO (2005) Available online at: http://www1.Paho.Org/English/Ad/Fch/Im/Fieldguide_Measles.Pdf

16. Zemke R, Zemke S. Adult learning: what do we know for sure? ERIC (1995) 32:31–4.

Keywords: epidemiology, public health, workforce development, training, case study

Citation: Nelson AL, Bradley L and MacDonald PDM (2018) Designing an Interactive Field Epidemiology Case Study Training for Public Health Practitioners. Front. Public Health 6:275. doi: 10.3389/fpubh.2018.00275

Received: 02 July 2018; Accepted: 05 September 2018; Published: 26 September 2018.

Reviewed by:

Copyright © 2018 Nelson, Bradley and MacDonald. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY) . The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

*Correspondence: Amy L. Nelson, [email protected]

Disclaimer: All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.

Skip to content

Read the latest news stories about Mailman faculty, research, and events. 

Departments

We integrate an innovative skills-based curriculum, research collaborations, and hands-on field experience to prepare students.

Learn more about our research centers, which focus on critical issues in public health.

Our Faculty

Meet the faculty of the Mailman School of Public Health. 

Become a Student

Life and community, how to apply.

Learn how to apply to the Mailman School of Public Health. 

Cases Studies in Public Health Available Online

First study of mers animal host in saudi arabia.

Case studies aren’t just for business schools anymore. Columbia University’s Mailman School of Public Health has been using the case method to teach MPH students as part of the new Columbia Public Health Curriculum. Now, six of the School’s public health cases have been published online, making them available to classrooms anywhere.

“The case method can be a powerful tool for learning public health. It gives students the opportunity to gain experience making decisions in the face of uncertainty, much as they will have to do every day when they graduate and leave us to work in their chosen field,” says Melissa Begg, ScD , Vice Dean for Education at the Mailman School, who is leading the implementation of the new MPH curriculum.

Each case study tells a detailed story which stops mid-action, asking students to imagine themselves into the shoes of a decision-maker facing a tough call. One case developed with the help of David Abramson, PhD , assistant professor of Sociomedical Sciences, looks at the decision of whether to evacuate two hospitals during Superstorm Sandy. Another looks at how to win a community’s trust, as told through the experience of Mailman scientists conducting a federally-funded study of arsenic-tainted water in Bangladesh.

Classroom discussions are lively, and most important, there isn’t a single right answer. “Students practice taking positions and defending them based on the available evidence, while developing communication and critical thinking skills,” explains Dr. Begg. “They learn to argue persuasively for their points of view.

The Mailman School case studies are available through Columbia University’s Case Consortium website, which also features cases by Columbia’s Journalism School and the School for International and Public Affairs (SIPA). They are available free (after registration) to educators and at a nominal cost to students, professionals, and other interested parties. 

“While most existing case curriculum remains paper-based, the Mailman School cases are online and multimedia, meeting students where they live in the digital media world,” says Kirsten Lundberg, MPA, director of the Case Consortium @ Columbia.

The “discussion-based” case study approach has historic roots reaching back to Socrates, and was popularized in the 20th century by business schools. To see the cases, please visit  Case Consortium  and click on Cases.

Public Health Case Studies

Voluntary or Regulated? The Trans Fat Campaign in New York City

This case takes students behind the scenes in the world of public health policymaking.  Students follow the New York City Department of Health and Mental Hygiene, and the process it went through to craft a policy to reduce public consumption of trans fats in restaurants. In 2005, after considerable internal negotiations, the department’s Bureau of Chronic Disease Prevention and Control elected to launch a public awareness campaign aimed equally at consumers, restaurants and their suppliers. But after a year, the awareness campaign had not budged the rate of trans fat use in restaurants. In 2006, the department decided to resort to regulation, despite the risks of triggering protests of a “nanny state,” not to mention pushback from industry.

When BEST Intentions Go Awry: Arsenic Mitigation in Bangladesh

This case is about a public health response to the widespread arsenic contamination of groundwater in Bangladesh. It examines the lead-up to a 2008 media crisis that confronted a Columbia University clinical trial of a potential treatment for arsenic poisoning. The case raises for discussion the challenges of conducting research in rural, less developed and culturally insular communities. It also asks how to help communities while studying them—complicated by funding restrictions and a possible skewing of results. 

Community Savings, or Community Threat? California Policy for Ill and Elderly Inmates

This case looks at the challenges that confront public health professionals who work in a corrections environment. By 2011, a court-appointed Receiver had made progress in fixing a broken system of medical care for prisoners in California. But costs spiraled ever higher for elderly and ailing inmates. Public health officials had to balance competing public priorities: save taxpayer dollars while treating patients. A new law allowed the sickest prisoners to move to community-based care—but now public health doctors had to decide: who qualified for medical parole?

Credible Voice: WHO-Beijing and the SARS Crisis

This case study takes students into the Beijing office of the World Health Organization as it dealt with the SARS crisis in early 2003. The WHO serves as the world’s monitor of disease outbreak and control. It is able to mobilize legions of the world’s best scientists to analyze, diagnose, prescribe treatments for and contain diseases. However, it depends on the cooperation of the countries experiencing an epidemic. What happens when that cooperation is limited or nonexistent?

The Elusive Tuberculosis Case: The CDC and Andrew Speaker

This case explores the ethical and logistical challenges that doctors face when an infectious disease patient does not cooperate with advice to stay out of public spaces in order to protect the general welfare. In April 2007, a young Atlanta lawyer, Andrew Speaker, was diagnosed with active tuberculosis. Initially cooperative, Speaker departs without notice for Greece and his scheduled wedding even though it is clear that his strain of TB is more lethal and difficult-to-treat than anticipated. 

Evacuate or Stay? Northshore LIJ and Hurricane Sandy

This case examines the pros and cons of evacuating medical facilities in the face of a looming natural disaster. In October 2012, the North Shore-Long Island Jewish Hospitals (North Shore-LIJ) network braced—together with rest of the East coast—for the advent of Hurricane Sandy. Weather forecasters painted a grim picture, and North Shore-LIJ had three hospitals in low-lying areas. Vice President of Protective Services James Romagnoli and COO Mark Solazzo had seen this scenario only a year earlier, when in August 2011 they evacuated hospitals in advance of Hurricane Irene. But Irene had, at the last moment, spared New York City. With that unnecessary evacuation fresh in their minds, the two officials had to decide what to do as Sandy approached.

• Utility Menu

harvardchan_logo.png

Harvard T.H. Chan School of Public Health Case-Based Teaching & Learning Initiative

Teaching cases & active learning resources for public health education, case writing.

An overview of the case planning and writing process, by experienced case-writer and CBTL workshop leader Kirsten Lundberg.

2019. The Case Centre . Visit website A non-profit clearing house for materials on the case method, the Case Centre holds a large and diverse collection of cases, articles, book chapters and teaching materials, including the collections of leading business schools across the globe.

Austin, J. , 1993. Teaching Notes: Communicating the Teacher's Wisdom , Harvard Business School Publishing. Publisher's Version "Provides guidance for the preparation of teaching notes. Sets forth the rationale for teaching notes, what they should contain and why, and how they can be prepared. Based on the experiences of Harvard Business School faculty."

Abell, D. , 1997. What makes a good case? . ECCHO–The Newsletter of the European Case Clearing House , 17 (1) , pp. 4-7. Read online "Case writing is both art and science. There are few, if any, specific prescriptions or recipes, but there are key ingredients that appear to distinguish excellent cases from the run-of-the-mill. This technical note lists ten ingredients to look for if you are teaching somebody else''s case - and to look out for if you are writing it yourself."

Roberts, M.J. , 2001. Developing a teaching case (abridged) , Harvard Business School Publishing. Publisher's Version A straightforward and comprehensive overview of how to write a teaching case, including sections on what makes a good case; sources for and types of cases; and steps in writing a case.

• Writing a case (8)
• Writing a teaching note (4)
• Active learning (12)
• Active listening (1)
• Asking effective questions (5)
• Assessing learning (1)
• Engaging students (5)
• Leading discussion (10)
• Managing the classroom (4)
• Planning a course (1)
• Problem-based learning (1)
• Teaching & learning with the case method (14)
• Teaching inclusively (3)

• MEMBER DIRECTORY
• Member Login
• Publications
• Clinician Well-Being
• Culture of Health and Health Equity
• Fellowships and Leadership Programs
• Future of Nursing
• U.S. Health Policy and System Improvement
• Healthy Longevity
• Human Gene Editing
• U.S. Opioid Epidemic
• Staff Directory
• Opportunities
• Action Collaborative on Decarbonizing the U.S. Health Sector
• Climate Communities Network
• Communicating About Climate Change & Health
• Research and Innovation
• Culture of Health
• Fellowships
• Emerging Leaders in Health & Medicine
• Culture & Inclusiveness
• Digital Health
• Evidence Mobilization
• Value Incentives & Systems
• Substance Use & Opioid Crises
• Reproductive Health, Equity, & Society
• Credible Sources of Health Information
• Emerging Science, Technology, & Innovation
• Pandemic & Seasonal Influenza Vaccine Preparedness and Response
• Preventing Firearm-Related Injuries and Deaths
• Vital Directions for Health & Health Care
• NAM Perspectives
• All Publications
• Upcoming Events
• Past Events
• MEMBER HOME

Systems Thinking for Public Health: A Case Study Using U.S. Public Education

ABSTRACT | The initial response to the COVID-19 pandemic in the United States largely focused on addressing the immediate health consequences from the emergent pathogen. This initial focus often ignored the related impacts from the pandemic and from mitigation measures, including how existing social determinants of health compounded physical, social, and economic impacts on individuals who have historically been marginalized. The consequences of decisions around closing and reopening primary and secondary (K–12 in the United States) public schools exemplify the complex impacts of pandemic mitigation measures. Ongoing COVID-19 mitigation and recovery efforts have gradually begun addressing indirect consequences, but these efforts were slow to be identified and adopted through much of the acute phase of the pandemic, mirroring the decades-long neglect of contributors to the overall health and well-being of populations that have been made to be vulnerable.

A systems approach for decision-making and problem solving holistically considers the effects of complex interacting factors. Taking a systems approach at the start of the next health emergency could encourage response strategies that consider various competing public health needs throughout different sectors of society, account for existing disparities, and preempt undesirable consequences before and during response implementation. There is a need to understand how a systems approach can be better integrated into decision-making to improve future responses to public health emergencies. A wide range of stakeholders should contribute expertise to these models, and these partnerships should be formed in advance of a public health emergency.

Introduction

In September 2021 the National Academies of Sciences, Engineering, and Medicine hosted a workshop titled “Towards a Post-Pandemic World: Lessons from COVID-19 for Now and the Future.” (NASEM, 2022) In this article, select workshop participants further explore the application of systems thinking in evaluating COVID-19 mitigation measures.

Systems Thinking in Public Health

A systems science approach to outbreak response planning is a useful tool for broadening strategic thinking to consider critical factors driving the short- and long-term consequences of crisis response measures, including how such decisions will impact health disparities (Bradley et al., 2020). A conceptual framework, systems thinking accounts for the relationship between individual factors within a scenario as well as their contributions to the whole, and can facilitate the synthesis of response plans that match the scale and complexity of the problem at hand (Trochim et al., 2006).

Specifically for public health, a systems approach “applies scientific insights to understand the elements that influence health outcomes; models the relationships between those elements; and alters design, processes or policies based on the resultant knowledge” (Kaplan et al., 2013). Complex and interconnected risk factors collectively influenced health outcomes in the COVID-19 pandemic. Response to an evolving public health emergency requires a systems approach that can weigh disparate needs and account for systemic inequities to quickly generate solutions while remaining adaptable as new data emerges.

In this article, we use the issue of K–12 public school closures in the United States to illustrate the need for systems approaches in public health situations. Mapping tools, such as causal loop diagrams, can show the complexity of interconnected factors and their use should be prioritized to guide evidence-based decisions in complex and evolving circumstances. This article argues for the adoption of a systems science approach to outbreak decision-making that:

• addresses the inherent complexity of societal impacts during public health emergencies,
• accounts for social determinants of health, and
• includes perspectives from a wide range of stakeholders

COVID-19 Decision-Making and Unintended Consequences

At the start of the COVID-19 pandemic, policy decisions and responses were enacted quickly to contain the spread of disease. However, the public health implications of COVID-19 extend beyond the disease itself, as the pandemic exacerbated disparities in health outcomes closely correlated with social determinants of health and structural inequalities (Karmakar et al., 2021; Liao and De Maio 2021; Webb Hooper et al., 2020). While strong infection control measures, such as lockdowns and school closures, were considered essential when COVID-19 was an emergent disease, these responses resulted in unintended consequences that were not prioritized in the early decision-making process (Turcotte-Tremblay et al., 2021).

This trade-off may have been necessary at the time, given the rapid disease spread and lack of data about the disease to guide initial decisions. However, as the potential for containment or eradication of COVID-19 dimmed, decision-makers were slow to update mitigation measures based on evolving knowledge and accounting for the broader population health needs. The COVID-19 response stemmed largely from concern about acute infections, reflecting a mindset that was more focused on medical response than broader public health impacts.

Biological factors (e.g., age or comorbidities such as hypertension, diabetes, lung disease, or immunodeficiencies) and social determinants of health (e.g., disparities stemming from marginalized socioeconomic status, lack of access to housing and transportation, race and ethnicity, and language and literacy barriers) interact to affect health and well-being (WHO, 2023; Gao et al., 2021). While awareness of biological risk factors for severe illness grew rapidly and mitigation measures were enacted to protect individuals at risk, consideration for social risk factors in COVID response plans were not equally prioritized (Laylavi, 2021).

For example, while the federal government heavily invested in the development of vaccines and anti-viral treatments early in the pandemic (Lalani et al., 2023), expanded unemployment support to address pandemic-related job losses and educational support for students during school closures were deprioritized and debated at length in government. This inaction slowed critical support for populations disproportionately impacted by pandemic spread-related closures.

The neglect of programs that would create a social safety net for the populations most marginalized is not specific to the pandemic, but is an exacerbation of systematic neglect over decades (Mody et al., 2022; Dorn et al., 2020; Saenz and Sparks, 2020). Even when educational support programs were rolled out, they were implemented inconsistently and did not specifically consider the additional needs of populations that have been made to be vulnerable and that were more likely to be disproportionately impacted by school closures and loss of income due to pandemic restrictions (Wright, 2021).

Officials did not give significant attention to the secondary impacts of the COVID-19 pandemic as the pandemic progressed. While these social disparities existed before the onset of COVID-19, decisions made in response to the pandemic widened many of these gaps.

There have been earlier calls to apply a systems approach to improve public health outcomes, and many examples exist to illustrate the strength of a systems approach in successfully addressing complex public health challenges (Kaplan et al., 2013; Honoré et al., 2011). The example of public school closures demonstrates how the social impacts of mitigation measures widened existing disparities. The example also highlights the need for holistic, systems-based approaches in addressing future public health crises.

Public School Closures and Remote Learning: A Case for Applying Systems Thinking to Improve Health Outcomes during Future Disease Outbreaks

The issue of school closures during the pandemic serves as a case study for how factors affecting health were not holistically considered during decision-making. School closures can exacerbate social and health disparities, with long-lasting consequences (NASEM, 2020). Many students rely on school systems for adequate nutrition, safety, supervision, and socioemotional and cognitive development (Van Lancker and Parolin, 2020). In addition, substantial evidence shows that remote learning is an inadequate and unequitable substitute for in-person learning and does not completely mitigate learning losses during school closures (Agostinelli et al., 2022; Engzell et al., 2021; Bettinger and Loeb, 2017).

Furthermore, school closures may have a greater impact on students in underserved communities. Systemically disadvantaged students (e.g., those who are experiencing poverty or are from racial or ethnic minority communities) are less likely to have access to the technology or broadband internet that is necessary for remote learning. They are less likely to have parents who are able to work from home and supervise them and often encounter other barriers to achieving learning goals (Smith and Reeves, 2020). Students with special educational needs have had disproportionate learning losses and have limited access to other supportive resources otherwise provided through schools while schools are closed (Hurwitz et al., 2021; Nelson and Murakami, 2020).

Importantly, education access and achievement are associated with improved health outcomes, and the above-mentioned educational disparities may translate to worsened health disparities among the different communities (Dorn et al., 2021; Zajacova and Lawrence, 2018).

The decision-making surrounding school closures is complex (Allen, 2021; World Bank Group Education, 2020). While decision makers now know that K–12 public school children have reduced physical risk to severe disease outcomes from COVID-19 compared to adults, school closures were implemented early in the pandemic, when this risk was unknown and there was limited time for decision-making. Students experienced related impacts from pandemic mitigation measures, and some have suffered mentally, emotionally, and developmentally as a direct result of school closures specifically (Viner et al., 2022; Engzell et al., 2021).

However, decisions about school closures and transitions to remote learning at the start of the COVID-19 pandemic generally focused on physical health risk factors (e.g., preventing transmission and mortality) rather than holistic evaluations of children’s multifaceted developmental needs (e.g., socialization in cognitive and emotional development; Viner et al., 2022). Factors such as public fear and parental pressure may have also affected decisions both to close and reopen schools. Many under-resourced schools may have also had limited ability to facilitate a safe return to in-person learning. The many factors affecting school closure decisions further demonstrates the overall need for a systematic, context-specific model for decision-making in future emergencies.

Widespread school closures lasted well into 2021, despite early and repeated warnings about the potential costs to student well-being (Allen, 2021; Kaffenberger, 2021; Balingit and Meckler, 2020) and evidence that with adequate interventions, in-person schooling could be made safe (Alonso et al., 2022; Rotevatn et al., 2022; Head et al., 2021).

Furthermore, school closures were experienced unequally. A nationwide study by Parolin and Lee (2021) found a correlation between school closures in fall 2021 and the racial and ethnic composition of the student body, with nearly 70 percent in-person attendance in schools with a high majority of White students and more than 70 percent closure among schools with large proportions of non-White students. This difference was observed across the United States and within local metro areas.

For example, in Los Angeles County, schools with the highest proportion of racial and ethnic minority students stayed closed at higher rates and for longer durations than schools with the highest proportion of White students (see Figure 1 ). Many factors could have contributed to this observation, including governance, demographic distribution in urban and suburban areas, differences in resource availability in public schools (including school health services), and differences in transmission rates due to population density.

A separate study by Grossmann et al. (2021) also suggested that other outside factors, such as political pressure and strength of teachers unions, may have had significant influence over school closure decisions. A diversity of factors impact student well-being; thus, a systems approach would support informed decision-making in school closure policies.

Multiple factors must also be accounted for in remediation plans, not just initial decision-making, in response to a public health crisis. In July 2021, the Center on Reinventing Public Education (CRPE, 2022) evaluated published plans from 100 major US school districts on spending the more than $43 billion allocated from the Elementary and Secondary School Emergency Relief Fund. While most districts included learning loss and social, emotional, and mental health as key target areas for remediation, only about 30 percent of schools accounted for special needs, equity, and community engagement in their remediation plans (see Figure 2 ). This data revealed that many school districts have attempted to address pandemic-related health outcomes, but these efforts can be further improved with a more holistic approach to decision-making regarding public education and student health.

Students’ well-being and long-term health outcomes are not the only considerations in deciding when best to resume in-person learning. Plans for safe and sustainable resumption of in-person learning also need to consider the needs and concerns of other stakeholders, such as parents, school staff (including nurses and health human resources), and public officials. For example, federal school reopening strategies included practices to safeguard the well-being of educators and other school staff (Department of Education, 2021). Other concerns include the need for data to understand and mitigate transmission dynamics within classrooms and in the local community, especially with the emergence of new viral variants (Honein et al., 2021). These complexities further underline the need for a holistic decision-making strategy that accounts for different needs and dynamics as information unfolds during a public health emergency such as the COVID-19 pandemic.

Using Systems Thinking to Redefine Strategies for Public Health Preparedness

Implementing a systems approach to public health planning requires tools, trained experts, and collaboration with stakeholders. Causal loop diagrams (CLDs) are analytical tools used to map a complex set of factors and forces in a system. They can be used to analyze interplay between factors or develop response strategies. CLDs are gaining attention in public health spheres and can be developed for various purposes, including for influencing policy and practice and for system dynamics modeling (Baugh Littlejohns et al., 2021).

Several CLDs have been developed to demonstrate the variety and interconnectedness of issues related to COVID-19, including mitigation measures. In a series of workshops, Sahin et al. (2020) gathered a group of subject matter experts in various fields (e.g., public health, social science, systems thinking) to develop a CLD that maps the unintended impacts of COVID-19 mitigation measures on socioeconomic systems (see Figure 3 ). One of the loops shows that social distancing will likely decrease virus transmission but also has negative, lasting mental health consequences (loop B3). Sahin et al. (2020) note there is a “a high risk of catastrophic social order demise” if enacted policies do not account for impacts on society.

Tools such as CLDs can facilitate understanding of varying factors within a public health system, a view that is needed to enact holistic solutions. This model captures the severity of social consequences, which were largely overlooked throughout the pandemic.

To further demonstrate their potential, we have created an example CLD that highlights components that could inform a more complex CLD addressing public education issues for children (see Figure 4 ). This illustrative CLD integrates several of the factors that have been discussed in this article (e.g., children’s physical health, mental and emotional health, family stressors). While not developed with the intent of immediate application, this example CLD could be modified and used for decision-making.

An analysis of COVID-19 CLDs by Strelkovskii and Rovenskaya (2021) concluded that these tools can “draw the attention of policy makers to areas where unintended and unwanted effects may be anticipated”; they identified CLDs as useful tools for highlighting the diverse impacts of the pandemic. Their analysis also found that, while there have been numerous calls to apply systems thinking approaches to the impacts of COVID-19, there are few examples of practical applications. The authors highlighted that there have been relatively few examples of CLDs developed for COVID-19, and these have been developed for purposes other than influencing decision-making.

As with many aspects of the COVID-19 pandemic, there is an opportunity to develop tools, such as CLDs, that are more actionable and policy related. The means of developing the CLD are also critical to its use. Such development should include an interdisciplinary group of experts to capture the multiple layers of a complex system. Stakeholder and community participation in developing CLDs represent a step toward developing tools that are more comprehensive and that may be more actionable from a policy standpoint (Baugh Littlejohns et al., 2021). Collaborative groups that include experts, community members, and policy makers can be better poised to develop a dynamic model that can be useful in depicting complex social, physical, and economic relationships. These nuanced models could serve as critical tools for weighing the impacts of mitigation measures in a public health emergency, and developing system models in advance will facilitate immediate action at the onset of an emergency. While providing substantial benefits, developing CLDs also presents challenges. Because systems are inherently complex, it is difficult to capture all relevant factors in a diagram while maintaining the detail that is needed to be useful. Also, translating a CLD into action can be challenging, as evidenced by the lack of actionable CLDs that address the impacts of the COVID-19 pandemic. Despite these challenges, CLDs remain a useful tool for providing a decision-making framework in complex situations with interconnected factors.

The U.S. response strategy to the COVID-19 pandemic suffered from a lack of a holistic and systems-oriented approach to decision-making. This paper outlines the complexities that should have been considered in making the shift to fully remote learning inK–12 schools during COVID-19. There is a need to integrate diverse perspectives from interdisciplinary experts, stakeholders, and community members in developing models that influence decision-making. In the example of school closures, educators, parents, school health leaders, and community leaders are relevant stakeholders for public health decisions that affect health outcomes in schools.

Systems approaches facilitate more comprehensive assessments to inform decision-making, and CLDs are a valuable tool that can be used for response planning. Time is of the essence in a public health emergency, especially when there is minimal information about an emerging threat. Systems models can be built to respond to an emerging threat and developed as information is gained.

We assert that using CLDs as part of a systems approach can improve the transparency, inclusiveness, and credibility of the decision-making process during future public health emergencies. Systems thinking, and tools such as CLDs, should be prioritized in future public health emergencies.

Despite the widely acknowledged usefulness of CLDs, there are few examples of CLDs that were applied during the COVID-19 pandemic to influence decision-making. Partnerships between public health experts and decision-makers should be developed in advance of public health emergencies, so they will be poised to respond immediately. Further, perspectives from the economic and social sectors should also be sought, to understand the complex impact of emergencies, including the impacts of mitigation measures. Increased stakeholder engagement can result in tools that are more actionable and effective.

A commitment to incorporate systems thinking will require broadening the preparedness planning approach for public health decision-making, emphasizing the inclusion of physical and related impacts, and securing buy-in from decision-makers (Zięba, 2021; Klement, 2020). This type of thinking would also require training, so the public health workforce can learn to design and implement these methods.

Join the conversation!

Download the graphics below and share them on social media!

• Agostinelli, F., M. Doepke, G. Sorrenti, and F. Zilibotti. 2022. When the great equalizer shuts down: Schools, peers, and parents in pandemic times. Journal of Public Economics 206:104574. https://doi.org/10.1016/j.jpubeco.2021.104574.
• Allen, J. G. 2021. We learned our lesson last year: Do not close schools. The New York Times , December 20. Available at: https://www.nytimes.com/2021/12/20/opinion/omicron-schools-do-not-close.html (accessed October 17, 2023).
• Alonso, S., M. Català, D. López, E. Álvarez-Lacalle, I. Jordan, J. J. García-García, V. Fumadó, C. Muñoz-Almagro, E. Gratacós, N. Balanza, R. Varo, P. Millat, B. Baro, S. Ajanovic, S. Arias, J. Claverol, M. F. de Sevilla, E. Bonet-Carne, A. Garcia-Miquel, E. Coma, M. Medina-Peralta, F. Fina, C. Prats, and Q. Bassat. 2022. Individual prevention and containment measures in schools in Catalonia, Spain, and community transmission of SARS-CoV-2 after school re-opening. PLOS ONE 17(2):e0263741. https://doi.org/10.1371/journal.pone.0263741.
• Azevedo, J. P. W. D., F. H. Rogers, S. E. Ahlgren, M.-H. Cloutier, B. Chakroun, G.-C. Chang, S. Mizunoya, N. J. Reuge, M. Brossard, and J. L. Bergmann. 2010. The state of the global education crisis: A path to recovery. Washington, DC: World Bank Group. Available at: http://documents.worldbank.org/curated/en/416991638768297704/The-State-of-the-Global-Education-Crisis-A-Path-to-Recovery (accessed October 18, 2023).
• Balingit, M., and L. Meckler. 2020. Schools close over coronavirus threat, raising concerns about disruption. The Washington Post , March 9. Available at: https://www.washingtonpost.com/local/education/schools-close-over-coronavirus-threat-raising-concerns-about-disruption/2020/03/09/ea74e528-622b-11ea-acca-80c22bbee96f_story.html (accessed October 17, 2023).
• Baugh Littlejohns, L., C. Hill, and C. Neudorf. 2021. Diverse approaches to creating and using causal loop diagrams in public health research: Recommendations from a scoping review. Public Health Reviews 42. https://doi.org/10.3389/phrs.2021.1604352.
• Bettinger, E., and S. Loeb. 2017. Promises and pitfalls of online education. Evidence Speaks 2(15):1–4. Available at: https://www.brookings.edu/research/promises-and-pitfalls-of-online-education/ (accessed October 17, 2023).
• Bradley, D. T., M. A. Mansouri, F. Kee, and L. M. T. Garcia. 2020. A systems approach to preventing and responding to COVID-19. eClinicalMedicine 21:100325. https://doi.org/10.1016/j.eclinm.2020.100325.
• Center on Reinventing Public Education. 2022. 2021–2022 School District Plans Database. Available at: https://crpe.org (accessed October 13, 2023).
• Department of Education. 2021. ED COVID-19 handbook: Strategies for safely reopening elementary and secondary schools, Vol. 1. Washington, DC. Available at: https://www2.ed.gov/documents/coronavirus/reopening.pdf (accessed October 17, 2023).
• Dorn, E., B. Hancock, J. Sarakatsannis, and E. Viruleg. 2021. COVID-19 and education: The lingering effects of unfinished learning. Chicago, IL: McKinsey and Company. Available at: https://www.mckinsey.com/industries/education/our-insights/covid-19-and-education-the-lingering-effects-of-unfinished-learning (accessed October 17, 2023).
• Dorn, E., B. Hancock, J. Sarakatsannis, and E. Viruleg. 2020. COVID-19 and learning loss—disparities grow and students need help. McKinsey and Company, December 8. Available at: https://www.mckinsey.com/industries/public-and-social-sector/our-insights/covid-19-and-learning-loss-disparities-grow-and-students-need-help (accessed October 17, 2023).
• Engzell, P., A. Frey, and M. D. Verhagen. 2021. Learning loss due to school closures during the COVID-19 pandemic. Proceedings of the National Academy of Sciences 118(17):e2022376118. https://doi.org/10.1073/pnas.2022376118.
• Gao, Y. D., M. Ding, X. Dong, J. J. Zhang, A. Kursat Azkur, D. Azkur, H. Gan, Y. L. Sun, W. Fu, W. Li, H. L. Liang, Y. Y. Cao, Q. Yan, C. Cao, H. Y. Gao, M. C. Bruggen, W. van de Veen, M. Sokolowska, M. Akdis, and C. A. Akdis. 2021. Risk factors for severe and critically ill COVID-19 patients: A review. Allergy 76(2):428–455. https://doi.org/10.1111/all.14657.
• Grossmann, M., S. Reckhow, K. O. Strunk, and M. Turner. 2021. All states close but red districts reopen: The politics of in-person schooling during the COVID-19 pandemic. Educational Researcher 50(9):637–648. https://doi.org/10.3102/0013189×211048840.
• Head, J. R., K. L. Andrejko, Q. Cheng, P. A. Collender, S. Phillips, A. Boser, A. K. Heaney, C. M. Hoover, S. L. Wu, G. R. Northrup, K. Click, N. S. Bardach, J. A. Lewnard, and J. V. Remais. 2021. School closures reduced social mixing of children during COVID-19 with implications for transmission risk and school reopening policies. Journal of the Royal Society Interface 18(177):20200970. https://doi.org/10.1098/rsif.2020.0970.
• Honein, M. A., L. C. Barrios, and J. T. Brooks. 2021. Data and policy to guide opening schools safely to limit the spread of SARS-CoV-2 infection. JAMA 325(9):823–824. https://doi.org/10.1001/jama.2021.0374.
• Honoré, P. A., D. Wright, D. M. Berwick, C. M. Clancy, P. Lee, J. Nowinski, and H. K. Koh. 2011. Creating a framework for getting quality into the public health system. Health Affairs 30(4):737–745. https://doi.org/10.1377/hlthaff.2011.0129.
• Hurwitz, S., B. Garman-McClaine, and K. Carlock. 2021. Special education for students with autism during the COVID-19 pandemic: “Each day brings new challenges.” Autism 26(4):889–899. https://doi.org/10.1177/13623613211035935.
• Kaffenberger, M. 2021. Modelling the long-run learning impact of the Covid-19 learning shock: Actions to (more than) mitigate loss. International Journal of Educational Development 81:102326. https://doi.org/https://doi.org/10.1016/j.ijedudev.2020.102326.
• Kaplan, G. S., G. W. Bo-Linn, P. Carayon, P. J. Pronovost, W. B. Rouse, P. P. Reid, and R. S. Saunders. 2013. Bringing a systems approach to health. NAM Perspectives . Discussion Paper, National Academy of Medicine, Washington, DC. https://doi.org/10.31478/201307a.
• Karmakar, M., P. M. Lantz, and R. Tipirneni. 2021. Association of social and demographic factors with COVID-19 incidence and death rates in the US. JAMA Network Open 4(1):e2036462–e2036462. https://doi.org/10.1001/jamanetworkopen.2020.36462.
• Klement, R. J. 2020. Systems thinking about SARS-CoV-2. Frontiers in Public Health 8. https://doi.org/10.3389/fpubh.2020.585229.
• Lalani H. S., S. Nagar, A. Sarpatwari, R. E. Barenie, J. Avorn, B. N. Rome, and A. S. Kesselheim. 2023. US public investment in development of mRNA covid-19 vaccines: Retrospective cohort study. BMJ 380:e073747. doi:10.1136/bmj-2022-073747.
• Laylavi, F. 2021. Social vulnerability to COVID-19: Preliminary indicators and research agenda. In COVID-19 pandemic, geospatial information, and community resilience, edited by A. Rajabifard, G. Foliente, and D. Paez. Boca Raton, FL: CRC Press. pp. 87–99. https://doi.org/10.1201/9781003181590
• Liao, T. F., and F. De Maio. 2021. Association of social and economic inequality with coronavirus disease 2019 incidence and mortality across US counties. JAMA Network Open 4(1):e2034578–e2034578. https://doi.org/10.1001/jamanetworkopen.2020.34578.
• Los Angeles Almanac. 2020. Ethnic distribution of pupils by school districts: Los Angeles County: School year 2019–2020. Available at: http://www.laalmanac.com/education/ed05.php (accessed January 13, 2023).
• Mody, A., C. Bradley, S. Redkar, B. Fox, I. Eshun-Wilson, M. G. Hlatshwayo, A. Trolard, K. H. Tram, L. M. Filiatreau, F. Thomas, M. Haslam, G. Turabelidze, V. Sanders-Thompson, W. G. Powderly, and E. H. Geng. 2022. Quantifying inequities in COVID-19 vaccine distribution over time by social vulnerability, race and ethnicity, and location: A population-level analysis in St. Louis and Kansas City, Missouri. PLOS Medicine 19(8):e1004048. https://doi.org/10.1371/journal.pmed.1004048.
• NASEM (National Academies of Sciences, Engineering, and Medicine). 2020. Reopening K-12 schools during the COVID-19 pandemic: Prioritizing health, equity, and communities. Washington, DC: The National Academies Press. https://doi.org/10.17226/25858.
• NASEM. 2022. Toward a post-pandemic world: Lessons from COVID-19 for now and the future: Proceedings of a workshop. Washington, DC: The National Academies Press. https://doi.org/10.17226/26556.
• Nelson, M., and E. Murakami. 2020. Special education students in public high schools during COVID-19 in the USA. Journal of the Commonwealth Council for Educational Administration & Managemen t 48(3):109–115. Available at: https://cceam.net/wp-content/uploads/2020/10/ISEA-2020-48-3.pdf#page=115 (accessed October 13, 2023).
• Parolin, Z., and E. K. Lee. 2021. Large socio-economic, geographic and demographic disparities exist in exposure to school closures. Nature Human Behaviour 5(4):522–528. https://doi.org/10.1038/s41562-021-01087-8.
• Rotevatn, T. A., P. Elstrøm, M. Greve-Isdahl, P. Surén, T. K. B. Johansen, and E. Astrup. 2022. School closure versus targeted control measures for SARS-CoV-2 infection. Pediatrics 149(5):e2021055071. https://doi.org/10.1542/peds.2021-055071.
• Saenz, R., and C. Sparks. 2020. The inequities of job loss and recovery amid the COVID-19 pandemic . University of New Hampshire Carsey School of Public Policy, August 11. Available at: https://carsey.unh.edu/publication/inequities-job-loss-recovery-amid-COVID-pandemic (accessed October 17, 2023).
• Sahin, O., H. Salim, E. Suprun, R. Richards, S. MacAskill, S. Heilgeist, S. Rutherford, R. A. Stewart, and C. D. Beal. 2020. Developing a preliminary causal loop diagram for understanding the wicked complexity of the COVID-19 pandemic. Systems 8(2):20. https://doi.org/10.3390/systems8020020.
• Smith, E., and R. V. Reeves. 2020. Students of color most likely to be learning online: Districts must work even harder on race equity . Brookings Institution, September 23. Available at: https://www.brookings.edu/blog/how-we-rise/2020/09/23/students-of-color-most-likely-to-be-learning-online-districts-must-work-even-harder-on-race-equity/ (accessed October 17, 2023).
• Strelkovskii N., and E. Rovenskaya. 2021. Causal loop diagramming of socioeconomic impacts of COVID-19: State-of-the-art, gaps and good practices. Systems 9(3):65. https://doi.org/10.3390/systems9030065.
• Trochim, W. M., D. A. Cabrera, B. Milstein, R. S. Gallagher, and S. J. Leischow. 2006. Practical challenges of systems thinking and modeling in public health. American Journal of Public Health 96(3):538–546. https://doi.org/10.2105/AJPH.2005.066001.
• Turcotte-Tremblay, A.-M., I. A. Gali Gali, and V. Ridde. 2021. The unintended consequences of COVID-19 mitigation measures matter: Practical guidance for investigating them. BMC Medical Research Methodology 21(1):28. https://doi.org/10.1186/s12874-020-01200-x.
• Van Lancker, W., and Z. Parolin. 2020. COVID-19, school closures, and child poverty: A social crisis in the making. The Lancet Public Health 5(5):e243–e244. https://doi.org/10.1016/S2468-2667(20)30084-0.
• Viner, R., S. Russell, R. Saulle, H. Croker, C. Stansfield, J. Packer, D. Nicholls, A.-L. Goddings, C. Bonell, L. Hudson, S. Hope, J. Ward, N. Schwalbe, A. Morgan, and S. Minozzi. 2022. School closures during social lockdown and mental health, health behaviors, and well-being among children and adolescents during the first COVID-19 wave: A systematic review. JAMA Pediatrics 176(4):400–409. https://doi.org/10.1001/jamapediatrics.2021.5840.
• Walsh, S., A. Chowdhury, V. Braithwaite, S. Russell, J. M. Birch, J. L. Ward, C. Waddington, C. Brayne, C. Bonell, R. M. Viner, and O. T. Mytton. 2021. Do school closures and school reopenings affect community transmission of COVID-19? A systematic review of observational studies. BMJ Open 11(8):e053371. https://doi.org/10.1136/bmjopen-2021-053371.
• Webb Hooper, M., A. M. Nápoles, and E. J. Pérez-Stable. 2020. COVID-19 and racial/ethnic disparities. JAMA 323(24):2466–2467. https://doi.org/10.1001/jama.2020.8598.
• WHO (World Health Organization). 2023. Social determinants of health . Available at: https://www.who.int/health-topics/social-determinants-of-health#tab=tab_1 (accessed October 13, 2023).
• World Bank Group Education. 2020. The COVID-19 pandemic: Shocks to education and policy responses. Washington, DC: World Bank. https://openknowledge.worldbank.org/bitstream/handle/10986/33696/148198.pdf (accessed October 17, 2023)
• Wright, A. 2021. “It’s patchwork”: Rural teachers struggle to connect in pandemic. Stateline , March 3. Available at: https://www.pewtrusts.org/en/research-and-analysis/blogs/stateline/2021/03/03/its-patchwork-rural-teachers-struggle-to-connect-in-pandemic (accessed October 17, 2023).
• Zajacova, A., and E. M. Lawrence. 2018. The relationship between education and health: Reducing disparities through a contextual approach. Annual Review of Public Health 39(1):273–289. https://doi.org/10.1146/annurev-publhealth-031816-044628.
• Zięba, K. 2021. How can systems thinking help us in the COVID-19 crisis? Knowledge and Process Management 29(3):221–230. https://doi.org/https://doi.org/10.1002/kpm.1680.

https://doi.org/10.31478/202311a

Suggested Citation

Ashby, E., C. Minicucci, J. Liao, D. Buonsenso, S. González- Dambrauskas, R. Obregón, M. Zahn, W. Hallman, and C. John. 2023. Systems thinking for public health: A case study using U.S. public education. NAM Perspectives . Discussion Paper, National Academy of Medicine, Washington, DC. https://doi.org/10.31478/202311a .

Author Information

Elizabeth Ashby, MS, is Associate Program Officer, National Academies of Sciences, Engineering, and Medicine. Charlie Minicucci, BS, is Research Associate, National Academies of Sciences, Engineering, and Medicine.  Julie Liao, PhD, is Program Officer, National Academies of Sciences, Engineering, and Medicine. Danilo Buonsenso, MD, PhD, is Pediatric Infectious Disease Physician, Department of Woman & Child Health & Public Health, Fondazione Policlinico Universitario Agostino Gemelli IRCCS. Sebastián González-Dambrauskas, MD, is Founder and Chair, LARed: Red Colaborativa Pediátrica de Latinoamérica and Adjunct Professor, Departamento de Pediatría y Unidad de Cuidados Intensivos de Niños, Universidad de la República. Rafael Obregón, PhD, is Country Representative, Paraguay, UNICEF. Matt Zahn, MD, is Pediatric Infectious Disease Physician, Children’s Hospital of Orange County. William Hallman, PhD, is Professor and Chair, Department of Human Ecology, School of Environmental and Biological Sciences, Rutgers University. Chandy John, MD, MS, is Professor of Pediatrics, Indiana University School of Medicine.

Acknowledgments

Charlie Minicucci and Elizabeth Ashby contributed equally to this work.

This manuscript benefited from the thoughtful input of Jessica G. Burke , University of Pittsburgh; Erin D. Maughan , George Mason University; and Carol Walsh , National Association of School Nurses.

Conflict-of-Interest Disclosures

Danilo Buonsenso reports funding from Pfizer outside the submitted work.

Correspondence

Questions or comments about this paper should be directed to Charlie Minicucci at [email protected].

The views expressed in this paper are those of the authors and not necessarily of the authors’ organizations, the National Academy of Medicine (NAM), or the National Academies of Sciences, Engineering, and Medicine (the National Academies). The paper is intended to help inform and stimulate discussion. It is not a report of the NAM or the National Academies. Copyright by the National Academy of Sciences. All rights reserved.

Join Our Community

Sign up for nam email updates.

• Langson Library
• Science Library
• Grunigen Medical Library
• Law Library
• Connect From Off-Campus
• Accessibility
• Gateway Study Center

Email this link

Public health 2: case studies in public health practice.

• Assignments
• Conducting Research - Finding Books & eBooks
• Finding Journal, News Articles & Statistics
• Data & Statistics, and Sites of Interest

Research Librarian for the Health Sciences, Medicine, Pharmaceutical Sciences, Pharmacology, Public Health

Introduction - Background

This Course Page will introduce you to resources and methodologies that will aid you in successfully completing all the assignments for this course.  Your point of departure should be the UCI Libraries Webpage and you may also find the Subject Guide for Public Health helpful.

Evaluating Information

When using Online or Internet Resources , consider Search Engines vs. metasites - evaluate resource - be attentive to domain -may include .com, .edu, .org, .gov, .net

Thinking Critically about World Wide Web Resources - http://www.library.ucla.edu/libraries/college/help/critical/index.htm

Take the CRAAP test:  Currency, Relevance, Authority, Accuracy, Purpose - https://www.csuchico.edu/lins/handouts/eval_websites.pdf

• Scope of coverage
• Verifiable information - can be documented
• Authority control - authorship - who, affiliation, where
• Currency - note the date, the update, does it reflect the right period of time
• Evaluate the source - establish criteria that is meaningful to covering the topic>
• Objectivity - reduce bias
• Design & Presentation
• Ease of use
• Citing Internet resources - URL & date of the search
• Capturing and citing
• Establishing relevancy

Citing Sources

In order to avoid plagiarism and to honor intellectual integrity, make sure that you cite the authority in a bibliographic reference to anything that is not your original writing or creation - that means when you quote a passage, insert a graphic image, figures, or illustration, that you cite the original source. The style manual you choose to follow should document how you cite electronic resources. Standard formats include the following reference elements:

Standard formats include the following reference elements:

For a Journal article or conference proceeding:

Author(s) - last name, first name, MI, - [include multiple authors if noted] (date), Title of article. Source of Article/Title of Journal. volume #, (issue #): pages. If it is only an electronic publication with no reference to print pages, then you cite the DOI - Digital Object Identifier and the date last visited.

If it is a conference paper, then you cite the Source of the Publication, Title of conference, date and location of meeting.

For books, the format is:

Author, editor of volume or chapter, (imprint date). Title of chapter in Title of Book, edited by editor if different. City of Publication: Publisher, page references. Note if it is an eBook.

For full volume:

Author, (date). Title of Book. City of Publisher, Publisher: pages

ACM Style Guide   - there are some examples in the link to MS Word in the list of references provided that can be used as a model for different writing elements

  IEEE Citation Style - The  IEEE Style Manual  notes the specific ways that references and footnotes are to be handled.  You will be writing with a model of California Engineer - check it out at the Science Library Current Periodicals Room by journal title or for back issues, by call number TA 1 C28 in the Science Library Drum (round part of the Library)

MLA Style Format is documented in the MLA Handbook for Writers of Research Papers, 2009 at any of the UCI Library Reference Desks at:                   REF LB 2369 G53 2009

APA Style - the 6th edition of the Publication Manual of the American Psychological Association, 2010 is the current Style Manual and is found at all REF desks at: REF BF 76.7 P83 2010 - for additional support with the APA style manual please consult the APA Style & Citation Resources  tab at https://guides.lib.uci.edu/psychology

Other hints:

• very important now
• when interviewing for job

Bibliographic Management Software

BIBLIOGRAPHIC MANAGEMENT SOFTWARE - allows you to discover, find, manage, recall and apply the citations you retrieve from all formats of work - journal articles, book chapters, edited volumes, standards, specifications, patents, conference papers & proceedings, lectures, etc. Neither Zotero nor EndNote support BibTeX or LaTeX, only the usual wordprocessing languages or Legal Style Manual.  For additional information, please visit the  Bibliographic Management Software page.  A comparison of all kinds of different products are noted at http://en.wikipedia.org/wiki/Comparison_of_reference_management_software

• EndNote more... less... There is an abbreviated free web version available via the Web of Science Databases at https://access.clarivate.com/login?app=endnote.
• EndNote Basic Web-based, free service.
• Zotero more... less... Researchers can add searchable notecards to their citations, organize them with tags, attach files to them, arrange them into folders, or export them as formatted citations. Zotero can capture a single citation or an entire results list of sources, which researchers can work with either online or offline.

    Library Hours

During the academic year, Library Hours are:

Monday-Thursday from 8am-11pm

Friday from 8am-6pm

Saturday from 1-5pm

Sunday from 1-5pm

Gateway Commons will be open during the week from 10am-3am; Friday from 10am-9pm, Saturday from 5pm-9pm, and Sunday from 5pm-3am.  The Libraries will be closed November 10-11 for Veterans Day and November 23-25 for Thanksgiving. 

Connecting from Off-Campus

Researching from home? Use the VPN Links below to access our databases.   Instructions on connecting from off-campus should be followed - for additional information, consult with NACS at 949-824-2222

Software VPN

More Information Regarding the VPN

Ask A Librarian

For Chat, Text, eMail and to schedule a Research Consultation with a Librarian, use:

• Next: Assignments >>
• Last Updated: Apr 18, 2024 12:58 PM
• URL: https://guides.lib.uci.edu/PH2-CaseStudies

Off-campus? Please use the Software VPN and choose the group UCIFull to access licensed content. For more information, please Click here

Software VPN is not available for guests, so they may not have access to some content when connecting from off-campus.

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• HHS Author Manuscripts

Reporting the methods used in public health research and practice

Donna f. stroup.

1 Data for Solutions, Inc., Decatur, GA, USA

C. Kay Smith

2 Office of the Associate Director for Science, National Center for HIV/AIDS, Viral Hepatitis, STD, and TB Prevention, Centers for Disease Control and Prevention, Atlanta, GA, USA

Benedict I. Truman

Associated data.

The methods section of a scientific article often receives the most scrutiny from journal editors, peer reviewers, and skeptical readers because it allows them to judge the validity of the results. The methods section also facilitates critical interpretation of study activities, explains how the study avoided or corrected for bias, details how the data support the answer to the study question, justifies generalizing the findings to other populations, and facilitates comparison with past or future studies. In 2006, the Enhancing the QUAlity and Transparency Of health Research (EQUATOR) Programme began collecting and disseminating guidelines for reporting health research studies. In addition, guidelines for reporting public health investigations not classified as research have also been developed. However, regardless of the type of study or scientific report, the methods section should describe certain core elements: the study design; how participants were selected; the study setting; the period of interest; the variables and their definitions used for analysis; the procedures or instruments used to measure exposures, outcomes, and their association; and the analyses. Specific requirements for each study type should be consulted during the project planning phase and again when writing begins. We present requirements for reporting methods for public health activities, including outbreak investigations, public health surveillance programs, prevention and intervention program evaluations, research, surveys, systematic reviews, and meta-analyses.

“People’s views about contradictory health studies tend to vary depending on their level of science knowledge. An overwhelming majority of those with high science knowledge say studies with findings that conflict with prior research are a sign that understanding of disease prevention is improving (85%). A smaller majority of those with low science knowledge say the same (65%), while 31% say that the research cannot really be trusted because so many studies conflict with each other.” Pew Research Center, February 2017

Introduction

The methods section of a scientific article often receives the most scrutiny from journal editors, peer reviewers, and skeptical readers. Effective reporting of the methods used in public health research and practice enables readers to judge the validity of the study results and other scientists to repeat the study during efforts to validate the findings. Although word limits for a scientific article can hinder complete reporting, full information can usually be provided in online-only or supplemental appendixes. The methods section also facilitates critical interpretation of study results; explains how the study avoided or corrected for bias in selecting participants, measuring exposures and outcomes, and estimating associations between exposures and outcomes; and explains how the data support the answer to the study question. The methods justify generalizing the findings from the sample studied to the population it represents. Finally, complete reporting of methods facilitates comparing the study findings with past and in future studies (e.g., in systematic reviews or meta-analyses).

For example, consider two different conclusions from the same study ( 1 ) highlighted in two newspaper headlines ( Figure 1 ). A reader must carefully read the methods section of the original study report (not those of newspaper reports) to assess the validity of each headline. Although the vaccine seemed to substantially lower the infection rate among African Americans and other non-Hispanic minorities in the trial, the numbers of participants from these racial/ ethnic groups were too small to support statistically significant conclusions about vaccine efficacy. This “no effect” conclusion is the more valid one, given details in the study methods.

Differences in media interpretation of a single study.

The methods section is usually the easiest and often the first section of the manuscript to be written, often during the protocol development or study-planning phase, then revised and updated after completion of the study to describe what was actually done during the study, including documentation of any changes from initial protocol. Describing the methods completely in the methods section for all aspects of the study is crucial; the readers should not discover something about the methods buried in the results or discussion. Of note, publishers typically specify their requirements for the methods sections in their journals, and those specifications might be beyond those presented here; therefore, reviewing the publisher’s instructions to authors thoroughly before writing begins is always advisable.

This article provides practical guidance for improving the clarity and completeness of the methods section of a scientific article or technical report. We provide a framework based on published standards for reporting the findings of public health research and practice according to major categories of public health activities. This is not a guide for conducting research. Instead, it describes the essential content of the methods section of a scientific article or technical report. Further, we restrict our attention to public health research and practice, excluding laboratory studies. Finally, we mention important ethical considerations and statistical methods only briefly; details of these topics can be found in “Research and Publication Ethics” and “Reporting Statistical Methods and Results” elsewhere in this issue of the Journal.

Guidelines for reporting public health investigation methods

Reporting guidelines have been developed for many types of public health investigations. Recognizing the proliferation of such guidelines, the Enhancing the QUAlity and Transparency Of health Research (EQUATOR) Programme began in 2006 to collect and disseminate guidelines for reporting health research studies ( Table S1 ) ( 2 ). We highlight example guidelines that are internationally recognized publishing standards and also include guidelines for reporting methods associated with fundamental public health investigations not classified as research ( Table S1 ).

Disease outbreak investigations

Investigating an outbreak or unexpected increase in the incidence of disease cases or a condition for a geographic area or period is a fundamental public health activity. The primary purpose of an outbreak investigation is to identify the source of the pathogen, its transmission mode (route), and modifiable risk factors for illness so that the most appropriate control and prevention activities can be implemented ( 3 ). However, subsequent publication of a scientific report summarizing detection, investigation, and control of the outbreak is useful for disseminating knowledge of new risk factors, investigation techniques, and effective interventions.

The Public Health Agency of Canada has developed a guide for reporting the investigation and findings of disease outbreaks ( 4 ). Their guidance recommends reporting of overview or background data (dates of first case and of investigation initiation and conclusion), and methods for case finding and data collection, case investigation, epidemiologic and statistical analysis, and interventions. When a disease outbreak involves a particular setting (e.g., infections acquired in hospitals or related facilities), the reporting guidelines recommend describing the study design, participants, setting, interventions, details of any laboratory diagnosis of the pathogen by culturing and typing, health outcomes, economic outcomes, potential threats to validity, sample size, and statistical methods ( 5 ).

Public health surveillance activities

Public health surveillance—the continuous, systematic collection, analysis, and interpretation of health-related data needed to plan, implement, and evaluate public health practice—is another fundamental public health activity ( 6 ). Publishing the results of surveillance activities is an essential part of public health action; therefore, the methods section of a surveillance report should include the following:

• ❖ Any legal mandates for data reporting;
• ❖ The methods used for data collection;
• ❖ The methods used for data transfer, management, and storage;
• ❖ Relevant case definitions for confirmed, probable, and suspected
• ❖ The performance attributes of the surveillance system.

These performance attributes of a surveillance system are defined in the Surveillance Evaluation Guidelines, published by the Centers for Disease Control and Prevention (United States) ( 7 ) and in Principles and Practice of Public Health Surveillance ( 8 ).

Intervention and prevention program evaluation

Another fundamental public health activity is evaluating intervention and prevention programs. A framework for program evaluation developed by the US Centers for Disease Control and Prevention summarizes key elements of the activity, specifies steps in the process, provides standards for measuring effectiveness, and clarifies the purposes of program evaluation ( 9 ). Guidelines for writing the methods section of a program evaluation paper exist for different types of evaluation designs. For example, the Transparent Reporting of Evaluations with Nonrandomized Designs (TREND) checklist is useful for reporting evaluations of behavioral and public health interventions with nonrandomized designs ( 10 ). The TREND checklist includes advice regarding how the methods should describe the participants, interventions, objectives, outcomes, sample size, exposure assignment, blinding (masking) of investigators or participant exposure, unit of analysis, and statistical methods. In contrast, for reporting the evaluation of interventions to change behavior, recommendations from the Workgroup for Intervention Development and Evaluation Research (WIDER) are more appropriate ( 11 ).

When randomization assignment is impractical, evaluations use observational designs and collect information on variables needed at the analysis stage of the investigation to correct for selection bias. Guidelines for reporting observational evaluations consider the observation method, the intervention and expected outcome, study design, information regarding the sample, measurement instruments, data quality control, and analysis methods ( 12 ).

Another approach to evaluation is a mixed-method or realist model, a theory-driven evaluation method increasingly used for studying the implementation of complex interventions within health systems, particularly in low- and middle-income countries ( 13 ). Theory-driven evaluation describes the associations between activities and outputs and short- and long-term outcomes. Theory-driven evaluation also attempts to address the problem that evaluations using traditional methods (e.g., experimental and quasi-experimental methods) do not always deal with intervention complexity. For example, in evaluating the effectiveness of community health workers in achieving improved maternal and child health outcomes in Nigeria, researchers chose a theory-driven approach because of countrywide and community-specific factors affecting the outcomes ( 14 ). As for other types of evaluation, reporting standards for realist evaluations include describing the reasons for using the method, the environment surrounding the evaluation, the program evaluated, the evaluation design, the data collection methods, the recruitment and sampling, and any statistical analysis ( 15 , 16 ).

If economic factors are important in the evaluation, specific guidelines should be consulted (e.g., the Consolidated Health Economic Evaluation Reporting Standards or CHEERS) ( 17 ). In using these reporting guidelines, some fundamental approaches in evaluation become important. Three approaches are commonly used to establish the effectiveness of a program or intervention: (I) comparing participants in the program with nonparticipants; (II) comparing results from different evaluations, each of which used different methods; and (III) case studies of programs and outcomes. In the comparison approach, random assignment of persons, facilities, or communities can be used to minimize selection bias. However, random assignment can be impractical for interventions involving, for example, mass media programs designed to reach the entire population. In such cases, other methods of assignment to intervention or control groups can be used.

Blinding the investigators and participants to whether a participant is assigned to an intervention or control group can help to avoid measurement bias. A participant’s knowledge that he or she is receiving an intervention can affect the outcome (the Hawthorne effect). The potential for bias is even more likely if the outcome of interest is behavioral, rather than biologic.

Furthermore, measurers of the outcomes must also be blind to what the recipients received to avoid biases in measurement associated with the measurers’ expectations (i.e., double-blinding). However, in certain evaluations, double-blinding is impractical. For example, a breastfeeding mother will be aware that she and her infant are in the breastfeeding intervention group, and that knowledge can affect other aspects of her behavior toward her infant. In such studies, rather than using double-blinding, the investigator might develop placebo interventions that expose mothers to the same amount and intensity of an educational intervention, but on a subject unrelated to breastfeeding ( 18 ). Participants and evaluators can be blinded by keeping treatment and control groups physically separate so that members of each group are unaware of the other group’s activities. However, the two groups would then have different experiences with interventions, exposures, and outcomes, presenting challenges for standardizing measures and development of appropriate informed consent procedures.

Public health research

Historically, many of the guidelines for reporting were developed in the context of clinical research studies. One of the earliest of these guidelines is for randomized controlled trials ( 19 ). These guidelines and subsequent extensions ( 20 ) formed the basis of much of the guideline development discussed in this paper. For each of these research designs, reporting of the methods should discuss how human participants were protected from harm caused by ethics errors, including informed consent, and how participants’ privacy and confidentiality was protected. These methods are covered in more detail in “Research and Publication Ethics” elsewhere in this issue of the Journal.

Clinical case-series reports

A report on a single case of a series of clinical cases with point-of-care data provides evidence of the effectiveness of high-quality patient care and approaches to treating rare or unusual conditions. To help reduce reporting bias, increase transparency, and provide early signals of what interventions work, depending on patient characteristics and circumstances, an international group of experts developed the CARE guidelines for reporting case studies and case-series ( 21 ). Even if such a report does not have a designated methods section, it should include information regarding patient characteristics, clinical findings, timelines (e.g., timeline graphs or epidemiologic curves), diagnostic assessments, therapeutic interventions, and outcomes.

Qualitative research

Many health problems can be addressed by combining interdisciplinary quantitative and qualitative methods. In such investigations, qualitative methods can be helpful by providing information about the meaning of text, images, and experiences and how the context surrounding study participants and their environments influence the concepts and theories being studied. For example, to investigate maternal knowledge of and attitudes toward childhood vaccination in Haiti, researchers used focus group discussions, physician observation, and semi-structured interviews with health providers ( 22 ). The Standards for Reporting Qualitative Research (SRQR) ( 23 ) checklist includes a qualitative research model, researcher characteristics, context, sampling strategy, ethics protections, data collection (e.g., methods, instruments, and technology), units of study, data processing and analysis, and techniques to enhance trustworthiness. A later extension (Consolidated Criteria for Reporting Qualitative Research or COREQ) ( 24 ) further defined the evaluation domains and added information about the research team.

Cross-sectional surveys and disease registry studies

If a controlled experiment or a quasi-experimental study design is impractical, an observational study is the only research option. The STrengthening the Reporting of OBservational studies in Epidemiology (STROBE) initiative recommends information that should be included in an accurate and complete report of any of three main observational study designs: cohort, case-control, and cross-sectional studies ( 25 ). For cross-sectional studies, surveys, and registry studies, the checklist includes study design, setting (including periods of recruitment), participants (methods of selection and exclusion criteria), variables (outcomes, exposures, confounders, and effect modifiers), data sources and measurements, efforts to address potential sources of bias, study size (including power calculation), quantitative variables and any groupings, and statistical methods (including handling of missing data and sensitivity analysis). The STROBE checklist does not include some important survey methods, such as nonresponse analysis, details of strategies used to increase response rates (e.g., multiple contacts or mode of contact of potential participants), and details of measurement methods (e.g., making the instrument available so that readers can consider questionnaire formatting, question framing, or choice of response categories) ( 26 ). Guidelines also have been developed for reporting the methods used for Internet-based surveys, the Checklist for Reporting Results of Internet E-Surveys (CHERRIES) ( 27 ).

Use of routinely collected health data, obtained for administrative and clinical purposes rather than research, is increasing in public health. To respond, guidelines for reporting methods used in registry studies were developed in 2015: the REporting of studies Conducted using Observational Routinely-collected health Data (RECORD) checklist ( 28 ).

Case-control studies

The case-control study design is useful for determining whether an exposure is directly associated with an outcome (i.e., a disease or condition of interest). This design is often used in public health because it is quick, inexpensive, and easy, compared with a cohort (follow-up) study design, making the case-control study particularly appropriate for (I) investigating outbreaks and (II) studying rare diseases or outcomes. The STROBE guidelines provide specific sections for reporting the methods used for case-control studies. The guidance includes attention to study design, setting, variables, data sources, bias, and study size. Specific to case-control studies is the necessity of reporting eligibility criteria, the sources and methods of case-patient ascertainment and control subject selection, and the rationale for the choice of case-patients versus control subjects. For matched case-control studies, reporting should include the matching criteria and the number of control subjects per case. In this study design, the methods must report how the statistical methods accounted for the matching.

A useful example of reporting methods for case-control studies can be found in “Reporting Participation in Case- Control Studies” by Olson et al. (readers should consider using the tables in that paper) ( 29 ). The effect size provides an estimate of the average difference, measured in standard deviation units so as to be scale independent, between a case’s score and the score of a randomly chosen member of the control population. Specific guidance for reporting effect sizes in case-control research is available ( 30 ).

Cohort studies

The general guidance of STROBE also is useful for reporting methods used in cohort studies. The adaptations for this study design include reporting methods for determining eligibility criteria, sources and methods of participant selection, and follow-up methods. For matched cohort studies, reporting should include the matching criteria used and the number of participants exposed and unexposed to the hypothesized cause of the outcome; in this case, the methods section also should describe how loss to follow-up was addressed. An example of reporting according to STROBE guidelines for cohort studies is available in Kunutsor et al. ’s investigation of the association of baseline serum magnesium concentrations associated with a risk for incident fractures ( 31 ).

Systematic reviews and meta-analyses

Meta-analysis is a statistical procedure for combining data from multiple studies, obtained through a systematic review of the literature. This can be used to identify a common effect (when the treatment effect is consistent among studies), to identify reasons for variation, or to assess important group differences. Pharmaceutical companies use meta-analyses to gain approval for new drugs, with regulatory agencies sometimes requiring a meta-analysis as part of the approval process. Researchers use meta-analyses to determine which interventions work and which ones work best. Many journals encourage researchers to submit systematic reviews and meta-analyses that summarize the body of evidence regarding a specific question, and systematic reviews are replacing the traditional narrative review. Meta-analyses can play a key role in planning new studies by identifying unanswered questions. Finally, meta-analyses can be used in grant applications to justify the need for a new study.

The earliest guideline for reporting meta-analyses was for meta-analyses of randomized controlled trials ( 32 ) and specified reporting of searches, selection, validity assessment, data abstraction, study characteristics, and quantitative data synthesis, and in the results with “trial flow,” study characteristics, and quantitative data synthesis. Research documentation was identified for only 8 of 18 items. Subsequently, this guidance was revised and expanded in the PRISMA Statement ( 33 ).

With the proliferation of meta-analyses of observational studies, reporting guidance followed ( 34 ). The MOOSE Statement requires a quantitative summary of the data; the degree to which coding of data from the articles was specified and objective; an assessment of confounding, study quality, and heterogeneity; the statistical methods used; and display of results (e.g., forest plots). The PRISMA statement also includes recommendations useful for reporting meta-analyses of observational studies; thus, both checklists should be consulted. Several extensions of the PRISMA checklist are available for network meta-analyses, health equity, and complex interventions ( 35 ).

The methods section of a scientific article must persuade readers that the study design, data collection, and analysis were appropriate for answering the study question and that the results are accurate and trustworthy. The methods section is often written first and is the easiest section of the manuscript to write because it is often written before the study begins. Writers should report the methods that were actually used in addition to methods that were planned but not used. Regardless of the type of study conducted and type of document written, the methods section should include certain core descriptive elements: the study design, how participants were selected, the setting, the period of interest, the variables and their definitions used for analysis, the procedures or instruments used to measure exposures, outcomes, and their association, and the analyses that produced the data that answer the study question. These core elements are common across all study designs, and the specific requirements for each study type should be consulted during project planning and again when writing begins.

Care should be taken to ensure that all methods are described in the main methods section or in supplementary online material, and not included in the results or discussion. Careful attention to reporting of methods can assist journal peer reviewers and readers, as well as other researchers who might use the methods to replicate the study or use the results in a meta-analysis or systematic review.

Supplementary Material

Acknowledgments.

The authors would like to thank Tom Lang for very helpful comments on an earlier draft.

Contributions: (I) Conception and design: All authors; (II) Administrative support: DF Stroup, CK Smith; (III) Provision of study materials or patients: All authors; (IV) Collection and assembly of data: DF Stroup, CK Smith; (V) Data analysis and interpretation: All authors; (VI) Manuscript writing: All authors; (VII) Final approval of manuscript: All authors.

Conflicts of Interest: The authors have no conflicts of interest to declare.

• Download PDF
• Share X Facebook Email LinkedIn
• Permissions

Fatal Traffic Risks With a Total Solar Eclipse in the US

• 1 Department of Medicine, University of Toronto, Toronto, Ontario, Canada
• 2 Evaluative Clinical Science Platform, Sunnybrook Research Institute, Toronto, Ontario, Canada
• 3 Institute for Clinical Evaluative Sciences, Toronto, Ontario, Canada
• 4 Division of General Internal Medicine, Sunnybrook Health Sciences Centre, Toronto, Ontario, Canada
• 5 Center for Leading Injury Prevention Practice Education & Research, Toronto, Ontario, Canada
• 6 Department of Medicine, University of British Columbia, Vancouver, British Columbia, Canada
• 7 Centre for Clinical Epidemiology & Evaluation, University of British Columbia, Vancouver, British Columbia, Canada

A total solar eclipse occurs when the moon temporarily obscures the sun and casts a dark shadow across the earth. This astronomical spectacle has been described for more than 3 millennia and can be predicted with high precision. Eclipse-related solar retinopathy (vision loss from staring at the sun) is an established medical complication; however, other medical outcomes have received little attention. 1

Read More About

Redelmeier DA , Staples JA. Fatal Traffic Risks With a Total Solar Eclipse in the US. JAMA Intern Med. Published online March 25, 2024. doi:10.1001/jamainternmed.2023.5234

Manage citations:

© 2024

Artificial Intelligence Resource Center

Best of JAMA Network 2022

Browse and subscribe to JAMA Network podcasts!

Others Also Liked

Select your interests.

Customize your JAMA Network experience by selecting one or more topics from the list below.

• Academic Medicine
• Acid Base, Electrolytes, Fluids
• Allergy and Clinical Immunology
• American Indian or Alaska Natives
• Anesthesiology
• Anticoagulation
• Art and Images in Psychiatry
• Artificial Intelligence
• Assisted Reproduction
• Bleeding and Transfusion
• Caring for the Critically Ill Patient
• Challenges in Clinical Electrocardiography
• Climate and Health
• Climate Change
• Clinical Challenge
• Clinical Decision Support
• Clinical Implications of Basic Neuroscience
• Clinical Pharmacy and Pharmacology
• Complementary and Alternative Medicine
• Consensus Statements
• Coronavirus (COVID-19)
• Critical Care Medicine
• Cultural Competency
• Dental Medicine
• Dermatology
• Diabetes and Endocrinology
• Diagnostic Test Interpretation
• Drug Development
• Electronic Health Records
• Emergency Medicine
• End of Life, Hospice, Palliative Care
• Environmental Health
• Equity, Diversity, and Inclusion
• Facial Plastic Surgery
• Gastroenterology and Hepatology
• Genetics and Genomics
• Genomics and Precision Health
• Global Health
• Guide to Statistics and Methods
• Hair Disorders
• Health Care Delivery Models
• Health Care Economics, Insurance, Payment
• Health Care Quality
• Health Care Reform
• Health Care Safety
• Health Care Workforce
• Health Disparities
• Health Inequities
• Health Policy
• Health Systems Science
• History of Medicine
• Hypertension
• Images in Neurology
• Implementation Science
• Infectious Diseases
• Innovations in Health Care Delivery
• JAMA Infographic
• Law and Medicine
• Leading Change
• Less is More
• LGBTQIA Medicine
• Lifestyle Behaviors
• Medical Coding
• Medical Devices and Equipment
• Medical Education
• Medical Education and Training
• Medical Journals and Publishing
• Mobile Health and Telemedicine
• Narrative Medicine
• Neuroscience and Psychiatry
• Notable Notes
• Nutrition, Obesity, Exercise
• Obstetrics and Gynecology
• Occupational Health
• Ophthalmology
• Orthopedics
• Otolaryngology
• Pain Medicine
• Palliative Care
• Pathology and Laboratory Medicine
• Patient Care
• Patient Information
• Performance Improvement
• Performance Measures
• Perioperative Care and Consultation
• Pharmacoeconomics
• Pharmacoepidemiology
• Pharmacogenetics
• Pharmacy and Clinical Pharmacology
• Physical Medicine and Rehabilitation
• Physical Therapy
• Physician Leadership
• Population Health
• Primary Care
• Professional Well-being
• Professionalism
• Psychiatry and Behavioral Health
• Public Health
• Pulmonary Medicine
• Regulatory Agencies
• Reproductive Health
• Research, Methods, Statistics
• Resuscitation
• Rheumatology
• Risk Management
• Scientific Discovery and the Future of Medicine
• Shared Decision Making and Communication
• Sleep Medicine
• Sports Medicine
• Stem Cell Transplantation
• Substance Use and Addiction Medicine
• Surgical Innovation
• Surgical Pearls
• Teachable Moment
• Technology and Finance
• The Art of JAMA
• The Arts and Medicine
• The Rational Clinical Examination
• Tobacco and e-Cigarettes
• Translational Medicine
• Trauma and Injury
• Treatment Adherence
• Ultrasonography
• Users' Guide to the Medical Literature
• Vaccination
• Venous Thromboembolism
• Veterans Health
• Women's Health
• Workflow and Process
• Wound Care, Infection, Healing
• Register for email alerts with links to free full-text articles
• Access PDFs of free articles
• Manage your interests
• Save searches and receive search alerts