what is peer reviewed research articles

What are Peer-Reviewed Journals?

• A Definition of Peer-Reviewed
• Research Guides and Tutorials
• Library FAQ Page This link opens in a new window

Research Help

540-828-5642 [email protected] 540-318-1962

• Bridgewater College
• Church of the Brethren
• regional history materials
• the Reuel B. Pritchett Museum Collection

Additional Resources

• What are Peer-reviewed Articles and How Do I Find Them? From Capella University Libraries

Introduction

Peer-reviewed journals (also called scholarly or refereed journals) are a key information source for your college papers and projects. They are written by scholars for scholars and are an reliable source for information on a topic or discipline. These journals can be found either in the library's online databases, or in the library's local holdings. This guide will help you identify whether a journal is peer-reviewed and show you tips on finding them.

What is Peer-Review?

Peer-review is a process where an article is verified by a group of scholars before it is published.

When an author submits an article to a peer-reviewed journal, the editor passes out the article to a group of scholars in the related field (the author's peers). They review the article, making sure that its sources are reliable, the information it presents is consistent with the research, etc. Only after they give the article their "okay" is it published.

The peer-review process makes sure that only quality research is published: research that will further the scholarly work in the field.

When you use articles from peer-reviewed journals, someone has already reviewed the article and said that it is reliable, so you don't have to take the steps to evaluate the author or his/her sources. The hard work is already done for you!

Identifying Peer-Review Journals

If you have the physical journal, you can look for the following features to identify if it is peer-reviewed.

Masthead (The first few pages) : includes information on the submission process, the editorial board, and maybe even a phrase stating that the journal is "peer-reviewed."

Publisher: Peer-reviewed journals are typically published by professional organizations or associations (like the American Chemical Society). They also may be affiliated with colleges/universities.

Graphics:  Typically there either won't be any images at all, or the few charts/graphs are only there to supplement the text information. They are usually in black and white.

Authors: The authors are listed at the beginning of the article, usually with information on their affiliated institutions, or contact information like email addresses.

Abstracts: At the beginning of the article the authors provide an extensive abstract detailing their research and any conclusions they were able to draw.

Terminology:  Since the articles are written by scholars for scholars, they use uncommon terminology specific to their field and typically do not define the words used.

Citations: At the end of each article is a list of citations/reference. These are provided for scholars to either double check their work, or to help scholars who are researching in the same general area.

Advertisements: Peer-reviewed journals rarely have advertisements. If they do the ads are for professional organizations or conferences, not for national products.

Identifying Articles from Databases

When you are looking at an article in an online database, identifying that it comes from a peer-reviewed journal can be more difficult. You do not have access to the physical journal to check areas like the masthead or advertisements, but you can use some of the same basic principles.

Points you may want to keep in mind when you are evaluating an article from a database:

• A lot of databases provide you with the option to limit your results to only those from peer-reviewed or refereed journals. Choosing this option means all of your results will be from those types of sources.  
• When possible, choose the PDF version of the article's full text. Since this is exactly as if you photocopied from the journal, you can get a better idea of its layout, graphics, advertisements, etc.  
• Even in an online database you still should be able to check for author information, abstracts, terminology, and citations.
• Next: Research Guides and Tutorials >>
• Last Updated: Dec 12, 2023 4:06 PM
• URL: https://libguides.bridgewater.edu/c.php?g=945314

• Science Notes Posts
• Contact Science Notes
• Todd Helmenstine Biography
• Anne Helmenstine Biography
• Free Printable Periodic Tables (PDF and PNG)
• Periodic Table Wallpapers
• Interactive Periodic Table
• Periodic Table Posters
• How to Grow Crystals
• Chemistry Projects
• Fire and Flames Projects
• Holiday Science
• Chemistry Problems With Answers
• Physics Problems
• Unit Conversion Example Problems
• Chemistry Worksheets
• Biology Worksheets
• Periodic Table Worksheets
• Physical Science Worksheets
• Science Lab Worksheets
• My Amazon Books

Understanding Peer Review in Science

Peer review is an essential element of the scientific publishing process that helps ensure that research articles are evaluated, critiqued, and improved before release into the academic community. Take a look at the significance of peer review in scientific publications, the typical steps of the process, and and how to approach peer review if you are asked to assess a manuscript.

What Is Peer Review?

Peer review is the evaluation of work by peers, who are people with comparable experience and competency. Peers assess each others’ work in educational settings, in professional settings, and in the publishing world. The goal of peer review is improving quality, defining and maintaining standards, and helping people learn from one another.

In the context of scientific publication, peer review helps editors determine which submissions merit publication and improves the quality of manuscripts prior to their final release.

Types of Peer Review for Manuscripts

There are three main types of peer review:

• Single-blind review: The reviewers know the identities of the authors, but the authors do not know the identities of the reviewers.
• Double-blind review: Both the authors and reviewers remain anonymous to each other.
• Open peer review: The identities of both the authors and reviewers are disclosed, promoting transparency and collaboration.

There are advantages and disadvantages of each method. Anonymous reviews reduce bias but reduce collaboration, while open reviews are more transparent, but increase bias.

Key Elements of Peer Review

Proper selection of a peer group improves the outcome of the process:

• Expertise : Reviewers should possess adequate knowledge and experience in the relevant field to provide constructive feedback.
• Objectivity : Reviewers assess the manuscript impartially and without personal bias.
• Confidentiality : The peer review process maintains confidentiality to protect intellectual property and encourage honest feedback.
• Timeliness : Reviewers provide feedback within a reasonable timeframe to ensure timely publication.

Steps of the Peer Review Process

The typical peer review process for scientific publications involves the following steps:

• Submission : Authors submit their manuscript to a journal that aligns with their research topic.
• Editorial assessment : The journal editor examines the manuscript and determines whether or not it is suitable for publication. If it is not, the manuscript is rejected.
• Peer review : If it is suitable, the editor sends the article to peer reviewers who are experts in the relevant field.
• Reviewer feedback : Reviewers provide feedback, critique, and suggestions for improvement.
• Revision and resubmission : Authors address the feedback and make necessary revisions before resubmitting the manuscript.
• Final decision : The editor makes a final decision on whether to accept or reject the manuscript based on the revised version and reviewer comments.
• Publication : If accepted, the manuscript undergoes copyediting and formatting before being published in the journal.

Pros and Cons

While the goal of peer review is improving the quality of published research, the process isn’t without its drawbacks.

• Quality assurance : Peer review helps ensure the quality and reliability of published research.
• Error detection : The process identifies errors and flaws that the authors may have overlooked.
• Credibility : The scientific community generally considers peer-reviewed articles to be more credible.
• Professional development : Reviewers can learn from the work of others and enhance their own knowledge and understanding.
• Time-consuming : The peer review process can be lengthy, delaying the publication of potentially valuable research.
• Bias : Personal biases of reviews impact their evaluation of the manuscript.
• Inconsistency : Different reviewers may provide conflicting feedback, making it challenging for authors to address all concerns.
• Limited effectiveness : Peer review does not always detect significant errors or misconduct.
• Poaching : Some reviewers take an idea from a submission and gain publication before the authors of the original research.

Steps for Conducting Peer Review of an Article

Generally, an editor provides guidance when you are asked to provide peer review of a manuscript. Here are typical steps of the process.

• Accept the right assignment: Accept invitations to review articles that align with your area of expertise to ensure you can provide well-informed feedback.
• Manage your time: Allocate sufficient time to thoroughly read and evaluate the manuscript, while adhering to the journal’s deadline for providing feedback.
• Read the manuscript multiple times: First, read the manuscript for an overall understanding of the research. Then, read it more closely to assess the details, methodology, results, and conclusions.
• Evaluate the structure and organization: Check if the manuscript follows the journal’s guidelines and is structured logically, with clear headings, subheadings, and a coherent flow of information.
• Assess the quality of the research: Evaluate the research question, study design, methodology, data collection, analysis, and interpretation. Consider whether the methods are appropriate, the results are valid, and the conclusions are supported by the data.
• Examine the originality and relevance: Determine if the research offers new insights, builds on existing knowledge, and is relevant to the field.
• Check for clarity and consistency: Review the manuscript for clarity of writing, consistent terminology, and proper formatting of figures, tables, and references.
• Identify ethical issues: Look for potential ethical concerns, such as plagiarism, data fabrication, or conflicts of interest.
• Provide constructive feedback: Offer specific, actionable, and objective suggestions for improvement, highlighting both the strengths and weaknesses of the manuscript. Don’t be mean.
• Organize your review: Structure your review with an overview of your evaluation, followed by detailed comments and suggestions organized by section (e.g., introduction, methods, results, discussion, and conclusion).
• Be professional and respectful: Maintain a respectful tone in your feedback, avoiding personal criticism or derogatory language.
• Proofread your review: Before submitting your review, proofread it for typos, grammar, and clarity.
• Couzin-Frankel J (September 2013). “Biomedical publishing. Secretive and subjective, peer review proves resistant to study”. Science . 341 (6152): 1331. doi: 10.1126/science.341.6152.1331
• Lee, Carole J.; Sugimoto, Cassidy R.; Zhang, Guo; Cronin, Blaise (2013). “Bias in peer review”. Journal of the American Society for Information Science and Technology. 64 (1): 2–17. doi: 10.1002/asi.22784
• Slavov, Nikolai (2015). “Making the most of peer review”. eLife . 4: e12708. doi: 10.7554/eLife.12708
• Spier, Ray (2002). “The history of the peer-review process”. Trends in Biotechnology . 20 (8): 357–8. doi: 10.1016/S0167-7799(02)01985-6
• Squazzoni, Flaminio; Brezis, Elise; Marušić, Ana (2017). “Scientometrics of peer review”. Scientometrics . 113 (1): 501–502. doi: 10.1007/s11192-017-2518-4

Related Posts

• USU Library

Articles: Finding (and Identifying) Peer-Reviewed Articles: What is Peer Review?

• What is Peer Review?
• Finding Peer Reviewed Articles
• Databases That Can Determine Peer Review

Peer Review in 3 Minutes

What is "Peer-Review"?

What are they.

Scholarly articles are papers that describe a research study. 

Why are scholarly articles useful?

They report original research projects that have been reviewed by other experts before they are accepted for publication, so you can reasonably be assured that they contain valid information. 

How do you identify scholarly or peer-reviewed articles?

• They are usually fairly lengthy - most likely at least 7-10 pages
• The authors and their credentials should be identified, at least the company or university where the author is employed
• There is usually a list of References or Works Cited at the end of the paper, listing the sources that the authors used in their research

How do you find them? 

Some of the library's databases contain scholarly articles, either exclusively or in combination with other types of articles. 

Google Scholar is another option for searching for scholarly articles. 

Know the Difference Between Scholarly and Popular Journals/Magazines

Peer reviewed articles are found in scholarly journals.  The checklist below can help you determine if what you are looking at is peer reviewed or scholarly.

• Both kinds of journals and magazines can be useful sources of information.
• Popular magazines and newspapers are good for overviews, recent news, first-person accounts, and opinions about a topic.
• Scholarly journals, often called scientific or peer-reviewed journals, are good sources of actual studies or research conducted about a particular topic. They go through a process of review by experts, so the information is usually highly reliable.

• Next: Finding Peer Reviewed Articles >>
• Last Updated: Feb 28, 2023 10:46 AM
• URL: https://libguides.usu.edu/peer-review

This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License

Faculty and researchers : We want to hear from you! We are launching a survey to learn more about your library collection needs for teaching, learning, and research. If you would like to participate, please complete the survey by May 17, 2024. Thank you for your participation!

• University of Massachusetts Lowell
• University Libraries

Peer Reviewed Articles: What Are They?

• What is a Scholarly or Peer Reviewed Article?
• How to Tell if a Journal Article is Peer Reviewed
• Review Articles
• Types of Literature Sources, (Grey Literature)
• What are Evidence Based Reviews?

Related Information

• Video: What is Intellectual Dishonesty

If you cannot access the above video, you can watch it here

Characteristics of Scholarly Literature

Look for these:

• Often have a formal appearance with tables, graphs, and diagrams
• Always cite their sources in the form of footnotes or bibliographies
• Usually have an abstract or summary paragraph above the text; may have sections describing methodology
• Articles are written by an authority or expert in the field
• The language includes specialized terms and the jargon of the discipline
• Titles of scholarly journals often contain the word "Journal", "Review", "Bulletin", or "Research"
• Usually have a narrow or specific subject focus
• Contains original research, experimentation, or in-depth studies in the field
• Written for researchers , professors, or students in the field
• Often reviewed by the author's peers before publication (peer-reviewed or refereed)
• Advertising is minimal or none.
• Anatomy of a Scholarly Article Interactive mock-up of a scholarly article explaining the various section. from NCSU Libraries

► Peer-reviewed (or refereed):    Refers to articles that have undergone a rigorous review process, often including revisions to the original manuscript, by peers in their discipline, before publication in a scholarly journal.  This can include empirical studies, review articles, meta-analyses among others.

► Empirical study (or primary article) :    An empirical study is one that aims to gain new knowledge on a topic through direct or indirect observation and research.  These include quantitative or qualitative data and analysis. In science, an empirical article will often include the following sections:  Introduction, Methods, Results, and Discussion.

► Review article:    In the scientific literature, this is a type of article that provides a synthesis of existing research on a particular topic.  These are useful when you want to get an idea of a body of research that you are not yet familiar with.  It differs from a systematic review in that it does not aim to capture ALL of the research on a particular topic.

► Systematic review : This is a methodical and thorough literature review focused on a particular research question.  It's aim is to identify and synthesize all of the scholarly research on a particular topic in an unbiased, reproducible way to provide evidence for practice and policy-making.  It may involve a meta-analysis (see below). 

► Meta-analysis :   This is a type of research study that combines or contrasts data from different independent studies in a new analysis in order to strengthen the understanding of a particular topic.  There are many methods, some complex, applied to performing this type of analysis.

(Adapted from Scholarly Literature Types , Cornell University)

• Next: How to Tell if a Journal Article is Peer Reviewed >>
• Last Updated: Feb 14, 2024 3:00 PM
• URL: https://libguides.uml.edu/healtharticle_types

• Collections
• Services & Support

Which Source Should I Use?

• The Right Source For Your Need-Authority
• Finding Subject Specific Sources: Research Guides
• Understanding Peer Reviewed Articles
• Understanding Peer Reviewed Articles- Arts & Humanities
• How to Read a Journal Article
• Locating Journals
• How to Find Streaming Media

The Peer Review Process

So you need to use scholarly, peer-reviewed articles for an assignment...what does that mean? 

Peer review  is a process for evaluating research studies before they are published by an academic journal. These studies typically communicate  original research  or analysis for other researchers. 

The Peer Review Process at a Glance:

Looking for peer-reviewed articles?  Try searching in OneSearch or a library database  and look for options to limit your results to scholarly/peer-reviewed or academic journals. Check out this brief tutorial to show you how:   How to Locate a Scholarly (Peer Reviewed) Article

Part 1: Watch the Video

Part 1: watch the video all about peer review (3 min.) and reflect on discussion questions..

Discussion Questions

After watching the video, reflect on the following questions:

• According to the video, what are some of the pros and cons of the peer review process?
• Why is the peer review process important to scholarship?
• Do you think peer reviewers should be paid for their work? Why or why not?

Part 2: Practice

Part 2: take an interactive tutorial on reading a research article for your major..

Includes a certification of completion to download and upload to Canvas.

Social Sciences

(e.g. Psychology, Sociology)

(e.g. Health Science, Biology)

Arts & Humanities

(e.g. Visual & Media Arts, Cultural Studies, Literature, History)

Click on the handout to view in a new tab, download, or print.

For Instructors

• Teaching Peer Review for Instructors

In class or for homework, watch the video “All About Peer Review” (3 min.) .

Video discussion questions:

• According to the video, what are some of the pros and cons of the peer review process

Assignment Ideas

• Ask students to conduct their own peer review of an important journal article in your field. Ask them to reflect on the process. What was hard to critique?
• Have students examine a journals’ web page with information for authors. What information is given to the author about the peer review process for this journal?
• Assign this reading by CSUDH faculty member Terry McGlynn, "Should journals pay for manuscript reviews?" What is the author's argument? Who profits the most from published research? You could also hold a debate with one side for paying reviewers and the other side against.
• Search a database like Cabell’s for information on the journal submission process for a particular title or subject. How long does peer review take for a particular title? Is it is a blind review? How many reviewers are solicited? What is their acceptance rate?
• Assign short readings that address peer review models. We recommend this issue of Nature on peer review debate and open review and this Chronicle of Higher Education article on open review in Shakespeare Quarterly .

Proof of Completion

Mix and match this suite of instructional materials for your course needs!

Questions about integrating a graded online component into your class, contact the Online Learning Librarian, Rebecca Nowicki ( [email protected] ).

Example of a certificate of completion:

• << Previous: Finding Subject Specific Sources: Research Guides
• Next: Understanding Peer Reviewed Articles- Arts & Humanities >>
• Last Updated: Dec 11, 2023 4:29 PM
• URL: https://libguides.sdsu.edu/WhichSource

• Library databases
• Library website

Evaluating Resources: Peer Review

What is peer review.

The term peer review can be confusing, since in some of your courses you may be asked to review the work of your peers. When we talk about peer-reviewed journal articles, this has nothing to do with your peers!

Peer-reviewed journals, also called refereed journals, are journals that use a specific scholarly review process to try to ensure the accuracy and reliability of published articles. When an article is submitted to a peer-reviewed journal for publication, the journal sends the article to other scholars/experts in that field and has them review the article for accuracy and reliability.

Find out more about peer review with our Peer Review Guide:

• Peer Review Guide

Types of peer review

Single blind.

In this process, the names of the reviewers are not known to the author(s). The reviewers do know the name of the author(s).

Double blind

Here, neither reviewers or authors know each other's names.

In the open review process, both reviewers and authors know each other's names.

What about editorial review?

Journals also use an editorial review process. This is not the same as peer review. In an editorial review process an article is evaluated for style guidelines and for clarity. Reviewers here do not look at technical accuracy or errors in data or methodology, but instead look at grammar, style, and whether an article is well written.

What is the difference between scholarly and peer review?

Not all scholarly journals are peer reviewed, but all peer-reviewed journals are scholarly.

• Things that are written for a scholarly or academic audience are considered scholarly writing.
• Peer-reviewed journals are a part of the larger category of scholarly writing.
• Scholarly writing includes many resources that are not peer reviewed, such as books, textbooks, and dissertations.

Scholarly writing does not come with a label that says scholarly . You will need to evaluate the resource to see if it is

• aimed at a scholarly audience
• reporting research, theories or other types of information important to scholars
• documenting and citing sources used to help authenticate the research done

The standard peer review process only applies to journals. While scholarly writing has certainly been edited and reviewed, peer review is a specific process only used by peer-reviewed journals. Books and dissertations may be scholarly, but are not considered peer reviewed.

Check out Select the Right Source for help with what kinds of resources are appropriate for discussion posts, assignments, projects, and more:

• Select the Right Source

How do I locate or verify peer-reviewed articles?

The peer review process is initiated by the journal publisher before an article is even published. Nowhere in the article will it tell you whether or not the article has gone through a peer review process.

You can locate peer-reviewed articles in the Library databases, typically by checking a limiter box.

• Quick Answer: How do I find scholarly, peer reviewed journal articles?

You can verify whether a journal uses a peer review process by using Ulrich's Periodicals Directory.

• Quick Answer: How do I verify that my article is peer reviewed?

What about resources that are not peer-reviewed?

Limiting your search to peer review is a way that you can ensure that you're looking at scholarly journal articles, and not popular or trade publications. Because peer-reviewed articles have been vetted by experts in the field, they are viewed as being held to a higher standard, and therefore are considered to be a high quality source. Professors often prefer peer-reviewed articles because they are considered to be of higher quality.

There are times, though, when the information you need may not be available in a peer-reviewed article.

• You may need to find original work on a theory that was first published in a book.
• You may need to find very current statistical data that comes from a government website.
• You may need background information that comes from a scholarly encyclopedia.

You will want to evaluate these resources to make sure that they are the best source for the information you need.

Note: If you are required for an assignment to find information from a peer-reviewed journal, then you will not be able to use non-peer-reviewed sources such as books, dissertations, or government websites. It's always best to clarify any questions over assignments with your professor.

• Previous Page: Evaluation Methods
• Next Page: Primary & Secondary Sources
• Office of Student Disability Services

Walden Resources

Departments.

• Academic Residencies
• Academic Skills
• Career Planning and Development
• Customer Care Team
• Field Experience
• Military Services
• Student Success Advising
• Writing Skills

Centers and Offices

• Center for Social Change
• Office of Academic Support and Instructional Services
• Office of Degree Acceleration
• Office of Research and Doctoral Services
• Office of Student Affairs

Student Resources

• Doctoral Writing Assessment
• Form & Style Review
• Quick Answers
• ScholarWorks
• SKIL Courses and Workshops
• Walden Bookstore
• Walden Catalog & Student Handbook
• Student Safety/Title IX
• Legal & Consumer Information
• Website Terms and Conditions
• Cookie Policy
• Accessibility
• Accreditation
• State Authorization
• Net Price Calculator
• Contact Walden

Walden University is a member of Adtalem Global Education, Inc. www.adtalem.com Walden University is certified to operate by SCHEV © 2024 Walden University LLC. All rights reserved.

Evaluating Information Sources: What Is A Peer-Reviewed Article?

• Should I Trust Internet Sources?

What Is A Peer-Reviewed Article?

Anali Perry, a librarian from Arizona State University Libraries, gives a quick definition of a peer-reviewed article.

The Library Minute: Academic Articles from ASU Libraries on Vimeo .

How Do Peer-Reviewed Articles Differ From Popular Ones?

This 3 minute video from the Peabody Library at Vanderbilt University talks about the differences between popular and scholarly articles.  It also mentions trade publications. 

What Is Peer Review?

In academic publishing, the goal of peer review is to assess the quality of articles submitted for publication in a scholarly journal. Before an article is deemed appropriate to be published in a peer-reviewed journal, it must undergo the following process:

• The author of the article must submit it to the journal editor who forwards the article to experts in the field. Because the reviewers specialize in the same scholarly area as the author, they are considered the author’s peers (hence “peer review”).
• These impartial reviewers are charged with carefully evaluating the quality of the submitted manuscript.
• The peer reviewers check the manuscript for accuracy and assess the validity of the research methodology and procedures.
• If appropriate, they suggest revisions. If they find the article lacking in scholarly validity and rigor, they reject it.

Because a peer-reviewed journal will not publish articles that fail to meet the standards established for a given discipline, peer-reviewed articles that are accepted for publication exemplify the best research practices in a field.

Features of a Peer-Reviewed Article

When you are determining whether or not the article you found is a peer-reviewed article, you should consider the following.

Does the article have the following features?

Also consider...

• Is the journal in which you found the article published or sponsored by a professional scholarly society, professional association, or university academic department? Does it describe itself as a peer-reviewed publication? (To know that, check the journal's website). 
• Did you find a citation for it in one of the  databases that includes scholarly publications? (Academic Search Complete, PsycINFO, etc.)?  Read the database description to see if it includes scholarly publications.
• In the database, did you limit your search to scholarly or peer-reviewed publications? (See video tutorial below for a demonstration.)
• Is the topic of the article narrowly focused and explored in depth ?
• Is the article based on either original research or authorities in the field (as opposed to personal opinion)?
• Is the article written for readers with some prior knowledge of the subject?
• If your field is social or natural science, is the article divided into sections with headings such as those listed below?

How Do I Find Peer-Reviewed Articles?

The easiest and fastest way to find peer-reviewed articles is to search the online library databases , many of which include peer-reviewed journals. To make sure your results come from peer-reviewed (also called "scholarly" or "academic") journals, do the following:

• Read the database description to determine if it features peer-reviewed articles.
• When you search for articles, choose the Advanced Search option. On the search screen, look for a check-box that allows you to limit your results to peer-reviewed only.
• If you didn't check off the "peer-reviewed articles only" box, try to see if your results can organized by source . For example, the database Criminal Justice Abstracts will let you choose the tab "Peer-Reviewed Journals."

Video tutorial

Watch this video through to the end. It will show you how to use a library database and how to narrow your search results down to just peer-reviewed articles.

• << Previous: Should I Trust Internet Sources?
• Last Updated: Aug 21, 2019 2:00 PM
• URL: https://guides.lib.jjay.cuny.edu/evaluatingsources

• University of Texas Libraries
• UT Libraries

Finding Journal Articles 101

Peer-reviewed or refereed.

• Research Article
• Review Article
• By Journal Title

What Does "Peer-reviewed" or "Refereed" Mean?

Peer review is a process that journals use to ensure the articles they publish represent the best scholarship currently available. When an article is submitted to a peer reviewed journal, the editors send it out to other scholars in the same field (the author's peers) to get their opinion on the quality of the scholarship, its relevance to the field, its appropriateness for the journal, etc.

Publications that don't use peer review (Time, Cosmo, Salon) just rely on the judgment of the editors whether an article is up to snuff or not. That's why you can't count on them for solid, scientific scholarship.

Note:This is an entirely different concept from " Review Articles ."

How do I know if a journal publishes peer-reviewed articles?

Usually, you can tell just by looking. A scholarly journal is visibly different from other magazines, but occasionally it can be hard to tell, or you just want to be extra-certain. In that case, you turn to Ulrich's Periodical Directory Online . Just type the journal's title into the text box, hit "submit," and you'll get back a report that will tell you (among other things) whether the journal contains articles that are peer reviewed, or, as Ulrich's calls it, Refereed.

Remember, even journals that use peer review may have some content that does not undergo peer review. The ultimate determination must be made on an article-by-article basis.

For example, the journal  Science  publishes  a mix  of peer-reviewed and non-peer-reviewed content. Here are two articles from the same issue of  Science . 

This one is not peer-reviewed:  https://science-sciencemag-org.ezproxy.lib.utexas.edu/content/303/5655/154.1  This one is a peer-reviewed research article:  https://science-sciencemag-org.ezproxy.lib.utexas.edu/content/303/5655/226

That is consistent with the Ulrichsweb  description of  Science , which states, "Provides news of recent international developments and research in all fields of science. Publishes original research results, reviews and short features."

Test these periodicals in Ulrichs :

• Advances in Dental Research
• Clinical Anatomy
• Molecular Cancer Research
• Journal of Clinical Electrophysiology
• Last Updated: Aug 28, 2023 9:25 AM
• URL: https://guides.lib.utexas.edu/journalarticles101

Peer Reviewed Literature

What is peer review, terminology, peer review what does that mean, what types of articles are peer-reviewed, what information is not peer-reviewed, what about google scholar.

• How do I find peer-reviewed articles?
• Scholarly vs. Popular Sources

Research Librarian

For more help on this topic, please contact our Research Help Desk: [email protected] or 781-768-7303. Stay up-to-date on our current hours . Note: all hours are EST.

This Guide was created by Carolyn Swidrak (retired).

Research findings are communicated in many ways.  One of the most important ways is through publication in scholarly, peer-reviewed journals.

Research published in scholarly journals is held to a high standard.  It must make a credible and significant contribution to the discipline.  To ensure a very high level of quality, articles that are submitted to scholarly journals undergo a process called peer-review.

Once an article has been submitted for publication, it is reviewed by other independent, academic experts (at least two) in the same field as the authors.  These are the peers.  The peers evaluate the research and decide if it is good enough and important enough to publish.  Usually there is a back-and-forth exchange between the reviewers and the authors, including requests for revisions, before an article is published. 

Peer review is a rigorous process but the intensity varies by journal.  Some journals are very prestigious and receive many submissions for publication.  They publish only the very best, most highly regarded research. 

The terms scholarly, academic, peer-reviewed and refereed are sometimes used interchangeably, although there are slight differences.

Scholarly and academic may refer to peer-reviewed articles, but not all scholarly and academic journals are peer-reviewed (although most are.)  For example, the Harvard Business Review is an academic journal but it is editorially reviewed, not peer-reviewed.

Peer-reviewed and refereed are identical terms.

From  Peer Review in 3 Minutes  [Video], by the North Carolina State University Library, 2014, YouTube (https://youtu.be/rOCQZ7QnoN0).

Peer reviewed articles can include:

• Original research (empirical studies)
• Review articles
• Systematic reviews
• Meta-analyses

There is much excellent, credible information in existence that is NOT peer-reviewed.  Peer-review is simply ONE MEASURE of quality. 

Much of this information is referred to as "gray literature."

Government Agencies

Government websites such as the Centers for Disease Control (CDC) publish high level, trustworthy information.  However, most of it is not peer-reviewed.  (Some of their publications are peer-reviewed, however. The journal Emerging Infectious Diseases, published by the CDC is one example.)

Conference Proceedings

Papers from conference proceedings are not usually peer-reviewed.  They may go on to become published articles in a peer-reviewed journal. 

Dissertations

Dissertations are written by doctoral candidates, and while they are academic they are not peer-reviewed.

Many students like Google Scholar because it is easy to use.  While the results from Google Scholar are generally academic they are not necessarily peer-reviewed.  Typically, you will find:

• Peer reviewed journal articles (although they are not identified as peer-reviewed)
• Unpublished scholarly articles (not peer-reviewed)
• Masters theses, doctoral dissertations and other degree publications (not peer-reviewed)
• Book citations and links to some books (not necessarily peer-reviewed)
• Next: How do I find peer-reviewed articles? >>
• Last Updated: Feb 12, 2024 9:39 AM
• URL: https://libguides.regiscollege.edu/peer_review

Research Methods: Peer-Reviewed Journal Articles

• Getting Started
• What Type of Source?
• Credible Sources
• Finding Background Information
• Library Databases
• Reference Books
• Electronic Books
• Online Reference Collections
• Databases Presenting Two Sides of an Argument
• Peer-Reviewed Journal Articles
• Writing Style Guides and Citing Sources
• Annotated Bibliographies

What is a Peer-Reviewed (Academic) Journal?

What Is a Peer-Reviewed Journal?

Peer Review is a process that journals use to ensure the articles they publish represent the best scholarship currently available. When an article is submitted to a peer reviewed journal, the editors send it out to other scholars in the same field (the author's peers) to get their opinion on the quality of the scholarship, its relevance to the field, its appropriateness for the journal, etc.

Publications that don't use peer review (Time, Cosmo, Salon) just rely on the judgement of the editors whether an article is up to snuff or not. That's why you can't count on them for solid, scientific scholarship. --University of Texas at Austin

Databases Containing Peer-Reviewed Journal Articles

Each database containing peer-reviewed journals has different content coverage and materials.  The databases listed in this Research Guide are available only to Truckee Meadows Community College students, faculty and staff. You will need your TMCC credentials (Username and Password) to access them off-campus.

When searching a database, a search term frequently will retrieve many articles.  Browse the article abstracts to find one or more relevant to your search.

Some of the databases provide citations for the articles.

Consult a librarian for assistance.

• Databases with peer-reviewed articles and content . This list can also be sorted by subject!

How to Read a Peer-Reviewed Journal Article

Tips for Reading a Research Article

Read the Abstract. It consists of a brief summary of the research questions and methods. It may also state the findings. Because it is short and often written in dense psychological language, you may need to read it a couple of times. Try to restate the abstract in your own nontechnical language.

• Read the Introduction. This is the beginning of the article, appearing first after the Abstract. This contains information about the authors' interest in the research, why they chose the topic, their hypothesis , and methods. This part also sets out the operational definitions of variables.
• Read the Discussion section. Skip over the Methods section for the time being. The Discussion section will explain the main findings in great detail and discuss any methodological problems or flaws that the researchers discovered.
• Read the Methods section. Now that you know the results and what the researchers claim the results mean, you are prepared to read about the Methods. This section explains the type of research and the techniques and assessment instruments used. If the research utilized self-reports and questionnaires, the questions and statements used may be set out either in this section or in an appendix that appears at the end of the report.
• Read the Results section. This is the most technically challenging part of a research report. But you already know the findings (from reading about them in the Discussion section). This section explains the statistical analyses that led the authors to their conclusions.
• Read the Conclusion. The last section of the report (before any appendices) summarizes the findings, but, more important for social research, it sets out what the researchers think is the value of their research for real-life application and for public policy. This section often contains suggestions for future research, including issues that the researchers became aware of in the course of the study.
• Following the conclusions are appendices, usually tables of findings, presentations of questions and statements used in self-reports and questionnaires, and examples of forms used (such as forms for behavioral assessments).

Modified from Net Lab

• << Previous: Databases Presenting Two Sides of an Argument
• Next: Writing Style Guides and Citing Sources >>
• Last Updated: Apr 15, 2024 9:36 AM
• URL: https://libguides.tmcc.edu/researchmethods

• Library Catalogue

What is peer review? What is a peer-reviewed journal?

Peer-reviewed (or refereed) journals.

Peer-reviewed or refereed journals have an editorial board of subject experts who review and evaluate submitted articles before accepting them for publication. A journal may be a scholarly journal but not a peer-reviewed journal.

Peer review (or referee) process

• An editorial board asks subject experts to review and evaluate submitted articles before accepting them for publication in a scholarly journal.
• Submissions are evaluated using criteria including the excellence, novelty and significance of the research or ideas.
• Scholarly journals use this process to protect and maintain the quality of material they publish.
• Members of the editorial board are listed near the beginning of each journal issue.

How to tell if a journal is peer-reviewed

• If you are searching for scholarly or peer-reviewed articles in a  database , you may be able to limit your results to peer-reviewed articles. 
• If you're looking at the journal itself, search for references to their peer-review process, such as in an editorial statement, or a section with instructions to authors. 
• You can also check the entry for a journal in the Library Catalogue . Many journal records will have a note in the Description section, e.g. to say "Refereed / Peer-reviewed."

For an overview of the different types of journals, see What is a scholarly (or peer-reviewed) journal?

• Harvard Library
• Research Guides
• Faculty of Arts & Sciences Libraries

Computer Science Library Research Guide

What is peer review.

• Get Started
• How to get the full-text
• Find Books in the SEC Library This link opens in a new window
• Find Conference Proceedings
• Find Dissertations and Theses
• Find Patents This link opens in a new window
• Find Standards
• Find Technical Reports
• Find Videos
• Ask a Librarian This link opens in a new window
• << Previous: How to get the full-text
• Next: Find Books in the SEC Library >>
• Last Updated: Feb 27, 2024 1:52 PM
• URL: https://guides.library.harvard.edu/cs

Harvard University Digital Accessibility Policy

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• 16 April 2024

Structure peer review to make it more robust

• Mario Malički 0

Mario Malički is associate director of the Stanford Program on Research Rigor and Reproducibility (SPORR) and co-editor-in-chief of the Research Integrity and Peer Review journal.

You can also search for this author in PubMed   Google Scholar

You have full access to this article via your institution.

In February, I received two peer-review reports for a manuscript I’d submitted to a journal. One report contained 3 comments, the other 11. Apart from one point, all the feedback was different. It focused on expanding the discussion and some methodological details — there were no remarks about the study’s objectives, analyses or limitations.

My co-authors and I duly replied, working under two assumptions that are common in scholarly publishing: first, that anything the reviewers didn’t comment on they had found acceptable for publication; second, that they had the expertise to assess all aspects of our manuscript. But, as history has shown, those assumptions are not always accurate (see Lancet 396 , 1056; 2020 ). And through the cracks, inaccurate, sloppy and falsified research can slip.

As co-editor-in-chief of the journal Research Integrity and Peer Review (an open-access journal published by BMC, which is part of Springer Nature), I’m invested in ensuring that the scholarly peer-review system is as trustworthy as possible. And I think that to be robust, peer review needs to be more structured. By that, I mean that journals should provide reviewers with a transparent set of questions to answer that focus on methodological, analytical and interpretative aspects of a paper.

For example, editors might ask peer reviewers to consider whether the methods are described in sufficient detail to allow another researcher to reproduce the work, whether extra statistical analyses are needed, and whether the authors’ interpretation of the results is supported by the data and the study methods. Should a reviewer find anything unsatisfactory, they should provide constructive criticism to the authors. And if reviewers lack the expertise to assess any part of the manuscript, they should be asked to declare this.

Anonymizing peer review makes the process more just

Other aspects of a study, such as novelty, potential impact, language and formatting, should be handled by editors, journal staff or even machines, reducing the workload for reviewers.

The list of questions reviewers will be asked should be published on the journal’s website, allowing authors to prepare their manuscripts with this process in mind. And, as others have argued before, review reports should be published in full. This would allow readers to judge for themselves how a paper was assessed, and would enable researchers to study peer-review practices.

To see how this works in practice, since 2022 I’ve been working with the publisher Elsevier on a pilot study of structured peer review in 23 of its journals, covering the health, life, physical and social sciences. The preliminary results indicate that, when guided by the same questions, reviewers made the same initial recommendation about whether to accept, revise or reject a paper 41% of the time, compared with 31% before these journals implemented structured peer review. Moreover, reviewers’ comments were in agreement about specific parts of a manuscript up to 72% of the time ( M. Malički and B. Mehmani Preprint at bioRxiv https://doi.org/mrdv; 2024 ). In my opinion, reaching such agreement is important for science, which proceeds mainly through consensus.

Stop the peer-review treadmill. I want to get off

I invite editors and publishers to follow in our footsteps and experiment with structured peer reviews. Anyone can trial our template questions (see go.nature.com/4ab2ppc ), or tailor them to suit specific fields or study types. For instance, mathematics journals might also ask whether referees agree with the logic or completeness of a proof. Some journals might ask reviewers if they have checked the raw data or the study code. Publications that employ editors who are less embedded in the research they handle than are academics might need to include questions about a paper’s novelty or impact.

Scientists can also use these questions, either as a checklist when writing papers or when they are reviewing for journals that don’t apply structured peer review.

Some journals — including Proceedings of the National Academy of Sciences , the PLOS family of journals, F1000 journals and some Springer Nature journals — already have their own sets of structured questions for peer reviewers. But, in general, these journals do not disclose the questions they ask, and do not make their questions consistent. This means that core peer-review checks are still not standardized, and reviewers are tasked with different questions when working for different journals.

Some might argue that, because different journals have different thresholds for publication, they should adhere to different standards of quality control. I disagree. Not every study is groundbreaking, but scientists should view quality control of the scientific literature in the same way as quality control in other sectors: as a way to ensure that a product is safe for use by the public. People should be able to see what types of check were done, and when, before an aeroplane was approved as safe for flying. We should apply the same rigour to scientific research.

Ultimately, I hope for a future in which all journals use the same core set of questions for specific study types and make all of their review reports public. I fear that a lack of standard practice in this area is delaying the progress of science.

Nature 628 , 476 (2024)

doi: https://doi.org/10.1038/d41586-024-01101-9

Reprints and permissions

Competing Interests

M.M. is co-editor-in-chief of the Research Integrity and Peer Review journal that publishes signed peer review reports alongside published articles. He is also the chair of the European Association of Science Editors Peer Review Committee.

Related Articles

• Scientific community
• Peer review

Londoners see what a scientist looks like up close in 50 photographs

Career News 18 APR 24

Researchers want a ‘nutrition label’ for academic-paper facts

Nature Index 17 APR 24

Deadly diseases and inflatable suits: how I found my niche in virology research

Spotlight 17 APR 24

Is ChatGPT corrupting peer review? Telltale words hint at AI use

News 10 APR 24

Three ways ChatGPT helps me in my academic writing

Career Column 08 APR 24

Is AI ready to mass-produce lay summaries of research articles?

Nature Index 20 MAR 24

Postdoctoral Position

We are seeking highly motivated and skilled candidates for postdoctoral fellow positions

Boston, Massachusetts (US)

Boston Children's Hospital (BCH)

Qiushi Chair Professor

Distinguished scholars with notable achievements and extensive international influence.

Hangzhou, Zhejiang, China

Zhejiang University

ZJU 100 Young Professor

Promising young scholars who can independently establish and develop a research direction.

Head of the Thrust of Robotics and Autonomous Systems

Reporting to the Dean of Systems Hub, the Head of ROAS is an executive assuming overall responsibility for the academic, student, human resources...

Guangzhou, Guangdong, China

The Hong Kong University of Science and Technology (Guangzhou)

Head of Biology, Bio-island

Head of Biology to lead the discovery biology group.

BeiGene Ltd.

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Undergraduate Research

• Research is a Process
• Topic to Research Question
• Finding Books
• Finding Articles
• Evaluating Information
• Notes, Quotes, and Citing
• Organizing a Literature Review
• Building a Literature Review
• Communicating with Faculty
• Peer Review & Predators

Publishing Peer Review

• What is Peer Review?
• How does Peer Review work?
• What models does Peer Review follow?
• Are there issues with Peer Review?

According to the Committee on Publication Ethics (COPE), "Peer review is the process by which a piece of scientific research is assessed by others - a researcher's fellow peers - who are suitable qualified and able to judge the piece of work under review in terms of novelty, soundness and significance."

Peer review helps to ensure that high-quality, accurate, and ethical work is being published. It is an integral part of the research process and a staple in the academic research community. Is is often predatory publishers and journals that exploit and deny the proper peer review process. Students can also benefit from learning the peer review process. In the book Student-Led Peer Review: A Practical Guide to Implementation Across Disciplines and Modalities , the authors state that "student-led peer review is a powerful tool for learning. It helps students master academic content, improve the quality of their work, and develop self-evaluative, interpersonal, and practical workplace skills" (Lowe et. al, 2022).

Once a manuscript is submitted to a journal for publication, an editor or editorial team collaborates with a group of peer reviewers to evaluate it. Usually the editor(s) function as the initial screening layer, through which manuscripts that are deemed within scope and of high quality are passed on to peer reviewers for commenting and review. The peer reviewers then read the manuscript, making notes or commenting as they see fit. Each journal has a set of guidelines for peer reviewers to follow that can be extensive or simple. Following the review, the annotated manuscript is then returned to the author, along with the decision of the peer reviewer. Decisions can fall under five major categories:

• Accept : the reviewer believes that the manuscript is ready for publication as is.
• Minor revision : the reviewer has made a few minor suggestions for the author that, once changed, the manuscript will be ready to publish without further review.
• Major revision : This feedback means that the manuscript needs substantial work before a decision can be made to publish it. Changes can range from "a broader literature review, additional data analysis, expanded discussion of findings, or re-writing or re-organization of text" (Meyer-Junco, 2023). A second review of the revised manuscript will then be necessary to move the pages on to publication.
• Reject & Resubmit : this decision shows that while the reviewer believes that the manuscript has potential, the author needs to make considerable revisions in order for it to be published, such as improving the quality or readability, creating more depth, or being more accurate on the topic of discussion. This decision allows the author to resubmit for publication.
• Rejection : the reviewer has deemed the manuscript unsuitable for publication, noting that even with revision or re-submission, the manuscript will still not be improved enough to publish. (Summarized from Peer Review Questions & Answers: What & Why? , Meyer-Junco, L., 2023)

While the editor(s) ultimately make the final call on accepting or rejecting a manuscript, reviewer recommendations play a major role in that decision. If you are submitting a manuscript to be published, it is necessary to take the annotations and recommendations of your peer reviewers seriously in order to ensure publication.

• Single-blind: the author's name is known to the reviewers, but the reviewers are kept anonymous to the author (unless they choose to self-identify)
• Double-blind: neither the author nor the reviewers names are made known to either party (if the research is published, reviewers will eventually know the name of the author)
• Open: both the author and reviewers names are made known to the other
• Transparent: if the manuscript is accepted for publication, the reviews are also made openly available (with the names of the reviewers redacted)
• Interactive or collaborative: this refers to a review process in which either the reviewers interact with each other or with the authors. It can be either open or anonymous at any of these levels of interaction
• Post-publication: a type of open peer review that takes place after the publication of the manuscript
• Post-publication commenting: this style of review makes the published manuscript open for public comment or annotation, which can be anonymous, open, or facilitated
• Pre-print commenting: similar to post-publication commenting, this type of review makes pubic commenting available on a pre-print archive or server

Even though peer review is still considered integral to the research process, it does not come without its own set of issues. Below are just a few:

• Timeliness: Often, the entire peer review process can take an extremely long time. Hoffman (2022) cites a study that concludes delayed reviewing time averages "168 days for accepted manuscripts". This can be caused by the influx of manuscripts to review while the pool of reviewers stays the same, as well as general burn out from doing peer review work.
• Perpetuation of systemic bias : It is unfortunately true that the publication of harmful research has used the peer review process to create legitimacy for its place in the scientific record. Harrison W. Inefuku, of Iowa State University, address this problem succinctly for The Scholarly Kitchen, stating, "Embedding anti-racist and anti-oppressive practices can make peer review, and the publishing process in general, more hospitable to marginalized communities, making it a more useful tool in upholding “research integrity"" ( We Asked the Community: Is Research Integrity Possible without Peer Review? , 2022).
• Ad hominem comments: Another prominent issue is that reviewers are not following ethical guidelines and treating the peer review process as one of collaboration for the greater good. Reviews that attack the author, instead of critiquing the scholarly work are unproductive and a general waste of everyone's time. 
• Confirmation bias: This can be described as, "confirming what we believe to be true while ignoring or de-emphasizing other existing information that may challenge our intuitive believes" (Hoffman, 2022). Reviewers should try to be impartial to the material they are reviewing, focusing more on the contribution to science and knowledge, and less on whether the manuscript proves or disproves their personal beliefs.
• Lack of training: A study from 2009 found that "over 56% of the researchers interviewed indicated that there "lacked coordination and guidance in how reviews were to be conducted" (Hoffman, 2022). In the years since this study, COPE and other organizations have released Peer Review toolkits and training programs to help ensure that reviewers have resources, even when the journal they are working for does not provide them.

Despite these issues with peer review, professionals in the field remain compassionate and hopeful. An excerpt taken from a blog post on The Scholarly Kitchen quotes Katie Duffy, Senior Director of Publications at the American Society for Bone and Mineral Research, stating "We also need to recognize that the pressure on our authors and referees to work quickly and produce more and more can sometimes invite shortcuts or lapses in judgement. Peer reviewer education and strong support from editorial teams is critical to ensuring that right balance between quality and speed" ( We Asked the Community: Is Research Integrity Possible without Peer Review? , 2022).

Reputable vs Predatory Journals

• << Previous: All Things Publishing
• Last Updated: Apr 17, 2024 11:23 AM
• URL: https://guides.library.txstate.edu/undergradresearch

Lemieux Library and McGoldrick Learning Commons

Catalog search, site search.

• Seattle University
• Lemieux Library

UCOR 1600-01 Politics of the End (Professor Patrick Schoettmer)

• Scholarly/Peer Reviewed Journals
• Course Guide
• Evaluating Sources
• Find Articles
• Find Books and More
• Search Strategically
• APA Reference List - Examples

Get Research Help

Your librarian is ....

Scholarly Journals, Popular Magazine and Trade Publications

What is a Scholarly Journal?

Scholarly journals are generally published by and for experts. A publication is considered to be peer reviewed if its articles go through an official editorial process that involves review and approval by the author’s peers (people who are experts in the same subject area.) Articles in scholarly journals present new, previously unpublished research. Scholarly sources will almost always include:

• Bibliography and footnotes
• Author’s name and academic credentials

Use scholarly journals for highly focused original research.

Articles in popular magazines tend to be written by staff writers or freelance journalists and are geared towards a general audience . While most magazines adhere to editorial standards, articles do not go through a peer review process and rarely contain bibliographic citations. Popular magazines are periodicals that one typically finds at grocery stores, airport newsstands, or bookstores. Use popular magazines for a general overview of current news and opinions, or firsthand accounts of an event.

Trade publications focus on a specific profession or trade. Articles in trade magazines cover the interest of skilled laborers, technicians, and artisans. Professional magazines cover the interests of professors, librarians, and members of other fields that require advanced degrees. Subject magazines cover a topic of interest to one or more professions. Use trade magazines for overviews of news and research in a particular field .

What are the types of scholarly articles?

Scholarly articles usually fall into one of five major types: empirical studies, review articles theoretical articles, methodological or case studies. A typical article will have an abstract to summarize the article which follows. The article will introduce the problem, present a thesis statement followed by the body/methodology section.  If there is raw data, there will be a results section or if not, it could be a section called the findings section. A discussion section interprets the results in light of other studies. The last section is the conclusion which restates the thesis and suggest future research.

An empirical article contains original research. It can be either quantitative or qualitative. In format, it has an introduction (problem statement/purpose) followed by sections covering methods, results and discussion. Usually arranged chronologically.

Review articles evaluate existing published research and shows how current research relates to previous research. In the introduction, the article will define the problem of the research, then summarizes and evaluates previous research. The conclusion usually recommends possible next steps for inquiry.

Theoretical articles either advance a theory or critique a current theory.

Methodological articles either advances or modifies a methodological approach. Uses empirical data

Case studies use an individual or organzation as an illustration of a problem or solution

• << Previous: Course Guide
• Next: Evaluating Sources >>
• Last Updated: Apr 19, 2024 9:19 AM
• URL: https://library.seattleu.edu/guides/politicsend

Evidence Review of the Adverse Effects of COVID-19 Vaccination and Intramuscular Vaccine Administration

Vaccines are a public health success story, as they have prevented or lessened the effects of many infectious diseases. To address concerns around potential vaccine injuries, the Health Resources and Services Administration (HRSA) administers the Vaccine Injury Compensation Program (VICP) and the Countermeasures Injury Compensation Program (CICP), which provide compensation to those who assert that they were injured by routine vaccines or medical countermeasures, respectively. The National Academies of Sciences, Engineering, and Medicine have contributed to the scientific basis for VICP compensation decisions for decades.

HRSA asked the National Academies to convene an expert committee to review the epidemiological, clinical, and biological evidence about the relationship between COVID-19 vaccines and specific adverse events, as well as intramuscular administration of vaccines and shoulder injuries. This report outlines the committee findings and conclusions.

Read Full Description

• Digital Resource: Evidence Review of the Adverse Effects of COVID-19 Vaccination
• Digital Resource: Evidence Review of Shoulder Injuries from Intramuscular Administration of Vaccines
• Press Release

Recent News

NAS Launches Science and Innovation Fund for Ukraine

Science Academies Issue Statements to Inform G7 Talks

Supporting Family Caregivers in STEMM

A Vision for High-Quality Preschool for All

• Load More...

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• v.2(6); 2022 Nov

Effective Peer Review: Who, Where, or What?

Peer review is widely viewed as one of the most critical elements in assuring the integrity of scientific literature ( Baldwin, 2018 ; Smith, 2006 ). Despite the widespread acceptance and utilization of peer review, many difficulties with the process have been identified ( Hames, 2014 ; Horrobin, 2001 ; Smith, 2006 ). One of the primary goals of the peer review process is to identify flaws in the work and, by so doing, help editors choose which manuscripts to publish. It is surprising that one of the persistent problems in peer review is assessing the quality of the reviews. Both authors and journal editors expect peer review to detect errors in experimental design and methodology and to ensure that the interpretation of the findings is presented in an objective and thoughtful manner. In traditional peer review, two or more reviewers are asked to evaluate a manuscript on the basis of the expectation that if the two reviewers agree on the quality of the submission, the likelihood of a high-quality review is increased. Unfortunately, studies have not consistently confirmed a high degree of agreement among reviewers. Rothwell and Martynn (2000) evaluated the reproducibility of peer review in neuroscience journals and meeting abstracts and found that agreement was approximately what would be expected by chance. Similarly, Scharschmidt et al. (1994) found similar results in the evaluation of 1,000 manuscripts submitted to the Journal of Clinical Investigation, where clustering of grades in the middle resulted in an agreement being “…only marginally…” better than chance. These observations suggest that we cannot rely on the agreement of reviewers to be an indication of the quality of the reviews. Another potential way to evaluate the quality of reviews would be to assess the ability of reviewers to detect errors in submissions. It is generally accepted that detection of intentional fraud is beyond the scope of typical peer review, but we do expect reviewers to detect major and minor errors as a primary function of the traditional peer review system ( Hwang, 2006 ; Weissman, 2006 ). Schroter et al. (2008) evaluated the ability of reviewers to detect major and minor errors by introducing errors into three previously published papers describing randomized controlled clinical trials. Reviewers detected approximately three of the nine errors introduced in each manuscript. Unfortunately, reviewers who had undergone training in how to conduct a high-quality peer review were not significantly better than untrained reviewers. Similar results have been reported by Godlee et al. (1998) and Baxt et al. (1998) . Baxt et al. (1998) did report that reviewers who rejected or suggested revision of a manuscript identified more errors than those who accepted the manuscript (decision: 17.3% of major errors detected [accept], 29.6% of major errors detected [revise], and 39.1% of major errors detected [reject]). It is almost certainly true that the extent of the failure to recognize errors in submitted manuscripts may differ among scientific disciplines and journals. It also however seems likely that these observations do have some applicability to journals such as JID Innovations . It is critical that both authors and editors are cognizant of these limitations of peer review in their assessment of reviews. These findings compel journals to continue to work to develop new strategies to train and evaluate reviewers. The findings also suggest that factors beyond the failure to detect objective mistakes in a manuscript may be playing a role in the discrepancy in reviewers’ evaluations. One area of ongoing concern in the peer review process is the role of reviewer bias in assessing the scientific work of colleagues ( Kuehn, 2017 ; Lee et al, 2013 ; Tvina et al, 2019 ).

Bias in the peer review process can take many forms, including collaborator/competitor bias, affiliation bias based on an investigator’s institution or department, geographical bias based on the region or country of origin, racial bias, and gender or sex bias ( Kuehn, 2017 ; Lee et al, 2013 ; Tvina et al, 2019 ). All of these forms of bias present the risk that a decision of the reviewer will not be based solely on the quality or merit of the work but rather be influenced by a bias of the reviewer. We and other journals routinely seek to avoid selecting individuals to review work from their own institutions and ask all reviewers to declare any potential personal conflicts of interest. All these methods require either the editor or the reviewer to identify a bias and fail to address the issue of implicit or unconscious reviewer bias. The dominant method currently utilized for peer review is the so-called single-blind review, in which the identity and affiliations of the authors are known to the reviewers, whereas the identity of reviewers remains unknown to the authors. This has led to concern that knowledge of the identity of the authors and their institutions may be the source of significant reviewer bias, especially implicit bias, in the evaluation of manuscripts. Double anonymized peer review (DAPR), also known as double-blind peer review, has been suggested as a way to address this issue ( Bazi, 2020 ; Lee et al, 2013 ). Studies have compared single-blind with double-blind reviewing and reported that there is no significant difference in the quality of the reviews ( Alam et al, 2011 ; Godlee et al, 1998 ; Justice et al, 1998 ; van Rooyen et al, 1998 ). Although these studies looked at measures such as the number of errors detected, acceptance rate, and distribution of initial reviewer scores, they were not designed to address specific sources of bias such as authors’ gender, institution, or geographic location. Other studies have been undertaken to directly address the issue of bias in the peer review process. Ross et al (2006) compared the acceptance of abstracts submitted to the American Heart Association’s annual scientific meeting during a period when the reviewers knew the identity and origin of the authors (i.e., single-blind review) with when this information was not known by the reviewers (i.e., double-anonymized peer review). They found a significant increase in acceptance of non‒United States abstracts and abstracts from non-English speaking countries when the reviewers were unaware of the country of origin of the abstracts ( Ross et al, 2006 ). They also found a significant decrease in the acceptance of abstracts from prestigious institutions when the reviewers were unaware of the institutions where the work was done. In a similar study, Tomkins et al. (2017) found that papers submitted to a prestigious computer science meeting were more likely to be accepted if they were from famous authors, top universities, and top companies. Okike et al. (2016) documented similar results for manuscripts submitted to the orthopedic literature. They submitted a fabricated manuscript that was presented as being written by two prominent orthopedic surgeons (past Presidents of the American Academy of Orthopedic Surgeons) from prestigious institutions. When reviewed in the traditional single-blind fashion, which included the identity of the authors, the manuscript was accepted by 87% of the reviewers. By contrast, when the identity of the authors was unknown, the manuscript was accepted by 68% of the reviewers ( P  = 0.02) ( Blank, 1991 ). A study conducted at The American Economic Review found that authors at near-top-ranked universities experienced lower acceptance rates when authorship was anonymized ( Blank, 1991 ). Of interest, they also found that for women, there was no difference in the acceptance rate between the double-anonymized and single-blinded reviews; however, for men, the acceptance rate was lower with double-anonymized reviews.

These studies provide strong evidence that knowledge of who and where the study was performed can impact the acceptance of abstracts and manuscripts. This conflicts with the goal of the review process to base our judgments on the quality of what the results demonstrate. It is difficult to estimate how much this may affect the fate of a manuscript at JID Innovations . We do not have evidence that our review process has been impacted by bias as is reported in the studies discussed. However, neither can we state with certainty that such bias is not a factor in the reviews we receive. One of the goals of JID Innovations is to be a truly open-access journal available to all investigators in skin science from around the world. We have sought to be an outlet for studies that challenge existing paradigms or that may report negative results. We want to be seen as providing fair and objective reviews for all authors, regardless of where they work or who they are. If we are to achieve this goal, it is imperative that the who and where of a specific manuscript do not negatively impact the evaluation of the what. We want young investigators, investigators at less prestigious institutions or from less well-known laboratories, and investigators from any country around the world to be confident that their work will be judged by what they report and not by the who and the where.

To be true to this mission, JID Innovations will be initiating DAPR starting in October 2022. This is not being done because we are aware of any issues of bias with our current process of peer review but because we realize that the absence of proof is not proof of absence. As a part of this process, authors will be asked to remove identifying material from manuscripts at the time of submission in preparation for the review process ( https://www.jidinnovations.org/content/authorinfo ). As a result, primary reviewers will see only the what of the manuscript. We realize that this process involves extra work for both the authors and our staff, but we feel the benefits will outweigh this small cost. Indeed, in other journals that have taken this step, surveys have shown that both authors and reviewers ultimately prefer double-anonymized reviews ( Bennett et al, 2018 ; Moylan et al, 2014 ). We realize that achieving 100% anonymization of a manuscript is nearly impossible. Studies have shown that the rate of successful anonymizing, where the reviewers cannot discern the authorship of a manuscript, ranged from 47 to 73%. It is however interesting that even with this rate of success in the anonymizing process, a meta-analysis of trials of double- versus that of single-blind peer review has suggested an impact, with lower acceptance rates with double-anonymized peer review ( Ucci et al, 2022 ). More work clearly needs to be done to assess the value of the DAPR process, and we will be monitoring our results carefully.

The institution of DAPR in JID Innovations will assure our authors that the what of their manuscript is our focus. It does not matter who you are or where you are from. It will also emphasize to our reviewers that our focus is on the what. We will be carefully monitoring the results of this new policy and plan to report back on our experience. We also welcome your feedback on your experience as a reviewer and author for JID Innovations ; send your comments to us at [email protected] .

Finally, this decision should be seen not as the end of our efforts to improve the peer review process but merely as a first step. We will continue to work to improve all aspects of the peer review process for JID Innovations . We firmly believe that the use of double-blind -anonymized peer review will bring us closer to ensuring to our authors and readers that the work that is published by JID Innovations has been selected on the basis of what the paper reports and not on who performed the studies or where they were located.

Conflict of Interest

The author states no conflicts of interest.

• Alam M., Kim N.A., Havey J., Rademaker A., Ratner D., Tregre B., et al. Blinded vs. unblinded peer review of manuscripts submitted to a dermatology journal: a randomized multi-rater study. Br J Dermatol. 2011; 165 563‒7. [ PubMed ] [ Google Scholar ]
• Baldwin M. Scientific autonomy, public accountability, and the rise of “peer review” in the Cold War United States. Isis. 2018; 109 :538–558. [ Google Scholar ]
• Baxt W.G., Waeckerle J.F., Berlin J.A., Callaham M.L. Who reviews the reviewers? Feasibility of using a fictitious manuscript to evaluate peer reviewer performance. Ann Emerg Med. 1998; 32 310‒7. [ PubMed ] [ Google Scholar ]
• Bazi T. Peer review: single-blind, double-blind, or all the way-blind? Int Urogynecol J. 2020; 31 :481–483. [ PubMed ] [ Google Scholar ]
• Bennett K.E., Jagsi R., Zietman A. Radiation oncology authors and reviewers prefer double-blind peer review. Proc Natl Acad Sci USA. 2018; 115 :E1940. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Blank R.M. The effects of double-blind versus single-blind reviewing: experimental evidence from the American Economic Review. Am Econ Rev. 1991; 81 :1041–1067. [ Google Scholar ]
• Godlee F., Gale C.R., Martyn C.N. Effect on the quality of peer review of blinding reviewers and asking them to sign their reports: a randomized controlled trial. JAMA. 1998 280237‒40. [ PubMed ] [ Google Scholar ]
• Hames I. Peer review at the beginning of the 21st century. Sci Ed. 2014; 1 :4–8. [ Google Scholar ]
• Horrobin D.F. Something rotten at the core of science? Trends Pharmacol Sci. 2001; 22 51‒2. [ PubMed ] [ Google Scholar ]
• Hwang W.S. Can peer review police fraud? Nat Neurosci. 2006; 9 :149. [ PubMed ] [ Google Scholar ]
• Justice A.C., Cho M.K., Winker M.A., Berlin J.A., Rennie D. Does masking author identity improve peer review quality? A randomized controlled trial. PEER Investigators [published correction appears in JAMA 1998;280:968. JAMA. 1998; 280 240‒2. [ PubMed ] [ Google Scholar ]
• Kuehn B.M. Rooting out bias. ELife. 2017; 6 [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Lee C.J., Sugimoto C.R., Zhang G., Cronin B. Bias in peer review. JASIST. 2013; 64 :2–17. [ Google Scholar ]
• Moylan E.C., Harold S., O'Neill C., Kowalczuk M.K. Open, single-blind, double-blind: which peer review process do you prefer? BMC Pharmacol Toxicol. 2014; 15 :55. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Okike K., Hug K.T., Kocher M.S., Leopold S.S. Single-blind vs double-blind peer review in the setting of author prestige. JAMA. 2016; 316 1315‒6. [ PubMed ] [ Google Scholar ]
• Ross J.S., Gross C.P., Desai M.M., Hong Y., Grant A.O., Daniels S.R., et al. Effect of blinded peer review on abstract acceptance. JAMA. 2006; 295 1675‒80. [ PubMed ] [ Google Scholar ]
• Rothwell P.M., Martyn C.N. Reproducibility of peer review in clinical neuroscience. Is agreement between reviewers any greater than would be expected by chance alone? Brain. 2000; 123 1964‒9. [ PubMed ] [ Google Scholar ]
• Scharschmidt B.F., DeAmicis A., Bacchetti P., Held M.J. Chance, concurrence, and clustering. Analysis of reviewers' recommendations on 1,000 submissions to the Journal of Clinical Investigation. J Clin Invest. 1994; 93 1877‒80. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Schroter S., Black N., Evans S., Godlee F., Osorio L., Smith R. What errors do peer reviewers detect, and does training improve their ability to detect them? J R Soc Med. 2008; 101 507‒14. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Smith R. Peer review: a flawed process at the heart of science and journals. J R Soc Med. 2006; 99 178‒82. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Tomkins A., Zhang M., Heavlin W.D. Reviewer bias in single- versus double-blind peer review. Proc Natl Acad Sci USA. 2017; 114 12708‒13. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Tvina A., Spellecy R., Palatnik A. Bias in the peer review process: can we do better? Obstet Gynecol. 2019; 133 1081‒3. [ PubMed ] [ Google Scholar ]
• Ucci M.A., D'Antonio F., Berghella V. Double- vs single-blind peer review effect on acceptance rates: a systematic review and meta-analysis of randomized trials. Am J Obstet Gynecol MFM. 2022; 4 [ PubMed ] [ Google Scholar ]
• van Rooyen S., Godlee F., Evans S., Smith R., Black N. Effect of blinding and unmasking on the quality of peer review: a randomized trial. JAMA. 1998; 280 234‒7. [ PubMed ] [ Google Scholar ]
• Weissmann G. Science fraud: from patchwork mouse to patchwork data. FASEB J. 2006; 20 587‒90. [ PubMed ] [ Google Scholar ]

• Open access
• Published: 11 April 2024

The role of champions in the implementation of technology in healthcare services: a systematic mixed studies review

• Sissel Pettersen 1 ,
• Hilde Eide 2 &
• Anita Berg 1  

BMC Health Services Research volume  24 , Article number:  456 ( 2024 ) Cite this article

229 Accesses

Metrics details

Champions play a critical role in implementing technology within healthcare services. While prior studies have explored the presence and characteristics of champions, this review delves into the experiences of healthcare personnel holding champion roles, as well as the experiences of healthcare personnel interacting with them. By synthesizing existing knowledge, this review aims to inform decisions regarding the inclusion of champions as a strategy in technology implementation and guide healthcare personnel in these roles.

A systematic mixed studies review, covering qualitative, quantitative, or mixed designs, was conducted from September 2022 to March 2023. The search spanned Medline, Embase, CINAHL, and Scopus, focusing on studies published from 2012 onwards. The review centered on health personnel serving as champions in technology implementation within healthcare services. Quality assessments utilized the Mixed Methods Appraisal Tool (MMAT).

From 1629 screened studies, 23 were included. The champion role was often examined within the broader context of technology implementation. Limited studies explicitly explored experiences related to the champion role from both champions’ and health personnel’s perspectives. Champions emerged as promoters of technology, supporting its adoption. Success factors included anchoring and selection processes, champions’ expertise, and effective role performance.

The specific tasks and responsibilities assigned to champions differed across reviewed studies, highlighting that the role of champion is a broad one, dependent on the technology being implemented and the site implementing it. Findings indicated a correlation between champion experiences and organizational characteristics. The role’s firm anchoring within the organization is crucial. Limited evidence suggests that volunteering, hiring newly graduated health personnel, and having multiple champions can facilitate technology implementation. Existing studies predominantly focused on client health records and hospitals, emphasizing the need for broader research across healthcare services.

Conclusions

With a clear mandate, dedicated time, and proper training, health personnel in champion roles can significantly contribute professional, technological, and personal competencies to facilitate technology adoption within healthcare services. The review finds that the concept of champions is a broad one and finds varied definitions of the champion role concept. This underscores the importance of describing organizational characteristics, and highlights areas for future research to enhance technology implementation strategies in different healthcare settings with support of a champion.

Peer Review reports

Digital health technologies play a transformative role in healthcare service systems [ 1 , 2 ]. The utilization of technology and digitalization is essential for ensuring patient safety, delivering high quality, cost-effective, and sustainable healthcare services [ 3 , 4 ]. The implementation of technology in healthcare services is a complex process that demands systematic changes in roles, workflows, and service provision [ 5 , 6 ].

The successful implementation of new technologies in healthcare services relies on the adaptability of health professionals [ 7 , 8 , 9 ]. Champions have been identified as a key factor in the successful implementation of technology among health personnel [ 10 , 11 , 12 ]. However, they have rarely been studied as an independent strategy; instead, they are often part of a broader array of strategies in implementation studies (e.g., Hudson [ 13 ], Gullslett and Bergmo [ 14 ]). Prior research has frequently focused on determining the presence or absence of champions [ 10 , 12 , 15 ], as well as investigating the characteristics of individuals assuming the champion role (e.g., George et al. [ 16 ], Shea and Belden [ 17 ]).

Recent reviews on champions [ 18 , 19 , 20 ] have studied their effects on adherence to guidelines, implementation of innovations and facilitation of evidence-based practice. While these reviews suggest that having champions yields positive effects, they underscore the importance for studies that offer detailed insights into the champion’s role concerning specific types of interventions.

There is limited understanding of the practical role requirements and the actual experiences of health personnel in performing the champion role in the context of technology implementation within healthcare services. Further, this knowledge is needed to guide future research on the practical, professional, and relational prerequisites for health personnel in this role and for organizations to successfully employ champions as a strategy in technology implementation processes.

This review seeks to synthesize the existing empirical knowledge concerning the experiences of those in the champion role and the perspectives of health personnel involved in technology implementation processes. The aim is to contribute valuable insights that enhance our understanding of practical role requirements, the execution of the champion role, and best practices in this domain.

The term of champions varies [ 10 , 19 ] and there is a lack of explicit conceptualization of the term ‘champion’ in the implementation literature [ 12 , 18 ]. Various terms for individuals with similar roles also exist in the literature, such as implementation leader, opinion leader, facilitator, change agent, superuser and facilitator. For the purpose of this study, we have adopted the terminology utilized in the recent review by Rigby, Redley and Hutchinson [ 21 ] collectively referring to these roles as ‘champions’. This review aims to explore the experiences of health personnel in their role as champions and the experiences of health personnel interacting with them in the implementation of technology in the healthcare services.

Prior review studies on champions in healthcare services have employed various designs [ 10 , 18 , 19 , 20 ]. In this review, we utilized a comprehensive mixed studies search to identify relevant empirical studies [ 22 ]. The search was conducted utilizing the Preferred Reporting Items for Systematic and Meta-Analysis (PRISMA) guidelines, ensuring a transparent and comprehensive overview that can be replicated or updated by others [ 23 ]. The study protocol is registered in PROSPERO (ID CRD42022335750), providing a more comprehensive description of the methods [ 24 ]. A systematic mixed studies review, examining research using diverse study designs, is well-suited for synthesizing existing knowledge and identifying gaps by harnessing the strengths of both qualitative and quantitative methods [ 22 ]. Our search encompassed qualitative, quantitative, and mixed methods design to capture experiences with the role of champions in technology implementation.

Search strategy and study selection

Search strategy.

The first author, in collaboration with a librarian, developed the search strategy based on initial searches to identify appropriate terms and truncations that align with the eligibility criteria. The search was constructed utilizing a combination of MeSH terms and keywords related to technology, implementation, champion, and attitudes/experiences. Conducted in August/September 2022, the search encompassed four databases: Medline, Embase, CINAHL, and Scopus, with an updated search conducted in March 2023. The full search strategy for Medline is provided in Appendix  1 . The searches in Embase, CINAHL and Scopus employed the same strategy, with adopted terms and phrases to meet the requirements of each respective database.

Eligibility criteria

We included all empirical studies employing qualitative, quantitative, and mixed methods designs that detailed the experiences and/or attitudes of health personnel regarding the champions role in the implementation of technology in healthcare services. Articles in the English language published between 2012 and 2023 were considered. The selected studies involved technology implemented or adapted within healthcare services.

Conference abstract and review articles were excluded from consideration. Articles published prior 2012 were excluded as a result of the rapid development of technology, which could impact the experiences reported. Furthermore, articles involving surgical technology and pre-implementation studies were also excluded, as the focus was on capturing experiences and attitudes from the adoption and daily use of technology. The study also excluded articles that involved champions without clinical health care positions.

Study selection

A total of 1629 studies were identified and downloaded from the selected databases, with Covidence [ 25 ] utilized as a software platform for screening. After removing 624 duplicate records, all team members collaborated to calibrate the screening process utilizing the eligibility criteria on the initial 50 studies. Subsequently, the remaining abstracts were independently screened by two researchers, blinded to each other, to ensure adherence to the eligibility criteria. Studies were included if the title and abstract included the term champion or its synonyms, along with technology in healthcare services, implementation, and health personnel’s experiences or attitudes. Any discrepancies were resolved through consensus among all team members. A total of 949 abstracts were excluded for not meeting this inclusion condition. During the initial search, 56 remaining studies underwent full-text screening, resulting in identification of 22 studies qualified for review.

In the updated search covering the period September 2022 to March 2023, 64 new studies were identified. Of these, 18 studies underwent full-text screening, and one study was included in our review. The total number of included studies is 23. The PRISMA flowchart (Fig.  1 ) illustrates the process.

Flow Chart illustrating the study selection and screening process

Data extraction

The research team developed an extraction form for the included studies utilizing an Excel spreadsheet. Following data extraction, the information included the Name of Author(s) Year of publication, Country/countries, Title of the article, Setting, Aim, Design, Participants, and Sample size of the studies, Technology utilized in healthcare services, name/title utilized to describe the Champion Role, how the studies were analyzed and details of Attitude/Experience with the role of champion. Data extraction was conducted by SP, and the results were deliberated in a workshop with the other researchers AB, and HE until a consensus was reached. Any discrepancies were resolved through discussions. The data extraction was categorized into three categories: qualitative, quantitative, and mixed methods, in preparation for quality appraisal.

Quality appraisal

The MMAT [ 26 ] was employed to assess the quality of the 23 included studies. Specifically designed for mixed studies reviews, the MMAT allows for the appraisal of the methodological quality of studies falling into five categories. The studies in our review encompassed qualitative, quantitative descriptive, and mixed methods studies. The MMAT begins with two screening questions to confirm the empirical nature of this study. Subsequently, all studies were categorized by type and evaluated utilizing specific criteria based on their research methods, with ratings of ‘Yes,’ ‘No’ or ‘Can’t tell.’ The MMAT discourages overall scores in favor of providing a detailed explanation for each criterion. Consequently, we did not rely on the MMAT’s overall methodical quality scores and continued to include all 23 studies for our review. Two researchers independently scored the studies, and any discrepancies were discussed among all team members until a consensus was reached. The results of the MMAT assessments are provided in Appendix  2 .

Data synthesis

Based on discussions of this material, additional tables were formulated to present a comprehensive overview of the study characteristics categorized by study design, study settings, technology included, and descriptions/characteristics of the champion role. To capture attitudes and experiences associated with the champion role, the findings from the included studies were translated into narrative texts [ 22 ]. Subsequently, the reviewers worked collaboratively to conduct a thematic analysis, drawing inspiration from Braun and Clarke [ 27 ]. Throughout the synthesis process, multiple meetings were conducted to discern and define the emerging themes and subthemes.

The adopting of new technology in healthcare services can be perceived as both an event and a process. According to Iqbal [ 28 ], experience is defined as the knowledge and understanding gained after an event or the process of living through or undergoing an event. This review synthesizes existing empirical knowledge regarding the experiences of occupying the champion role, and the perspectives of health personnel interacting with champions in technology implementation processes.

Study characteristics

The review encompassed a total of 23 studies, and an overview of these studies is presented in Table  1 . Of these, fourteen studies employed a qualitative design, four had quantitative design, and five utilized a mixed method design. The geographical distribution revealed that the majority of studies were conducted in the USA (8), followed by Australia (5), England (4), Canada (2), Norway (2), Ireland (1), and Malaysia (1). In terms of settings, 11 studies were conducted in hospitals, five in primary health care, three in home-based care settings, and four in a mixed settings where two or more settings collaborated. Various technologies were employed across these studies, with client health records (7) and telemedicine (5) being the most frequently utilized. All studies included experiences from champions or health personnel collaborating with champions in their respective healthcare services. Only three studies had the champion role as a main objective [ 29 , 30 , 31 ]. The remaining studies described champions as one of the strategies in technology implementation processes, including 10 evaluation studies (including feasibility studies [ 32 , 33 , 34 ] and one cost-benefit study [ 30 ]).

Several studies underscored the importance of champions for successful implementation [ 29 , 30 , 31 , 34 , 35 , 36 , 37 , 38 , 40 , 41 , 42 , 43 , 49 ]. Four studies specifically highlighted champions as a key factor for success [ 34 , 36 , 37 , 43 ], and one study went further to describe champions as the most important factor for successful implementation [ 39 ]. Additionally, one study associated champions with reduced labor cost [ 30 ].

Thin descriptions, yet clear expectations for technology champions’ role and -attributes

The analyses revealed that the concept of champions in studies pertaining to technology implementation in healthcare services varies, primarily as a result of the diversity of terms utilized to describe the role combined with short role descriptions. Nevertheless, the studies indicated clear expectations for the champion’s role and associated attributes.

The term champion

The term champion was expressed in 20 different forms across the 23 studies included in our review. Three studies utilized multiple terms within the same study [ 32 , 47 , 48 ] and 15 different authors [ 29 , 32 , 33 , 35 , 36 , 37 , 39 , 40 , 41 , 42 , 43 , 44 , 46 , 47 , 50 ] employed the term with different compositions (Table  1 ). Furthermore, four authors utilized the term Super user [ 30 , 31 , 49 , 51 ], while four authors employed the terms Facilitator [ 38 ], IT clinician [ 48 ], Leader [ 45 ], and Manager [ 34 ], each in combination with more specific terms (such as local opinion leaders, IT nurse, or practice manager).

Most studies associated champion roles with specific professions. In seven studies, the professional title was explicitly linked to the concept of champions, such as physician champions or clinical nurse champions, or through the strategic selection of specific professions [ 29 , 33 , 36 , 40 , 43 , 47 , 50 ]. Additionally, some studies did not specify professions, but utilized terms like clinicians [ 45 ] or health professionals [ 41 ].

All included articles portray the champion’s role as facilitating implementation and daily use of technology among staff. In four studies, the champion’s role was not elaborated beyond indicating that the individual holding the role is confident with an interest in technology [ 35 , 41 , 42 , 44 ]. The champion’s role was explicitly examined in six studies [ 29 , 30 , 31 , 33 , 46 , 50 ]. Furthermore, seven studies described the champion in both the methods and results [ 32 , 36 , 38 , 47 , 48 , 49 , 51 ]. In ten of the studies, champions were solely mentioned in the results [ 34 , 35 , 37 , 39 , 38 , 39 , 40 , 41 , 42 , 43 , 44 , 45 ].

Eight studies provided a specific description or definition of the champion [ 29 , 30 , 31 , 32 , 38 , 48 , 49 , 50 ]. The champion’s role was described as involving training in the specific technology, being an expert on the technology, providing support and assisting peers when needed. In some instance, the champion had a role in leading the implementation [ 50 ], while in other situations, the champion operated as a mediator [ 48 ].

The champions tasks

In the included studies, the champion role encompassed two interrelated facilitators tasks: promoting the technology and supporting others in adopting the technology in their daily practice. Promoting the technology involved encouraging staff adaptation [ 32 , 34 , 35 , 37 , 40 , 41 , 49 ], generally described as being enthusiastic about the technology [ 32 , 35 , 37 , 41 , 48 ], influencing the attitudes and beliefs of colleagues [ 42 , 45 ] and legitimizing the introduction of the technology [ 42 , 46 , 48 ]. Supporting others in technology adaption involved training and teaching [ 31 , 35 , 38 , 40 , 51 ], as well as providing technical support [ 30 , 31 , 39 , 43 , 49 ] and social support [ 49 ]. Only four studies reported that the champions received their own training to enable them able to support their colleagues [ 30 , 31 , 39 , 48 ]. Furthermore, eight studies [ 32 , 34 , 38 , 40 , 48 , 49 , 50 , 51 ], specified that the champion role included leadership and management responsibilities, mentioning tasks such as planning, organizing, coordinating, and mediating technology adaption without providing further details.

Desirable champion attributes

To effectively fulfill their role, champions should ideally possess clinical expertise and experience [ 29 , 35 , 38 , 40 , 48 ], stay professionally updated [ 37 , 48 ], and possess knowledge of the organization and workflows [ 29 , 34 , 46 ]. They should have the ability to understand and communicate effectively with healthcare personnel [ 31 , 32 , 46 , 49 ] and be proficient in IT language [ 51 ]. Moreover, champions should demonstrate a general technological interest and competence, and competence, along with specific knowledge of the technology to be implemented [ 32 , 37 , 49 ]. It is also emphasized that they should command formal and/or informal respect and authority in the organization [ 36 , 45 ], be accessible to others [ 39 , 43 ], possess leadership qualities [ 34 , 37 , 38 , 46 ], and understand and balance the needs of stakeholders [ 43 ]. Lastly, the champions should be enthusiastic promoters of the technology, engaging and supporting others [ 31 , 32 , 33 , 34 , 37 , 39 , 40 , 41 , 43 , 49 ], while also effectively coping with cultural resistance to change [ 31 , 46 ].

Anchoring and recruiting for the champion role

The champions were organized differently within services, holding various positions in the organizations, and being recruited for the role in different ways.

Anchoring the champion role

The champion’s role is primarily anchored at two levels: the management level and/or the clinical level, with two studies having champions at both levels [ 34 , 49 ]. Those working with the management actively participated in the planning of the technology implementation [ 29 , 36 , 40 , 41 , 45 ]. Serving as advisors to management, they leveraged their clinical knowledge to guide the implementation in alignment with the necessities and possibilities of daily work routines in the clinics. Champions in this capacity experienced having a clear formal position that enabled them to fulfil their role effectively [ 29 , 40 ]. Moreover, these champions served as bridge builders between the management and department levels [ 36 , 45 ], ensuring the necessary flow of information in both directions.

Champions anchored at the clinic level played a pivotal role in the practical implementation and facilitation of the daily use of technology [ 31 , 33 , 35 , 37 , 38 , 43 , 48 , 51 ]. Additionally, these champions actively participated in meetings with senior management to discuss the technology and its implementation in the clinic. This position conferred potential influence over health personnel [ 33 , 35 ]. Champions at the clinic level facilitated collaboration between employees, management, and suppliers [ 48 ]. Fontaine et al. [ 36 ] identified respected champions at the clinical level, possessing authority and formal support from all leadership levels, as the most important factor for success.

Only one study reported that the champions received additional compensation for their role [ 36 ], while another study mentioned champions having dedicated time to fulfil their role [ 46 ]. The remaining studies did not provide this information.

Recruiting for the role as champion

Several studies have reported different experiences regarding the management’s selection of champions. A study highlighted the distinctions between a volunteered role and an appointed champion’s role [ 31 ]. Some studies underscored that appointed champions were chosen based on technological expertise and skills [ 41 , 48 , 51 ]. Moreover, the selection criteria included champions’ interest in the specific technology [ 42 ] or experiential skills [ 40 ]. The remaining studies did not provide this information.

While the champion role was most frequently held by health personnel with clinical experience, one study deviated by hiring 150 newly qualified nurses as champions [ 30 ] for a large-scale implementation of an Electronic Health Record (EHR). Opting for clinical novices assisted in reducing implementation costs, as it avoided disrupting daily tasks and interfering with daily operations. According to Bullard [ 30 ], these super-user nurses became highly sought after post-implementation as a result of their technological confidence and competence.

Reported experiences of champions and health personnel

Drawing from the experiences of both champions and health personnel, it is essential for a champion to possess a combination of general knowledge and specific champion characteristics. Furthermore, champions are required to collaborate with individuals both within and outside the organization. The subsequent paragraphs delineate these experiences, categorizing them into four subsets: champions’ contextual knowledge and expertise, preferred performance of the champion role, recognizing that a champion alone is insufficient, and distinguishing between reactive and proactive champions.

Champions’ contextual knowledge and know-how

Health personnel with experience interacting with champions emphasized that a champion must be familiar with the department and its daily work routines [ 35 , 40 ]. Knowledge of the department’s daily routines made it easier for champions to facilitate the adaptation of technology. However, there was a divergence of opinions on whether champions were required to possess extensive clinical experience to fulfil their role. In most studies, having an experienced and competent clinician as a champion instilled a sense of confidence among health personnel. Conversely, Bullard’s study [ 30 ] exhibited that health personnel were satisfied with newly qualified nurses in the role of champion, despite their initial skepticism.

It is a generally expected that champions should possess technological knowledge beyond that of other health professionals [ 37 , 41 ]. Some health personnel perceived the champions as uncritical promoters of technology, with the impression that health personnel were being compelled to utilize technology [ 46 ]. Champions could also overestimate the readiness of health personnel to implement a technology, especially during the early phases of the implementation process [ 32 ]. Regardless of whether the champion is at the management level or the clinic level, champions themselves have acknowledged the importance of providing time and space for innovation. Moreover, the recruitment of champions should span all levels of the organization [ 34 , 46 ]. Furthermore, champions must be familiar with daily work routines, work tools, and work surfaces [ 38 , 40 , 43 ].

Preferable performance of the champion role

The studies identified several preferable characteristics of successful champions. Health personnel favored champions utilizing positive words when discussing technology and exhibiting positive attitudes while facilitating and adapting it [ 33 , 34 , 37 , 38 , 41 , 46 ]. Additionally, champions who were enthusiastic and engaging were considered good role models for the adoption of technology. Successful champions were perceived as knowledgeable and adept problem solvers who motivated and supported health personnel [ 41 , 43 , 44 , 48 ]. They were also valued for being available and responding promptly when contacted [ 42 ]. Health professionals noted that champions perceived as competent garnered respect in the organization [ 40 ]. Moreover, some health personnel felt that some certain champions wielded a greater influence based on how they encouraged the use of the system [ 48 ]. It was also emphasized that health personnel needed to feel it was safe to provide feedback to champions, especially when encountering difficulties or uncertainties [ 49 ].

A champion is not enough

The role of champions proved to be more demanding than expected [ 29 , 31 , 38 ], involving tasks such as handling an overwhelming number of questions or actively participating in the installation process to ensure the technology functions effectively in the department [ 29 ]. Regardless of the organizational characteristics or the champion’s profile, appointing the champion as a “solo implementation agent” is deemed unsuitable. If the organization begins with one champion, it is recommended that this individual promptly recruits others into the role [ 42 ].

Health personnel, reliant on champions’ expertise, found it beneficial to have champions in all departments, and these champions had to be actively engaged in day-to-day operations [ 31 , 33 , 34 , 37 ]. Champions themselves also noted that health personnel increased their technological expertise through their role as champions in the department [ 39 ].

Furthermore, the successful implementation of technology requires the collaboration of various professions and support functions, a task that cannot be solely addressed by a champion [ 29 , 43 , 48 ]. In Orchard et. al.‘s study [ 34 ], champions explicitly emphasized the necessity of support from other personnel in the organization, such as those responsible for the technical aspects and archiving routines, to provide essential assistance.

According to health personnel, the role of champions is vulnerable in case they become sick or leave their position [ 42 , 51 ]. In some of the included studies, only one or a few hold the position of champion [ 37 , 38 , 42 , 48 ]. Two studies observed that their implementations were not completed because champions left or reassigned for various reasons [ 32 , 51 ]. The health professionals in the study by Owens and Charles [ 32 ] expressed that champions must be replaced in such cases. Further, the study of Olsen et al., 2021 [ 42 ] highlights the need for quicky building a champion network within the organization.

Reactive and proactive champions

Health personnel and champions alike noted that champions played both a reactive and proactive role. The proactive role entailed facilitating measures such as training and coordination [ 31 , 32 , 33 , 34 , 37 , 39 , 40 , 41 , 43 , 48 , 49 ] as initiatives to generate enthusiasm for the technology [ 31 , 32 , 33 , 34 , 35 , 37 , 39 , 40 , 41 , 43 , 49 ]. On the other hand, the reactive role entailed hands-on support and troubleshooting [ 30 , 31 , 39 , 43 , 49 ].

In a study presenting experiences from both health personnel and champions, Yuan et al. [ 31 ] found that personnel observed differences in the assistance provided by appointed and self-chosen champions. Appointed champions demonstrated the technology, answered questions from health personnel, but quickly lost patience and track of employees who had received training [ 31 ]. Health personnel perceived that self-chosen champions were proactive and well-prepared to facilitate the utilization of technology, communicating with the staff as a group and being more competent in utilizing the technology in daily practice [ 31 ]. Health personnel also noted that volunteer champions were supportive, positive, and proactive in promoting the technology, whereas appointed champions acted on request and had a more reactive approach [ 31 ].

This review underscores the breadth of the concept of champion and the significant variation in the champion’s role in implementation of technology in healthcare services. This finding supports the results from previous reviews [ 10 , 18 , 19 , 20 ]. The majority of studies meeting our inclusion criteria did not specifically focus on the experiences of champions and health personnel regarding the champion role, with the exception of studies by Bullard [ 30 ], Gui et al. [ 29 ], Helmer-Smith et al. [ 33 ], Hogan-Murphy et al. [ 46 ], Rea et al. [ 50 ], and Yuan et al. [ 31 ].

The 23 studies encompassed in this review utilized 20 different terms for the champion role. In most studies, the champion’s role was briefly described in terms of the duties it entailed or should entail. This may be linked to the fact that the role of champions was not the primary focus of the study, but rather one of the strategies in the implementation process being investigated. This result reinforces the conclusions drawn by Miech et al. [ 10 ] and Shea et al. [ 12 ] regarding the lack of united understandings of the concept. Furthermore, in Santos et al.‘s [ 19 ] review, champions were only operationalized through presence or absence in 71.4% of the included studies. However, our review finds that there is a consistent and shared understanding that champions should promote and support technology implementation.

Several studies advocate for champions as an effective and recommended strategy for implementing technology [ 30 , 31 , 33 , 34 , 35 , 36 , 37 , 38 , 39 , 40 , 42 , 43 , 45 , 46 ]. However, we identified that few studies exclusively explore health personnel`s experiences within the champion role when implementing technology in healthcare services.

This suggests a general lack of information essential for understanding the pros, cons, and prerequisites for champions as a strategy within this field of knowledge. However, this review identifies, on a general basis, the types of support and structures required for champions to perform their role successfully from the perspectives of health personnel, contributing to Shea’s conceptual model [ 12 ].

Regarding the organization of the role, this review identified champions holding both formal appointed and informal roles, working in management or clinical settings, being recruited for their clinical and/or technological expertise, and either volunteering or being hired with specific benefits for the role. Regardless of these variations, anchoring the role is crucial for both the individuals holding the champion role and the health personnel interacting with them. Anchoring, in this context, is associated with the clarity of the role’s content and a match between role expectations and opportunities for fulfilment. Furthermore, the role should be valued by the management, preferably through dedicated time and/or salary support [ 34 , 36 , 46 ]. Additionally, our findings indicate that relying on a “solo champion” is vulnerable to issues such as illness, turnover, excessive workload, and individual champion performance [ 32 , 37 ]. Based on these insights, it appears preferable to appoint multiple champions, with roles at both management and clinical levels [ 33 ].

Some studies have explored the selection of champions and its impact on role performance, revealing diverse experiences [ 30 , 31 ]. Notably, Bullard [ 30 ], stands out for emphasizing long clinical experience, and hiring newly trained nurses as superusers to facilitate the use of electronic health records. Despite facing initial reluctance, these newly trained nurses gradually succeeded in their roles. This underscores the importance of considering contextual factors in the champion selection [ 30 , 52 ]. In Bullard’s study [ 30 ], the collaboration between newly trained nurses as digital natives and clinical experienced health personnel proved beneficial, highlighting the need to align champion selection with the organization’s needs based on personal characteristics. This finding aligns with Melkas et al.‘s [ 9 ] argument that implementing technology requires a deeper understanding of users, access to contextual know-how, and health personnel’s tacit knowledge.

To meet role expectations and effectively leverage their professional and technological expertise, champions should embody personal qualities such as the ability to engage others, take a leadership role, be accessible, supportive, and communicate clearly. These qualities align with the key attributes for change in healthcare champions described by Bonawitz et al. [ 15 ]. These attributes include influence, ownership, physical presence, persuasiveness, grit, and a participative leadership style (p.5). These findings suggest that the active performance of the role, beyond mere presence, is crucial for champions to be a successful strategy in technology implementation. Moreover, the recruitment process is not inconsequential. Identifying the right person for the role and providing them with adequate training, organizational support, and dedicated time to fulfill their responsibilities emerge as an important factor based on the insights from champions and health personnel.

Strengths and limitations

While this study benefits from identifying various terms associated with the role of champions, it acknowledges the possibility of missing some studies as a result of diverse descriptions of the role. Nonetheless, a notable strength of the study lies in its specific focus on the health personnel’s experiences in holding the champion role and the broader experiences of health personnel concerning champions in technology implementation within healthcare services. This approach contributes valuable insights into the characteristics of experiences and attitudes toward the role of champions in implementing technology. Lastly, the study emphasizes the relationship between the experiences with the champion role and the organizational setting’s characteristics.

The champion role was frequently inadequately defined [ 30 , 33 , 34 , 35 , 36 , 37 , 39 , 41 , 42 , 43 , 44 , 45 , 46 , 47 , 51 ], aligning with previous reviews [ 17 , 19 , 21 ]. As indicated by van Laere and Aggestam [ 52 ], this lack of clarity complicates the identification and comparison of champions across studies. Studies that lacking a distinct definition of the champion’s role were consequently excluded. Only studies written in English were included, introducing the possibility of overlooking relevant studies based on our chosen terms for identifying the champion’s role. Most of the included studies focused on technology implementation in a general context, with champions being just one of several measures. This approach resulted in scant descriptions, as champions were often discussed in the results, discussion, or implications sections rather than being the central focus of the research.

As highlighted by Hall et al. [ 18 ]., methodological issues and inadequate reporting in studies of the champion role create challenges for conducting high-quality reviews, introducing uncertainty around the findings. We have adopted a similar approach to Santos et al. [ 19 ], including all studies even when some issues were identified during the quality assessment. Our review shares the same limitations as previous review by Santos et al. [ 19 ] on the champion role.

Practical implications, policy, and future research

The findings emphasize the significance of the relationship between experiences with the champion role and characteristics of organizational settings as crucial factors for success in the champion role. Clear anchoring of the role within the organization is vital and may impact routines, workflows, staffing, and budgets. Despite limited evidence on the experience of the champion’s role, volunteering, hiring newly graduated health personnel, and appointing more than one champion are identified as facilitators of technology implementation. This study underscores the need for future empirical research including clear descriptions of the champion roles, details on study settings and the technologies to be adopted. This will enable the determination of outcomes and success factors in holding champions in technology implementation processes, transferability of knowledge between contexts and technologies as well as enhance the comparability of studies. Furthermore, there is a need for studies to explore experiences with the champion role, preferably from the perspective of multiple stakeholders, as well as focus on the champion role within various healthcare settings.

This study emphasizes that champions can hold significant positions when provided with a clear mandate, dedicated time, and training, contributing their professional, technological, and personal competencies to expedite technology adoption within services. It appears to be an advantage if the health personnel volunteer or apply for the role to facilitate engaged and proactive champions. The implementation of technology in healthcare services demands efforts from the entire service, and the experiences highlighted in this review exhibits that champions can play an important role. Consequently, empirical studies dedicated to the champion role, employing robust designs based current knowledge, are still needed to provide solid understanding of how champions can be a successful initiative when implementing technology in healthcare services.

Data availability

This review relies exclusively on previously published studies. The datasets supporting the conclusions of this article are included within the article and its supplementary files: Description and characteristics of included studies in Table  1 , Study characteristics. The search strategy is provided in Appendix  1 , and the Critical Appraisal Summary of included studies utilizing MMAT is presented in Appendix  2 .

Abbreviations

Electronic Health Record

Implementation Outcomes Framework

Preferred Reporting Items for Systematics and Meta-Analysis

Meskó B, Drobni Z, Bényei É, Gergely B, Győrffy Z. Digital health is a cultural transformation of traditional healthcare. mHealth. 2017;3:38. https://doi.org/10.21037/mhealth.2017.08.07 .

Article   PubMed   PubMed Central   Google Scholar  

Pérez Sust P, Solans O, Fajardo JC, Medina Peralta M, Rodenas P, Gabaldà J, et al. Turning the crisis into an opportunity: Digital health strategies deployed during the COVID-19 outbreak. JMIR Public Health Surveill. 2020;6:e19106. https://doi.org/10.2196/19106 .

Alotaibi YK, Federico F. The impact of health information technology on patient safety. Saudi MedJ. 2017;38:117380. https://doi.org/10.15537/smj.2017.12.20631 .

Article   Google Scholar  

Kuoppamäki S. The application and deployment of welfare technology in Swedish municipal care: a qualitative study of procurement practices among municipal actors. BMC Health Serv Res. 2021;21:918. https://doi.org/10.1186/s12913-021-06944-w .

Kraus S, Schiavone F, Pluzhnikova A, Invernizzi AC. Digital transformation in healthcare: analyzing the current state-of-research. J Bus Res. 2021;123:557–67. https://doi.org/10.1016/j.jbusres.2020.10.030 .

Frennert S. Approaches to welfare technology in municipal eldercare. JTechnolHum. 2020;38:22646. https://doi.org/10.1080/15228835.2020.1747043 .

Konttila J, Siira H, Kyngäs H, Lahtinen M, Elo S, Kääriäinen M, et al. Healthcare professionals’ competence in digitalisation: a systematic review. Clin Nurs. 2019;28:74561. https://doi.org/10.1111/jocn.14710 .

Jacob C, Sanchez-Vazquez A, Ivory C. Social, organizational, and technological factors impacting clinicians’ adoption of mobile health tools: systematic literature review. JMIR mHealth uHealth. 2020;8:e15935. https://doi.org/10.2196/15935 .

Melkas H, Hennala L, Pekkarinen S, Kyrki V. Impacts of robot implementation on care personnel and clients in elderly-care institutions. Int J Med Inf. 2020;134:104041. https://doi.org/10.1016/j.ijmedinf.2019.104041 .

Miech EJ, Rattray NA, Flanagan ME, Damschroder L, Schmid AA, Damush TM. Inside help: an integrative review of champions in healthcare-related implementation. SAGE Open Med. 2018;6. https://doi.org/10.1177/2050312118773261 .

Foong HF, Kyaw BM, Upton Z, Tudor Car L. Facilitators and barriers of using digital technology for the management of diabetic foot ulcers: a qualitative systematic review. Int Wound J. 2020;17:126681. https://doi.org/10.1111/iwj.13396 .

Shea CM. A conceptual model to guide research on the activities and effects of innovation champions. Implement Res Pract. 2021;2. https://doi.org/10.1177/2633489521990443 .

Hudson D. Physician engagement strategies in health information system implementations. Healthc Manage Forum. 2023;36:86–9. https://doi.org/10.1177/08404704221131921 .

Article   PubMed   Google Scholar  

Gullslett MK, Strand Bergmo T. Implementation of E-prescription for multidose dispensed drugs: qualitative study of general practitioners’’ experiences. JMIR HumFactors. 2022;9:e27431. https://doi.org/10.2196/27431 .

Bonawitz K, Wetmore M, Heisler M, Dalton VK, Damschroder LJ, Forman J, et al. Champions in context: which attributes matter for change efforts in healthcare? Implement Sci. 2020;15:62. https://doi.org/10.1186/s13012-020-01024-9 .

George ER, Sabin LL, Elliott PA, Wolff JA, Osani MC, McSwiggan Hong J, et al. Examining health care champions: a mixed-methods study exploring self and peer perspectives of champions. Implement Res Pract. 2022;3. https://doi.org/10.1177/26334895221077880 .

Shea CM, Belden CM. What is the extent of research on the characteristics, behaviors, and impacts of health information technology champions? A scoping review. BMC Med Inf Decis Mak. 2016;16:2. https://doi.org/10.1186/s12911-016-0240-4 .

Hall AM, Flodgren GM, Richmond HL, Welsh S, Thompson JY, Furlong BM, Sherriff A. Champions for improved adherence to guidelines in long-term care homes: a systematic review. Implement Sci Commun. 2021;2(1):85–85. https://doi.org/10.1186/s43058-021-00185-y .

Santos WJ, Graham ID, Lalonde M, Demery Varin M, Squires JE. The effectiveness of champions in implementing innovations in health care: a systematic review. Implement Sci Commun. 2022;3(1):1–80. https://doi.org/10.1186/s43058-022-00315-0 .

Wood K, Giannopoulos V, Louie E, Baillie A, Uribe G, Lee KS, Haber PS, Morley KC. The role of clinical champions in facilitating the use of evidence-based practice in drug and alcohol and mental health settings: a systematic review. Implement Res Pract. 2020;1:2633489520959072–2633489520959072. https://doi.org/10.1177/2633489520959072 .

Rigby K, Redley B, Hutchinson AM. Change agent’s role in facilitating use of technology in residential aged care: a systematic review. Int J Med Informatics. 2023;105216. https://doi.org/10.1016/j.ijmedinf.2023.105216 .

Pluye P, Hong QN. Combining the power of stories and the power of numbers: mixed methods research and mixed studies reviews. Annu Rev Public Health. 2014;35:29–45. https://doi.org/10.1146/annurev-publhealth-032013-182440 .

Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ. 2021;372:n71. https://doi.org/10.1136/bmj.n71 .

Pettersen S, Berg A, Eide H. Experiences and attitudes to the role of champions in implementation of technology in health services. A systematic review. PROSPERO. 2022. https://www.crd.york.ac.uk/prospero/display_record.php?ID=CRD42022335750 . Accessed [15 Feb 2023].

Covidence. Better Syst Rev Manag. https://www.covidence.org/ . 2023;26.

Hong QN, Fàbregues S, Bartlett G, Boardman F, Cargo M, Dagenais P, et al. The mixed methods Appraisal Tool (MMAT) version 2018 for information professionals and researchers. Educ Inf. 2018;34:285–91. https://doi.org/10.3233/EFI-180221 .

Braun V, Clarke V. Thematic analysis: a practical guide. 1st ed. SAGE; 2022.

Iqbal MP, Manias E, Mimmo L, Mears S, Jack B, Hay L, Harrison R. Clinicians’ experience of providing care: a rapid review. BMC Health Serv Res. 2020;20:1–10. https://doi.org/10.1186/s12913-020-05812-3 .

Gui X, Chen Y, Zhou X, Reynolds TL, Zheng K, Hanauer DA. Physician champions’ perspectives and practices on electronic health records implementation: challenges and strategies. JAMIA open. 2020;3:5361. https://doi.org/10.1093/jamiaopen/ooz051 .

Bullard KL. Cost effective staffing for an EHR implementation. Nurs Econ. 2016;34:726.

Google Scholar  

Yuan CT, Bradley EH, Nembhard IM. A mixed methods study of how clinician ‘super users’ influence others during the implementation of electronic health records. BMC Med Inf Decis Mak. 2015;15:26. https://doi.org/10.1186/s12911-015-0154-6 .

Owens C, Charles N. Implementation of a text-messaging intervention for adolescents who self-harm (TeenTEXT): a feasibility study using normalisation process theory. Child Adolesc Psychiatry Ment Health. 2016;10:14. https://doi.org/10.1186/s13034-016-0101-z .

Helmer-Smith M, Fung C, Afkham A, Crowe L, Gazarin M, Keely E, et al. The feasibility of using electronic consultation in long-term care homes. JAm Med Dir Assoc. 2020;21:11661170e2. https://doi.org/10.1016/j.jamda.2020.03.003 .

Orchard J, Lowres N, Freedman SB, Ladak L, Lee W, Zwar N, et al. Screening for atrial fibrillation during influenza vaccinations by primary care nurses using a smartphone electrocardiograph (iECG): a feasibility study. Eur J Prev Cardiol. 2016;23:13–20. https://doi.org/10.1177/2047487316670255 .

Bee P, Lovell K, Airnes Z, Pruszynska A. Embedding telephone therapy in statutory mental health services: a qualitative, theory-driven analysis. BMC Psychiatry. 2016;16:56. https://doi.org/10.1186/s12888-016-0761-5 .

Fontaine P, Whitebird R, Solberg LI, Tillema J, Smithson A, Crabtree BF. Minnesota’s early experience with medical home implementation: viewpoints from the front lines. J Gen Intern Med. 2015;30(7):899–906. https://doi.org/10.1007/s11606-014-3136-y .

Kolltveit B-CH, Gjengedal E, Graue M, Iversen MM, Thorne S, Kirkevold M. Conditions for success in introducing telemedicine in diabetes foot care: a qualitative inquiry. BMC Nurs. 2017;16:2. https://doi.org/10.1186/s12912-017-0201-y .

Salbach NM, McDonald A, MacKay-Lyons M, Bulmer B, Howe JA, Bayley MT, et al. Experiences of physical therapists and professional leaders with implementing a toolkit to advance walking assessment poststroke: a realist evaluation. Phys Ther. 2021;101:1–11. https://doi.org/10.1093/ptj/pzab232 .

Schwarz M, Coccetti A, Draheim M, Gordon G. Perceptions of allied health staff of the implementation of an integrated electronic medical record across regional and metropolitan settings. Aust Health Rev. 2020;44:965–72. https://doi.org/10.1071/AH19024 .

Stewart J, McCorry N, Reid H, Hart N, Kee F. Implementation of remote asthma consulting in general practice in response to the COVID-19 pandemic: an evaluation using extended normalisation process theory. BJGP Open. 2022;6:1–10. https://doi.org/10.3399/BJGPO.2021.0189 .

Bennett-Levy J, Singer J, DuBois S, Hyde K. Translating mental health into practice: what are the barriers and enablers to e-mental health implementation by aboriginal and Torres Strait Islander health professionals? JMed. Internet Res. 2017;19:e1. https://doi.org/10.2196/jmir.6269 .

Olsen J, Peterson S, Stevens A. Implementing electronic health record-based National Diabetes Prevention Program referrals in a rural county. Public Health Nurs (Boston Mass). 2021;38(3):464–9. https://doi.org/10.1111/phn.12860 .

Yang L, Brown-Johnson CG, Miller-Kuhlmann R, Kling SMR, Saliba-Gustafsson EA, Shaw JG, et al. Accelerated launch of video visits in ambulatory neurology during COVID-19: key lessons from the Stanford experience. Neurology. 2020;95:305–11. https://doi.org/10.1212/WNL.0000000000010015 .

Article   CAS   PubMed   Google Scholar  

Buckingham SA, Sein K, Anil K, Demain S, Gunn H, Jones RB, et al. Telerehabilitation for physical disabilities and movement impairment: a service evaluation in South West England. JEval Clin Pract. 2022;28:108495. https://doi.org/10.1111/jep.13689 .

Chung OS, Robinson T, Johnson AM, Dowling NL, Ng CH, Yücel M, et al. Implementation of therapeutic virtual reality into psychiatric care: clinicians’ and service managers’’ perspectives. Front Psychiatry. 2022;12:791123. https://doi.org/10.3389/fpsyt.2021.791123 .

Hogan-Murphy D, Stewart D, Tonna A, Strath A, Cunningham S. Use of normalization process theory to explore key stakeholders’ perceptions of the facilitators and barriers to implementing electronic systems for medicines management in hospital settings. Res SocialAdm Pharm. 2021;17:398405. https://doi.org/10.1016/j.sapharm.2020.03.005 .

Moss SR, Martinez KA, Nathan C, Pfoh ER, Rothberg MB. Physicians’ views on utilization of an electronic health record-embedded calculator to assess risk for venous thromboembolism among medical inpatients: a qualitative study. TH Open. 2022;6:e33–9. https://doi.org/10.1055/s-0041-1742227 .

Yusof MM. A case study evaluation of a critical Care Information System adoption using the socio-technical and fit approach. Int J Med Inf. 2015;84:486–99. https://doi.org/10.1016/j.ijmedinf.2015.03.001 .

Dugstad J, Sundling V, Nilsen ER, Eide H. Nursing staff’s evaluation of facilitators and barriers during implementation of wireless nurse call systems in residential care facilities. A cross-sectional study. BMC Health Serv Res. 2020;20:163. https://doi.org/10.1186/s12913-020-4998-9 .

Rea K, Le-Jenkins U, Rutledge C. A technology intervention for nurses engaged in preventing catheter-associated urinary tract infections. Comput Inf Nurs. 2018;36:305–13. https://doi.org/10.1097/CIN.0000000000000429 .

Bail K, Davey R, Currie M, Gibson J, Merrick E, Redley B. Implementation pilot of a novel electronic bedside nursing chart: a mixed-methods case study. Aust Health Rev. 2020;44:672–6. https://doi.org/10.1071/AH18231 .

van Laere J, Aggestam L. Understanding champion behaviour in a health-care information system development project – how multiple champions and champion behaviours build a coherent whole. Eur J Inf Syst. 2016;25:47–63. https://doi.org/10.1057/ejis.2015.5 .

Download references

Acknowledgements

We would like to thank the librarian Malin E. Norman, at Nord university, for her assistance in the development of the search, as well as guidance regarding the scientific databases.

This study is a part of a PhD project undertaken by the first author, SP, and funded by Nord University, Norway. This research did not receive any specific grant from funding agencies in the public, commercial, as well as not-for-profit sectors.

Open access funding provided by Nord University

Author information

Authors and affiliations.

Faculty of Nursing and Health Sciences, Nord university, P.O. Box 474, N-7801, Namsos, Norway

Sissel Pettersen & Anita Berg

Centre for Health and Technology, Faculty of Health Sciences, University of South-Eastern Norway, PO Box 7053, N-3007, Drammen, Norway

You can also search for this author in PubMed   Google Scholar

Contributions

The first author/SP has been the project manager and was mainly responsible for all phases of the study. The second and third authors HE and AB have contributed to screening, quality assessment, analysis and discussion of findings. Drafting of the final manuscript has been a collaboration between the first/SP and third athor/AB. The final manuscript has been approved by all authors.

Corresponding author

Correspondence to Sissel Pettersen .

Ethics declarations

Ethics approval and consent to participate.

This review does not involve the processing of personal data, and given the nature of this study, formal consent is not applicable.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary Material 1

Supplementary material 2, rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Pettersen, S., Eide, H. & Berg, A. The role of champions in the implementation of technology in healthcare services: a systematic mixed studies review. BMC Health Serv Res 24 , 456 (2024). https://doi.org/10.1186/s12913-024-10867-7

Download citation

Received : 19 June 2023

Accepted : 14 March 2024

Published : 11 April 2024

DOI : https://doi.org/10.1186/s12913-024-10867-7

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Technology implementation
• Healthcare personnel
• Healthcare services
• Mixed methods
• Organizational characteristics
• Technology adoption
• Role definitions
• Healthcare settings
• Systematic review

BMC Health Services Research

ISSN: 1472-6963

• General enquiries: [email protected]

SYSTEMATIC REVIEW article

This article is part of the research topic.

Ethnic Inequalities in Diabetes Care and Outcomes

Acceptability of Community Health Worker and Peer Supported Interventions for Minoritized Populations with Type 2 Diabetes: A Qualitative Systematic Review Provisionally Accepted

• 1 University of Birmingham, United Kingdom

The final, formatted version of the article will be published soon.

Ethnic minority groups in high income countries in North America, Europe, and elsewhere are disproportionately affected by T2DM with a higher risk of mortality and morbidity. The use of community health workers and peer supporters offer a way of ensuring the benefits of self-management support observed in the general population are shared by those in minoritized communities.The major databases were searched for existing qualitative evidence of participants' experiences and perspectives of self-management support for type 2 diabetes delivered by community health workers and peer supporters (CHWPs) in ethnically minoritized populations. The data were analysed using Sekhon's Theoretical Framework of Acceptability.The results are described within five domains of the framework of acceptability collapsed from seven for reasons of clarity and concision: Affective attitude described participants' satisfaction with CHWPs delivering the intervention including the open, trusting relationships that developed in contrast to those with clinical providers. In considering Burden and Opportunity Costs, participants reflected on the impact of health, transport, and the responsibilities of work and childcare on their attendance, alongside a lack of resources necessary to maintain healthy diets and active lifestyles. In relation to Cultural Sensitivity participants appreciated the greater understanding of the specific cultural needs and challenges exhibited by CHWPs. The evidence related to Intervention Coherence indicated that participants responded positively to the practical and applied content, the range of teaching materials, and interactive practical sessions. Finally, in examining the impact of Effectiveness and Self-efficacy participants described how they changed a range of healthrelated behaviours, had more confidence in dealing with their condition and interacting with senior clinicians and benefitted from the social support of fellow participants and CHWPs.Many of the same barriers around attendance and engagement adherence towith usual self-management support interventions delivered to general populations were observed, including lack of time and resource. However, the insight of CHWPs, their culturallysensitive and specific strategies for self-management and their development of trusting relationships presented considerable advantages.

Keywords: type 2 diabetes, self-management, Community Health, health inequalities, Ethnic minorities Library, Places of Worship, Sports centres, Community Halls, a Mixed methods studies

Received: 03 Oct 2023; Accepted: 26 Feb 2024.

Copyright: © 2024 Grant and Litchfield. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY) . The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

* Correspondence: Mx. Ian Litchfield, University of Birmingham, Birmingham, United Kingdom

People also looked at