critical thinking in clinical research pdf

Search Menu
Browse content in Arts and Humanities
Browse content in Archaeology
Anglo-Saxon and Medieval Archaeology
Archaeological Methodology and Techniques
Archaeology by Region
Archaeology of Religion
Archaeology of Trade and Exchange
Biblical Archaeology
Contemporary and Public Archaeology
Environmental Archaeology
Historical Archaeology
History and Theory of Archaeology
Industrial Archaeology
Landscape Archaeology
Mortuary Archaeology
Prehistoric Archaeology
Underwater Archaeology
Urban Archaeology
Zooarchaeology
Browse content in Architecture
Architectural Structure and Design
History of Architecture
Residential and Domestic Buildings
Theory of Architecture
Browse content in Art
Art Subjects and Themes
History of Art
Industrial and Commercial Art
Theory of Art
Biographical Studies
Byzantine Studies
Browse content in Classical Studies
Classical Literature
Classical Reception
Classical History
Classical Philosophy
Classical Mythology
Classical Art and Architecture
Classical Oratory and Rhetoric
Greek and Roman Archaeology
Greek and Roman Papyrology
Greek and Roman Epigraphy
Greek and Roman Law
Late Antiquity
Religion in the Ancient World
Digital Humanities
Browse content in History
Colonialism and Imperialism
Diplomatic History
Environmental History
Genealogy, Heraldry, Names, and Honours
Genocide and Ethnic Cleansing
Historical Geography
History by Period
History of Agriculture
History of Education
History of Emotions
History of Gender and Sexuality
Industrial History
Intellectual History
International History
Labour History
Legal and Constitutional History
Local and Family History
Maritime History
Military History
National Liberation and Post-Colonialism
Oral History
Political History
Public History
Regional and National History
Revolutions and Rebellions
Slavery and Abolition of Slavery
Social and Cultural History
Theory, Methods, and Historiography
Urban History
World History
Browse content in Language Teaching and Learning
Language Learning (Specific Skills)
Language Teaching Theory and Methods
Browse content in Linguistics
Applied Linguistics
Cognitive Linguistics
Computational Linguistics
Forensic Linguistics
Grammar, Syntax and Morphology
Historical and Diachronic Linguistics
History of English
Language Variation
Language Families
Language Evolution
Language Reference
Language Acquisition
Lexicography
Linguistic Theories
Linguistic Typology
Linguistic Anthropology
Phonetics and Phonology
Psycholinguistics
Sociolinguistics
Translation and Interpretation
Writing Systems
Browse content in Literature
Bibliography
Children's Literature Studies
Literary Studies (Modernism)
Literary Studies (Romanticism)
Literary Studies (American)
Literary Studies (Asian)
Literary Studies (European)
Literary Studies (Eco-criticism)
Literary Studies - World
Literary Studies (1500 to 1800)
Literary Studies (19th Century)
Literary Studies (20th Century onwards)
Literary Studies (African American Literature)
Literary Studies (British and Irish)
Literary Studies (Early and Medieval)
Literary Studies (Fiction, Novelists, and Prose Writers)
Literary Studies (Gender Studies)
Literary Studies (Graphic Novels)
Literary Studies (History of the Book)
Literary Studies (Plays and Playwrights)
Literary Studies (Poetry and Poets)
Literary Studies (Postcolonial Literature)
Literary Studies (Queer Studies)
Literary Studies (Science Fiction)
Literary Studies (Travel Literature)
Literary Studies (War Literature)
Literary Studies (Women's Writing)
Literary Theory and Cultural Studies
Mythology and Folklore
Shakespeare Studies and Criticism
Browse content in Media Studies
Browse content in Music
Applied Music
Dance and Music
Ethics in Music
Ethnomusicology
Gender and Sexuality in Music
Medicine and Music
Music Cultures
Music and Culture
Music and Media
Music and Religion
Music Education and Pedagogy
Music Theory and Analysis
Musical Scores, Lyrics, and Libretti
Musical Structures, Styles, and Techniques
Musicology and Music History
Performance Practice and Studies
Race and Ethnicity in Music
Sound Studies
Browse content in Performing Arts
Browse content in Philosophy
Aesthetics and Philosophy of Art
Epistemology
Feminist Philosophy
History of Western Philosophy
Metaphysics
Moral Philosophy
Non-Western Philosophy
Philosophy of Action
Philosophy of Law
Philosophy of Religion
Philosophy of Language
Philosophy of Mind
Philosophy of Perception
Philosophy of Science
Philosophy of Mathematics and Logic
Practical Ethics
Social and Political Philosophy
Browse content in Religion
Biblical Studies
Christianity
East Asian Religions
History of Religion
Judaism and Jewish Studies
Qumran Studies
Religion and Education
Religion and Health
Religion and Politics
Religion and Science
Religion and Law
Religion and Art, Literature, and Music
Religious Studies
Browse content in Society and Culture
Cookery, Food, and Drink
Cultural Studies
Customs and Traditions
Ethical Issues and Debates
Hobbies, Games, Arts and Crafts
Lifestyle, Home, and Garden
Natural world, Country Life, and Pets
Popular Beliefs and Controversial Knowledge
Sports and Outdoor Recreation
Technology and Society
Travel and Holiday
Visual Culture
Browse content in Law
Arbitration
Browse content in Company and Commercial Law
Commercial Law
Company Law
Browse content in Comparative Law
Systems of Law
Competition Law
Browse content in Constitutional and Administrative Law
Government Powers
Judicial Review
Local Government Law
Military and Defence Law
Parliamentary and Legislative Practice
Construction Law
Contract Law
Browse content in Criminal Law
Criminal Procedure
Criminal Evidence Law
Sentencing and Punishment
Employment and Labour Law
Environment and Energy Law
Browse content in Financial Law
Banking Law
Insolvency Law
History of Law
Human Rights and Immigration
Intellectual Property Law
Browse content in International Law
Private International Law and Conflict of Laws
Public International Law
IT and Communications Law
Jurisprudence and Philosophy of Law
Law and Society
Law and Politics
Browse content in Legal System and Practice
Courts and Procedure
Legal Skills and Practice
Primary Sources of Law
Regulation of Legal Profession
Medical and Healthcare Law
Browse content in Policing
Criminal Investigation and Detection
Police and Security Services
Police Procedure and Law
Police Regional Planning
Browse content in Property Law
Personal Property Law
Study and Revision
Terrorism and National Security Law
Browse content in Trusts Law
Wills and Probate or Succession
Browse content in Medicine and Health
Browse content in Allied Health Professions
Arts Therapies
Clinical Science
Dietetics and Nutrition
Occupational Therapy
Operating Department Practice
Physiotherapy
Radiography
Speech and Language Therapy
Browse content in Anaesthetics
General Anaesthesia
Neuroanaesthesia
Clinical Neuroscience
Browse content in Clinical Medicine
Acute Medicine
Cardiovascular Medicine
Clinical Genetics
Clinical Pharmacology and Therapeutics
Dermatology
Endocrinology and Diabetes
Gastroenterology
Genito-urinary Medicine
Geriatric Medicine
Infectious Diseases
Medical Oncology
Medical Toxicology
Pain Medicine
Palliative Medicine
Rehabilitation Medicine
Respiratory Medicine and Pulmonology
Rheumatology
Sleep Medicine
Sports and Exercise Medicine
Community Medical Services
Critical Care
Emergency Medicine
Forensic Medicine
Haematology
History of Medicine
Medical Ethics
Browse content in Medical Skills
Clinical Skills
Communication Skills
Nursing Skills
Surgical Skills
Browse content in Medical Dentistry
Oral and Maxillofacial Surgery
Paediatric Dentistry
Restorative Dentistry and Orthodontics
Surgical Dentistry
Medical Statistics and Methodology
Browse content in Neurology
Clinical Neurophysiology
Neuropathology
Nursing Studies
Browse content in Obstetrics and Gynaecology
Gynaecology
Occupational Medicine
Ophthalmology
Otolaryngology (ENT)
Browse content in Paediatrics
Neonatology
Browse content in Pathology
Chemical Pathology
Clinical Cytogenetics and Molecular Genetics
Histopathology
Medical Microbiology and Virology
Patient Education and Information
Browse content in Pharmacology
Psychopharmacology
Browse content in Popular Health
Caring for Others
Complementary and Alternative Medicine
Self-help and Personal Development
Browse content in Preclinical Medicine
Cell Biology
Molecular Biology and Genetics
Reproduction, Growth and Development
Primary Care
Professional Development in Medicine
Browse content in Psychiatry
Addiction Medicine
Child and Adolescent Psychiatry
Forensic Psychiatry
Learning Disabilities
Old Age Psychiatry
Psychotherapy
Browse content in Public Health and Epidemiology
Epidemiology
Public Health
Browse content in Radiology
Clinical Radiology
Interventional Radiology
Nuclear Medicine
Radiation Oncology
Reproductive Medicine
Browse content in Surgery
Cardiothoracic Surgery
Gastro-intestinal and Colorectal Surgery
General Surgery
Neurosurgery
Paediatric Surgery
Peri-operative Care
Plastic and Reconstructive Surgery
Surgical Oncology
Transplant Surgery
Trauma and Orthopaedic Surgery
Vascular Surgery
Browse content in Science and Mathematics
Browse content in Biological Sciences
Aquatic Biology
Biochemistry
Bioinformatics and Computational Biology
Developmental Biology
Ecology and Conservation
Evolutionary Biology
Genetics and Genomics
Microbiology
Molecular and Cell Biology
Natural History
Plant Sciences and Forestry
Research Methods in Life Sciences
Structural Biology
Systems Biology
Zoology and Animal Sciences
Browse content in Chemistry
Analytical Chemistry
Computational Chemistry
Crystallography
Environmental Chemistry
Industrial Chemistry
Inorganic Chemistry
Materials Chemistry
Medicinal Chemistry
Mineralogy and Gems
Organic Chemistry
Physical Chemistry
Polymer Chemistry
Study and Communication Skills in Chemistry
Theoretical Chemistry
Browse content in Computer Science
Artificial Intelligence
Computer Architecture and Logic Design
Game Studies
Human-Computer Interaction
Mathematical Theory of Computation
Programming Languages
Software Engineering
Systems Analysis and Design
Virtual Reality
Browse content in Computing
Business Applications
Computer Games
Computer Security
Computer Networking and Communications
Digital Lifestyle
Graphical and Digital Media Applications
Operating Systems
Browse content in Earth Sciences and Geography
Atmospheric Sciences
Environmental Geography
Geology and the Lithosphere
Maps and Map-making
Meteorology and Climatology
Oceanography and Hydrology
Palaeontology
Physical Geography and Topography
Regional Geography
Soil Science
Urban Geography
Browse content in Engineering and Technology
Agriculture and Farming
Biological Engineering
Civil Engineering, Surveying, and Building
Electronics and Communications Engineering
Energy Technology
Engineering (General)
Environmental Science, Engineering, and Technology
History of Engineering and Technology
Mechanical Engineering and Materials
Technology of Industrial Chemistry
Transport Technology and Trades
Browse content in Environmental Science
Applied Ecology (Environmental Science)
Conservation of the Environment (Environmental Science)
Environmental Sustainability
Environmentalist Thought and Ideology (Environmental Science)
Management of Land and Natural Resources (Environmental Science)
Natural Disasters (Environmental Science)
Nuclear Issues (Environmental Science)
Pollution and Threats to the Environment (Environmental Science)
Social Impact of Environmental Issues (Environmental Science)
History of Science and Technology
Browse content in Materials Science
Ceramics and Glasses
Composite Materials
Metals, Alloying, and Corrosion
Nanotechnology
Browse content in Mathematics
Applied Mathematics
Biomathematics and Statistics
History of Mathematics
Mathematical Education
Mathematical Finance
Mathematical Analysis
Numerical and Computational Mathematics
Probability and Statistics
Pure Mathematics
Browse content in Neuroscience
Cognition and Behavioural Neuroscience
Development of the Nervous System
Disorders of the Nervous System
History of Neuroscience
Invertebrate Neurobiology
Molecular and Cellular Systems
Neuroendocrinology and Autonomic Nervous System
Neuroscientific Techniques
Sensory and Motor Systems
Browse content in Physics
Astronomy and Astrophysics
Atomic, Molecular, and Optical Physics
Biological and Medical Physics
Classical Mechanics
Computational Physics
Condensed Matter Physics
Electromagnetism, Optics, and Acoustics
History of Physics
Mathematical and Statistical Physics
Measurement Science
Nuclear Physics
Particles and Fields
Plasma Physics
Quantum Physics
Relativity and Gravitation
Semiconductor and Mesoscopic Physics
Browse content in Psychology
Affective Sciences
Clinical Psychology
Cognitive Neuroscience
Cognitive Psychology
Criminal and Forensic Psychology
Developmental Psychology
Educational Psychology
Evolutionary Psychology
Health Psychology
History and Systems in Psychology
Music Psychology
Neuropsychology
Organizational Psychology
Psychological Assessment and Testing
Psychology of Human-Technology Interaction
Psychology Professional Development and Training
Research Methods in Psychology
Social Psychology
Browse content in Social Sciences
Browse content in Anthropology
Anthropology of Religion
Human Evolution
Medical Anthropology
Physical Anthropology
Regional Anthropology
Social and Cultural Anthropology
Theory and Practice of Anthropology
Browse content in Business and Management
Business History
Business Ethics
Business Strategy
Business and Technology
Business and Government
Business and the Environment
Comparative Management
Corporate Governance
Corporate Social Responsibility
Entrepreneurship
Health Management
Human Resource Management
Industrial and Employment Relations
Industry Studies
Information and Communication Technologies
International Business
Knowledge Management
Management and Management Techniques
Operations Management
Organizational Theory and Behaviour
Pensions and Pension Management
Public and Nonprofit Management
Strategic Management
Supply Chain Management
Browse content in Criminology and Criminal Justice
Criminal Justice
Criminology
Forms of Crime
International and Comparative Criminology
Youth Violence and Juvenile Justice
Development Studies
Browse content in Economics
Agricultural, Environmental, and Natural Resource Economics
Asian Economics
Behavioural Finance
Behavioural Economics and Neuroeconomics
Econometrics and Mathematical Economics
Economic Methodology
Economic History
Economic Systems
Economic Development and Growth
Financial Markets
Financial Institutions and Services
General Economics and Teaching
Health, Education, and Welfare
History of Economic Thought
International Economics
Labour and Demographic Economics
Law and Economics
Macroeconomics and Monetary Economics
Microeconomics
Public Economics
Urban, Rural, and Regional Economics
Welfare Economics
Browse content in Education
Adult Education and Continuous Learning
Care and Counselling of Students
Early Childhood and Elementary Education
Educational Equipment and Technology
Educational Strategies and Policy
Higher and Further Education
Organization and Management of Education
Philosophy and Theory of Education
Schools Studies
Secondary Education
Teaching of a Specific Subject
Teaching of Specific Groups and Special Educational Needs
Teaching Skills and Techniques
Browse content in Environment
Applied Ecology (Social Science)
Climate Change
Conservation of the Environment (Social Science)
Environmentalist Thought and Ideology (Social Science)
Natural Disasters (Environment)
Social Impact of Environmental Issues (Social Science)
Browse content in Human Geography
Cultural Geography
Economic Geography
Political Geography
Browse content in Interdisciplinary Studies
Communication Studies
Museums, Libraries, and Information Sciences
Browse content in Politics
African Politics
Asian Politics
Chinese Politics
Comparative Politics
Conflict Politics
Elections and Electoral Studies
Environmental Politics
European Union
Foreign Policy
Gender and Politics
Human Rights and Politics
Indian Politics
International Relations
International Organization (Politics)
International Political Economy
Irish Politics
Latin American Politics
Middle Eastern Politics
Political Theory
Political Behaviour
Political Economy
Political Institutions
Political Methodology
Political Communication
Political Philosophy
Political Sociology
Politics and Law
Public Policy
Public Administration
Quantitative Political Methodology
Regional Political Studies
Russian Politics
Security Studies
State and Local Government
UK Politics
US Politics
Browse content in Regional and Area Studies
African Studies
Asian Studies
East Asian Studies
Japanese Studies
Latin American Studies
Middle Eastern Studies
Native American Studies
Scottish Studies
Browse content in Research and Information
Research Methods
Browse content in Social Work
Addictions and Substance Misuse
Adoption and Fostering
Care of the Elderly
Child and Adolescent Social Work
Couple and Family Social Work
Developmental and Physical Disabilities Social Work
Direct Practice and Clinical Social Work
Emergency Services
Human Behaviour and the Social Environment
International and Global Issues in Social Work
Mental and Behavioural Health
Social Justice and Human Rights
Social Policy and Advocacy
Social Work and Crime and Justice
Social Work Macro Practice
Social Work Practice Settings
Social Work Research and Evidence-based Practice
Welfare and Benefit Systems
Browse content in Sociology
Childhood Studies
Community Development
Comparative and Historical Sociology
Economic Sociology
Gender and Sexuality
Gerontology and Ageing
Health, Illness, and Medicine
Marriage and the Family
Migration Studies
Occupations, Professions, and Work
Organizations
Population and Demography
Race and Ethnicity
Social Theory
Social Movements and Social Change
Social Research and Statistics
Social Stratification, Inequality, and Mobility
Sociology of Religion
Sociology of Education
Sport and Leisure
Urban and Rural Studies
Browse content in Warfare and Defence
Defence Strategy, Planning, and Research
Land Forces and Warfare
Military Administration
Military Life and Institutions
Naval Forces and Warfare
Other Warfare and Defence Issues
Peace Studies and Conflict Resolution
Weapons and Equipment

Critical Thinking in Clinical Research: Applied Theory and Practice Using Case Studies (1)

< Previous chapter
Next chapter >

2 Selection of the Research Question

Published: March 2018
Cite Icon Cite
Permissions Icon Permissions

Selecting a research question is the first step of any research project. This chapter discusses how to formulate a specific research question from a variety of scientific interests. The reader will learn that a good research question needs to consider several aspects, such as feasibility, innovation, and significance, and that merging all these aspects into one research question may be challenging. This chapter reviews the importance of generating a strong research question using the PICOT format: population (P), intervention (I), control (C), outcomes (O), and time (T). This chapter also discusses the selection of appropriate outcome variables—surrogates or clinical endpoints, based on the types of questions or study phase. The final goal of this chapter is to refine the researcher’s general idea into the process of shaping a strong research question that will be feasible, important, ethical, and answerable.

The difficulty in most scientific work lies in framing the questions rather than in finding the answers. —Arthur E. Boycott (Pathologist, 1877–1938)

The grand aim of all science is to cover the greatest number of empirical facts by logical deduction from the smallest number of hypotheses or axioms. —Albert Einstein (Physicist, 1879–1955)

Introduction

The previous chapter provided the reader with an overview of the history of clinical research, followed by an introduction to fundamental concepts of clinical research and clinical trials. It is important to be aware of and to learn lessons from the mistakes of past and current research in order to be prepared to conduct your own research. As you will soon learn, developing your research project is an evolutionary process, and research itself is a continuously changing and evolving field.

Careful conceptual design and planning are crucial for conducting a reproducible, compelling, and ethically responsible research study. In this chapter, we will discuss what should be the first step of any research project, that is, how to develop your own research question. The basic process is to select a topic of interest, identify a research problem within this area of interest, formulate a research question, and finally state the overall research objectives (i.e., the specific aims that define what you want to accomplish).

You will learn how to define your research question, starting from broad interests and then narrowing these down to your primary research question. We will address the key elements you will need to define for your research question: the study population, the intervention ( x , independent variable[s]), and the outcome(s) ( y , dependent variable[s]). Later chapters in this volume will discuss popular study designs and elements such as covariates, confounders, and effect modifiers (interaction) that will help you to further delineate your research question and your data analysis plan.

Although this chapter is not a grant-writing tutorial, most of what you will learn here has very important implications for writing a grant proposal. In fact, the most important part of a grant proposal is the “specific aims” page, where you state your research question, hypotheses, and objectives.

How to Select a Research Question

What is a research question.

A research question is an inquiry about an unanswered scientific problem. The purpose of your research project is to find the answer to this particular research question. Defining a research question can be the most difficult task during the design of your study. Nevertheless, it is fundamental to start with the research question, as it is strongly associated with the study design and predetermines all the subsequent steps in the planning and analysis of the research study.

What Is the Importance of a Research Question?

Defining the research question is instrumental for the success of your study. It determines the study population, outcome, intervention, and statistical analysis of the research study, and therefore the scope of the entire project.

A novice researcher will often jump to methodology and design, eager to collect data and analyze them. It is always tempting to try out a new or “fancy” method (e.g., “Let’s test this new proteomic biomarker in a pilot study!” or “With this Luminex assay we can test 20 cytokines simultaneously in our patient serum!”), but this mistake all too often makes the research project a “fishing expedition,” with the unfortunate outcome that a researcher has invested hours of work and has obtained reams of data, only to find herself at an impasse, and never figuring out what to do with all the information collected. Although it is not wrong to plan an exploratory study (or a hypothesis-generating study), such study has a high risk of not yielding any useful information; thus all the effort to have the study performed will be lost. When planning an exploratory or pilot study (with no defined research question), the investigator must understand the goals and the risks (for additional discussion on pilot studies, see Lancaster et al. 2002 ).

It is important to first establish a concept for your research. You must have a preset idea or a working hypothesis in order to be able to understand the data you will generate. Otherwise, you will not be able to differentiate whether your data were obtained by chance, by mistake, or if they actually reflect a true finding. Also, have in mind ahead of time how you would like to present your study at a conference, in a manuscript, or in a grant proposal. You should be able to present your research to your audience in a well-designed manner that reflects a logical approach and appropriate reasoning.

A good research question leads to useful findings that may have a significant impact on clinical practice and health care, regardless of whether the results are positive or negative. It also gives rise to the next generation of research questions. Therefore, taking enough time to develop the research question is essential.

Where Do Research Questions Come From?

How do we find research questions? As a clinical research scientist, your motivation to conduct a study might be driven by a perceived knowledge gap, the urge to deepen your understanding in a certain phenomenon, or perhaps to clarify contradictory existing findings. Maybe your bench research implies that your findings warrant translation into a study involving patients in a clinical setting. Maybe your clinical work experience gives you the impression that a new intervention would be more effective for your patients compared to standard treatment. For example, your results could lead you to ask, “Does this drug really prolong life in patients with breast cancer?” or, “Does this procedure really decrease pain in patients with chronic arthritis?”

Once you have identified a problem in the area you want to study, you can refine your idea into a research question by gaining a firm grasp of “what is known” and “what is unknown.” To better understand the research problem, you should learn as much as you can about the background pertaining to the topic of interest and specify the gap between current understanding and unsolved problems. As an early step, you should consult the literature, using tools such as MEDLINE or EMBASE, to gauge the current level of knowledge relevant to your potential research question. This is essential in order to avoid spending unnecessary time and effort on questions that have already been solved by other investigators. Meta-analyses and systematic reviews are especially useful to understand the combined level of evidence from a large number of studies and to obtain an overview of clinical trials associated with your questions. You should also pay attention to unpublished results and the progress of important studies whose results are not yet published. It is important to realize that there likely are negative results produced but never published. You can inspect funnel plots obtained from meta-analyses or generated from your own research (see Chapter 14 in this volume for more details) to estimate if there has been publication bias toward positive studies. Also, be aware that clinical trials with aims similar to those of your study might still be ongoing. To find this information, you can check the public registration of trials using sites such as clinicaltrials.gov.

How to Develop the Research Question: Narrow Down the Question

Once you have selected your research topic, you need to develop it into a more specific question. The first step in refining a research question is to narrow down a broad research topic into a specific description ( narrow research question ) that covers the four points of importance, feasibility, answerability , and ethicality.

Importance: Interesting, Novel, and Relevant

Your research can be descriptive, exploratory, or experimental. The purpose of your research can be for diagnostic or treatment purposes, or to discover or elucidate a certain mechanism. The point you will always have to consider when making a plan for your study, however, is how to justify your research proposal. Does your research question have scientific relevance? Can you answer the “so what” question? You need to describe the importance of your research question with careful consideration of the following elements:

The disease (condition or problem) : Novelty, unmet need, or urgency are important. What is the prevalence of this disease/condition? Is there a pressing need for further discoveries regarding this topic because of well-established negative prognoses (e.g., HIV, pancreatic cancer, or Alzheimer’s disease)? Are existing treatment options limited, too complex or costly, or otherwise not satisfactory (e.g., limb replacement, face transplantation)? Does the research topic reflect a major problem in terms of health policy, medical, social, and/or economic aspects (e.g., smoking, hypertension, or obesity)?

The intervention : Is it a new drug, procedure, technology, or medical device (e.g., stem-cell derived pacemaker or artificial heart)? Does it concern an existing drug approved by the Food and Drug Administration (FDA) for a different indication (e.g., is Rituximab, a drug normally indicated for malignant lymphoma, effective for systemic lupus erythematosus or rheumatoid arthritis)? Is there new evidence for application of an existing intervention in a different population (e.g., is Palivizumab also effective in immunodeficiency infants, not only in premature infants to prevent respiratory syncytial virus)? Have recent findings supported the testing of a new intervention in a particular condition (e.g., is a β-blocker effective in preventing cardiovascular events in patients with chronic renal failure)? Even a research question regarding a standard of care intervention can be valuable if in the end it can improve the effectiveness of clinical practice.

Feasibility

In short, be realistic: novel research tends to jump right away into very ambitious projects. You should carefully prove the feasibility of your research idea to prevent wasting precious resources such as time and money:

Patients : Can you recruit the required number of subjects? Do you think your recruitment goal is realistic? Rare diseases such as Pompe or Fabry’s disease will pose a challenge in obtaining a sufficient sample size. Even common diseases, depending on your inclusion criteria and regimen of intervention, may be difficult to recruit. Does your hospital have enough patients? If not, you may have to consider a multi-center study. What about protocol adherence and dropouts ? Do you expect significant deviations from the protocol? Do you need to adjust your sample size accordingly?

Technical expertise : Are there any established measurements or diagnostic tools for your study? Can the outcome be measured? Is there any established diagnostic tool? Do we have any standard techniques for using the device (e.g., guidelines for echocardiographic diagnosis for congenital heart disease)? Is there a defined optimal dose? Can you operate the device, or can the skill be learned appropriately (e.g., training manual for transcutaneous atrial valve replacement)? A pilot study or small preliminary study can be helpful at this stage to help answer these preliminary questions.

Time : Do you have the required time to recruit your patients? Is it possible to follow up with patients for the entire time of the proposed study period (e.g., can you follow preterm infant development at 3, 6, and 9 years of age)? When do you need to have your results in order to apply for your next grant?

Funding : Does you budget allow for the scope of your study? Are there any research grants you can apply for? Do the funding groups’ interests align with those of your study? How realistic are your chances of obtaining the required funding? If there are available funds, how do you apply for the grant?

Team : How about your research environment? Do your mentors and colleagues share your interests? What kind of specialists do you need to invite for your research? Do you have the staff to support your project (technicians, nurses, administrators, etc.)?

Answerability

New knowledge can only originate from questions that are answerable. A broad research problem is still a theoretical idea, and even if it is important and feasible, it still needs to be further specified. You should carefully investigate your research idea and consider the following:

Precisely define what is known or not known and identify what area your research will address. The research question should demonstrate an understanding of the biology, physiology , and epidemiology relevant to your research topic. For example, you may want to investigate the prevalence and incidence of stroke after catheterization and its prognosis before you begin research on the efficacy of a new anticoagulant for patients who received catheter procedures. Again, you may need to conduct a literature review in order to clarify what is already known. Conducting surveys ( interviews or questionnaires ) initially could also be useful to understand the current status of your issues (e.g., how many patients a year are diagnosed with stroke after catherization in your hospital? What kind of anticoagulant is already being used for the patients? How old are the patients? How about the duration of cauterization techniques? etc.).

The standard treatment should be well known before testing a new treatment. Are there any established treatments in your research field? Could your new treatment potentially replace the standard treatment or be complementary to the current treatment of choice? Guidelines can be helpful for discussion (e.g., American College of Cardiology/American Heart Association guidelines for anticoagulant therapy). Without knowing the current practice, your new treatment may never find its clinical relevance.

We also need information about clinical issues for diagnostic tests and interventions . Are you familiar with the diagnoses and treatment of this disease (e.g., computerized tomography or magnetic resonance imaging to rule out stroke after catherization)? Do you know the current guidelines ?

Ethical Aspects

Ethical issues should be discussed before conducting research. Is the subject of your research a controversial topic? The possible ethical issues will often depend largely on whether the study population is considered vulnerable (e.g., children, pregnant women, etc.; see Chapter 1 ) [ 1 ]. You must always determine the possible risks and benefits of your study intervention [ 1 ].

Finally, you may want to ask for expert opinions about whether your research question is answerable and relevant (no matter how strong your personal feelings may be about the relevance). To this end, a presentation of your idea or preliminary results at a study meeting early on in the project development can help refine your question.

How to Build the Research Question

The next step of formulating a narrow research question is to focus on the primary interest ( primary question ): What is the most critical question for your research problem? You will define this primary question by addressing the key elements using the useful acronym PICOT (population, intervention, control, outcome, and time), while keeping in mind the importance, feasibility, answerability, and ethicality. Although PICOT is a useful framework, it does not cover all types of studies, especially some observational studies, for instance those investigating predictors of response (E [exposure] instead of I [intervention] is used for observational studies). But for an experimental study (e.g., a clinical trial), the PICOT framework is extremely useful to guide formulation of the research question.

Building the Research Question: PICOT

P (population or patient).

What is the target population of your research? The target population is the population of interest from which you want to draw conclusions and inferences. Do you want to study mice or rabbits? Adults or children? Nurses or doctors? What are the characteristics of the study subjects, and what are the given problems that should be considered? You may want to consider the pathophysiology (acute or chronic?) and the severity of the disease (severe end stage or early stage?), as well as factors such as geographical background and socioeconomic status.

Once you decide on the target population, you may select a sample as the study population for your study. The study population is a subset of the target population under investigation. However, it is important to remember that the study population is not always a perfect representation of the target population, even when sampled at random. Thus, defining the study population by the inclusion and exclusion criteria is a critical step (see Chapter 3 ).

Since only in rare cases will you be able to study every patient of interest, you will have to identify and select whom from the target population you want to study. This is referred to as the study sample. To do this requires choosing a method of selection or recruitment (see Chapter 7 ).

A specific study sample defined by restricted criteria will have a reduced number of covariates and will be more homogeneous, therefore increasing the chance of higher internal validity for your study. This also typically allows for the study to be smaller and potentially less expensive. In contrast, a restricted population might make it more difficult to recruit a sufficient number of subjects. On the other hand, recruitment can be easier if you define a broad population, which also increases the generalizability of your study results. However, a broad population can make the study larger and more expensive [ 2 ].

I (Intervention)

The I of the acronym usually refers to “intervention.” However, a more general and therefore preferable term would be “independent variable.” The independent variable is the explanatory variable of primary interest, also declared as x in the statistical analysis. The independent variable can be an intervention (e.g., a drug or a specific drug dose), a prognostic factor, or a diagnostic test. I can also be the exposure in an observational study. In an experimental study, I is referred to as the fixed variable (controlled by the investigator), whereas in an observational study, I refers to an exposure that occurs outside of the experimenter’s control.

The independent variable precedes the outcome in time or in its causal path, and thus it “drives” the outcome in a cause-effect relationship.

C (Control)

What comparison or control is being considered? This is an important component when comparing the efficacy of two interventions. The new treatment should be superior to the placebo , when there is no standard treatment available. Placebo is a simulated treatment that has no pharmaceutical effects and is used to mask the recipients to potential expectation biases associated with participating in clinical trials. On the other hand, active controls could be used when an established treatment exists and the efficacy of the new intervention should be examined at least within the context of non-inferiority to the standard treatment. Also the control could be baseline in a one-group study.

O (Outcomes)

O is the dependent variable, or the outcome variable of primary interest; in the statistical analysis, it is also referred to as y . The outcome of interest is a random variable and can be a clinical (e.g., death) or a surrogate endpoint (e.g., hormone level, bone density, antibody titer). Selection of the primary outcome depends on several considerations: What can you measure in a timely and efficient manner? Which measurement will be relevant to understand the effectiveness of the new intervention? What is routinely accepted and established within the clinical community? We will discuss the outcome variable in more detail later in the chapter.

Time is sometimes added as another criterion and often refers to the follow-up time necessary to assess the outcome or the time necessary to recruit the study sample. Rather than viewing time as a separate aspect, it is usually best to consider time in context with the other PICOT criteria.

What Is the Primary Interest in your Research?

Once you have selected your study population, as well as the dependent and independent variables, you are ready to formulate your primary research question, the major specific aim, and a hypothesis. Even if you have several different ideas regarding your research problem, you still need to clearly define what the most important question of your research is. This is called your primary question . A research project may also contain additional secondary questions.

The primary question is the most relevant question of your research that should be driven by the hypothesis. Usually only one primary question should be defined at the beginning of the study, and it must be stated explicitly upfront [ 3 ]. This question is relevant for your sample size calculation (and in turn, for the power of your study—see Chapter 11 ).

The specific aim is a statement of what you are proposing to do in your research project.

The primary hypothesis states your anticipated results by describing how the independent variable will affect the dependent variable. Your hypothesis cannot be just speculation, but rather it must be grounded on the research you have performed and must have a reasonable chance of being proven true.

We can define more than one question for a study, but aside from the primary question, all others associated with your research are treated as secondary questions. Secondary questions may help to clarify the primary question and may add some information to the research study. What potential problems do we encounter with secondary questions? Usually, they are not sufficiently powered to be answered because the sample size is determined based on th e primary question. Also, type I errors (i.e., false positives) may occur due to multiple comparisons if not adjusted for by the proper statistical analysis. Therefore, findings from secondary questions should be considered exploratory and hypothesis generating in nature, with new confirmatory studies needed to further support the results.

An ancillary study is a sub-study built into the main study design. Previous evidence may convince you of the need to test a hypothesis within a sub-group ancillary to the main population of interest (e.g., females, smokers). While this kind of study enables you to perform a detailed analysis of the subpopulation, there are limitations on the generalizability of an ancillary study since the population is usually more restricted (see Further Readings, Examples of Ancillary Studies).

It is important to understand thoroughly the study variables when formulating the study question. Here we will discuss some of the important concepts regarding the variables, which will be discussed in more detail in Chapter 8 .

We have already learned that the dependent variable is the outcome, and the independent variable is the intervention. For study design purposes, it is important to also discuss how the outcome variables are measured. A good measurement requires reliability (precision), validity (“trueness”), and responsiveness to change. Reliability refers to how consistent the measurement is if it is repeated. Validity of a measurement refers to the degree to which it measures what it is supposed to measure. Responsiveness of a measurement means that it can detect differences that are proportional to the change of what is being measured with clinical meaningfulness and statistical significance.

Covariates are independent variables of secondary interest that may influence the relationship between the independent and dependent variables. Age, race, and gender are well-known examples. Since covariates can affect the study results, it is critical to control or adjust for them. Covariates can be controlled for by both planning (inclusion and exclusion criteria, placebo and blinding, sampling and randomization, etc.) and analytical methods (e.g., covariate adjustment [see Chapter 13 ], and propensity scores [see Chapter 17 ]).

Continuous (ratio and interval scale), discrete, ordinal, nominal (categorical, binary) variables: Continuous data represent all numbers (including fractions of numbers, floating point data) and are the common type of raw data. Discrete data are full numbers (i.e., integer data type; e.g., number of hospitalizations). Ordinal data are ordered categories (e.g., mild, moderate, severe). Nominal data can be either categorical (e.g., race) or dichotomous/binary (e.g., gender). Compared to other variables, continuous variables have more power, which is the ability of the study to detect an effect (e.g., differences between study groups) when it is truly present, but they don’t always reflect clinical meaningfulness and therefore make interpretation more difficult. Ordinal and nominal data may better reflect the clinical significance (e.g., dead or alive, relapse or no relapse, stage 1 = localized carcinoma, etc.). However, ordinal and categorical data typically have less power, and important information may be lost (e.g., if an IQ less than 70 is categorized as developmental delay in infants, IQs of 50, 58, and 69 will all fall into the same category, while an IQ of 70 or more is considered to be normal development, although the difference is just 1 point). This approach is called categorization of continuous data, where a certain clinically meaningful threshold is set to make it easier to quickly assess study results. It is important to note that some authors differentiate between continuous and discrete variables by defining the former as having a quantitative characteristic and the latter as having a qualitative characteristic. This is a somewhat problematic classification, especially when it comes to ordinal data.

Single and multiple variables : Having a single variable is simpler, as it is easier for clinical interpretation. Multiple valuables are efficient because we can evaluate many variables within a single trial, but these can be difficult to disentangle and interpret. Composite endpoints are combined multiple variables and are also sometimes used. Because each clinical outcome may separately require a long duration and a large sample size, combining many possible outcomes increases overall efficiency and enables one to reduce sample size requirements and to capture the overall impact of therapeutic interventions. Common examples include MACE (major adverse cardiac events) and TVF (target vessel failure: myocardial infarction in target vessel, target vessel reconstruction, cardiac death, etc.). Interpretation of the results has to proceed with caution, however (see section on case-specific questions) [ 9 ].

Surrogate variables (endpoints) and clinical variables (endpoints): Clinical variables directly assess the effect of therapeutic interventions on patient function and survival, which is the ultimate goal of a clinical trial. Clinical variables may include mortality, events (e.g., myocardial infarction, stroke), and occurrence of disease (e.g., HIV). A clinical endpoint is the most definitive outcome to assess the efficacy of an intervention. Thus, clinical endpoints are preferably used in clinical research. However, it is not always feasible to use clinical outcomes in trials. The evaluation of clinical outcomes presents some methodological problems since they require long-term follow-up (with problems of adherence, dropouts, competing risks, requiring larger sample sizes) and can make a trial more costly. At the same time, the clinical endpoint may be difficult to observe. For this reason, clinical scientists often use alternative outcomes to substitute for the clinical outcomes. So-called surrogate endpoints are a more practical measure to reflect the benefit of a new treatment. Surrogate endpoints (e.g., cholesterol levels, blood sugar, blood pressure, viral load) are defined based on the understanding of the mechanism of a disease that suggests a clear relationship between a marker and a clinical outcome [ 8 ]. Also, a biological rationale provided by epidemiological data, other clinical trials, or animal data should be previously demonstrated. A surrogate is frequently a continuous variable that can be measured early and repeatedly and therefore requires shorter follow-up time, smaller sample size, and reduced costs for conducting a trial. Surrogate endpoints are often used to accelerate the process of new drug development and early stages of development, such as in phase 2 [ 10 ]. As a word of caution, too much reliance on surrogate endpoints alone can be misleading if the results are not interpreted with regard to validation, measurability, and reproducibility (see Further Reading) [ 4 ].

How to Express a Research Question

Once a narrow research question is defined, you should clearly specify a hypothesis in the study protocol. A hypothesis is a statement about the expected results that predicts the effect of the independent on the dependent variable. A research hypothesis is essential to frame the experimental and statistical plan (statistics will be discussed in Unit II of this volume) and is also important to support the aim of the study in a scientific manuscript.

Types of Research Questions

To refine the research question and form the research hypothesis, we will discuss three types of research questions that investigate group differences, correlations, or descriptive measures. This classification is particularly important in discussing which statistical analysis is appropriate for your research question [ 5 ].

Basic/complex difference (group comparison) questions : Samples split into groups by levels associated with the independent variable are compared by considering whether there is a difference in the dependent variable. If you have only one independent variable, the question is classified as a basic difference question (e.g., drug A will reduce time to primary closure in a 5-mm punch biopsy vs. placebo) and you would rely on a t-test or one-way analysis of variance (ANOVA) for the analysis. If you have two or more independent variables (e.g., drugs A and B led to a 15-mg/dl reduction in LDL cholesterol versus placebo, but there was no reduction with only drug A), this then becomes a complex difference question and is analyzed by other statistical methods, such as a factorial ANOVA.

Basic/complex associational (relational/correlation) questions : The independent variable is correlated with the dependent variable. If there is only one dependent variable and one independent variable (e.g., is there a relationship between weight and natriuretic peptide levels?), it is called a basic associational question , and in this situation, a correlation analysis is used. If there is more than one independent variable associated with one dependent variable (e.g., smoking and drinking alcohol are associated with lung cancer), it is called a complex associational question , and multiple regression is used for statistical analysis.

Basic/complex descriptive question : The data are described and summarized using measures of central tendency (means, median, and mode), variability, and percentage (prevalence, frequency). If there is only one variable, it is called a basic descriptive question (e.g., how much MRSA isolates occur after the 15th day of hospitalization?); for more than one variable, a classification of basic/complex descriptive question is used.

Where Should You State Your Research Question?

Finally, where should you state your hypothesis? You may be writing for a research grant, research protocol, or manuscript. Usually, research questions should be stated in the introduction, immediately following the justification (“so what”) section. Research questions should be clearly stated in the form of a hypothesis, such as “We hypothesize that in this particular population (P), the new intervention (I) will improve the outcome (O) more than the standard of care (C).”

A Research Question Should Be Developed over Time

It is important that the investigator spend a good amount of time developing his or her study question. During this process, everything we discussed in this chapter needs to be reviewed and the research question then needs to be refined as this process takes place. A good planning, starting with the research question, is one of the key components for a study’s success.

Using Adverse Events as the Primary Research Question

Important questions concerning adverse effects can be answered in a clinical trial. However, as the typical clinical trial is performed in a controlled setting, the information regarding adverse effects is not always generalizable to the real-world setting. Thus, the clinical translation of the results needs careful consideration when carrying out a safety-focused study. The adverse reports from phase 4 (post-marketing marketing surveillance) are considered more generalizable information in drug development, although minimum safety data from phase 1 are required to proceed to subsequent study phases.

Also, it might not be easy to formulate a specific research question regarding adverse effects, as they might not be fully known in the early stage of drug development. This will also make it difficult to power the study properly (e.g., how many patients do we need to examine to show the statistically meaningful difference?).

When the Research Question Leads to Other Research Questions

Medical history is filled with interesting stories about research questions. And sometimes, it is not the intended hypothesis to be proven that yields a big discovery. For example, Sildenafil (Viagra) was initially developed by Pfizer for the treatment of cardiovascular conditions. Although clinical trials showed Sildenafil to have only little effect on the primary outcomes, it was quickly realized that an unexpected but marked “side effect” occurred in men. Careful investigation of clinical and pharmacological data generated the new research question, “Can Sildenafil improve erectile dysfunction?” This question was then answered in clinical trials with nearly 5,000 patients, which led to Sildenafil’s FDA approval in 1998 as the first oral treatment for male erectile dysfunction [ 7 ]. The investigator must be attentive to novel hypothesis that can be learned from a negative study.

Case Study: Finding the Research Question

Dr. L. Heart is a scientist working on cardiovascular diseases in a large, busy emergency room of a tertiary hospital specialized in acute coronarian syndromes. While searching PUBMED, she found an interesting article on a new drug—which animal studies have demonstrated to be a powerful anti-thrombotic agent—showing its safety in healthy volunteers. She then feels that it would be the right time to perform a phase II trial, testing this new drug in patients presenting myocardial infarction (MI). She sees this as her big career breakthrough. However, when Dr. Heart starts writing a study proposal for the internal review board (ethics committee), she asks herself, “What is my research question?” 1

Defining the research question is, perhaps, the most important part of the planning of a research study. That is because the wrong question will eventually lead to a poor study design and therefore all the results will be useless; on the other hand, choosing an elegant, simple question will probably lead to a good study that will be meaningful to the scientific community, even if the results are negative. In fact, the best research question is one that, regardless of the results (negative or positive), produces interesting findings. In addition, a study should be designed with only one main question in mind.

However, choosing the most appropriate question is not always easy, as such a question might not be feasible to be answered. For instance, when researching acute MI, the most important question would be whether or not a new drug decreases mortality. However, for economic and ethical reasons, such an approach can only be considered when previous studies have already suggested that the new drug is a potential candidate. Therefore, the investigator needs to deal with the important issue of feasibility versus clinical relevance. Dr. Heart soon realized that her task would not be an easy one, and also that this task may take some time; she kept thinking about one of the citations in an article she recently read: “One-third of a trial’s time between the germ of your idea and its publication in the New England Journal of Medicine should be spent fighting about the research question.” 2

“So What?” Test for the Research Question

Dr. Heart knows that an important test for the research question is to ask, “So what?” In other words, does the research question address an important issue? She knows, for example, that the main agency funding in the United States, the NIH (National Institutes of Health), considers significance and innovation as important factors to fund grant applications. Dr. Heart also remembers something that her mentor used to tell her at the beginning of her career: “A house built on a weak foundation will not stand.” She knows that even if she has the most refined design and uses the optimal statistical tests, her research will be of very little interest or utility if it does not advance the field. But regarding this point, she is confident that her research will have a significant impact in the field.

Next Step for the Research Question: How to Measure the Efficacy of the Intervention

Dr. L. Heart is in a privileged position. She works in a busy hospital that receives a significant amount of acute cardiovascular patients. She also has received huge departmental support for her research, meaning that she can run a wide range of blood exams to measure specific biological markers related to death in myocardial infarction. Finally, she has a PhD student who is a psychologist working with quality of life post-MI. Therefore, she asks herself whether she should rely on biological markers, on the assessment of quality of life, or if she should go to a more robust outcome to prove the efficacy of the new drug. She knows that this is one of the most critical decisions she has to make. It was a Friday afternoon. She had just packed up her laptop and the articles she was reading, knowing that she will have to make a decision by the end of the weekend.

Dr. Heart is facing a common problem: What outcome should be used in a research study? This needs to be defined for the research question. She knows that there are several options. For instance, the outcome might be mortality, new MI, days admitted to the emergency room, quality of life, specific effect of disease such as angina, a laboratory measure (cholesterol levels), or the cost of the intervention. Also, she might use continuous or categorical outcomes. For instance, if she is measuring angina, she might measure the number of days with angina (continuous outcome) or dichotomize the number of angina days in two categories (less than 100 days with angina vs. more or equal to 100 days with angina). She then lays out her options:

Use of clinical outcomes (such as mortality or new myocardial infarction) : She knows that by using this outcome, her results would be easily accepted by her colleagues; however, using these outcomes will increase the trial duration and costs.

Use of surrogates (for instance, laboratorial measurements) : One attractive alternative for her is to use some biomarkers or radiological exams (such as a catheterism). She knows a colleague in the infectious disease field who only uses CD4 for HIV trials as the main outcome. This would increase the trial feasibility. However, she is concerned that her biomarkers might not really represent disease progression.

Use of quality of life scales : This might be an intermediate solution for her. However, she is still concerned with the interpretation of the results if she decides to use quality of life scales.

More on the Response Variable: Categorical or Continuous?

Even before making the final decision, Dr. Heart needs to decide whether she will use a continuous or categorical variable. She wishes now that she knew the basic concepts of statistics. However, she calls a colleague, who explains to her the main issue of categorical versus continuous outcomes—in summary, the issue is the trade-off of power versus clinical significance.

A categorical outcome usually has two categories (e.g., a yes/no answer), while a continuous outcome can express any value. A categorical approach might be more robust than a continuous one, and it also has more clinical significance, but it also decreases the power of the study due to the use of less information. 3 She is now at the crossroad of feasibility versus clinical significance.

Choosing the Study Population

Now that Dr. Heart has gone through the difficult decision of finding the best outcome measure, she needs to define the target population—that is, in which patients is she going to test the new drug? Her first idea is to select only patients who have a high probability of dying—for instance, males who smoke, are older than 75 years, with insulin-dependent diabetes and hypercholesterolemia. “Then,” she thinks, “it will be easier to prove that the new drug is useful regardless of the population I study. But does that really sound like a good idea?”

The next step is to define the target population. Dr. Heart is inclined to restrict the study population, as she knows that this drug might be effective to a particular population of patients and therefore this increases her chances of getting a good result. In addition, she does remember from her statistical courses that this would imply a smaller variability and therefore she would gain power (power is an important currency in research, as it makes the study more efficient, decreasing costs and time to complete the study). On the other hand, she is concerned that she might put all her efforts in one basket—this is a risky approach, as this specific population might not respond, and she knows that broadening the population also has some advantages, for instance, the results would be more generalizable and it would be easier to recruit patients. But this would also increase the costs of the study.

But How about Other Ideas?

After a weekend of reflection, Dr. Heart called the staff for a team meeting and proudly explained the scenario and stated her initial thoughts. The staff was very eager to start a new study, and they made several suggestions: “We should also use echocardiography to assess the outcome!”; “Why don’t we perform a genotypic analysis on these patients?”; “We need to follow them until one year after discharge.” She started to become anxious again. What should she do with these additional suggestions? They all seem to be good ideas.

When designing a clinical trial, researchers expose a number of subjects to a new intervention. Therefore, they want to extract as much data as possible from studies. On the other hand, it might not be possible to ask all of the questions, since this will increase the study’s duration, costs, and personnel. Also, researchers should be aware that all the other outcomes assessed will be exploratory (i.e., their usefulness remains in suggesting possible associations and future studies) because studies are designed to answer a primary question only—and, as a principle of statistics, there is a 5% probability of observing a positive result just by chance (if you perform 20 tests, for instance, one of them will be positive just by chance!). But Dr. Heart knows that she can test additional hypotheses as secondary questions. She knows that there is another issue to go through: the issue of primary versus secondary questions.

Defining Her Hypothesis

After going through this long process, Dr. Heart is getting close to her research question. But now she needs to define the study hypotheses. In other words, what is her educated guess regarding the study outcome?

An important step when formulating a research question is to define the hypothesis of the study. This is important in terms of designing the analysis plan, as well as estimating the study sample size. Usually, researchers come up with study hypotheses after reviewing the literature and preliminary data. Dr. Heart can choose between a simple and a complex hypothesis. In the first case, her hypothesis would only have one dependent variable (i.e., the response variable) and one independent variable (e.g., the intervention). Complex hypotheses have more than one independent and/or dependent variable and might not be easy to use in planning the data analysis.

By the end of the day, Dr. Heart was overwhelmed with the first steps to put this study together. Although she is confident that this study might be her breakthrough and she needs to get her tenure track position at the institution where she works, she also knows she has only one chance and must be very careful at this stage. After wrestling with her thoughts, she finished her espresso and walked back to her office, confident that she knew what to do.

Case Discussion

Dr. Heart is a busy and ambitious clinical scientist and wants to establish herself within the academic ranks of her hospital. She has some background in statistics but seems to be quite inexperienced in conducting clinical research. She is looking for an idea to write up a research proposal and rightly conducts a literature research in her field of expertise, cardiovascular diseases. She finds an interesting article about a compound that has been demonstrated to be effective in an animal model and safe in healthy volunteers (results of a phase I trial). She now plans to conduct a phase II trial, but struggles to come up with a study design. The most vexing problem for her is formulating the research question.

Dr. Heart then reviews and debates aspects that have to be considered when delineating a research question. The main points she ponders include the following: determining the outcome with regard to feasibility (mainly concerning the time of follow-up when using a clinical outcome) versus clinical relevance (when using a surrogate outcome) and with regard to the data type to be used for the outcome (categorical vs. continuous); the importance of the research proposal (the need for a new anticoagulant drug); whether to use a narrow versus broad study population; whether to include only a primary or also secondary questions; and whether to use a basic versus complex hypothesis. Important aspects that Dr. Heart has not considered include the following: whether to test versus a control (although not mandatory in a phase II trial, it deserves consideration since she is investigating the effects of an anticoagulant and therefore adverse events should be expected, thus justifying the inclusion of a control arm) or to test several dosages (to observe a dose-response effect); logistics; the budget; and the overall scope of her project.

All these aspects are important and need careful consideration, but you have to wonder how this will help Dr. Heart come up with a compelling research question. Rather than assessing each aspect separately and making decisions based on advantages and disadvantages, it is recommended to start from a broad research interest and then develop and further specify the idea into a specific research question.

While Dr. Heart should be applauded for her ambition, she should also try to balance the level of risk of her research given her level of experience.

Finally, we should also question Dr. Heart’s motives for conducting this study. What is her agenda?

Case Questions for Reflection

What are the main challenges faced by Dr. Heart?

Should she be really concerned with the study question?

Which variable response should she choose? Justify your choice to your colleagues (remember, there is no right or wrong).

How should she select the study population?

Should her study have secondary questions?

Finally, try to create a study question for Dr. Heart in the PICOT format, depending on your previous selections.

Surrogate Outcomes

D’Agostino RB. Debate: The slippery slope of surrogate outcomes. Curr Contr Trials Cardiovasc Med . 2000 ; 1: 76–78. 10.1186/CVM-1-2-076

Echt DS , Liebson PR , Mitchell LB , et.al., Mortality and morbidity in patients receiving encainide, flecainide, or placebo: The Cardiac Arrhythmia Suppression Trial. N Eng J Med . 1991 ; 324: 781–788. 10.1056/NEJM199103213241201

Web of Science

Feng M , Balter JM , Normolle D , et al. Characterization of pancreatic tumor motion using Cine- MRI: surrogates for tumor position should be used with caution. Int J Radiat Oncol Biol Phys . 2009 July 1; 74(3): 884–891. 10.1016/j.ijrobp.2009.02.003

Katz R. Biomarkers and surrogate markers: an FDA perspective. NeuroRx . 2004 April; 1(2): 189–195. 10.1602/neurorx.1.2.189

Lonn E. The use of surrogate endpoints in clinical trials: focus on clinical trials in cardiovascular diseases. Pharmacoepidemiol Drug Safety . 2001 ; 10: 497–508. 10.1002/pds.654

Composite Endpoint

Cordoba G , Schwartz L , Woloshin S , et al. Definition, reporting, and interpretation of composite outcomes in clinical trials: systematic review. BMJ . 2010 ; 341: c3920. 10.1136/bmj.c3920

Kip KE , Hollabaugh K , Marroquin OC , et al. The problem with composite end points in cardiovascular studies. The story of major adverse cardiac events and percutaneous coronary intervention. JACC . 2008 ; 51(7): 701–707. 10.1016/j.jacc.2007.10.034

Examples of Ancillary Studies

Krishnan JA , Bender BG , Wamboldt FS , et al. Adherence to inhaled corticosteroids: an ancillary study of the Childhood Asthma Management Program clinical trial. J Allergy Clin Immunol. 2012 ; 129 (1): 112–118. 10.1016/j.jaci.2011.10.030

Udelson JE , Pearte CA , Kimmelstiel CD , et al. The Occluded Artery Trial (OAT) Viability Ancillary Study (OAT-NUC): influence of infarct zone viability on left ventricular remodeling after percutaneous coronary intervention versus optimal medical therapy alone. Am Heart J . 2011 Mar; 161(3): 611–621. 10.1016/j.ahj.2010.11.020

Controls, Sham/Placebo

Finnissa DG , Kaptchukb TJ , Millerc F et.al., Placebo effects: biological, clinical and ethical advances. Lancet . 2010 February 20; 375(9715): 686–695. 10.1016/S0140-6736(09)61706-2

Macklin R. The ethical problems with sham surgery in clinical research. N Engl J Med . 1999 Sep 23; 341(13): 992–996. 10.1056/NEJM199909233411312

Pilot Studies

Lancaster GA , Dodd S , Williamson PR. Design and analysis of pilot studies: recommendations for good practice. J Eval Clin Pract . 2002 ; 10(2): 307–312. 10.1111/j..2002.384.doc.x

1. The Belmont Report. Office of the Secretary. Ethical principles and guidelines for the protection of human subjects of research . The National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research. Washington, DC: U.S. Government Printing Office, 1979 .

2. Ferguson L. External validity, generalizability, and knowledge utilization. J of Nursing Scholarship . 2004 ; 36:1, 16–22. 10.1111/j.1547-5069.2004.04006.x

3. CONSORT statement, http://www.consort-statement.org/consort-statement/ ]

4. Echt DS , Liebson PR , Mitchell LB , et al. Mortality and morbidity in patients receiving encainide, flecainide, or placebo: The Cardiac Arrhythmia Suppression Trial. N Engl J Med . 1991 ; 324: 781–788. 10.1056/NEJM199103213241201

5. Morgan GA , Harmon RJ. Clinician’s guide to research methods and statistics: research question and hypotheses. J Am Acad Child Adolesc Psychiatry . 2000 ; 39(2): 261–263. 10.1097/00004583-200002000-00028

6. Macklin R. The ethical problems with sham surgery in clinical research. N Engl J Med . 1999 Sep 23; 341(13): 992–996. 10.1056/NEJM199909233411312

7. Campbell SF. Science, art and drug discovery: a personal perspective. Clin Sci (Lond) . 2000 Oct; 99(4): 255–260.

8. Lonn E. The use of surrogate endpoints in clinical trials: focus on clinical trial in cardiovascular disease. Pharmacoepidemiol Drug Safety . 2001 ; 10: 497–508. 10.1002/pds.654

9. Kip KE , Hollabaugh K , Marroquin OC , et al. The problem with composite end points in cardiovascular studies: the story of major adverse cardiac events and percutaneous coronary intervention. JACC . 2008 ; 51(7): 701–707 10.1016/j.jacc.2007.10.034

10. Katz R. Biomarkers and surrogate markers: an FDA perspective. NeuroRx . 2004 April; 1(2): 189–195. 10.1602/neurorx.1.2.189

Dr. André Brunoni and Professor Felipe Fregni prepared this case. Course cases are developed solely as the basis for class discussion. The situation in this case is fictional. Cases are not intended to serve as endorsements or sources of primary data. All rights reserved to the authors of this case.

Riva JJ , Malik KM , Burnie SJ , Endicott AR , Busse JW . What is your research question? An introduction to the PICOT format for clinicians. J Can Chiropr Assoc . 2012 Sep; 56(3):167–71 .

These concepts will be discussed in details in Unit II of this volume.

About Oxford Academic
Publish journals with us
University press partners
What we publish
New features
Open access
Institutional account management
Rights and permissions
Get help with access
Accessibility
Advertising
Media enquiries
Oxford University Press
Oxford Languages
University of Oxford

Oxford University Press is a department of the University of Oxford. It furthers the University's objective of excellence in research, scholarship, and education by publishing worldwide

Copyright © 2024 Oxford University Press
Cookie settings
Cookie policy
Privacy policy
Legal notice

This Feature Is Available To Subscribers Only

This PDF is available to Subscribers Only

For full access to this pdf, sign in to an existing account, or purchase an annual subscription.

Open access
Published: 24 May 2023

Embracing critical thinking to enhance our practice

Luis Martí-Bonmatí ORCID: orcid.org/0000-0002-8234-010X 1

Insights into Imaging volume 14 , Article number: 97 ( 2023 ) Cite this article

3520 Accesses

1 Citations

2 Altmetric

Metrics details

Miguel de Cervantes, the great Spanish writer, once wrote that those “who read much and walk much, go far and know much" [ 1 ]. The same is true in medicine; reading and gathering experience are the main pillars on which one should develop the knowledge of solving clinical problems in the ever-changing field of healthcare. If properly done, these newly acquired skills will continuously enhance our critical thinking strategies with which we try to identify the best possible improvements in the clinical pathway of radiology. As gaps in knowledge are always present, medicine is rooted in consolidated knowledge based on validated scientific studies and clinical experience reproducibility and accuracy [ 2 ]. This represents our best approach to evidence-based decisions. Medical knowledge must be well-established before it can be considered as the basis for decision making and patients guidance in daily practice.

The practice of critical thinking helps us understand the disease manifestations and the related processes and actions that might be relevant to prevent, diagnose and treat diseases. To critically appraise the way we perform evidence-based practice, we must combine best quality research with clinical expertise. This link between exploration and practice will allow radiologists and related disciplines to impact the way medicine is practiced.

These concepts are the cornerstones of Insights into Imaging , and it is my privilege as editor-in-chief to describe in this editorial how the journal, and each author, can contribute quality through critical thinking, and hence improve the way we practice radiology by re-shaping our understandings.

It is universally recognized that, in medical imaging, strong levels of evidence are needed to assess the results of the different possible actions and to guide decisions (i.e., to demonstrate a sufficient causal relationship between a specific diagnostic criterion and a disease grading, or a given radiological intervention versus another option in a given condition) toward the most effective or safe outcome considering the benefit of patients and value-based healthcare pathways. Consequently, solid levels of evidence are required to assess the results of different possible actions derived from imaging findings. And, in doing so, we continuously generate more data in our diagnostic and therapeutic activities, whether they are processes or outcomes. This new information will then be transformed into new evidence, real world evidence. In this way, the observed relationship between action and outcome generates causality course actions that will improve our understanding of the best clinical pathways, eliminating the many confounding thoughts that we unconsciously carry during the process of learning and implementing our clinical practice.

Socratic inquiry and Skepticism as foundation. Critical thinking can be understood as the process of analyzing and questioning existing and established knowledge with the intention of improving it. Previous knowledge, either eminence- or evidence-based, should continuously be critically reconsidered and reevaluated for the benefit of the patients, as knowledge is always changing in Precision Medicine. In the real world of medical imaging, this critical thinking must be focused on the evaluation of the effectiveness and clinical impact of all those processes in which images are involved, from the acquisition with different modalities to the processing of the data, from the biological correlation of radiomics as an image biomarker to the therapeutic orientation, and finally in image-guided interventional treatments. Developing critical thinking helps to improve any medical discipline by asking ourselves how to establish better and more precise processes based on existing accumulated evidence, how to recognize and control the biases when approaching a clinical problem, and how to adapt the new clinical information in service of the best solutions. Socratic inquiry and a skeptic attitude can be used to consolidate the best knowledge and construct new associations to be more efficient and to approach excellence in our daily work. Critical thinking is therefore necessary to improve both clinical practice and research in radiology, avoiding disruptive uncertainties and wrong assumptions.

These “questioning and solving” skills require learning, practice, and experience [ 3 ], but mainly a recognition of the many uncertainties we do have despite the important scientific advances. Precisely, a good example of the importance of critical thinking is its contribution to Precision Medicine through medical imaging data and information. In daily practice, we should ask ourselves why should we accept a reliable diagnostic method that fails 15% of the time, or an appropriate treatment that is not effective in almost 25% of patients? As scientists, we can improve these clinical decisions in the daily practice. Artificial intelligence (AI) solutions integrating different imaging, clinical, molecular, and genetic data as inputs are being implemented as a suitable pathway to solve clinical problems. The design and methodology of these AI algorithms must allow for their explainability and critical thinking evaluation before they are implemented in clinical practice [ 4 ].

In summary, critical thinking develops evidence-based knowledge, provides continuous improvements, and avoids spurious technical and clinical misconceptions. Insights into Imaging is dedicated to manuscripts with a clear critical approach, focusing on excellence in clinical practice, evidence-based knowledge and causal reasoning in radiology. Science is based on long-lived critiques and authors are encouraged to systematically identify, analyze, and solve problems by identifying inconsistencies and correcting errors.

To foster this, Insights into Imaging welcomes critical thinking papers and will incorporate a new “Critical Relevance Statement” in all their publications, where authors are asked to summarize in one sentence the question they are trying to answer and the improvement they are providing to the issue at hand.

Availability of data and materials

Not applicable.

De Cervantes M (1986) The adventures of don Quixote de la Mancha. New York, Farrar, Straus, Giroux

Martí-Bonmatí L (2021) Evidence levels in radiology: the insights into imaging approach. Insights Imaging 12(1):45. https://doi.org/10.1186/s13244-021-00995-7

Article PubMed PubMed Central Google Scholar

Ho YR, Chen BY, Li CM (2023) Thinking more wisely: using the Socratic method to develop critical thinking skills amongst healthcare students. BMC Med Educ 23(1):173. https://doi.org/10.1186/s12909-023-04134-2

Cerdá-Alberich L, Solana J, Mallol P et al (2023) MAIC-10 brief quality checklist for publications using artificial intelligence and medical images. Insights Imaging 14(1):11. https://doi.org/10.1186/s13244-022-01355-9

Download references

Acknowledgements

To the Insights into Imaging ’s Office for their help in preparing this editorial.

Author information

Authors and affiliations.

Medical Imaging Department and Biomedical Imaging Research Group, Hospital Universitario y Politécnico La Fe and Health Research Institute, Valencia, Spain

Luis Martí-Bonmatí

You can also search for this author in PubMed Google Scholar

Contributions

LBM is the only author.The author have read and approved the final version of the manuscript.

Corresponding author

Correspondence to Luis Martí-Bonmatí .

Ethics declarations

Ethics approval and consent to participate, consent for publication, competing interests.

LMB is the Editor in Chief of Insights into Imaging . He has not taken part in the review or selection process of this article.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Martí-Bonmatí, L. Embracing critical thinking to enhance our practice. Insights Imaging 14 , 97 (2023). https://doi.org/10.1186/s13244-023-01435-4

Download citation

Received : 05 April 2023

Accepted : 24 April 2023

Published : 24 May 2023

DOI : https://doi.org/10.1186/s13244-023-01435-4

Teaching Strategies for Developing Clinical Reasoning Skills in Nursing Students: A Systematic Review of Randomised Controlled Trials

Associated data.

Data are contained within the article.

Background: Clinical reasoning (CR) is a holistic and recursive cognitive process. It allows nursing students to accurately perceive patients’ situations and choose the best course of action among the available alternatives. This study aimed to identify the randomised controlled trials studies in the literature that concern clinical reasoning in the context of nursing students. Methods: A comprehensive search of PubMed, Scopus, Embase, and the Cochrane Controlled Register of Trials (CENTRAL) was performed to identify relevant studies published up to October 2023. The following inclusion criteria were examined: (a) clinical reasoning, clinical judgment, and critical thinking in nursing students as a primary study aim; (b) articles published for the last eleven years; (c) research conducted between January 2012 and September 2023; (d) articles published only in English and Spanish; and (e) Randomised Clinical Trials. The Critical Appraisal Skills Programme tool was utilised to appraise all included studies. Results: Fifteen papers were analysed. Based on the teaching strategies used in the articles, two groups have been identified: simulation methods and learning programs. The studies focus on comparing different teaching methodologies. Conclusions: This systematic review has detected different approaches to help nursing students improve their reasoning and decision-making skills. The use of mobile apps, digital simulations, and learning games has a positive impact on the clinical reasoning abilities of nursing students and their motivation. Incorporating new technologies into problem-solving-based learning and decision-making can also enhance nursing students’ reasoning skills. Nursing schools should evaluate their current methods and consider integrating or modifying new technologies and methodologies that can help enhance students’ learning and improve their clinical reasoning and cognitive skills.

1. Introduction

Clinical reasoning (CR) is a holistic cognitive process. It allows nursing students to accurately perceive patients’ situations and choose the best course of action among the available alternatives. This process is consistent, dynamic, and flexible, and it helps nursing students gain awareness and put their learning into perspective [ 1 ]. CR is an essential competence for nurses’ professional practice. It is considered crucial that its development begin during basic training [ 2 ]. Analysing clinical data, determining priorities, developing plans, and interpreting results are primary skills in clinical reasoning during clinical nursing practise [ 3 ]. To develop these skills, nursing students must participate in caring for patients and working in teams during clinical experiences. Among clinical reasoning skills, we can identify communication skills as necessary for connecting with patients, conducting health interviews, engaging in shared decision-making, eliciting patients’ concerns and expectations, discussing clinical cases with colleagues and supervisors, and explaining one’s reasoning to others [ 4 ].

Educating students in nursing practise to ensure high-quality learning and safe clinical practise is a constant challenge [ 5 ]. Facilitating the development of reasoning is challenging for educators due to its complexity and multifaceted nature [ 6 ], but it is necessary because clinical reasoning must be embedded throughout the nursing curriculum [ 7 ]. Such being the case, the development of clinical reasoning is encouraged, aiming to promote better performance in indispensable skills, decision-making, quality, and safety when assisting patients [ 8 ].

Nursing education is targeted at recognising clinical signs and symptoms, accurately assessing the patient, appropriately intervening, and evaluating the effectiveness of interventions. All these clinical processes require clinical reasoning, and it takes time to develop [ 9 ]. This is a significant goal of nursing education [ 10 ] in contemporary teaching and learning approaches [ 6 ].

Strategies to mitigate errors, promote knowledge acquisition, and develop clinical reasoning should be adopted in the training of health professionals. According to the literature, different methods and teaching strategies can be applied during nursing training, as well as traditional teaching through lectures. However, the literature explains that this type of methodology cannot enhance students’ clinical reasoning alone. Therefore, nursing educators are tasked with looking for other methodologies that improve students’ clinical reasoning [ 11 ], such as clinical simulation. Clinical simulation offers a secure and controlled setting to encounter and contemplate clinical scenarios, establish relationships, gather information, and exercise autonomy in decision-making and problem-solving [ 12 ]. Different teaching strategies have been developed in clinical simulation, like games or case studies. Research indicates a positive correlation between the use of simulation to improve learning outcomes and how it positively influences the development of students’ clinical reasoning skills [ 13 ].

The students of the 21st century utilise information and communication technologies. With their technological skills, organisations can enhance their productivity and achieve their goals more efficiently. Serious games are simulations that use technology to provide nursing students with a safe and realistic environment to practise clinical reasoning and decision-making skills [ 14 ] and can foster the development of clinical reasoning through an engaging and motivating experience [ 15 ].

New graduate nurses must possess the reasoning skills required to handle complex patient situations. Aware that there are different teaching methodologies, with this systematic review we intend to discover which RCTs published focus on CR in nursing students, which interventions have been developed, and their effectiveness, both at the level of knowledge and in increasing clinical reasoning skills. By identifying the different techniques used during the interventions with nursing students in recent years and their effectiveness, it will help universities decide which type of methodology to implement to improve the reasoning skills of nursing students and, therefore, obtain better healthcare results.

This study aims to identify and analyse randomised controlled trials concerning clinical reasoning in nursing students. The following questions guide this literature review:

Which randomised controlled trials have been conducted in the last eleven years regarding nursing students’ clinical reasoning? What are the purposes of the identified RCTs? Which teaching methodologies or strategies were used in the RCTs studies? What were the outcomes of the teaching strategies used in the RCTs?

2. Materials and Methods

This review follows the PRISMA 2020 model statement for systematic reviews. That comprises three documents: the 27-item checklist, the PRISMA 2020 abstract checklist, and the revised flow diagram [ 16 ].

2.1. Search Strategy

A systematic literature review was conducted on PubMed, Scopus, Embase, and the Cochrane Controlled Register of Trials (CENTRAL) up to 15th October 2023.

The PICOS methodology guided the bibliographic search [ 17 ]: “P” being the population (nursing students), “I” the intervention (clinical reasoning), “C” comparison (traditional teaching), “O” outcome (dimension, context, and attributes of clinical reasoning in the students’ competences and the results of the teaching method on nursing students), and “S” study type (RCTs).

The search strategy used in each database was the following: (“nursing students” OR “nursing students” OR “pupil nurses” OR “undergraduate nursing”) AND (“clinical reasoning” OR “critical thinking” OR “clinical judgment”). The filters applied were full text, randomised controlled trial, English, Spanish, and from 1 January 2012 to 15 October 2023. The search strategy was performed using the same process for each database. APP performed the search, and AZ supervised the process.

During the search, the terms clinical reasoning, critical thinking, and clinical judgement were used interchangeably since clinical judgement is part of clinical reasoning and is defined by the decision to act. It is influenced by an individual’s previous experiences and clinical reasoning skills [ 18 ]. Critical thinking and clinical judgement involve reflective and logical thinking skills and play a vital role in the decision-making and problem-solving processes [ 19 ].

The first search was conducted between March and September 2022, and an additional search was conducted during October 2023, adding the new articles published between September 2022 and September 2023, following the same strategy. The search strategy was developed using words from article titles, abstracts, and index terms. Parallel to this process, the PRISMA protocol was used to systematise the collection of all the information presented in each selected article. This systematic review protocol was registered in the international register PROSPERO: CRD42022372240.

2.2. Eligibility Criteria and Study Selection

The following inclusion criteria were examined: (a) clinical reasoning, clinical judgment, and critical thinking in nursing students as a primary aim; (b) articles published in the last eleven years; (c) research conducted between January 2012 and September 2023; (d) articles published only in English and Spanish; and (e) RCTs. On the other hand, the exclusion criteria were studies conducted with students from other disciplines other than nursing, not random studies or review articles.

2.3. Data Collection and Extraction

After this study selection, the following information was extracted from each article: bibliographic information, study aims, teaching methodology, sample size and characteristics, time of intervention, and conclusions.

2.4. Risk of Bias

The two reviewers, APP and AZ, worked independently to minimise bias and mistakes. The titles and abstracts of all papers were screened for inclusion. All potential articles underwent a two-stage screening process based on the inclusion criteria. All citations were screened based on title, abstract, and text. Reviewers discussed the results to resolve minor discrepancies. All uncertain citations were included for full-text review. The full text of each included citation was obtained. Each study was read thoroughly and assessed for inclusion following the inclusion and exclusion criteria explained in the methodology. The CASP tool was utilised to appraise all included studies. The CASP Randomized Controlled Trial Standard Checklist is an 11-question checklist [ 20 ], and the components assessed included the appropriateness of the objective and aims, methodology, study design, sampling method, data collection, reflexivity of the researchers, ethical considerations, data analysis, rigour of findings, and significance of this research. These items of the studies were then rated (“Yes” = with three points; “Cannot tell” = with two points; “No” = with one point). The possible rates for every article were between 0 and 39 points.

2.5. Ethical Considerations

Since this study was a comprehensive, systematic review of the existing published literature, there was no need for us to seek ethical approval.

3.1. Search Results

The initial search identified 158 articles using the above-mentioned strategy (SCOPUS ® n = 72, PUBMED ® n = 56, CENTRAL ® n = 23, and EMBASE ® n= 7), and the results are presented in Figure 1 . After retrieving the articles and excluding 111, 47 were selected for a full reading. Finally, 17 articles were selected. To comply with the methodology, the independent reviewers analysed all the selected articles one more time after the additional search, and they agreed to eliminate two of them because this study sample included nursing students as well as professional nurses. Therefore, to have a clear outcome focused on nursing students, two articles were removed, and the very final sample size was fifteen articles, following the established selection criteria ( Figure 1 ). The reasons for excluding studies from the systematic review were: nurses as targets; other design types of studies different from RCTs; focusing on other health professionals such as medical students; review studies; and being published in full text in other languages other than Spanish or English.

An external file that holds a picture, illustration, etc.
Object name is healthcare-12-00090-g001.jpg

Flowchart of screening of clinical reasoning RCTs that underwent review.

3.2. Risk of Bias in CASP Results

All studies included in the review were screened with the CASP tool. Each study was scored out of a maximum of 39 points, showing the high quality of the randomised control trial methodology. The studies included had an average score of 33.1, ranging from 30 to 36 points. In addition, this quantitative rate of the items based on CASP, there were 13 studies that missed an item in relation to assessing/analysing outcome/s ‘blinded or not’ or not, and 11 studies that missed the item whether the benefits of the experimental intervention outweigh the harms and costs.

3.3. Data Extraction

Once the articles had undergone a full reading and the inclusion criteria were applied, data extraction was performed with a data extraction table ( Appendix A ). Their contents were summarised into six different cells: (1) CASP total points result, (2) purpose of this study, (3) teaching strategy, (4) time of intervention, (5) sample size, and (6) author and year of publication. After the review by the article’s readers, fifteen RCTs were selected. Of the fifteen, the continent with the highest number of studies was Asia, with 53.33% of the studies (n = 8) (Korea n = 4, Taiwan n = 2, and China n = 2), followed by Europe with 26.66% (n = 4) (Turkey n = 2, Paris n = 1, and Norway n = 1), and lastly South America with 20% (n = 3), all of them from Brazil.

3.4. Teaching Strategies

Different teaching strategies have been identified in the reviewed studies: simulation methods (seven articles) and learning programmes (eight articles). There are also two studies that focus on comparing different teaching methodologies.

3.4.1. Clinical Simulation

The simulation methods focused on in the studies were virtual simulation (based on mobile applications), simulation games, and high-fidelity clinical simulation. Of the total number of nursing students in the studies referring to clinical simulations, 43.85% were in their second year, while 57.1% were senior-year students. The most used method in the clinical simulation group was virtual simulation, and 57.14% of studies included only one-day teaching interventions.

Virtual simulations were used to increase knowledge about medication administration and nasotracheal suctioning in different scenarios [ 21 ], to evaluate the effect of interactive nursing skills, knowledge, and self-efficacy [ 11 ], and to detect patient deterioration in two different cases [ 22 ]. Simulation game methodology was used to improve nursing students’ cognitive and attention skills, strengthen judgment, time management, and decision-making [ 14 ].

Clinical simulation was used to develop nursing students’ clinical reasoning in evaluating wounds and their treatments [ 12 ], to evaluate and compare the perception of stressors, with the goal of determining whether simulations promote students’ self-evaluation and critical-thinking skills [ 23 ], and also to evaluate the impact of multiple simulations on students’ self-reported clinical decision-making skills and self-confidence [ 24 ].

3.4.2. Learning Programs

Different types of learning programmes have been identified in this systematic review: team-based learning, reflective training programs, person-centred educational programmes, ethical reasoning programmes, case-based learning, mapping, training problem-solving skills, and self-instructional guides. Of the total number of nursing students in the studies referring to learning programs, 57.1% were junior-year students, while 43.85% were in their senior year.

Team-based learning is a learner-centred educational strategy that promotes active learning to improve students’ problem-solving, knowledge, and practise performance. It can be implemented in small or large groups divided into teams with an instructor and reading material based on case scenarios [ 25 ]. Reflective training is based on a new mentoring practise to explore, think about, and solve problems actively during an internship. During the reflective training program, the mentors lead students to uncover clinical nursing problems through conversations with them and discussing feedback for their professional portfolios [ 26 ]. The person-centred educational programme focuses on how nursing students perceive individualised care, using design thinking to improve their perception. The use of design thinking gave the students opportunities to apply their theoretical knowledge of the person-centred program to plan innovative solutions that may effectively resolve real-life situations [ 27 ]. Another educational programme identified is the ethical reasoning program, and the aim of this is to improve nursing students’ handling of ethical decision-making situations [ 28 ], engaging the students in complex ethical clinical situations based on real cases.

Case-based learning was used to explore and demonstrate the feasibility of implementing unfolding cases in lectures to develop students’ critical-thinking abilities [ 29 ]. The web-based concept mapping of nursing students was also investigated to determine its impact on critical-thinking skills [ 30 ]. Training problem-solving skills were used to find out how it affected the rate of self-handicapping among nursing students [ 31 ]. And the last article evaluated the effect of the self-instructional guide to improve clinical reasoning skills on diagnostic accuracy in undergraduate nursing students [ 32 ].

4. Discussion

Although 158 studies were initially identified, only 15 articles were finally included in this review. The excluded articles were mainly from other disciplines other than nursing and used a less rigorous study design than RCT.

The three longest interventions were developed in Asia [ 26 , 28 , 29 ]. The longest was 300 h in duration, through one year [ 30 ]. These interventions were based on learning programs, case-based learning, person-centred care (PCC), and reflective training programs. However, it is important to take into account that Asian nursing curriculum programmes are different from European or United States curriculum because their internship is carried out only during the last academic degree year, while in Europe, following the European directive 2005/36/CE, 2013/55/UE nursing education requirements of 4600 h (2300 h of clinical practice) is carried out along the 3–4 years of the academic degree [ 33 ]. On the other hand, the intervention with the biggest sample was 419 nursing students [ 30 ], 210 in the experimental group, and 209 in the control group, and the one with the lowest sample was 51, with 24 students in the control group and 27 in the intervention group [ 32 ]. Therefore, all the included studies had a good sample size.

This systematic review has detected different methodologies to help nursing students improve their reasoning and decision-making skills. Virtual simulation was the most frequently used teaching method, both as a mobile application and as a serious game. In terms of its effectiveness in a study carried out in Taiwan, the use of a mobile application resulted in significantly higher knowledge scores, better skill performance, and higher satisfaction in students than traditional paper materials [ 21 ]. Virtual simulation [ 11 , 14 , 21 ] has also proven to be an effective tool for enhancing knowledge and confidence in recognising and responding to rapidly deteriorating patients, but studies that combined two educational strategies were more effective [ 29 ], like clinical simulation combined with another teaching strategy such as lectures or videos [ 12 ].

An interactive learner-centred nursing education mobile application with systematic contents effectively allowed students to experience positive practical nursing skills [ 11 ]. However, in a study comparing serious game simulation versus traditional teaching methods, no significant difference was found immediately or in the month following the training [ 22 ], but serious games can improve nursing students’ cognitive skills to detect patient deterioration and to make safe decisions about patient care [ 14 ]. Although the innovative teaching method was well received by the students, who expressed higher levels of satisfaction and motivation [ 22 ]. We can affirm that the development of a mobile application and its application can be effectively used by nursing students at all levels [ 11 ]. However, the performance of all these studies was measured on its short-term outcomes, only 40 min [ 21 ], 2 h [ 22 ], and 1 week [ 11 , 14 ] of intervention, and was performed with a mean sample size of 97 nursing students.

The data obtained in a study developed in Brazil [ 12 ] confirm that clinical simulation is effective for the development of nursing students’ clinical reasoning in wound evaluation and treatment and that clinical simulation in conjunction with other educational methods promotes the acquisition of knowledge by facilitating the transition from what the student knows to rational action. Moreover, the high-fidelity simulation strategy increases the perception of stressors related to a lack of competence and interpersonal relationships with patients, multidisciplinary teams, and colleagues compared with the conventional practice class in the skill laboratory. This increase was related to the students’ capacity for self-evaluation and critical reflection, concerning their learning responsibility and the need to acquire the required skills for patient care [ 23 ]. However, in the case of the effect of multiple simulations on students, there are no differences found between the double-versus single-scenario simulations [ 24 ]. The intervention time in these three studies was 30 min [ 23 ], 3.5 h [ 12 ], and 4 days [ 24 ]; then the time used to implement the intervention can determine the results obtained.

The different learning methods have an impact on various learning outcomes and students’ variables. Team-based learning [ 25 ], reflective training [ 26 ], the person-centred education programme [ 27 ], web-based concept mapping [ 30 ], and teaching cognitive-behavioural approaches [ 31 ] have proven to be effective in enhancing problem-solving abilities, knowledge, and reasoning processes and consequently improving the quality of nursing practical education. Team-based learning increased problem-solving ability scores significantly, while those in the control group decreased [ 25 ]. Reflective training, developed in China based on the new mentoring approach, was effective in encouraging nursing students to explore, think about, and solve problems actively during an internship, consequently improving their disposition for critical thinking [ 26 ]. A person-centred education programme using design thinking can effectively improve how nursing students perceive individualised care. Using design thinking allowed the students to apply their theoretical knowledge of the programme to plan innovative solutions that may effectively resolve real health problems [ 27 ]. These programmes were developed in 5 or 6 days [ 27 , 31 ], 1 week or 3 weeks [ 25 , 30 ], and 1 year [ 26 ].

The education programme focused on improving ethical decision-making had statistically significant improvements in nursing students’ self-efficacy in communication confidence, complex ethical decision-making skills, and decreased communication difficulty [ 28 ]. Case-based learning was more effective with lectures than without them in developing students’ critical thinking abilities [ 29 ]. This study was one of the longest developed with 300 h during one school year. This long-term learning intervention could have a positive impact on this study sample. Therefore, the time of the learning intervention could be a limitation in the studied RCTs. The one-time self-instruction guide was ineffective in impacting students’ diagnostic accuracy in solving case studies [ 32 ], and it is possible that only one day of intervention is not enough.

Studies have shown that problem- and team-based learning [ 25 , 31 ] are more beneficial than traditional teaching [ 29 ], as they enhance nursing skills and improve problem-solving abilities, clinical performance, communication competencies, critical thinking, and self-leadership.

Researchers generally agree that clinical reasoning is an important ability and one of the most important competencies for good nursing practise to ensure optimal patient outcomes [ 29 ] and to recognise and address patient deterioration effectively. However, effective communication is crucial in clinical reasoning. It is required to establish a rapport with patients, conduct health evaluations, make collaborative decisions, and discuss clinical cases with colleagues and supervisors. Developing clinical reasoning skills during training is essential to improving nursing professionals’ practice. To enhance clinical reasoning abilities, nursing schools should integrate simulations at every level of education to ultimately improve patient care. Improving nursing students’ preparation will impact the quality of patient care. In addition, new innovative teaching methodologies based on the use of technology could be a motivational driver in nursing clinical reasoning [ 22 ].

5. Limitations

This systematic review did not perform a search on CINAHL. Although most of the journals included in this database are included in MEDLINE, this should be addressed in the future because of the relevance of the database to nursing research. The results of the included studies could have also been influenced by the different times of the interventions and the different contexts. In addition, the reviewers have identified other studies published in languages other than those required by the inclusion criteria. It seems that many articles are published by Asian researchers, but some of them are not in English, so they cannot be analysed.

6. Conclusions

As society progresses, the new generation of nursing students poses a challenge; new technologies are ingrained in their daily lives with access to increasingly advanced technologies like artificial intelligence, and we must adapt training to capture their interest and increase their learning skills. The utilisation of mobile apps, digital simulations, and learning games has a positive impact on the clinical reasoning abilities of nursing students and their motivation. Incorporating new technologies into problem-solving-based learning and decision-making can also enhance nursing students’ reasoning skills. As a result, it is crucial to incorporate these tools into the learning process to maintain students’ interest, motivation, and satisfaction in education. Clinical simulation is particularly important in the training of students in terms of clinical performance. Still, it is necessary to add another teaching method to increase the efficacy of clinical simulations. Therefore, nursing schools should evaluate their current teaching methods and consider integrating or modifying new technologies and methodologies that can help enhance students’ learning, improve their clinical reasoning and cognitive skills, and potentially improve nursing students’ ability to affect patient care positively. By doing so, students will be better equipped to provide high-quality patient care in the future.

Funding Statement

This research received external funding from the European programme Eramus +2021-1-BE02-KA220-HED-000023194.

Author Contributions

Conceptualisation, A.P.-P. and A.Z.; methodology, A.P.-P. and A.Z.; formal analysis, A.P.-P.; writing—original draft preparation, A.P.-P.; writing—review and editing, A.Z.; visualisation, A.Z.; supervision, A.Z. All authors have read and agreed to the published version of the manuscript.

Institutional Review Board Statement