Researchmate.net logo

9 Best Examples of Research Instruments in Qualitative Research Explained

Introduction.

Qualitative research is a valuable approach that allows researchers to explore complex phenomena and gain in-depth insights into the experiences and perspectives of individuals. In order to conduct qualitative research effectively, researchers often utilize various research methodologies and instruments. These methodologies and instruments serve as tools to collect and analyze data, enabling researchers to uncover rich and nuanced information.

Qualitative research instruments are tools used to gather non-numerical data, providing researchers with detailed insights into participants' experiences, emotions, and social contexts.

In this article, we will delve into the world of qualitative research instruments, specifically focusing on research instrument examples. We will explore the different types of qualitative research instruments, provide specific examples, and discuss the advantages and limitations of using these instruments in qualitative research. By the end of this article, you will have a comprehensive understanding of the role and significance of research instruments in qualitative research.

Goals of Research Instruments in Qualitative Research

Qualitative research instruments are tools that researchers use to collect and analyze data in qualitative research studies. These instruments help researchers gather rich and detailed information about a particular phenomenon or topic.

One of the main goals of qualitative research is to understand the subjective experiences and perspectives of individuals. To achieve this, researchers need to use instruments that allow for in-depth exploration and interpretation of data. Qualitative research instruments can take various forms, including interviews, questionnaires, observations, and focus groups. Each instrument has its own strengths and limitations, and researchers need to carefully select the most appropriate instrument for their study objectives.

Exploring qualitative research instruments involves understanding the characteristics and features of each instrument, as well as considering the research context and the specific research questions being addressed. Researchers also need to consider the ethical implications of using qualitative research instruments, such as ensuring informed consent and maintaining confidentiality and anonymity of participants.

Examples of Qualitative Research Instruments

Qualitative research instruments are tools used to collect data and gather information in qualitative research studies. These instruments help researchers explore and understand complex social phenomena in depth. There are several types of qualitative research instruments that can be used depending on the research objectives and the nature of the study.

Interviews are a fundamental qualitative research instrument that allows researchers to gather in-depth and personalized information directly from participants through structured, semi-structured, or unstructured formats.

Interviews are one of the most commonly used qualitative research instruments. They involve direct communication between the researcher and the participant, allowing for in-depth exploration of the participant’s experiences, perspectives, and opinions. Interviews can be structured, semi-structured, or unstructured , depending on the level of flexibility in the questioning process. They involve researchers asking open-ended questions to participants to gather in-depth information and insights. Interviews can be conducted face-to-face, over the phone, or through video conferencing.

Focus Groups

Focus groups are a qualitative research instrument that involves guided group discussions, enabling researchers to collect diverse perspectives and explore group dynamics on a particular topic.

Focus groups are another example of qualitative research instrument that involves a group discussion led by a researcher or moderator. Participants in a focus group share their thoughts, ideas, and experiences on a specific topic. This instrument allows for the exploration of group dynamics and the interaction between participants. It also allow researchers to gather multiple perspectives and generate rich qualitative data.

Observations

Observations are a powerful qualitative research instrument that involves systematic and careful observation of participants in their natural settings. This type of qualitative research instrument allows researchers to gather data on behavior, interactions, and social processes. Observations can be participant observations, where the researcher actively participates in the setting, or non-participant observations, where the researcher remains an observer.

Document Analysis

Document analysis is a qualitative research instrument that involves the examination, analyzation and interpretation of written or recorded materials such as documents, texts, audio/video recordings or other written materials. Researchers analyze documents to gain insights into social, cultural, or historical contexts, as well as to understand the perspectives and meanings embedded in the documents.

Visual Methods

Visual methods, such as photography, video recording, or drawings, can be used as qualitative research instruments. These methods allow participants to express their experiences and perspectives visually, providing rich and nuanced data. Visual methods can be particularly useful in studying topics related to art, culture, or visual communication.

Diaries or Journals

Diaries or journals are qualitative research instruments that allow participants to record their thoughts, experiences, and reflections over time, providing researchers with rich, longitudinal data.

Diaries or journals can be used as qualitative research instruments to collect data on participants’ thoughts, feelings, and experiences over a period of time. Participants record their daily activities, reflections, and emotions, providing valuable insights into their lived experiences.

While surveys are commonly associated with quantitative research, they can also be used as qualitative research instruments. Qualitative surveys typically include open-ended questions that allow participants to provide detailed responses. Surveys can be administered online, through interviews, or in written form.

Case Studies

Case studies are in-depth investigations of a particular individual, group, or phenomenon. They involve collecting and analyzing qualitative data from various sources such as interviews, observations, and document analysis. Case studies provide rich and detailed insights into specific contexts or situations.

Ethnography

Ethnography is a qualitative research instrument that involves immersing researchers in a particular social or cultural group to observe and understand their behaviors, beliefs, and practices. Ethnographic research often includes participant observation, interviews, and document analysis.

These are just a few examples of qualitative research instruments. Researchers can choose the most appropriate data collection method or combination of methods based on their research objectives, the nature of the research question, and the available resources.

Advantages of Using Qualitative Research Instruments

Gathering in-depth and detailed information.

Qualitative research instruments offer several advantages that make them valuable tools in the research process. Firstly, qualitative research instruments allow researchers to gather in-depth and detailed information. Unlike quantitative research instruments that focus on numerical data, qualitative instruments provide rich and descriptive data about participants’ feelings, opinions, and experiences. This depth of information allows researchers to gain a comprehensive understanding of the research topic .

Flexibility and Adaptability in Qualitative Research

Another advantage of qualitative research instruments is their flexibility. Researchers can adapt their methods and questions during data collection to respond to emerging insights. This flexibility allows for a more dynamic and responsive research process, enabling researchers to explore new avenues and uncover unexpected findings.

Capturing Data in Natural Settings

Qualitative research instruments also offer the advantage of capturing data in natural settings. Unlike controlled laboratory settings often used in quantitative research, qualitative research takes place in real-world contexts. This natural setting allows researchers to observe participants’ behaviors and interactions in their natural environment, providing a more authentic and realistic representation of their experiences.

Promoting Participant Engagement and Collaboration

Furthermore, qualitative research instruments promote participant engagement and collaboration. By using methods such as interviews and focus groups, researchers can actively involve participants in the research process. This engagement fosters a sense of ownership and empowerment among participants, leading to more meaningful and insightful data.

Exploring Complex Issues Through Qualitative Research

Lastly, qualitative research instruments allow for the exploration of complex issues. Qualitative research is particularly useful when studying complex phenomena that cannot be easily quantified or measured. It allows researchers to delve into the underlying meanings, motivations, and social dynamics that shape individuals’ behaviors and experiences.

Limitations of Qualitative Research Instruments

Qualitative research instruments have several limitations that researchers need to consider when conducting their studies. In this section, we will delve into the limitations of qualitative research instruments as compared to quantitative research.

Time-Consuming Nature of Qualitative Research

One of the main drawbacks of qualitative research is that the process is time-consuming. Unlike quantitative research, which can collect data from a large sample size in a relatively short period of time, qualitative research requires in-depth interviews, observations, and analysis, which can take a significant amount of time.

Subjectivity and Potential Bias in Qualitative Research

Another limitation of qualitative research instruments is that the interpretations are subjective. Since qualitative research focuses on understanding the meaning and context of phenomena, the interpretations of the data can vary depending on the researcher’s perspective and biases. This subjectivity can introduce potential bias and affect the reliability and validity of the findings.

Complexity of Data Analysis

Additionally, qualitative research instruments often involve complex data analysis. Unlike quantitative research, which can use statistical methods to analyze data, qualitative research requires researchers to analyze textual or visual data, which can be time-consuming and challenging. The analysis process involves coding, categorizing, and interpreting the data, which requires expertise and careful attention to detail.

Challenges in Maintaining Anonymity and Privacy

Furthermore, qualitative research instruments may face challenges in maintaining anonymity. In some cases, researchers may need to collect sensitive or personal information from participants, which can raise ethical concerns . Ensuring the privacy and confidentiality of participants’ data can be challenging, and researchers need to take appropriate measures to protect the participants’ identities and maintain their trust.

Limited Generalizability of Qualitative Research Findings

Another limitation of qualitative research instruments is the limited generalizability of the findings. Qualitative research often focuses on a specific context or a small sample size, which may limit the generalizability of the findings to a larger population. While qualitative research provides rich and detailed insights into a particular phenomenon, it may not be representative of the broader population or applicable to other settings.

Difficulty in Replicating Qualitative Research Findings

Lastly, replicating findings in qualitative research can be difficult. Since qualitative research often involves in-depth exploration of a specific phenomenon, replicating the exact conditions and context of the original study can be challenging. This can make it difficult for other researchers to validate or replicate the findings, which is an essential aspect of scientific research.

Despite these limitations, qualitative research instruments offer valuable insights and understanding of complex phenomena. By acknowledging and addressing these limitations, researchers can enhance the rigor and validity of their qualitative research studies.

In conclusion, qualitative research instruments are powerful tools that enable researchers to explore and uncover the complexities of human experiences. By utilizing a range of instruments and considering their advantages and limitations, researchers can enhance the rigor and depth of their qualitative research studies.

Leave a Comment Cancel reply

Save my name, email, and website in this browser for the next time I comment.

Related articles

Writing engaging introduction in research papers : tips and tricks.

Comparative-Frameworks-

Understanding Comparative Frameworks: Their Importance, Components, Examples and 8 Best Practices

artificial-intelligence-in-thesis-writing-for-phd-students

Revolutionizing Effective Thesis Writing for PhD Students Using Artificial Intelligence!

Interviews-as-One-of-Qualitative-Research-Instruments

3 Types of Interviews in Qualitative Research: An Essential Research Instrument and Handy Tips to Conduct Them

highlight abstracts

Highlight Abstracts: An Ultimate Guide For Researchers!

Critical abstracts

Crafting Critical Abstracts: 11 Expert Strategies for Summarizing Research

Informative Abstract

Crafting the Perfect Informative Abstract: Definition, Importance and 8 Expert Writing Tips

Descriptive Abstracts: A Complete Guide to Crafting Effective Summaries in Research Writing

Descriptive Abstracts: A Complete Guide to Crafting Effective Summaries in Research Writing

U.S. flag

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

  • Publications
  • Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

  • Advanced Search
  • Journal List
  • Neurol Res Pract

Logo of neurrp

How to use and assess qualitative research methods

Loraine busetto.

1 Department of Neurology, Heidelberg University Hospital, Im Neuenheimer Feld 400, 69120 Heidelberg, Germany

Wolfgang Wick

2 Clinical Cooperation Unit Neuro-Oncology, German Cancer Research Center, Heidelberg, Germany

Christoph Gumbinger

Associated data.

Not applicable.

This paper aims to provide an overview of the use and assessment of qualitative research methods in the health sciences. Qualitative research can be defined as the study of the nature of phenomena and is especially appropriate for answering questions of why something is (not) observed, assessing complex multi-component interventions, and focussing on intervention improvement. The most common methods of data collection are document study, (non-) participant observations, semi-structured interviews and focus groups. For data analysis, field-notes and audio-recordings are transcribed into protocols and transcripts, and coded using qualitative data management software. Criteria such as checklists, reflexivity, sampling strategies, piloting, co-coding, member-checking and stakeholder involvement can be used to enhance and assess the quality of the research conducted. Using qualitative in addition to quantitative designs will equip us with better tools to address a greater range of research problems, and to fill in blind spots in current neurological research and practice.

The aim of this paper is to provide an overview of qualitative research methods, including hands-on information on how they can be used, reported and assessed. This article is intended for beginning qualitative researchers in the health sciences as well as experienced quantitative researchers who wish to broaden their understanding of qualitative research.

What is qualitative research?

Qualitative research is defined as “the study of the nature of phenomena”, including “their quality, different manifestations, the context in which they appear or the perspectives from which they can be perceived” , but excluding “their range, frequency and place in an objectively determined chain of cause and effect” [ 1 ]. This formal definition can be complemented with a more pragmatic rule of thumb: qualitative research generally includes data in form of words rather than numbers [ 2 ].

Why conduct qualitative research?

Because some research questions cannot be answered using (only) quantitative methods. For example, one Australian study addressed the issue of why patients from Aboriginal communities often present late or not at all to specialist services offered by tertiary care hospitals. Using qualitative interviews with patients and staff, it found one of the most significant access barriers to be transportation problems, including some towns and communities simply not having a bus service to the hospital [ 3 ]. A quantitative study could have measured the number of patients over time or even looked at possible explanatory factors – but only those previously known or suspected to be of relevance. To discover reasons for observed patterns, especially the invisible or surprising ones, qualitative designs are needed.

While qualitative research is common in other fields, it is still relatively underrepresented in health services research. The latter field is more traditionally rooted in the evidence-based-medicine paradigm, as seen in " research that involves testing the effectiveness of various strategies to achieve changes in clinical practice, preferably applying randomised controlled trial study designs (...) " [ 4 ]. This focus on quantitative research and specifically randomised controlled trials (RCT) is visible in the idea of a hierarchy of research evidence which assumes that some research designs are objectively better than others, and that choosing a "lesser" design is only acceptable when the better ones are not practically or ethically feasible [ 5 , 6 ]. Others, however, argue that an objective hierarchy does not exist, and that, instead, the research design and methods should be chosen to fit the specific research question at hand – "questions before methods" [ 2 , 7 – 9 ]. This means that even when an RCT is possible, some research problems require a different design that is better suited to addressing them. Arguing in JAMA, Berwick uses the example of rapid response teams in hospitals, which he describes as " a complex, multicomponent intervention – essentially a process of social change" susceptible to a range of different context factors including leadership or organisation history. According to him, "[in] such complex terrain, the RCT is an impoverished way to learn. Critics who use it as a truth standard in this context are incorrect" [ 8 ] . Instead of limiting oneself to RCTs, Berwick recommends embracing a wider range of methods , including qualitative ones, which for "these specific applications, (...) are not compromises in learning how to improve; they are superior" [ 8 ].

Research problems that can be approached particularly well using qualitative methods include assessing complex multi-component interventions or systems (of change), addressing questions beyond “what works”, towards “what works for whom when, how and why”, and focussing on intervention improvement rather than accreditation [ 7 , 9 – 12 ]. Using qualitative methods can also help shed light on the “softer” side of medical treatment. For example, while quantitative trials can measure the costs and benefits of neuro-oncological treatment in terms of survival rates or adverse effects, qualitative research can help provide a better understanding of patient or caregiver stress, visibility of illness or out-of-pocket expenses.

How to conduct qualitative research?

Given that qualitative research is characterised by flexibility, openness and responsivity to context, the steps of data collection and analysis are not as separate and consecutive as they tend to be in quantitative research [ 13 , 14 ]. As Fossey puts it : “sampling, data collection, analysis and interpretation are related to each other in a cyclical (iterative) manner, rather than following one after another in a stepwise approach” [ 15 ]. The researcher can make educated decisions with regard to the choice of method, how they are implemented, and to which and how many units they are applied [ 13 ]. As shown in Fig.  1 , this can involve several back-and-forth steps between data collection and analysis where new insights and experiences can lead to adaption and expansion of the original plan. Some insights may also necessitate a revision of the research question and/or the research design as a whole. The process ends when saturation is achieved, i.e. when no relevant new information can be found (see also below: sampling and saturation). For reasons of transparency, it is essential for all decisions as well as the underlying reasoning to be well-documented.

An external file that holds a picture, illustration, etc.
Object name is 42466_2020_59_Fig1_HTML.jpg

Iterative research process

While it is not always explicitly addressed, qualitative methods reflect a different underlying research paradigm than quantitative research (e.g. constructivism or interpretivism as opposed to positivism). The choice of methods can be based on the respective underlying substantive theory or theoretical framework used by the researcher [ 2 ].

Data collection

The methods of qualitative data collection most commonly used in health research are document study, observations, semi-structured interviews and focus groups [ 1 , 14 , 16 , 17 ].

Document study

Document study (also called document analysis) refers to the review by the researcher of written materials [ 14 ]. These can include personal and non-personal documents such as archives, annual reports, guidelines, policy documents, diaries or letters.

Observations

Observations are particularly useful to gain insights into a certain setting and actual behaviour – as opposed to reported behaviour or opinions [ 13 ]. Qualitative observations can be either participant or non-participant in nature. In participant observations, the observer is part of the observed setting, for example a nurse working in an intensive care unit [ 18 ]. In non-participant observations, the observer is “on the outside looking in”, i.e. present in but not part of the situation, trying not to influence the setting by their presence. Observations can be planned (e.g. for 3 h during the day or night shift) or ad hoc (e.g. as soon as a stroke patient arrives at the emergency room). During the observation, the observer takes notes on everything or certain pre-determined parts of what is happening around them, for example focusing on physician-patient interactions or communication between different professional groups. Written notes can be taken during or after the observations, depending on feasibility (which is usually lower during participant observations) and acceptability (e.g. when the observer is perceived to be judging the observed). Afterwards, these field notes are transcribed into observation protocols. If more than one observer was involved, field notes are taken independently, but notes can be consolidated into one protocol after discussions. Advantages of conducting observations include minimising the distance between the researcher and the researched, the potential discovery of topics that the researcher did not realise were relevant and gaining deeper insights into the real-world dimensions of the research problem at hand [ 18 ].

Semi-structured interviews

Hijmans & Kuyper describe qualitative interviews as “an exchange with an informal character, a conversation with a goal” [ 19 ]. Interviews are used to gain insights into a person’s subjective experiences, opinions and motivations – as opposed to facts or behaviours [ 13 ]. Interviews can be distinguished by the degree to which they are structured (i.e. a questionnaire), open (e.g. free conversation or autobiographical interviews) or semi-structured [ 2 , 13 ]. Semi-structured interviews are characterized by open-ended questions and the use of an interview guide (or topic guide/list) in which the broad areas of interest, sometimes including sub-questions, are defined [ 19 ]. The pre-defined topics in the interview guide can be derived from the literature, previous research or a preliminary method of data collection, e.g. document study or observations. The topic list is usually adapted and improved at the start of the data collection process as the interviewer learns more about the field [ 20 ]. Across interviews the focus on the different (blocks of) questions may differ and some questions may be skipped altogether (e.g. if the interviewee is not able or willing to answer the questions or for concerns about the total length of the interview) [ 20 ]. Qualitative interviews are usually not conducted in written format as it impedes on the interactive component of the method [ 20 ]. In comparison to written surveys, qualitative interviews have the advantage of being interactive and allowing for unexpected topics to emerge and to be taken up by the researcher. This can also help overcome a provider or researcher-centred bias often found in written surveys, which by nature, can only measure what is already known or expected to be of relevance to the researcher. Interviews can be audio- or video-taped; but sometimes it is only feasible or acceptable for the interviewer to take written notes [ 14 , 16 , 20 ].

Focus groups

Focus groups are group interviews to explore participants’ expertise and experiences, including explorations of how and why people behave in certain ways [ 1 ]. Focus groups usually consist of 6–8 people and are led by an experienced moderator following a topic guide or “script” [ 21 ]. They can involve an observer who takes note of the non-verbal aspects of the situation, possibly using an observation guide [ 21 ]. Depending on researchers’ and participants’ preferences, the discussions can be audio- or video-taped and transcribed afterwards [ 21 ]. Focus groups are useful for bringing together homogeneous (to a lesser extent heterogeneous) groups of participants with relevant expertise and experience on a given topic on which they can share detailed information [ 21 ]. Focus groups are a relatively easy, fast and inexpensive method to gain access to information on interactions in a given group, i.e. “the sharing and comparing” among participants [ 21 ]. Disadvantages include less control over the process and a lesser extent to which each individual may participate. Moreover, focus group moderators need experience, as do those tasked with the analysis of the resulting data. Focus groups can be less appropriate for discussing sensitive topics that participants might be reluctant to disclose in a group setting [ 13 ]. Moreover, attention must be paid to the emergence of “groupthink” as well as possible power dynamics within the group, e.g. when patients are awed or intimidated by health professionals.

Choosing the “right” method

As explained above, the school of thought underlying qualitative research assumes no objective hierarchy of evidence and methods. This means that each choice of single or combined methods has to be based on the research question that needs to be answered and a critical assessment with regard to whether or to what extent the chosen method can accomplish this – i.e. the “fit” between question and method [ 14 ]. It is necessary for these decisions to be documented when they are being made, and to be critically discussed when reporting methods and results.

Let us assume that our research aim is to examine the (clinical) processes around acute endovascular treatment (EVT), from the patient’s arrival at the emergency room to recanalization, with the aim to identify possible causes for delay and/or other causes for sub-optimal treatment outcome. As a first step, we could conduct a document study of the relevant standard operating procedures (SOPs) for this phase of care – are they up-to-date and in line with current guidelines? Do they contain any mistakes, irregularities or uncertainties that could cause delays or other problems? Regardless of the answers to these questions, the results have to be interpreted based on what they are: a written outline of what care processes in this hospital should look like. If we want to know what they actually look like in practice, we can conduct observations of the processes described in the SOPs. These results can (and should) be analysed in themselves, but also in comparison to the results of the document analysis, especially as regards relevant discrepancies. Do the SOPs outline specific tests for which no equipment can be observed or tasks to be performed by specialized nurses who are not present during the observation? It might also be possible that the written SOP is outdated, but the actual care provided is in line with current best practice. In order to find out why these discrepancies exist, it can be useful to conduct interviews. Are the physicians simply not aware of the SOPs (because their existence is limited to the hospital’s intranet) or do they actively disagree with them or does the infrastructure make it impossible to provide the care as described? Another rationale for adding interviews is that some situations (or all of their possible variations for different patient groups or the day, night or weekend shift) cannot practically or ethically be observed. In this case, it is possible to ask those involved to report on their actions – being aware that this is not the same as the actual observation. A senior physician’s or hospital manager’s description of certain situations might differ from a nurse’s or junior physician’s one, maybe because they intentionally misrepresent facts or maybe because different aspects of the process are visible or important to them. In some cases, it can also be relevant to consider to whom the interviewee is disclosing this information – someone they trust, someone they are otherwise not connected to, or someone they suspect or are aware of being in a potentially “dangerous” power relationship to them. Lastly, a focus group could be conducted with representatives of the relevant professional groups to explore how and why exactly they provide care around EVT. The discussion might reveal discrepancies (between SOPs and actual care or between different physicians) and motivations to the researchers as well as to the focus group members that they might not have been aware of themselves. For the focus group to deliver relevant information, attention has to be paid to its composition and conduct, for example, to make sure that all participants feel safe to disclose sensitive or potentially problematic information or that the discussion is not dominated by (senior) physicians only. The resulting combination of data collection methods is shown in Fig.  2 .

An external file that holds a picture, illustration, etc.
Object name is 42466_2020_59_Fig2_HTML.jpg

Possible combination of data collection methods

Attributions for icons: “Book” by Serhii Smirnov, “Interview” by Adrien Coquet, FR, “Magnifying Glass” by anggun, ID, “Business communication” by Vectors Market; all from the Noun Project

The combination of multiple data source as described for this example can be referred to as “triangulation”, in which multiple measurements are carried out from different angles to achieve a more comprehensive understanding of the phenomenon under study [ 22 , 23 ].

Data analysis

To analyse the data collected through observations, interviews and focus groups these need to be transcribed into protocols and transcripts (see Fig.  3 ). Interviews and focus groups can be transcribed verbatim , with or without annotations for behaviour (e.g. laughing, crying, pausing) and with or without phonetic transcription of dialects and filler words, depending on what is expected or known to be relevant for the analysis. In the next step, the protocols and transcripts are coded , that is, marked (or tagged, labelled) with one or more short descriptors of the content of a sentence or paragraph [ 2 , 15 , 23 ]. Jansen describes coding as “connecting the raw data with “theoretical” terms” [ 20 ]. In a more practical sense, coding makes raw data sortable. This makes it possible to extract and examine all segments describing, say, a tele-neurology consultation from multiple data sources (e.g. SOPs, emergency room observations, staff and patient interview). In a process of synthesis and abstraction, the codes are then grouped, summarised and/or categorised [ 15 , 20 ]. The end product of the coding or analysis process is a descriptive theory of the behavioural pattern under investigation [ 20 ]. The coding process is performed using qualitative data management software, the most common ones being InVivo, MaxQDA and Atlas.ti. It should be noted that these are data management tools which support the analysis performed by the researcher(s) [ 14 ].

An external file that holds a picture, illustration, etc.
Object name is 42466_2020_59_Fig3_HTML.jpg

From data collection to data analysis

Attributions for icons: see Fig. ​ Fig.2, 2 , also “Speech to text” by Trevor Dsouza, “Field Notes” by Mike O’Brien, US, “Voice Record” by ProSymbols, US, “Inspection” by Made, AU, and “Cloud” by Graphic Tigers; all from the Noun Project

How to report qualitative research?

Protocols of qualitative research can be published separately and in advance of the study results. However, the aim is not the same as in RCT protocols, i.e. to pre-define and set in stone the research questions and primary or secondary endpoints. Rather, it is a way to describe the research methods in detail, which might not be possible in the results paper given journals’ word limits. Qualitative research papers are usually longer than their quantitative counterparts to allow for deep understanding and so-called “thick description”. In the methods section, the focus is on transparency of the methods used, including why, how and by whom they were implemented in the specific study setting, so as to enable a discussion of whether and how this may have influenced data collection, analysis and interpretation. The results section usually starts with a paragraph outlining the main findings, followed by more detailed descriptions of, for example, the commonalities, discrepancies or exceptions per category [ 20 ]. Here it is important to support main findings by relevant quotations, which may add information, context, emphasis or real-life examples [ 20 , 23 ]. It is subject to debate in the field whether it is relevant to state the exact number or percentage of respondents supporting a certain statement (e.g. “Five interviewees expressed negative feelings towards XYZ”) [ 21 ].

How to combine qualitative with quantitative research?

Qualitative methods can be combined with other methods in multi- or mixed methods designs, which “[employ] two or more different methods [ …] within the same study or research program rather than confining the research to one single method” [ 24 ]. Reasons for combining methods can be diverse, including triangulation for corroboration of findings, complementarity for illustration and clarification of results, expansion to extend the breadth and range of the study, explanation of (unexpected) results generated with one method with the help of another, or offsetting the weakness of one method with the strength of another [ 1 , 17 , 24 – 26 ]. The resulting designs can be classified according to when, why and how the different quantitative and/or qualitative data strands are combined. The three most common types of mixed method designs are the convergent parallel design , the explanatory sequential design and the exploratory sequential design. The designs with examples are shown in Fig.  4 .

An external file that holds a picture, illustration, etc.
Object name is 42466_2020_59_Fig4_HTML.jpg

Three common mixed methods designs

In the convergent parallel design, a qualitative study is conducted in parallel to and independently of a quantitative study, and the results of both studies are compared and combined at the stage of interpretation of results. Using the above example of EVT provision, this could entail setting up a quantitative EVT registry to measure process times and patient outcomes in parallel to conducting the qualitative research outlined above, and then comparing results. Amongst other things, this would make it possible to assess whether interview respondents’ subjective impressions of patients receiving good care match modified Rankin Scores at follow-up, or whether observed delays in care provision are exceptions or the rule when compared to door-to-needle times as documented in the registry. In the explanatory sequential design, a quantitative study is carried out first, followed by a qualitative study to help explain the results from the quantitative study. This would be an appropriate design if the registry alone had revealed relevant delays in door-to-needle times and the qualitative study would be used to understand where and why these occurred, and how they could be improved. In the exploratory design, the qualitative study is carried out first and its results help informing and building the quantitative study in the next step [ 26 ]. If the qualitative study around EVT provision had shown a high level of dissatisfaction among the staff members involved, a quantitative questionnaire investigating staff satisfaction could be set up in the next step, informed by the qualitative study on which topics dissatisfaction had been expressed. Amongst other things, the questionnaire design would make it possible to widen the reach of the research to more respondents from different (types of) hospitals, regions, countries or settings, and to conduct sub-group analyses for different professional groups.

How to assess qualitative research?

A variety of assessment criteria and lists have been developed for qualitative research, ranging in their focus and comprehensiveness [ 14 , 17 , 27 ]. However, none of these has been elevated to the “gold standard” in the field. In the following, we therefore focus on a set of commonly used assessment criteria that, from a practical standpoint, a researcher can look for when assessing a qualitative research report or paper.

Assessors should check the authors’ use of and adherence to the relevant reporting checklists (e.g. Standards for Reporting Qualitative Research (SRQR)) to make sure all items that are relevant for this type of research are addressed [ 23 , 28 ]. Discussions of quantitative measures in addition to or instead of these qualitative measures can be a sign of lower quality of the research (paper). Providing and adhering to a checklist for qualitative research contributes to an important quality criterion for qualitative research, namely transparency [ 15 , 17 , 23 ].

Reflexivity

While methodological transparency and complete reporting is relevant for all types of research, some additional criteria must be taken into account for qualitative research. This includes what is called reflexivity, i.e. sensitivity to the relationship between the researcher and the researched, including how contact was established and maintained, or the background and experience of the researcher(s) involved in data collection and analysis. Depending on the research question and population to be researched this can be limited to professional experience, but it may also include gender, age or ethnicity [ 17 , 27 ]. These details are relevant because in qualitative research, as opposed to quantitative research, the researcher as a person cannot be isolated from the research process [ 23 ]. It may influence the conversation when an interviewed patient speaks to an interviewer who is a physician, or when an interviewee is asked to discuss a gynaecological procedure with a male interviewer, and therefore the reader must be made aware of these details [ 19 ].

Sampling and saturation

The aim of qualitative sampling is for all variants of the objects of observation that are deemed relevant for the study to be present in the sample “ to see the issue and its meanings from as many angles as possible” [ 1 , 16 , 19 , 20 , 27 ] , and to ensure “information-richness [ 15 ]. An iterative sampling approach is advised, in which data collection (e.g. five interviews) is followed by data analysis, followed by more data collection to find variants that are lacking in the current sample. This process continues until no new (relevant) information can be found and further sampling becomes redundant – which is called saturation [ 1 , 15 ] . In other words: qualitative data collection finds its end point not a priori , but when the research team determines that saturation has been reached [ 29 , 30 ].

This is also the reason why most qualitative studies use deliberate instead of random sampling strategies. This is generally referred to as “ purposive sampling” , in which researchers pre-define which types of participants or cases they need to include so as to cover all variations that are expected to be of relevance, based on the literature, previous experience or theory (i.e. theoretical sampling) [ 14 , 20 ]. Other types of purposive sampling include (but are not limited to) maximum variation sampling, critical case sampling or extreme or deviant case sampling [ 2 ]. In the above EVT example, a purposive sample could include all relevant professional groups and/or all relevant stakeholders (patients, relatives) and/or all relevant times of observation (day, night and weekend shift).

Assessors of qualitative research should check whether the considerations underlying the sampling strategy were sound and whether or how researchers tried to adapt and improve their strategies in stepwise or cyclical approaches between data collection and analysis to achieve saturation [ 14 ].

Good qualitative research is iterative in nature, i.e. it goes back and forth between data collection and analysis, revising and improving the approach where necessary. One example of this are pilot interviews, where different aspects of the interview (especially the interview guide, but also, for example, the site of the interview or whether the interview can be audio-recorded) are tested with a small number of respondents, evaluated and revised [ 19 ]. In doing so, the interviewer learns which wording or types of questions work best, or which is the best length of an interview with patients who have trouble concentrating for an extended time. Of course, the same reasoning applies to observations or focus groups which can also be piloted.

Ideally, coding should be performed by at least two researchers, especially at the beginning of the coding process when a common approach must be defined, including the establishment of a useful coding list (or tree), and when a common meaning of individual codes must be established [ 23 ]. An initial sub-set or all transcripts can be coded independently by the coders and then compared and consolidated after regular discussions in the research team. This is to make sure that codes are applied consistently to the research data.

Member checking

Member checking, also called respondent validation , refers to the practice of checking back with study respondents to see if the research is in line with their views [ 14 , 27 ]. This can happen after data collection or analysis or when first results are available [ 23 ]. For example, interviewees can be provided with (summaries of) their transcripts and asked whether they believe this to be a complete representation of their views or whether they would like to clarify or elaborate on their responses [ 17 ]. Respondents’ feedback on these issues then becomes part of the data collection and analysis [ 27 ].

Stakeholder involvement

In those niches where qualitative approaches have been able to evolve and grow, a new trend has seen the inclusion of patients and their representatives not only as study participants (i.e. “members”, see above) but as consultants to and active participants in the broader research process [ 31 – 33 ]. The underlying assumption is that patients and other stakeholders hold unique perspectives and experiences that add value beyond their own single story, making the research more relevant and beneficial to researchers, study participants and (future) patients alike [ 34 , 35 ]. Using the example of patients on or nearing dialysis, a recent scoping review found that 80% of clinical research did not address the top 10 research priorities identified by patients and caregivers [ 32 , 36 ]. In this sense, the involvement of the relevant stakeholders, especially patients and relatives, is increasingly being seen as a quality indicator in and of itself.

How not to assess qualitative research

The above overview does not include certain items that are routine in assessments of quantitative research. What follows is a non-exhaustive, non-representative, experience-based list of the quantitative criteria often applied to the assessment of qualitative research, as well as an explanation of the limited usefulness of these endeavours.

Protocol adherence

Given the openness and flexibility of qualitative research, it should not be assessed by how well it adheres to pre-determined and fixed strategies – in other words: its rigidity. Instead, the assessor should look for signs of adaptation and refinement based on lessons learned from earlier steps in the research process.

Sample size

For the reasons explained above, qualitative research does not require specific sample sizes, nor does it require that the sample size be determined a priori [ 1 , 14 , 27 , 37 – 39 ]. Sample size can only be a useful quality indicator when related to the research purpose, the chosen methodology and the composition of the sample, i.e. who was included and why.

Randomisation

While some authors argue that randomisation can be used in qualitative research, this is not commonly the case, as neither its feasibility nor its necessity or usefulness has been convincingly established for qualitative research [ 13 , 27 ]. Relevant disadvantages include the negative impact of a too large sample size as well as the possibility (or probability) of selecting “ quiet, uncooperative or inarticulate individuals ” [ 17 ]. Qualitative studies do not use control groups, either.

Interrater reliability, variability and other “objectivity checks”

The concept of “interrater reliability” is sometimes used in qualitative research to assess to which extent the coding approach overlaps between the two co-coders. However, it is not clear what this measure tells us about the quality of the analysis [ 23 ]. This means that these scores can be included in qualitative research reports, preferably with some additional information on what the score means for the analysis, but it is not a requirement. Relatedly, it is not relevant for the quality or “objectivity” of qualitative research to separate those who recruited the study participants and collected and analysed the data. Experiences even show that it might be better to have the same person or team perform all of these tasks [ 20 ]. First, when researchers introduce themselves during recruitment this can enhance trust when the interview takes place days or weeks later with the same researcher. Second, when the audio-recording is transcribed for analysis, the researcher conducting the interviews will usually remember the interviewee and the specific interview situation during data analysis. This might be helpful in providing additional context information for interpretation of data, e.g. on whether something might have been meant as a joke [ 18 ].

Not being quantitative research

Being qualitative research instead of quantitative research should not be used as an assessment criterion if it is used irrespectively of the research problem at hand. Similarly, qualitative research should not be required to be combined with quantitative research per se – unless mixed methods research is judged as inherently better than single-method research. In this case, the same criterion should be applied for quantitative studies without a qualitative component.

The main take-away points of this paper are summarised in Table ​ Table1. 1 . We aimed to show that, if conducted well, qualitative research can answer specific research questions that cannot to be adequately answered using (only) quantitative designs. Seeing qualitative and quantitative methods as equal will help us become more aware and critical of the “fit” between the research problem and our chosen methods: I can conduct an RCT to determine the reasons for transportation delays of acute stroke patients – but should I? It also provides us with a greater range of tools to tackle a greater range of research problems more appropriately and successfully, filling in the blind spots on one half of the methodological spectrum to better address the whole complexity of neurological research and practice.

Take-away-points

• Assessing complex multi-component interventions or systems (of change)

• What works for whom when, how and why?

• Focussing on intervention improvement

• Document study

• Observations (participant or non-participant)

• Interviews (especially semi-structured)

• Focus groups

• Transcription of audio-recordings and field notes into transcripts and protocols

• Coding of protocols

• Using qualitative data management software

• Combinations of quantitative and/or qualitative methods, e.g.:

• : quali and quanti in parallel

• : quanti followed by quali

• : quali followed by quanti

• Checklists

• Reflexivity

• Sampling strategies

• Piloting

• Co-coding

• Member checking

• Stakeholder involvement

• Protocol adherence

• Sample size

• Randomization

• Interrater reliability, variability and other “objectivity checks”

• Not being quantitative research

Acknowledgements

Abbreviations.

EVTEndovascular treatment
RCTRandomised Controlled Trial
SOPStandard Operating Procedure
SRQRStandards for Reporting Qualitative Research

Authors’ contributions

LB drafted the manuscript; WW and CG revised the manuscript; all authors approved the final versions.

no external funding.

Availability of data and materials

Ethics approval and consent to participate, consent for publication, competing interests.

The authors declare no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

  • Privacy Policy

Research Method

Home » Qualitative Research – Methods, Analysis Types and Guide

Qualitative Research – Methods, Analysis Types and Guide

Table of Contents

Qualitative Research

Qualitative Research

Qualitative research is a type of research methodology that focuses on exploring and understanding people’s beliefs, attitudes, behaviors, and experiences through the collection and analysis of non-numerical data. It seeks to answer research questions through the examination of subjective data, such as interviews, focus groups, observations, and textual analysis.

Qualitative research aims to uncover the meaning and significance of social phenomena, and it typically involves a more flexible and iterative approach to data collection and analysis compared to quantitative research. Qualitative research is often used in fields such as sociology, anthropology, psychology, and education.

Qualitative Research Methods

Types of Qualitative Research

Qualitative Research Methods are as follows:

One-to-One Interview

This method involves conducting an interview with a single participant to gain a detailed understanding of their experiences, attitudes, and beliefs. One-to-one interviews can be conducted in-person, over the phone, or through video conferencing. The interviewer typically uses open-ended questions to encourage the participant to share their thoughts and feelings. One-to-one interviews are useful for gaining detailed insights into individual experiences.

Focus Groups

This method involves bringing together a group of people to discuss a specific topic in a structured setting. The focus group is led by a moderator who guides the discussion and encourages participants to share their thoughts and opinions. Focus groups are useful for generating ideas and insights, exploring social norms and attitudes, and understanding group dynamics.

Ethnographic Studies

This method involves immersing oneself in a culture or community to gain a deep understanding of its norms, beliefs, and practices. Ethnographic studies typically involve long-term fieldwork and observation, as well as interviews and document analysis. Ethnographic studies are useful for understanding the cultural context of social phenomena and for gaining a holistic understanding of complex social processes.

Text Analysis

This method involves analyzing written or spoken language to identify patterns and themes. Text analysis can be quantitative or qualitative. Qualitative text analysis involves close reading and interpretation of texts to identify recurring themes, concepts, and patterns. Text analysis is useful for understanding media messages, public discourse, and cultural trends.

This method involves an in-depth examination of a single person, group, or event to gain an understanding of complex phenomena. Case studies typically involve a combination of data collection methods, such as interviews, observations, and document analysis, to provide a comprehensive understanding of the case. Case studies are useful for exploring unique or rare cases, and for generating hypotheses for further research.

Process of Observation

This method involves systematically observing and recording behaviors and interactions in natural settings. The observer may take notes, use audio or video recordings, or use other methods to document what they see. Process of observation is useful for understanding social interactions, cultural practices, and the context in which behaviors occur.

Record Keeping

This method involves keeping detailed records of observations, interviews, and other data collected during the research process. Record keeping is essential for ensuring the accuracy and reliability of the data, and for providing a basis for analysis and interpretation.

This method involves collecting data from a large sample of participants through a structured questionnaire. Surveys can be conducted in person, over the phone, through mail, or online. Surveys are useful for collecting data on attitudes, beliefs, and behaviors, and for identifying patterns and trends in a population.

Qualitative data analysis is a process of turning unstructured data into meaningful insights. It involves extracting and organizing information from sources like interviews, focus groups, and surveys. The goal is to understand people’s attitudes, behaviors, and motivations

Qualitative Research Analysis Methods

Qualitative Research analysis methods involve a systematic approach to interpreting and making sense of the data collected in qualitative research. Here are some common qualitative data analysis methods:

Thematic Analysis

This method involves identifying patterns or themes in the data that are relevant to the research question. The researcher reviews the data, identifies keywords or phrases, and groups them into categories or themes. Thematic analysis is useful for identifying patterns across multiple data sources and for generating new insights into the research topic.

Content Analysis

This method involves analyzing the content of written or spoken language to identify key themes or concepts. Content analysis can be quantitative or qualitative. Qualitative content analysis involves close reading and interpretation of texts to identify recurring themes, concepts, and patterns. Content analysis is useful for identifying patterns in media messages, public discourse, and cultural trends.

Discourse Analysis

This method involves analyzing language to understand how it constructs meaning and shapes social interactions. Discourse analysis can involve a variety of methods, such as conversation analysis, critical discourse analysis, and narrative analysis. Discourse analysis is useful for understanding how language shapes social interactions, cultural norms, and power relationships.

Grounded Theory Analysis

This method involves developing a theory or explanation based on the data collected. Grounded theory analysis starts with the data and uses an iterative process of coding and analysis to identify patterns and themes in the data. The theory or explanation that emerges is grounded in the data, rather than preconceived hypotheses. Grounded theory analysis is useful for understanding complex social phenomena and for generating new theoretical insights.

Narrative Analysis

This method involves analyzing the stories or narratives that participants share to gain insights into their experiences, attitudes, and beliefs. Narrative analysis can involve a variety of methods, such as structural analysis, thematic analysis, and discourse analysis. Narrative analysis is useful for understanding how individuals construct their identities, make sense of their experiences, and communicate their values and beliefs.

Phenomenological Analysis

This method involves analyzing how individuals make sense of their experiences and the meanings they attach to them. Phenomenological analysis typically involves in-depth interviews with participants to explore their experiences in detail. Phenomenological analysis is useful for understanding subjective experiences and for developing a rich understanding of human consciousness.

Comparative Analysis

This method involves comparing and contrasting data across different cases or groups to identify similarities and differences. Comparative analysis can be used to identify patterns or themes that are common across multiple cases, as well as to identify unique or distinctive features of individual cases. Comparative analysis is useful for understanding how social phenomena vary across different contexts and groups.

Applications of Qualitative Research

Qualitative research has many applications across different fields and industries. Here are some examples of how qualitative research is used:

  • Market Research: Qualitative research is often used in market research to understand consumer attitudes, behaviors, and preferences. Researchers conduct focus groups and one-on-one interviews with consumers to gather insights into their experiences and perceptions of products and services.
  • Health Care: Qualitative research is used in health care to explore patient experiences and perspectives on health and illness. Researchers conduct in-depth interviews with patients and their families to gather information on their experiences with different health care providers and treatments.
  • Education: Qualitative research is used in education to understand student experiences and to develop effective teaching strategies. Researchers conduct classroom observations and interviews with students and teachers to gather insights into classroom dynamics and instructional practices.
  • Social Work : Qualitative research is used in social work to explore social problems and to develop interventions to address them. Researchers conduct in-depth interviews with individuals and families to understand their experiences with poverty, discrimination, and other social problems.
  • Anthropology : Qualitative research is used in anthropology to understand different cultures and societies. Researchers conduct ethnographic studies and observe and interview members of different cultural groups to gain insights into their beliefs, practices, and social structures.
  • Psychology : Qualitative research is used in psychology to understand human behavior and mental processes. Researchers conduct in-depth interviews with individuals to explore their thoughts, feelings, and experiences.
  • Public Policy : Qualitative research is used in public policy to explore public attitudes and to inform policy decisions. Researchers conduct focus groups and one-on-one interviews with members of the public to gather insights into their perspectives on different policy issues.

How to Conduct Qualitative Research

Here are some general steps for conducting qualitative research:

  • Identify your research question: Qualitative research starts with a research question or set of questions that you want to explore. This question should be focused and specific, but also broad enough to allow for exploration and discovery.
  • Select your research design: There are different types of qualitative research designs, including ethnography, case study, grounded theory, and phenomenology. You should select a design that aligns with your research question and that will allow you to gather the data you need to answer your research question.
  • Recruit participants: Once you have your research question and design, you need to recruit participants. The number of participants you need will depend on your research design and the scope of your research. You can recruit participants through advertisements, social media, or through personal networks.
  • Collect data: There are different methods for collecting qualitative data, including interviews, focus groups, observation, and document analysis. You should select the method or methods that align with your research design and that will allow you to gather the data you need to answer your research question.
  • Analyze data: Once you have collected your data, you need to analyze it. This involves reviewing your data, identifying patterns and themes, and developing codes to organize your data. You can use different software programs to help you analyze your data, or you can do it manually.
  • Interpret data: Once you have analyzed your data, you need to interpret it. This involves making sense of the patterns and themes you have identified, and developing insights and conclusions that answer your research question. You should be guided by your research question and use your data to support your conclusions.
  • Communicate results: Once you have interpreted your data, you need to communicate your results. This can be done through academic papers, presentations, or reports. You should be clear and concise in your communication, and use examples and quotes from your data to support your findings.

Examples of Qualitative Research

Here are some real-time examples of qualitative research:

  • Customer Feedback: A company may conduct qualitative research to understand the feedback and experiences of its customers. This may involve conducting focus groups or one-on-one interviews with customers to gather insights into their attitudes, behaviors, and preferences.
  • Healthcare : A healthcare provider may conduct qualitative research to explore patient experiences and perspectives on health and illness. This may involve conducting in-depth interviews with patients and their families to gather information on their experiences with different health care providers and treatments.
  • Education : An educational institution may conduct qualitative research to understand student experiences and to develop effective teaching strategies. This may involve conducting classroom observations and interviews with students and teachers to gather insights into classroom dynamics and instructional practices.
  • Social Work: A social worker may conduct qualitative research to explore social problems and to develop interventions to address them. This may involve conducting in-depth interviews with individuals and families to understand their experiences with poverty, discrimination, and other social problems.
  • Anthropology : An anthropologist may conduct qualitative research to understand different cultures and societies. This may involve conducting ethnographic studies and observing and interviewing members of different cultural groups to gain insights into their beliefs, practices, and social structures.
  • Psychology : A psychologist may conduct qualitative research to understand human behavior and mental processes. This may involve conducting in-depth interviews with individuals to explore their thoughts, feelings, and experiences.
  • Public Policy: A government agency or non-profit organization may conduct qualitative research to explore public attitudes and to inform policy decisions. This may involve conducting focus groups and one-on-one interviews with members of the public to gather insights into their perspectives on different policy issues.

Purpose of Qualitative Research

The purpose of qualitative research is to explore and understand the subjective experiences, behaviors, and perspectives of individuals or groups in a particular context. Unlike quantitative research, which focuses on numerical data and statistical analysis, qualitative research aims to provide in-depth, descriptive information that can help researchers develop insights and theories about complex social phenomena.

Qualitative research can serve multiple purposes, including:

  • Exploring new or emerging phenomena : Qualitative research can be useful for exploring new or emerging phenomena, such as new technologies or social trends. This type of research can help researchers develop a deeper understanding of these phenomena and identify potential areas for further study.
  • Understanding complex social phenomena : Qualitative research can be useful for exploring complex social phenomena, such as cultural beliefs, social norms, or political processes. This type of research can help researchers develop a more nuanced understanding of these phenomena and identify factors that may influence them.
  • Generating new theories or hypotheses: Qualitative research can be useful for generating new theories or hypotheses about social phenomena. By gathering rich, detailed data about individuals’ experiences and perspectives, researchers can develop insights that may challenge existing theories or lead to new lines of inquiry.
  • Providing context for quantitative data: Qualitative research can be useful for providing context for quantitative data. By gathering qualitative data alongside quantitative data, researchers can develop a more complete understanding of complex social phenomena and identify potential explanations for quantitative findings.

When to use Qualitative Research

Here are some situations where qualitative research may be appropriate:

  • Exploring a new area: If little is known about a particular topic, qualitative research can help to identify key issues, generate hypotheses, and develop new theories.
  • Understanding complex phenomena: Qualitative research can be used to investigate complex social, cultural, or organizational phenomena that are difficult to measure quantitatively.
  • Investigating subjective experiences: Qualitative research is particularly useful for investigating the subjective experiences of individuals or groups, such as their attitudes, beliefs, values, or emotions.
  • Conducting formative research: Qualitative research can be used in the early stages of a research project to develop research questions, identify potential research participants, and refine research methods.
  • Evaluating interventions or programs: Qualitative research can be used to evaluate the effectiveness of interventions or programs by collecting data on participants’ experiences, attitudes, and behaviors.

Characteristics of Qualitative Research

Qualitative research is characterized by several key features, including:

  • Focus on subjective experience: Qualitative research is concerned with understanding the subjective experiences, beliefs, and perspectives of individuals or groups in a particular context. Researchers aim to explore the meanings that people attach to their experiences and to understand the social and cultural factors that shape these meanings.
  • Use of open-ended questions: Qualitative research relies on open-ended questions that allow participants to provide detailed, in-depth responses. Researchers seek to elicit rich, descriptive data that can provide insights into participants’ experiences and perspectives.
  • Sampling-based on purpose and diversity: Qualitative research often involves purposive sampling, in which participants are selected based on specific criteria related to the research question. Researchers may also seek to include participants with diverse experiences and perspectives to capture a range of viewpoints.
  • Data collection through multiple methods: Qualitative research typically involves the use of multiple data collection methods, such as in-depth interviews, focus groups, and observation. This allows researchers to gather rich, detailed data from multiple sources, which can provide a more complete picture of participants’ experiences and perspectives.
  • Inductive data analysis: Qualitative research relies on inductive data analysis, in which researchers develop theories and insights based on the data rather than testing pre-existing hypotheses. Researchers use coding and thematic analysis to identify patterns and themes in the data and to develop theories and explanations based on these patterns.
  • Emphasis on researcher reflexivity: Qualitative research recognizes the importance of the researcher’s role in shaping the research process and outcomes. Researchers are encouraged to reflect on their own biases and assumptions and to be transparent about their role in the research process.

Advantages of Qualitative Research

Qualitative research offers several advantages over other research methods, including:

  • Depth and detail: Qualitative research allows researchers to gather rich, detailed data that provides a deeper understanding of complex social phenomena. Through in-depth interviews, focus groups, and observation, researchers can gather detailed information about participants’ experiences and perspectives that may be missed by other research methods.
  • Flexibility : Qualitative research is a flexible approach that allows researchers to adapt their methods to the research question and context. Researchers can adjust their research methods in real-time to gather more information or explore unexpected findings.
  • Contextual understanding: Qualitative research is well-suited to exploring the social and cultural context in which individuals or groups are situated. Researchers can gather information about cultural norms, social structures, and historical events that may influence participants’ experiences and perspectives.
  • Participant perspective : Qualitative research prioritizes the perspective of participants, allowing researchers to explore subjective experiences and understand the meanings that participants attach to their experiences.
  • Theory development: Qualitative research can contribute to the development of new theories and insights about complex social phenomena. By gathering rich, detailed data and using inductive data analysis, researchers can develop new theories and explanations that may challenge existing understandings.
  • Validity : Qualitative research can offer high validity by using multiple data collection methods, purposive and diverse sampling, and researcher reflexivity. This can help ensure that findings are credible and trustworthy.

Limitations of Qualitative Research

Qualitative research also has some limitations, including:

  • Subjectivity : Qualitative research relies on the subjective interpretation of researchers, which can introduce bias into the research process. The researcher’s perspective, beliefs, and experiences can influence the way data is collected, analyzed, and interpreted.
  • Limited generalizability: Qualitative research typically involves small, purposive samples that may not be representative of larger populations. This limits the generalizability of findings to other contexts or populations.
  • Time-consuming: Qualitative research can be a time-consuming process, requiring significant resources for data collection, analysis, and interpretation.
  • Resource-intensive: Qualitative research may require more resources than other research methods, including specialized training for researchers, specialized software for data analysis, and transcription services.
  • Limited reliability: Qualitative research may be less reliable than quantitative research, as it relies on the subjective interpretation of researchers. This can make it difficult to replicate findings or compare results across different studies.
  • Ethics and confidentiality: Qualitative research involves collecting sensitive information from participants, which raises ethical concerns about confidentiality and informed consent. Researchers must take care to protect the privacy and confidentiality of participants and obtain informed consent.

Also see Research Methods

About the author

' src=

Muhammad Hassan

Researcher, Academic Writer, Web developer

You may also like

Quasi-Experimental Design

Quasi-Experimental Research Design – Types...

Descriptive Research Design

Descriptive Research Design – Types, Methods and...

Survey Research

Survey Research – Types, Methods, Examples

Exploratory Research

Exploratory Research – Types, Methods and...

Qualitative Research Methods

Qualitative Research Methods

Case Study Research

Case Study – Methods, Examples and Guide

Root out friction in every digital experience, super-charge conversion rates, and optimize digital self-service

Uncover insights from any interaction, deliver AI-powered agent coaching, and reduce cost to serve

Increase revenue and loyalty with real-time insights and recommendations delivered to teams on the ground

Know how your people feel and empower managers to improve employee engagement, productivity, and retention

Take action in the moments that matter most along the employee journey and drive bottom line growth

Whatever they’re are saying, wherever they’re saying it, know exactly what’s going on with your people

Get faster, richer insights with qual and quant tools that make powerful market research available to everyone

Run concept tests, pricing studies, prototyping + more with fast, powerful studies designed by UX research experts

Track your brand performance 24/7 and act quickly to respond to opportunities and challenges in your market

Explore the platform powering Experience Management

  • Free Account
  • Product Demos
  • For Digital
  • For Customer Care
  • For Human Resources
  • For Researchers
  • Financial Services
  • All Industries

Popular Use Cases

  • Customer Experience
  • Employee Experience
  • Net Promoter Score
  • Voice of Customer
  • Customer Success Hub
  • Product Documentation
  • Training & Certification
  • XM Institute
  • Popular Resources
  • Customer Stories
  • Artificial Intelligence

Market Research

  • Partnerships
  • Marketplace

The annual gathering of the experience leaders at the world’s iconic brands building breakthrough business results, live in Salt Lake City.

  • English/AU & NZ
  • Español/Europa
  • Español/América Latina
  • Português Brasileiro
  • REQUEST DEMO
  • Experience Management
  • Qualitative Research

Try Qualtrics for free

Your ultimate guide to qualitative research (with methods and examples).

16 min read You may be already using qualitative research and want to check your understanding, or you may be starting from the beginning. Learn about qualitative research methods and how you can best use them for maximum effect.

What is qualitative research?

Qualitative research is a research method that collects non-numerical data. Typically, it goes beyond the information that quantitative research provides (which we will cover below) because it is used to gain an understanding of underlying reasons, opinions, and motivations.

Qualitative research methods focus on the thoughts, feelings, reasons, motivations, and values of a participant, to understand why people act in the way they do .

In this way, qualitative research can be described as naturalistic research, looking at naturally-occurring social events within natural settings. So, qualitative researchers would describe their part in social research as the ‘vehicle’ for collecting the qualitative research data.

Qualitative researchers discovered this by looking at primary and secondary sources where data is represented in non-numerical form. This can include collecting qualitative research data types like quotes, symbols, images, and written testimonials.

These data types tell qualitative researchers subjective information. While these aren’t facts in themselves, conclusions can be interpreted out of qualitative that can help to provide valuable context.

Because of this, qualitative research is typically viewed as explanatory in nature and is often used in social research, as this gives a window into the behavior and actions of people.

It can be a good research approach for health services research or clinical research projects.

Free eBook: The qualitative research design handbook

Quantitative vs qualitative research

In order to compare qualitative and quantitative research methods, let’s explore what quantitative research is first, before exploring how it differs from qualitative research.

Quantitative research

Quantitative research is the research method of collecting quantitative research data – data that can be converted into numbers or numerical data, which can be easily quantified, compared, and analyzed .

Quantitative research methods deal with primary and secondary sources where data is represented in numerical form. This can include closed-question poll results, statistics, and census information or demographic data.

Quantitative research data tends to be used when researchers are interested in understanding a particular moment in time and examining data sets over time to find trends and patterns.

The difference between quantitative and qualitative research methodology

While qualitative research is defined as data that supplies non-numerical information, quantitative research focuses on numerical data.

In general, if you’re interested in measuring something or testing a hypothesis, use quantitative research methods. If you want to explore ideas, thoughts, and meanings, use qualitative research methods.

While qualitative research helps you to properly define, promote and sell your products, don’t rely on qualitative research methods alone because qualitative findings can’t always be reliably repeated. Qualitative research is directional, not empirical.

The best statistical analysis research uses a combination of empirical data and human experience ( quantitative research and qualitative research ) to tell the story and gain better and deeper insights, quickly.

Where both qualitative and quantitative methods are not used, qualitative researchers will find that using one without the other leaves you with missing answers.

For example, if a retail company wants to understand whether a new product line of shoes will perform well in the target market:

  • Qualitative research methods could be used with a sample of target customers, which would provide subjective reasons why they’d be likely to purchase or not purchase the shoes, while
  • Quantitative research methods into the historical customer sales information on shoe-related products would provide insights into the sales performance, and likely future performance of the new product range.

Approaches to qualitative research

There are five approaches to qualitative research methods:

  • Grounded theory: Grounded theory relates to where qualitative researchers come to a stronger hypothesis through induction, all throughout the process of collecting qualitative research data and forming connections. After an initial question to get started, qualitative researchers delve into information that is grouped into ideas or codes, which grow and develop into larger categories, as the qualitative research goes on. At the end of the qualitative research, the researcher may have a completely different hypothesis, based on evidence and inquiry, as well as the initial question.
  • Ethnographic research : Ethnographic research is where researchers embed themselves into the environment of the participant or group in order to understand the culture and context of activities and behavior. This is dependent on the involvement of the researcher, and can be subject to researcher interpretation bias and participant observer bias . However, it remains a great way to allow researchers to experience a different ‘world’.
  • Action research: With the action research process, both researchers and participants work together to make a change. This can be through taking action, researching and reflecting on the outcomes. Through collaboration, the collective comes to a result, though the way both groups interact and how they affect each other gives insights into their critical thinking skills.
  • Phenomenological research: Researchers seek to understand the meaning of an event or behavior phenomenon by describing and interpreting participant’s life experiences. This qualitative research process understands that people create their own structured reality (‘the social construction of reality’), based on their past experiences. So, by viewing the way people intentionally live their lives, we’re able to see the experiential meaning behind why they live as they do.
  • Narrative research: Narrative research, or narrative inquiry, is where researchers examine the way stories are told by participants, and how they explain their experiences, as a way of explaining the meaning behind their life choices and events. This qualitative research can arise from using journals, conversational stories, autobiographies or letters, as a few narrative research examples. The narrative is subjective to the participant, so we’re able to understand their views from what they’ve documented/spoken.

Web Graph of Qualitative Research

Qualitative research methods can use structured research instruments for data collection, like:

Surveys for individual views

A survey is a simple-to-create and easy-to-distribute qualitative research method, which helps gather information from large groups of participants quickly. Traditionally, paper-based surveys can now be made online, so costs can stay quite low.

Qualitative research questions tend to be open questions that ask for more information and provide a text box to allow for unconstrained comments.

Examples include:

  • Asking participants to keep a written or a video diary for a period of time to document their feelings and thoughts
  • In-Home-Usage tests: Buyers use your product for a period of time and report their experience

Surveys for group consensus (Delphi survey)

A Delphi survey may be used as a way to bring together participants and gain a consensus view over several rounds of questions. It differs from traditional surveys where results go to the researcher only. Instead, results go to participants as well, so they can reflect and consider all responses before another round of questions are submitted.

This can be useful to do as it can help researchers see what variance is among the group of participants and see the process of how consensus was reached.

  • Asking participants to act as a fake jury for a trial and revealing parts of the case over several rounds to see how opinions change. At the end, the fake jury must make a unanimous decision about the defendant on trial.
  • Asking participants to comment on the versions of a product being developed , as the changes are made and their feedback is taken onboard. At the end, participants must decide whether the product is ready to launch .

Semi-structured interviews

Interviews are a great way to connect with participants, though they require time from the research team to set up and conduct, especially if they’re done face-to-face.

Researchers may also have issues connecting with participants in different geographical regions. The researcher uses a set of predefined open-ended questions, though more ad-hoc questions can be asked depending on participant answers.

  • Conducting a phone interview with participants to run through their feedback on a product . During the conversation, researchers can go ‘off-script’ and ask more probing questions for clarification or build on the insights.

Focus groups

Participants are brought together into a group, where a particular topic is discussed. It is researcher-led and usually occurs in-person in a mutually accessible location, to allow for easy communication between participants in focus groups.

In focus groups , the researcher uses a set of predefined open-ended questions, though more ad-hoc questions can be asked depending on participant answers.

  • Asking participants to do UX tests, which are interface usability tests to show how easily users can complete certain tasks

Direct observation

This is a form of ethnographic research where researchers will observe participants’ behavior in a naturalistic environment. This can be great for understanding the actions in the culture and context of a participant’s setting.

This qualitative research method is prone to researcher bias as it is the researcher that must interpret the actions and reactions of participants. Their findings can be impacted by their own beliefs, values, and inferences.

  • Embedding yourself in the location of your buyers to understand how a product would perform against the values and norms of that society

Qualitative data types and category types

Qualitative research methods often deliver information in the following qualitative research data types:

  • Written testimonials

Through contextual analysis of the information, researchers can assign participants to category types:

  • Social class
  • Political alignment
  • Most likely to purchase a product
  • Their preferred training learning style

Advantages of qualitative research

  • Useful for complex situations: Qualitative research on its own is great when dealing with complex issues, however, providing background context using quantitative facts can give a richer and wider understanding of a topic. In these cases, quantitative research may not be enough.
  • A window into the ‘why’: Qualitative research can give you a window into the deeper meaning behind a participant’s answer. It can help you uncover the larger ‘why’ that can’t always be seen by analyzing numerical data.
  • Can help improve customer experiences: In service industries where customers are crucial, like in private health services, gaining information about a customer’s experience through health research studies can indicate areas where services can be improved.

Disadvantages of qualitative research

  • You need to ask the right question: Doing qualitative research may require you to consider what the right question is to uncover the underlying thinking behind a behavior. This may need probing questions to go further, which may suit a focus group or face-to-face interview setting better.
  • Results are interpreted: As qualitative research data is written, spoken, and often nuanced, interpreting the data results can be difficult as they come in non-numerical formats. This might make it harder to know if you can accept or reject your hypothesis.
  • More bias: There are lower levels of control to qualitative research methods, as they can be subject to biases like confirmation bias, researcher bias, and observation bias. This can have a knock-on effect on the validity and truthfulness of the qualitative research data results.

How to use qualitative research to your business’s advantage?

Qualitative methods help improve your products and marketing in many different ways:

  • Understand the emotional connections to your brand
  • Identify obstacles to purchase
  • Uncover doubts and confusion about your messaging
  • Find missing product features
  • Improve the usability of your website, app, or chatbot experience
  • Learn about how consumers talk about your product
  • See how buyers compare your brand to others in the competitive set
  • Learn how an organization’s employees evaluate and select vendors

6 steps to conducting good qualitative research

Businesses can benefit from qualitative research by using it to understand the meaning behind data types. There are several steps to this:

  • Define your problem or interest area: What do you observe is happening and is it frequent? Identify the data type/s you’re observing.
  • Create a hypothesis: Ask yourself what could be the causes for the situation with those qualitative research data types.
  • Plan your qualitative research: Use structured qualitative research instruments like surveys, focus groups, or interviews to ask questions that test your hypothesis.
  • Data Collection: Collect qualitative research data and understand what your data types are telling you. Once data is collected on different types over long time periods, you can analyze it and give insights into changing attitudes and language patterns.
  • Data analysis: Does your information support your hypothesis? (You may need to redo the qualitative research with other variables to see if the results improve)
  • Effectively present the qualitative research data: Communicate the results in a clear and concise way to help other people understand the findings.

Qualitative data analysis

Evaluating qualitative research can be tough when there are several analytics platforms to manage and lots of subjective data sources to compare.

Qualtrics provides a number of qualitative research analysis tools, like Text iQ , powered by Qualtrics iQ, provides powerful machine learning and native language processing to help you discover patterns and trends in text.

This also provides you with:

  • Sentiment analysis — a technique to help identify the underlying sentiment (say positive, neutral, and/or negative) in qualitative research text responses
  • Topic detection/categorisation — this technique is the grouping or bucketing of similar themes that can are relevant for the business & the industry (eg. ‘Food quality’, ‘Staff efficiency’ or ‘Product availability’)

How Qualtrics products can enhance & simplify the qualitative research process

Even in today’s data-obsessed marketplace, qualitative data is valuable – maybe even more so because it helps you establish an authentic human connection to your customers. If qualitative research doesn’t play a role to inform your product and marketing strategy, your decisions aren’t as effective as they could be.

The Qualtrics XM system gives you an all-in-one, integrated solution to help you all the way through conducting qualitative research. From survey creation and data collection to textual analysis and data reporting, it can help all your internal teams gain insights from your subjective and categorical data.

Qualitative methods are catered through templates or advanced survey designs. While you can manually collect data and conduct data analysis in a spreadsheet program, this solution helps you automate the process of qualitative research, saving you time and administration work.

Using computational techniques helps you to avoid human errors, and participant results come in are already incorporated into the analysis in real-time.

Our key tools, Text IQ™ and Driver IQ™ make analyzing subjective and categorical data easy and simple. Choose to highlight key findings based on topic, sentiment, or frequency. The choice is yours.

Qualitative research Qualtrics products

Some examples of your workspace in action, using drag and drop to create fast data visualizations quickly:

Qualitative research Qualtrics products

Related resources

Market intelligence 10 min read, marketing insights 11 min read, ethnographic research 11 min read, qualitative vs quantitative research 13 min read, qualitative research questions 11 min read, qualitative research design 12 min read, primary vs secondary research 14 min read, request demo.

Ready to learn more about Qualtrics?

UpMetrics Blog

Read the latest expert insights, trends, and best practices around impact measurement and leveraging actionable data to drive meaningful change.

8 Essential Qualitative Data Collection Methods

Qualitative data methods allow you to deep dive into the mindset of your audience to discover areas for growth, development, and improvement. 

British mathematician and marketing mastermind Clive Humby once famously stated that “Data is the new oil.”  He has a point. Without data, nonprofit organizations are left second-guessing what their clients and supporters think, how their brand compares to others in the market, whether their messaging is on-point, how their campaigns are performing, where improvements can be made, and how overall results can be optimized. 

There are two primary data collection methodologies: qualitative data collection and quantitative data collection. At UpMetrics, we believe that relying on quantitative, static data is no longer an option to drive effective impact. In the nonprofit sector, where financial gain is not the sole purpose of your organization’s existence. In this guide, we’ll focus on qualitative data collection methods and how they can help you gather, analyze, and collate information that can help drive your organization forward. 

What is Qualitative Data? 

Data collection in qualitative research focuses on gathering contextual information. Unlike quantitative data, which focuses primarily on numbers to establish ‘how many’ or ‘how much,’ qualitative data collection tools allow you to assess the ‘why’s’ and ‘how’s’ behind those statistics. This is vital for nonprofits as it enables organizations to determine:

  • Existing knowledge surrounding a particular issue.
  • How social norms and cultural practices impact a cause.
  • What kind of experiences and interactions people have with your brand.
  • Trends in the way people change their opinions.
  • Whether meaningful relationships are being established between all parties.

In short, qualitative data collection methods collect perceptual and descriptive information that helps you understand the reasoning and motivation behind particular reactions and behaviors. For that reason, qualitative data methods are usually non-numerical and center around spoken and written words rather than data extrapolated from a spreadsheet or report. 

Qualitative vs. Quantitative Data 

Quantitative and qualitative data represent both sides of the same coin. There will always be some degree of debate over the importance of quantitative vs. qualitative research, data, and collection. However, successful organizations should strive to achieve a balance between the two. 

Organizations can track their performance by collecting quantitative data based on metrics including dollars raised, membership growth, number of people served, overhead costs, etc. This is all essential information to have. However, the data lacks value without the additional details provided by qualitative research because it doesn’t tell you anything about how your target audience thinks, feels, and acts. 

Qualitative data collection is particularly relevant in the nonprofit sector as the relationships people have with the causes they support are fundamentally personal and cannot be expressed numerically. Qualitative data methods allow you to deep dive into the mindset of your audience to discover areas for growth, development, and improvement. 

8 Types of Qualitative Data Collection Methods  

As we have firmly established the need for qualitative data, it’s time to answer the next big question: how to collect qualitative data. 

Here is a list of the most common qualitative data collection methods. You don’t need to use them all in your quest for gathering information. However, a foundational understanding of each will help you refine your research strategy and select the methods that are likely to provide the highest quality business intelligence for your organization. 

1. Interviews

One-on-one interviews are one of the most commonly used data collection methods in qualitative research because they allow you to collect highly personalized information directly from the source. Interviews explore participants' opinions, motivations, beliefs, and experiences and are particularly beneficial in gathering data on sensitive topics because respondents are more likely to open up in a one-on-one setting than in a group environment. 

Interviews can be conducted in person or by online video call. Typically, they are separated into three main categories:

  • Structured Interviews - Structured interviews consist of predetermined (and usually closed) questions with little or no variation between interviewees. There is generally no scope for elaboration or follow-up questions, making them better suited to researching specific topics. 
  • Unstructured Interviews – Conversely, unstructured interviews have little to no organization or preconceived topics and include predominantly open questions. As a result, the discussion will flow in completely different directions for each participant and can be very time-consuming. For this reason, unstructured interviews are generally only used when little is known about the subject area or when in-depth responses are required on a particular subject.
  • Semi-Structured Interviews – A combination of the two interviews mentioned above, semi-structured interviews comprise several scripted questions but allow both interviewers and interviewees the opportunity to diverge and elaborate so more in-depth reasoning can be explored. 

While each approach has its merits, semi-structured interviews are typically favored as a way to uncover detailed information in a timely manner while highlighting areas that may not have been considered relevant in previous research efforts. Whichever type of interview you utilize, participants must be fully briefed on the format, purpose, and what you hope to achieve. With that in mind, here are a few tips to follow: 

  • Give them an idea of how long the interview will last
  • If you plan to record the conversation, ask permission beforehand
  • Provide the opportunity to ask questions before you begin and again at the end. 

2. Focus Groups

Focus groups share much in common with less structured interviews, the key difference being that the goal is to collect data from several participants simultaneously. Focus groups are effective in gathering information based on collective views and are one of the most popular data collection instruments in qualitative research when a series of one-on-one interviews proves too time-consuming or difficult to schedule. 

Focus groups are most helpful in gathering data from a specific group of people, such as donors or clients from a particular demographic. The discussion should be focused on a specific topic and carefully guided and moderated by the researcher to determine participant views and the reasoning behind them. 

Feedback in a group setting often provides richer data than one-on-one interviews, as participants are generally more open to sharing when others are sharing too. Plus, input from one participant may spark insight from another that would not have come to light otherwise. However, here are a couple of potential downsides:

  • If participants are uneasy with each other, they may not be at ease openly discussing their feelings or opinions.
  • If the topic is not of interest or does not focus on something participants are willing to discuss, data will lack value. 

The size of the group should be carefully considered. Research suggests over-recruiting to avoid risking cancellation, even if that means moderators have to manage more participants than anticipated. The optimum group size is generally between six and eight for all participants to be granted ample opportunity to speak. However, focus groups can still be successful with as few as three or as many as fourteen participants. 

3. Observation

Observation is one of the ultimate data collection tools in qualitative research for gathering information through subjective methods. A technique used frequently by modern-day marketers, qualitative observation is also favored by psychologists, sociologists, behavior specialists, and product developers. 

The primary purpose is to gather information that cannot be measured or easily quantified. It involves virtually no cognitive input from the participants themselves. Researchers simply observe subjects and their reactions during the course of their regular routines and take detailed field notes from which to draw information. 

Observational techniques vary in terms of contact with participants. Some qualitative observations involve the complete immersion of the researcher over a period of time. For example, attending the same church, clinic, society meetings, or volunteer organizations as the participants. Under these circumstances, researchers will likely witness the most natural responses rather than relying on behaviors elicited in a simulated environment. Depending on the study and intended purpose, they may or may not choose to identify themselves as a researcher during the process. 

Regardless of whether you take a covert or overt approach, remember that because each researcher is as unique as every participant, they will have their own inherent biases. Therefore, observational studies are prone to a high degree of subjectivity. For example, one researcher’s notes on the behavior of donors at a society event may vary wildly from the next. So, each qualitative observational study is unique in its own right. 

4. Open-Ended Surveys and Questionnaires

Open-ended surveys and questionnaires allow organizations to collect views and opinions from respondents without meeting in person. They can be sent electronically and are considered one of the most cost-effective qualitative data collection tools. Unlike closed question surveys and questionnaires that limit responses, open-ended questions allow participants to provide lengthy and in-depth answers from which you can extrapolate large amounts of data. 

The findings of open-ended surveys and questionnaires can be challenging to analyze because there are no uniform answers. A popular approach is to record sentiments as positive, negative, and neutral and further dissect the data from there. To gather the best business intelligence, carefully consider the presentation and length of your survey or questionnaire. Here is a list of essential considerations:

  • Number of questions : Too many can feel intimidating, and you’ll experience low response rates. Too few can feel like it’s not worth the effort. Plus, the data you collect will have limited actionability. The consensus on how many questions to include varies depending on which sources you consult. However, 5-10 is a good benchmark for shorter surveys that take around 10 minutes and 15-20 for longer surveys that take approximately 20 minutes to complete. 
  • Personalization: Your response rate will be higher if you greet patients by name and demonstrate a historical knowledge of their interactions with your brand. 
  • Visual elements : Recipients can be easily turned off by poorly designed questionnaires. Besides, it’s a good idea to customize your survey template to include brand assets like colors, logos, and fonts to increase brand loyalty and recognition.
  • Reminders : Sending survey reminders is the best way to improve your response rate. You don’t want to hassle respondents too soon, nor do you want to wait too long. Sending a follow-up at around the 3-7 mark is usually the most effective. 
  • Building a feedback loop : Adding a tick-box requesting permission for further follow-ups is a proven way to elicit more in-depth feedback. Plus, it gives respondents a voice and makes their opinion feel valued.

5. Case Studies

Case studies are often a preferred method of qualitative research data collection for organizations looking to generate incredibly detailed and in-depth information on a specific topic. Case studies are usually a deep dive into one specific case or a small number of related cases. As a result, they work well for organizations that operate in niche markets.

Case studies typically involve several qualitative data collection methods, including interviews, focus groups, surveys, and observation. The idea is to cast a wide net to obtain a rich picture comprising multiple views and responses. When conducted correctly, case studies can generate vast bodies of data that can be used to improve processes at every client and donor touchpoint. 

The best way to demonstrate the purpose and value of a case study is with an example: A Longitudinal Qualitative Case Study of Change in Nonprofits – Suggesting A New Approach to the Management of Change . 

The researchers established that while change management had already been widely researched in commercial and for-profit settings, little reference had been made to the unique challenges in the nonprofit sector. The case study examined change and change management at a single nonprofit hospital from the viewpoint of all those who witnessed and experienced it. To gain a holistic view of the entire process, research included interviews with employees at every level, from nursing staff to CEOs, to identify the direct and indirect impacts of change. Results were collated based on detailed responses to questions about preparing for change, experiencing change, and reflecting on change.

6. Text Analysis

Text analysis has long been used in political and social science spheres to gain a deeper understanding of behaviors and motivations by gathering insights from human-written texts. By analyzing the flow of text and word choices, relationships between other texts written by the same participant can be identified so that researchers can draw conclusions about the mindset of their target audience. Though technically a qualitative data collection method, the process can involve some quantitative elements, as often, computer systems are used to scan, extract, and categorize information to identify patterns, sentiments, and other actionable information. 

You might be wondering how to collect written information from your research subjects. There are many different options, and approaches can be overt or covert. 

Examples include:

  • Investigating how often certain cause-related words and phrases are used in client and donor social media posts.
  • Asking participants to keep a journal or diary.
  • Analyzing existing interview transcripts and survey responses.

By conducting a detailed analysis, you can connect elements of written text to specific issues, causes, and cultural perspectives, allowing you to draw empirical conclusions about personal views, behaviors, and social relations. With small studies focusing on participants' subjective experience on a specific theme or topic, diaries and journals can be particularly effective in building an understanding of underlying thought processes and beliefs. 

7. Audio and Video Recordings

Similarly to how data is collected from a person’s writing, you can draw valuable conclusions by observing someone’s speech patterns, intonation, and body language when you watch or listen to them interact in a particular environment or within specific surroundings. 

Video and audio recordings are helpful in circumstances where researchers predict better results by having participants be in the moment rather than having them think about what to write down or how to formulate an answer to an email survey. 

You can collect audio and video materials for analysis from multiple sources, including:

  • Previously filmed records of events
  • Interview recordings
  • Video diaries

Utilizing audio and video footage allows researchers to revisit key themes, and it's possible to use the same analytical sources in multiple studies – providing that the scope of the original recording is comprehensive enough to cover the intended theme in adequate depth. 

It can be challenging to present the results of audio and video analysis in a quantifiable form that helps you gauge campaign and market performance. However, results can be used to effectively design concept maps that extrapolate central themes that arise consistently. Concept Mapping offers organizations a visual representation of thought patterns and how ideas link together between different demographics. This data can prove invaluable in identifying areas for improvement and change across entire projects and organizational processes. 

8. Hybrid Methodologies

It is often possible to utilize data collection methods in qualitative research that provide quantitative facts and figures. So if you’re struggling to settle on an approach, a hybrid methodology may be a good starting point. For instance, a survey format that asks closed and open questions can collect and collate quantitative and qualitative data. 

A Net Promoter Score (NPS) survey is a great example. The primary goal of an NPS survey is to collect quantitative ratings of various factors on a score of 1-10. However, they also utilize open-ended follow-up questions to collect qualitative data that helps identify insights into the trends, thought processes, reasoning, and behaviors behind the initial scoring. 

Collect and Collate Actionable Data with UpMetrics

Most nonprofits believe data is strategically important. It has been statistically proven that organizations with advanced data insights achieve their missions more efficiently. Yet, studies show that despite 90% of organizations collecting data, only 5% believe internal decision-making is data-driven. At UpMetrics, we’re here to help you change that. 

UpMetrics specializes in bringing technology and humanity together to serve social good. Our unique  social impact software  combines quantitative and qualitative data collection methods and analysis techniques, enabling social impact organizations to gain insights, drive action, and inspire change. By reporting and analyzing quantitative and qualitative data in one intuitive platform, your impact organization gains the understanding it needs to identify the drivers of positive outcomes, achieve transparency, and increase knowledge sharing across stakeholders.

Contact us today  to learn more about our  nonprofit impact measurement  solutions and discover the power of a partnership with UpMetrics. 

Kauffman Foundation Case Study Preview

Disrupting Tradition

Download our case study to discover how the Ewing Marion Kauffman Foundation is using a Learning Mindset to Support Grantees in Measuring Impact

research instrument used in qualitative

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • View all journals
  • Explore content
  • About the journal
  • Publish with us
  • Sign up for alerts
  • Published: 22 March 2008

Methods of data collection in qualitative research: interviews and focus groups

  • P. Gill 1 ,
  • K. Stewart 2 ,
  • E. Treasure 3 &
  • B. Chadwick 4  

British Dental Journal volume  204 ,  pages 291–295 ( 2008 ) Cite this article

1.70m Accesses

1044 Citations

43 Altmetric

Metrics details

Interviews and focus groups are the most common methods of data collection used in qualitative healthcare research

Interviews can be used to explore the views, experiences, beliefs and motivations of individual participants

Focus group use group dynamics to generate qualitative data

Qualitative research in dentistry

Conducting qualitative interviews with school children in dental research

Analysing and presenting qualitative data

This paper explores the most common methods of data collection used in qualitative research: interviews and focus groups. The paper examines each method in detail, focusing on how they work in practice, when their use is appropriate and what they can offer dentistry. Examples of empirical studies that have used interviews or focus groups are also provided.

You have full access to this article via your institution.

Similar content being viewed by others

research instrument used in qualitative

Interviews in the social sciences

research instrument used in qualitative

A review of technical and quality assessment considerations of audio-visual and web-conferencing focus groups in qualitative health research

Professionalism in dentistry: deconstructing common terminology, introduction.

Having explored the nature and purpose of qualitative research in the previous paper, this paper explores methods of data collection used in qualitative research. There are a variety of methods of data collection in qualitative research, including observations, textual or visual analysis (eg from books or videos) and interviews (individual or group). 1 However, the most common methods used, particularly in healthcare research, are interviews and focus groups. 2 , 3

The purpose of this paper is to explore these two methods in more detail, in particular how they work in practice, the purpose of each, when their use is appropriate and what they can offer dental research.

Qualitative research interviews

There are three fundamental types of research interviews: structured, semi-structured and unstructured. Structured interviews are, essentially, verbally administered questionnaires, in which a list of predetermined questions are asked, with little or no variation and with no scope for follow-up questions to responses that warrant further elaboration. Consequently, they are relatively quick and easy to administer and may be of particular use if clarification of certain questions are required or if there are likely to be literacy or numeracy problems with the respondents. However, by their very nature, they only allow for limited participant responses and are, therefore, of little use if 'depth' is required.

Conversely, unstructured interviews do not reflect any preconceived theories or ideas and are performed with little or no organisation. 4 Such an interview may simply start with an opening question such as 'Can you tell me about your experience of visiting the dentist?' and will then progress based, primarily, upon the initial response. Unstructured interviews are usually very time-consuming (often lasting several hours) and can be difficult to manage, and to participate in, as the lack of predetermined interview questions provides little guidance on what to talk about (which many participants find confusing and unhelpful). Their use is, therefore, generally only considered where significant 'depth' is required, or where virtually nothing is known about the subject area (or a different perspective of a known subject area is required).

Semi-structured interviews consist of several key questions that help to define the areas to be explored, but also allows the interviewer or interviewee to diverge in order to pursue an idea or response in more detail. 2 This interview format is used most frequently in healthcare, as it provides participants with some guidance on what to talk about, which many find helpful. The flexibility of this approach, particularly compared to structured interviews, also allows for the discovery or elaboration of information that is important to participants but may not have previously been thought of as pertinent by the research team.

For example, in a recent dental public heath study, 5 school children in Cardiff, UK were interviewed about their food choices and preferences. A key finding that emerged from semi-structured interviews, which was not previously thought to be as highly influential as the data subsequently confirmed, was the significance of peer-pressure in influencing children's food choices and preferences. This finding was also established primarily through follow-up questioning (eg probing interesting responses with follow-up questions, such as 'Can you tell me a bit more about that?') and, therefore, may not have emerged in the same way, if at all, if asked as a predetermined question.

The purpose of research interviews

The purpose of the research interview is to explore the views, experiences, beliefs and/or motivations of individuals on specific matters (eg factors that influence their attendance at the dentist). Qualitative methods, such as interviews, are believed to provide a 'deeper' understanding of social phenomena than would be obtained from purely quantitative methods, such as questionnaires. 1 Interviews are, therefore, most appropriate where little is already known about the study phenomenon or where detailed insights are required from individual participants. They are also particularly appropriate for exploring sensitive topics, where participants may not want to talk about such issues in a group environment.

Examples of dental studies that have collected data using interviews are 'Examining the psychosocial process involved in regular dental attendance' 6 and 'Exploring factors governing dentists' treatment philosophies'. 7 Gibson et al . 6 provided an improved understanding of factors that influenced people's regular attendance with their dentist. The study by Kay and Blinkhorn 7 provided a detailed insight into factors that influenced GDPs' decision making in relation to treatment choices. The study found that dentists' clinical decisions about treatments were not necessarily related to pathology or treatment options, as was perhaps initially thought, but also involved discussions with patients, patients' values and dentists' feelings of self esteem and conscience.

There are many similarities between clinical encounters and research interviews, in that both employ similar interpersonal skills, such as questioning, conversing and listening. However, there are also some fundamental differences between the two, such as the purpose of the encounter, reasons for participating, roles of the people involved and how the interview is conducted and recorded. 8

The primary purpose of clinical encounters is for the dentist to ask the patient questions in order to acquire sufficient information to inform decision making and treatment options. However, the constraints of most consultations are such that any open-ended questioning needs to be brought to a conclusion within a fairly short time. 2 In contrast, the fundamental purpose of the research interview is to listen attentively to what respondents have to say, in order to acquire more knowledge about the study topic. 9 Unlike the clinical encounter, it is not to intentionally offer any form of help or advice, which many researchers have neither the training nor the time for. Research interviewing therefore requires a different approach and a different range of skills.

The interview

When designing an interview schedule it is imperative to ask questions that are likely to yield as much information about the study phenomenon as possible and also be able to address the aims and objectives of the research. In a qualitative interview, good questions should be open-ended (ie, require more than a yes/no answer), neutral, sensitive and understandable. 2 It is usually best to start with questions that participants can answer easily and then proceed to more difficult or sensitive topics. 2 This can help put respondents at ease, build up confidence and rapport and often generates rich data that subsequently develops the interview further.

As in any research, it is often wise to first pilot the interview schedule on several respondents prior to data collection proper. 8 This allows the research team to establish if the schedule is clear, understandable and capable of answering the research questions, and if, therefore, any changes to the interview schedule are required.

The length of interviews varies depending on the topic, researcher and participant. However, on average, healthcare interviews last 20-60 minutes. Interviews can be performed on a one-off or, if change over time is of interest, repeated basis, 4 for example exploring the psychosocial impact of oral trauma on participants and their subsequent experiences of cosmetic dental surgery.

Developing the interview

Before an interview takes place, respondents should be informed about the study details and given assurance about ethical principles, such as anonymity and confidentiality. 2 This gives respondents some idea of what to expect from the interview, increases the likelihood of honesty and is also a fundamental aspect of the informed consent process.

Wherever possible, interviews should be conducted in areas free from distractions and at times and locations that are most suitable for participants. For many this may be at their own home in the evenings. Whilst researchers may have less control over the home environment, familiarity may help the respondent to relax and result in a more productive interview. 9 Establishing rapport with participants prior to the interview is also important as this can also have a positive effect on the subsequent development of the interview.

When conducting the actual interview it is prudent for the interviewer to familiarise themselves with the interview schedule, so that the process appears more natural and less rehearsed. However, to ensure that the interview is as productive as possible, researchers must possess a repertoire of skills and techniques to ensure that comprehensive and representative data are collected during the interview. 10 One of the most important skills is the ability to listen attentively to what is being said, so that participants are able to recount their experiences as fully as possible, without unnecessary interruptions.

Other important skills include adopting open and emotionally neutral body language, nodding, smiling, looking interested and making encouraging noises (eg, 'Mmmm') during the interview. 2 The strategic use of silence, if used appropriately, can also be highly effective at getting respondents to contemplate their responses, talk more, elaborate or clarify particular issues. Other techniques that can be used to develop the interview further include reflecting on remarks made by participants (eg, 'Pain?') and probing remarks ('When you said you were afraid of going to the dentist what did you mean?'). 9 Where appropriate, it is also wise to seek clarification from respondents if it is unclear what they mean. The use of 'leading' or 'loaded' questions that may unduly influence responses should always be avoided (eg, 'So you think dental surgery waiting rooms are frightening?' rather than 'How do you find the waiting room at the dentists?').

At the end of the interview it is important to thank participants for their time and ask them if there is anything they would like to add. This gives respondents an opportunity to deal with issues that they have thought about, or think are important but have not been dealt with by the interviewer. 9 This can often lead to the discovery of new, unanticipated information. Respondents should also be debriefed about the study after the interview has finished.

All interviews should be tape recorded and transcribed verbatim afterwards, as this protects against bias and provides a permanent record of what was and was not said. 8 It is often also helpful to make 'field notes' during and immediately after each interview about observations, thoughts and ideas about the interview, as this can help in data analysis process. 4 , 8

Focus groups

Focus groups share many common features with less structured interviews, but there is more to them than merely collecting similar data from many participants at once. A focus group is a group discussion on a particular topic organised for research purposes. This discussion is guided, monitored and recorded by a researcher (sometimes called a moderator or facilitator). 11 , 12

Focus groups were first used as a research method in market research, originating in the 1940s in the work of the Bureau of Applied Social Research at Columbia University. Eventually the success of focus groups as a marketing tool in the private sector resulted in its use in public sector marketing, such as the assessment of the impact of health education campaigns. 13 However, focus group techniques, as used in public and private sectors, have diverged over time. Therefore, in this paper, we seek to describe focus groups as they are used in academic research.

When focus groups are used

Focus groups are used for generating information on collective views, and the meanings that lie behind those views. They are also useful in generating a rich understanding of participants' experiences and beliefs. 12 Suggested criteria for using focus groups include: 13

As a standalone method, for research relating to group norms, meanings and processes

In a multi-method design, to explore a topic or collect group language or narratives to be used in later stages

To clarify, extend, qualify or challenge data collected through other methods

To feedback results to research participants.

Morgan 12 suggests that focus groups should be avoided according to the following criteria:

If listening to participants' views generates expectations for the outcome of the research that can not be fulfilled

If participants are uneasy with each other, and will therefore not discuss their feelings and opinions openly

If the topic of interest to the researcher is not a topic the participants can or wish to discuss

If statistical data is required. Focus groups give depth and insight, but cannot produce useful numerical results.

Conducting focus groups: group composition and size

The composition of a focus group needs great care to get the best quality of discussion. There is no 'best' solution to group composition, and group mix will always impact on the data, according to things such as the mix of ages, sexes and social professional statuses of the participants. What is important is that the researcher gives due consideration to the impact of group mix (eg, how the group may interact with each other) before the focus group proceeds. 14

Interaction is key to a successful focus group. Sometimes this means a pre-existing group interacts best for research purposes, and sometimes stranger groups. Pre-existing groups may be easier to recruit, have shared experiences and enjoy a comfort and familiarity which facilitates discussion or the ability to challenge each other comfortably. In health settings, pre-existing groups can overcome issues relating to disclosure of potentially stigmatising status which people may find uncomfortable in stranger groups (conversely there may be situations where disclosure is more comfortable in stranger groups). In other research projects it may be decided that stranger groups will be able to speak more freely without fear of repercussion, and challenges to other participants may be more challenging and probing, leading to richer data. 13

Group size is an important consideration in focus group research. Stewart and Shamdasani 14 suggest that it is better to slightly over-recruit for a focus group and potentially manage a slightly larger group, than under-recruit and risk having to cancel the session or having an unsatisfactory discussion. They advise that each group will probably have two non-attenders. The optimum size for a focus group is six to eight participants (excluding researchers), but focus groups can work successfully with as few as three and as many as 14 participants. Small groups risk limited discussion occurring, while large groups can be chaotic, hard to manage for the moderator and frustrating for participants who feel they get insufficient opportunities to speak. 13

Preparing an interview schedule

Like research interviews, the interview schedule for focus groups is often no more structured than a loose schedule of topics to be discussed. However, in preparing an interview schedule for focus groups, Stewart and Shamdasani 14 suggest two general principles:

Questions should move from general to more specific questions

Question order should be relative to importance of issues in the research agenda.

There can, however, be some conflict between these two principles, and trade offs are often needed, although often discussions will take on a life of their own, which will influence or determine the order in which issues are covered. Usually, less than a dozen predetermined questions are needed and, as with research interviews, the researcher will also probe and expand on issues according to the discussion.

Moderating a focus group looks easy when done well, but requires a complex set of skills, which are related to the following principles: 15

Participants have valuable views and the ability to respond actively, positively and respectfully. Such an approach is not simply a courtesy, but will encourage fruitful discussions

Moderating without participating: a moderator must guide a discussion rather than join in with it. Expressing one's own views tends to give participants cues as to what to say (introducing bias), rather than the confidence to be open and honest about their own views

Be prepared for views that may be unpalatably critical of a topic which may be important to you

It is important to recognise that researchers' individual characteristics mean that no one person will always be suitable to moderate any kind of group. Sometimes the characteristics that suit a moderator for one group will inhibit discussion in another

Be yourself. If the moderator is comfortable and natural, participants will feel relaxed.

The moderator should facilitate group discussion, keeping it focussed without leading it. They should also be able to prevent the discussion being dominated by one member (for example, by emphasising at the outset the importance of hearing a range of views), ensure that all participants have ample opportunity to contribute, allow differences of opinions to be discussed fairly and, if required, encourage reticent participants. 13

Other relevant factors

The venue for a focus group is important and should, ideally, be accessible, comfortable, private, quiet and free from distractions. 13 However, while a central location, such as the participants' workplace or school, may encourage attendance, the venue may affect participants' behaviour. For example, in a school setting, pupils may behave like pupils, and in clinical settings, participants may be affected by any anxieties that affect them when they attend in a patient role.

Focus groups are usually recorded, often observed (by a researcher other than the moderator, whose role is to observe the interaction of the group to enhance analysis) and sometimes videotaped. At the start of a focus group, a moderator should acknowledge the presence of the audio recording equipment, assure participants of confidentiality and give people the opportunity to withdraw if they are uncomfortable with being taped. 14

A good quality multi-directional external microphone is recommended for the recording of focus groups, as internal microphones are rarely good enough to cope with the variation in volume of different speakers. 13 If observers are present, they should be introduced to participants as someone who is just there to observe, and sit away from the discussion. 14 Videotaping will require more than one camera to capture the whole group, as well as additional operational personnel in the room. This is, therefore, very obtrusive, which can affect the spontaneity of the group and in a focus group does not usually yield enough additional information that could not be captured by an observer to make videotaping worthwhile. 15

The systematic analysis of focus group transcripts is crucial. However, the transcription of focus groups is more complex and time consuming than in one-to-one interviews, and each hour of audio can take up to eight hours to transcribe and generate approximately 100 pages of text. Recordings should be transcribed verbatim and also speakers should be identified in a way that makes it possible to follow the contributions of each individual. Sometimes observational notes also need to be described in the transcripts in order for them to make sense.

The analysis of qualitative data is explored in the final paper of this series. However, it is important to note that the analysis of focus group data is different from other qualitative data because of their interactive nature, and this needs to be taken into consideration during analysis. The importance of the context of other speakers is essential to the understanding of individual contributions. 13 For example, in a group situation, participants will often challenge each other and justify their remarks because of the group setting, in a way that perhaps they would not in a one-to-one interview. The analysis of focus group data must therefore take account of the group dynamics that have generated remarks.

Focus groups in dental research

Focus groups are used increasingly in dental research, on a diverse range of topics, 16 illuminating a number of areas relating to patients, dental services and the dental profession. Addressing a special needs population difficult to access and sample through quantitative measures, Robinson et al . 17 used focus groups to investigate the oral health-related attitudes of drug users, exploring the priorities, understandings and barriers to care they encounter. Newton et al . 18 used focus groups to explore barriers to services among minority ethnic groups, highlighting for the first time differences between minority ethnic groups. Demonstrating the use of the method with professional groups as subjects in dental research, Gussy et al . 19 explored the barriers to and possible strategies for developing a shared approach in prevention of caries among pre-schoolers. This mixed method study was very important as the qualitative element was able to explain why the clinical trial failed, and this understanding may help researchers improve on the quantitative aspect of future studies, as well as making a valuable academic contribution in its own right.

Interviews and focus groups remain the most common methods of data collection in qualitative research, and are now being used with increasing frequency in dental research, particularly to access areas not amendable to quantitative methods and/or where depth, insight and understanding of particular phenomena are required. The examples of dental studies that have employed these methods also help to demonstrate the range of research contexts to which interview and focus group research can make a useful contribution. The continued employment of these methods can further strengthen many areas of dentally related work.

Silverman D . Doing qualitative research . London: Sage Publications, 2000.

Google Scholar  

Britten N . Qualitative interviews in healthcare. In Pope C, Mays N (eds) Qualitative research in health care . 2nd ed. pp 11–19. London: BMJ Books, 1999.

Legard R, Keegan J, Ward K . In-depth interviews. In Ritchie J, Lewis J (eds) Qualitative research practice: a guide for social science students and researchers . pp 139–169. London: Sage Publications, 2003.

May K M . Interview techniques in qualitative research: concerns and challenges. In Morse J M (ed) Qualitative nursing research . pp 187–201. Newbury Park: Sage Publications, 1991.

Stewart K, Gill P, Treasure E, Chadwick B . Understanding about food among 6-11 year olds in South Wales. Food Culture Society 2006; 9 : 317–333.

Article   Google Scholar  

Gibson B, Drenna J, Hanna S, Freeman R . An exploratory qualitative study examining the social and psychological processes involved in regular dental attendance. J Public Health Dent 2000; 60 : 5–11.

Kay E J, Blinkhorn A S . A qualitative investigation of factors governing dentists' treatment philosophies. Br Dent J 1996; 180 : 171–176.

Pontin D . Interviews. In Cormack D F S (ed) The research process in nursing . 4th ed. pp 289–298. Oxford: Blackwell Science, 2000.

Kvale S . Interviews . Thousand Oaks: Sage Publications, 1996.

Hammersley M, Atkinson P . Ethnography: principles in practice . 2nd ed. London: Routledge, 1995.

Kitzinger J . The methodology of focus groups: the importance of interaction between research participants. Sociol Health Illn 1994; 16 : 103–121.

Morgan D L . The focus group guide book . London: Sage Publications, 1998.

Book   Google Scholar  

Bloor M, Frankland J, Thomas M, Robson K . Focus groups in social research . London: Sage Publications, 2001.

Stewart D W, Shamdasani P M . Focus groups. Theory and practice . London: Sage Publications, 1990.

Krueger R A . Moderating focus groups . London: Sage Publications, 1998.

Chestnutt I G, Robson K F. Focus groups – what are they? Dent Update 2002; 28 : 189–192.

Robinson P G, Acquah S, Gibson B . Drug users: oral health related attitudes and behaviours. Br Dent J 2005; 198 : 219–224.

Newton J T, Thorogood N, Bhavnani V, Pitt J, Gibbons D E, Gelbier S . Barriers to the use of dental services by individuals from minority ethnic communities living in the United Kingdom: findings from focus groups. Primary Dent Care 2001; 8 : 157–161.

Gussy M G, Waters E, Kilpatrick M . A qualitative study exploring barriers to a model of shared care for pre-school children's oral health. Br Dent J 2006; 201 : 165–170.

Download references

Author information

Authors and affiliations.

Senior Research Fellow, Faculty of Health, Sport and Science, University of Glamorgan, Pontypridd, CF37 1DL,

Research Fellow, Academic Unit of Primary Care, University of Bristol, Bristol, BS8 2AA,

Dean and Professor of Dental Public Health, School of Dentistry, Dental Health and Biological Sciences, School of Dentistry, Cardiff University, Heath Park, Cardiff, CF14 4XY,

E. Treasure

Professor of Paediatric Dentistry, School of Dentistry, Dental Health and Biological Sciences, School of Dentistry, Cardiff University, Heath Park, Cardiff, CF14 4XY,

B. Chadwick

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to P. Gill .

Additional information

Refereed paper

Rights and permissions

Reprints and permissions

About this article

Cite this article.

Gill, P., Stewart, K., Treasure, E. et al. Methods of data collection in qualitative research: interviews and focus groups. Br Dent J 204 , 291–295 (2008). https://doi.org/10.1038/bdj.2008.192

Download citation

Published : 22 March 2008

Issue Date : 22 March 2008

DOI : https://doi.org/10.1038/bdj.2008.192

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

This article is cited by

Imprisonment for south ethiopian people living with hiv presents a double health burden: lived experiences of prisoners.

  • Terefe Gone Fuge
  • George Tsourtos
  • Emma R Miller

BMC Health Services Research (2024)

Employees’ experiences of a large-scale implementation in a public care setting: a novel mixed-method approach to content analysis

  • My Säfström
  • Ulrika Löfkvist

Translating brand reputation into equity from the stakeholder’s theory: an approach to value creation based on consumer’s perception & interactions

  • Olukorede Adewole

International Journal of Corporate Social Responsibility (2024)

Nurses’ and patients’ perceptions of physical health screening for patients with schizophrenia spectrum disorders: a qualitative study

  • Camilla Långstedt
  • Daniel Bressington
  • Maritta Välimäki

BMC Nursing (2024)

Heteronormative biases and distinctive experiences with prostate cancer among men who have sex with men: a qualitative focus group study

  • Evan Panken
  • Noah Frydenlund
  • Channa Amarasekera

BMC Urology (2024)

Quick links

  • Explore articles by subject
  • Guide to authors
  • Editorial policies

research instrument used in qualitative

  • Skip to main content
  • Skip to primary sidebar
  • Skip to footer
  • QuestionPro

survey software icon

  • Solutions Industries Gaming Automotive Sports and events Education Government Travel & Hospitality Financial Services Healthcare Cannabis Technology Use Case NPS+ Communities Audience Contactless surveys Mobile LivePolls Member Experience GDPR Positive People Science 360 Feedback Surveys
  • Resources Blog eBooks Survey Templates Case Studies Training Help center

research instrument used in qualitative

Home Market Research

Qualitative Data Collection: What it is + Methods to do it

qualitative-data-collection

Qualitative data collection is vital in qualitative research. It helps researchers understand individuals’ attitudes, beliefs, and behaviors in a specific context.

Several methods are used to collect qualitative data, including interviews, surveys, focus groups, and observations. Understanding the various methods used for gathering qualitative data is essential for successful qualitative research.

In this post, we will discuss qualitative data and its collection methods of it.

Content Index

What is Qualitative Data?

What is qualitative data collection, what is the need for qualitative data collection, effective qualitative data collection methods, qualitative data analysis, advantages of qualitative data collection.

Qualitative data is defined as data that approximates and characterizes. It can be observed and recorded.

This data type is non-numerical in nature. This type of data is collected through methods of observations, one-to-one interviews, conducting focus groups, and similar methods.

Qualitative data in statistics is also known as categorical data – data that can be arranged categorically based on the attributes and properties of a thing or a phenomenon.

It’s pretty easy to understand the difference between qualitative and quantitative data. Qualitative data does not include numbers in its definition of traits, whereas quantitative research data is all about numbers.

  • The cake is orange, blue, and black in color (qualitative).
  • Females have brown, black, blonde, and red hair (qualitative).

Qualitative data collection is gathering non-numerical information, such as words, images, and observations, to understand individuals’ attitudes, behaviors, beliefs, and motivations in a specific context. It is an approach used in qualitative research. It seeks to understand social phenomena through in-depth exploration and analysis of people’s perspectives, experiences, and narratives. In statistical analysis , distinguishing between categorical data and numerical data is essential, as categorical data involves distinct categories or labels, while numerical data consists of measurable quantities.

The data collected through qualitative methods are often subjective, open-ended, and unstructured and can provide a rich and nuanced understanding of complex social phenomena.

Qualitative research is a type of study carried out with a qualitative approach to understand the exploratory reasons and to assay how and why a specific program or phenomenon operates in the way it is working. A researcher can access numerous qualitative data collection methods that he/she feels are relevant.

LEARN ABOUT: Best Data Collection Tools

Qualitative data collection methods serve the primary purpose of collecting textual data for research and analysis , like the thematic analysis. The collected research data is used to examine:

  • Knowledge around a specific issue or a program, experience of people.
  • Meaning and relationships.
  • Social norms and contextual or cultural practices demean people or impact a cause.

The qualitative data is textual or non-numerical. It covers mostly the images, videos, texts, and written or spoken words by the people. You can opt for any digital data collection methods , like structured or semi-structured surveys, or settle for the traditional approach comprising individual interviews, group discussions, etc.

Data at hand leads to a smooth process ensuring all the decisions made are for the business’s betterment. You will be able to make informed decisions only if you have relevant data.

Well! With quality data, you will improve the quality of decision-making. But you will also enhance the quality of the results expected from any endeavor.

Qualitative data collection methods are exploratory. Those are usually more focused on gaining insights and understanding the underlying reasons by digging deeper.

Although quantitative data cannot be quantified, measuring it or analyzing qualitative data might become an issue. Due to the lack of measurability, collection methods of qualitative data are primarily unstructured or structured in rare cases – that too to some extent.

Let’s explore the most common methods used for the collection of qualitative data:

research instrument used in qualitative

Individual interview

It is one of the most trusted, widely used, and familiar qualitative data collection methods primarily because of its approach. An individual or face-to-face interview is a direct conversation between two people with a specific structure and purpose.

The interview questionnaire is designed in the manner to elicit the interviewee’s knowledge or perspective related to a topic, program, or issue.

At times, depending on the interviewer’s approach, the conversation can be unstructured or informal but focused on understanding the individual’s beliefs, values, understandings, feelings, experiences, and perspectives on an issue.

More often, the interviewer chooses to ask open-ended questions in individual interviews. If the interviewee selects answers from a set of given options, it becomes a structured, fixed response or a biased discussion.

The individual interview is an ideal qualitative data collection method. Particularly when the researchers want highly personalized information from the participants. The individual interview is a notable method if the interviewer decides to probe further and ask follow-up questions to gain more insights.

Qualitative surveys

To develop an informed hypothesis, many researchers use qualitative research surveys for data collection or to collect a piece of detailed information about a product or an issue. If you want to create questionnaires for collecting textual or qualitative data, then ask more open-ended questions .

LEARN ABOUT: Research Process Steps

To answer such qualitative research questions , the respondent has to write his/her opinion or perspective concerning a specific topic or issue. Unlike other collection methods, online surveys have a wider reach. People can provide you with quality data that is highly credible and valuable.

Paper surveys

Online surveys, focus group discussions.

Focus group discussions can also be considered a type of interview, but it is conducted in a group discussion setting. Usually, the focus group consists of 8 – 10 people (the size may vary depending on the researcher’s requirement). The researchers ensure appropriate space is given to the participants to discuss a topic or issue in a context. The participants are allowed to either agree or disagree with each other’s comments. 

With a focused group discussion, researchers know how a particular group of participants perceives the topic. Researchers analyze what participants think of an issue, the range of opinions expressed, and the ideas discussed. The data is collected by noting down the variations or inconsistencies (if any exist) in the participants, especially in terms of belief, experiences, and practice. 

The participants of focused group discussions are selected based on the topic or issues for which the researcher wants actionable insights. For example, if the research is about the recovery of college students from drug addiction. The participants have to be college students studying and recovering from drug addiction.

Other parameters such as age, qualification, financial background, social presence, and demographics are also considered, but not primarily, as the group needs diverse participants. Frequently, the qualitative data collected through focused group discussion is more descriptive and highly detailed.

Record keeping

This method uses reliable documents and other sources of information that already exist as the data source. This information can help with the new study. It’s a lot like going to the library. There, you can look through books and other sources to find information that can be used in your research.

Case studies

In this method, data is collected by looking at case studies in detail. This method’s flexibility is shown by the fact that it can be used to analyze both simple and complicated topics. This method’s strength is how well it draws conclusions from a mix of one or more qualitative data collection methods.

Observations

Observation is one of the traditional methods of qualitative data collection. It is used by researchers to gather descriptive analysis data by observing people and their behavior at events or in their natural settings. In this method, the researcher is completely immersed in watching people by taking a participatory stance to take down notes.

There are two main types of observation:

  • Covert: In this method, the observer is concealed without letting anyone know that they are being observed. For example, a researcher studying the rituals of a wedding in nomadic tribes must join them as a guest and quietly see everything. 
  • Overt: In this method, everyone is aware that they are being watched. For example, A researcher or an observer wants to study the wedding rituals of a nomadic tribe. To proceed with the research, the observer or researcher can reveal why he is attending the marriage and even use a video camera to shoot everything around him. 

Observation is a useful method of qualitative data collection, especially when you want to study the ongoing process, situation, or reactions on a specific issue related to the people being observed.

When you want to understand people’s behavior or their way of interaction in a particular community or demographic, you can rely on the observation data. Remember, if you fail to get quality data through surveys, qualitative interviews , or group discussions, rely on observation.

It is the best and most trusted collection method of qualitative data to generate qualitative data as it requires equal to no effort from the participants.

LEARN ABOUT: Behavioral Research

You invested time and money acquiring your data, so analyze it. It’s necessary to avoid being in the dark after all your hard work. Qualitative data analysis starts with knowing its two basic techniques, but there are no rules.

  • Deductive Approach: The deductive data analysis uses a researcher-defined structure to analyze qualitative data. This method is quick and easy when a researcher knows what the sample population will say.
  • Inductive Approach: The inductive technique has no structure or framework. When a researcher knows little about the event, an inductive approach is applied.

Whether you want to analyze qualitative data from a one-on-one interview or a survey, these simple steps will ensure a comprehensive qualitative data analysis.

Step 1: Arrange your Data

After collecting all the data, it is mostly unstructured and sometimes unclear. Arranging your data is the first stage in qualitative data analysis. So, researchers must transcribe data before analyzing it.

Step 2: Organize all your Data

After transforming and arranging your data, the next step is to organize it. One of the best ways to organize the data is to think back to your research goals and then organize the data based on the research questions you asked.

Step 3: Set a Code to the Data Collected

Setting up appropriate codes for the collected data gets you one step closer. Coding is one of the most effective methods for compressing a massive amount of data. It allows you to derive theories from relevant research findings.

Step 4: Validate your Data

Qualitative data analysis success requires data validation. Data validation should be done throughout the research process, not just once. There are two sides to validating data:

  • The accuracy of your research design or methods.
  • Reliability—how well the approaches deliver accurate data.

Step 5: Concluding the Analysis Process

Finally, conclude your data in a presentable report. The report should describe your research methods, their pros and cons, and research limitations. Your report should include findings, inferences, and future research.

QuestionPro is a comprehensive online survey software that offers a variety of qualitative data analysis tools to help businesses and researchers in making sense of their data. Users can use many different qualitative analysis methods to learn more about their data.

Users of QuestionPro can see their data in different charts and graphs, which makes it easier to spot patterns and trends. It can help researchers and businesses learn more about their target audience, which can lead to better decisions and better results.

LEARN ABOUT: Steps in Qualitative Research

Qualitative data collection has several advantages, including:

research instrument used in qualitative

  • In-depth understanding: It provides in-depth information about attitudes and behaviors, leading to a deeper understanding of the research.
  • Flexibility: The methods allow researchers to modify questions or change direction if new information emerges.
  • Contextualization: Qualitative research data is in context, which helps to provide a deep understanding of the experiences and perspectives of individuals.
  • Rich data: It often produces rich, detailed, and nuanced information that cannot capture through numerical data.
  • Engagement: The methods, such as interviews and focus groups, involve active meetings with participants, leading to a deeper understanding.
  • Multiple perspectives: This can provide various views and a rich array of voices, adding depth and complexity.
  • Realistic setting: It often occurs in realistic settings, providing more authentic experiences and behaviors.

LEARN ABOUT: 12 Best Tools for Researchers

Qualitative research is one of the best methods for identifying the behavior and patterns governing social conditions, issues, or topics. It spans a step ahead of quantitative data as it fails to explain the reasons and rationale behind a phenomenon, but qualitative data quickly does. 

Qualitative research is one of the best tools to identify behaviors and patterns governing social conditions. It goes a step beyond quantitative data by providing the reasons and rationale behind a phenomenon that cannot be explored quantitatively.

With QuestionPro, you can use it for qualitative data collection through various methods. Using Our robust suite correctly, you can enhance the quality and integrity of the collected data.

LEARN MORE         FREE TRIAL

MORE LIKE THIS

Employee listening strategy

Employee Listening Strategy: What it is & How to Build One

Jul 17, 2024

As your relationships grow, you’ll find that people will come to you for a different perspective or creative way to solve a problem, and it spirals from there.

Winning the Internal CX Battles — Tuesday CX Thoughts

Jul 16, 2024

Knowledge Management

Knowledge Management: What it is, Types, and Use Cases

Jul 12, 2024

Response Weighting: Enhancing Accuracy in Your Surveys

Response Weighting: Enhancing Accuracy in Your Surveys

Jul 11, 2024

Other categories

  • Academic Research
  • Artificial Intelligence
  • Assessments
  • Brand Awareness
  • Case Studies
  • Communities
  • Consumer Insights
  • Customer effort score
  • Customer Engagement
  • Customer Experience
  • Customer Loyalty
  • Customer Research
  • Customer Satisfaction
  • Employee Benefits
  • Employee Engagement
  • Employee Retention
  • Friday Five
  • General Data Protection Regulation
  • Insights Hub
  • Life@QuestionPro
  • Market Research
  • Mobile diaries
  • Mobile Surveys
  • New Features
  • Online Communities
  • Question Types
  • Questionnaire
  • QuestionPro Products
  • Release Notes
  • Research Tools and Apps
  • Revenue at Risk
  • Survey Templates
  • Training Tips
  • Tuesday CX Thoughts (TCXT)
  • Uncategorized
  • What’s Coming Up
  • Workforce Intelligence

Banner

Research Methodologies: Research Instruments

  • Research Methodology Basics
  • Research Instruments
  • Types of Research Methodologies

Header Image

research interview survey bibguru

Types of Research Instruments

A research instrument is a tool you will use to help you collect, measure and analyze the data you use as part of your research.  The choice of research instrument will usually be yours to make as the researcher and will be whichever best suits your methodology. 

There are many different research instruments you can use in collecting data for your research:

  • Interviews  (either as a group or one-on-one). You can carry out interviews in many different ways. For example, your interview can be structured, semi-structured, or unstructured. The difference between them is how formal the set of questions is that is asked of the interviewee. In a group interview, you may choose to ask the interviewees to give you their opinions or perceptions on certain topics.
  • Surveys  (online or in-person). In survey research, you are posing questions in which you ask for a response from the person taking the survey. You may wish to have either free-answer questions such as essay style questions, or you may wish to use closed questions such as multiple choice. You may even wish to make the survey a mixture of both.
  • Focus Groups.  Similar to the group interview above, you may wish to ask a focus group to discuss a particular topic or opinion while you make a note of the answers given.
  • Observations.  This is a good research instrument to use if you are looking into human behaviors. Different ways of researching this include studying the spontaneous behavior of participants in their everyday life, or something more structured. A structured observation is research conducted at a set time and place where researchers observe behavior as planned and agreed upon with participants.

These are the most common ways of carrying out research, but it is really dependent on your needs as a researcher and what approach you think is best to take. It is also possible to combine a number of research instruments if this is necessary and appropriate in answering your research problem.

Data Collection

How to Collect Data for Your Research   This article covers different ways of collecting data in preparation for writing a thesis.

  • << Previous: Research Methodology Basics
  • Next: Types of Research Methodologies >>
  • Last Updated: Sep 27, 2022 12:28 PM
  • URL: https://paperpile.libguides.com/research-methodologies

The Chicago School Library Logo

  • The Chicago School
  • The Chicago School Library
  • Research Guides

Research Methods: Qualitative

What is qualitative research, about this guide, introduction.

  • Qualitative Research Approaches
  • Key Resources
  • Finding Qualitative Studies

 The purpose of this guide is to provide a starting point for learning about qualitative research. In this guide, you'll find:

  • Resources on diverse types of qualitative research.
  • An overview of resources for data, methods, coding & analysis
  • Popular qualitative software options
  • Information on how to find qualitative studies

Qualitative research involves collecting and analyzing non-numerical data (e.g., text, video, or audio) to understand concepts, opinions, or experiences. It can be used to gather in-depth insights into a problem or generate new ideas for research. It answers the hows and whys instead of how many or how much. It could be structured as a stand-alone study, purely relying on qualitative data or it could be part of mixed-methods research that combines qualitative and quantitative data.

Qualitative researchers use multiple systems of inquiry for the study of human phenomena including biography, case study, historical analysis, discourse analysis, ethnography, grounded theory, and phenomenology.

Watch the following video to learn more about Qualitative Research:

(Video best viewed in Edge and Chrome browsers, or click here to view in the Sage Research Methods Database )

Qualitative Approaches

The case study approach is useful to employ when there is a need to obtain an in-depth appreciation of an issue, event or phenomenon of interest, in its natural real-life context.

Ethnography

Ethnographies are an in-depth, holistic type of research used to capture cultural practices, beliefs, traditions, and so on. Here, the researcher observes and interviews members of a culture an ethnic group, a clique, members of a religion, etc. and then analyzes their findings.

Grounded Theory

Researchers will create and test a hypothesis using qualitative data. Often, researchers use grounded theory to understand decision-making, problem-solving, and other types of behavior.

Narrative Research

Researchers use this type of framework to understand different aspects of the human experience and how their subjects assign meaning to their experiences. Researchers use interviews to collect data from a small group of subjects, then discuss those results in the form of a narrative or story.

Phenomenology

This type of research attempts to understand the lived experiences of a group and/or how members of that group find meaning in their experiences. Researchers use interviews, observation, and other qualitative methods to collect data.

Watch the video "Choosing among the Five Qualitative Approaches" from Sage Research Methods database for more on these qualitative approaches:

Note: Video is best viewed using Chrome, Edge, or Safari browsers.

Poth, C. (Academic). (2023). Choosing among five qualitative approaches [Video]. Sage Research Methods. doi.org/10.4135/9781529629866

  • Next: Key Resources >>
  • Last Updated: Jul 18, 2024 12:37 PM
  • URL: https://library.thechicagoschool.edu/qualitative
  • Open access
  • Published: 27 May 2020

How to use and assess qualitative research methods

  • Loraine Busetto   ORCID: orcid.org/0000-0002-9228-7875 1 ,
  • Wolfgang Wick 1 , 2 &
  • Christoph Gumbinger 1  

Neurological Research and Practice volume  2 , Article number:  14 ( 2020 ) Cite this article

758k Accesses

330 Citations

90 Altmetric

Metrics details

This paper aims to provide an overview of the use and assessment of qualitative research methods in the health sciences. Qualitative research can be defined as the study of the nature of phenomena and is especially appropriate for answering questions of why something is (not) observed, assessing complex multi-component interventions, and focussing on intervention improvement. The most common methods of data collection are document study, (non-) participant observations, semi-structured interviews and focus groups. For data analysis, field-notes and audio-recordings are transcribed into protocols and transcripts, and coded using qualitative data management software. Criteria such as checklists, reflexivity, sampling strategies, piloting, co-coding, member-checking and stakeholder involvement can be used to enhance and assess the quality of the research conducted. Using qualitative in addition to quantitative designs will equip us with better tools to address a greater range of research problems, and to fill in blind spots in current neurological research and practice.

The aim of this paper is to provide an overview of qualitative research methods, including hands-on information on how they can be used, reported and assessed. This article is intended for beginning qualitative researchers in the health sciences as well as experienced quantitative researchers who wish to broaden their understanding of qualitative research.

What is qualitative research?

Qualitative research is defined as “the study of the nature of phenomena”, including “their quality, different manifestations, the context in which they appear or the perspectives from which they can be perceived” , but excluding “their range, frequency and place in an objectively determined chain of cause and effect” [ 1 ]. This formal definition can be complemented with a more pragmatic rule of thumb: qualitative research generally includes data in form of words rather than numbers [ 2 ].

Why conduct qualitative research?

Because some research questions cannot be answered using (only) quantitative methods. For example, one Australian study addressed the issue of why patients from Aboriginal communities often present late or not at all to specialist services offered by tertiary care hospitals. Using qualitative interviews with patients and staff, it found one of the most significant access barriers to be transportation problems, including some towns and communities simply not having a bus service to the hospital [ 3 ]. A quantitative study could have measured the number of patients over time or even looked at possible explanatory factors – but only those previously known or suspected to be of relevance. To discover reasons for observed patterns, especially the invisible or surprising ones, qualitative designs are needed.

While qualitative research is common in other fields, it is still relatively underrepresented in health services research. The latter field is more traditionally rooted in the evidence-based-medicine paradigm, as seen in " research that involves testing the effectiveness of various strategies to achieve changes in clinical practice, preferably applying randomised controlled trial study designs (...) " [ 4 ]. This focus on quantitative research and specifically randomised controlled trials (RCT) is visible in the idea of a hierarchy of research evidence which assumes that some research designs are objectively better than others, and that choosing a "lesser" design is only acceptable when the better ones are not practically or ethically feasible [ 5 , 6 ]. Others, however, argue that an objective hierarchy does not exist, and that, instead, the research design and methods should be chosen to fit the specific research question at hand – "questions before methods" [ 2 , 7 , 8 , 9 ]. This means that even when an RCT is possible, some research problems require a different design that is better suited to addressing them. Arguing in JAMA, Berwick uses the example of rapid response teams in hospitals, which he describes as " a complex, multicomponent intervention – essentially a process of social change" susceptible to a range of different context factors including leadership or organisation history. According to him, "[in] such complex terrain, the RCT is an impoverished way to learn. Critics who use it as a truth standard in this context are incorrect" [ 8 ] . Instead of limiting oneself to RCTs, Berwick recommends embracing a wider range of methods , including qualitative ones, which for "these specific applications, (...) are not compromises in learning how to improve; they are superior" [ 8 ].

Research problems that can be approached particularly well using qualitative methods include assessing complex multi-component interventions or systems (of change), addressing questions beyond “what works”, towards “what works for whom when, how and why”, and focussing on intervention improvement rather than accreditation [ 7 , 9 , 10 , 11 , 12 ]. Using qualitative methods can also help shed light on the “softer” side of medical treatment. For example, while quantitative trials can measure the costs and benefits of neuro-oncological treatment in terms of survival rates or adverse effects, qualitative research can help provide a better understanding of patient or caregiver stress, visibility of illness or out-of-pocket expenses.

How to conduct qualitative research?

Given that qualitative research is characterised by flexibility, openness and responsivity to context, the steps of data collection and analysis are not as separate and consecutive as they tend to be in quantitative research [ 13 , 14 ]. As Fossey puts it : “sampling, data collection, analysis and interpretation are related to each other in a cyclical (iterative) manner, rather than following one after another in a stepwise approach” [ 15 ]. The researcher can make educated decisions with regard to the choice of method, how they are implemented, and to which and how many units they are applied [ 13 ]. As shown in Fig.  1 , this can involve several back-and-forth steps between data collection and analysis where new insights and experiences can lead to adaption and expansion of the original plan. Some insights may also necessitate a revision of the research question and/or the research design as a whole. The process ends when saturation is achieved, i.e. when no relevant new information can be found (see also below: sampling and saturation). For reasons of transparency, it is essential for all decisions as well as the underlying reasoning to be well-documented.

figure 1

Iterative research process

While it is not always explicitly addressed, qualitative methods reflect a different underlying research paradigm than quantitative research (e.g. constructivism or interpretivism as opposed to positivism). The choice of methods can be based on the respective underlying substantive theory or theoretical framework used by the researcher [ 2 ].

Data collection

The methods of qualitative data collection most commonly used in health research are document study, observations, semi-structured interviews and focus groups [ 1 , 14 , 16 , 17 ].

Document study

Document study (also called document analysis) refers to the review by the researcher of written materials [ 14 ]. These can include personal and non-personal documents such as archives, annual reports, guidelines, policy documents, diaries or letters.

Observations

Observations are particularly useful to gain insights into a certain setting and actual behaviour – as opposed to reported behaviour or opinions [ 13 ]. Qualitative observations can be either participant or non-participant in nature. In participant observations, the observer is part of the observed setting, for example a nurse working in an intensive care unit [ 18 ]. In non-participant observations, the observer is “on the outside looking in”, i.e. present in but not part of the situation, trying not to influence the setting by their presence. Observations can be planned (e.g. for 3 h during the day or night shift) or ad hoc (e.g. as soon as a stroke patient arrives at the emergency room). During the observation, the observer takes notes on everything or certain pre-determined parts of what is happening around them, for example focusing on physician-patient interactions or communication between different professional groups. Written notes can be taken during or after the observations, depending on feasibility (which is usually lower during participant observations) and acceptability (e.g. when the observer is perceived to be judging the observed). Afterwards, these field notes are transcribed into observation protocols. If more than one observer was involved, field notes are taken independently, but notes can be consolidated into one protocol after discussions. Advantages of conducting observations include minimising the distance between the researcher and the researched, the potential discovery of topics that the researcher did not realise were relevant and gaining deeper insights into the real-world dimensions of the research problem at hand [ 18 ].

Semi-structured interviews

Hijmans & Kuyper describe qualitative interviews as “an exchange with an informal character, a conversation with a goal” [ 19 ]. Interviews are used to gain insights into a person’s subjective experiences, opinions and motivations – as opposed to facts or behaviours [ 13 ]. Interviews can be distinguished by the degree to which they are structured (i.e. a questionnaire), open (e.g. free conversation or autobiographical interviews) or semi-structured [ 2 , 13 ]. Semi-structured interviews are characterized by open-ended questions and the use of an interview guide (or topic guide/list) in which the broad areas of interest, sometimes including sub-questions, are defined [ 19 ]. The pre-defined topics in the interview guide can be derived from the literature, previous research or a preliminary method of data collection, e.g. document study or observations. The topic list is usually adapted and improved at the start of the data collection process as the interviewer learns more about the field [ 20 ]. Across interviews the focus on the different (blocks of) questions may differ and some questions may be skipped altogether (e.g. if the interviewee is not able or willing to answer the questions or for concerns about the total length of the interview) [ 20 ]. Qualitative interviews are usually not conducted in written format as it impedes on the interactive component of the method [ 20 ]. In comparison to written surveys, qualitative interviews have the advantage of being interactive and allowing for unexpected topics to emerge and to be taken up by the researcher. This can also help overcome a provider or researcher-centred bias often found in written surveys, which by nature, can only measure what is already known or expected to be of relevance to the researcher. Interviews can be audio- or video-taped; but sometimes it is only feasible or acceptable for the interviewer to take written notes [ 14 , 16 , 20 ].

Focus groups

Focus groups are group interviews to explore participants’ expertise and experiences, including explorations of how and why people behave in certain ways [ 1 ]. Focus groups usually consist of 6–8 people and are led by an experienced moderator following a topic guide or “script” [ 21 ]. They can involve an observer who takes note of the non-verbal aspects of the situation, possibly using an observation guide [ 21 ]. Depending on researchers’ and participants’ preferences, the discussions can be audio- or video-taped and transcribed afterwards [ 21 ]. Focus groups are useful for bringing together homogeneous (to a lesser extent heterogeneous) groups of participants with relevant expertise and experience on a given topic on which they can share detailed information [ 21 ]. Focus groups are a relatively easy, fast and inexpensive method to gain access to information on interactions in a given group, i.e. “the sharing and comparing” among participants [ 21 ]. Disadvantages include less control over the process and a lesser extent to which each individual may participate. Moreover, focus group moderators need experience, as do those tasked with the analysis of the resulting data. Focus groups can be less appropriate for discussing sensitive topics that participants might be reluctant to disclose in a group setting [ 13 ]. Moreover, attention must be paid to the emergence of “groupthink” as well as possible power dynamics within the group, e.g. when patients are awed or intimidated by health professionals.

Choosing the “right” method

As explained above, the school of thought underlying qualitative research assumes no objective hierarchy of evidence and methods. This means that each choice of single or combined methods has to be based on the research question that needs to be answered and a critical assessment with regard to whether or to what extent the chosen method can accomplish this – i.e. the “fit” between question and method [ 14 ]. It is necessary for these decisions to be documented when they are being made, and to be critically discussed when reporting methods and results.

Let us assume that our research aim is to examine the (clinical) processes around acute endovascular treatment (EVT), from the patient’s arrival at the emergency room to recanalization, with the aim to identify possible causes for delay and/or other causes for sub-optimal treatment outcome. As a first step, we could conduct a document study of the relevant standard operating procedures (SOPs) for this phase of care – are they up-to-date and in line with current guidelines? Do they contain any mistakes, irregularities or uncertainties that could cause delays or other problems? Regardless of the answers to these questions, the results have to be interpreted based on what they are: a written outline of what care processes in this hospital should look like. If we want to know what they actually look like in practice, we can conduct observations of the processes described in the SOPs. These results can (and should) be analysed in themselves, but also in comparison to the results of the document analysis, especially as regards relevant discrepancies. Do the SOPs outline specific tests for which no equipment can be observed or tasks to be performed by specialized nurses who are not present during the observation? It might also be possible that the written SOP is outdated, but the actual care provided is in line with current best practice. In order to find out why these discrepancies exist, it can be useful to conduct interviews. Are the physicians simply not aware of the SOPs (because their existence is limited to the hospital’s intranet) or do they actively disagree with them or does the infrastructure make it impossible to provide the care as described? Another rationale for adding interviews is that some situations (or all of their possible variations for different patient groups or the day, night or weekend shift) cannot practically or ethically be observed. In this case, it is possible to ask those involved to report on their actions – being aware that this is not the same as the actual observation. A senior physician’s or hospital manager’s description of certain situations might differ from a nurse’s or junior physician’s one, maybe because they intentionally misrepresent facts or maybe because different aspects of the process are visible or important to them. In some cases, it can also be relevant to consider to whom the interviewee is disclosing this information – someone they trust, someone they are otherwise not connected to, or someone they suspect or are aware of being in a potentially “dangerous” power relationship to them. Lastly, a focus group could be conducted with representatives of the relevant professional groups to explore how and why exactly they provide care around EVT. The discussion might reveal discrepancies (between SOPs and actual care or between different physicians) and motivations to the researchers as well as to the focus group members that they might not have been aware of themselves. For the focus group to deliver relevant information, attention has to be paid to its composition and conduct, for example, to make sure that all participants feel safe to disclose sensitive or potentially problematic information or that the discussion is not dominated by (senior) physicians only. The resulting combination of data collection methods is shown in Fig.  2 .

figure 2

Possible combination of data collection methods

Attributions for icons: “Book” by Serhii Smirnov, “Interview” by Adrien Coquet, FR, “Magnifying Glass” by anggun, ID, “Business communication” by Vectors Market; all from the Noun Project

The combination of multiple data source as described for this example can be referred to as “triangulation”, in which multiple measurements are carried out from different angles to achieve a more comprehensive understanding of the phenomenon under study [ 22 , 23 ].

Data analysis

To analyse the data collected through observations, interviews and focus groups these need to be transcribed into protocols and transcripts (see Fig.  3 ). Interviews and focus groups can be transcribed verbatim , with or without annotations for behaviour (e.g. laughing, crying, pausing) and with or without phonetic transcription of dialects and filler words, depending on what is expected or known to be relevant for the analysis. In the next step, the protocols and transcripts are coded , that is, marked (or tagged, labelled) with one or more short descriptors of the content of a sentence or paragraph [ 2 , 15 , 23 ]. Jansen describes coding as “connecting the raw data with “theoretical” terms” [ 20 ]. In a more practical sense, coding makes raw data sortable. This makes it possible to extract and examine all segments describing, say, a tele-neurology consultation from multiple data sources (e.g. SOPs, emergency room observations, staff and patient interview). In a process of synthesis and abstraction, the codes are then grouped, summarised and/or categorised [ 15 , 20 ]. The end product of the coding or analysis process is a descriptive theory of the behavioural pattern under investigation [ 20 ]. The coding process is performed using qualitative data management software, the most common ones being InVivo, MaxQDA and Atlas.ti. It should be noted that these are data management tools which support the analysis performed by the researcher(s) [ 14 ].

figure 3

From data collection to data analysis

Attributions for icons: see Fig. 2 , also “Speech to text” by Trevor Dsouza, “Field Notes” by Mike O’Brien, US, “Voice Record” by ProSymbols, US, “Inspection” by Made, AU, and “Cloud” by Graphic Tigers; all from the Noun Project

How to report qualitative research?

Protocols of qualitative research can be published separately and in advance of the study results. However, the aim is not the same as in RCT protocols, i.e. to pre-define and set in stone the research questions and primary or secondary endpoints. Rather, it is a way to describe the research methods in detail, which might not be possible in the results paper given journals’ word limits. Qualitative research papers are usually longer than their quantitative counterparts to allow for deep understanding and so-called “thick description”. In the methods section, the focus is on transparency of the methods used, including why, how and by whom they were implemented in the specific study setting, so as to enable a discussion of whether and how this may have influenced data collection, analysis and interpretation. The results section usually starts with a paragraph outlining the main findings, followed by more detailed descriptions of, for example, the commonalities, discrepancies or exceptions per category [ 20 ]. Here it is important to support main findings by relevant quotations, which may add information, context, emphasis or real-life examples [ 20 , 23 ]. It is subject to debate in the field whether it is relevant to state the exact number or percentage of respondents supporting a certain statement (e.g. “Five interviewees expressed negative feelings towards XYZ”) [ 21 ].

How to combine qualitative with quantitative research?

Qualitative methods can be combined with other methods in multi- or mixed methods designs, which “[employ] two or more different methods [ …] within the same study or research program rather than confining the research to one single method” [ 24 ]. Reasons for combining methods can be diverse, including triangulation for corroboration of findings, complementarity for illustration and clarification of results, expansion to extend the breadth and range of the study, explanation of (unexpected) results generated with one method with the help of another, or offsetting the weakness of one method with the strength of another [ 1 , 17 , 24 , 25 , 26 ]. The resulting designs can be classified according to when, why and how the different quantitative and/or qualitative data strands are combined. The three most common types of mixed method designs are the convergent parallel design , the explanatory sequential design and the exploratory sequential design. The designs with examples are shown in Fig.  4 .

figure 4

Three common mixed methods designs

In the convergent parallel design, a qualitative study is conducted in parallel to and independently of a quantitative study, and the results of both studies are compared and combined at the stage of interpretation of results. Using the above example of EVT provision, this could entail setting up a quantitative EVT registry to measure process times and patient outcomes in parallel to conducting the qualitative research outlined above, and then comparing results. Amongst other things, this would make it possible to assess whether interview respondents’ subjective impressions of patients receiving good care match modified Rankin Scores at follow-up, or whether observed delays in care provision are exceptions or the rule when compared to door-to-needle times as documented in the registry. In the explanatory sequential design, a quantitative study is carried out first, followed by a qualitative study to help explain the results from the quantitative study. This would be an appropriate design if the registry alone had revealed relevant delays in door-to-needle times and the qualitative study would be used to understand where and why these occurred, and how they could be improved. In the exploratory design, the qualitative study is carried out first and its results help informing and building the quantitative study in the next step [ 26 ]. If the qualitative study around EVT provision had shown a high level of dissatisfaction among the staff members involved, a quantitative questionnaire investigating staff satisfaction could be set up in the next step, informed by the qualitative study on which topics dissatisfaction had been expressed. Amongst other things, the questionnaire design would make it possible to widen the reach of the research to more respondents from different (types of) hospitals, regions, countries or settings, and to conduct sub-group analyses for different professional groups.

How to assess qualitative research?

A variety of assessment criteria and lists have been developed for qualitative research, ranging in their focus and comprehensiveness [ 14 , 17 , 27 ]. However, none of these has been elevated to the “gold standard” in the field. In the following, we therefore focus on a set of commonly used assessment criteria that, from a practical standpoint, a researcher can look for when assessing a qualitative research report or paper.

Assessors should check the authors’ use of and adherence to the relevant reporting checklists (e.g. Standards for Reporting Qualitative Research (SRQR)) to make sure all items that are relevant for this type of research are addressed [ 23 , 28 ]. Discussions of quantitative measures in addition to or instead of these qualitative measures can be a sign of lower quality of the research (paper). Providing and adhering to a checklist for qualitative research contributes to an important quality criterion for qualitative research, namely transparency [ 15 , 17 , 23 ].

Reflexivity

While methodological transparency and complete reporting is relevant for all types of research, some additional criteria must be taken into account for qualitative research. This includes what is called reflexivity, i.e. sensitivity to the relationship between the researcher and the researched, including how contact was established and maintained, or the background and experience of the researcher(s) involved in data collection and analysis. Depending on the research question and population to be researched this can be limited to professional experience, but it may also include gender, age or ethnicity [ 17 , 27 ]. These details are relevant because in qualitative research, as opposed to quantitative research, the researcher as a person cannot be isolated from the research process [ 23 ]. It may influence the conversation when an interviewed patient speaks to an interviewer who is a physician, or when an interviewee is asked to discuss a gynaecological procedure with a male interviewer, and therefore the reader must be made aware of these details [ 19 ].

Sampling and saturation

The aim of qualitative sampling is for all variants of the objects of observation that are deemed relevant for the study to be present in the sample “ to see the issue and its meanings from as many angles as possible” [ 1 , 16 , 19 , 20 , 27 ] , and to ensure “information-richness [ 15 ]. An iterative sampling approach is advised, in which data collection (e.g. five interviews) is followed by data analysis, followed by more data collection to find variants that are lacking in the current sample. This process continues until no new (relevant) information can be found and further sampling becomes redundant – which is called saturation [ 1 , 15 ] . In other words: qualitative data collection finds its end point not a priori , but when the research team determines that saturation has been reached [ 29 , 30 ].

This is also the reason why most qualitative studies use deliberate instead of random sampling strategies. This is generally referred to as “ purposive sampling” , in which researchers pre-define which types of participants or cases they need to include so as to cover all variations that are expected to be of relevance, based on the literature, previous experience or theory (i.e. theoretical sampling) [ 14 , 20 ]. Other types of purposive sampling include (but are not limited to) maximum variation sampling, critical case sampling or extreme or deviant case sampling [ 2 ]. In the above EVT example, a purposive sample could include all relevant professional groups and/or all relevant stakeholders (patients, relatives) and/or all relevant times of observation (day, night and weekend shift).

Assessors of qualitative research should check whether the considerations underlying the sampling strategy were sound and whether or how researchers tried to adapt and improve their strategies in stepwise or cyclical approaches between data collection and analysis to achieve saturation [ 14 ].

Good qualitative research is iterative in nature, i.e. it goes back and forth between data collection and analysis, revising and improving the approach where necessary. One example of this are pilot interviews, where different aspects of the interview (especially the interview guide, but also, for example, the site of the interview or whether the interview can be audio-recorded) are tested with a small number of respondents, evaluated and revised [ 19 ]. In doing so, the interviewer learns which wording or types of questions work best, or which is the best length of an interview with patients who have trouble concentrating for an extended time. Of course, the same reasoning applies to observations or focus groups which can also be piloted.

Ideally, coding should be performed by at least two researchers, especially at the beginning of the coding process when a common approach must be defined, including the establishment of a useful coding list (or tree), and when a common meaning of individual codes must be established [ 23 ]. An initial sub-set or all transcripts can be coded independently by the coders and then compared and consolidated after regular discussions in the research team. This is to make sure that codes are applied consistently to the research data.

Member checking

Member checking, also called respondent validation , refers to the practice of checking back with study respondents to see if the research is in line with their views [ 14 , 27 ]. This can happen after data collection or analysis or when first results are available [ 23 ]. For example, interviewees can be provided with (summaries of) their transcripts and asked whether they believe this to be a complete representation of their views or whether they would like to clarify or elaborate on their responses [ 17 ]. Respondents’ feedback on these issues then becomes part of the data collection and analysis [ 27 ].

Stakeholder involvement

In those niches where qualitative approaches have been able to evolve and grow, a new trend has seen the inclusion of patients and their representatives not only as study participants (i.e. “members”, see above) but as consultants to and active participants in the broader research process [ 31 , 32 , 33 ]. The underlying assumption is that patients and other stakeholders hold unique perspectives and experiences that add value beyond their own single story, making the research more relevant and beneficial to researchers, study participants and (future) patients alike [ 34 , 35 ]. Using the example of patients on or nearing dialysis, a recent scoping review found that 80% of clinical research did not address the top 10 research priorities identified by patients and caregivers [ 32 , 36 ]. In this sense, the involvement of the relevant stakeholders, especially patients and relatives, is increasingly being seen as a quality indicator in and of itself.

How not to assess qualitative research

The above overview does not include certain items that are routine in assessments of quantitative research. What follows is a non-exhaustive, non-representative, experience-based list of the quantitative criteria often applied to the assessment of qualitative research, as well as an explanation of the limited usefulness of these endeavours.

Protocol adherence

Given the openness and flexibility of qualitative research, it should not be assessed by how well it adheres to pre-determined and fixed strategies – in other words: its rigidity. Instead, the assessor should look for signs of adaptation and refinement based on lessons learned from earlier steps in the research process.

Sample size

For the reasons explained above, qualitative research does not require specific sample sizes, nor does it require that the sample size be determined a priori [ 1 , 14 , 27 , 37 , 38 , 39 ]. Sample size can only be a useful quality indicator when related to the research purpose, the chosen methodology and the composition of the sample, i.e. who was included and why.

Randomisation

While some authors argue that randomisation can be used in qualitative research, this is not commonly the case, as neither its feasibility nor its necessity or usefulness has been convincingly established for qualitative research [ 13 , 27 ]. Relevant disadvantages include the negative impact of a too large sample size as well as the possibility (or probability) of selecting “ quiet, uncooperative or inarticulate individuals ” [ 17 ]. Qualitative studies do not use control groups, either.

Interrater reliability, variability and other “objectivity checks”

The concept of “interrater reliability” is sometimes used in qualitative research to assess to which extent the coding approach overlaps between the two co-coders. However, it is not clear what this measure tells us about the quality of the analysis [ 23 ]. This means that these scores can be included in qualitative research reports, preferably with some additional information on what the score means for the analysis, but it is not a requirement. Relatedly, it is not relevant for the quality or “objectivity” of qualitative research to separate those who recruited the study participants and collected and analysed the data. Experiences even show that it might be better to have the same person or team perform all of these tasks [ 20 ]. First, when researchers introduce themselves during recruitment this can enhance trust when the interview takes place days or weeks later with the same researcher. Second, when the audio-recording is transcribed for analysis, the researcher conducting the interviews will usually remember the interviewee and the specific interview situation during data analysis. This might be helpful in providing additional context information for interpretation of data, e.g. on whether something might have been meant as a joke [ 18 ].

Not being quantitative research

Being qualitative research instead of quantitative research should not be used as an assessment criterion if it is used irrespectively of the research problem at hand. Similarly, qualitative research should not be required to be combined with quantitative research per se – unless mixed methods research is judged as inherently better than single-method research. In this case, the same criterion should be applied for quantitative studies without a qualitative component.

The main take-away points of this paper are summarised in Table 1 . We aimed to show that, if conducted well, qualitative research can answer specific research questions that cannot to be adequately answered using (only) quantitative designs. Seeing qualitative and quantitative methods as equal will help us become more aware and critical of the “fit” between the research problem and our chosen methods: I can conduct an RCT to determine the reasons for transportation delays of acute stroke patients – but should I? It also provides us with a greater range of tools to tackle a greater range of research problems more appropriately and successfully, filling in the blind spots on one half of the methodological spectrum to better address the whole complexity of neurological research and practice.

Availability of data and materials

Not applicable.

Abbreviations

Endovascular treatment

Randomised Controlled Trial

Standard Operating Procedure

Standards for Reporting Qualitative Research

Philipsen, H., & Vernooij-Dassen, M. (2007). Kwalitatief onderzoek: nuttig, onmisbaar en uitdagend. In L. PLBJ & H. TCo (Eds.), Kwalitatief onderzoek: Praktische methoden voor de medische praktijk . [Qualitative research: useful, indispensable and challenging. In: Qualitative research: Practical methods for medical practice (pp. 5–12). Houten: Bohn Stafleu van Loghum.

Chapter   Google Scholar  

Punch, K. F. (2013). Introduction to social research: Quantitative and qualitative approaches . London: Sage.

Kelly, J., Dwyer, J., Willis, E., & Pekarsky, B. (2014). Travelling to the city for hospital care: Access factors in country aboriginal patient journeys. Australian Journal of Rural Health, 22 (3), 109–113.

Article   Google Scholar  

Nilsen, P., Ståhl, C., Roback, K., & Cairney, P. (2013). Never the twain shall meet? - a comparison of implementation science and policy implementation research. Implementation Science, 8 (1), 1–12.

Howick J, Chalmers I, Glasziou, P., Greenhalgh, T., Heneghan, C., Liberati, A., Moschetti, I., Phillips, B., & Thornton, H. (2011). The 2011 Oxford CEBM evidence levels of evidence (introductory document) . Oxford Center for Evidence Based Medicine. https://www.cebm.net/2011/06/2011-oxford-cebm-levels-evidence-introductory-document/ .

Eakin, J. M. (2016). Educating critical qualitative health researchers in the land of the randomized controlled trial. Qualitative Inquiry, 22 (2), 107–118.

May, A., & Mathijssen, J. (2015). Alternatieven voor RCT bij de evaluatie van effectiviteit van interventies!? Eindrapportage. In Alternatives for RCTs in the evaluation of effectiveness of interventions!? Final report .

Google Scholar  

Berwick, D. M. (2008). The science of improvement. Journal of the American Medical Association, 299 (10), 1182–1184.

Article   CAS   Google Scholar  

Christ, T. W. (2014). Scientific-based research and randomized controlled trials, the “gold” standard? Alternative paradigms and mixed methodologies. Qualitative Inquiry, 20 (1), 72–80.

Lamont, T., Barber, N., Jd, P., Fulop, N., Garfield-Birkbeck, S., Lilford, R., Mear, L., Raine, R., & Fitzpatrick, R. (2016). New approaches to evaluating complex health and care systems. BMJ, 352:i154.

Drabble, S. J., & O’Cathain, A. (2015). Moving from Randomized Controlled Trials to Mixed Methods Intervention Evaluation. In S. Hesse-Biber & R. B. Johnson (Eds.), The Oxford Handbook of Multimethod and Mixed Methods Research Inquiry (pp. 406–425). London: Oxford University Press.

Chambers, D. A., Glasgow, R. E., & Stange, K. C. (2013). The dynamic sustainability framework: Addressing the paradox of sustainment amid ongoing change. Implementation Science : IS, 8 , 117.

Hak, T. (2007). Waarnemingsmethoden in kwalitatief onderzoek. In L. PLBJ & H. TCo (Eds.), Kwalitatief onderzoek: Praktische methoden voor de medische praktijk . [Observation methods in qualitative research] (pp. 13–25). Houten: Bohn Stafleu van Loghum.

Russell, C. K., & Gregory, D. M. (2003). Evaluation of qualitative research studies. Evidence Based Nursing, 6 (2), 36–40.

Fossey, E., Harvey, C., McDermott, F., & Davidson, L. (2002). Understanding and evaluating qualitative research. Australian and New Zealand Journal of Psychiatry, 36 , 717–732.

Yanow, D. (2000). Conducting interpretive policy analysis (Vol. 47). Thousand Oaks: Sage University Papers Series on Qualitative Research Methods.

Shenton, A. K. (2004). Strategies for ensuring trustworthiness in qualitative research projects. Education for Information, 22 , 63–75.

van der Geest, S. (2006). Participeren in ziekte en zorg: meer over kwalitatief onderzoek. Huisarts en Wetenschap, 49 (4), 283–287.

Hijmans, E., & Kuyper, M. (2007). Het halfopen interview als onderzoeksmethode. In L. PLBJ & H. TCo (Eds.), Kwalitatief onderzoek: Praktische methoden voor de medische praktijk . [The half-open interview as research method (pp. 43–51). Houten: Bohn Stafleu van Loghum.

Jansen, H. (2007). Systematiek en toepassing van de kwalitatieve survey. In L. PLBJ & H. TCo (Eds.), Kwalitatief onderzoek: Praktische methoden voor de medische praktijk . [Systematics and implementation of the qualitative survey (pp. 27–41). Houten: Bohn Stafleu van Loghum.

Pv, R., & Peremans, L. (2007). Exploreren met focusgroepgesprekken: de ‘stem’ van de groep onder de loep. In L. PLBJ & H. TCo (Eds.), Kwalitatief onderzoek: Praktische methoden voor de medische praktijk . [Exploring with focus group conversations: the “voice” of the group under the magnifying glass (pp. 53–64). Houten: Bohn Stafleu van Loghum.

Carter, N., Bryant-Lukosius, D., DiCenso, A., Blythe, J., & Neville, A. J. (2014). The use of triangulation in qualitative research. Oncology Nursing Forum, 41 (5), 545–547.

Boeije H: Analyseren in kwalitatief onderzoek: Denken en doen, [Analysis in qualitative research: Thinking and doing] vol. Den Haag Boom Lemma uitgevers; 2012.

Hunter, A., & Brewer, J. (2015). Designing Multimethod Research. In S. Hesse-Biber & R. B. Johnson (Eds.), The Oxford Handbook of Multimethod and Mixed Methods Research Inquiry (pp. 185–205). London: Oxford University Press.

Archibald, M. M., Radil, A. I., Zhang, X., & Hanson, W. E. (2015). Current mixed methods practices in qualitative research: A content analysis of leading journals. International Journal of Qualitative Methods, 14 (2), 5–33.

Creswell, J. W., & Plano Clark, V. L. (2011). Choosing a Mixed Methods Design. In Designing and Conducting Mixed Methods Research . Thousand Oaks: SAGE Publications.

Mays, N., & Pope, C. (2000). Assessing quality in qualitative research. BMJ, 320 (7226), 50–52.

O'Brien, B. C., Harris, I. B., Beckman, T. J., Reed, D. A., & Cook, D. A. (2014). Standards for reporting qualitative research: A synthesis of recommendations. Academic Medicine : Journal of the Association of American Medical Colleges, 89 (9), 1245–1251.

Saunders, B., Sim, J., Kingstone, T., Baker, S., Waterfield, J., Bartlam, B., Burroughs, H., & Jinks, C. (2018). Saturation in qualitative research: Exploring its conceptualization and operationalization. Quality and Quantity, 52 (4), 1893–1907.

Moser, A., & Korstjens, I. (2018). Series: Practical guidance to qualitative research. Part 3: Sampling, data collection and analysis. European Journal of General Practice, 24 (1), 9–18.

Marlett, N., Shklarov, S., Marshall, D., Santana, M. J., & Wasylak, T. (2015). Building new roles and relationships in research: A model of patient engagement research. Quality of Life Research : an international journal of quality of life aspects of treatment, care and rehabilitation, 24 (5), 1057–1067.

Demian, M. N., Lam, N. N., Mac-Way, F., Sapir-Pichhadze, R., & Fernandez, N. (2017). Opportunities for engaging patients in kidney research. Canadian Journal of Kidney Health and Disease, 4 , 2054358117703070–2054358117703070.

Noyes, J., McLaughlin, L., Morgan, K., Roberts, A., Stephens, M., Bourne, J., Houlston, M., Houlston, J., Thomas, S., Rhys, R. G., et al. (2019). Designing a co-productive study to overcome known methodological challenges in organ donation research with bereaved family members. Health Expectations . 22(4):824–35.

Piil, K., Jarden, M., & Pii, K. H. (2019). Research agenda for life-threatening cancer. European Journal Cancer Care (Engl), 28 (1), e12935.

Hofmann, D., Ibrahim, F., Rose, D., Scott, D. L., Cope, A., Wykes, T., & Lempp, H. (2015). Expectations of new treatment in rheumatoid arthritis: Developing a patient-generated questionnaire. Health Expectations : an international journal of public participation in health care and health policy, 18 (5), 995–1008.

Jun, M., Manns, B., Laupacis, A., Manns, L., Rehal, B., Crowe, S., & Hemmelgarn, B. R. (2015). Assessing the extent to which current clinical research is consistent with patient priorities: A scoping review using a case study in patients on or nearing dialysis. Canadian Journal of Kidney Health and Disease, 2 , 35.

Elsie Baker, S., & Edwards, R. (2012). How many qualitative interviews is enough? In National Centre for Research Methods Review Paper . National Centre for Research Methods. http://eprints.ncrm.ac.uk/2273/4/how_many_interviews.pdf .

Sandelowski, M. (1995). Sample size in qualitative research. Research in Nursing & Health, 18 (2), 179–183.

Sim, J., Saunders, B., Waterfield, J., & Kingstone, T. (2018). Can sample size in qualitative research be determined a priori? International Journal of Social Research Methodology, 21 (5), 619–634.

Download references

Acknowledgements

no external funding.

Author information

Authors and affiliations.

Department of Neurology, Heidelberg University Hospital, Im Neuenheimer Feld 400, 69120, Heidelberg, Germany

Loraine Busetto, Wolfgang Wick & Christoph Gumbinger

Clinical Cooperation Unit Neuro-Oncology, German Cancer Research Center, Heidelberg, Germany

Wolfgang Wick

You can also search for this author in PubMed   Google Scholar

Contributions

LB drafted the manuscript; WW and CG revised the manuscript; all authors approved the final versions.

Corresponding author

Correspondence to Loraine Busetto .

Ethics declarations

Ethics approval and consent to participate, consent for publication, competing interests.

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Busetto, L., Wick, W. & Gumbinger, C. How to use and assess qualitative research methods. Neurol. Res. Pract. 2 , 14 (2020). https://doi.org/10.1186/s42466-020-00059-z

Download citation

Received : 30 January 2020

Accepted : 22 April 2020

Published : 27 May 2020

DOI : https://doi.org/10.1186/s42466-020-00059-z

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

  • Qualitative research
  • Mixed methods
  • Quality assessment

Neurological Research and Practice

ISSN: 2524-3489

  • Submission enquiries: Access here and click Contact Us
  • General enquiries: [email protected]

research instrument used in qualitative

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

  • Knowledge Base

Methodology

  • Questionnaire Design | Methods, Question Types & Examples

Questionnaire Design | Methods, Question Types & Examples

Published on July 15, 2021 by Pritha Bhandari . Revised on June 22, 2023.

A questionnaire is a list of questions or items used to gather data from respondents about their attitudes, experiences, or opinions. Questionnaires can be used to collect quantitative and/or qualitative information.

Questionnaires are commonly used in market research as well as in the social and health sciences. For example, a company may ask for feedback about a recent customer service experience, or psychology researchers may investigate health risk perceptions using questionnaires.

Table of contents

Questionnaires vs. surveys, questionnaire methods, open-ended vs. closed-ended questions, question wording, question order, step-by-step guide to design, other interesting articles, frequently asked questions about questionnaire design.

A survey is a research method where you collect and analyze data from a group of people. A questionnaire is a specific tool or instrument for collecting the data.

Designing a questionnaire means creating valid and reliable questions that address your research objectives , placing them in a useful order, and selecting an appropriate method for administration.

But designing a questionnaire is only one component of survey research. Survey research also involves defining the population you’re interested in, choosing an appropriate sampling method , administering questionnaires, data cleansing and analysis, and interpretation.

Sampling is important in survey research because you’ll often aim to generalize your results to the population. Gather data from a sample that represents the range of views in the population for externally valid results. There will always be some differences between the population and the sample, but minimizing these will help you avoid several types of research bias , including sampling bias , ascertainment bias , and undercoverage bias .

Here's why students love Scribbr's proofreading services

Discover proofreading & editing

Questionnaires can be self-administered or researcher-administered . Self-administered questionnaires are more common because they are easy to implement and inexpensive, but researcher-administered questionnaires allow deeper insights.

Self-administered questionnaires

Self-administered questionnaires can be delivered online or in paper-and-pen formats, in person or through mail. All questions are standardized so that all respondents receive the same questions with identical wording.

Self-administered questionnaires can be:

  • cost-effective
  • easy to administer for small and large groups
  • anonymous and suitable for sensitive topics

But they may also be:

  • unsuitable for people with limited literacy or verbal skills
  • susceptible to a nonresponse bias (most people invited may not complete the questionnaire)
  • biased towards people who volunteer because impersonal survey requests often go ignored.

Researcher-administered questionnaires

Researcher-administered questionnaires are interviews that take place by phone, in-person, or online between researchers and respondents.

Researcher-administered questionnaires can:

  • help you ensure the respondents are representative of your target audience
  • allow clarifications of ambiguous or unclear questions and answers
  • have high response rates because it’s harder to refuse an interview when personal attention is given to respondents

But researcher-administered questionnaires can be limiting in terms of resources. They are:

  • costly and time-consuming to perform
  • more difficult to analyze if you have qualitative responses
  • likely to contain experimenter bias or demand characteristics
  • likely to encourage social desirability bias in responses because of a lack of anonymity

Your questionnaire can include open-ended or closed-ended questions or a combination of both.

Using closed-ended questions limits your responses, while open-ended questions enable a broad range of answers. You’ll need to balance these considerations with your available time and resources.

Closed-ended questions

Closed-ended, or restricted-choice, questions offer respondents a fixed set of choices to select from. Closed-ended questions are best for collecting data on categorical or quantitative variables.

Categorical variables can be nominal or ordinal. Quantitative variables can be interval or ratio. Understanding the type of variable and level of measurement means you can perform appropriate statistical analyses for generalizable results.

Examples of closed-ended questions for different variables

Nominal variables include categories that can’t be ranked, such as race or ethnicity. This includes binary or dichotomous categories.

It’s best to include categories that cover all possible answers and are mutually exclusive. There should be no overlap between response items.

In binary or dichotomous questions, you’ll give respondents only two options to choose from.

White Black or African American American Indian or Alaska Native Asian Native Hawaiian or Other Pacific Islander

Ordinal variables include categories that can be ranked. Consider how wide or narrow a range you’ll include in your response items, and their relevance to your respondents.

Likert scale questions collect ordinal data using rating scales with 5 or 7 points.

When you have four or more Likert-type questions, you can treat the composite data as quantitative data on an interval scale . Intelligence tests, psychological scales, and personality inventories use multiple Likert-type questions to collect interval data.

With interval or ratio scales , you can apply strong statistical hypothesis tests to address your research aims.

Pros and cons of closed-ended questions

Well-designed closed-ended questions are easy to understand and can be answered quickly. However, you might still miss important answers that are relevant to respondents. An incomplete set of response items may force some respondents to pick the closest alternative to their true answer. These types of questions may also miss out on valuable detail.

To solve these problems, you can make questions partially closed-ended, and include an open-ended option where respondents can fill in their own answer.

Open-ended questions

Open-ended, or long-form, questions allow respondents to give answers in their own words. Because there are no restrictions on their choices, respondents can answer in ways that researchers may not have otherwise considered. For example, respondents may want to answer “multiracial” for the question on race rather than selecting from a restricted list.

  • How do you feel about open science?
  • How would you describe your personality?
  • In your opinion, what is the biggest obstacle for productivity in remote work?

Open-ended questions have a few downsides.

They require more time and effort from respondents, which may deter them from completing the questionnaire.

For researchers, understanding and summarizing responses to these questions can take a lot of time and resources. You’ll need to develop a systematic coding scheme to categorize answers, and you may also need to involve other researchers in data analysis for high reliability .

Question wording can influence your respondents’ answers, especially if the language is unclear, ambiguous, or biased. Good questions need to be understood by all respondents in the same way ( reliable ) and measure exactly what you’re interested in ( valid ).

Use clear language

You should design questions with your target audience in mind. Consider their familiarity with your questionnaire topics and language and tailor your questions to them.

For readability and clarity, avoid jargon or overly complex language. Don’t use double negatives because they can be harder to understand.

Use balanced framing

Respondents often answer in different ways depending on the question framing. Positive frames are interpreted as more neutral than negative frames and may encourage more socially desirable answers.

Positive frame Negative frame
Should protests of pandemic-related restrictions be allowed? Should protests of pandemic-related restrictions be forbidden?

Use a mix of both positive and negative frames to avoid research bias , and ensure that your question wording is balanced wherever possible.

Unbalanced questions focus on only one side of an argument. Respondents may be less likely to oppose the question if it is framed in a particular direction. It’s best practice to provide a counter argument within the question as well.

Unbalanced Balanced
Do you favor…? Do you favor or oppose…?
Do you agree that…? Do you agree or disagree that…?

Avoid leading questions

Leading questions guide respondents towards answering in specific ways, even if that’s not how they truly feel, by explicitly or implicitly providing them with extra information.

It’s best to keep your questions short and specific to your topic of interest.

  • The average daily work commute in the US takes 54.2 minutes and costs $29 per day. Since 2020, working from home has saved many employees time and money. Do you favor flexible work-from-home policies even after it’s safe to return to offices?
  • Experts agree that a well-balanced diet provides sufficient vitamins and minerals, and multivitamins and supplements are not necessary or effective. Do you agree or disagree that multivitamins are helpful for balanced nutrition?

Keep your questions focused

Ask about only one idea at a time and avoid double-barreled questions. Double-barreled questions ask about more than one item at a time, which can confuse respondents.

This question could be difficult to answer for respondents who feel strongly about the right to clean drinking water but not high-speed internet. They might only answer about the topic they feel passionate about or provide a neutral answer instead – but neither of these options capture their true answers.

Instead, you should ask two separate questions to gauge respondents’ opinions.

Strongly Agree Agree Undecided Disagree Strongly Disagree

Do you agree or disagree that the government should be responsible for providing high-speed internet to everyone?

You can organize the questions logically, with a clear progression from simple to complex. Alternatively, you can randomize the question order between respondents.

Logical flow

Using a logical flow to your question order means starting with simple questions, such as behavioral or opinion questions, and ending with more complex, sensitive, or controversial questions.

The question order that you use can significantly affect the responses by priming them in specific directions. Question order effects, or context effects, occur when earlier questions influence the responses to later questions, reducing the validity of your questionnaire.

While demographic questions are usually unaffected by order effects, questions about opinions and attitudes are more susceptible to them.

  • How knowledgeable are you about Joe Biden’s executive orders in his first 100 days?
  • Are you satisfied or dissatisfied with the way Joe Biden is managing the economy?
  • Do you approve or disapprove of the way Joe Biden is handling his job as president?

It’s important to minimize order effects because they can be a source of systematic error or bias in your study.

Randomization

Randomization involves presenting individual respondents with the same questionnaire but with different question orders.

When you use randomization, order effects will be minimized in your dataset. But a randomized order may also make it harder for respondents to process your questionnaire. Some questions may need more cognitive effort, while others are easier to answer, so a random order could require more time or mental capacity for respondents to switch between questions.

Step 1: Define your goals and objectives

The first step of designing a questionnaire is determining your aims.

  • What topics or experiences are you studying?
  • What specifically do you want to find out?
  • Is a self-report questionnaire an appropriate tool for investigating this topic?

Once you’ve specified your research aims, you can operationalize your variables of interest into questionnaire items. Operationalizing concepts means turning them from abstract ideas into concrete measurements. Every question needs to address a defined need and have a clear purpose.

Step 2: Use questions that are suitable for your sample

Create appropriate questions by taking the perspective of your respondents. Consider their language proficiency and available time and energy when designing your questionnaire.

  • Are the respondents familiar with the language and terms used in your questions?
  • Would any of the questions insult, confuse, or embarrass them?
  • Do the response items for any closed-ended questions capture all possible answers?
  • Are the response items mutually exclusive?
  • Do the respondents have time to respond to open-ended questions?

Consider all possible options for responses to closed-ended questions. From a respondent’s perspective, a lack of response options reflecting their point of view or true answer may make them feel alienated or excluded. In turn, they’ll become disengaged or inattentive to the rest of the questionnaire.

Step 3: Decide on your questionnaire length and question order

Once you have your questions, make sure that the length and order of your questions are appropriate for your sample.

If respondents are not being incentivized or compensated, keep your questionnaire short and easy to answer. Otherwise, your sample may be biased with only highly motivated respondents completing the questionnaire.

Decide on your question order based on your aims and resources. Use a logical flow if your respondents have limited time or if you cannot randomize questions. Randomizing questions helps you avoid bias, but it can take more complex statistical analysis to interpret your data.

Step 4: Pretest your questionnaire

When you have a complete list of questions, you’ll need to pretest it to make sure what you’re asking is always clear and unambiguous. Pretesting helps you catch any errors or points of confusion before performing your study.

Ask friends, classmates, or members of your target audience to complete your questionnaire using the same method you’ll use for your research. Find out if any questions were particularly difficult to answer or if the directions were unclear or inconsistent, and make changes as necessary.

If you have the resources, running a pilot study will help you test the validity and reliability of your questionnaire. A pilot study is a practice run of the full study, and it includes sampling, data collection , and analysis. You can find out whether your procedures are unfeasible or susceptible to bias and make changes in time, but you can’t test a hypothesis with this type of study because it’s usually statistically underpowered .

If you want to know more about statistics , methodology , or research bias , make sure to check out some of our other articles with explanations and examples.

  • Student’s  t -distribution
  • Normal distribution
  • Null and Alternative Hypotheses
  • Chi square tests
  • Confidence interval
  • Quartiles & Quantiles
  • Cluster sampling
  • Stratified sampling
  • Data cleansing
  • Reproducibility vs Replicability
  • Peer review
  • Prospective cohort study

Research bias

  • Implicit bias
  • Cognitive bias
  • Placebo effect
  • Hawthorne effect
  • Hindsight bias
  • Affect heuristic
  • Social desirability bias

A questionnaire is a data collection tool or instrument, while a survey is an overarching research method that involves collecting and analyzing data from people using questionnaires.

Closed-ended, or restricted-choice, questions offer respondents a fixed set of choices to select from. These questions are easier to answer quickly.

Open-ended or long-form questions allow respondents to answer in their own words. Because there are no restrictions on their choices, respondents can answer in ways that researchers may not have otherwise considered.

A Likert scale is a rating scale that quantitatively assesses opinions, attitudes, or behaviors. It is made up of 4 or more questions that measure a single attitude or trait when response scores are combined.

To use a Likert scale in a survey , you present participants with Likert-type questions or statements, and a continuum of items, usually with 5 or 7 possible responses, to capture their degree of agreement.

You can organize the questions logically, with a clear progression from simple to complex, or randomly between respondents. A logical flow helps respondents process the questionnaire easier and quicker, but it may lead to bias. Randomization can minimize the bias from order effects.

Questionnaires can be self-administered or researcher-administered.

Researcher-administered questionnaires are interviews that take place by phone, in-person, or online between researchers and respondents. You can gain deeper insights by clarifying questions for respondents or asking follow-up questions.

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

Bhandari, P. (2023, June 22). Questionnaire Design | Methods, Question Types & Examples. Scribbr. Retrieved July 16, 2024, from https://www.scribbr.com/methodology/questionnaire/

Is this article helpful?

Pritha Bhandari

Pritha Bhandari

Other students also liked, survey research | definition, examples & methods, what is a likert scale | guide & examples, reliability vs. validity in research | difference, types and examples, get unlimited documents corrected.

✔ Free APA citation check included ✔ Unlimited document corrections ✔ Specialized in correcting academic texts

Banner

HOW to Find Research Instruments

Books about research.

Cover Art

Information Literacy Coordinator

Profile Photo

What are Research Instruments?

Research instruments are tools used to collect, measure, and analyze data related to your subject.

Research instruments can be  tests ,  surveys ,  scales ,  questionnaires , or even  checklists .

To assure the strength of your study, it is important to use previously validated instruments!

Finding Research Instruments

Sage Research Methods

This database contains information suited to all levels of researchers, from undergrads starting their first projects to the most senior faculty. It contains books, reference works, case studies, sample datasets, and videos. There is everything a researcher needs to design and execute a research project. 

You can explore the Methods Map in this database for guidance on:

  • Designing a research project
  • Quantitative methods design
  • Qualitative methods design
  • The practice of data analysis
  • and more...

PsycINFO 1887-Current (EbscoHost)  

  • This APA database contains useful information on Tests & Measures.                                                                     
  • In searching, opt for the Tests & Measures selection to retrieve articles with relevant tests and measures
  • Also refer to this LINK for more details about using PsycInfo
  • Link to some educational resources on research instruments
  • Selecting, developing, and evaluating research instruments

BOOKS from the ZU Library

Cover Art

Developing Research Instruments

Suggested sources of information:

  • Electronic Questionnaires Design and Implementation
  • Fundamental issues in questionnaire design

Instrument Development : Sage Research Methods                                     

Tips for developing and testing questionnaires

Using Cronbach’s Alpha When Developing and Reporting Research Instruments

  • Magowe, M. K. M. (2012). Procedures for an instrument development study : The Botswana experience: Research instrument development procedures.  International Nursing Review,  59 (2), 281-288.  https://doi.org/10.1111/j.1466-7657.2011.00950.x
  • Zhang, H., & Schuster, T. (2018). Questionnaire instrument development in primary health care research : A plea for the use of Bayesian inference.  Canadian Family Physician,  64 (9), 699-700.

Assessing the Reliability and Validity of Research Instruments

It is important to assess an instrument's validity and reliability before you try to obtain its full text.

  • Open  this link  for information on  How to Determine the Validity and Reliability of an Instrument
  • Article: Mohamad, M. M., Sulaiman, N. L., Sern, L. C., & Salleh, K. M. (2015). Measuring the validity and reliability of research instruments.  Procedia-Social and Behavioral Sciences ,  204 , 164-171.  https://doi.org/10.1016/j.sbspro.2015.08.129
  • Chapter: Stapleton, L. M. (2019). In Hancock G. R., Stapleton L. M. and Mueller R. O.(Eds.),  The Reviewer’s guide to quantitative methods in the social sciences   (2nd ed.). Routledge. https://doi.org/10.4324/9781315755649. Chapter 35

Where you find that data depends on whether the instrument is "published" or "unpublished." 

Published Instruments

Published means commercially published, and that the instrument is typically available for sale. you can find reviews of many published instruments, including validity and reliability data, in  mental measurements yearbook , unpublished instruments, unpublished means that the instrument has not been commercially published.  if the instrument is   unpublished ,   contact the author directly., you may be able to find the full text of unpublished instruments using the  library's databases..

  • If you find the full text, read the  permission terms  to determine if it is available for reuse or if you will need to contact the author/publisher.
  • Look for the author's email address or phone number to contact them, also letting them know that you are a student.
  • If the email bounces back or phone number doesn't work, search for their institution affiliation as this may lead to their contact information. 
  • Ask your professor or  a librarian  for help! They might be able to help. 
  • If you have tried all of the above and still cannot locate the author, see if the author has any co-authors (in other papers) that you can contact.
  • Contact authors of articles that mention the instrument you are looking for, and ask them how they obtained permission.
  • Last Updated: Aug 31, 2022 3:49 PM
  • URL: https://zu.libguides.com/c.php?g=1210895

research instrument used in qualitative

Research Instruments

  • Resources for Identifying Instruments
  • Assessing Instruments
  • Obtaining the Full Instrument
  • Getting Help

What are Research Instruments?

A research instrument is a tool used to collect, measure, and analyze data related to  your subject.

Research instruments can  be tests , surveys , scales ,  questionnaires , or even checklists .

To assure the strength of your study, it is important to use previously validated instruments!

Getting Started

Already know the full name of the instrument you're looking for? 

  • Start here!

Finding a research instrument can be very time-consuming!

This process involves three concrete steps:

research instrument used in qualitative

It is common that sources will not provide the full instrument, but they will provide a citation with the publisher. In some cases, you may have to contact the publisher to obtain the full text.

Research Tip :  Talk to your departmental faculty. Many of them have expertise in working with research instruments and can help you with this process.

  • Next: Identifying a Research Instrument >>
  • Last Updated: Aug 27, 2023 9:34 AM
  • URL: https://guides.library.duq.edu/researchinstruments

U.S. flag

A .gov website belongs to an official government organization in the United States.

A lock ( ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

  • Collaborating Center for Questionnaire Design and Evaluation Research
  • Question Evaluation
  • Participate
  • CCQDER Products

National Center for Health Statistics

Other research methods.

  • Researchers from the Collaborative Center for Questionnaire Design and Evaluation Research (CCQDER) design and evaluate survey questions and questionnaires.
  • CCQDER uses a variety of evaluation methods, including focus groups, ethnographic interviews, usability testing, and web probing.
  • Project goals and timelines can inform which methods are best.

A person answering survey questions on a laptop.

About CCQDER additional research methods

Cognitive interviewing is a popular method for evaluating survey questions . In addition to cognitive interviewing, Collaborative Center for Questionnaire Design and Evaluation Research (CCQDER) researchers can also use several other methods to evaluate questions. These other methods may be used alongside of or instead of cognitive interviewing.

The goals and timeline of each project can help inform which other research methods are used. Some methods, like focus groups and ethnographic interviews, are used primarily during the questionnaire design phase. Other methods enhance cognitive interviewing during the evaluation phase.

Resource‎

Focus groups.

Focus groups, or group discussions, are typically used during the question or questionnaire design phase of a project. These groups allow researchers to explore how potential respondents understand and talk about specific topics.

CCQDER researchers rely on the group dynamics and conversation that focus groups provide to get information about—

  • Particular words or phrases respondents use to describe facts or events
  • Which topics may be sensitive or burdensome

Focus groups are often combined with individual activities, such as short surveys or free-listing tasks. Individual activities can provide even more information about potential respondents' perceptions and understanding of the topic areas being studied.

Ethnographic interviews

Ethnographic interviews are one-on-one interviews that follow a flexible plan. These interviews are conducted during the question and questionnaire design phase of a project. Like focus groups, they allow CCQDER researchers to understand how potential respondents think and speak about a particular topic. This allows researchers to write questions about complex topics in ways that respondents understand.

The one-on-one nature of these interviews allows researchers to get a much deeper understanding about the way that potential respondents think and speak about a particular topic and how their position in society, personal characteristics and experiences affect how they understand the subject under investigation.

Usability testing

Usability testing involves observing how people interact with a questionnaire or other data collection instrument. When evaluating survey instruments that respondents will complete on their own, CCQDER often will combine usability testing with cognitive interviewing. Whether it's a pen and paper or web-based instrument, CCQDER researchers are interested in understanding how—

  • Respondents interact with a questionnaire
  • Their interactions affect their response processes

During usability testing, CCQDER researchers will typically observe these interactions and ask the respondent about them.

Web probing

Web probes are questions designed to draw out information about how respondents understand, think about, and respond to the questions that are being evaluated. Web probes encourage the respondent to write freely (open-ended) or require them to select from a list of responses (closed-ended). These probes are embedded in a web questionnaire after the question or questions being evaluated.

CCQDER is one of the world's leading research groups in developing and using web probes. While CCQDER has experience using and analyzing both open- and closed-ended probes, it focuses on close-ended probes. CCQDER uses closed-ended web probes developed directly from the findings of cognitive interviews.

Closed-ended web probes produce results that can be counted or measured (quantitative results). Quantitative results can be analyzed alongside the information, ideas, and observations from cognitive interviews (qualitative results). This allows for an in-depth, mixed-method approach that lets CCQDER researchers identify the distribution of interpretations and sources of measurement error. Researchers can make these identifications across a survey's intended population and within that population's subgroups.

This allows researchers to learn if certain groups are more likely than others to use out-of-scope interpretations beyond those intended by the researchers. Certain groups may include respondents with different education levels or from different areas of the country. Out-of-scope interpretations by certain groups could affect the accuracy of the data collected from the final survey.

CCQDER uses its semi-annual Research and Development Survey to evaluate questions using this innovative method.

Contact us‎

NCHS collects, analyzes, and disseminates timely, relevant, and accurate health data and statistics to inform the public and guide program and policy decisions to improve our nation’s health.

Collaborating Center for Questionnaire Design and Evaluation Research (CCQDER)

  • Open access
  • Published: 12 July 2024

Unlocking potential: a qualitative exploration guiding the implementation and evaluation of professional role substitution models in healthcare

  • Rumbidzai N. Mutsekwa   ORCID: orcid.org/0000-0001-8200-3026 1 , 2 , 3 ,
  • Katrina L. Campbell   ORCID: orcid.org/0000-0002-4479-1284 3 , 4 , 5 ,
  • Russell Canavan   ORCID: orcid.org/0000-0002-0155-3219 6 ,
  • Rebecca L. Angus   ORCID: orcid.org/0000-0002-1839-9641 1 , 7 ,
  • Liza-Jane McBride   ORCID: orcid.org/0000-0002-6336-5934 8 &
  • Joshua M. Byrnes   ORCID: orcid.org/0000-0001-6562-711X 3 , 4  

Implementation Science Communications volume  5 , Article number:  73 ( 2024 ) Cite this article

108 Accesses

3 Altmetric

Metrics details

As role substitution models gain prominence in healthcare, understanding the factors shaping their effectiveness is paramount. This study aimed to investigate factors that impact the implementation and performance evaluation of professional role substitution models in healthcare, with a focus on understanding the variables that determine their success or failure in adoption, execution, continuity, and outcomes.

The exploratory qualitative study used semi-structured interviews with key opinion leaders, decision makers, facilitators, recipients, and frontline implementers, who had influence and involvement in the implementation of professional role substitution models. Data analysis was guided by the Consolidated Framework for Implementation Research (CFIR).

Between November 2022 and April 2023, 39 stakeholders were interviewed. Factors influencing implementation and evaluation of allied health professional role substitution models of care aligned with the five core CFIR domains (innovation, outer setting, inner setting, individuals, implementation process) and outcome domain incorporating implementation and innovation outcomes. The six themes identified within these CFIR domains were, respectively; i) Examining the dynamics of innovation catalysts, evidence, advantages, and disadvantages; ii) Navigating the complex landscape of external factors that influence implementation and evaluation; iii) Impact of internal structural, political, and cultural contexts; iv) The roles and contributions of individuals in the process; v) Essential phases and strategies for effective implementation; and vi) The assessment of outcomes derived from allied health professional role substitution models.

Conclusions

The study highlights the complex interplay of contextual and individual factors that influence the implementation and performance evaluation of professional role substitution models. It emphasises the need for collaboration among diverse stakeholders to navigate the challenges and leverage the opportunities presented by expanded healthcare roles. Understanding these multifaceted factors can contribute to the development of an empowered workforce and a healthcare system that is more efficient, effective, safe, and sustainable, ultimately benefiting patients.

Peer Review reports

Contributions to literature

• There is limited understanding of the complex interplay of contextual and individual factors that influence implementation and performance evaluation of professional role substitution models of care.

• This study provides comprehensive guidance on successful implementation and evaluation of new models of care which influences efficient use of resources in healthcare.

• This study contributes to recognised gaps in literature, seeking to demonstrate value proposition of professional role substitution models of care. This study has identified outcome measures that can determine the successful implementation and impact of these models of care

The healthcare sector plays a crucial role in ensuring the well-being of individuals and society, but it is facing challenges due to a growing and ageing population. The demand for high-quality healthcare has increased significantly, while the shortage of healthcare workers has become a pressing concern [ 1 , 2 ]. Workforce reforms are now being prioritised in healthcare to shape the future of healthcare delivery. These reforms include initiatives to increase the number of healthcare workers, enhance the quality and duration of healthcare education and training, and diversify the healthcare workforce.

One key strategy to address healthcare challenges is the expanded scope of practice for non-medical healthcare professionals [ 3 ]. This expansion entails a discrete knowledge and skill base beyond the recognised scope of practice within a specific jurisdiction's regulatory framework [ 4 ]. It empowers healthcare practitioners such as nurse practitioners, allied health professionals, and physician assistants to practice to the full extent of their training and education, or to extend their scope of practice beyond traditional boundaries [ 5 , 6 , 7 ]. Consequently, they can perform a broader range of tasks, including those previously reserved for medical doctors.

Professional role substitution models have improved patients' access to healthcare services [ 8 , 9 , 10 ]. Moreover, there is a growing body of evidence suggesting that these alternative healthcare delivery models can provide safe and effective care that patients find acceptable. Nurse practitioners and advanced nurses in the US, Canada, the UK, and Australia expand primary care roles, including diagnosis, prescribing, patient education, managing long-term conditions, and minor surgeries [ 9 , 11 , 12 , 13 ]. Physician assistants (PAs) in countries like the US, Canada, and the Netherlands work closely with physicians, conducting assessments, diagnosing, treating common illnesses, and providing patient education. PAs improve healthcare access, especially in underserved and rural areas with physician shortages [ 14 , 15 , 16 ].

In developing countries with limited healthcare resources, professional role substitution models are vital for addressing shortages of skilled healthcare providers and improving access to essential services. For instance, in sub-Saharan Africa, task shifting from physicians to nurses and community health workers addresses the scarcity of skilled providers [ 17 , 18 ]. Community health workers, trained to deliver basic healthcare services and education, play crucial roles in preventive and promotive interventions, particularly in rural and underserved areas [ 18 ]. Nurse-led clinics have also proven successful in delivering comprehensive primary care services, such as antenatal care and family planning, alleviating pressure on strained healthcare systems [ 19 , 20 , 21 ].

Allied health professionals, encompassing disciplines such as speech pathology, pharmacy, dietetics, physiotherapy, occupational therapy, radiography, sonography, psychology, and social work, are increasingly vital in diverse healthcare settings. Supported by mounting evidence of their effectiveness, their role continues to expand [ 10 , 22 ]. Despite substantial growth, particularly notable in Australia where they rank as the second-largest healthcare group, [ 23 ] the implementation of professional role substitution within allied health is relatively new compared to fields like nursing and physician assistants [ 24 ].

The successful implementation of all professional role substitution models including allied health is complex and contingent on various factors which are not currently well understood or defined [ 10 , 22 , 25 ]. To ensure success, it is essential to consider the impact on patients, healthcare professionals, and the healthcare system [ 25 ]. This must be approached from a multi-stakeholder perspective, involving experts in the field, key opinion leaders, healthcare leaders, decision makers, policy makers, recipients, and frontline implementers.

Research into the expanded scope of practice within allied health disciplines, including implementation and performance evaluation, is crucial [ 6 , 23 , 25 , 26 ]. Previous studies have highlighted patients' perceptions and experiences of healthcare quality in role substitution models [ 27 , 28 ]. While clinicians express support for performance evaluation, there's a gap between support and effective implementation [ 29 ]. There's also a lack of agreed-upon approaches for measuring performance [ 25 , 29 ]. Collaborative efforts involving multiple stakeholders are essential for understanding robust evaluation methods and optimising alternative models of care for healthcare transformation and sustainability [ 25 ].

To address this gap in knowledge and practice, this study aimed to describe the individual and contextual factors that influence the implementation and performance evaluation of allied health professional role substitution models from a multi-stakeholder perspective. Furthermore, the study aimed to identify outcome measures that can demonstrate the successful implementation and impact of these models of care.

Study approach and design

An exploratory qualitative approach was used to explore expectations, perceptions, and experiences of stakeholders involved in the implementation and performance evaluation of professional role substitution models of care. Semi-structured interviews were chosen as the primary method of data collection to allow for flexible exploration of specific topics and issues, maximising the richness of the data [ 30 ]. The study adhered to the Consolidated Criteria for Reporting Qualitative Research (COREQ) guidelines [ 31 ]. Please see Additional file 1

Study setting

This study was conducted within the public healthcare system of the State of Queensland, Australia. This comprises 16 hospital and health services and approximately 35,000 allied health professionals [ 32 ]. Queensland initiated an allied health strategy in 2014 to expand professionals' scope of practice, resulting in the establishment of 133 distinct models of care by 2019 [ 24 , 33 , 34 ]. Examining this system offers valuable insights into implementing and evaluating professional role substitution models, providing practical understanding within a specific healthcare context.

Study participants and recruitment

A purposeful sampling strategy was employed to recruit key stakeholders at various levels of the healthcare system who were involved in some way in the implementation and performance evaluation of allied health professional role substitution models of care. Participants included experts in the field, key opinion leaders, decision makers, recipients, and frontline implementers, implementation facilitators and support teams. A sampling matrix was used to consider factors such as location, affiliation, organisational role, tenure, and profession ensuring diversity and representation across the different dimension of the healthcare system. While a specific target number of participants was not predetermined, our aim was to achieve saturation in the sample, ensuring comprehensive coverage of perspectives and experiences relevant to our research objectives. Email invitations were sent to potential participants/participant groups, along with study information and consent forms. Those who agreed to participate contacted the principal investigator to arrange a suitable interview time.

Positionality of researchers

The research team comprised individuals with diverse backgrounds and roles, including experts in professional role substitution, health services research, economics, qualitative study methodology, and healthcare management.

This study was performed in line with the principles of the Declaration of Helsinki with approval granted by Gold Coast Hospital and Health Service (HREC/2020/QGC/62104) and Griffith University (GU Ref No: 2020/876). All participants provided written informed consent.

Data collection

An interview guide was developed by the research team to ensure coverage of the study aims and objectives (Additional file 2). The guide was pilot tested with three eligible participants, resulting in minor wording adjustments for clarity. Interviews were conducted either face-to-face or via video conferencing with only interviewer and participant present. The semi-structured interviews were designed to elicit open-ended responses from participants, with the interviewer using prompts and probing techniques as needed. Data collection continued until data saturation was reached, indicating that no new themes were emerging [ 35 ]. All interviews were audio-recorded, transcribed, and supplemented with field notes for additional context and consistency. Each participant was allocated an anonymous identifier, comprising their participant number along with a descriptor of their role or professional background. (e.g., P34, Workforce and Education). Participants were offered the opportunity to check their transcript.

Data analysis and interpretation

Descriptive statistics were used to analyse demographic data, such as participants' time in their current role, age, gender, and education level. Exploration of contextual influences on implementation and performance evaluation was guided by the Consolidated Framework for Implementation Research (CFIR) [ 36 ]. The CFIR is a comprehensive framework that focuses on understanding and improving the implementation and evaluation of health innovations. Its adaptability enables integration into various contexts, fostering analysis and facilitating cross-study comparisons. This versatility supports a systematic approach to evaluating implementation processes and outcomes, thereby enriching our understanding of innovation dynamics across diverse settings [ 36 ].

It consists of six domains: 1. Innovation domain (the model of care being implemented), 2. Outer setting (the healthcare system in which the inner setting exists) 3. Inner setting (the site in which the model of care is implemented e.g., hospital) 4. Individuals (the roles and characteristics of individuals involved in the implementation process), 5. Implementation process (the activities and strategies used to implement the model of care), 6. Implementation outcomes (perceptions and measures of implementation success or failure), and Innovation outcomes (outcomes that capture success or failure of model of care) [ 36 , 37 , 38 ].

A reflexive thematic approach was taken for qualitative analysis [ 39 ]. The analysis began deductively with codes derived from the CFIR, followed by inductive coding to identify additional categories. These codes were assigned using CFIR definitions, inclusion/exclusion criteria, and appropriate quote examples. NVivo V10 software (QSR International Ltd.) was used to facilitate data management.

Investigator triangulation was employed, with the principal researcher (R.N.M) coding all interviews and 20% of the interviews coded by a second researcher (R.L.A) to enhance reliability and provide different perspectives [ 40 ]. All authors participated in summarising codes prioritised for analysis and interpreting the results. A matrix was created to compare the ratings of each CFIR construct, focusing on any differences among stakeholders. Data extracts were selected to illustrate themes and subthemes, incorporating multiple perspectives for interpretation.

Study population

A total of 39 stakeholders from various hospital and health services across Queensland were interviewed. The stakeholders represented a broad spectrum of positions and roles within the healthcare system, categorised into eight groups: allied health clinicians, medical practitioners/general practitioners, nursing staff, allied health leadership, hospital and health services/statewide leadership, recipients, implementation support personnel, workforce and education. Table 1 provides demographic details of the participants.

Participants had been in their roles on average 11 years, (range 1-27 years). Interviews had an average duration of 32 minutes (range 15-59 minutes). Five interviews were conducted face to face with the remainder ( n =34) conducted through video conferencing.

Six themes were identified which aligned with the five CFIR domains and the outcomes domain. Twenty-seven underlying constructs and subconstructs of the CFIR were identified as factors influencing implementation of professional role substitution in our analysis. Ten constructs were identified in the implementation and innovation outcome categories. Main domains and constructs are illustrated in Fig. 1 .

figure 1

Key implementation and evaluation constructs for professional role substitution models of care

Innovation domain

Examining the dynamics of innovation catalysts, evidence, advantages, and disadvantages in allied health professional role substitution models of care.

The following section delineates the three primary constructs aligning with CFIR domains and an additional domain, namely "relative disadvantage." These constructs were identified from the perspectives of participants regarding professional role substitution models of care as an innovative approach.

Innovation source

Participants recognised healthcare system strain due to workforce shortages, rising costs, and increased needs with policies now prioritising workforce reform as a key healthcare strategy. The 2006 Australian Productivity Commission review, focusing on optimising scope, competencies, and job redesign was frequently cited by interviewees as the catalyst for change. The Ministerial Taskforce on Health Practitioner Expanded Scope of Practice (Queensland), alongside similar taskforces nationwide, played a vital role in implementing allied-health professional role substitution models of care. “There were a broad range of stakeholders involved in the task force across Queensland Health and external to Queensland in 2014. There was a number of recommendations in the report with overall endorsement from the Minister.” (P34, Workforce and Education)

Furthermore, the Allied Health Professions' Office supported these efforts by funding care models, addressing legislative barriers, developing training, supporting research, monitoring progress, and sharing achievements. A participant explained, ‘The office was charged with implementing the recommendations and to test these models of care. Particularly things like requesting and interpreting forms for diagnostic imaging and requesting pathology.” ( P34, Workforce and Education)

Evidence base

Participants expressed varying perspectives on the evidence base for professional role substitution models of care. Some noted a reliance on grey literature or information from pilots, highlighting the limited evidence supporting certain models. Conversely, others believed the evidence base was robust and questioned the need for further piloting. “It should be business as usual and that’s something we’ve tried to promote where we’ve got evidence from other jurisdictions and internationally. There should then be efforts to implement and try and replicate those results and take it to scale.” (P34, Workforce and Education).

Established services in other countries and professions influenced the implementation in Australia. Clinician leads or facilitators with prior experience in allied health models were identified as key enablers of this process. One participant shared their experience stating, “I was involved with that over there in the UK and so I came with that mentality to Queensland. When I worked as a fellow, I was surprised that there wasn’t that model, and I advocated for it and was told we don’t do that here. So, we ended up running extra clinics as fellows to see the long-wait patients when I knew that back in the UK it would have been [allied health discipline ].” (P19, Medical Specialist)

Relative advantage

Participants, healthcare professional and patients alike identified several advantages of allied health professional role substitution models. These models improved access to care, particularly benefiting underserved areas, and boosted efficiency by “streamlining decision-making and minimising duplication” (P19, Medical specialist). A patient shared their positive experience, stating, “If anything, I thought I was really special. I got pushed ahead really. I didn’t have to wait so long, and I wasn’t made to feel silly for my symptoms and they were investigated. The whole experience was positive.” ( P39, Consumer/Recipient)

While considered cost-effective by those interviewed due to reduced reliance on specialists in resource-limited settings, many highlighted the need for further cost-effectiveness data. One participant mentioned,

“You can get comparable or sometimes a better service at a lower cost using alternate models of care.” (P17, Medical Specialist)

Participants indicated that these models enabled allied health professionals to provide comprehensive, patient-centred care, enhancing overall healthcare experiences and patient satisfaction. Another viewpoint shared was, “ It's about getting patients to clinicians with holistic skill sets rather than just the medical model. Traditionally, patients wait a long time to see a medical professional, only to be referred back to the same clinician “, ( P15, Implementation Support). Additionally, clinicians working in these roles noted , “So, they’re kind of getting that one stop assessment, where the speech pathologist looks at the functional component, as well as pathology or organic disease” (P13, Allied Health Clinician). Moreover, they promoted professional growth, job satisfaction, and workforce retention through expanded roles and skill development opportunities, fostering collaboration among healthcare professionals from various disciplines for improved patient outcomes. One individual expressed “That responsibility and that extra challenge for me is where I get the buzz. (P 12, Allied Health Clinician )

Relative disadvantage

In addition to the benefits of professional role substitution in healthcare, participants emphasised other key factors. Patient safety and care quality surfaced as paramount concerns. A participant with workforce and education background stated, “There was a lot of the discussion and particularly the negative media coverage around the model of care. I was quite driven to answer the questions, or the concerns raised by the health professionals around safety .” (P29, Workforce and Education)

The imperative of ensuring skill, competence, and appropriate clinical governance was strongly emphasised. In some settings, participants flagged the potential for resistance and conflicts with traditional providers and organisations, driven by apprehensions about expertise encroachment, de-skilling, and role ambiguity. A Medical Specialist (P19) highlighted this, “The risk is that if you promote therapists from being treating therapists to being screening and treating therapists, you’re on the risk of deskilling your (medical) workers.”

Building public and patient trust, especially in unfamiliar models, highlighted the importance of transparent communication and educational efforts, as noted by both consumers and healthcare professionals. A patient shared, “I really didn’t know what to expect because I hadn’t been to a clinic like that before and I didn’t know what they were going to do”. (P39, Consumer/recipient). A healthcare professional suggested, “ Another barrier is patient perception, especially if they are expecting to see a doctor” but went on to add , “In my experience this has often not been the case with patients often reassured once they have had a thorough assessment ” (P13, Allied Health Clinician) Initial challenges in interaction with General Practitioners (GPs), were also highlighted with one participant noting, “ See the problems at the beginning where the GPs would ring up and say, I wanted a specialist opinion, and I got a physiotherapist. But once they were educated, those complaints dropped off especially when the patient satisfaction scores were high” (P19, Medical Specialist).

The implementation of these models often demands additional investments in training and supervision, with a consideration of their economic and logistical impact on the healthcare system required. Lastly, “striking a delicate balance between expanded scopes and core responsibilities” (P 25, Allied Health Leadership) is essential. Another participant noted, “ It is also worth considering the amount of time it takes for this training and to set up these roles. It is also important to consider the cost. Once you have a model of care set up well, what’s my sustainability plan for this model in relation to, succession planning, leave management, etcetera ? “(P15, Implementation Support)

Outer setting

Navigating the complex landscape of external factors that influence implementation and evaluation of allied health professional role substitution models of care, partnerships and connections.

Collaborative care teams and strong referral networks emerged as crucial elements for successful role substitution practice. Participants emphasised the importance of interdisciplinary collaboration, where professionals from various disciplines worked together to provide holistic patient care. Furthermore, partnerships with specialists, hospitals, community resources, and primary care facilities were highlighted as essential for ensuring seamless transitions and continuity of care. This was articulated by one GP, (P7) “I think for me and my style of medicine, it’s helpful. I really enjoyed that sort of team, that real MDT and holistic approach to patient care.”

Policies and laws

Implementing professional role substitution and scope extension may require legal and regulatory adjustments, including redefining boundaries and establishing standards which participants noted as a challenge. Variations across jurisdictions, were highlighted emphasising the need for a national approach to align state and federal policies. An occupational therapist identified legislative barriers stating, “Legislation prevents us from ordering imaging, but we all have local agreements with our departments that enable us to order basic radiology. But we want to be able to order that radiology in our general role as well and potentially expand that into other forms of the imaging down the road. This role has expanded even further in the UK to some of those therapists prescribing and referring people for MRIs and CT scans.” (P11, Allied Health Clinician)

Both allied health clinicians and medical doctors expressed concerns about legal accountability in the event of adverse events or complications in professional role substitution models. Stakeholders, including allied health clinicians, medical doctors, and healthcare leaders, emphasised the importance of assurance of indemnity through health services. “We’re protected by public indemnity in this system. And ultimately the directors are responsible for all the patients, even the ones we don’t directly treat. So that model protected our junior doctors and subsequently protects the therapists as well ” (P19, Medical Specialist). Participants also acknowledged the need for regular training and re-assessment of knowledge and skills for medical professionals but were uncertain about the lack of similar scrutiny and regulation mechanisms for allied health clinicians in professional role substitution roles.

Participants had differing perspectives on funding for new models of care. Implementing professional role substitution models of care often relied on short-term funding and grants to pilot services. A participant with an allied health clinician background highlighted complexities in healthcare funding and incentives, pointing out “General practice won’t make money unless the patient sees the GP. They would need to look at some sort of MBS (Medicare Benefits Schedule) item number so that the practice or hospital and health service can generate money from those expanded roles.” (P10, Allied Health Clinician)

Suggestions were made to review Medicare and activity-based funding structures to provide support for professional role substitution models ensuring their viability. A participant who has supported implementation of a professional role substitution model noted, “That’s also based on the fact that with Activity-Based Funding framework, we have to demonstrate that the model can generate enough activity to be viable and valuable.” (P15, Implementation Support).

Additionally, participants emphasised the importance of funding models that prioritise outcomes rather than specific care delivery mechanisms. A healthcare executive highlighted, “W e don’t purchase models of care. I would like to think that we purchase outcomes, and we are quite agnostic in how health services go about achieving those outcomes. We’ve wanted to make sure that the funding model is enabled and that it’s not a barrier to people trying alternative ways using new and different models to achieve those outcomes that we’re interested in.” (P32, Hospital and Health Services/ State-wide Leadership).

Performance management pressure

Participants acknowledged the challenge of meeting patient waiting time targets set by federal and state governments. This was an enabling factor, with professional role substitution models of care implemented as strategies to reduce specialist outpatient waitlists and improve access to services, aligning with performance targets. “There was a wait list issue for the specialty area. There was a big project to see who else could help see patients and try and reduce the waitlists. They highlighted that the [allied health specialist area clinician] might be something that could help with that.” (P8, Allied Health Clinician).

Inner setting

Impact of internal structural, political, and cultural contexts on the implementation and performance evaluation of allied health professional role substitution models in healthcare, work infrastructure.

Implementing role substitution models had workforce implications, including assessing skills availability and workload management. Sustainability relied on individual commitment, posing threats to the longevity of these models of care. An allied health leader, (P22) noted, “Often the first people you get in are personally passionate about it. It’s hard to find those people all the time, but a succession plan is important for sustainability of extended scope roles.“ These sentiments were echoed by a physician who mentioned, “Workforce and sourcing the right resources and clinicians is something that is a bit of a challenge for the health services moving forward.” (P17, Medical specialist)

Relational connections

Participants identified strong relationships and networks as vital for implementing and sustaining professional role substitution models. Trust between medical doctors and allied health clinicians was essential. As one participant noted: “Most of the time when these models fall down, it’s because the relationships between the allied health and the multidisciplinary team, including the doctors, have broken down. The doctor’s left or there’s been an issue that they couldn’t resolve and then everything falls to pieces.” (P23, Allied Health Leadership) Key roles of advocates and clinical leads were emphasised, but overreliance on individuals was a concern. Building resilience in these models across all levels of leadership was an important consideration as are clear governance structures which include supervision and escalation pathways.

Communications

Effective communication was necessary for high-quality care, patient safety, and collaborative relationships in both implementing and sustaining professional role substitution models. Iterative modifications and a willingness to learn were recognised as important. Collaboration involved shared decision-making, regular communication, and joint management of complex cases. Specialist doctors provided guidance and medical expertise, while allied health professionals contributed their specialised skills including ability to provide holistic care. “We still needed to iron out all of the kinks, so each side still needs to continue to learn from each other. So, I would say it probably took a good 12 to 18 months before we felt like we had a system that was working well for both sides and streamlining the process.” (P11, Allied Health Clinician). Additionally, some participants emphasised transitioning from “substitution-focused to team-based approaches” (P27, General Practitioner and Healthcare Executive), promoting interdisciplinary and transdisciplinary care.

For some participants, professional role substitution raised concerns about autonomy with potential for conflicts among healthcare professionals. Cultivating a collaborative culture, renegotiating traditional hierarchies, and addressing professional dynamics were identified as strategies to enable interprofessional collaboration, promoting innovation and excellence in patient care. However, despite the progress made, some participants expressed reservations about barriers that still exist, even in allied health practitioners performing tasks that were within their scope of practice. One executive leader expressed frustration at the slow pace of change stating, “ It’s an imperative at the moment that we actively promote full scope of practice and give more support for our allied health staff to do extended scope of practice qualifications. So, we have a role to ensure that we have a culture that encourages the new models of care, because just to have the old models of care, it’s not simply sustainable, it’s not sustainable, at all. “ (P36, Hospital and Health Services/ State-wide Leadership)

Another participant, an allied health leader (P22), highlighted the positive impact of professional role substitution on organisational culture and the morale of younger professionals, stating, “It’s good for our culture and gives some sort of energy to the younger professionals. It also flows through to junior doctors particularly working alongside a consultant that already holds these clinicians and models of care in high standard.”

Mission alignment and tension for change

In many organisations, clinical demand drove professional role substitution adoption, facilitated by change management teams and frameworks. Professional role substitution models aligned with healthcare organisational goals and objectives, promoting innovation, equity, and sustainable use of resources. As articulated by a Medical Specialist (P19) “We have a limited number of specialists, and training for medical students and junior doctors hasn't significantly increased to meet demand. With advanced technology and reduced working hours, we need to expand services. Having other clinicians who can treat patients without surgery is invaluable.”

Additionally, participants acknowledged their role in healthcare delivery to underserved communities and advancing health equity in First Nations, rural, and regional areas, “ improving access and preventing, fragmented care ,” (P33, Nursing Health Professional). Furthermore, participants discussed the impact of population growth on surgical waitlists, revealing the pressing need for effective solutions to address increasing demand. An allied health clinician (P11) highlighted the challenges posed by population growth, stating, “ There's been a significant increase in people moving to Queensland now for many years and our surgical wait lists were continuing to grow. So, when I started in this role… the waitlist was almost four years long.”

Participants stressed the strategic importance of expanded scope in advancing organisational objectives. An Allied health leadership participant (P25), emphasised the multifaceted benefits of expanded scope, highlighting its alignment with strategic goals and the need to reassess care delivery models: “Expanded scope hits all the strategic goals really. We need to disinvest in some of the low value care because we know that we’re not getting any outcomes. We also need to look at the impact of these models of care.”

Available resources

Funding for professional role substitution models varied, with some implemented without dedicated funding which posed challenges in attracting skilled clinicians. Stakeholders recognised the benefits of co-locating allied health clinicians and medical doctors for interdisciplinary case discussions but sometimes faced challenges due to high demand for limited space. An allied health clinician (P8) highlighted the impact of dedicated funding on the feasibility and efficiency of implementing such models “They had a certain amount of funding for this project to set it all up. And I think that really made it feasible. So, then we got the right equipment, the right time to set it up. It was a very set process with money attached to it that got it off the ground quicker.”

Access to knowledge and information

Clinicians in extended scope roles actively sought professional development opportunities to expand their skills. Local credentialing and on-the-job training were the norm. A workforce development officer highlighted the rigorous process of credentialing for clinicians in such roles. “Our credentialling package is fairly intense. It takes months and months and months to become credentialed in a first point of contact clinic like this and needs [Health Service] approval before a clinician can work in a space like this.” (P29, Workforce and Education). In contrast to nurse practitioner programs offered by universities and specialised training institutions, formal education programs for allied health professionals were scarce. Many participants recommended development of formalised training and credentialing programs to ensure high quality and safe care. “We’re now in the process of developing our own course here in Australia in collaboration with the university in New South Wales so that we can provide that level of education that we need in these advanced scope roles ” (P11, Allied Health Clinician).

Individuals domain

The roles and contributions of individuals in the implementation of allied health professional role substitution models of care.

The implementation of allied health professional role substitution models of care heavily relies on the engagement of various individuals who play pivotal roles in the process. Through our interviews, participants identified nine key roles integral to the implementation and evaluation of these alternative healthcare delivery models. These roles, aligned with those in the individuals’ domain of the CFIR, encompassed high-level leaders, mid-level leaders, opinion leaders, implementation facilitators, implementation leads, implementation team members, other implementation support, innovation deliverers, and innovation recipients. Our analysis revealed representation across these roles within our study population, demonstrating the diverse range of contributions.

Participants described the characteristics of these individuals, which we analysed based on the Capability, Opportunity, Motivation-Behaviour (COM-B) theoretical behaviour change model integrated into the CFIR framework. This system evaluates individuals' influence on the implementation process across four constructs: Need, Capability, Opportunity, and Motivation. These constructs assess individuals' deficits addressed by the models of care, their interpersonal competence, availability and power, and commitment and motivation in fulfilling their roles respectively.

Participants emphasised the critical role of medical and executive buy-in for the success of these models. Without their support and commitment, implementation efforts often faced significant hurdles. As one participant stated, " Medical and executive buy-in, if they are not supportive, it doesn’t happen " (P34, Workforce and Education). Furthermore, participants highlighted the importance of strong endorsement from medical professionals and the need for active engagement from allied health clinicians and managers to ensure the sustainability of these models. As articulated by another participant, " Allied health clinicians and even up into the level of our managers, there's certain spheres of influence that we have, but to make something like this come together and to be able to make it sustainable, you really need strong medical endorsement and that real commitment to push it " (P12, Allied Health Clinician).

Moreover, participants identified the Allied Health Office as having a crucial role in facilitating implementation. However, they also expressed the need for greater visibility and recognition of successful implementation efforts. As one participant suggested, " The Allied Health Office has a role to play in that. I think we should certainly see more things up in lights, you know, presentations, success stories et cetera and opportunities for these models to be shared and celebrated more widely across the state " (P15, Implementation Support).

A matrix analysis (Table 2 ) provides detailed insights into the roles and characteristics of individuals within different groups/roles. This elucidates their contributions to the successful implementation of professional role substitution models of care, as perceived by the study participants.

Implementation process

Essential phases and strategies for effective implementation of allied professional role substitution healthcare models.

Participants in our study provided insights into key stages necessary for implementing allied health professional role substitution models of care. We analysed their responses and mapped them to constructs in the implementation process domain of the CFIR, finding alignment with five out of the nine constructs. In the planning phase, participants emphasised the importance of conducting needs assessments and developing comprehensive implementation plans to identify gaps, set objectives, and consider resources and stakeholders' roles. One participant described, "At the start-up of our model of care, we had a series of meetings involving all stakeholders... to develop very clear guidance and pathways for how patients would move through these services " (P13, Allied Health Clinician).

Engaging was highlighted as crucial focused on involving diverse stakeholders, from healthcare providers to patients, forming multi-stakeholder teams to ensure a variety of perspectives and support for long-term sustainability. " There were a broad range of stakeholders involved in the task force across Queensland Health and external to Queensland ." (P34, Workforce and Education) In the doing phase models of care often started as pilot projects, with services developing iteratively.

Reflecting and evaluation Participants stressed the importance of building evaluation into the model of care to ensure sustainability and strategic outcomes. However, challenges such as limited time and funding were acknowledged, as one participant stated, "We don't get the time or the funding in my experienc e" (P1, Allied Health Clinician). Lastly , in adapting , participants recognised the need for continuous learning and tailored strategies to the local context, acknowledging the necessity for flexibility in response to evolving healthcare systems. Moreover, strategies to enhance evaluation included dedicated funding, external evaluation to reduce bias, development of performance frameworks, and tailored technology and digital systems allowing data collection and analysis at the point of care. Collaboration with universities and the use of research frameworks and grants were also seen by participants as facilitators to enhance performance measurement.

Implementation and innovation outcomes

The assessment of outcomes derived from allied health professional role substitution models.

In our study, participants highlighted the importance of evaluating healthcare models' success and failure, focusing on both implementation process and innovation outcomes. They identified eight key domains, including implementation aspects such as adoptability, implementability, and sustainability, as well as innovation delivery outcomes like effectiveness, safety, patient-centeredness, healthcare provider experience, access, activity, and economic evaluation. One participant stressed the need for thoughtful measurement, stating , “You do need to think about what you need to measure to prove the value of your service .” (P35, Allied Health Leader). Figure 2 summarises these outcomes and provides examples of measures discussed by participants.

figure 2

Recommended outcomes and examples to measure the impact of professional role substitution models of care

Implementation outcomes

Participants shared diverse perspectives on implementation success for allied health professional role substitution models, with factors like regulatory environment, financing, medical acceptance, stakeholder engagement, and individual characteristics playing key roles. Sustainability was particularly highlighted, as expressed by a participant, “ You need to know that a service that has been implemented is still running after several years ” (Participant 1, Allied Health Clinician).

Innovation outcomes

Participants emphasised specific outcomes in evaluating the impact of allied health professional role substitution models of care. One participant stressed the importance of measuring performance and demonstrating improved access and cost saving, saying, " I think it is important to measure performance and to show that there is improved access and economic benefits. You know, to show that the service is doing what it was intended to do" (P5, Allied Health Clinician). They also highlighted the need to track activity data, with another participant mentioning, "We basically keep data on all of the occasions of service, how many patients are seen within the service, and how many patients are discharged." (Participant 8, Allied Health Clinician).

Healthcare provider experience, including clinician and patient satisfaction, emerged as essential, with one participant suggesting : "Surveying the general practitioners would be a good way of doing it as well, asking if they are happy with the service" (P36, Executive Leader). Continuity of care and patient centredness were also emphasised. "It's actually quite heartening hearing what our patients value and to see if our services line up with that " (P29, Allied Health Clinician). Participants in the study also stressed the importance of safety as a crucial outcome measure in evaluating allied health professional role substitution models of care. One general practitioner (P7) highlighted this by stating, "We need to know we are providing great healthcare to patients. You know that we are reducing harm, not causing harm, and hopefully not missing diagnoses”.

Finally, participants perceived effectiveness as paramount for assessing the success and impact of the model on patient care experiences and health outcomes. One executive leader (P36) expressed: "Forgetting about everything else, the patients' view of whether or not they've been treated adequately to me is the most important. If there are no outcomes with the model of care, the patients won't be satisfied, and they will say so."

This study investigated factors influencing the implementation and performance evaluation of allied health professional role substitution models of care using the CFIR framework. We identified six overarching themes aligned with CFIR domains and outcomes. These themes covered dynamics such as innovation catalysts, evidence, advantages, and disadvantages; external factors affecting implementation and evaluation; internal structural, political, and cultural contexts; roles and contributions of individuals; essential implementation phases and strategies; and assessment of model outcomes. Our analysis identified twenty-seven underlying constructs and subconstructs within the CFIR framework that influence professional role substitution implementation. Additionally, we identified ten key constructs across implementation and innovation outcome categories: adoptability, sustainability, implementability, effectiveness, safety, patient-centeredness, accessibility, healthcare provider experiences, service delivery metrics, and economic evaluations. These findings addressed critical questions regarding factors influencing implementation and methods for assessing the impact of care models. Overall, this study provides a robust framework for implementing and evaluating allied health professional role substitution models, effectively addressing gaps in literature and practice.

Priority areas of focus

While prior studies have demonstrated the potential benefits of these models in terms of providing safe, effective, and cost-efficient care, [ 10 , 11 ] the current research goes further by exploring stakeholders' perceptions and experiences in depth. Grounded in the Consolidated Framework for Implementation Research (CFIR), [ 36 , 37 , 38 ] it explores the multifaceted factors influencing the adoption and integration of these models within healthcare systems.

Healthcare organisations play a significant role in either facilitating or impeding the implementation of professional role substitution models of care [ 41 ]. Along with previous research , this work underscores the significance of supportive organisational cultures, adequate resources, leadership commitment, and medical endorsement as critical factors for the successful adoption of such models [ 42 ]. Conversely, factors like resistance to change, resource limitations, and insufficient infrastructure can pose significant barriers that must be addressed to ensure successful implementation.

Traditional healthcare structures were once considered conducive to advancing medical sciences [ 43 ]. However, recent reviews have highlighted how entrenched organisational cultures and long-held traditions within healthcare settings may now act as barriers to alternative models of practice and hinder improvements in healthcare access for the community [ 41 ]. Consequently, healthcare organisations must proactively assess their readiness for new models and develop strategies to overcome these barriers. Leveraging the constructs and principles identified in the inner setting domain of this study is essential for cultivating a culture that fosters role substitution and innovation in healthcare delivery.

Stakeholders' perceptions and attitudes play a significant role in shaping the success of professional role substitution models of care, influenced by factors like medical buy-in, leadership support, and engagement strategies [ 41 , 44 , 45 ]. Effective stakeholder engagement strategies, alongside tailored training, communication programs and ongoing support mechanisms, emerge as crucial tools for addressing individual concerns and fostering buy-in from all involved parties. These findings align with similar studies in physiotherapy and nursing, emphasising the universal importance of considering individual perspectives in healthcare implementation efforts [ 41 , 44 , 45 , 46 ].

This research emphasises the importance of incorporating perspectives from patients and innovation recipients to enhance the success of healthcare interactions. Integrating these viewpoints strengthens the potential for sustainable adoption of evidence-based innovations, promoting patient-centred care [ 47 , 48 , 49 ]. Patient involvement in co-designing and evaluating alternative healthcare models improves trust and acceptance, highlighting the significance of collaboration and patient engagement strategies for optimising implementation and evaluation processes [ 27 , 28 , 47 ].

Performance evaluation plays a pivotal role in assessing the implementation of professional role substitution models of care [ 25 ]. Monitoring various factors, including outcomes, patient satisfaction, quality of care, safety, healthcare professionals’ performance, healthcare system efficiency, and cost-effectiveness, can provide valuable insights for ongoing improvement, optimisation, and sustainability of models of care [ 25 , 50 ]. We address gaps highlighted in previous research, particularly concerning the lack of comprehensive evaluations and guidance on outcome measures [ 10 , 22 , 25 ]. Many current frameworks lack specificity in identifying key metrics relevant to professional role substitution models [ 25 , 26 ]. However, this study delineating eight key outcome measures emphasises a data-driven approach to decision-making. This represents an advancement in the field, providing a structured framework for assessing the impact and value of these models.

Implications for policy, practice and future research

In combination with existing literature in various alternative healthcare delivery models, this study highlights the shared challenges and opportunities across healthcare professions and settings [ 41 , 45 , 46 ]. Our analysis of implementation considerations, stakeholder perspectives, and outcome measures, advances theoretical understanding and also provides practical guidance for real-world implementation and evaluation. These insights can be extended beyond Australia's healthcare system, with implications for policy development, collaboration, knowledge exchange, and healthcare delivery practices in other regions.

In practice, maximising the effectiveness and sustainability of professional role substitution models necessitates comprehensive training and education initiatives for health professionals. Collaborating with professional bodies and universities can standardise training, provide continuous professional development opportunities, and address individual factors that impact implementation readiness for alternative healthcare delivery models.

Adapting regulatory frameworks to the evolving healthcare landscape is paramount, necessitating clear guidelines and legal frameworks to delineate practice boundaries and facilitate the seamless implementation of expanded roles. Adequate funding is critical to support various aspects, including staffing, infrastructure development, establishment of incentivising reimbursement models, research, evaluation, and ensuring ongoing sustainability. Prioritising evidence-based policy development, informed by comprehensive evaluation of care models, is essential to ensure alignment with best practices and standards of care. Integrating standard outcome measures into evaluation frameworks is crucial for accurately assessing the impact and effectiveness of care models, thereby enabling informed decision-making based on evidence. The research we have conducted supports these assertions, emphasising the importance of these factors for the successful implementation and sustainability of alternative healthcare delivery models.

Our findings may therefore serve as a catalyst for discussion and debate on allied health professional role substitution and other alternative healthcare delivery models, guiding future research endeavours. Exploring longitudinal studies to gauge sustainability and long-term impact, conducting comparative analyses across diverse settings and patient populations, and conducting qualitative inquiries to identify implementation and evaluation facilitators and barriers are critical. Additionally, research in health economics, health information technology, policy analysis, and interprofessional collaboration can provide valuable insights to optimise implementation practices and enhance the applicability of these models across different healthcare systems and cultural contexts.

Strengths and limitations

The strengths of this study lie in the diverse range of stakeholders involved, including key opinion leaders, decision makers, allied health clinicians, medical professionals, policymakers, healthcare administrators, university partners, professional bodies, advocates, and patients. The inclusion of participants with varied experiences enhances the robustness of the findings. Purposeful sampling with maximum variation further improves the transferability of the results.

The use of the COREQ-checklist and independent co-coding and discussions among the research team enhance the credibility, trustworthiness, and transparency of the study [ 31 ]. Another notable strength is the use of the Consolidated Framework for Implementation Research (CFIR) to guide the analysis, which helped identify and organise themes into multi-level intervention principles that influence implementation effectiveness. It should be noted that the CFIR was not used to guide data collection, as is often practiced, [ 38 ] as this may have limited the exploration of qualitative themes relevant to the research question but not explicitly aligned with CFIR domains and constructs.

As the study was conducted in Australia, the generalisability of the findings to other stakeholders or healthcare contexts in different countries and settings may be limited. Additionally, as with any research involving human subjects, the possibility of self-selection bias influencing the results cannot be excluded, and the findings should be interpreted with this in mind. Insights gained from this study may also have broader implications for other countries facing similar challenges in healthcare delivery. By examining similarities and differences in healthcare systems and regulatory environments, countries can however learn from Queensland’s experiences adopting strategies to support the implementation of role substitution models.

In conclusion, this study provides a systematic examination of the key elements and principles influencing the implementation and performance evaluation of professional role substitution models of care. By understanding the multifaceted nature of these factors and addressing the challenges and opportunities associated with expanded healthcare roles, healthcare systems can navigate complexities and capitalise on opportunities. This holistic approach, involving collaboration among stakeholders and considering patient safety, quality of care, and optimal healthcare outcomes will contribute to the development of more efficient, equitable, sustainable, and patient-centred models of care and healthcare systems.

Availability of data and materials

Data is available from corresponding author on reasonable request.

Abbreviations

Consolidated Framework for Implementation Research

Consolidated Criteria for Reporting Qualitative Research

Physician Assistants

Duckett SJ. Health workforce design for the 21st century. Aust Health Rev. 2005;29(2):201–10.

Article   PubMed   Google Scholar  

Segal L, Bolton T. Issues facing the future health care workforce: the importance of demand modelling. Aust New Zealand Health Policy. 2009;6(1):12.

Article   PubMed   PubMed Central   Google Scholar  

Commission AGP. Productity Commision Research Report https://www.pc.gov.au/inquiries/completed/health-workforce/report/healthworkforce.pdf2006 [Available from: https://www.pc.gov.au/inquiries/completed/health-workforce/report/healthworkforce.pdf .

Dower C, Moore J, Langelier M. It is time to restructure health professions scope-of-practice regulations to remove barriers to care. Health Affairs. 2013;32(11):1971–6.

Porter ME, Lee TH. The strategy that will fix health care. Harvard business review. 2013;91(10):1–19.

Google Scholar  

Baker A. Crossing the quality chasm: a new health system for the 21st century. Br Med J Publishing Group. 2001;323(7322):1192.

Owens DK, Qaseem A, Chou R, Shekelle P. High-value, cost-conscious health care: concepts for clinicians to evaluate the benefits, harms, and costs of medical interventions. Ann Intern Med. 2011;154(3):174–80.

Roberts N, Carrigan A, Clay-Williams R, Hibbert PD, Mahmoud Z, Pomare C, et al. Innovative models of healthcare delivery: an umbrella review of reviews. BMJ open. 2023;13(2):e066270.

Wang H, English M, Chakma S, Namedre M, Hill E, Nagraj S. The roles of physician associates and advanced nurse practitioners in the National Health Service in the UK: a scoping review and narrative synthesis. Human Resour Health. 2022;20(1):69.

Article   CAS   Google Scholar  

McPherson K, Kersten P, George S, Lattimer V, Breton A, Ellis B, et al. A systematic review of evidence about extended roles for allied health professionals. J Health Serv Res Policy. 2006;11(4):240–7.

Martin-Misener R, Kilpatrick K, Donald F, Bryant-Lukosius D, Rayner J, Valaitis R, et al. Nurse practitioner caseload in primary health care: scoping review. Int J Nurs Stud. 2016;62:170–82.

Barnett M, Balkissoon C, Sandhu J. The level of quality care nurse practitioners provide compared with their physician colleagues in the primary care setting: A systematic review. J Am Assoc Nurse Pract. 2022;34(3):457–64.

Middleton S, Gardner A, Della PR, Lam L, Allnutt N, Gardner G. How has the profile of Australian nurse practitioners changed over time? Collegian. 2016;23(1):69–77.

Freund T, Everett C, Griffiths P, Hudon C, Naccarella L, Laurant M. Skill mix, roles and remuneration in the primary care workforce: who are the healthcare professionals in the primary care teams across the world? Int J Nurs Stud. 2015;52(3):727–43.

Halter M, Drennan V, Chattopadhyay K, Carneiro W, Yiallouros J, de Lusignan S, et al. The contribution of physician assistants in primary care: a systematic review. BMC Health Serv Res. 2013;13:1–13.

Article   Google Scholar  

Henry LR, Hooker RS, Yates KL. The role of physician assistants in rural health care: a systematic review of the literature. J Rural Health. 2011;27(2):220–9.

Terry B, Bisanzo M, McNamara M, Dreifuss B, Chamberlain S, Nelson SW, et al. Task shifting: meeting the human resources needs for acute and emergency care in Africa. Afr J Emerg Med. 2012;2(4):182–7.

Mijovic H, McKnight J, English M. What does the literature tell us about health workers’ experiences of task-shifting projects in sub-Saharan Africa? A systematic, qualitative review. J Clin Nurs. 2016;25(15–16):2083–100.

Michel JE, D Tediosi F, D Egger MB, T McIntyre DR. Lest we forget, primary health care in Sub-Saharan Africa is nurse led. Is this reflected in the current health systems strengthening undertakings and initiatives? J Global Health Rep. 2018;2:e2018009.

Kengne AP, Sobngwi E, Fezeu LL, Awah PK, Dongmo S, Mbanya JC, et al. Nurse-led care for asthma at primary level in rural sub-Saharan Africa: the experience of Bafut in Cameroon. J Asthma. 2008;45(6):437–43.

Eberly LA, Rusingiza E, Park PH, Ngoga G, Dusabeyezu S, Mutabazi F, et al. 10-year heart failure outcomes from nurse-driven clinics in rural sub-Saharan Africa. J Am Coll Cardiol. 2019;73(8):977–80.

Saxon RL, Gray MA, Oprescu FI. Extended roles for allied health professionals: an updated systematic review of the evidence. J Multidisciplinary Healthcare. 2014;7:479–88.

Welfare AIoHa. Australia's health performance framework 2017 [Available from: https://www.aihw.gov.au/reports-data/indicators/australias-health-performance-framework .

Queensland Health Clinical Excellence Division. Allied Health Expanded Scope Strategy 2016–2021. 2016. Available from https://www.health.qld.gov.au/__data/assets/pdf_file/0024/449520/expanded-strategy.pdf . Accessed 09 July 2024.

Mutsekwa RN, Wright C, Byrnes JM, Canavan R, Angus RL, Spencer A, et al. Measuring performance of professional role substitution models of care against traditional medical care in healthcare—A systematic review. J Eval Clin Pract. 2022;28(2):208–17.

Levesque J-F, Sutherland K. Combining patient, clinical and system perspectives in assessing performance in healthcare: an integrated measurement framework. BMC Health Serv Res. 2020;20(1):1–14.

Mutsekwa RN, Byrnes JM, Larkins V, Canavan R, Angus RL, Campbell KL. Role substitution of specialist medical doctors with allied-health professionals: A qualitative exploration of patients’ experiences and perceptions of healthcare quality. J Eval Clin Pract. 2022;28(6):1096–105.

Mutsekwa RN, Campbell KL, Canavan R, Mulhern B, Angus RL, Byrnes JM. Patient preferences for attributes that characterise alternative models of care in gastroenterology: a discrete choice experiment. Patient. 2023;16(2):165–77.

Mutsekwa RN, Campbell KL, Canavan R, Angus RL, McBride L-J, Byrnes JM. Measuring performance in allied health professional role substitution models of care: a clinician survey. BMC Health Serv Res. 2024;24(1):79.

Galletta A. Mastering the semi-structured interview and beyond: From research design to analysis and publication, vol. 18. NYU press; 2013.

Tong A, Sainsbury P, Craig J. Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups. Int J Qual Health Care. 2007;19(6):349–57.

Government Q. Fact Sheet: Allied Health and Oral Health 2019 [Available from: https://www.health.qld.gov.au/__data/assets/pdf_file/0027/1067571/Allied-Health-and-Oral-Health-2019.pdf .

Queensland Health Allied Health Professions’ Office of Queensland. Ministerial Taskforce on health practitioner expanded scope of practice: final report. 2014. Available from: Ministerial taskforce final report ( health.qld.gov.au ). Accessed 09 July 2024.

Queensland Government. Hospital care, surgical procedures and waiting lists 2015 [Available from: https://www.qld.gov.au/health/services/hospital-care/waiting-lists .

Saunders B, Sim J, Kingstone T, Baker S, Waterfield J, Bartlam B, et al. Saturation in qualitative research: exploring its conceptualization and operationalization. Qual Quant. 2018;52:1893–907.

Damschroder LJ, Reardon CM, Widerquist MAO, Lowery J. The updated Consolidated Framework for Implementation Research based on user feedback. Implement Sci. 2022;17(1):1–16.

Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009;4(1):1–15.

Damschroder LJ, Reardon CM, Opra Widerquist MA, Lowery J. Conceptualizing outcomes for use with the Consolidated Framework for Implementation Research (CFIR): the CFIR Outcomes Addendum. Implement Sci. 2022;17(1):1–10.

Braun V, Clarke V. Reflecting on reflexive thematic analysis. Qualitat Res Sport Exerc Health. 2019;11(4):589–97.

Carter N. The use of triangulation in qualitative research. Number 5/September 2014. 1969;41(5):545–7.

Wiggins D, Downie A, Engel RM, Brown BT. Factors that influence scope of practice of the five largest health care professions in Australia: a scoping review. Human Resour Health. 2022;20(1):1–13.

Morris J, Grimmer K, Gilmore L, Perera C, Waddington G, Kyle G, et al. Principles to guide sustainable implementation of extended-scope-of-practice physiotherapy workforce redesign initiatives in Australia: stakeholder perspectives, barriers, supports, and incentives. J Multidiscip Healthc. 2014;7:249–58.

Fiorio CV, Gorli M, Verzillo S. Evaluating organizational change in health care: the patient-centered hospital model. BMC Health Serv Res. 2018;18:1–15.

Wiles L, Milanese S. Stakeholder perspectives of the Extended Scope Physiotherapy Practitioner (ESPP) role in Australia–a qualitative study. Physical Therapy Rev. 2016;21(3–6):228–35.

Parker RM, Desborough JL, Forrest LE. Stakeholder perceptions of a nurse led walk-in centre. BMC Health Serv Res. 2012;12:1–7.

Bryant-Lukosius D, DiCenso A. A framework for the introduction and evaluation of advanced practice nursing roles. J Adv Nurs. 2004;48(5):530–40.

Harding P, Prescott J, Block L, O’Flynn AM, Burge AT. Patient experience of expanded-scope-of-practice musculoskeletal physiotherapy in the emergency department: a qualitative study. Australian Health Rev. 2015;39(3):283–9.

Browne K, Roseman D, Shaller D, Edgman-Levitan S. Measuring patient experience as a strategy for improving primary care. Health Affairs. 2010;29(5):921–5.

Reeve S, May S. Exploration of patients’ perspectives of quality within an extended scope physiotherapists’ spinal screening service. Physiother Theory Pract. 2009;25(8):533–43.

Moretto N, Stute M, Sam S, Bhagwat M, Raymer M, Buttrum P, et al. A uniform data set for determining outcomes in allied health primary contact services in Australia. Aus J Prim Health. 2020;26(1):58–69.

Download references

Acknowledgements

A special thanks to the participants who gave up their time to share their experiences and perceptions on implementation and performance evaluation of professional role substitution models of care.

This work was supported by the Gold Coast Hospital and Health Service Collaborative Research Grant [grant number RGS20190041].

Author information

Authors and affiliations.

Gold Coast Hospital and Health Service, Nutrition and Food Services, 1 Hospital Boulevard, Southport, Queensland, 4215, Australia

Rumbidzai N. Mutsekwa & Rebecca L. Angus

Gold Coast Hospital and Health Service, Allied Health Research Team, 1 Hospital Boulevard, Southport, Queensland, 4215, Australia

Rumbidzai N. Mutsekwa

Centre for Applied Health Economics, School of Medicine, Griffith University, Sir Samuel Griffith Centre, 1 Parklands Drive, Nathan, Queensland, 4111, Australia

Rumbidzai N. Mutsekwa, Katrina L. Campbell & Joshua M. Byrnes

Menzies Health Institute Queensland, Griffith University, Gold Coast Campus, 1 Parklands Drive, Southport, Queensland, 4215, Australia

Katrina L. Campbell & Joshua M. Byrnes

Healthcare Excellence and Innovation, Metro North Hospital and Health Service, 153 Campbell Street, Bowen Hills, Queensland, 4029, Australia

Katrina L. Campbell

Gastroenterology Department, Gold Coast Hospital and Health Service, 1 Hospital Boulevard, Southport, Queensland, 4215, Australia

Russell Canavan

School of Health Sciences and Social Work, Griffith University, Gold Coast Campus, 1 Parklands Drive, Southport, Queensland, 4215, Australia

Rebecca L. Angus

Department of Health, Clinical Excellence, 15 Butterfield Street, Herston, Queensland, 4006, Australia

Liza-Jane McBride

You can also search for this author in PubMed   Google Scholar

Contributions

RNM, RC, KC, RLA, LJM, JB contributed to conception and design of the study. RM led the data collection and analysis and wrote the initial draft of the manuscript. RNM, KC, RC, RLA, LJM and JB interpreted results, critically revised the manuscript, and approved the final version.

Corresponding author

Correspondence to Rumbidzai N. Mutsekwa .

Ethics declarations

Ethics approval and consent to participate.

This study was performed in line with the principles of the Declaration of Helsinki. Approval was granted by the Gold Coast Hospital and Health Service (HREC/2020/QGC/62104) and Griffith University (GU Ref No: 2020/876) Human Research Ethics Committees

Participants provided informant consent and completed participant consent and information statements.

Consent for publication

All authors consented for the manuscript to be submitted for publication.

Competing interests

None declared. Participants provided informant consent and completed participant consent and information statements.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Supplementary material 1., supplementary material 2., rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Mutsekwa, R.N., Campbell, K.L., Canavan, R. et al. Unlocking potential: a qualitative exploration guiding the implementation and evaluation of professional role substitution models in healthcare. Implement Sci Commun 5 , 73 (2024). https://doi.org/10.1186/s43058-024-00611-x

Download citation

Received : 22 December 2023

Accepted : 28 June 2024

Published : 12 July 2024

DOI : https://doi.org/10.1186/s43058-024-00611-x

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

  • Professional role substitution
  • Scope of practice
  • Implementation
  • Measuring performance

Implementation Science Communications

ISSN: 2662-2211

  • Submission enquiries: Access here and click Contact Us
  • General enquiries: [email protected]

research instrument used in qualitative

This paper is in the following e-collection/theme issue:

Published on 17.7.2024 in Vol 10 (2024)

This is a member publication of National University of Singapore

Advancing Colorectal Cancer Detection With Blood-Based Tests: Qualitative Study and Discrete Choice Experiment to Elicit Population Preferences

Authors of this article:

Author Orcid Image

Original Paper

  • Clarence Ong 1 , BSocSci   ; 
  • Alex R Cook 1 , PhD   ; 
  • Ker-Kan Tan 1, 2 , MBBS, PhD   ; 
  • Yi Wang 1 , PhD  

1 Saw Swee Hock School of Public Health, National University of Singapore and National University Health System, Singapore, Singapore

2 Department of Surgery, Yong Loo Lin School of Medicine, National University of Singapore, Singapore, Singapore

Corresponding Author:

Yi Wang, PhD

Saw Swee Hock School of Public Health

National University of Singapore and National University Health System

12 Science Drive 2

Singapore, 117549

Phone: 65 65164988

Email: [email protected]

Background: Colorectal cancer (CRC) is the second most deadly form of cancer, inducing an estimated 1.9 million incidence cases and 0.9 million deaths worldwide in 2020. Despite the availability of screening tests, their uptake remains suboptimal. However, blood-based tests that look for signs of cancer-specific markers in the body are increasingly available as an alternative for more invasive tests for cancer. Compared with existing tests, the benefits of blood-based tests for CRC include not needing pretest preparation, stool handling, and dietary or medication restrictions.

Objective: This study aims to explore the population’s preferences for CRC screening tests, with a focus on blood-based tests, and investigate the factors influencing test uptake.

Methods: We used a mixed methods approach, combining semistructured interviews and a discrete choice experiment (DCE) survey. Interviews were analyzed using thematic analysis to identify salient attributes for CRC screening tests. These attributes informed the design of the DCE survey. The DCE data were analyzed using mixed logit and mixed-mixed multinomial logit models.

Results: Qualitative findings from 30 participants revealed that participants preferred blood-based tests due to their perceived low risk, minimal pain, and ease of sample collection. However, concerns about the test’s lower accuracy were also expressed. The DCE survey was completed by 1189 participants. In the mixed logit model, participants demonstrated a stronger preference for blood-based tests over a 2-day stool-based test. The mixed-mixed multinomial logit model identified 2 classes, strong supporters and weak supporters, for CRC screening. Weak supporters, but not strong supporters, had a higher preference for blood-based tests. Women, ethnic Chinese, and people aged 40 to 60 years were more likely to be weak supporters. Both models highlighted the high influence of cost and test sensitivity on participants’ preferences. Transitioning from a 2-day stool-based test to a blood-based test, assuming a national screening program at a base price of Singapore $5 (US $3.75), was estimated to have the potential to increase the relative uptake by 5.9% (95% CI 3.6%-8.2%).

Conclusions: These findings contribute to our understanding of CRC screening preferences and provide insights into the factors driving test uptake. This study highlights the perceived advantages of blood-based tests and identifies areas of concern regarding their accuracy. Further research is needed to determine the actual increase in uptake rate when blood-based tests are made available.

Introduction

Colorectal cancer (CRC) is the second most deadly form of cancer, inducing an estimated 1.9 million incidence cases and 0.9 million deaths worldwide in 2020 [ 1 ]. Regular screening for CRC, through methods such as colonoscopy and fecal immunochemical testing (FIT), helps detect CRC earlier, reduces the incidence and mortality of CRC, and brings about cost savings compared with not undergoing any screening [ 2 ]. Despite the availability of screening as a preventive intervention for the early detection of CRC, screening rates are suboptimal, even within high-income countries [ 3 - 5 ]. Factors that hamper screening include not just individual characteristics but characteristics of the screening tests as well [ 6 ].

The emergence of blood-based, early detection tests for cancer has the potential not only for detecting multiple cancers but also for improving patient compliance and acceptance [ 7 ]. The discovery of circulating and cell-free tumor DNA in the blood has ushered in new possibilities for the blood-based detection of CRC as well [ 8 ]. Epi proColon— a SEPT9 DNA methylation assay—remains to be the only US Food and Drug Administration–approved test [ 9 ]. However, there are at least 5 other blood-based tests in various stages of development, with tests ranging from CRC-specific tests to multi-cancer early detection tests. Some candidate analytic targets include cell-free DNA, methylated circulating tumor DNA, and a combination of methylated DNA and proteins [ 10 ]. Compared with existing tests, the benefits of blood-based tests for CRC include not needing pretest preparation, stool handling, and dietary or medication restrictions [ 9 ].

Challenges to the implementation of blood-based tests for screening include lower specificity relative to one-time FIT [ 11 ] and inferior sensitivity compared with next-generation FIT-DNA tests [ 12 ]. As a result, blood-based tests for CRC are not recommended for the general population in the health guidelines of the United States, Europe, China, and Singapore [ 13 - 16 ]. While there exist several clinical disadvantages to blood-based tests, it may serve as an alternative for patients refusing screening by colonoscopy or patients self-excluded from stool-based tests due to bleeding conditions such as hemorrhoids radiation proctitis [ 17 ]. In fact, the SEPT9 test was found to be more effective and cost-effective compared with no screening [ 18 ]. By making screening easier, blood-based tests have the potential to improve uptake if the benefits outweigh the downside of this screening modality [ 19 ]. Studies are required to understand how the population will make trade-offs between different procedures and their attributes.

In a randomized controlled trial (RCT) involving average-risk adults that offered blood-based tests and FIT in a clinical setting, higher screening participation rates were observed in the blood-based test arm [ 20 ]. The blood-based test was also found to be effective in increasing screening rates among medically underserved populations [ 19 ]. However, another RCT reported no statistically significant improvement in the uptake among the population familiar with FIT if a blood-based test was offered upfront as an option [ 21 ]. Conversely, studies offering the blood-based test as a rescue option for those who declined colonoscopy and stool-based tests showed an increase in participant rates [ 21 - 23 ].

Building on the existing literature, at least 4 questions are deserving of further investigation. First, what is the population’s inclination toward blood-based tests if the accuracy of blood-based tests can be improved to satisfactory levels akin to the FIT-DNA test? This insight can help assess the potential value of further investment in the test and inform the design of a target product profile [ 24 , 25 ]. Second, what is the general population’s preference for using the blood-based test in routine CRC screening? Results from RCTs may not be generalizable to the general population given the differences in the characteristics between the study participants and the general population. Third, considering heterogeneous preferences for blood-based tests, can we profile the population based on their preferences? Such profiling efforts can inform the crafting of targeted screening programs to cater to the heterogeneous preferences across different groups. Fourth, many preference studies were done in Western countries and very few were done in Asia [ 26 , 27 ]. Cultural and social norms could influence decision-making and outcomes. Studies understanding the acceptance of blood-based CRC tests in Asia are needed.

Acknowledging these gaps, we have designed a mixed methods study to delve into the population’s preference for blood-based testing modalities in Singapore, a multiracial Asian society, and to understand their decision-making process when choosing between blood-based tests and other existing screening methods. A discrete choice experiment (DCE) was used to construct hypothetical scenarios (eg, higher accuracy for the blood-based test). Furthermore, we intend to undertake subgroup analyses to examine potential variations in the preference for blood-based tests within distinct segments of the population, as highlighted by the mixed results of the RCTs. Our investigation will also delve into whether specific screening methods, such as the blood-based test, might positively impact participation rates, particularly in subpopulations identified with lower anticipated adherence based on prevailing screening recommendations.

This is a mixed methods study with interviews and a survey that incorporated a DCE. The methods for the qualitative and quantitative components will first be outlined, and subsequently, the qualitative and quantitative results will be presented.

Ethical Considerations

The study was approved by the National Healthcare Group Domain Specific Review Board (2021/00753) before data collection. Participants of the DCE were from a web-based cohort, and their participation in research was approved by the National University of Singapore (NUS) institutional review board (NUS-IRB: H-18-011).

Qualitative Component

Participant and sampling.

The recruitment and interview of participants for the qualitative component took place between December 2021 and March 2022. Convenience sampling was undertaken to include the Singapore population aged ≥40 years with varied engagement with CRC screening services. Recruitment was conducted via the NUS social media platforms and its email blast services, as well as other participant recruitment channels and word of mouth. Interested potential participants contacted the researchers, who verified their eligibility before taking informed consent.

Conducting the Interview

The interviews took place either on the web via a videoconferencing application or in a quiet room within the NUS that was convenient for recording. The interviews adopted a semistructured format using a topic guide. Each interview lasted approximately 30 to 45 minutes and was conducted in English.

Analysis of Interviews

The interviews were transcribed verbatim, and the data were analyzed using thematic analysis. A preliminary codebook with emerging themes of relevance from the first 5 transcripts was developed upon full familiarization of the transcripts. A deductive and semantic approach was used in the clustering of codes into metacodes and categories of interest. The coding framework was subsequently applied to the remaining transcripts. The identified themes were also further reviewed to ensure their usefulness and accuracy in representing the data.

Quantitative Component

Discrete choice experiment.

DCEs have increasingly become a popular method for investigating and eliciting patient and population preferences for health care [ 28 ]. The method is based on consumer choice theory [ 29 ], which posits that respondents make choices between hypothetical products or scenarios comprising of ≥2 alternatives based on the importance they place on the characteristics of these attributes. In a DCE, a product or scenario is described with a fixed number of attributes with varying combinations of levels. Per this paradigm, in choosing the ideal product or scenario, the respondent evaluates the overall desirability of the alternatives and makes trade-offs among the attributes. From the respondents’ choice, their preferences are indirectly revealed, determining the attributes that drive the respondents’ preferences as well as the way variations of the attributes and levels may affect the respective preferences [ 30 ].

Selection of Attributes and Levels

The selection of attributes and levels must be relevant to the policy process and the study population, while being consistent with the random utility theoretical foundation of DCEs [ 31 ]. An initial set of attributes and levels for the DCE was based on a scoping review of the existing literature, which yielded 13 attributes. During the aforementioned qualitative interviews, participants were asked to rank 3 attributes that they valued the most, and the weighted preferences of all participants helped shortlist the final attribute list for the DCE. Following that, a quantitative survey with eligible health care professionals (n=11), who had at least 1-year working experience with patients with CRC or RC screening, was conducted to ensure the validity of the selected attributes and their corresponding levels. After these iterative processes, six attributes were identified and ultimately used in the DCE: (1) procedure, (2) pain level, (3) sensitivity, (4) recommendation, (5) out-of-pocket cost, and (6) risk of test. Each attribute was assigned various levels based on the best information available. The blood-based test was one level of the procedure attribute.

To optimize the choice sets, a pilot study was conducted with 12 participants. Adjustments were then made to the text for the attributes and levels to improve clarity for the participants. The final set of 6 attributes and levels is presented in Textbox 1 . The total number of appearances and selections for each attribute and level may be found in Multimedia Appendix 1 .

  • Colonoscopy
  • Computed tomography colonography
  • Stool-based (2 days)
  • Stool-based (1 day)
  • Blood-based
  • Health Promotion Board
  • Family or friends
  • Singapore $0
  • Singapore $5 (approximately US $3.75)
  • Singapore $30 (approximately US $22.50)
  • Singapore $400 (approximately US $300)
  • Singapore $1000 (approximately US $750)
  • 1% risk of adverse event

Experimental Design

The DCE questionnaire was designed using Sawtooth Lighthouse Studio (version 9.13.2), and a 2-stage design was used. For each task, participants first selected the preferred choice from 2 test profiles and were then asked to choose between taking the test or opting out of it in real life. Correspondingly, the parameter labeled “Opt-Out” represents the utility associated with declining the preferred test in the first stage. A negative value thus indicates a participant’s preference to undergo the screening test. The questionnaire was designed using the random task generation method provided by Sawtooth. The DCE included a total of 20 blocks with 10 choice sets each. Each study participant saw 1 block of choice sets, consisting of 10 choice sets, from the 20 blocks. To test for internal validity, 1 fixed choice set offering 2 alternatives is common across all blocks, of which one is intended to be strictly dominant over the other.

The levels of cost were selected to reflect the costs of different procedures in reality. Participants in qualitative interviews also demonstrated a similar perception regarding the cost of the tests. Certain within-concept prohibitions were also specified to provide combinations of attributes that were realistic. This included prohibiting high out-of-pocket payment costs and the presence of mild pain for stool-based tests. However, we allowed the blood-based test to appear together with a higher cost given that commercial companies may set higher prices [ 32 ]. The coverage matrix of the DCE design was examined using Sawtooth Lighthouse Studio to ensure all the parameters can be estimated. In addition, considering the large range of costs, we treated cost as discrete variables rather than a continuous variable in our analysis. An example of a DCE choice task is presented in Figure 1 .

Participants reported any familial history of CRC and if they had attended any type of CRC screening in the past. Various sociodemographic information was also collected. All variables were coded as categorical variables, and some variables were subsequently recoded as binary or ternary variables to form meaningful subgroups for analyses. Psychosocial inventories—the Zimbardo Time Perspective Inventory [ 33 ], the Intolerance of Uncertainty Scale [ 34 ], and the Duke UNC Functional Social Support Questionnaire [ 35 ]—were also included to measure participants’ degree of present orientation, intolerance of uncertainty, and social support, respectively.

research instrument used in qualitative

This study was conducted as a web-based survey hosted on REDCap from May 19, 2022, to May 28, 2022. The target population for this study was Singapore citizens and permanent residents aged ≥40 years. The survey was sent to the participants from Singapore Population Health Studies web-based panel. This web-based panel consists of a cohort that is broadly representative of the general Singapore population. Participants gave their implicit consent by participating in the survey and were reimbursed with Singapore $10 (approximately US $7.50) for every successful completion of the survey.

In calculating the minimum sample size, we made use of the proposed formula of n>500*c/(t*a) by Johnson and Orme [ 36 ], where 500 is a fixed variable, c demotes the largest number of levels for a certain attribute, a indicates the number of DCE choice sets per block of questionnaire, and t refers to the number of alternatives per DCE choice set (excluding “Opt-out”). Accordingly, the sample size required for this study should be >125 participants (500*5/(2*10)=125). In addition, Lancsar and Louviere [ 30 ] suggested 20 responses per block or questionnaire, which led to a minimum sample size of 400. Considering that the DCE may be relatively difficult to answer, we expected a relatively high nonresponse rate of 20% to 30%. Hence, we set the target sample size to be 600. We subsequently tested the sample size using simulation functions with Sawtooth Lighthouse Studio, which deemed the sample size of 600 as sufficient. Owing to the overwhelming response, our final sample size of 1189 participants not only meets the minimum required sample size but also allows for the possibility of conducting flexible subgroup analyses.

Statistical Analysis

A total of 2 models were tested: a mixed logit (MXL) model and a mixed-mixed multinomial logit (MMML) model [ 37 ]. The MXL model was selected to account for the correlation introduced by the repeated observations from each participant and to relax the assumptions of independence from irrelevant alternatives [ 38 ]. This model assumes that the choices made by the same participant are correlated, and preference heterogeneity exists across the population sample. The interpretation of the mean preference weights is made in relation to the chosen base level, and the SD of each level indicates the variability in the mean preference weights. The MMML model was also selected as it incorporates both MXL and the latent class logit model. Unlike latent class logit where a homogeneous fixed preference is assumed within each latent class, a distribution of random coefficients is specified in the MMML model. Within each class, preference weights and the average probability of each demographic within each class can be derived. The number of classes was chosen by examining the Bayesian information criterion.

In addition, the conditional relative importance (CRI) for a given attribute, defined as the difference between the highest preference weight of the attribute level and the lowest preference weight of the attribute level, was reported. A higher CRI indicates the attribute is more important in designing the CRC screening program. Profile-based normalization was then applied to normalize the sum of CRI of all the attributes to 1.

A left-specific constant was included in each regression, with a statistically significant coefficient indicating a left-right bias in the study [ 39 ]. Participants in DCE may take shortcuts and use simplifying heuristics when answering DCE questions, which can introduce an unintended source of variability in the data. When using reading order heuristic, study participants may tend to choose the choice on one side [ 40 ]. Incorporating a variable indicating the position of the choice in the regression can disentangle the associated bias [ 41 , 42 ].

All levels were coded as dummy variables. Reference levels were selected based on the current recommendation for CRC screening for the average-risk adult population. However, we intentionally designated “No Recommendation” as the reference level for the “Recommendation” attribute to investigate the nuanced preferences arising from diverse recommendations made by different individuals. Continuous psychological variables were demedian by subtracting the median value from each data point, resulting in a centered distribution. As for the classification of the continuous sociodemographic variables, age bands were set according to the definition of senior citizen locally, which is at the age of ≥60 years. Household income was grouped to ensure a sufficient sample size in both groups.

Statistical significance was set at P <.05. All quantitative data analyses were carried out using the statistical software R (version 4.2.1; The R Foundation) [ 43 ].

Sample Characteristics

A total of 30 participants completed the interview. A summary of their sociodemographic characteristics is presented in Multimedia Appendix 2 . Of these participants, 14 had undergone a colonoscopy, 20 had taken a stool test, and 6 had taken a blood-based test. A total of 5 participants had not undergone any screening for CRC. Five main themes that elucidated the participants’ motivations and important attributes of CRC screening were identified: (1) accuracy, (2) cost, (3) perceived risk and pain, (4) convenience of the test, and (5) the method of sample collection.

When it came to blood-based tests, many participants were uncomfortable with its inferior accuracy relative to other tests. They saw it as “pointless” due to its possibility of giving rise to many false positives and negatives:

...half the chance of being accurate, then why waste my blood? [P6, female]
If [accuracy is] too low, then it defeats the purpose already...you want to know whether you have cancer or not, you see. [P24, female]

For participants who felt that they had no symptoms, the level of accuracy of the stool-based tests provided sufficient assurance that they felt a colonoscopy was unnecessary. Many cited an approach of escalation—to undergo a colonoscopy only in the event of a positive stool-based test:

I’m not in the high-risk category...I’ll go for the FIT first, and then based on [the results], I’ll go for the colonoscopy. [P22, female]
If I have concerns, colonoscopy will be the best. But if it’s more like a routine check, then I think stool-based would be better. [P25, male]

Participants who had undergone invasive tests such as colonoscopy either (1) were diagnosed with hemorrhoids or (2) had discovered blood in their stools. They had opted for the colonoscopy under their physician’s or friend’s advice, seeing that a colonoscopy was a more “comprehensive” or “complete” test. Many felt that the relatively higher accuracy of the colonoscopy provided them greater ease of mind:

[Colonoscopy] gives a more accurate reading, because you’re able to see...what’s inside. [P19, female]
[Colonoscopy] is not so comfortable, but it’s comfortable to the mind. [P30, male]

Perceived Risk and Pain

The stool-based and blood-based tests were more favorable to most participants as the process was “simple” and “pain-free.” Furthermore, many who had undergone a blood-based test did so as part of their annual comprehensive health check-up and saw little extra risk or pain in doing it:

It’s less traumatic to the patient. That kind of [needle] pain is bearable...no issue. [P1, female]
If it’s part of the blood works, might as well, right? Since they are already drawing blood? I don’t mind testing for [colorectal cancer] as well. [P17, female]

Many of these participants who had no prior experience undergoing colonoscopy were more likely to express fear of the risk of colon perforation from the procedure. Due to its invasive nature, many were also “scared” and “uncomfortable” with the pain the procedure might induce:

I don’t know how big the scope is...how difficult is it to insert? Will it damage anything permanently? [P11, male]
The sort that comes with pain...[I] may get cold feet. [P17, female]

Interestingly, most participants who had undergone colonoscopy had little qualms about the risk and pain of the procedure. Instead, many expressed difficulties adhering to the bowel preparation instead:

The agony part was the bowel preparation. I can’t finish the 4 litres...I gave up at 2 litres. [P14, male]

Method of Sample Collection

While being relatively easy to conduct, some participants shared reservations about the collection of stool samples. They saw the process as “dirty,” “disgusting,” or “troublesome,” especially when 2 separate stool samples were required. While some complained about the uncomfortable experience of stool collection, some expressed personal concerns about improper collection:

...because you have to do it on your own, especially when you have to dig the stool, I’m not sure whether we are doing correctly or not. [P28, male]

Nevertheless, participants who have done it across many years expressed little of such concerns, seeing it as a routine exercise that was necessary:

You’ve done it once and then you make it an annual exercise...it’s not a big deal. [P10, female]

Convenience of Test

As colonoscopies and blood-based tests require medical professionals to perform them, some participants felt that the need to arrange a doctor’s appointment was time-consuming. This was especially true for colonoscopies, where a referral from a primary care physician is required to receive a subsidized rate for the colonoscopy:

...you have to go to the [primary care] polyclinic and then get a referral, see the specialist and then wait for the appointment...you know, so it’s a bit more cumbersome. [P25, male]

However, participants shared that stool-based tests were relatively more accessible, with kits being easily obtained at pharmacies or mailed to them on request. Even when returning stool samples, some participants shared that mailing services to the laboratory were available, which saved them the shame and hassle of dropping them off at a clinic:

[It’s] kind of a deterrent because you have a book an appointment...compared to FIT test, you can just drop by any of the pharmacies...it’s a lot more convenient. [P20, female]

Many participants were aware of the stark difference in cost between a colonoscopy and a stool-based or blood-based test. While many participants, especially those younger and working, had employer and private insurances to subsidize a colonoscopy procedure, they highlighted that out-of-pocket cost was still substantially higher. Many cited that a higher willingness to pay must come in tandem with either higher accuracy and lower frequency of testing:

If the colonoscopy is 70-80% [accurate], and the other tests are also 70-80% [accurate], of course I will choose the simple one. No point to go for a colonoscopy...pay so much, go through the hassle... [P24, female]
[If] you do this scope, one time, last you for 10 years, [because them you] don’t have to collect stool sample at the next medical check-up. [P29, male]

Sample Statistics

A total of 1189 participants completed the study. Of these, 127 (10.7%) participants did not have complete sociodemographic information, while 44 (3.7%) participants failed the fixed choice task. This led to 168 (14.1%) participants dropping out, leaving a sample of 1021 participants for analysis. The demographic characteristics of all participants are presented in the Multimedia Appendix 2 .

The main results of the MXL model are presented in Table 1 .

All mean coefficients were significant at P <.05. On average, participants exhibited a higher preference for blood-based tests relative to a 2-day stool-based test (coefficient=0.40, 95% CI 0.24-0.55). Participants also exhibited a higher preference for a 1-day stool-based test relative to a 2-day stool-based test (coefficient=0.27, 95% CI 0.10-0.45). The preference for these 2 procedures were however not statistically different from each other (coefficient=0.12, 95% CI –0.04 to 0.29).

The profile-based normalized CRI of the 6 attributes based on the MXL model is presented in Figure 2 . Ranking the attributes, participants were most concerned with the cost and sensitivity of the screening test. This is followed by the procedure type, the risk level, the pain level, and ultimately the recommendation received for the screening test.


MeanSD

Coefficient (95% CI ) valueCoefficient (95% CI) value
Left0.13 (0.06 to 0.12)<.001
Opt-out−1.74 (−1.93 to −1.55)<.0013.86 (3.67 to 4.05)<.001

Colonoscopy−0.73 (−0.90 to −0.57)<.0010.98 (0.85 to 1.12)<.001

CT colonography−0.75 (−0.91 to −0.60)<.0010.65 (0.51 to 0.80)<.001

Stool-based (2 days)0.00 (Reference)0.00 (Reference)

Stool-based (1 day)0.27 (0.10 to 0.45)<.0010.93 (0.71 to 1.16)<.001

Blood-based0.40 (0.24 to 0.55)<.0010.91 (0.76 to 1.05)<.001

No pain0.00 (Reference)0.00 (Reference)

Mild pain−0.54 (−0.64 to −0.45)<.0010.26 (0.11 to 0.40)<.001

1001.63 (1.52 to 1.75)<.0010.09 (−0.05 to 0.24)0.19

950.70 (0.59 to 0.82)<.0010.02 (−0.13 to 0.18)0.79

800.00 (Reference)0.00 (Reference)

60−1.26 (−1.43 to −1.08)<.0011.74 (1.53 to 1.95)<.001

Health Promotion Board0.88 (0.77 to 1.00)<.0010.11 (−0.04 to 0.25).16

Doctors0.68 (0.56 to 0.80)<.0010.01 (−0.14 to 0.17).86

Family or friends0.35 (0.23 to 0.48)<.0010.21 (0.05 to 0.36).008

Neither0.00 (Reference)0.00 (Reference)
)

00.00 (Reference)0.00 (Reference)

5−0.35 (−0.50 to −0.19)<.0010.38 (0.16 to 0.59)<.001

30−0.81 (−0.93 to −0.69)<.0010.08 (−0.09 to 0.25).35

400−2.39 (−2.53 to −2.25)<.0010.46 (0.30 to 0.62)<.001

1000−3.88 (−4.07 to −3.68)<.0011.53 (1.33 to 1.73)<.001

No risk0.00 (Reference)0.00 (Reference)

1% risk of adverse event−0.74 (−0.86 to −0.62)<.0010.05 (−0.11 to 0.21).53

a Log-likelihood: −8353, Akaike information criteria: 16,777, and Bayesian information criteria: 17,054.

b Mean refers to the population mean. SD measures the individual preference heterogeneity. A significant value means that the preference for the corresponding level is heterogeneous at the individual levels.

c Not applicable.

d CT: computed tomography.

e Singapore $1=approximately US $0.75.

research instrument used in qualitative

MMML Model: Preference Analysis

The results from the MXL model suggested preference heterogeneity across the participants, with the SDs of several preferences being statistically significant. Thus, we ran an MMML model while also estimating the posterior class membership probabilities. A 2-class MMML model was the most appropriate based on the BIC. The classes were labeled post hoc as (1) strong supporters and (2) weak supporters, based on the coefficient value for “None.” A more negative coefficient value means people are more willing to take the screening test in real life. The class shares for strong supporters and weak supporters are 38.09% (n=339) and 61.91% (n=632), respectively. The main results of the MMML model are presented in Table 2 . The full table with SD is presented in Multimedia Appendix 3 .

Strong supporters did not exhibit a higher preference for blood-based tests relative to a 2-day stool-based test. However, weak supporters had a higher preference for blood-based tests compared with a 2-day stool-based test (coefficient=0.66, 95% CI 0.44-0.88). Similar to the results in the MXL model, strong and weak supporters were most concerned with the cost and sensitivity of the screening test. Weak supporters were more likely than strong supporters to be concerned with the procedure, pain level, and risk of test. Compared with an existing national screening program that is 2-day stool-based, has no pain, 80% sensitivity, recommended by the government’s Health Promotion Board, costs Singapore $5, and has no risk, the relative uptake rate of a blood-based screening test with all else constant will increase by 0.2% (95% CI −1.2% to 1.6%) for strong supporters and increase by 5.9% (95% CI 3.6% to 8.2%) for weak supporters.


Class 1: Strong supportersClass 2: Weak supporters

Coefficient (95% CI) valueCoefficient (95% CI) value

Left0.28 (0.15 to 0.42)<.0010.08 (−0.02 to 0.18).13

None−5.13 (−5.79 to −4.46)<.001−0.86 (−1.10 to −0.61)<.001



Colonoscopy−0.35 (−0.71 to 0.02).07−1.05 (−1.28 to −0.81)<.001


CT colonography−0.53 (−0.87 to −0.18)<.001−1.01 (−1.23 to −0.78)<.001


Stool-based (2 days)0.00 (Reference)0.00 (Reference)


Stool-based (1 day)−0.20 (−0.57 to 0.16).270.63 (0.39 to 0.87)<.001


Blood-based0.06 (−0.27 to 0.49).700.66 (0.44 to 0.88)<.001



No pain0.00 (Reference)0.00 (Reference)


Mild pain−0.25 (−0.46 to −0.04).02−0.84 (−0.98 to −0.70)<.001



1002.92 (2.61 to 3.24)<.0011.25 (1.10 to 1.41)<.001


951.58 (1.29 to 1.87)<.0010.42 (0.25 to 0.58)<.001


800.00 (Reference)0.00 (Reference)


60−1.65 (−2.04 to −1.27)<.001−0.96 (−1.19 to −0.72)<.001



Health Promotion Board1.24 (0.99 to 1.49)<.0010.79 (0.62 to 0.96)<.001


Doctors0.89 (0.64 to 1.14)<.0010.67 (0.50 to 0.83)<.001


Family or friends0.17 (−0.08 to 0.43).180.40 (0.22 to 0.58)<.001


Neither0.00 (Reference)0.00 (Reference)

)


00.00 (Reference)0.00 (Reference)


5−0.67 (−1.02 to −0.32)<.001−0.33 (−0.55 to −0.11)<.001


30−0.97 (−1.24 to −0.70)<.001−0.83 (−0.99 to −0.66)<.001


400−2.24 (−2.55 to −1.94)<.001−2.93 (−3.14 to −2.72)<.001


1000−4.00 (−4.41 to −3.59)<.001−4.46 (−4.75 to −4.18)<.001



No risk0.00 (Reference)0.00 (Reference)


1% risk of adverse event−0.60 (−0.85 to −0.35)<.001−1.04 (−1.21 to −0.87)<.001



Female0.19 (0.11 to 0.27)<.001


Male0.00 (Reference)



Chinese0.52 (0.41 to 0.63)<.001


Non-Chinese0.00 (Reference)



40-600.00 (Reference)


≥61−0.52 (−0.61 to −0.42)<.001



High income (≥6000)−0.13 (−0.22 to −0.05)<.001


Lower income (≤5999)0.00 (Reference)



Married−0.06 (−0.16 to 0.04).21


Single or divorced or widowed or separated0.00 (Reference)



Primary and secondary0.08 (−0.02 to 0.19).13


Preuniversity0.00 (Reference)


University and above0.02 (−0.22 to −0.05).61



Public housing0.00 (Reference)


Private housing−0.47 (−0.60 to −0.35)<.001



Currently working−0.24 (−0.34 to −0.14)<.001


Not working or retired or student0.00 (Reference)



Yes−0.74 (−0.85 to −0.63)<.001


No0.00 (Reference)



Yes−0.52 (−0.60 to −0.44)<.001


No0.00 (Reference)

Perceived safety of test score−0.16 (−0.17 to −0.14)<.001

Social support score−0.03 (−0.05 to −0.02)<.001

Present orientation−0.03 (−0.05 to −0.02)<.001

Intolerance of uncertainty−0.01 (−0.02 to −0.01)<.001

a Log-likelihood: −8041, Akaike information criteria: 16,257, Bayesian information criteria: 16,955.

b CT: computed tomography.

d Singapore $1=approximately US $0.75.

e CRC: colorectal cancer.

MMML Model: Analysis of Demographic Information

Weak supporters were more likely to be women, ethnic Chinese, and people aged 40 to 60 years. However, strong supporters were more likely to be working, have higher income levels, and live in private housing. Strong supporters were also more likely to have a family history of CRC and to have opted for CRC screening in the past. Compared with weak supporters, strong supporters were also found to have higher scores for the perceived safety of CRC screening tests, social support, present orientation, and intolerance of uncertainty.

Principal Findings

For every screening program to be successful, it is important to identify patterns in the population’s choice for CRC screening test to encourage uptake. By using a sequential exploratory mixed methods design and using a web-based cohort that is designed to be representative of the general Singapore population, our study is able to identify salient attributes of screening among participants to inform our understanding of the population’s preference. Furthermore, controlling for some of the salient attributes in our DCE allows for a better interpretation of the preference of specific procedures, which is likely to encompass the value of convenience and method of sample collection that is otherwise not considered in the DCE.

Our DCE results suggest that most participants preferred a blood-based test over a 2-day stool-based test and colonoscopy after accounting for the other attributes (eg, sensitivity). A blood-based test was perceived to be pain free, with a method of sample collection that was relatively simpler, compared with a colonoscopy. This pattern of aversion to aspects of pain and risk that comes with colonoscopy is supported by several studies [ 44 , 45 ]. Furthermore, studies have also supported the convenience and ease of the blood-based tests as compared with colonoscopy, as no preparation is involved [ 44 , 46 ]. Stool-based tests were regarded as unpleasant and disgusting to some participants in line with studies from the United States [ 47 ], Australia [ 48 ], and Germany [ 49 ], wherein participants expressed a preference for blood over stool sampling. However, comparing a blood-based test with a 1-day stool-based test, the utility for a blood-based test was not statistically significantly higher. This suggests that the requirement of testing twice was a key contributor to the perceived inconvenience and disutility of a stool-based test.

One advantage of DCE is its ability to profile the population and understand the preference and estimate the uptake in different subpopulations. We profiled the population into 2 classes, the strong supporters for CRC screening who were indifferent between the 2-day stool-based test and blood-based test, and the weak supporters for CRC screening who preferred the blood-based test over the 2-day stool-based test. Our results suggest that the sensitivity of tests was a key consideration of participants. For example, a 2-day stool-based test with a sensitivity of 80% was preferred by both strong supporters and weak supporters compared with a blood-based test with a sensitivity of 60%. Unfortunately, the shortcoming of existing blood-based tests lies in the relatively inferior sensitivity and specificity [ 11 , 12 ], which was a concern for participants based on the qualitative interviews as well. In reality, while the procedure of blood-based tests itself is preferred, the low sensitivity and specificity rate is unlikely to appeal to the masses. However, if blood-based tests can achieve the same sensitivity as 2-day stool-based tests, by offering blood-based tests to weak supporters, there is a potential to increase the relative CRC screening uptake by approximately 6%. The impact is likely to be significant as weak supporters make up approximately 62% of the population. Hence, further investment in research and development to improve the accuracy of blood-based tests could be beneficial to society.

The weak supporters identified in the study were less likely to do CRC screening in real life, but showed a higher preference for blood-based tests. Several observable demographic factors were associated with being weak supporters, including being female, ethnic Chinese, and younger and having lower income. The government may thus use targeted information campaigns when blood-based tests become a feasible screening option. The convenience of the blood-based test needs to be highlighted to weak supporters. However, one concern identified from the interview was the need for health care providers to draw blood. Strategies and logistic arrangements to reduce waiting time for taking the blood-based test need to be designed. For strong supporters, relative to the testing procedure, they cared more about the sensitivity of the test compared with weak supporters. Blood-based tests itself are not attractive to them. Information campaigns to strong supporters should focus on better accuracy (eg, a blood-based test with high accuracy). Both strong supporters and weak supporters valued the recommendation from the Health Promotion Board [ 50 ] most, the governmental agency driving preventive care under the Ministry of Health in Singapore.

While DCEs can inform patients’ preferences on specific tests, a proper health technology assessment should be performed on whether blood-based tests are appropriate in each country. Unfortunately, cost-effectiveness studies on SEPT9 were rarely included in systematic reviews of the cost-effectiveness of CRC screening strategies [ 51 , 52 ]. All found studies that included SEPT9 as a screening strategy discovered that an annual SEPT9 screening was more cost-effective than no screening [ 18 , 53 , 54 ]. A total of 2 studies that relied on test characteristics of the earlier version of SEPT9 found that annual screening with FIT dominated the SEPT9 strategy [ 18 , 54 ], meaning that using FIT provided superior outcomes at a lower cost compared with using SEPT9 . However, a more recent study using test characteristics of the improved version of SEPT9 with higher sensitivity and specificity found that the strategy resulted in higher quality-adjusted life years gained, CRC cases adverted, and CRC deaths adverted compared with other screening strategies [ 53 ]. Nevertheless, the strategy of using SEPT9 remained more costly than FIT as it resulted in a 63% higher referral for colonoscopy than FIT, increasing the cost by 26%. As a result, the strategy of FIT was still more cost-effective than SEPT9 . However, these conclusions were based on perfect adherence of strategies. On the basis of our findings, the blood-based tests like SEPT9 are more likely to have higher adherence than stool-based tests such as FIT if similar accuracy can be achieved. Indeed, 2 studies that considered imperfect uptake found that when the uptake rate of FIT fell below 70% relative to that of SEPT9 , FIT was no longer more cost-effective than SEPT9 [ 18 , 54 ]. Thus, the possibility of SEPT9 being more cost-effective than FIT likely hinges on (1) an improved version of SEPT9 with higher sensitivity and specificity and (2) a significantly higher uptake for SEPT9 over FIT .

Policy Implications

It is inevitable that the next frontier of cancer screening will be the adoption of blood-based tests [ 8 ]. Multiple such tests are being evaluated or studied currently [ 7 ]. Hence, it is no longer far-fetched that policy makers need to decide whether and how to include blood-based tests in the national cancer screening program. In the overall landscape of cancer screening, different stakeholders have different views on the matter. Patients and clinicians alike will want any patient with cancers to be diagnosed earlier, while developers and manufacturers of the test will ultimately focus on monetary returns as a primary consideration. Policy makers and government agencies have to determine the cost-effectiveness of such tests and be mindful of all the additional subsequent more invasive and expensive tests that would be performed in the presence of a positive test result, all of which can compound health care costs. Aside from the financial aspect, the other downsides of screening, such as patient anxiety and lead-time bias, will all need to be considered seriously. Most of the existing studies examining the attitude and preference for blood-based CRC tests were conducted in Western societies [ 26 , 27 ]. Preference of the technology in Asia is understudied. Therefore, studying preferences in Singapore, with its mix of East, South, and Southeast Asian cultures, provides a valuable addition to the literature.

Screening test only serves as the first step—follow-up diagnostic tests are required to complete the process. The gaps between stool-based tests and follow-up colonoscopies have been documented in the literature [ 55 , 56 ], which compromises the benefit of the screening program. Nonetheless, blood-based tests give policy makers another option to improve CRC screening. While blood-based tests in themselves may result in higher colonoscopy referral rates, blood-based tests may be combined with the existing screening methods rather than replacing them. This potentially improves on current CRC screening program by reducing the burden of colonoscopy through a 2-step screening approach that triages positive stool-based test patients [ 57 ] and serves as an alternative for people who reject stool-based tests [ 22 ]. Additional research is required to address these practical issues and understand the value brought by blood-based tests before advocating for the inclusion of blood-based cancer screening tests into the national guidelines.

Limitations

We acknowledge several limitations in this study. First, the DCE could not encompass the entirety of decision attributes pertinent to CRC screening, potentially limiting the comprehensive representation of individual decision matrices. However, we prioritized the most salient attributes of the population through qualitative interviews. Moreover, the qualitative interviews helped furnish and provide supplementary perspectives beyond the finalized attributes. Second, it is imperative to note that our study was conducted within a relatively affluent nation, thereby limiting the generalizability of economic considerations, such as income sensitivity and trade-offs to settings with lower income levels. In addition, the availability, accessibility, and quality perception of essential infrastructure, services, and resources may be influenced by local contexts, and their differential manifestations in various settings could yield disparate research conclusions.

Nevertheless, this study advances our understanding of the preferences of the population for CRC screening tests with respect to the type of procedure. The quantified value of the population’s preferences can help in the design of more targeted policies to promote optimal screening behavior and improve screening rates. Given the constrained available resources, more resources can be allocated in the short term to (1) increase the awareness of noninvasive tests and (2) the accessibility of noninvasive tests. Given that stool-based tests are nationally recommended in Singapore, efforts addressing the emotional barriers of embarrassment and disgust of stool collection should be promoted to encourage the collection of stool as something fundamental and shameless. In the long-run, policy makers should consider investing in research and development to improve the accuracy of blood-based tests, as they are generally preferred over invasive tests and may lead to greater uptake. With an improved blood-based test that yields higher sensitivity and specificity rates, institutionalizing CRC screening alongside other routine blood works is likely to be widely acceptable.

Acknowledgments

YW, KKT, and ARC were responsible for the concept and design of the study and obtaining the funding for this study. CO, KKT, and ARC were responsible for the recruitment of participants and data collection. CO and YW were responsible for the analysis of data. YW, KKT, and ARC provided supervision and administrative support. CO and YW wrote the first draft of the manuscript. All authors were involved in the interpretation of the data and worked on critical revisions of the manuscript. All authors read and approved the final manuscript.

This research is supported by the Singapore Ministry of Health’s (MOH) National Medical Research Council under its Health Services Research New Investigator Grant (MOH-000740-00). Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not reflect the views of MOH and National Medical Research Council.

Data Availability

Data are available upon request, subject to the approval of the review boards.

Conflicts of Interest

None declared.

Attributes and levels of the discrete choice experiment with the total number of appearances and selections.

Sociodemographic of participants from the qualitative survey and the discrete choice experiment.

Full mixed-mixed multinomial logit analysis table.

  • Xi Y, Xu P. Global colorectal cancer burden in 2020 and projections to 2040. Transl Oncol. Oct 2021;14(10):101174. [ CrossRef ] [ Medline ]
  • Shaukat A, Mongin SJ, Geisser MS, Lederle FA, Bond JH, Mandel JS, et al. Long-term mortality after screening for colorectal cancer. N Engl J Med. Sep 19, 2013;369(12):1106-1114. [ CrossRef ] [ Medline ]
  • Moutel G, Duchange N, Lièvre A, Orgerie MB, Jullian O, Sancho-Garnier H, et al. Low participation in organized colorectal cancer screening in France: underlying ethical issues. Eur J Cancer Prev. Jan 2019;28(1):27-32. [ FREE Full text ] [ CrossRef ] [ Medline ]
  • Kim A, Gitlin M, Fadli E, McGarvey N, Cong Z, Chung KC. Breast, colorectal, lung, prostate, and cervical cancer screening prevalence in a large commercial and Medicare advantage plan, 2008-2020. Prev Med Rep. Dec 2022;30:102046. [ FREE Full text ] [ CrossRef ] [ Medline ]
  • Chan TK, Tan LW, van Dam RM, Seow WJ. Cancer screening knowledge and behavior in a multi-ethnic Asian population: the Singapore Community Health Study. Front Oncol. Aug 12, 2021;11:684917. [ FREE Full text ] [ CrossRef ] [ Medline ]
  • de Bekker-Grob EW, Donkers B, Veldwijk J, Jonker MF, Buis S, Huisman J, et al. What factors influence non-participation most in colorectal cancer screening? A discrete choice experiment. Patient. Mar 05, 2021;14(2):269-281. [ FREE Full text ] [ CrossRef ] [ Medline ]
  • Schrag D, Beer TM, McDonnell CH3, Nadauld L, Dilaveri CA, Reid R, et al. Blood-based tests for multicancer early detection (PATHFINDER): a prospective cohort study. Lancet. Oct 07, 2023;402(10409):1251-1260. [ CrossRef ] [ Medline ]
  • Wang CP, Miller SJ, Shaukat A, Jandorf LH, Greenwald DA, Itzkowitz SH. Blood-based colorectal cancer screening: are we ready for the next frontier? Lancet Gastroenterol Hepatol. Oct 2023;8(10):870-872. [ FREE Full text ] [ CrossRef ] [ Medline ]
  • Epi proColon. Epigenomics, Inc. 2016. URL: https://www.labcorp.com/tests/related-documents/EpiPatient [accessed 2023-09-27]
  • Shaukat A, Levin TR. Current and future colorectal cancer screening strategies. Nat Rev Gastroenterol Hepatol. Aug 2022;19(8):521-531. [ FREE Full text ] [ CrossRef ] [ Medline ]
  • Johnson DA, Barclay RL, Mergener K, Weiss G, König T, Beck J, et al. Plasma Septin9 versus fecal immunochemical testing for colorectal cancer screening: a prospective multicenter study. PLoS One. 2014;9(6):e98238. [ FREE Full text ] [ CrossRef ] [ Medline ]
  • Ahlquist DA, Taylor WR, Mahoney DW, Zou H, Domanico M, Thibodeau SN, et al. The stool DNA test is more accurate than the plasma septin 9 test in detecting colorectal neoplasia. Clin Gastroenterol Hepatol. Mar 2012;10(3):272-7.e1. [ FREE Full text ] [ CrossRef ] [ Medline ]
  • Colorectal cancer: screening. U.S. Preventive Services Task Force. May 18, 2021. URL: https://www.uspreventiveservicestaskforce.org/uspstf/recommendation/colorectal-cancer-screening [accessed 2023-09-27]
  • Colorectal cancer (bowel cancer) symptoms, screening and diagnosis: screening and testing. Digestive Cancers Europe. URL: https://digestivecancers.eu/colorectal-screening/ [accessed 2023-09-27]
  • Cancer screening. National Cancer Centre Singapore. URL: https://www.nccs.com.sg/patient-care/cancer-types/pages/cancer-screening.aspx [accessed 2023-09-27]
  • Chen H, Lu B, Dai M. Colorectal cancer screening in china: status, challenges, and prospects - China, 2022. China CDC Wkly. Apr 15, 2022;4(15):322-328. [ FREE Full text ] [ CrossRef ] [ Medline ]
  • Symonds EL, Cock C, Meng R, Cole SR, Fraser RJ, Young GP. Uptake of a colorectal cancer screening blood test in people with elevated risk for cancer who cannot or will not complete a faecal occult blood test. Eur J Cancer Prev. Sep 2018;27(5):425-432. [ CrossRef ] [ Medline ]
  • Ladabaum U, Allen J, Wandell M, Ramsey S. Colorectal cancer screening with blood-based biomarkers: cost-effectiveness of methylated septin 9 DNA versus current strategies. Cancer Epidemiol Biomarkers Prev. Sep 2013;22(9):1567-1576. [ CrossRef ] [ Medline ]
  • Ioannou S, Sutherland K, Sussman DA, Deshpande AR. Increasing uptake of colon cancer screening in a medically underserved population with the addition of blood-based testing. BMC Cancer. Aug 28, 2021;21(1):966. [ FREE Full text ] [ CrossRef ] [ Medline ]
  • Liles EG, Coronado GD, Perrin N, Harte AH, Nungesser R, Quigley N, et al. Uptake of a colorectal cancer screening blood test is higher than of a fecal test offered in clinic: a randomized trial. Cancer Treat Res Commun. 2017;10:27-31. [ FREE Full text ] [ CrossRef ]
  • Symonds EL, Hughes D, Flight I, Woodman R, Chen G, Ratcliffe J, et al. A randomized controlled trial testing provision of fecal and blood test options on participation for colorectal cancer screening. Cancer Prev Res (Phila). Sep 2019;12(9):631-640. [ CrossRef ] [ Medline ]
  • Liang PS, Zaman A, Kaminsky A, Cui Y, Castillo G, Tenner CT, et al. Blood test increases colorectal cancer screening in persons who declined colonoscopy and fecal immunochemical test: a randomized controlled trial. Clin Gastroenterol Hepatol. Oct 2023;21(11):2951-7.e2. [ FREE Full text ] [ CrossRef ] [ Medline ]
  • Coronado GD, Jenkins CL, Shuster E, Johnson C, Amy D, Cook J, et al. Blood-based colorectal cancer screening in an integrated health system: a randomised trial of patient adherence. Gut. Mar 07, 2024;73(4):622-628. [ CrossRef ] [ Medline ]
  • Cocco P, Ayaz-Shah A, Messenger MP, West RM, Shinkins B. Target Product Profiles for medical tests: a systematic review of current methods. BMC Med. May 11, 2020;18(1):119. [ FREE Full text ] [ CrossRef ] [ Medline ]
  • Wang Y, Rattanavipapong W, Teerawattananon Y. Using health technology assessment to set priority, inform target product profiles, and design clinical study for health innovation. Technol Forecast Soc Change. Nov 2021;172:121000. [ FREE Full text ] [ CrossRef ] [ Medline ]
  • Ali O, Gupta S, Brain K, Lifford KJ, Paranjothy S, Dolwani S. Acceptability of alternative technologies compared with faecal immunochemical test and/or colonoscopy in colorectal cancer screening: a systematic review. J Med Screen. Mar 2023;30(1):14-27. [ FREE Full text ] [ CrossRef ] [ Medline ]
  • Brinkmann M, Fricke LM, Diedrich L, Robra BP, Krauth C, Dreier M. Attributes in stated preference elicitation studies on colorectal cancer screening and their relative importance for decision-making among screenees: a systematic review. Health Econ Rev. Sep 22, 2022;12(1):49. [ FREE Full text ] [ CrossRef ] [ Medline ]
  • Ryan M, Gerard K, Amaya-Amaya M. Using Discrete Choice Experiments to Value Health and Health Care. Dordrecht, The Netherlands. Springer; 2008.
  • Lancaster KJ. A new approach to consumer theory. J Political Econ. Apr 1966;74(2):132-157. [ CrossRef ]
  • Lancsar E, Louviere J. Conducting discrete choice experiments to inform healthcare decision making: a user's guide. Pharmacoeconomics. 2008;26(8):661-677. [ CrossRef ] [ Medline ]
  • Coast J, Al-Janabi H, Sutton EJ, Horrocks SA, Vosper AJ, Swancutt DR, et al. Using qualitative methods for attribute development for discrete choice experiments: issues and recommendations. Health Econ. Jun 2012;21(6):730-741. [ CrossRef ] [ Medline ]
  • Inadomi JM. Cost-effectiveness of blood-based biomarkers for colorectal cancer screening-an ounce of prevention is worth a pound of cure. JAMA Netw Open. Nov 01, 2023;6(11):e2343346. [ FREE Full text ] [ CrossRef ] [ Medline ]
  • Crockett RA, Weinman J, Hankins M, Marteau T. Time orientation and health-related behaviour: measurement in general population samples. Psychol Health. Mar 2009;24(3):333-350. [ FREE Full text ] [ CrossRef ] [ Medline ]
  • Carleton RN, Norton MA, Asmundson GJ. Fearing the unknown: a short version of the Intolerance of Uncertainty Scale. J Anxiety Disord. 2007;21(1):105-117. [ CrossRef ] [ Medline ]
  • Saracino R, Kolva E, Rosenfeld B, Breitbart W. Measuring social support in patients with advanced medical illnesses: an analysis of the Duke-UNC Functional Social Support Questionnaire. Palliat Support Care. Oct 2015;13(5):1153-1163. [ FREE Full text ] [ CrossRef ] [ Medline ]
  • Johnson R, Orme B. Getting the most from CBC. Sawtooth Software Research Paper Series, Sawtooth Software. 2003. URL: https:/​/scholar.​google.com/​scholar_lookup?title=Getting+the+most+from+CBC&author=R+Johnson&author=B+Orme&publication_year=2003& [accessed 2021-10-01]
  • Sarrias M, Daziano R. Multinomial logit models with continuous and discrete individual heterogeneity in R: the gmnl package. J Stat Softw. 2017;79(2):1-46. [ FREE Full text ] [ CrossRef ]
  • McFadden D, Train K. Mixed MNL models for discrete response. J Appl Econ. 2000;15(5):447-470. [ FREE Full text ] [ CrossRef ]
  • Veldwijk J, Marceta SM, Swait JD, Lipman SA, de Bekker-Grob EW. Taking the shortcut: simplifying heuristics in discrete choice experiments. Patient. Jul 2023;16(4):301-315. [ FREE Full text ] [ CrossRef ] [ Medline ]
  • Johnson FR, Yang JC, Reed SD. The internal validity of discrete choice experiment data: a testing tool for quantitative assessments. Value Health. Feb 2019;22(2):157-160. [ FREE Full text ] [ CrossRef ] [ Medline ]
  • Wang Y, Faradiba D, Del Rio Vilas VJ, Asaria M, Chen YT, Babigumira JB, et al. The relative importance of vulnerability and efficiency in COVID-19 contact tracing programmes: a discrete choice experiment. Int J Public Health. Jul 20, 2022;67:1604958. [ FREE Full text ] [ CrossRef ] [ Medline ]
  • Ang IY, Wang Y, Tyagi S, Koh GC, Cook AR. Preferences and willingness-to-pay for a blood pressure telemonitoring program using a discrete choice experiment. NPJ Digit Med. Sep 25, 2023;6(1):176. [ FREE Full text ] [ CrossRef ] [ Medline ]
  • R Core Team. R: a language and environment for statistical computing. R Foundation for Statistical Computing. 2013. URL: http://www.R-project.org [accessed 2024-06-11]
  • Xu Y, Levy BT, Daly JM, Bergus GR, Dunkelberg JC. Comparison of patient preferences for fecal immunochemical test or colonoscopy using the analytic hierarchy process. BMC Health Serv Res. Apr 23, 2015;15:175. [ FREE Full text ] [ CrossRef ] [ Medline ]
  • Schroy PC3, Lal S, Glick JT, Robinson PA, Zamor P, Heeren TC. Patient preferences for colorectal cancer screening: how does stool DNA testing fare? Am J Manag Care. Jul 2007;13(7):393-400. [ FREE Full text ] [ Medline ]
  • DeBourcy AC, Lichtenberger S, Felton S, Butterfield KT, Ahnen DJ, Denberg TD. Community-based preferences for stool cards versus colonoscopy in colorectal cancer screening. J Gen Intern Med. Feb 21, 2008;23(2):169-174. [ FREE Full text ] [ CrossRef ] [ Medline ]
  • Taber JM, Aspinwall LG, Heichman KA, Kinney AY. Preferences for blood-based colon cancer screening differ by race/ethnicity. Am J Health Behav. May 01, 2014;38(3):351-361. [ CrossRef ]
  • Osborne JM, Wilson C, Moore V, Gregory T, Flight I, Young GP. Sample preference for colorectal cancer screening tests: blood or stool? Open J Prevent Med. Aug 2012;02(03):326-331. [ CrossRef ]
  • Adler A, Geiger S, Keil A, Bias H, Schatz P, deVos T, et al. Improving compliance to colorectal cancer screening using blood and stool based tests in patients refusing screening colonoscopy in Germany. BMC Gastroenterol. Oct 17, 2014;14(1):1-8. [ FREE Full text ] [ CrossRef ]
  • Health Promotion Board homepage. Health Promotion Board. URL: https://hpb.gov.sg [accessed 2023-09-27]
  • Ran T, Cheng CY, Misselwitz B, Brenner H, Ubels J, Schlander M. Cost-effectiveness of colorectal cancer screening strategies-a systematic review. Clin Gastroenterol Hepatol. Sep 2019;17(10):1969-81.e15. [ CrossRef ] [ Medline ]
  • Gheysariyeha F, Rahimi F, Tabesh E, Hemami MR, Adibi P, Rezayatmand R. Cost-effectiveness of colorectal cancer screening strategies: a systematic review. Eur J Cancer Care (Engl). Nov 16, 2022;31(6):e13673. [ CrossRef ] [ Medline ]
  • Peterse EF, Meester RG, de Jonge L, Omidvari AH, Alarid-Escudero F, Knudsen AB, et al. Comparing the cost-effectiveness of innovative colorectal cancer screening tests. J Natl Cancer Inst. Feb 01, 2021;113(2):154-161. [ FREE Full text ] [ CrossRef ] [ Medline ]
  • Ladabaum U, Alvarez-Osorio L, Rösch T, Brueggenjuergen B. Cost-effectiveness of colorectal cancer screening in Germany: current endoscopic and fecal testing strategies versus plasma methylated Septin 9 DNA. Endosc Int Open. Jun 6, 2014;2(2):E96-104. [ FREE Full text ] [ CrossRef ] [ Medline ]
  • Mohl JT, Ciemins EL, Miller-Wilson LA, Gillen A, Luo R, Colangelo F. Rates of follow-up colonoscopy after a positive stool-based screening test result for colorectal cancer among health care organizations in the US, 2017-2020. JAMA Netw Open. Jan 03, 2023;6(1):e2251384. [ FREE Full text ] [ CrossRef ] [ Medline ]
  • Escaron AL, Garcia J, Petrik AF, Ruiz E, Nyongesa DB, Thompson JH, et al. Colonoscopy following an abnormal fecal test result from an annual colorectal cancer screening program in a federally qualified health center. J Prim Care Community Health. Nov 30, 2022;13:21501319221138423. [ FREE Full text ] [ CrossRef ] [ Medline ]
  • Petersen MM, Kleif J, Jørgensen LN, Hendel JW, Seidelin JB, Madsen MR, et al. Optimizing screening for colorectal cancer: an algorithm combining fecal immunochemical test, blood-based cancer-associated proteins and demographics to reduce colonoscopy burden. Clin Colorectal Cancer. Jun 2023;22(2):199-210. [ FREE Full text ] [ CrossRef ] [ Medline ]

Abbreviations

colorectal cancer
conditional relative importance
discrete choice experiment
fecal immunochemical testing
mixed-mixed multinomial logit
mixed logit
National University of Singapore
randomized controlled trial

Edited by A Mavragani; submitted 03.10.23; peer-reviewed by R Liu, Hsiao-Yu Yang; comments to author 10.01.24; revised version received 29.01.24; accepted 21.05.24; published 17.07.24.

©Clarence Ong, Alex R Cook, Ker-Kan Tan, Yi Wang. Originally published in JMIR Public Health and Surveillance (https://publichealth.jmir.org), 17.07.2024.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Public Health and Surveillance, is properly cited. The complete bibliographic information, a link to the original publication on https://publichealth.jmir.org, as well as this copyright and license information must be included.

  • Environment
  • Science & Technology
  • Business & Industry
  • Health & Public Welfare
  • Topics (CFR Indexing Terms)
  • Public Inspection
  • Presidential Documents
  • Document Search
  • Advanced Document Search
  • Public Inspection Search
  • Reader Aids Home
  • Office of the Federal Register Announcements
  • Using FederalRegister.Gov
  • Understanding the Federal Register
  • Recent Site Updates
  • Federal Register & CFR Statistics
  • Videos & Tutorials
  • Developer Resources
  • Government Policy and OFR Procedures
  • Congressional Review
  • My Clipboard
  • My Comments
  • My Subscriptions
  • Sign In / Sign Up
  • Site Feedback
  • Search the Federal Register

The Federal Register

The daily journal of the united states government.

  • Legal Status

This site displays a prototype of a “Web 2.0” version of the daily Federal Register. It is not an official legal edition of the Federal Register, and does not replace the official print version or the official electronic version on GPO’s govinfo.gov.

The documents posted on this site are XML renditions of published Federal Register documents. Each document posted on the site includes a link to the corresponding official PDF file on govinfo.gov. This prototype edition of the daily Federal Register on FederalRegister.gov will remain an unofficial informational resource until the Administrative Committee of the Federal Register (ACFR) issues a regulation granting it official legal status. For complete information about, and access to, our official publications and services, go to About the Federal Register on NARA's archives.gov.

The OFR/GPO partnership is committed to presenting accurate and reliable regulatory information on FederalRegister.gov with the objective of establishing the XML-based Federal Register as an ACFR-sanctioned publication in the future. While every effort has been made to ensure that the material on FederalRegister.gov is accurately displayed, consistent with the official SGML-based PDF version on govinfo.gov, those relying on it for legal research should verify their results against an official edition of the Federal Register. Until the ACFR grants it official status, the XML rendition of the daily Federal Register on FederalRegister.gov does not provide legal notice to the public or judicial notice to the courts.

Order Granting Conditional Substituted Compliance in Connection With Certain Capital and Financial Reporting Requirements Applicable to Nonbank Swap Dealers Subject to Regulation by the United Kingdom Prudential Regulation Authority

A Rule by the Commodity Futures Trading Commission on 07/18/2024

Document Details

Information about this document as published in the Federal Register .

Published Document

This document has been published in the Federal Register . Use the PDF linked in the document sidebar for the official electronic format.

Enhanced Content - Table of Contents

This table of contents is a navigational tool, processed from the headings within the legal text of Federal Register documents. This repetition of headings to form internal navigation links has no substantive legal effect.

FOR FURTHER INFORMATION CONTACT:

Supplementary information:, i. introduction, a. regulatory background—cftc capital, margin, and financial reporting requirements for swap dealers and major swap participants, b. commission capital comparability determinations for non-u.s. nonbank swap dealers and non-u.s. nonbank major swap participants, c. application for a comparability determination for nonbank swap dealers domiciled in the united kingdom and subject to regulation by the prudential regulation authority, d. proposed comparability determination and proposed comparability order for pra-designated uk nonbank swap dealers, e. general comments on the uk application and the commission's proposed finding of comparability between the cftc capital rules and cftc financial reporting rules and the uk pra capital rules and the uk pra financial reporting rules, ii. final capital and financial reporting comparability determination and comparability order, a. regulatory objectives of cftc capital rules and cftc financial reporting rules and uk pra capital rules and uk pra financial reporting rules, 1. preliminary determination, 2. comment analysis and final determination, b. nonbank swap dealer qualifying capital, c. nonbank swap dealer minimum capital requirement, 1. introduction to nonbank swap dealer minimum capital requirements, 2. preliminary determination and comment analysis, a. fixed amount minimum capital requirement, b. minimum capital requirement based on risk-weighted assets, c. minimum capital requirement based on the uncleared swap margin amount, 3. final determination, d. nonbank swap dealer financial reporting requirements, 1. proposed determination, e. notice requirements, 2. comments and final determination, f. supervision and enforcement, iii. final capital comparability determination and comparability order, a. commission's final comparability determination, b. order providing conditional capital comparability determination for certain pra-designated uk nonbank swap dealers, appendices to order granting conditional substituted compliance in connection with certain capital and financial reporting requirements applicable to nonbank swap dealers subject to regulation by the united kingdom prudential regulation authority—voting summary and chairman's and commissioners' statements, appendix 1—voting summary, appendix 2—statement of chairman rostin behnam, appendix 3—statement of commissioner kristin n. johnson, today's final comparability determinations, appendix 4—statement of commissioner caroline d. pham, enhanced content - submit public comment.

  • This feature is not available for this document.

Enhanced Content - Read Public Comments

Enhanced content - sharing.

  • Email this document to a friend

Enhanced Content - Document Print View

  • Print this document

Enhanced Content - Document Tools

These tools are designed to help you understand the official document better and aid in comparing the online edition to the print edition.

These markup elements allow the user to see how the document follows the Document Drafting Handbook that agencies use to create their documents. These can be useful for better understanding how a document is structured but are not part of the published document itself.

Enhanced Content - Developer Tools

This document is available in the following developer friendly formats:.

  • JSON: Normalized attributes and metadata
  • XML: Original full text XML
  • MODS: Government Publishing Office metadata

More information and documentation can be found in our developer tools pages .

Official Content

  • View printed version (PDF)

This PDF is the current document as it appeared on Public Inspection on 07/17/2024 at 8:45 am. It was viewed 3 times while on Public Inspection.

If you are using public inspection listings for legal research, you should verify the contents of the documents against a final, official edition of the Federal Register. Only official editions of the Federal Register provide legal notice of publication to the public and judicial notice to the courts under 44 U.S.C. 1503 & 1507 . Learn more here .

Commodity Futures Trading Commission.

On February 5, 2024, the Commodity Futures Trading Commission issued a notice and request for comment on an application submitted by the Institute of International Bankers, International Swaps and Derivatives Association, and Securities Industry and Financial Markets Association requesting that the Commission determine that registered nonbank swap dealers organized and domiciled in the United Kingdom may comply with certain capital and financial reporting requirements under the Commodity Exchange Act and Commission regulations by being subject to, and complying with, corresponding capital and financial reporting requirements of the United Kingdom Prudential Regulation Authority. The Commission also solicited public comment on a proposed comparability determination and related order providing for the conditional availability of substituted compliance in connection with the application.

The Commission is adopting the proposed order with certain modifications and clarifications to address comments. The final order provides that a nonbank swap dealer organized and domiciled in the United Kingdom may satisfy the capital requirements under the Commodity Exchange Act and Commission applicable Commission regulations and the financial reporting rules under the Commodity Exchange Act and applicable Commission regulations by complying with certain specified United Kingdom laws and regulations and conditions set forth in the order.

This determination was made by the Commission on June 24, 2024.

Amanda L. Olear, Director, 202-418-5283, [email protected] ; Thomas Smith, Deputy Director, 202-418-5495, [email protected] ; Rafael Martinez, Associate Director, 202-418-5462, [email protected] ; Liliya Bozhanova, Special Counsel, 202-418-6232, [email protected] ; Joo Hong, Risk Analyst, 202-418-6221, [email protected] ; Justin McPhee, Risk Analyst, 202-418-6223; [email protected] , Market Participants Division; Commodity Futures Trading Commission, Three Lafayette Centre, 1155 21st Street NW, Washington, DC 20581.

The Commodity Futures Trading Commission (“Commission” or “CFTC”) is issuing an order providing that registered nonbank swap dealers (“SDs”) organized and domiciled in the Start Printed Page 58536 United Kingdom (“UK”) may satisfy certain capital and financial reporting requirements under the Commodity Exchange Act (“CEA”)  [ 1 ] and Commission regulations  [ 2 ] by being subject to, and complying with, comparable capital and financial reporting requirements under relevant UK laws and regulations, subject to certain conditions set forth in the order below. The order is based on the proposed comparability determination and related proposed order published by the Commission on February 5, 2024, [ 3 ] as modified in certain aspects to address comments and to clarify its terms.

Section 4s(e) of the CEA  [ 4 ] directs the Commission and “prudential regulators”  [ 5 ] to impose capital requirements on SDs and major swap participants (“MSPs”) registered with the Commission. [ 6 ] Section 4s(e) also directs the Commission and prudential regulators to adopt regulations imposing initial and variation margin requirements on swaps entered into by SDs and MSPs that are not cleared by a registered derivatives clearing organization (“uncleared swaps”).

Section 4s(e) applies a bifurcated approach with respect to the above Congressional directives, requiring each SD and MSP that is subject to the regulation of a prudential regulator (“bank SD” and “bank MSP,” respectively) to meet the minimum capital requirements and uncleared swaps margin requirements adopted by the applicable prudential regulator, and requiring each SD and MSP that is not subject to the regulation of a prudential regulator (“nonbank SD” and “nonbank MSP,” respectively) to meet the minimum capital requirements and uncleared swaps margin requirements adopted by the Commission. [ 7 ] Therefore, the Commission's authority to impose capital requirements and margin requirements for uncleared swap transactions extends to nonbank SDs and nonbank MSPs, including nonbanking subsidiaries of bank holding companies regulated by the Federal Reserve Board. [ 8 ]

The prudential regulators implemented section 4s(e) in 2015 by amending existing capital requirements applicable to bank SDs and bank MSPs to incorporate swap transactions into their respective bank capital frameworks, and by adopting rules imposing initial and variation margin requirements on bank SDs and bank MSPs that engage in uncleared swap transactions. [ 9 ] The Commission adopted final rules imposing initial and variation margin obligations on nonbank SDs and nonbank MSPs for uncleared swap transactions on January 6, 2016. [ 10 ] The Commission also approved final capital requirements for nonbank SDs and nonbank MSPs on July 24, 2020, which were published in the Federal Register on September 15, 2020 with a compliance date of October 6, 2021 (“CFTC Capital Rules”). [ 11 ]

Section 4s(f) of the CEA addresses SD and MSP financial reporting requirements. [ 12 ] Section 4s(f) authorizes the Commission to adopt rules imposing financial condition reporting obligations on all SDs and MSPs ( i.e., nonbank SDs, nonbank MSPs, bank SDs, and bank MSPs). Specifically, section 4s(f)(1)(A) provides, in relevant part, that each registered SD and MSP must make financial condition reports as required by regulations adopted by the Commission. [ 13 ] The Commission's financial reporting obligations were adopted with the Commission's nonbank SD and nonbank MSP capital requirements, and also had a compliance date of October 6, 2021 (“CFTC Financial Reporting Rules”). [ 14 ]

Commission Regulation 23.106 establishes a substituted compliance framework whereby the Commission may determine that compliance by a non-U.S. domiciled nonbank SD or non-U.S. domiciled nonbank MSP with its home country's capital and financial reporting requirements will satisfy all or parts of the CFTC Capital Rules and all or parts of the CFTC Financial Reporting Rules (such a determination referred to as a “Comparability Determination”). [ 15 ] The Commission's capital adequacy and financial reporting requirements are designed to address and manage risks Start Printed Page 58537 that arise from a firm's operation as an SD or MSP. Given their functions, both sets of requirements and rules must be applied on an entity-level basis (meaning that the rules apply on a firm-wide basis, irrespective of the type of transactions involved) to effectively address risk to the firm as a whole. The availability of such substituted compliance is conditioned upon the Commission issuing a Comparability Determination finding that the relevant foreign jurisdiction's capital adequacy and financial reporting requirements for non-U.S. nonbank SDs and/or non-U.S. nonbank MSPs are comparable to the corresponding CFTC Capital Rules and CFTC Financial Reporting Rules. The Commission would issue a Comparability Determination in the form of an order (“Comparability Order”). [ 16 ]

The Commission's approach for conducting a Comparability Determination with respect to the CFTC Capital Rules and the CFTC Financial Reporting Rules is a principles-based, holistic approach that focuses on assessing whether the applicable foreign jurisdiction's capital and financial reporting requirements have comparable objectives with, and achieve comparable outcomes to, corresponding CFTC requirements. [ 17 ] The Commission's assessment is not a line-by-line evaluation or comparison of a foreign jurisdiction's regulatory requirements with the Commission's requirements. [ 18 ] In performing the analysis, the Commission recognizes that jurisdictions may adopt differing approaches to achieving regulatory objectives and outcomes, and the Commission will focus on whether the foreign jurisdiction's capital and financial reporting requirements are based on regulatory objectives, and produce regulatory outcomes, that are comparable to the Commission's in purpose and effect, and not whether they are comparable in every aspect or contain identical elements.

A person requesting a Comparability Determination is required to submit an application to the Commission containing: (i) a description of the objectives of the relevant foreign jurisdiction's capital adequacy and financial reporting requirements applicable to entities that are subject to the CFTC Capital Rules and the CFTC Financial Reporting Rules; (ii) a description (including specific legal and regulatory provisions) of how the relevant foreign jurisdiction's capital adequacy and financial reporting requirements address the elements of the CFTC Capital Rules and CFTC Financial Reporting Rules, including, at a minimum, the methodologies for establishing and calculating capital adequacy requirements and whether such methodologies comport with international standards; and (iii) a description of the ability of the relevant foreign regulatory authority to supervise and enforce compliance with the relevant foreign jurisdiction's capital adequacy and financial reporting requirements. The applicant must also submit, upon request, such other information and documentation as the Commission deems necessary to evaluate the comparability of the capital adequacy and financial reporting requirements of the foreign jurisdiction. [ 19 ]

The Commission will consider an application for a Comparability Determination to be a representation by the applicant that the laws and regulations of the foreign jurisdiction that are submitted in support of the application are finalized and in force, that the description of such laws and regulations is accurate and complete, and that, unless otherwise noted, the scope of such laws and regulations encompasses the relevant non-U.S. nonbank SDs and/or non-U.S. nonbank MSPs domiciled in the foreign jurisdiction. [ 20 ] Each non-U.S. nonbank SD or non-U.S. nonbank MSP that seeks to rely on a Comparability Order is responsible for determining whether it is subject to the foreign laws and regulations found comparable in the Comparability Order. A non-U.S. nonbank SD or non-U.S. nonbank MSP that is not legally required to comply with a foreign jurisdiction's laws and/or regulations determined to be comparable in a Comparability Order may not voluntarily comply with such laws and/or regulations in lieu of compliance with the CFTC Capital Rules or the CFTC Financial Reporting Rules.

The Commission may consider all relevant factors in making a Comparability Determination, including: (i) the scope and objectives of the relevant foreign jurisdiction's capital and financial reporting requirements; (ii) whether the relevant foreign jurisdiction's capital and financial reporting requirements achieve comparable outcomes to the Commission's corresponding capital requirements and financial reporting requirements; (iii) the ability of the relevant foreign regulatory authority or authorities to supervise and enforce compliance with the relevant foreign jurisdiction's capital adequacy and financial reporting requirements; and (iv) any other facts or circumstances the Commission deems relevant, including whether the Commission and foreign regulatory authority or authorities have a memorandum of understanding (“MOU”) or similar arrangement that would facilitate supervisory cooperation. [ 21 ]

In performing the comparability assessment for foreign nonbank SDs, the Commission's review will include the extent to which the foreign jurisdiction's requirements address: (i) the process of establishing minimum capital requirements for nonbank SDs and how such process addresses risk, including market risk and credit risk of the nonbank SD's on-balance sheet and off-balance sheet exposures; (ii) the types of equity and debt instruments that qualify as regulatory capital in meeting minimum requirements; (iii) the financial reports and other financial information submitted by a nonbank SD to its relevant regulatory authority and whether such information provides the regulatory authority with the means necessary to effectively monitor the financial condition of the nonbank SD; and (iv) the regulatory notices and other communications between a nonbank SD and its foreign regulatory authority that address potential adverse financial or operational issues that may impact the firm. With respect to the ability of the relevant foreign regulatory authority to supervise and enforce compliance with the foreign jurisdiction's capital adequacy and financial reporting requirements, the Commission's review will include an assessment of the foreign jurisdiction's surveillance program for monitoring nonbank SDs' compliance with such capital adequacy and financial reporting requirements, and the disciplinary process imposed on firms that fail to comply with such requirements. [ 22 ]

Commission Regulation 23.106 further provides that the Commission may impose any terms or conditions that it deems appropriate in issuing a Comparability Determination. [ 23 ] Any specific terms or conditions with respect to capital adequacy or financial reporting requirements will be set forth in the Commission's Comparability Order. As a general condition to all Comparability Orders, the Commission will require notification from the applicants of any material changes to information submitted by the applicants in support of a comparability finding, including, but not limited to, changes in the foreign jurisdiction's relevant laws and regulations, as well as changes to the relevant supervisory or regulatory regime.

To rely on a Comparability Order, a nonbank SD or nonbank MSP domiciled in the foreign jurisdiction and subject to supervision by the relevant regulatory authority (or authorities) in the foreign jurisdiction must file a notice with the Commission of its intent to comply with the applicable capital adequacy and financial reporting requirements of the foreign jurisdiction set forth in the Comparability Order in lieu of all or parts of the CFTC Capital Rules and/or CFTC Financial Reporting Rules. [ 24 ] Notices must be filed electronically with the Commission's Market Participants Division (“MPD”). [ 25 ] The filing of a notice by a non-U.S. nonbank SD or non-U.S. nonbank MSP provides MPD staff with the opportunity to engage with the firm and to obtain representations that it is subject to, and complies with, the laws and regulations cited in the Comparability Order and that it will comply with any listed conditions. MPD will issue a letter under delegated authority from the Commission confirming that the non-U.S. nonbank SD or non-U.S. nonbank MSP may comply with the foreign laws and regulations cited in the Comparability Order in lieu of complying with the CFTC Capital Rules and CFTC Financial Reporting Rules upon MPD's confirmation through discussions with the non-U.S. nonbank SD or non-U.S. nonbank MSP that the firm is subject to, and complies with, such foreign laws and regulations, is subject to the jurisdiction of the applicable foreign regulatory authority (or authorities), and can meet the conditions in the Comparability Order. [ 26 ]

Each non-U.S. nonbank SD and each non-U.S. nonbank MSP that receives confirmation from the Commission that it may comply with a foreign jurisdiction's capital adequacy and financial reporting requirements will be deemed by the Commission to be in compliance with the corresponding CFTC Capital Rules and/or CFTC Financial Reporting Rules. [ 27 ] A non-U.S. nonbank SD or non-U.S. nonbank MSP that receives confirmation of substituted compliance remains subject, however, to the Commission's examination and enforcement authority. [ 28 ] Accordingly, if a nonbank SD or nonbank MSP fails to comply with the foreign jurisdiction's capital adequacy and/or financial reporting requirements, the Commission may initiate an action for a violation of the corresponding CFTC Capital Rules and/or CFTC Financial Reporting Rules. [ 29 ] In addition, a finding of a violation by a foreign jurisdiction's regulatory authority is not a prerequisite for the exercise of such examination and enforcement authority by the Commission.

On May 4, 2021, the Institute of International Bankers (“IIB”), International Swaps and Derivatives Association (“ISDA”), and Securities Industry and Financial Markets Association (“SIFMA”) (together, the “Applicants”) submitted an application (the “UK Application”) requesting that the Commission conduct a Comparability Determination and issue a Comparability Order finding that compliance with certain designated capital and financial reporting requirements of the United Kingdom satisfy certain Commission capital rules and financial reporting rules for nonbank SDs. [ 30 ] Specifically, the Applicants requested that the Commission determine that registered nonbank SDs  [ 31 ] organized and domiciled within the UK, licensed as investment firms, and designated for prudential supervision by the UK Prudential Regulation Authority (“PRA-designated UK nonbank SDs”), may satisfy corresponding CFTC Capital Rules and CFTC Financial Reporting Rules applicable to a nonbank SD under sections 4s(e) and (f) of the CEA and Commission Regulations 23.101 and 23.105. [ 32 ]

To be designated for prudential supervision by the UK Prudential Regulation Authority (“PRA”), a UK-domiciled investment firm must be authorized, or have requested authorization, to deal in investments as principal. [ 33 ] For an investment firm that is authorized, or has requested authorization, to deal in investments as principal, the PRA may designate the firm for prudential supervision if the PRA determines that the dealing activities of the firm should be a PRA-regulated activity. The PRA considers the following in determining whether an investment firm should be subject to PRA supervision: (i) the assets of the investment firm; and (ii) where the investment firm is a member of a group, (a) the assets of other firms within the group that are authorized, or have sought authorization, to deal in investments as principal, (b) whether any other member of the group is subject to prudential supervision by the PRA, and (c) whether the investment firm's activities have, or might have, a material impact on the ability of the PRA to advance any of its objectives in relation to a PRA-authorized person in its group. [ 34 ] The PRA also must consult Start Printed Page 58539 with the FCA before designating a person for prudential supervision. [ 35 ]

The PRA also has issued a Statement of Policy providing further detail regarding the factors that are considered in assessing an investment firm for prudential supervision. [ 36 ] The factors include: (i) whether the firm's balance sheet exceeds an average of GBP 15 billion total gross assets over four quarters; (ii) where the investment firm is part of a group, whether the sum of the balance sheets of all firms within the group that are authorized, or have requested authorization, to deal in investments as principals exceeds an average of GBP 15 billion over four quarters; and/or (iii) where the firm is part of a group subject to PRA supervision, whether the investment firm's revenues, balance sheet and risk taking is significant relative to the group's revenues, balance sheet, and risk-taking. [ 37 ] There are currently six PRA-designated UK nonbank SDs registered with the Commission: Citigroup Global Markets Limited, Goldman Sachs International, Merrill Lynch International, Morgan Stanley & Co. International Plc, MUFG Securities EMEA Plc, and Nomura International Plc.

The Applicants represented that the capital and financial reporting framework applicable to PRA-designated UK nonbank SDs is primarily based on the framework established by the European Union's (“EU”) Capital Requirements Regulation  [ 38 ] and Capital Requirements Directive, [ 39 ] which set forth capital and financial reporting requirements applicable to “credit institutions”  [ 40 ] and “investment firms.”  [ 41 ] CRR, as a regulation, is directly applicable in all member states of the EU (“EU Member States”) and was, therefore, binding law in the UK during the UK's membership in the EU. [ 42 ] CRD, as a directive, was required to be transposed into EU Member States' national law, including UK law. [ 43 ] With regard to PRA-designated UK nonbank SDs, the UK implemented CRD primarily through a series of regulations, including the Capital Requirements Regulations 2013  [ 44 ] and the Capital Requirements (Capital Buffers and Macro-prudential Measures) Regulations 2014, [ 45 ] and the rules of the PRA. [ 46 ]

Following the UK's withdrawal from EU membership (“Brexit”), EU laws that were in effect and applicable as of December 31, 2020, were retained in UK law subject to certain non-substantive amendments seeking to reflect the UK's new position outside of the EU. [ 47 ] As such, directly applicable EU law, such as CRR, was converted into domestic UK law and UK legislation implementing EU directives, such as CRD, was preserved. The UK subsequently adopted additional changes, generally consistent with amendments introduced by the EU to CRR, CRD and other relevant EU provisions, [ 48 ] and incorporated certain CRR provisions in the PRA Rulebook. [ 49 ] The CRR provisions as applicable in the UK are referred hereafter as “UK CRR.”  [ 50 ] The UK capital and financial reporting framework also comprises UK-specific requirements in respect of certain matters. Requirements applicable to PRA-designated UK nonbank SDs are included in the PRA Rulebook. In addition, Commission Delegated Regulation (EU) 2015/61, [ 51 ] which supplements UK CRR with regard to liquidity coverage requirement for credit institutions, applies to PRA-designated UK nonbank SDs and imposes separate liquidity requirements to these firms. [ 52 ]

The Applicants also represented that in addition to UK CRR and the PRA Rulebook, the Banking Act 2009 and its related secondary legislation, through which the UK transposed the Bank Recovery and Resolution Directive (“BRRD”), include relevant UK capital requirements. [ 53 ] Specifically, pursuant to the Banking Act 2009 and its secondary legislation, the Bank of Start Printed Page 58540 England, in its role as resolution authority, requires certain investment firms, including PRA-designated UK nonbank SDs, to satisfy a firm-specific minimum requirement for own funds and eligible liabilities (“MREL”). [ 54 ]

UK CRR, Capital Requirements Regulations 2013, Capital Requirements (Capital Buffers and Macro-prudential Measures) Regulations 2014, Liquidity Coverage Delegated Regulation, relevant provisions of Banking Act 2009 and its secondary legislation, and relevant parts of the PRA Rulebook are referred to hereafter as the “UK PRA Capital Rules.”

The Applicants further represented that with respect to supervisory financial reporting, the framework applicable to PRA-designated UK nonbank SDs is also based on the EU requirements. In addition, the framework comprises PRA-specific rules for matters not addressed by the EU-based requirements. Specifically, Commission Implementing Regulation (EU) 680/2014, [ 55 ] which was initially retained in UK law following Brexit, supplemented CRR with implementing technical standards (“CRR Reporting ITS”) specifying, among other things, uniform formats and frequencies for the financial and capital requirements reporting required under CRR. [ 56 ] CRR Reporting ITS included templates for the common reporting (“COREP”) and the financial reporting (“FINREP”) that specify the contents of the EU-based supervisory reporting requirements. As part of the regulatory reforms that followed Brexit and sought to implement Basel standards, the PRA incorporated the entire body of the UK version of COREP and FINREP requirements into the PRA Rulebook to create a single source for reporting requirements for firms. [ 57 ] For PRA-designated UK nonbank SDs that are not subject to the EU-based FINREP requirements, the PRA Rulebook includes PRA-specific requirements. [ 58 ]

The Applicants also represented that the Companies Act 2006 contains provisions related to financial reporting, including a mandate that entities of a certain size be required to prepare annual audited financial statements and a strategic report. [ 59 ] UK CRR, relevant provisions of the PRA Rulebook, and relevant provisions of the Companies Act 2006, are collectively referred to hereafter as the “UK PRA Financial Reporting Rules.”

The Applicants also noted that the U.S. Securities and Exchange Commission (“SEC”) has issued orders permitting an SEC-registered nonbank security-based swap dealer domiciled in the UK (“UK nonbank SBSD”)  [ 60 ] to satisfy SEC capital  [ 61 ] and financial reporting requirements via substituted compliance with applicable UK capital and financial reporting. [ 62 ] The UK Order conditioned substituted compliance for capital requirements on a UK nonbank SBSD complying with specified laws and regulations, including relevant parts of UK CRR and the PRA Rulebook, and also maintaining total liquid assets in an amount that exceeds the UK nonbank SBSD's total liabilities by at least $100 million and by at least $20 million after applying certain deductions to the value of the liquid assets to reflect market, credit, and other potential risks to the value of the assets. [ 63 ]

On February 5, 2024, the Commission published the 2024 Proposal, seeking comment on the Application and the Commission's proposed Comparability Determination and related Comparability Order. [ 64 ] The 2024 Proposal set forth the Commission's preliminary Comparability Determination and proposed Comparability Order providing that, based on its review of the UK Application and applicable UK laws and/or rules, the Commission preliminarily found that the UK PRA Capital Rules and the UK PRA Financial Reporting Rules, subject to the conditions set forth in the proposed Comparability Order, achieve comparable outcomes and are comparable in purpose and effect to the CFTC Capital Rules and CFTC Financial Reporting Rules. [ 65 ] The Commission, however, noted that there were certain differences between the UK PRA Capital Rules and CFTC Capital Rules and certain differences between the UK PRA Financial Reporting Rules and the CFTC Financial Reporting Rules. As such, the Commission proposed certain conditions to the Comparability Order. [ 66 ] The proposed conditions were designed to promote consistency in regulatory outcomes, to reflect the scope of substituted compliance that would be available notwithstanding the differences, and to ensure that the Commission and National Futures Association (“NFA”) receive information to monitor PRA-designated UK nonbank SDs for ongoing compliance with the Comparability Start Printed Page 58541 Order. [ 67 ] The Commission further stated that, in its preliminary view, the identified differences would not be inconsistent with providing a substituted compliance framework for PRA-designated UK nonbank SDs subject to the conditions specified in the proposed Comparability Order. [ 68 ]

The proposed Comparability Order was limited to the comparison of the UK PRA Capital Rules to the CFTC Capital Rules' Bank-Based Capital Approach (“Bank-Based Approach”) for computing regulatory capital for nonbank SDs, which is based on certain capital requirements imposed by the Federal Reserve Board for bank holding companies. [ 69 ] As noted by the Commission in the 2024 Proposal, the Applicants have not requested, nor has the Commission performed, a comparison of the UK PRA Capital Rules to the Commission's TNW Approach or NLA Approach. [ 70 ]

The public comment period on the UK Application, the proposed Comparability Determination, and the proposed Comparability Order ended on March 24, 2024. The Commission received comments from the following four interested parties: Michael Ravnitzky (“Ravnitzky”); William J. Harrington (“Harrington”); Better Markets, Inc. (“Better Markets”); and the Applicants. [ 71 ]

The Applicants filed a comment letter generally expressing support for the proposed Comparability Determination and Comparability Order, agreeing with the Commission's overall analysis and determination of comparability of the CFTC Capital Rules and CFTC Financial Reporting Rules and the UK PRA Capital Rules and UK PRA Financial Reporting Rules. [ 72 ] The Applicants also included several technical comments, further discussed in section II. below, on the proposed conditions requiring PRA-designated UK nonbank SDs to file a notice with the Commission and NFA upon the occurrence of certain events. Finally, the Applicants recommended that the Commission refine the condition defining the scope of the UK PRA Capital Rules to specify that only the MREL-related provisions of the Banking Act 2009 would be considered part of UK PRA Capital Rules. [ 73 ] In support of their request, the Applicants stated that the reference to the Banking Act 2009 is included only because it imposes MREL on PRA-designated UK nonbank SDs. [ 74 ] The Commission notes that in the process leading to this Comparability Determination, the Commission has considered the Banking Act 2009 more broadly, including as it relates to the powers conferred to the PRA in its role as resolution authority. With respect to the definition of the UK PRA Capital Rules with which a PRA-designated UK nonbank SD must comply, however, the Commission believes that referring to the Banking Act 2009 only to the extent it imposes MREL on PRA-designated UK nonbank SDs is appropriate. Accordingly, the Commission has adjusted the language in final Condition 4 consistent with the Applicants' recommendation.

Conversely, two commenters disagreed with the CFTC's proposed Comparability Determination and proposed Comparability Order. [ 75 ] Better Markets asserted that the principles-based, holistic approach applied by the Commission, which assesses whether the applicable foreign jurisdiction's capital and financial requirements achieve comparable outcomes to the corresponding Commission requirements, “is insufficiently rigorous, leaving far too much room for inaccurate and unwarranted comparability determinations.”  [ 76 ] Better Markets further asserted that in an attempt to restore London to its status of a global financial center in the post-Brexit environment, both major political parties in the UK are promising “light touch” regulation and incentivizing regulatory arbitrage. [ 77 ]

The Commission does not believe that the principles-based, holistic assessment that it conducted on the comparability of the UK PRA Capital Rules and UK PRA Financial Reporting Rules with the CFTC Capital Rules and CFTC Financial Reporting Rules was “insufficiently rigorous,” nor does the Commission believe that it left “room for inaccurate and unwarranted comparability determinations.” The principles-based, holistic approach employed in the Comparability Determination was performed in accordance with the substituted compliance assessment framework adopted by the Commission for capital and financial reporting requirements for foreign nonbank SDs and set out in Commission Regulation 23.106. Consistent with this assessment framework, the Commission focused on whether the UK PRA Capital Rules and UK PRA Financial Reporting Rules are designed with the objective of ensuring overall safety and soundness of the PRA-designated UK nonbank SDs in a Start Printed Page 58542 manner that is comparable with the Commission's overall objective of ensuring the safety and soundness of nonbank SDs.

As stated in the 2024 Proposal, due to the detailed and complex nature of the capital frameworks, differences in how jurisdictions approach and implement the requirements are expected, even among jurisdictions that base their requirements on the principles and standards set forth in the BCBS framework. [ 78 ] Furthermore, as discussed in section I.B. above, when adopting Commission Regulation 23.106, the Commission stated that “its approach to substituted compliance is a principles-based, holistic approach that focuses on whether the foreign regulations are designed with the objectives of ensuring the overall safety and soundness of the [non-US nonbank SD] in a manner that is comparable with the Commission's overall capital and financial reporting requirements, and is not based on a line-by-line assessment or comparison of a foreign jurisdiction's regulatory requirements with the Commission's requirements.”  [ 79 ]

The approach and standards set forth in Commission Regulation 23.106, with the focus on “comparable outcomes,” are also consistent with the Commission's precedents of undertaking a principles-based, holistic assessment of the comparability of foreign regulatory regimes for purposes of substituted compliance for cross-border swap transactions. The Commission first outlined its approach to substituted compliance with respect to swaps requirements in 2013, when it issued an Interpretive Guidance and Policy Statement Regarding Compliance with Certain Swap Regulations. [ 80 ] In the Guidance, the Commission stated that “[i]n evaluating whether a particular category of foreign regulatory requirement(s) is comparable and comprehensive to the applicable requirement(s) under the CEA and Commission regulations, the Commission will take into consideration all relevant factors, including but not limited to, the comprehensiveness of those requirement(s), the scope and objectives of the relevant regulatory requirement(s), the comprehensiveness of the foreign regulator's supervisory compliance program, as well as the home jurisdiction's authority to support and enforce its oversight of the registrant.”  [ 81 ] The Commission emphasized that in this context, “comparable does not necessarily mean identical.”  [ 82 ] Rather, the Commission stated that it would evaluate whether the home jurisdiction's regulatory requirement is comparable to, and as comprehensive as, the corresponding U.S. regulatory requirement(s). [ 83 ] In conducting comparability determinations based on the policy set forth in the Guidance, the Commission noted that the “outcome-based” approach recognizes that “foreign regulatory systems differ and their approaches vary and may differ from how the Commission chose to address an issue, but that the foreign jurisdiction's regulatory requirements nonetheless achieve the regulatory outcome sought to be achieved by a certain provision of the CEA or Commission regulation.”  [ 84 ]

The Commission further elaborated on the required elements of comparability in 2016, when it issued final rules to address the cross-border application of the Commission's margin requirements for uncleared swap transactions. Specifically, the Commission stated that its substituted compliance approach reflects an outcome-based assessment of the comparability of a foreign jurisdiction's margin requirements with the Commission's corresponding requirements. [ 85 ] The Commission further stated that it would evaluate the objectives and outcomes of the foreign margin requirements in light of foreign regulator(s)' supervisory and enforcement authority. [ 86 ] Consistent with its previously stated position, the Commission recognized that jurisdictions may adopt different approaches to achieving the same outcome and, therefore, the assessment would focus on whether the foreign jurisdiction's margin requirements are comparable to the Commission's in purpose and effect, not whether they are comparable in every aspect or contain identical elements. [ 87 ] The Commission's policy thus reflects an understanding that a line-by-line evaluation of a foreign jurisdiction's regulatory regime is not the optimum approach to assessing the comparability of complex structures whose individual components may differ based on jurisdiction-specific considerations, but which achieve the objective and outcomes set forth in the Commission's framework.

With respect to the UK Application, the process leading to the Commission's Comparability Determination involved Commission staff reviewing relevant UK laws, rules, and regulations cited in the UK Application. Staff verified the assertions and citations contained in the UK Application regarding the specific UK PRA Capital Rules and UK PRA Financial Reporting Rules to the relevant UK laws, rules, and regulations. [ 88 ]

Commission staff also evaluated the comparability of the UK PRA Capital Rules and UK PRA Financial Reporting Rules with the CFTC Capital Rules and CFTC Financial Reporting Rules with respect to the following areas: (i) the process of establishing minimum capital requirements for PRA-designated UK nonbank SDs and how such process addresses risk, including market risk and credit risk of the PRA-designated UK nonbank SD's on-balance sheet and off-balance sheet exposures; (ii) the types of equity and debt instruments that qualify as regulatory capital in meeting a PRA-designated UK nonbank SD's minimum capital requirements; (iii) the financial reports and other financial information submitted by a PRA-designated UK nonbank SD to the PRA, and whether such information provides the PRA with the means necessary to effectively monitor the financial condition of the PRA-designated UK nonbank SD; and (iv) the regulatory notices and other communications between a PRA-designated UK nonbank SD and the PRA that address potential adverse financial or operational issues that may impact the firm. [ 89 ] With respect to the ability of the PRA to supervise and enforce compliance with the UK PRA Capital Rules and UK PRA Financial Reporting Rules, the Commission's assessment included a review of the PRA's surveillance program for monitoring compliance by PRA-designated UK nonbank SDs with the UK PRA Capital Rules and the UK PRA Financial Reporting Rules, and the disciplinary process imposed on firms that fail to comply with such requirements. [ 90 ] In conducting its assessment of the PRA's Start Printed Page 58543 regulatory and supervisory framework, the Commission did not identify elements supporting Better Markets' assertion that the framework is characterized by “light touch” regulation. [ 91 ]

Contrary to the position articulated by Better Markets regarding the nature of the comparability assessment, the Commission believes that the principles-based, holistic assessment of the UK PRA Capital Rules and UK PRA Financial Reporting Rules against the CFTC Capital Rules and CFTC Financial Reporting Rules, as outlined above and discussed in detail in section II below, was sufficiently rigorous for purposes of determining if the UK PRA regulations are comparable in purpose and effect to the CEA and Commission regulations. Better Markets further asserted that even under a principles-based, holistic approach, the UK PRA capital and financial reporting requirements for PRA-designated UK nonbank SDs do not satisfy the test for an order granting substituted compliance as the PRA's regulatory framework governing capital and financial reporting is not comparable to the corresponding CFTC requirements. [ 92 ] Better Markets cited the Commission's inclusion of conditions in the proposed Comparability Order as demonstrating the Commission's need “to compensate for the acknowledged gaps in the UK PRA framework” and as a “de facto admission that the regulations are not comparable and that the [UK Application] should be denied.”  [ 93 ] Better Markets claimed that the Commission proposed 12 filing requirements that must be met as a condition for the comparability determination, and stated that the Commission was not issuing a comparability finding, but was engaging in a “de facto rewriting” of the PRA's laws and rules in the form of conditions. [ 94 ]

Conversely, another commenter, Ravnitzky, noted that the “CFTC need not be limited to finding a binary yes or no answer to the comparability determination” and “has the flexibility to grant conditional substituted compliance.”  [ 95 ] In this regard, Ravnitzky recommended that the Commission exercise its authority “to make a flexible and nuanced decision, and strive to impose only the necessary conditions for approving the UK PRA rules as substitutes, to minimize the regulatory burden while achieving the necessary risk reduction.”  [ 96 ]

The Commission disagrees that the inclusion of conditions in the Comparability Order precludes a finding of comparability with respect to the UK PRA Capital Rules and UK PRA Financial Reporting Rules. The Commission's comparability assessment process, consistent with the holistic approach, contemplates the potential need for a Comparability Order to contain conditions. Specifically, Commission Regulation 23.106(a)(5) states that the Commission may impose any terms and conditions it deems appropriate in issuing a Comparability Order, including conditions with respect to capital adequacy and financial reporting requirements of non-U.S. nonbank SDs. [ 97 ]

The process employed in this Comparability Determination is consistent with the Commission's established approach to conducting comparability assessments. Upon a finding of comparability, the Commission's policy generally is that eligible entities may comply with a substituted compliance regime subject to the conditions the Commission places on its finding, and subject to the Commission's retention of its examination authority and its enforcement authority. [ 98 ] In this regard, the Commission has stated that certain conditions included in a Comparability Order may be designed to ensure the Commission's direct access to books and records required to be maintained by an SD registered with the Commission. [ 99 ] Other conditions may address areas where the foreign jurisdiction lacks analogous requirements. [ 100 ] The inclusion of conditions in a Comparability Order was contemplated as an integral part of the Commission's holistic, principles-based approach to conducting comparability assessments and is not inconsistent with a grant of substituted compliance.

In particular, Commission Regulation 23.106(a)(5) states the Commission's authority to impose conditions in issuing a Comparability Determination in connection with the CFTC Capital Rules and the CFTC Financial Reporting Rules. As further discussed below, the conditions proposed in the 2024 Proposal are clearly of the nature contemplated by Commission Regulation 23.106(a)(5).

The Commission also does not believe that the inclusion of the conditions in the Comparability Order reflects a “rewriting” of the UK laws and regulations as asserted by Better Markets. Consistent with the Commission's policy described above, a majority of the conditions contained in the Comparability Order are designed to ensure that: (i) the PRA-designated UK nonbank SD is eligible for substituted compliance based on the UK laws and regulations that were reviewed by the Commission in performing the comparability assessment, and (ii) the Commission and NFA receive timely financial information and notices to effectively monitor a PRA-designated nonbank SD's compliance with relevant UK capital and financial reporting rules and to assess the ongoing safety and soundness of the PRA-designated UK nonbank SD. Specifically, there are 25 conditions in the final Comparability Order. Six conditions set forth criteria that a PRA-designated UK nonbank SD must meet to be eligible for substituted compliance pursuant to the Comparability Order. [ 101 ] The six Start Printed Page 58544 conditions ensure that only PRA-designated UK nonbank SDs that are within the scope of, and comply with, the UK PRA Capital Rules and UK PRA Financial Reporting Rules that were part of the Commission's comparability assessment may apply for substituted compliance. Ten additional conditions require PRA-designated UK nonbank SDs within the scope of the Comparability Order to provide notice to the Commission and NFA of certain defined events, [ 102 ] and a further two conditions require PRA-designated nonbank SDs to file with the Commission and NFA copies of certain unaudited and audited financial reports that the firms provide to the PRA. [ 103 ] In addition, two additional conditions reflect administrative matters necessary to implement the substituted compliance framework. [ 104 ] Lastly, five conditions impose obligations on PRA-designated UK nonbank SDs that align with certain of the Commission's requirements for nonbank SDs. The five conditions require a PRA-designated UK nonbank SD to: (i) maintain common equity tier 1 capital denominated in GBP equal to or in excess of the equivalent of $20 million (Condition 7); (ii) prepare and keep current financial books and records (Condition 9); (iii) file a monthly schedule of the firm's financial positions on Schedule 1 of appendix B to Subpart E of part 23 of the Commission's regulations (Condition 12); (iv) file a monthly report listing the custodians holding margin posted by, and collected by, the PRA-designated UK nonbank SD, the amount of margin held by each custodian, and the aggregate amount of margin required to be posted and collected by the PRA-designated UK nonbank SD (Condition 14); and (v) submit, with each filing of financial information, a statement by an authorized representative that, to the best knowledge and belief of the person making the representation, the information is true and correct (Condition 13).

As the substance of these conditions demonstrates, the primary objective of a majority of the conditions is not to compensate for regulatory gaps in the UK PRA capital and financial reporting framework, but rather to ensure that the Commission and NFA receive information to conduct ongoing monitoring of PRA-designated UK nonbank SDs for compliance with relevant capital and financial reporting requirements and to assess the firm's overall safety and soundness. As discussed above, in issuing the Comparability Order, the Commission is not ceding its supervisory and enforcement authorities. The Comparability Order permits PRA-designated UK nonbank SDs to satisfy the Commission's capital and financial reporting requirements by complying with certain UK laws and/or regulations that have been found comparable to the Commission's laws and/or regulations in purpose and effect. The Commission and NFA, however, have a continuing obligation to conduct ongoing oversight, including potential examination, of PRA-designated UK nonbank SDs that operate under a Comparability Order to ensure compliance with the Comparability Order, including its conditions. [ 105 ] To that effect, the notice and financial reporting conditions set forth in the Comparability Order provide the Commission and NFA with information necessary to monitor for such compliance and to evaluate the operational condition and ongoing financial condition of PRA-designated UK nonbank SDs. The Commission may also initiate an enforcement action against a PRA-designated UK nonbank SD that fails to comply with the conditions of the Comparability Order.

Furthermore, to the extent that a condition imposes a new regulatory obligation on PRA-designated UK nonbank SDs, the imposition of such condition is also consistent with Commission Regulation 23.106 and the Commission's established policy with regard to comparability determinations. As discussed above, the Commission contemplated that even in circumstances where the Commission finds two regulatory regimes comparable, the Commission may impose requirements on entities relying on substituted compliance where the Commission determines that the home jurisdiction's regime lacks comparable and comprehensive regulation on a specific issue. [ 106 ] The Commission's authority to impose such conditions is set out in Commission Regulation 23.106(a)(5), which states that the Commission may impose “any terms and conditions it deems appropriate, including certain capital adequacy and financial reporting requirements [on SDs].”  [ 107 ]

Better Markets further stated that, if the Commission grants substituted compliance with regard to materially Start Printed Page 58545 different regulatory requirements, it must make a well-supported, evidence-based determination that those different requirements nevertheless will, in fact, lead to comparable regulatory outcomes. [ 108 ] In this connection, Better Markets stated that if the Commission grants the Comparability Determination and Comparability Order, it must, at a minimum, clearly and specifically set forth the desired regulatory outcome and provide a detailed, evidence-based explanation as to how the jurisdiction's different legal requirements nonetheless lead to that regulatory outcome. [ 109 ] Better Markets further asserted that “[a] determination that a foreign jurisdiction's nonbank SDs rules would produce comparable regulatory outcomes is the beginning, not the end, of the CFTC's obligation to ensure that the activities of the foreign nonbank SD entities do not pose risks to the U.S. financial system. As time goes on, regulatory requirements that, in theory, are expected to produce one regulatory outcome may, in practice, produce a different one. And, of course, the regulatory requirements may themselves be changed in a variety of ways. Finally, the effectiveness of an authority's supervision and enforcement program can become weakened for any number of reasons—the CFTC cannot assume that an enforcement program that it believes is presently effective will continue to be effective.”  [ 110 ] Better Markets further asserted that to fulfill its obligation to protect the U.S. financial system, the CFTC must ensure, on an ongoing basis, that each grant of substituted compliance remains appropriate over time by requiring, at a minimum, each order of substituted compliance, and each MOU with a foreign regulatory authority, to impose an obligation on the applicant, as appropriate, to: (i) periodically apprise the Commission of the activities and results of its supervision and enforcement programs, to ensure that they remain sufficiently robust to deter and address violations of the law; and (ii) immediately apprise the Commission of any material changes to the regulatory regime, including changes to rules or interpretations of rules. [ 111 ]

Although the Commission disagrees that the UK PRA Capital Rules and the UK PRA Financial Reporting Rules, as a whole, are materially different or do not achieve comparable regulatory outcomes, the Commission concurs that granting substituted compliance should be the result of a well-supported comparability assessment. Consistent with that view, the Commission believes that this final Comparability Determination clearly states the desired regulatory outcomes, articulates the Commission's analysis in sufficient detail, and provides an appropriate explanation of how the foreign jurisdiction's requirements are comparable in purpose and effect with the Commission's requirements, and lead to comparable regulatory outcomes with the Commission's requirements. Specifically, section III of the 2024 Proposal and section II of the final Comparability Determination reflect, among other observations, the Commission's detailed analysis with respect to each of the elements for consideration listed in Commission Regulation 23.106(a)(3).

The Commission also concurs that the availability of substituted compliance is conditioned upon a non-US nonbank SD's ongoing compliance with the terms and conditions of the final Comparability Order, and the Commission's ongoing assessment that the UK PRA Capital Rules and UK PRA Financial Reporting Rules remain comparable in purpose and effect with the CFTC Capital Rules and CFTC Financial Reporting Rules. As noted above, and discussed in more detail in sections II.D. and E. below, PRA-designated UK nonbank SDs are subject to notice and financial reporting requirements under the final Comparability Order that provide Commission and NFA staff with the ability to monitor the PRA-designated UK nonbank SDs' ongoing compliance with the conditions set forth in the final Comparability Order. In addition, the final Comparability Order requires a PRA-designated UK nonbank SD, or an entity acting on its behalf, to inform the Commission of changes to the relevant UK PRA Capital Rules and UK PRA Financial Reporting Rules so that the Commission may assess the continued effectiveness of the Comparability Order in ensuring that the relevant UK laws and regulations have the comparable regulatory objectives of the CEA and Commission regulations of ensuring the safety and soundness of nonbank SDs. [ 112 ] Commission staff will also monitor the PRA-designated UK nonbank SDs directly as part of its supervisory program and will discuss with the firms any proposed or pending revisions to specific rules cited in the final Comparability Order. Lastly, in addition to assessing the effectiveness of the Comparability Order as a result of revisions or proposed revisions to the UK laws, regulations, or supervisory regime administered by the PRA, the Commission further notes that future material changes to the CFTC Capital Rules or CFTC Financial Reporting Rules, or the Commission's or NFA's supervisory programs, may necessitate an amendment to the Comparability Determination and Comparability Order to reflect those changes. [ 113 ]

Another commenter, Harrington, stated that the Commission must condition the Comparability Order on an “outright prohibition against regulated entities providing [swap contracts that include a “flip clause”].”  [ 114 ] Harrington has elsewhere referred to a description of a “flip clause” as a provision in swap contracts with structured debt issuers that reverses or “flips” the priority of payment obligations owed to the swap counterparty on the one hand and the noteholders on the other, following a specified event of default. [ 115 ] Based on Harrington's description, flip clauses present a risk to the SD in synthetic transactions where payments under a swap contract are secured with the same collateral that would serve to cover payments under the notes issued by a structured debt issuer. In such circumstances, an “event of default” by the SD would cause the SD's priority of Start Printed Page 58546 payment from the collateral under a swap to “flip” to a more junior priority position, including for mark-to-market gains on “in the money” swaps. [ 116 ] Harrington argued that swap contracts with a flip clause incentivize SDs to “self-sabotage by under-sourcing themselves.”  [ 117 ] Harrington recognized, however, that the CFTC margin requirements for uncleared swap transactions address his concerns associated with the inclusion of a flip clause. [ 118 ] Nonetheless, according to Harrington, risks arise in circumstances when non-U.S. margin rules exempt SDs from margin obligations in connection with swaps with a structured debt issuer. [ 119 ]

The Commission recognizes that given some definitional differences and differences in the activity thresholds with respect to the scope of application of the CFTC margin requirements and non-U.S. margin requirements, some transactions that are subject to the CFTC margin requirements for uncleared swaps may not be subject to margin requirements in another jurisdiction. In connection with this Comparability Determination, however, the Commission notes that both under the CFTC Capital Rules and the UK PRA Capital Rules, uncollateralized exposures from uncleared swap transactions would generate a higher counterparty credit risk amount than the exposures resulting from transactions under which the counterparties have posted collateral. [ 120 ] Accordingly, the Commission does not believe that the respective sets of rules adopt a conflicting approach or lead to a disparate outcome with respect to the capital treatment of uncollateralized uncleared swap exposures that would warrant a finding of non-comparability of the CFTC Capital Rules and the UK PRA Capital Rules.

Finally, one commenter, Ravnitzky, noted that due to differences in how the respective jurisdictions define the regulatory categories of registrants involved in swap dealing activity ( i.e., differences between the term “swap dealer” as defined under the Commission's regulations and the term “investment firm” as defined under the PRA's framework), it may be “unclear or inconsistent which entities can use substituted compliance under the [proposed Comparability Order].”  [ 121 ] The Commission notes, as discussed above, that the Comparability Order will apply with respect to UK-domiciled, PRA-designated investment firms that are registered with the Commission as SDs and not subject to regulation by a U.S. prudential regulator. In this regard, the Commission believes that proposed Conditions 1 through 4, which the Commission adopts without material changes, clearly define the scope of entities that may request to rely on the Comparability Order.

The following section provides the Commission's comparative analysis of the UK PRA Capital Rules and the UK PRA Financial Reporting Rules with the corresponding CFTC Capital Rules and CFTC Financial Reporting Rules, as described in the 2024 Proposal, further modified to address comments received. As emphasized in the 2024 Proposal, the capital and financial reporting regimes are complex structures comprised of a number of interrelated regulatory components. [ 122 ] Differences in how jurisdictions approach and implement these regimes are expected, even among jurisdictions that base their requirements on the principles and standards set forth in the BCBS framework.

The Commission performed the analysis by assessing the comparability of the UK PRA Capital Rules for PRA-designated UK nonbank SDs as set forth in the UK Application and in certain applicable UK laws and regulations with the Commission's Bank-Based Approach for nonbank SDs. The Commission understands that all PRA-designated UK nonbank SDs addressed by the UK Application, as of the date of the final Comparability Determination, are subject to a bank-based capital approach under the UK PRA Capital Rules. Accordingly, when the Commission makes its final determination herein about the comparability of the UK PRA Capital Rules with the CFTC Capital Rules, the determination pertains to the comparability of the UK PRA Capital Rules with the Bank-Based Approach under the CFTC Capital Rules. The Commission notes that any material changes to the information submitted in the UK Application, including, but not limited to, proposed and final material changes to the UK PRA Capital Rules or UK PRA Financial Reporting Rules, as well as any proposed and final material changes to the PRA's supervisory authority or supervisory regime over PRA-designated UK nonbank SDs, will require notification to the Commission and NFA pursuant to Condition 24 of the final Comparability Order. [ 123 ] Therefore, if there are subsequent material changes to the UK PRA Capital Rules, UK PRA Financial Reporting Rules, or PRA's supervisory authority or regime, the Commission will review and assess the impact of such changes on the final Comparability Determination and Comparability Order as they are then in effect, and may amend or supplement the Comparability Order as appropriate. [ 124 ]

As reflected in the 2024 Proposal and discussed above, the Commission preliminarily determined that the overall objectives of the UK PRA Capital Rules and the CFTC Capital Rules are comparable in that both sets of rules are intended to ensure the safety and soundness of nonbank SDs by establishing regulatory regimes that require nonbank SDs to maintain a sufficient amount of qualifying regulatory capital to absorb losses, including losses from swaps and other trading activities, and to absorb decreases in the value of firm assets and increases in the value of firm liabilities without the nonbank SDs becoming insolvent. [ 125 ] The Commission further noted that the UK PRA Capital Rules and CFTC Capital Rules are based on, and consistent with, the BCBS framework, which was designed to ensure that banking entities hold sufficient levels of capital to absorb losses and decreases in the value of firm assets and increases in the value of firm liabilities without the banks becoming insolvent. [ 126 ]

The Commission also preliminarily found that the UK PRA Capital Rules are comparable in purpose and effect to the CFTC Capital Rules given that both regulatory approaches compute the minimum capital requirements based on the level of a nonbank SD's on-balance sheet and off-balance sheet exposures, with the objective and purpose of ensuring that the nonbank SD's capital is adequate to absorb losses or decreases in the value of firm assets or increases in the value of firm liabilities resulting from such exposures. The Commission observed that the UK PRA Capital Rules and CFTC Capital Rules provide for a comparable approach to the calculation of market risk and credit risk exposures using standardized or internal model-based approaches. [ 127 ] In addition, as discussed in the 2024 Proposal, the UK PRA Capital Rules' and CFTC Capital Rules' requirements for identifying and measuring on-balance sheet and off-balance sheet exposures under standardized or internal model-based approaches are also consistent with the requirements set forth under the BCBS framework for identifying and measuring on-balance sheet and off-balance sheet exposures. [ 128 ]

Finally, the Commission preliminarily noted that the UK PRA Capital Rules and CFTC Capital Rules further achieve comparable outcomes and are comparable in purpose and effect in that both sets of rules limit the types of capital instruments that qualify as regulatory capital to cover the on-balance sheet and off-balance sheet risk exposures to high quality equity capital and qualifying subordinated debt instruments that meet conditions designed to ensure that the holders of the debt have effectively subordinated their claims to other creditors of the nonbank SD. [ 129 ] As discussed in the 2024 Proposal and in section II.B. below, both the UK PRA Capital Rules and the CFTC Capital Rules define high quality capital by the degree to which the capital represents permanent capital that is contributed, or readily available to a nonbank SD, on an unrestricted basis to absorb unexpected losses, including losses from swaps trading and other activities, without the nonbank SD becoming insolvent. [ 130 ]

The Commission further stated that it preliminarily found the UK PRA Financial Reporting Rules to be comparable in purpose and effect to the CFTC Financial Reporting Rules as both the PRA and CFTC require nonbank SDs to file periodic financial reports, including unaudited financial reports and an annual audited financial report, detailing their financial operations and demonstrating their compliance with minimum capital requirements. [ 131 ] As discussed in the 2024 Proposal, in addition to providing the CFTC and the PRA with information necessary to comprehensively assess the financial condition of a nonbank SD on an ongoing basis, the financial reports further provide the CFTC and the PRA with information regarding potential changes in a nonbank SD's risk profile by disclosing changes in account balances reported over a period of time. [ 132 ] Such changes in account balances may indicate, among other things, that the nonbank SD has entered into new lines of business, has increased its activity in an existing line of business relative to other activities, or has terminated a previous line of business. [ 133 ]

In assessing the comparability between the CFTC Financial Reporting Rules and the UK PRA Financial Reporting Rules, the Commission noted that the prompt and effective monitoring of the financial condition of nonbank SDs through the receipt and review of periodic financial reports supports the Commission and the PRA in meeting their respective objectives of ensuring the safety and soundness of nonbank SDs. In this regard, the Commission stated that the early identification of potential financial issues provides the Commission and the PRA with an opportunity to address such issues with the nonbank SD before they develop to a state where the financial condition of the firm is impaired such that it may no longer hold a sufficient amount of qualifying regulatory capital to absorb decreases in the value of firm assets, absorb increases in the value of firm liabilities, or cover losses from its business activities, including the firm's swap dealing activities and obligations to swap counterparties. [ 134 ]

In response to the Commission's request for comment, Better Markets identified certain differences between the CFTC Capital Rules and Financial Reporting Rules and the UK PRA Capital Rules and Financial Reporting Rules and stated that the differences mandated denial of the request for a comparability determination. [ 135 ] Better Markets further stated that the nature and number of conditions that the Commission deemed necessary to impose are inconsistent with a finding of comparability. [ 136 ] In this connection, Better Markets also noted that the imposition of conditions will exacerbate complexity as the Commission will have to monitor compliance with the conditions, including reviewing the financial reports of the PRA-designated UK nonbank SDs and tracking developments in the UK PRA regulatory regime more generally. [ 137 ] Finally, Better Markets asserted that the proposed Comparability Order failed to provide sufficient analysis as to exactly how and why the Commission concluded that the UK and U.S. frameworks would produce “comparable outcomes.”  [ 138 ]

As described herein and in the 2024 Proposal, Commission staff has engaged in a detailed, comprehensive study and evaluation of the UK PRA capital and financial reporting framework and has confirmed that its understanding of the elements and application of the framework is accurate. The Commission has also concluded, based on its evaluation, that the PRA has a comprehensive oversight program for monitoring PRA-designated UK nonbank SDs' compliance with relevant UK PRA Capital Rules.

Furthermore, as discussed in section I.E. above, the conditions set forth in the Comparability Order are generally intended to ensure that: (i) only PRA-designated UK nonbank SDs that are subject to the laws and regulations assessed under the Comparability Determination are eligible for substituted compliance; (ii) the PRA-designated UK nonbank SDs are subject to supervision by the PRA; and (iii) the PRA-designated UK nonbank SDs provide information to the Commission and NFA that is relevant to the ongoing supervision of their operations and financial condition. Considering this thorough analysis, and the ongoing requirement for PRA-designated UK nonbank SDs to provide information to the Commission and NFA demonstrating compliance with the Comparability Order, the Commission is confident that it is capable of effectively conducting, together with NFA, oversight of the PRA-designated UK nonbank SDs consistent with the conduct of oversight of U.S.-domiciled nonbank SDs. In light of the Commission's ultimate conclusion that the UK PRA capital and financial reporting requirements are comparable based on the standards articulated in Commission Regulation 23.106(a)(3), the Commission believes that a failure to issue a Comparability Determination and Comparability Order would in fact “exacerbate complexity” as it would impose duplicative requirements that would result in increased costs for registrants and market participants without a commensurate benefit from an oversight perspective.

As discussed in sections I.B. and E. above, and detailed herein, the Commission finds that the CFTC Capital Rules and Financial Reporting Rules and the UK PRA Capital Rules and Financial Reporting Rules are comparable in purpose and effect, and have overall comparable objectives, notwithstanding the identified differences. In this regard, the Commission notes that, as described above, instead of conducting a line-by-line assessment or comparison of the UK PRA Capital and Financial Reporting Rules and the CFTC Capital and Financial Reporting Rules, it has applied in the assessment set forth in the determination and order, a principles-based, holistic approach in assessing the comparability of both regimes, consistent with the standard of review it adopted in Commission Regulation 23.106(a)(3). Based on that principles-based, holistic assessment, the individual elements of which are described in more detail in sections II.B. through II.F. below, the Commission has determined that both sets of rules are designed to ensure the safety and soundness of nonbank SDs and achieve comparable outcomes. As such, the Commission adopts the Comparability Determination and Comparability Order as proposed with respect to the analysis of the regulatory objectives of the CFTC Capital Rules and Financial Reporting Rules and the UK PRA Capital and Financial Reporting Rules.

As discussed in the 2024 Proposal, the Commission preliminarily determined that the UK PRA Capital Rules are comparable in purpose and effect to the CFTC Capital Rules with regard to the types and characteristics of a nonbank SD's equity that qualifies as regulatory capital in meeting its minimum requirements. [ 139 ] The Commission explained that the UK PRA Capital Rules and the CFTC Capital Rules for nonbank SDs both require a nonbank SD to maintain a quantity of high-quality and permanent capital that, based on the firm's activities and on-balance sheet and off-balance sheet exposures, is sufficient to absorb losses and decreases in the value of firm assets and increases in the value of firm liabilities without resulting in the firm becoming insolvent. [ 140 ] The Commission observed that the UK PRA Capital Rules and the CFTC Capital Rules permit nonbank SDs to recognize comparable forms of equity capital and qualifying subordinated debt instruments toward meeting minimum capital requirements, with both the UK PRA Capital Rules and the CFTC Capital Rules emphasizing high quality capital instruments. [ 141 ]

In support of its preliminary Comparability Determination, the Commission noted that the CFTC Capital Rules require a nonbank SD electing the Bank-Based Approach to maintain regulatory capital in the form of common equity tier 1 capital, additional tier 1 capital, and tier 2 capital in amounts that meet certain stated minimum requirements set forth in Commission Regulation 23.101. [ 142 ] Common equity tier 1 capital is generally composed of an entity's common stock instruments, and any related surpluses, retained earnings, and accumulated other comprehensive income, and is a more conservative or permanent form of capital that is last in line to receive distributions in the event of the entity's insolvency. [ 143 ] Additional tier 1 capital is generally composed of equity instruments such as preferred stock and certain hybrid securities that may be converted to common stock if triggering events occur and may have a preference in distributions over common equity tier 1 capital in the event of an insolvency. [ 144 ] Total tier 1 capital is composed of common equity tier 1 capital and further includes additional tier 1 capital. Tier 2 capital includes certain types of instruments that include both debt and equity characteristics such as qualifying subordinated debt. [ 145 ] Subordinated debt must meet certain conditions to qualify as tier 2 capital under the CFTC Capital Rules. [ 146 ]

The preliminary Comparability Determination also noted that the UK PRA Capital Rules require a PRA-designated nonbank SD to maintain an amount of regulatory capital ( i.e., equity capital and qualifying subordinated debt) equal to or greater than 8 percent of the PRA-designated UK nonbank SD's total risk exposure, which is calculated as the sum of the firm's: (i) capital charges for market risk; (ii) risk-weighted exposure amounts for credit risk; (iii) capital charges for settlement risk; (iv) credit valuation adjustment (“CVA”) risk of over-the-counter (“OTC”) derivatives instruments; and (v) capital charges for operational risk. The UK PRA Capital Rules limit the composition of regulatory capital to Start Printed Page 58549 common equity tier 1 capital, additional tier 1 capital, and tier 2 capital in a manner consistent with the BCBS framework. Specifically, the UK PRA Capital Rules provide that a PRA-designated UK nonbank SD's regulatory capital may be composed of: (i) common equity tier 1 capital instruments, which generally include the PRA-designated UK nonbank SD's common equity (stock), retained earnings, and accumulated other comprehensive income; (ii) additional tier 1 capital instruments, which includes other forms of capital instruments and certain long-term convertible debt instruments; and (iii) tier 2 capital instruments, which include other reserves, hybrid capital instruments, and certain qualifying subordinated term debt. [ 147 ] Capital instruments that qualify as common equity tier 1 capital under the UK PRA Capital Rules include instruments that: (i) are issued directly by the PRA-designated UK nonbank SD; (ii) are paid in full and not funded directly or indirectly by the PRA-designated UK nonbank SD; and (iii) are perpetual. [ 148 ] In addition, the principal amount of the common equity tier 1 capital instruments may not be reduced or repaid, except in the liquidation of the PRA-designated UK nonbank SD. [ 149 ] Furthermore, to qualify as additional tier 1 capital, the capital instruments must meet certain conditions including: (i) the instruments are issued directly by the PRA-designated UK nonbank SD and paid in full; (ii) the instruments are not owned by the PRA-designated UK nonbank SD or its subsidiaries; (iii) the purchase of the instruments is not funded directly or indirectly by the PRA-designated UK nonbank SD; (iv) the instruments rank below tier 2 instruments in the event of the insolvency of the PRA-designated UK nonbank SD; (v) the instruments are not secured or guaranteed by the PRA-designated UK nonbank SD or an affiliate; (vi) the instruments are perpetual and do not include an incentive for the PRA-designated UK nonbank SD to redeem them; and (vii) distributions under the instruments are pursuant to defined terms and may be cancelled under the full discretion of the PRA-designated UK nonbank SD. [ 150 ] Lastly, subordinated debt instruments must meet certain conditions to qualify as tier 2 regulatory capital under the UK PRA Capital Rules, including that the: (i) loans are not granted by the PRA-designated UK nonbank SD or its subsidiaries; (ii) claims on the principal amount of the subordinated loans under the provisions governing the subordinated loan agreement rank below any claim from eligible liabilities instruments ( i.e., certain non-capital instruments), meaning that they are effectively subordinated to claims of all non-subordinated creditors of the PRA-designated UK nonbank SD; (iii) subordinated loans are not secured, or subject to a guarantee that enhances the seniority of the claim, by the PRA-designated UK nonbank SD, its subsidiaries, or affiliates; (iv) loans have an original maturity of at least five years; and (v) provisions governing the loans do not include any incentive for the principal amount to be repaid by the PRA-designated UK nonbank SD prior to the loans' maturity. [ 151 ]

Based on its comparative assessment, the Commission preliminarily found that the types and characteristics of the equity instruments that qualify as common equity tier 1 capital and additional tier 1 capital under the UK PRA Capital Rules are comparable to the types and characteristics of equity instruments comprising common equity tier 1 capital and additional tier 1 capital under the CFTC Capital Rules. [ 152 ] Specifically, the Commission noted that the UK PRA Capital Rules' common equity tier 1 capital and additional tier 1 capital and the CFTC Capital Rules' common equity tier 1 capital and additional tier 1 capital are comparable in that these forms of equity capital have similar characteristics ( e.g., the equity must be in the form of high-quality, committed, and permanent capital) and represent contributed equity capital that generally has no priority to the distribution of firm assets or income with respect to other shareholders or creditors of the firm, which allows a nonbank SD to use this equity to absorb decreases in the value of firm assets, absorb increases in the value of firm liabilities, and cover losses from business activities, including the firm's swap dealing activities. [ 153 ]

The Commission also found subordinated debt under the UK PRA Capital Rules comparable to tier 2 capital under the CFTC Capital Rules. [ 154 ] Specifically, the Commission noted that the qualifying conditions imposed on subordinated debt instruments are comparable under the UK PRA Capital Rules and the CFTC Capital Rules in that they are designed to ensure that the debt has qualities supporting its recognition by a nonbank SD as equity for capital purposes, including by effectively subordinating the debt lenders' claims for repayment on the debt to other creditors of the nonbank SD and by limiting or restricting repayment of the subordinated loans if such repayments result in the nonbank SD's equity falling below certain defined thresholds. [ 155 ] The Commission preliminarily concluded that these terms and conditions provided assurances that the subordinated debt is appropriate to be recognized as regulatory capital available to a nonbank SD to meet its obligations and to absorb business losses and decreases in the value of firm assets and increases in the value of firm liabilities. [ 156 ]

The Commission did not receive comments regarding its preliminary determination that the UK PRA Capital Rules are comparable in purpose and effect to the CFTC Capital Rules with regard to the types and characteristics of a nonbank SD's equity and subordinated debt that qualifies as regulatory capital in meeting its minimum requirements. In conclusion, the Commission finds that the UK PRA Capital Rules and the CFTC Capital Rules are comparable in purpose and effect, and achieve comparable regulatory outcomes, with respect to the types of capital instruments that qualify as regulatory capital. Both the UK PRA Capital Rules and the CFTC Capital Rules limit regulatory capital to permanent and conservative forms of capital, including common equity, capital surpluses, retained earnings, and subordinate debt where debt holders effectively subordinate their claims to repayment to all other creditors of the nonbank SD in the event of the firm's insolvency. Limiting regulatory capital to the above categories of equity and debt instruments promotes the safety and soundness of the nonbank SD by helping to ensure that the regulatory capital is not withdrawn or converted to other equity instruments that may have rights or priority with respect to payments, such as dividends or distributions in insolvency, over other creditors, including swap counterparties. The Commission, therefore, is adopting the Comparability Order as proposed with respect to the types and characteristics of equity and subordinated debt that qualifies as regulatory capital to meet minimum Start Printed Page 58550 capital requirements under the UK PRA Capital Rules.

As reflected in the 2024 Proposal, the CFTC Capital Rules require a nonbank SD electing the Bank-Based Approach to maintain regulatory capital that satisfies each of the following criteria: (i) an amount of common equity tier 1 capital of at least $20 million; (ii) an aggregate amount of common equity tier 1 capital, additional tier 1 capital, and tier 2 capital equal to or greater than 8 percent of the nonbank SD's total risk-weighted assets, provided that common equity tier 1 capital comprises at least 6.5 percent of the 8 percent; (iii) an aggregate of common equity tier 1 capital, additional tier 1 capital, and tier 2 capital in an amount equal to or in excess of 8 percent of the nonbank SD's uncleared swap margin amount;  [ 157 ] and (iv) the amount of capital required by NFA. [ 158 ]

In comparison, the UK PRA Capital Rules, consistent with the BCBS framework, require each PRA-designated UK nonbank SD to maintain sufficient levels of capital to satisfy the following, expressed as a percentage of the PRA-designated UK nonbank SD's “total risk exposure amount” ( i.e., the sum of the PRA-designated UK nonbank SD's risk-weighted assets and exposures): (i) a common equity tier 1 capital ratio of 4.5 percent; (ii) a tier 1 capital ratio of 6 percent; and (iii) a total capital ratio of 8 percent. Furthermore, PRA-designated UK nonbank SDs must maintain a capital conservation buffer composed of common equity tier 1 capital in an amount equal to 2.5 percent of the firm's total risk exposure. The common equity tier 1 capital used to meet the capital conservation buffer must be separate and in addition to the 4.5 percent of common equity tier 1 capital required to meet its core 8 percent capital requirement. [ 159 ] As explained in the 2024 Proposal, the “total risk exposure amount” is calculated as the sum of the PRA-designated UK nonbank SD's: (i) capital requirements for market risk; (ii) risk-weighted exposure amounts for credit risk; (iii) capital requirements for CVA risk of OTC derivatives; and (iv) capital requirements for operational risk. [ 160 ] Capital charges for market risk and credit risk are computed based on a PRA-designated UK nonbank SD's on-balance sheet and off-balance sheet exposures, weighted according to risk. [ 161 ]

While noting certain differences in the minimum capital requirements and calculation of regulatory capital between the UK PRA Capital Rules and the CFTC Capital Rules, the Commission preliminarily found that the UK PRA Capital Rules and CFTC Capital Rules achieve, subject to the conditions in the proposed Comparability Determination and proposed Comparability Order, comparable outcomes by requiring a nonbank SD to maintain a minimum level of qualifying regulatory capital and subordinated debt to absorb losses from the firm's business activities, including its swap dealing activities, and decreases in the value of the firm's assets and increases in the firm's liabilities without the nonbank SD becoming insolvent. [ 162 ] As further discussed below, the Commission's preliminary finding of comparability was based on a principles-based, holistic comparative analysis of the three minimum capital requirement thresholds of the CFTC Capital Rules' Bank-Based Approach referenced above and the respective elements of the UK PRA Capital Rules' requirements.

As noted above, prong (i) of the CFTC Capital Rules requires each nonbank SD electing the Bank-Based Approach to maintain a minimum of $20 million of common equity tier 1 capital. The CFTC's $20 million fixed-dollar minimum capital requirement is intended to ensure that each nonbank SD maintains a level of regulatory capital, without regard to the level of the firm's dealing and other activities, sufficient to meet its obligations to swap market participants given the firm's status as a CFTC-registered nonbank SD and to help ensure the safety and soundness of the nonbank SD. [ 163 ] In comparison, the UK PRA Capital Rules also contain a requirement that a PRA-designated UK nonbank SD maintain a fixed amount of minimum initial capital of GBP 750,000. [ 164 ]

The Commission, in the 2024 Proposal, recognized that the $20 million fixed-dollar minimum capital required under the CFTC Capital Rules is substantially higher than the GBP 750,000 minimum base capital required under the UK PRA Capital Rules. Therefore, the Commission preliminarily proposed a condition that each PRA-designated UK nonbank SD would be required to maintain, at all times, a minimum amount of common equity tier 1 capital, as defined in Article 26 of UK CRR, denominated in GBP equal to or in excess of the equivalent of $20 million. [ 165 ]

One commenter, Better Markets, argued that the establishment in the UK PRA Capital Rules of a base level requirement that is substantially lower than the CFTC Capital Rules' fixed amount minimum requirement “demonstrates a fatal lack of comparability.”  [ 166 ] Better Markets further stated that to compensate for this gap, the Commission proposed a condition requiring PRA-designated UK nonbank SDs to maintain a minimum amount of common equity tier 1 capital denominated in GBP equal to or in excess of the equivalent of $20 million. [ 167 ]

As noted above, the Commission recognized the material difference in the requirement under the UK PRA Capital Rules and the CFTC Capital Rules with respect to the $20 million minimum dollar amount of regulatory capital a nonbank SD is required to maintain. The Commission's proposed condition, however, effectively addresses this difference by providing that a PRA- Start Printed Page 58551 designated UK nonbank SD may not avail itself of substituted compliance unless it maintains a minimum amount of common equity tier 1 capital denominated in GBP equal to or excess of the equivalent of $20 million. Furthermore, the imposition of conditions in a Comparability Order, as discussed in section I.E. above, is authorized by Commission Regulation 23.106(a)(5), which provides that the Commission may issue terms and conditions as it deems appropriate. In addition, as further noted in section I.E. above, the Guidance also provides that the Commission may impose conditions as part of the substituted compliance process to address a lack of comparable and comprehensive regulation in a home jurisdiction. [ 168 ] In this connection, the Commission concludes that requiring PRA-designated UK nonbank SDs to maintain an amount of regulatory capital in the form of common equity tier 1 items, as defined in Article 26 of UK CRR, equal to or in excess of the equivalent of $20 million will impose an equally stringent standard to the analogue requirement under the CFTC Capital Rules and will appropriately address the substantially lower minimum fixed amount capital requirement under the UK PRA Capital Rules.

In conclusion, the Commission finds that the UK PRA Capital Rules and the CFTC Capital Rules, with the imposition of the condition for PRA-designated UK nonbank SDs to maintain a minimum level of common equity tier 1 capital in an amount equivalent to at least $20 million, are comparable in purpose and effect and achieve comparable outcomes with respect to capital requirements based on a minimum dollar amount. The requirement for a nonbank SD with limited swap dealing or other business activities to maintain a minimum level of regulatory capital equivalent to $20 million helps to ensure the firm's safety and soundness by allowing it to absorb decreases in firm assets, absorb increases in firm liabilities, and meet obligations to swap counterparties, other creditors, and market participants, without the firm becoming insolvent.

Prong (ii) of the CFTC Capital Rules' minimum capital requirements described above requires each nonbank SD electing the Bank-Based Approach to maintain an aggregate of common equity tier 1 capital, additional tier 1 capital, and tier 2 capital in an amount equal to or greater than 8 percent of the nonbank SD's total risk-weighted assets, with common equity tier 1 capital comprising at least 6.5 percent of the 8 percent. [ 169 ] Risk-weighted assets are a nonbank SD's on-balance sheet and off-balance sheet market risk and credit risk exposures, including exposures associated with proprietary swap, security-based swap, equity, and futures positions, weighted according to risk. The requirements and capital ratios set forth in prong (ii) are based on the Federal Reserve Board's capital requirements for bank holding companies and are consistent with the BCBS framework. The requirement for each nonbank SD to maintain regulatory capital in an amount that equals or exceeds 8 percent of the firm's total risk-weighted assets is intended to help ensure that the nonbank SD's level of capital is sufficient to absorb decreases in the value of the firm's assets and increases in the value of the firm's liabilities, and to cover unexpected losses resulting from the firm's business activities, including losses resulting from uncollateralized defaults from swap counterparties, without the nonbank SD becoming insolvent. [ 170 ]

The UK PRA Capital Rules contain capital requirements for PRA-designated UK nonbank SDs that the Commission preliminarily found comparable to the requirements in prong (ii) of the CFTC Capital Requirements. [ 171 ] Specifically, the UK PRA Capital Rules require a PRA-designated UK nonbank SD to maintain: (i) common equity tier 1 capital equal to at least 4.5 percent of the PRA-designated UK nonbank SD's total risk exposure amount; (ii) total tier 1 capital ( i.e., common equity tier 1 capital plus additional tier 1 capital) equal to at least 6 percent of the PRA-designated UK nonbank SD's total risk exposure amount; and (iii) total capital ( i.e., an aggregate amount of common equity tier 1 capital, additional tier 1 capital, and tier 2 capital) equal to at least 8 percent of the PRA-designated UK nonbank SD's total risk exposure amount. The UK PRA Capital Rules further require each PRA-designated UK nonbank SD to maintain an additional capital conservation buffer equal to 2.5 percent of the PRA-designated UK nonbank SD's total risk exposure amount, which must be met with common equity tier 1 capital. Thus, a PRA-designated UK nonbank SD is effectively required to maintain total qualifying regulatory capital in an amount equal to or in excess of 10.5 percent of the market risk, credit risk, CVA risk, settlement risk, and operational risk of the firm ( i.e., total capital requirement of 8 percent of risk-weighted assets and an additional 2.5 percent of risk-weighted assets as a capital conservation buffer), which is a higher capital ratio than the 8 percent required of nonbank SDs under prong (ii) of the CFTC Capital Rules. [ 172 ]

The Commission also preliminarily found that the UK PRA Capital Rules and the CFTC Capital Rules are comparable with respect to the approaches used in the calculation of risk-weighted asset amounts for market risk and credit risk in determining the nonbank SD's risk-weighted assets. [ 173 ] In that regard, the Commission noted that both regimes require a nonbank SD to use standardized approaches to compute market risk and credit risk amounts, unless the firm is approved to use internal models. [ 174 ]

As the Commission observed, the standardized approaches to calculating risk-weighted asset amounts for market risk and credit risk under both sets of rules follow the same structure that is now the common global standard: (i) allocating assets to categories according to risk and assigning each a risk weight; (ii) allocating counterparties according to risk assessments and assigning each a risk factor; (iii) calculating gross exposures based on valuation of assets; (iv) calculating a net exposure allowing offsets following well defined procedures and subject to clear limitations; (v) adjusting the net exposure by the market risk weights; and finally, (vi) for credit risk exposures, multiplying the sum of net exposures to each counterparty by their corresponding risk factor. [ 175 ]

More specifically, with respect to the calculation of standardized risk-weighted asset amounts for market risk, the Commission explained that the CFTC Capital Rules incorporate by reference the standardized market risk charges set forth in Commission Regulation 1.17 for FCMs and SEC Rule 18a-1 for nonbank security-based swap dealers (“SBSDs”). [ 176 ] The standardized market risk charges under Commission Regulation 1.17 and SEC Rule 18a-1 are calculated as a standardized or table-based percentage of the market value or notional value of the nonbank SD's marketable securities and derivatives positions, with the percentages applied Start Printed Page 58552 to the market value or notional value increasing as the expected or anticipated risk of the positions increases. [ 177 ] For example, CFTC Capital Rules require nonbank SDs to calculate standardized market risk-weighted asset amounts for uncleared swaps based on notional values of the swap positions multiplied by percentages set forth in the applicable rules. [ 178 ] In addition, market risk-weighted asset amounts for readily marketable equity securities are calculated by multiplying the fair market value of the securities by 15 percent. [ 179 ]

Under the CFTC Capital Rules, the resulting total market risk-weighted asset amount is multiplied by a factor of 12.5 to cancel the effect of the 8 percent multiplication factor applied to all of the nonbank SD's risk-weighted assets under prong (ii) of the rules' minimum capital requirements described above. As a result, a nonbank SD is effectively required to hold qualifying regulatory capital equal to or greater than 100 percent of the amount of its market risk exposure amount. [ 180 ]

Comparable to the CFTC Capital Rules, the UK PRA Capital Rules require a PRA-designated UK nonbank SD to calculate its standardized risk-weighted asset amounts for market risk by multiplying the notional or carrying amount of net positions by risk-weighting factors, which are based on the underlying market risk of each asset or exposure and increase as the expected risk of the positions increases. [ 181 ] The Commission further explained that a PRA-designated UK nonbank SD is required to calculate market risk requirements for debt instruments and equity instruments separately, by computing each category as the sum of specific risk and general risk of the positions. [ 182 ] As further discussed in the 2024 Proposal, the UK PRA Capital Rules also require PRA-designated UK nonbank SDs to include in their risk-weighted assets market risk exposures to certain foreign currency and gold positions. Specifically, a PRA-designated UK nonbank SD with net positions in foreign exchange and gold that exceed 2 percent of the firm's total capital must calculate capital requirements for foreign exchange risk. [ 183 ] The capital requirement for foreign exchange risk under the standardized approach is 8 percent of the PRA-designated UK nonbank SD's net positions in foreign exchange and gold. [ 184 ] The UK PRA Capital Rules further require PRA-designated UK nonbank SDs to include exposures to commodity positions in calculating the firm's risk-weighted assets. The standardized calculation of commodity risk exposures may follow one of three approaches depending on type of position or exposure. The first is the sum of a flat percentage rate for net positions, with netting allowed among tightly defined sets, plus another flat percentage rate for the gross position. [ 185 ] The other two standardized approaches are based on maturity-ladders, where unmatched portions of each maturity band ( i.e., portions that do not net out to zero) are charged at a step-up rate in comparison to the base charges for matched portions. [ 186 ]

With respect to standardized risk-weighted asset amounts for credit risk, the Commission explained that under the CFTC Capital Rules, a nonbank SD must compute its on-balance sheet and off-balance sheet exposures in accordance with the standardized risk-weighting requirements adopted by the Federal Reserve Board and set forth in Subpart D of 12 CFR 217 as if the SD itself were a bank holding company subject to Subpart D. [ 187 ] Standardized risk-weighted asset amounts for credit risk are computed by multiplying the amount of the exposure by defined counterparty credit risk factors that range from 0 percent to 150 percent. [ 188 ] A nonbank SD with off-balance sheet exposures is required to calculate a risk-weighted amount for credit risk by multiplying each exposure by a credit conversion factor that ranges from 0 percent to 100 percent, depending on the type of exposure. [ 189 ]

In comparison, the Commission noted that the UK PRA Capital Rules require a PRA-designated UK nonbank SD to calculate its standardized risk-weighted asset amounts for credit risk in a manner aligned with the Commission's Bank-Based Approach and the BCBS framework by taking the carrying value or notional value of each of the PRA-designated UK nonbank SD's on-balance sheet and off-balance sheet exposures, making certain additional credit risk adjustments, and then applying specific risk weights based on the type of counterparty and the asset's credit quality. [ 190 ] For instance, exposures to the ECB, the UK government, and the Bank of England, carry a zero percent risk weight; exposures to other central governments and central banks may carry risk weights between 0 and 150, depending on the credit rating available for the central government or central bank; and exposures to banks, PRA-designated investment firms, or other businesses may carry risk weights between 20 percent and 150 percent depending on the credit ratings available for the entity or, for exposures to banks and investment firms, for the central government of the jurisdiction in which the entity is incorporated. [ 191 ] If no credit rating is available, the PRA-designated UK nonbank SD must generally apply a 100 percent risk weight, meaning the total accounting value of the exposure is used. [ 192 ]

With respect to counterparty credit risk for derivatives positions, the Commission explained that under the CFTC Capital Rules, a nonbank SD may compute standardized credit risk Start Printed Page 58553 exposures, using either the current exposure method (“CEM”) or the standardized approach for measuring counterparty credit risk (“SA-CCR”). [ 193 ] Both CEM and SA-CCR are non-model, rules-based approaches to calculating counterparty credit risk exposures for derivatives positions. Credit risk exposure under CEM is the sum of: (i) the current exposure ( i.e., the positive mark-to-market) of the derivatives contract; and (ii) the potential future exposure, which is calculated as the product of the notional principal amount of the derivatives contract multiplied by a standard credit risk conversion factor set forth in the rules of the Federal Reserve Board. [ 194 ] Credit risk exposure under SA-CCR is defined as the exposure at default amount of a derivatives contract, which is computed by multiplying a factor of 1.4 by the sum of: (i) the replacement costs of the contract ( i.e., the positive mark-to market); and (ii) the potential future exposure of the contract. [ 195 ] In comparison, the UK PRA Capital Rules require a PRA-designated UK nonbank SD that is not approved to use credit risk models to calculate its exposure using the SA-CCR. [ 196 ] The exposure amount under the SA-CCR is computed, under both the UK PRA Capital Rules and the Commission's Bank-Based Approach, as the sum of the replacement cost of the contract and the potential future exposure of the contract, multiplied by a factor of 1.4. [ 197 ]

UK PRA Capital Rules also require a PRA-designated UK nonbank SD to include its exposures to settlement risk in its calculation of its risk-weighted assets. [ 198 ] Consistent with the BCBS framework, the risk-weighted asset amount for settlement risk for transactions settled on a delivery-versus-payment basis is computed by multiplying the price difference to which a PRA-designated UK nonbank SD is exposed as a result of an unsettled transaction by a percentage factor that varies from 8 percent to 100 percent based on the number of working days after the settlement due date during which the transaction remains unsettled. [ 199 ] The CFTC's Bank-Based Approach provides for a similar calculation methodology for risk-weighted asset amounts for unsettled transactions involving securities, foreign exchange instruments, and commodities. [ 200 ]

Consistent with the BCBS framework, a PRA-designated UK nonbank SD is also required to calculate a CVA risk-weighted asset amount for OTC derivative instruments to reflect the current market value of the credit risk of the counterparty to the PRA-designated UK nonbank SD. [ 201 ] Risk-weighted asset amounts for CVA risk can be calculated following similar methodologies as those described in Subpart E of the Federal Reserve Board's part 217 regulations. [ 202 ]

As discussed in the 2024 Proposal, both the CFTC Capital Rules and the UK PRA Capital Rules also provide that, if approved by NFA or the PRA, respectively, nonbank SDs may also use internal models to calculate market and/or credit risk exposures. [ 203 ] The Commission noted that the internal market and credit risk models under the UK PRA Capital Rules and the CFTC Capital Rules are based on the BCBS framework and preliminarily found that such models must meet comparable quantitative and qualitative requirements covering the same risks, though with slightly different categorization, and including comparable model risk management requirements. [ 204 ] In this regard, the Commission observed that both rule sets address the same types of risk, with similar allowed methodologies and under similar controls. [ 205 ] The Commission also preliminarily determined that the UK PRA Capital Rules and the CFTC Capital Rules are comparable with respect to the requirement that nonbank SDs account for operational risk in computing their minimum capital requirements. [ 206 ] In this connection, the Commission noted that the UK PRA Capital Rules require a PRA-designated UK nonbank SD to calculate an operational risk exposure as a component of the firm's total risk exposure amount. [ 207 ] PRA-designated UK nonbank SDs may use either a standardized approach or, if the PRA-designated UK nonbank has obtained regulatory permission, an internal approach based on the firm's own measurement systems, to calculate their risk-weighted asset amounts for Start Printed Page 58554 operational risk. The CFTC Capital Rules address operational risk both as a stand-alone, separate minimum capital requirement that a nonbank SD is required to meet under prong (iii) of the Bank-Based Approach and as a component of the calculation of risk-weighted assets for nonbank SDs that use Subpart E of the Federal Reserve Board's part 217 regulations to calculate their credit risk-weighted assets via internal models. [ 208 ]

Only one commenter specifically addressed the Commission's comparative analysis of the minimum capital requirement based on risk-weighted assets. The commenter, Ravnitzky, stated that the UK PRA Capital Rules and the CFTC Capital Rules differ in several areas, including in their approaches to calculating risk-weighted amounts for market risk and credit risk. [ 209 ] Ravnitzky asserted that unlike the UK PRA Capital Rules, which use a standardized approach, the CFTC Capital Rules use a model-based approach to calculating risk-weighted amounts. [ 210 ] The Commission notes that this description of the respective rule sets is not accurate. As discussed above, the currently applicable UK PRA Capital Rules and CFTC Capital Rules both incorporate standardized and model-based approaches to calculating market risk and credit risk amounts. [ 211 ]

In conclusion, the Commission finds that the UK PRA Capital Rules and the CFTC Capital Rules are comparable in purpose and effect with respect to the computation of minimum capital requirements based on a nonbank SD's risk-weighted assets. In this regard, the Commission finds that the UK PRA Capital Rules and the CFTC Capital rules have a comparable approach to the computation of market risk exposure amounts and credit risk exposure amounts for on-balance sheet and off-balance sheet exposures, which are intended to ensure that a nonbank SD maintains a sufficient level of regulatory capital to absorb decreases in firm assets, absorb increases in firm liabilities, and meet obligations to counterparties and creditors, without the firm becoming insolvent.

As noted above, prong (iii) of the CFTC Capital Rules' Bank-Based Approach requires a nonbank SD to maintain regulatory capital in an amount equal to or greater than 8 percent of the firm's total uncleared swap margin amount associated with its uncleared swap transactions to address potential operational, legal, and liquidity risks. [ 212 ]

The UK PRA Capital Rules differ from the CFTC Capital Rules in that they do not impose a capital requirement on PRA-designated UK nonbank SDs based on a percentage of the margin for uncleared swap transactions. [ 213 ] In the 2024 Proposal, the Commission described, however, how certain UK PRA capital and liquidity requirements may compensate for the lack of direct analogue to the 8 percent uncleared swap margin amount requirement. [ 214 ] Specifically, the Commission noted that under the UK PRA Capital Rules the total risk exposure amount is computed as the sum of the PRA-designated UK nonbank SD's risk-weighted asset amounts for market risk, credit risk, settlement risk, CVA risk of OTC derivatives instruments, and operational risk. [ 215 ] Notably, the UK PRA Capital Rules require that PRA-designated UK nonbank SDs, including firms that do not use internal models, calculate capital charges for operational risk as a separate component of the total risk exposure amount. The UK PRA Capital Rules also impose separate liquidity requirements designed to ensure that the PRA-designated UK nonbank SDs can meet both short- and long-term obligations, in addition to the general requirement to maintain processes and systems for the identification of liquidity risk. [ 216 ] In comparison, the Commission requires nonbank SDs to maintain a risk management program covering liquidity risk, among other risk categories, but does not have a distinct liquidity requirement. [ 217 ]

Addressing the Commission's request for comment on the comparability between the CFTC's capital requirement based on a percentage of the margin for uncleared swap transactions and the UK PRA Capital Rules' requirements with respect to operational risk and liquidity risk, one commenter, Better Markets, asserted that the requirement for PRA-designated UK nonbank SDs to hold qualifying regulatory capital to cover operational risk is not comparable to the CFTC's requirement for nonbank SDs to hold qualifying capital in an amount equal to at least 8 percent of the nonbank SD's uncleared swap margin amount. [ 218 ] Better Markets further asserted that the proposed Comparability Determination fell short in furnishing an adequate analysis substantiating that the incorporation of an operational risk charge and the existence of separate liquidity requirements would genuinely yield an equivalent result. [ 219 ] Furthermore, Better Markets argued that the Commission should have undertaken “an examination to ascertain whether the PRA-designated UK nonbank SD's operational risk charge and liquidity requirements would adequately cover [its] cumulative amounts of uncleared swaps margin.”  [ 220 ]

The Applicants offered a contrasting view in their comment letter, stating that, although the UK PRA Capital Rules do not “have a direct analogue to the 8 percent uncleared swap margin requirement” under the CFTC Capital Start Printed Page 58555 Rules, they have “various other measures that achieve the same regulatory objective of ensuring that an SD maintains an amount of capital that is sufficient to cover the full range of risks a PRA-designated UK nonbank SD may face.”  [ 221 ] In support of the statement, the Applicants discussed, among other measures, the various categories of risk charges that a PRA-designated UK nonbank SD is required to include in its total risk exposure amount, as well as the capital conservation buffer, leverage ratio floor, and liquidity requirements that the UK PRA Capital Rules impose on PRA-designated UK nonbank SDs. [ 222 ]

The Commission finds that the additional categories of risk-weighted asset amounts that PRA-designated UK nonbank SDs are required to include in the total risk-weighted assets amount, as well as the various regulatory measures seeking to ensure that PRA-designated UK nonbank SDs hold sufficient capital to cover the full range of risks that they may face, support the comparability of the UK PRA Capital Rules and the CFTC Capital Rules even in the absence of a separate capital requirement in the UK PRA Capital Rules requiring PRA-designated UK nonbank SDs to have qualified capital equal to or greater than 8 percent of the amount of uncleared swap margin. The Commission notes that the minimum capital requirement based on a percentage of the nonbank SD's uncleared swap margin amount was conceived as a proxy, not an exact measure, for inherent risk in the SD's positions and operations, including operational risk, legal risk, and liquidity risk. [ 223 ] As the Commission noted in adopting the CFTC Capital Rules, although the amount of capital required of a nonbank SD under the uncleared swap margin calculation is directly related to the volume, size, complexity, and risk of the covered SD's positions, the minimum capital requirement is intended to cover a multitude of potential risks faced by the SD. [ 224 ] The Commission understands that other jurisdictions may adopt alternative measures to cover the same risks. As such, a strict comparison between the amounts that a PRA-designated UK nonbank SD holds to account for operational risk and liquidity risk pursuant to the UK PRA Capital Rules and the amount of uncleared swap margin that a PRA-designated UK nonbank SD would have been required to hold pursuant to the CFTC Capital Rules is not warranted. As discussed in section I.E. above, the Commission's analysis in ascertaining the comparability of a foreign jurisdiction's capital rules to the CFTC Capital Rules is focused on determining whether the foreign jurisdiction's rules have comparable regulatory objectives and achieve comparable outcomes. Following this standard of review, the Commission finds that the various measures that the UK PRA Capital Rules have established to help ensure that PRA-designated UK nonbank SDs hold sufficient capital to cover the full range of risks that they face have comparable objectives and achieve comparable outcomes.

In conclusion, the Commission finds that the UK PRA Capital Rules and the CFTC Capital Rules are comparable in purpose and effect with respect to the requirement that a nonbank SD's minimum level of regulatory capital reflects potential operational risk exposures in addition to market risk and credit risk exposures. The Commission emphasizes that the intent of the minimum capital requirement based on a percentage of the nonbank SD's uncleared swap margin is to establish a minimum capital requirement that would help ensure that the nonbank SD meets its obligations as an SD to market participants, and to cover potential operational risk, legal risk, and liquidity risk in addition to the risks associated with its trading portfolio. [ 225 ] The UK PRA Capital Rules address comparable risks albeit not through a requirement based on a UK nonbank SD's uncleared swap margin amount. In this regard, UK nonbank SDs are required to maintain a minimum level of regulatory capital based on an aggregate of the firm's total risk-weighted asset amounts for market risk, credit risk, and operational risk. Accordingly, the Commission has determined that, notwithstanding the differences in approaches, the UK PRA Capital Rules and CFTC Capital Rules are comparable in purpose and effect in requiring nonbank SDs to maintain a minimum level of regulatory capital that addresses potential market risk, credit risk, and operational risk to help ensure the safety and soundness of the firm, and to ensure that the firm has sufficient capital to absorb decreases in firm assets, absorb increases in firm liabilities, and meet obligations to counterparties and creditors, without the firm becoming insolvent.

Based on its analysis of comments and its holistic assessment of the respective requirements discussed in sections II.C.2.a., b., and c. above, the Commission adopts the Comparability Determination and Comparability Order as proposed with respect to the minimum capital requirements and calculation of regulatory capital, subject to the condition that PRA-designated UK nonbank SDs must maintain a minimum level of regulatory capital in the form of common equity tier 1 capital that equals or exceeds the equivalent of $20 million U.S. dollars.

The Commission detailed the requirements of the CFTC Financial Reporting Rules in the 2024 Proposal. [ 226 ] Specifically, the 2024 Proposal noted that the CFTC Financial Reporting Rules require nonbank SDs to file with the Commission and NFA periodic unaudited and annual audited financial reports. [ 227 ] The unaudited financial reports must include: (i) a statement of financial condition; (ii) a statement of income/loss; (iii) a statement demonstrating compliance with, and calculation of, the applicable regulatory minimum capital requirement; (iv) a statement of changes in ownership equity; (v) a statement of changes in liabilities subordinated to claims of general creditors; and (vi) such further material information necessary to make the required statements not misleading. [ 228 ] The annual audited financial reports must include the same financial statements that are required to be included in the unaudited financial reports, and must further include: (i) a statement of cash flows; (ii) appropriate footnote disclosures; and (iii) a reconciliation of any material differences between the financial statements contained in the annual audited financial reports and the financial statements contained in the Start Printed Page 58556 unaudited financial reports prepared as of the nonbank SD's year-end date. [ 229 ] In addition, a nonbank SD must attach to each unaudited and audited financial report an oath or affirmation that to the best knowledge and belief of the individual making the affirmation the information contained in the financial report is true and correct. [ 230 ] The individual making the oath or affirmation must be a duly authorized officer if the nonbank SD is a corporation, or one of the persons specified in the regulation for business organizations that are not corporations. [ 231 ]

The CFTC Financial Reporting Rules also require a nonbank SD to file the following financial information with the Commission and NFA on a monthly basis: (i) a schedule listing the nonbank SD's financial positions reported at fair market value;  [ 232 ] (ii) schedules showing the nonbank SD's counterparty credit concentration for the 15 largest exposures in derivatives, a summary of its derivatives exposures by internal credit ratings, and the geographic distribution of derivatives exposures for the 10 largest countries;  [ 233 ] and (iii) for nonbank SDs approved to use internal capital models, certain model metrics, such as aggregate value-at-risk (“VaR”), a graph reflecting the daily intra-month VaR for each business line, and counterparty credit risk information. [ 234 ]

The CFTC Financial Reporting Rules further require a nonbank SD to provide the Commission and NFA with information regarding the custodianship of margin for uncleared swap transactions (“Margin Report”). [ 235 ] The Margin Report must contain: (i) the name and address of each custodian holding initial margin or variation margin on behalf of the nonbank SD or its swap counterparties; (ii) the amount of initial and variation margin required by the uncleared margin rules held by each custodian on behalf of the nonbank SD and on behalf its swap counterparties; and (iii) the aggregate amount of initial margin that the nonbank SD is required to collect from, or post with, swap counterparties for uncleared swap transactions subject to the uncleared margin rules. [ 236 ]

A nonbank SD electing the Bank-Based Capital Approach is required to file the unaudited financial report, Schedule 1, schedules of counterparty credit exposures, and the Margin Report with the Commission and NFA no later than 17 business days after the applicable month-end reporting date. [ 237 ] A nonbank SD must file its annual report with the Commission and NFA no later than 60 calendar days after the end of its fiscal year. [ 238 ]

The 2024 Proposal also detailed relevant financial reporting requirements of the UK PRA Financial Reporting Rules. [ 239 ] The UK PRA Financial Reporting Rules require a PRA-designated UK nonbank SD to report information to the PRA concerning its capital and financial condition sufficient to provide a comprehensive view of the firm's risk profile, including information on the firm's capital requirements, leverage ratio, large exposures, and liquidity requirements. [ 240 ] PRA-designated UK nonbank SDs must follow the templates and instructions provided in the PRA Rulebook for purposes of the prudential requirements reporting referred to COREP. [ 241 ] Under the COREP requirements, PRA-designated UK nonbank SDs are required to provide, on a quarterly basis, [ 242 ] calculations in relation to the PRA-designated UK nonbank SD's capital and capital requirements, [ 243 ] capital ratios and capital levels, [ 244 ] and market risk, [ 245 ] among other items.

In addition to the prudential requirements reporting, Article 430(3) of the Reporting (CRR) Part of the PRA Rulebook imposes financial information reporting on PRA-designated UK nonbank SDs that are subject to section 403(1) of the Companies Act 2006 ( i.e., entities that are parent companies  [ 246 ] and report on a consolidated basis using UK-adopted International Financial Reporting Standards (IFRS) and that issue securities admitted to trading on a UK-regulated market). [ 247 ] The relevant reporting templates and instructions, referred to as FINREP, are included in Chapter 6 of the Reporting (CRR) Part of the PRA Rulebook. Under the FINREP requirements, PRA-designated UK nonbank SDs subject to the requirements of Article 430(3) of the Reporting (CRR) Part of the PRA Rulebook are required to provide the following documents to the PRA, among other items: (i) on a quarterly basis, a balance sheet statement (or statement of financial position) that reflects the PRA-designated UK nonbank SD's financial condition;  [ 248 ] (ii) on a quarterly basis, a statement of profit or loss;  [ 249 ] (iii) on a quarterly basis, a breakdown of financial liabilities by product and by counterparty sector;  [ 250 ] (iv) on a quarterly basis, a listing of subordinated financial liabilities;  [ 251 ] and (v) on an annual basis, a statement of changes in equity. [ 252 ]

Under the FINREP requirements, a PRA-designated UK nonbank SD subject to the requirements of Article 430(3) of the Reporting (CRR) Part of the PRA Rulebook is also required to provide the PRA with additional financial information, including a breakdown of Start Printed Page 58557 its loans and advances by product and type of counterparty, [ 253 ] as well as detailed information regarding its derivatives trading activities, [ 254 ] collateral, and guarantees. [ 255 ]

For PRA-designated UK nonbank SD that are not subject to financial information reporting under Article 430(3) of the Reporting (CRR) Part of the PRA Rulebook, the Regulatory Reporting Part of the PRA Rulebook dictates the applicable reporting requirements. [ 256 ] Specifically, as firms that fall into Regulated Activity Group 3 (“RAG 3”), PRA-designated UK nonbank SDs are required to provide the following documents to the PRA, among other items: (i) on a quarterly basis, a balance sheet statement (or statement of financial position) that reflects the PRA-designated UK nonbank SD's financial condition;  [ 257 ] (ii) on a quarterly basis, a statement of profit or loss;  [ 258 ] and (iii) on an annual basis, an annual report and accounts. [ 259 ] The Applicants represented that the six UK PRA-designated nonbank SDs currently registered with the Commission are designated as RAG 3 firms and are required to provide the aforementioned documents. [ 260 ]

Furthermore, all PRA-designated UK nonbank SDs are required to prepare annual audited accounts and a strategic report (together, “annual audited financial report”) pursuant to Parts 15 and 16 of the Companies Act 2006. [ 261 ] The audit of the accounts and report is required to be performed by one or more independent statutory auditors, which have the required skill, resources, and experience to perform their duties based on the complexity of the firm's business and the regulatory requirements to which the firm is subject. [ 262 ] PRA-designated UK nonbank SDs must submit the annual audited financial report to the PRA within 80 business days from the firm's accounting reference date. [ 263 ] In addition, under generally applicable company law requirements, PRA-designated UK nonbank SDs are required to submit the annual audited financial report to the UK Registrar of Companies. [ 264 ] The registrar makes the report available to the public on its website, free of charge. [ 265 ]

The annual audited accounts must comprise, at a minimum, a balance sheet, a profit and loss statement, and notes about the accounts. [ 266 ] The auditor's audit report must include: (i) a description of the annual accounts subject to the audit and the financial reporting framework that was applied in their preparation; (ii) a description of the scope of the audit, which must specify the auditing standards used to conduct the audit; (iii) an audit opinion stating whether the annual accounts give a true and fair view of the state of affairs and/or the profit and loss of the firm, as applicable, and whether the annual accounts have been prepared in accordance with the relevant financial reporting framework; and (iv) a reference to any matters emphasized by the auditor that did not qualify the audit opinion. [ 267 ]

The strategic report is required to include a review of the development and performance of the PRA-designated UK nonbank SD's during the financial year and a description of the principal risks and uncertainties that the firm faces. [ 268 ] The auditors are required to express an opinion on whether the strategic report is consistent with the accounts for the same financial year, and whether the strategic report has been prepared in accordance with applicable legal requirements. [ 269 ] The opinion also must state whether the auditor has identified material misstatements in the strategic report and, if so, describe the misstatement. [ 270 ]

In addition, as noted in the 2024 Proposal, the SEC's UK Order granting substituted compliance for financial reporting to UK nonbank SBSDs, as supplemented by the SEC Order on Manner and Format of Filing Unaudited Financial and Operational Information, require a UK nonbank SBSD to file an unaudited FOCUS Report with the SEC on a monthly basis. [ 271 ] The FOCUS Report is required to include, among other statements and schedules: (i) a statement of financial condition; (ii) a statement of the UK nonbank SBSD's capital computation in accordance with home country Basel-based requirements; (iii) a statement of income/loss; and (iv) a statement of capital withdrawals. [ 272 ] A UK nonbank SBSD is required to file its FOCUS Report with the SEC within 35 calendar days of the month end. [ 273 ]

Based on its review of the UK Application and the relevant UK laws and regulations, the Commission preliminarily determined that, subject to the conditions specified in the 2024 Proposal, the UK PRA Financial Reporting Rules are comparable to CFTC Financial Reporting Rules in purpose and effect. The Commission noted that both sets of rules provide the PRA, Start Printed Page 58558 Commission, and NFA with financial information to monitor a nonbank SD's compliance with capital requirements, and to assess a nonbank SD's overall safety and soundness. [ 274 ] Specifically, the Commission preliminarily found that the UK PRA Financial Reporting Rules impose reporting requirements that are comparable with respect to overall form and content to the CFTC Financial Reporting Rules. [ 275 ] In this regard, both the CFTC Financial Reporting Rules and the UK PRA Financial Reporting Rules require a nonbank SD to file statements of financial condition, statements of profit and loss, and statements of regulatory capital that, collectively, provide information for the PRA, Commission, and NFA to assess a nonbank SD's overall ability to absorb decreases in the value of firm assets, absorb increases in the value of firm liabilities, and cover losses from business activities, including swap dealing activities, without the firm becoming insolvent. [ 276 ]

The proposed conditions would ensure that the Commission and NFA receive appropriate and timely financial information from PRA-designated UK nonbank SDs to monitor the firms' compliance with UK PRA capital requirements and to assess the firms' overall safety and soundness. The proposed conditions would require a PRA-designated UK nonbank SD to provide the Commission and NFA with copies of the relevant templates of the FINREP reports and COREP reports that correspond to the PRA-designated UK nonbank SD's statement of financial condition, statement of income/loss, and statement of regulatory capital, total risk exposure, and capital ratios. These templates consist of FINREP templates 1.1 (Balance Sheet Statement: assets), 1.2 (Balance Sheet Statement: liabilities), 1.3 (Balance Sheet Statement: equity), and 2 (Statement of profit or loss), and COREP templates 1 (Own Funds), 2 (Own Funds Requirements) and 3 (Capital Ratios). In addition, the Commission proposed to require PRA-designated UK nonbank SDs to submit to the Commission and NFA copies of the PRA-designated UK nonbank SD's annual audited financial report. [ 277 ]

The proposed conditions would also require that the PRA-designated UK nonbank SD provide the reports and statements with balances converted to U.S. dollars. [ 278 ] The Commission further recognized that the requirement to convert accounts denominated in British pound to U.S. dollars on the annual audited financial report may have an unintended impact on the opinion expressed by the independent auditor. The Commission, therefore, proposed to accept the annual audited financial report denominated in British pound. [ 279 ]

The proposed conditions also would require a PRA-designated UK nonbank SD to file with the Commission and NFA its: (i) FINREP reports and COREP reports within 35 calendar days of the end of each month; and (ii) annual audited financial report on the on the earlier of the date the report is filed with the PRA or the date the report is required to be filed with the PRA. [ 280 ]

The Commission also proposed a condition to require PRA-designated UK nonbank SDs to file with the Commission and NFA, on a monthly basis, Schedule 1 showing the aggregate securities, commodities, and swap positions of the firm at fair market value as of the reporting date. [ 281 ] The Commission explained that Schedule 1 provides the Commission and NFA with detailed information regarding the financial positions that a nonbank SD holds as of the end of each month, including the firm's swaps positions, which allows the Commission and NFA to monitor the types of investments and other activities that the firm engages in and would assist the Commission and NFA in monitoring the safety and soundness of the firm. [ 282 ] The Commission proposed to require that Schedule 1 be filed by a PRA-designated UK nonbank SD along with the firm's monthly submission of selected FINREP and COREP templates. [ 283 ] The Commission also proposed to require that Schedule 1 be prepared with balances reported in U.S. dollars.

The Commission further proposed that, in lieu of filing FINREP and COREP reports, PRA-designated UK nonbank SDs that are registered with the SEC as UK nonbank SBSDs could satisfy this condition by filing with the CFTC and NFA, on a monthly basis, copies of the unaudited FOCUS Reports that the PRA-designated UK nonbank SDs are required to file with the SEC pursuant to the SEC UK Order, as supplemented by the SEC Order on Manner and Format of Filing Unaudited Financial and Operational Information. The filing of a FOCUS Report was proposed as an elective option for the PRA-designated UK nonbank SD, as an alternative to the filing of unaudited FINREP templates, COREP templates, and Schedule 1 that such firms would otherwise be required to file with the Commission and NFA pursuant to the proposed Comparability Order. In this connection, the Commission noted that all six of the PRA-designated UK nonbank SDs are currently registered with the SEC as UK nonbank SBSDs and would be eligible to file copies of their monthly FOCUS Report with the Commission and NFA in lieu of the FINREP and COREP templates and Schedule 1. A PRA-designated UK nonbank SD electing to file copies of its monthly FOCUS Report would be required to submit the reports to the Commission and NFA within 35 calendar days of the end of each month.

Proposing that PRA-designated UK nonbank SDs that are registered with the SEC as UK nonbank SBSDs file the FOCUS Report in lieu of the FINREP and COREP templates and Schedule 1 as an elective option was consistent with Commission Regulation 23.105(d)(3), which at the time the 2024 Proposal was issued, provided that a nonbank SD or nonbank MSP that is also registered with the SEC as a broker or dealer, an SBSD, or a major security-based swap participant might elect to file a FOCUS Report in lieu of the financial reports required by the Commission. On April Start Printed Page 58559 30, 2024, the Commission amended Commission Regulation 23.105(d)(3) to mandate the filing of a FOCUS Report by such dually-registered entities, including dually-registered non-U.S. nonbank SDs, in lieu of the Commission's financial reports. [ 284 ] As such, the Commission is also adopting as final a revised Condition 10 to require that PRA-designated UK nonbank SDs registered as UK nonbank SBSDs comply with the requirement to file periodic financial statements by filing a copy of the FOCUS Report that the PRA-designated UK nonbank SDs are required to file with the SEC.

The Commission also proposed a condition to require a PRA-designated UK nonbank SD to submit with each set of selected FINREP and COREP templates, annual audited financial report, and the applicable Schedule 1, a statement by an authorized representative or representatives of the PRA-designated UK nonbank SD that to the best knowledge and belief of the person(s) the information contained in the respective reports and statements is true and correct, including the conversion of balances in the statements to U.S. dollars, as applicable. [ 285 ] The statement by the authorized representative or representatives of the PRA-designated UK nonbank SD was intended to be a substitute of the oath or affirmation required of nonbank SDs under Commission Regulation 23.105(f), [ 286 ] to ensure that reports and statements filed with the Commission and NFA are prepared and submitted by firm personnel with knowledge of the financial reporting of the firm who can attest to the accuracy of the reporting and conversion. [ 287 ]

The Commission noted that a Margin Report would assist the Commission and NFA in their assessment of the safety and soundness of the PRA-designated UK nonbank SDs by providing information regarding the firm's swap book and the extent to which it has uncollateralized exposures to counterparties or has not met its financial obligations to counterparties. The Commission explained that this information, along with the list of custodians holding both the firms' and counterparties' collateral for swap transactions, would assist with identifying potential financial impacts to the nonbank SD resulting from defaults on its swap transactions. The Commission further proposed to require a PRA-designated UK nonbank SD to file the Margin Report with the Commission and NFA within 35 calendar days of the end of each month, which corresponds with the proposed timeframe for the PRA-designated UK nonbank SD to file the selected FINREP and COREP templates or FOCUS Report, as applicable. The Commission also proposed to require the Margin Report to be provided with balances reported in U.S. dollars.

The Commission's preliminary determination did not require a PRA-designated UK nonbank SD to file the model metrics and counterparty credit exposure information required by Commission Regulations 23.105(k) and (l)  [ 288 ] in recognition that NFA's current SD risk monitoring program requires all SDs, including PRA-designated UK nonbank SDs, to file with NFA on a monthly basis certain risk metrics that are comparable with the risk metrics contained in Commission Regulation 23.105(k) and (l) and address the market risk and credit risk of the SD's positions. [ 289 ] Specifically, the Commission noted that NFA's monthly risk metric information includes: (i) VaR for interest rates, credit, foreign exchange, equities, commodities, and total VaR; (ii) total stressed VaR; (iii) interest rate, credit spread, foreign exchange market, and commodity sensitivities; (iv) total swaps current exposure both before and after offsetting against collateral held by the firm; and (v) a list of the 15 largest swaps counterparty current exposures before collateral and net of collateral. [ 290 ]

Furthermore, the Commission recognized that although the UK PRA Financial Reporting Rules do not contain an analogue to the CFTC's requirements for nonbank SDs to file monthly model metric information and counterparty exposures information, the PRA has access to comparable information. More specifically, the Commission noted that, under the UK PRA Financial Reporting Rules, the PRA has broad powers to request any information necessary for the exercise of its functions. [ 291 ] As such, the PRA has access to information allowing it to assess the ongoing performance of risk models and to monitor the PRA-designated UK nonbank SD's credit exposures, which may be comprised of credit exposures to primarily other UK and EU counterparties. In addition, the COREP reports, which PRA-designated UK nonbank SDs are required to file with the PRA on a quarterly basis, include information regarding the PRA-designated UK nonbank SD's risk exposure amounts, including risk-weighted exposure amounts for credit risk. [ 292 ]

The Commission received comments regarding the comparability of financial reporting and specific comments addressing several of the financial reporting issues on which the Commission solicited feedback. Better Markets expressed a general disagreement with the Commission's preliminary finding of comparability, arguing that the number and variety of conditions regarding financial reporting are the most compelling evidence that the requirements are not comparable. [ 293 ] More generally, Better Markets asserted that the 2024 Proposal did not provide a sufficient analysis supporting the Commission's preliminary conclusion that the UK PRA and the U.S. financial Start Printed Page 58560 reporting frameworks would produce comparable outcomes. [ 294 ]

Better Markets also disagreed with the 2024 Proposal to the extent that the Commission proposed not to require PRA-designated UK nonbank SDs that have been approved by the PRA to use capital models to file the monthly model metric information required by Commission Regulation 23.105(k) with the Commission or NFA. [ 295 ] Commission Regulation 23.105(k) requires nonbank SDs that have been approved by the Commission or NFA to use models to compute market risk or credit risk for computing capital requirements to file certain information with the Commission and NFA on a monthly basis. [ 296 ] As noted above, the information required to be filed includes: (i) for nonbank SDs approved to use market risk models, a listing of any products that the nonbank SD excludes from the approved market risk model and the amount of the standardized market risk charge taken on such products; (ii) a graph reflecting, for each business line of the nonbank SD, the daily intra-month VaR; (iii) the aggregate VaR for the nonbank SD; (iv) certain credit risk information for swaps, mixed swaps and security-based swaps, including: (a) overall current exposure, (b) current exposure listed by counterparty for the 15 largest exposures, (c) the 10 largest commitments listed by counterparty, (d) maximum potential exposure listed by counterparty for the 15 largest exposures, (e) aggregate maximum potential exposure, (f) a summary report reflecting the SD's current and maximum potential exposures by credit rating category, and (g) a summary report reflecting current exposure for each of the top ten countries to which the nonbank SD is exposed. [ 297 ] Better Markets stated that by not requiring the information contained in Commission Regulation 23.105(k), the Commission was proposing to take a back seat to the UK and blindly accept the assessments resulting from the PRA-designated UK nonbank SDs' use of internal models to calculate risk. [ 298 ]

With respect to Better Markets' statement that the number and variety of conditions regarding financial reporting are the most compelling evidence that the requirements are not comparable, the Commission disagrees that the inclusion of conditions in the Comparability Order demonstrates that the UK PRA Financial Reporting Requirement are not comparable to CFTC Financial Reporting Requirements in achieving the overall objective of ensuring the safety and soundness of nonbank SDs. As discussed in section I.E. above, the conditions impose obligations on PRA-designated UK nonbank SDs to provide information to the Commission and NFA necessary for the effective oversight of the PRA-designated UK nonbank SDs on an ongoing basis. As also discussed in section I.E. above, Commission staff engaged in a thorough analysis of the UK PRA Capital Rules and UK PRA Financial Reporting Rules, which supports the Commission's conclusion that the respective regulatory frameworks would produce comparable outcomes.

The Commission also does not agree that its approach is effectively deferring model oversight to the PRA or that it is otherwise “blindly” accepting the internal model-based assessments of the PRA-designated UK nonbank SDs. As noted above, pursuant to NFA rules, all registered SDs, including PRA-designated UK nonbank SDs, are required to submit to NFA, on a monthly basis, a list of specified risk metrics related to the SD's market risk and credit risk exposures. [ 299 ] Specifically, the risk metrics include: (i) VaR for interest rates, credit, foreign exchange, equities, commodities, and total VaR; (ii) total stressed VaR; (iii) interest rate, credit spread, foreign exchange market, and commodity sensitivities; (iv) total swaps current exposure both before and after offsetting against collateral held by the firm; and (v) a list of the 15 largest swaps counterparty current exposures. [ 300 ] As part of its regulatory oversight program, NFA uses the risk metrics information to identify firms that may pose heightened risk and to allocate appropriate oversight resources. NFA also may request additional information from a nonbank SD to the extent it determines that information in the risk metrics or other financial filings warrants a need for additional follow-up. Furthermore, Commission staff has access to the collected risks metrics information and participates in NFA's risk monitoring function by regularly exchanging information and discussing potential risks with NFA staff.

As the list of specified risk metrics discussed above indicates, although the information collected by NFA is not identical to the information required under Commission Regulation 23.105(k), there is a significant overlap in the data items. The Commission also notes that NFA, in its role of primary supervisor of nonbank SDs' risk management practices, has identified the risk data items listed in NFA Notice I-17-10 as the most relevant risk metrics to be collected for oversight purposes. As such, the Commission finds that the information required pursuant to NFA Notice I-17-10 would provide the Commission and NFA with key data allowing them to monitor nonbank SDs' risk exposures. In addition, the Commission has the ability to request additional information from its registrants, including PRA-designated UK nonbank SDs, at any time. Finally, the Commission notes that the PRA, which will be conducting the initial approval and ongoing assessment of the performance of the PRA-designated UK nonbank SDs' internal models, under a regulatory framework that the Commission finds comparable to the CFTC Capital Rules, will have access to additional information that the PRA deems relevant in the conduct of such approval and assessment. The Commission, therefore, concludes that it is not necessary to require PRA-designated UK nonbank SDs relying on the final Comparability Order to submit the model metric information and credit risk information mandated by Commission Regulations 23.105(k) and (l).

Finally, the Applicants addressed the Commission's request for comment on the compliance dates for the reporting conditions that the proposed Comparability Order would impose on PRA-designated UK nonbank SDs. [ 301 ] The Applicants requested that the Commission set the compliance date at least six months following the issue date of the final Comparability Order to allow PRA-designated UK nonbank SDs to adequately prepare for compliance with the reporting conditions imposed by the Comparability Order. [ 302 ]

The Commission believes that granting an additional period of time to allow PRA-designated UK nonbank SDs to develop and implement the necessary systems and processes for compliance with the Comparability Order is appropriate with respect to the new reporting obligations imposed on PRA- Start Printed Page 58561 designated UK nonbank SDs under the final Order. For other reporting obligations, for which a process already exists, such as the reports that PRA-designated UK nonbank SDs currently submit to the Commission and NFA pursuant to CFTC Staff Letter 22-10, [ 303 ] prepare pursuant to the UK PRA Financial Reporting Rules, and/or submit to the SEC ( i.e., FOCUS Reports), additional time for compliance does not appear necessary. Accordingly, the Commission is setting a compliance date of 180 calendar days from the date of publication of the final Comparability Order in the Federal Register , to comply with final Condition 14, which requires PRA-designated UK nonbank SDs to file monthly Margin Reports with the Commission and NFA.

For purposes of clarity, the Commission also notes that PRA-designated UK nonbank SDs may present the financial information required to be provided to the Commission and NFA under the final Comparability Order in accordance with generally accepted accounting principles that the PRA-designated UK nonbank SD uses to prepare general purpose financial statements in the UK. This clarification is consistent with proposed Condition 9, which the Commission adopts without modification in the final Comparability Order, requiring that the PRA-designated UK nonbank SD prepares and keeps current ledgers and other similar records “in accordance with [the PRA Rulebook] and conforming with the applicable accounting principles.”  [ 304 ] In taking the position that PRA-designated UK nonbank SDs may provide financial reporting prepared in accordance with the accounting standards applicable in their home jurisdiction, the Commission considered the nature of the financial reporting information required from nonbank SDs for purposes of monitoring their overall financial condition and compliance with capital requirements. Specifically, the Commission notes that the requirements for how nonbank SDs calculate their risk-weighted assets and capital ratio, in both the UK and the U.S., follow a rules-based approach consistent with the Basel standards, and, consequently, the Commission does not anticipate that a variation in the applicable accounting standards would materially impact this calculation. [ 305 ] In this regard, the Commission notes that PRA-designated UK nonbank SDs currently submit financial reports, including a statement of financial condition and a statement of regulatory capital, pursuant to CFTC Staff Letter 22-10. [ 306 ] The reports provide the Commission with appropriate information to assess the financial and operational condition of PRA-designated UK nonbank SDs, as well as the firms' compliance with the capital ratios imposed on PRA-designated UK nonbank SDs under the UK PRA Capital Rules.

In summary, the Commission adopts the final Comparability Order and conditions substantially as proposed with respect to the comparability of the CFTC Financial Reporting Rules and UK PRA Financial Reporting Requirements, subject to the amendment in Condition 10 to mandate the filing by EU nonbank SDs registered as EU nonbank SBSDs of a copy of the FOCUS Report that such dually-registered PRA-designated UK nonbank SDs are required to file with the SEC. The Commission also specifies, in final Conditions 10, 12, and 14, that the conversion of balances to U.S. dollars must be done using a commercially reasonable and observable British pound/U.S. dollar spot rate as of the date of the respective report. Finally, the Commission also grants an additional compliance period for the new reporting obligations imposed on PRA-designated UK nonbank SDs under the final Order set forth below.

The Commission noted in the 2024 Proposal that the CFTC Financial Reporting Rules require nonbank SDs to provide the Commission and NFA with written notice of certain defined events. [ 307 ] Commission Regulation 23.105(c) requires a nonbank SD to file written notice with the Commission and NFA of the following events: (i) the nonbank SD's regulatory capital is less than the minimum amount required; (ii) the nonbank SD's regulatory capital is less than 120 percent of the minimum amount required; (iii) the nonbank SD fails to make or to keep current required financial books and records; (iv) the nonbank SD experiences a reduction in the level of its excess regulatory capital of 30 percent or more from the amount last reported in a financial report filed with the Commission; (v) the nonbank SD plans to distribute capital to equity holders in an amount in excess of 30 percent of the firm's excess regulatory capital; (vi) the nonbank SD fails to post to, or collect from, a counterparty (or group of counterparties under common ownership or control) required initial and variation margin, and the aggregate amount of such margin equals or exceeds 25 percent of the nonbank SD's minimum capital requirement; (vii) the nonbank SD fails to post to, or collect from, swap counterparties required initial and variation margin, and the aggregate amount of such margin equals or exceeds 50 percent of the nonbank SD's minimum capital requirement; and (viii) the nonbank SD is registered with the SEC as an SBSD and files a notice with the SEC under applicable SEC Rules. [ 308 ]

The notices are part of the Commission's overall program of helping to ensure the safety and soundness of nonbank SDs and the Start Printed Page 58562 swaps markets in general. [ 309 ] Notices provide the Commission and NFA with an opportunity to assess whether there is an actual or potential financial and/or operational issue at a nonbank SD. In situations where there is an underlying issue, Commission and NFA staff engage with the nonbank SD in an effort to minimize potential adverse impacts on the firm, swap counterparties, and the larger swaps market. [ 310 ]

The UK capital and resolution framework, in turn, require PRA-designated UK nonbank SDs to provide certain notices to the PRA concerning the firm's compliance with relevant laws and regulations. [ 311 ] The Commission noted that the UK PRA Financial Reporting Rules require a PRA-designated UK nonbank SD to provide notice to the PRA within five business days if the firm fails to meet its combined buffer requirement, which at a minimum consists of a capital conservation buffer of 2.5 percent of the PRA-designated UK nonbank SD's total risk exposure amount. [ 312 ] To meet its capital buffer requirements, a PRA-designated UK nonbank SD must hold common equity tier 1 capital in addition to the minimum common equity tier 1 ratio requirement of 4.5 percent of the firm's core capital requirement of 8 percent of the firm's total risk exposure amount. [ 313 ] The notice to the PRA must be accompanied by a capital conservation plan that sets out how the PRA-designated UK nonbank SD will restore its capital levels. [ 314 ] The capital conservation plan is required to include: (i) the “maximum distributable amount” calculated in accordance with the PRA rules; (ii) estimates of income and expenditures and a forecast balance sheet; (iii) measures to increase the capital ratios of the PRA-designated UK nonbank SD; and (iv) a plan and timeframe for the increase in the capital of the PRA-designated UK nonbank SD with the objective of meeting fully the combined buffer requirement. [ 315 ] The PRA is required to assess the capital conservation plan and may approve the plan only if it considers that the plan would be reasonably likely to conserve or raise sufficient capital to enable the PRA-designated UK nonbank SD to meet its combined capital buffer requirement within a timeframe that the PRA considers to be appropriate. [ 316 ] A PRA-designated UK nonbank SD is required to notify the PRA as early as possible where it has identified a material risk to its ability to meet the combined buffer according to the capital conservation plan and timeframe approved by the PRA. [ 317 ]

In addition, a PRA-designated UK nonbank SD must notify the PRA if the firm's management considers that the firm is failing or will in the near future fail to satisfy one or more of the “threshold conditions,” which are the minimum requirements that a PRA-designated UK nonbank SD must meet to be permitted to carry the regulated activities in which it engages. [ 318 ] In broad terms, the PRA's threshold conditions include, among other things, requirements that the firm has appropriate financial resources and capacity to measure, monitor and manage risks. [ 319 ]

Emphasizing that the requirement for a nonbank SD to file notice with the Commission and NFA if the firm becomes undercapitalized or if the firm experiences a decrease of excess regulatory capital below defined levels is a central component of the Commission's and NFA's oversight program for nonbank SDs, the Commission proposed a condition to require a PRA-designated UK nonbank SD to file with the Commission and NFA copies of notices filed under the Capital Buffers Part of the PRA Rulebook by PRA-designated UK nonbank SDs alerting the PRA of a breach of the PRA-designated UK nonbank SD's combined capital buffer. [ 320 ] The Commission proposed to require that the notice be filed by the PRA-designated UK nonbank SD within 24 hours of the filing of the notice with the PRA.

The Commission, however, preliminarily determined that the requirement for a PRA-designated UK nonbank SD to provide notice of a breach of its capital buffer requirements to the PRA is not sufficiently comparable in purpose and effect to the CFTC notice provisions contained in Commission Regulation 23.105(c)(1) and (2), [ 321 ] which require a nonbank SD to provide notice to the Commission and to NFA if the firm fails to meet its minimum capital requirement or if the firm's regulatory capital falls below 120 percent of its minimum capital requirement (“Early Warning Level”). The Commission noted that, in its preliminary view, the requirement for a PRA-designated UK nonbank SD to provide notice of a breach of its capital buffer requirements does not achieve a comparable outcome to the CFTC's Early Warning Level requirement due to the difference in the thresholds triggering a notice requirement in the respective rule sets. Therefore, the Commission proposed a condition to require a PRA-designated UK nonbank SD to file a notice with the Commission and NFA if the firm's capital ratio does not equal or exceed 12.6 percent. [ 322 ] The proposed condition would further require the PRA-designated UK nonbank SD to file the notice with the Commission and NFA within 24 hours of when the firm knows or should have known that its regulatory capital was below 120 percent of its minimum capital requirement. [ 323 ]

The Commission also noted that the UK PRA Financial Reporting Rules also do not contain an explicit requirement for a PRA-designated UK nonbank SD to notify the PRA if the firm fails to maintain current books and records, experiences a decrease in regulatory capital over levels previously reported, or fails to collect or post initial margin with uncleared swap counterparties that exceed certain threshold levels. [ 324 ] The UK PRA Financial Reporting Rules also do not require a PRA-designated UK nonbank SD to provide the PRA with advance notice of equity withdrawals initiated by equity holders that exceed defined amounts or percentages of the firm's excess regulatory capital. [ 325 ]

To ensure that the Commission and NFA receive prompt information concerning potential operational or financial issues that may adversely Start Printed Page 58563 impact the safety and soundness of a PRA-designated UK nonbank SD, the Commission proposed to condition the Comparability Order to require PRA-designated UK nonbank SDs to file certain notices mandated by Commission Regulation 23.105(c) with the Commission and NFA as discussed below. Pursuant to the proposed conditions, a PRA-designated UK nonbank SD would be required to file a notice the Commission and NFA if the firm fails to maintain current books and records with respect to its financial condition and financial reporting requirements. [ 326 ] The Commission stated that, in this context, books and records would include current ledgers or other similar records which show or summarize, with appropriate references to supporting documents, each transaction affecting the PRA-designated UK nonbank SD's asset, liability, income, expense, and capital accounts in accordance with the accounting principles accepted by the relevant authorities. [ 327 ] The Commission further stated that it preliminarily believed that the maintenance of current books and records is a fundamental and essential component of operating as a registered nonbank SD and that the failure to comply with such a requirement may indicate an inability of the firm to promptly and accurately record transactions and to ensure compliance with regulatory requirements, including regulatory capital requirements. As such, the Commission proposed to condition the proposed Order on a PRA-designated UK nonbank SD providing the Commission and NFA with a written notice within 24 hours if the firm fails to maintain books and records on a current basis. [ 328 ]

The Commission further proposed to condition the Comparability Order on a PRA-designated UK nonbank SD filing a notice with the Commission and NFA if: (i) a single counterparty, or group of counterparties under common ownership or control, fails to post required initial margin or pay required variation margin on uncleared swap and security-based swap positions that, in the aggregate, exceeds 25 percent of the PRA-designated UK nonbank SD's minimum capital requirement; (ii) counterparties fail to post required initial margin or pay required variation margin to the PRA-designated UK nonbank SD for uncleared swap and security-based swap positions that, in the aggregate, exceeds 50 percent of the PRA-designated UK nonbank SD's minimum capital requirement; (iii) a PRA-designated UK nonbank SD fails to post required initial margin or pay required variation margin for uncleared swap and security-based swap positions to a single counterparty or group of counterparties under common ownership and control that, in the aggregate, exceeds 25 percent of the PRA-designated UK nonbank SD's minimum capital requirement; and (iv) a PRA-designated UK nonbank SD fails to post required initial margin or pay required variation margin to counterparties for uncleared swap and security-based swap positions that, in the aggregate, exceeds 50 percent of the PRA-designated UK nonbank SD's minimum capital requirement. The Commission proposed to require this notice so that, in the event that such a notice is filed, the Commission and NFA may commence communication with the PRA-designated UK nonbank SD and the PRA to obtain an understanding of the facts that have led to the failure to exchange material amounts of initial margin and variation margin in accordance with the applicable margin rules, and to assess whether there is a concern regarding the financial condition of the firm that may impair its ability to meet its financial obligations to customers, counterparties, creditors, and general market participants, or otherwise adversely impact the firm's safety and soundness. [ 329 ]

The Commission did not propose to require a PRA-designated UK nonbank SD to file notices with the Commission and NFA concerning withdrawals of capital or changes in capital levels as such information would be reflected in the financial statement reporting filed with the Commission and NFA as conditions of the order, and because the PRA-designated UK nonbank SD's capital levels are monitored by the PRA. As such, the Commission preliminarily considered that the separate reporting of the information to the Commission would be superfluous. [ 330 ]

The Commission proposed to require that a PRA-designated UK nonbank SD file any notices required under the Order with the Commission and NFA reflecting any balances, where applicable, in U.S. dollars. The Commission stated that each notice required by the proposed Comparability Order had to be filed in accordance with instructions issued by the Commission or NFA. [ 331 ]

Based on its review of the UK Application and the relevant UK laws and regulations, and subject to the proposed conditions discussed above and specified in the proposed Comparability Order, the Commission preliminarily determined that the UK PRA Financial Reporting Rules related to notice provisions are comparable in purpose and effect to the notice provisions of the CFTC Financial Reporting Rules. [ 332 ]

With respect to the proposed requirements in Condition 20 that a PRA-designated UK nonbank SD file a notice with the Commission and NFA within 24 hours of when the firm knew or should have known that its regulatory capital fell below 120 percent of its minimum capital requirement, the Applicants asserted that the wording of the proposed condition raises practical challenges as it would require notification prior to the discovery of the relevant event. [ 333 ] The Applicants recommended that the Commission amend the proposed condition to require notice within 24 hours of when the firm “knew” that its regulatory capital fell below 120 percent of the minimum capital requirement. [ 334 ] Similarly, with respect to proposed Condition 21, which would require a PRA-designated UK nonbank SD to file a notice with the Commission and NFA within 24 hours if the firm fails to make or keep current the financial books and records, the Applicants recommended that the Commission amend the condition to require that a PRA-designated UK file a notice within 24 hours “of when it knows it has failed to make or keep current the financial books and records.”  [ 335 ] In addition, with respect to proposed Condition 20, the Applicants asserted that, pursuant to the condition, a PRA-designated UK nonbank SD would calculate the Early Warning Level by applying a buffer of 20 percent in excess capital, in the form of common equity tier 1 capital, on top of the firm's capital conservation buffer, which, at a minimum, equals 2.5 percent of the firm's total risk exposure amount and must be met in the form of common equity tier 1 capital. In the Applicants' view, an aggregate notification trigger of 12.6 percent of total risk exposure amount would be too Start Printed Page 58564 high. The Applicants recommended that the Commission set the notification trigger at 120 percent of the minimum total capital requirement. [ 336 ]

The Early Warning Level notice requirement is a central component of the Commission's and NFA's oversight programs. The Commission, however, recognizes that by requiring a PRA-designated UK nonbank SD to provide notice if its capital ratio falls below 120 percent of the firm's minimum capital requirement, as defined to comprise the applicable capital buffers, the Commission would be imposing a higher threshold level for the notice trigger than is currently applicable to nonbank SDs under the CFTC Capital Rules. To achieve the condition's goal of providing the Commission and NFA with information on decreases in capital that may indicate financial or operational challenges at the firm, the Commission is revising proposed Condition 20 to require instead that a PRA-designated UK nonbank SD provide notice to the Commission if it experiences a 30 percent or more decrease in its excess regulatory capital as compared to the last reported. [ 337 ] The condition is consistent with the requirement applicable to nonbank SDs under Commission Regulation 23.105(c)(4). [ 338 ] The Commission believes that this condition, combined with the condition requiring a PRA-designated UK nonbank SD to file with the Commission and NFA copies of notices filed with the PRA of a breach of the PRA-designated UK nonbank SD's combined capital buffer, will provide a timely opportunity to the Commission and NFA to initiate conversations and fact finding with a PRA-designated UK nonbank SD that may be experiencing operational or financial issues that may adversely impact the firm's ability to meet its obligations to market participants, including customers or swap counterparties.

In connection with the Applicants' general request that the Commission set the compliance date of the Comparability Order at least six months following the issuance of the final Order, the Commission believes, as stated above, that granting an additional period of time to allow PRA-designated UK nonbank SDs to establish and implement the necessary systems and processes to comply with the notice reporting obligations imposed by the Comparability Order is appropriate with respect to certain notice obligations. Specifically, the Commission understands that establishing a system and process for monitoring material decreases in excess regulatory capital as required by final Condition 20 or for monitoring failures to collect or post initial margin or variation margin for uncleared swap transactions that exceed specified thresholds for purposes of complying with final Condition 22 may take time. [ 339 ] Conversely, the Commission does not believe that additional time is necessary for implementing a system and process of providing a notice to the Commission and NFA in connection with the occurrence of events that PRA-designated UK nonbank SDs currently monitor and/or report to the PRA. The Commission is also of the view that, given the nature of the notice obligation, PRA-designated UK nonbank SDs should be in a position to comply with all other notice obligations, including those requiring PRA-designated UK nonbanks SDs to provide notice to the Commission and NFA if they fail to make or keep current financial books and records or if they fail to maintain regulatory capital in the form of common equity tier 1 equal or in excess of the U.S. dollar equivalent of $20 million, immediately upon effectiveness of the Comparability Order. Specifically, with respect to the requirement in Condition 21 that a PRA-designated UK nonbank SD notify the Commission and NFA if the firm fails to make or keep current the financial books and records, the Commission notes that maintaining current books and records of all financial transactions is a fundamental recordkeeping requirement for a registered nonbank SD, and is essential to provide management with the information necessary to ensure that transactions are timely and accurately reported and that the firm complies with capital and other regulatory requirements. The Commission finds that it is necessary for a nonbank SD to maintain internal controls and procedures to affirmatively monitor that financial books and records are being maintained on a current basis. The Commission also notes that the language of Condition 21 is consistent with the timing standard of Commission Regulation 23.105(c)(3). [ 340 ] As such, the Commission is adopting Condition 21 as proposed. The Commission, however, is setting a compliance date of 180 calendar days after the publication of the final Comparability Order in the Federal Register with respect to the notice reporting obligations under final Conditions 20 and 22 of the Comparability Order.

With respect to the notice requirement in final Condition 22, the Applicants also recommended that the Commission clarify the term “minimum capital requirement,” used in connection with the thresholds triggering a notice requirement. [ 341 ] In response, the Commission will amend the condition to indicate that, in the context of final Condition 22, the PRA-designated UK nonbank SD's “minimum capital requirement” is the core capital requirement under the UK PRA Capital Rules, excluding capital buffers.

Finally, the Applicants recommended that the Commission amend proposed Condition 24 to require that a PRA-designated UK nonbank SDs, or an entity acting on its behalf, notify the Commission and NFA of “material changes” to the UK PRA Capital Rules or UK PRA Financial Reporting Rules instead of “proposed or final material changes” to the UK PRA Capital Rules or UK PRA Financial Reporting Rules. [ 342 ] Separately, the Applicants noted that the language of proposed Condition 24 is confusing in that it differentiates between rules that are “imposed on” and those that “apply to” PRA-designated UK nonbank SDs. [ 343 ] The Commission did not intend to distinguish between rules that are “imposed on” and rules that “apply to” PRA-designated UK nonbank SDs and will use instead the defined terms “UK PRA Capital Rules” and “UK PRA Financial Reporting Rules” to address the potential for confusion. The Start Printed Page 58565 Commission, however, believes that it is necessary that the Commission and NFA receive an advance notice of potential material changes to the foreign jurisdiction's rules to allow the Commission a sufficient time to assess the potential impact of the proposed amendments and to address potential changes to the Comparability Determination and Comparability Order. As such, the Commission is adopting Condition 24 as proposed with regard to the required notice of “proposed and final material changes” to the UK PRA Capital Rules and UK PRA Financial Reporting Rules.

The Commission did not receive any comments with respect to the following proposed notice conditions: (i) the PRA-designated UK nonbank SD files notice with the Commission and NFA within 24 hours of being informed by the PRA that the firm is not in compliance with any component of the UK PRA Capital Rules or UK PRA Financial Reporting Rules (proposed Condition 15); (ii) the PRA-designated UK nonbank SD files notice with the Commission and NFA within 24 hours if the firm fails to maintain regulatory capital in the form of common equity tier 1 capital, as defined in Article 26 of UK CRR, equal to or in excess of the U.S. dollar equivalent of $20 million (proposed Condition 16); (iii) the PRA-designated UK nonbank SD provides the Commission and NFA with notice within 24 hours of filing a capital conservation plan (proposed Condition 17); (iv) the PRA-designated UK nonbank SD files notice with the Commission and NFA within 24 hours of being required by the PRA to maintain additional capital or additional liquidity requirements, or to restrict its business operations, or to comply with certain other additional requirements that the PRA may impose pursuant to the UK PRA Capital Rules and the UK PRA Financial Reporting Rules (proposed Condition 18); (v) the PRA-designated UK nonbank SD files a notice with the Commission and NFA within 24 hours if it fails to maintain its MREL (proposed Condition 19); or (vi) the PRA-designated UK nonbank SD files notice of PRA approving a change in the firm's fiscal year-end date, which must be filed with the Commission and NFA at least 15 business days prior to the effective date of the change (proposed Condition 23).

With regard to the proposed condition requiring that the PRA-designated UK nonbank SD file a notice with the Commission and NFA within 24 hours of filing a capital conservation plan, the Commission will revise the condition to require that the notice be filed within 24 hours of when the PRA-designated UK nonbank SD breaches its combined capital buffer requirement and is required to file a capital conservation plan. Thus, the Commission will help ensure that the PRA-designated UK nonbank SD provides a timely notice within 24 hours of breaching its combined capital buffer requirement instead of 24 hours of filing the capital conservation plan, which may occur up to five business days after the breach of the combined buffer requirement.

In conclusion, the Commission finds that the regulatory notice provisions of the UK PRA Financial Reporting Rules and the CFTC Financial Reporting Rules, after consideration of the conditions imposed in the final Comparability Order, are comparable in purpose and effect, and achieve comparable outcomes, by providing timely notice to the PRA, and to the Commission and NFA, of specified events at a nonbank SD that may potentially indicate an ongoing issue with the safety and soundness of the firm and/or its ability to meet its obligations to swap counterparties, creditors, or other market participants without the firm becoming insolvent. As such, the Commission adopts the final Comparability Order and conditions as proposed with respect to the Commission's analysis of comparability of the PRA and Commission's nonbank SD notice reporting requirements, subject to the revisions in final Conditions 17 and 20, and the clarifying changes to final Condition 24 discussed above. The Commission is also adopting a compliance date for certain notice reporting requirements as discussed above in the final Comparability Order.

In the 2024 Proposal, the Commission discussed the oversight of nonbank SDs, noting that the Commission and NFA conduct ongoing supervision of nonbank SDs to assess their compliance with the CEA, Commission regulations, and NFA rules by reviewing financial reports, notices, risk exposure reports, and other filings that nonbank SDs are required to file with the Commission and NFA. [ 344 ] The 2024 Proposal also noted that the Commission and NFA also conduct periodic examinations as part of the supervision of nonbank SDs, including routine onsite examinations of nonbank SDs' books, records, and operations to ensure compliance with CFTC and NFA requirements. [ 345 ] In this regard, as noted in section I.E. above, section 17(p) of the CEA requires NFA, as a registered futures association, to establish minimum capital and financial requirements for nonbank SDs and to implement a program to audit and enforce compliance with such requirements. [ 346 ]

The Commission also discussed the financial reports and notices required under the CFTC Financial Reporting Rules, noting that the reports and notices provide the Commission and NFA with information necessary to: ensure the nonbank SD's compliance with minimum capital requirements; assess the firm's overall safety and soundness by being able to meet its financial obligations to customers, counterparties, creditors, and general market participants; and identify potential issues at a nonbank SD that may impact the firm's ability to maintain compliance with the CEA and Commission regulations. [ 347 ] As discussed in the 2024 Proposal, the Commission and NFA also have the authority to require a nonbank SD to provide any additional financial and/or operational information as the Commission or NFA may specify to monitor the safety and soundness of the firm. [ 348 ] The Commission further noted that it has authority to take disciplinary actions against a nonbank SD for failing to comply with the CEA and Commission regulations. In this regard, section 4b-1(a) of the CEA provides the Commission with exclusive authority to enforce the capital requirements imposed on nonbank SDs adopted under section 4s(e) of the CEA. [ 349 ]

With respect to PRA-designated UK nonbank SDs, the Commission noted in the 2024 Proposal that the PRA conducts oversight of the firm's compliance with the UK PRA Capital Rules and the UK PRA Financial Reporting Rules. In this regard, the Commission noted that the PRA has supervision, audit, and investigation powers with respect to PRA-designated UK nonbank SDs, which include the powers to obtain specified information reasonably required in connection with the exercise of the PRA's functions, the power to conduct or order investigations, and the power to impose sanctions on PRA-designated UK nonbank SDs that breach their regulatory obligations, including those deriving from the UK PRA Capital Rules Start Printed Page 58566 and the UK PRA Financial Reporting Rules. [ 350 ]

The PRA also monitors the capital adequacy of PRA-designated UK nonbank SDs through supervisory measures on an ongoing basis. The monitoring includes assessing the notices and the capital conservation plan discussed in section II.E.1. above. In addition, the PRA is empowered with a variety of measures to address a PRA-designated UK nonbank SD's financial deterioration. [ 351 ] Under its general supervisory powers, the PRA may impose new requirements to a PRA-designated UK nonbank SD if the firm is failing, or likely to fail, to satisfy the threshold conditions for which the PRA is responsible. [ 352 ] More specifically, a breach in a PRA-designated UK nonbank SD's capital buffers automatically triggers restrictions on the firm's ability to make certain distributions ( e.g., pay certain dividends or employee bonuses). [ 353 ] In addition, the PRA may impose administrative penalties or other administrative measures, including prudential charges, if a PRA-designated nonbank SD's liquidity position falls below the liquidity and stable funding requirements. [ 354 ]

In case of non-compliance with the capital and liquidity thresholds, the PRA may also order PRA-designated UK nonbank SDs to comply with additional requirements, including: (i) maintaining additional capital in excess of the minimum requirements, if certain conditions are met; (ii) requiring that the PRA-designated UK nonbank SD submit a plan to restore compliance with applicable capital or liquidity thresholds; (iii) imposing restrictions on the business or operations of the PRA-designated UK nonbank SD; (iv) imposing restrictions or prohibitions on distributions or interest payments to shareholders or holders of additional tier 1 capital instruments; (v) requiring additional or more frequent reporting requirements; and (vi) imposing additional specific liquidity requirements. [ 355 ] The PRA may also sanction the PRA-designated UK nonbank SD if the firm's capital or liquidity fall below the applicable thresholds or the PRA has evidence that the firm will breach such thresholds in the next 12 months. [ 356 ] The PRA may also withdraw a PRA-designated UK nonbank SD's authorization if the firm no longer meets its minimum capital requirements. [ 357 ]

In addition, if the capital and liquidity requirements are breached, the PRA may take early measures to intervene, such as requiring management to take certain actions, order members of management to be removed or replaced, or require changes to the firm's business strategy or legal or operational structure, among other measures. [ 358 ]

Although the PRA generally has broad discretion as to what powers it may exercise, the UK PRA Capital Rules and the UK PRA Financial Reporting Rules specifically mandate that the PRA require PRA-designated UK nonbank SDs to hold increased capital when: (i) risks or elements of risks are not covered by the capital requirements imposed by the UK PRA Capital Rules; (ii) the PRA-designated UK nonbank SD lacks robust governance arrangements, appropriate resolution and recovery plans, processes to manage large exposures or effective processes to maintain on an ongoing basis the amounts, types, and distribution of capital needed to cover the nature and level of risks to which it might be exposed; or (iii) the sole application of other administrative measures would be unlikely to timely and sufficiently improve the firm's arrangements and processes. [ 359 ]

Based on its review of the Application and its analysis of the relevant laws and regulations, the Commission preliminarily found that the PRA has the necessary powers to supervise, investigate, and discipline PRA-designated UK nonbank SDs for compliance with the applicable capital and financial reporting requirements, and to detect and deter violations of, and ensure compliance with, the applicable UK capital and financial reporting requirements. [ 360 ] Furthermore, the Commission noted that it retains supervision, examination, and enforcement authority over PRA-designated UK nonbank SDs that are covered by the Comparability Order. [ 361 ] Specifically, the Commission noted that a non-U.S. nonbank SD that operates under substituted compliance remains subject to the Commission's examination authority and may be subject to a Commission enforcement action if the firm fails to comply with a foreign jurisdiction's capital adequacy or financial reporting requirements. [ 362 ] The ability of the Commission to exercise its enforcement authority over a PRA-designated UK nonbank SD is not conditioned upon a finding by the PRA of a violation of the UK PRA Capital Rules or UK PRA Financial Reporting Rules. In addition, as each PRA-designated UK nonbank SD is a member of NFA, the firm is subject to NFA membership rules, examination authority, and disciplinary process. [ 363 ]

The Commission did not receive comments directly related to its analysis set forth in the proposed Comparability Determination and Comparability Order, or on its preliminary determination that the PRA has the necessary powers to supervise, investigate, and discipline PRA-designated UK nonbank SDs for non-compliance with the applicable UK capital and financial reporting requirements. The Commission has reviewed its preliminary Comparability Determination and finds that the PRA-designated UK nonbank SDs are subject to a supervisory and enforcement framework that is comparable to the Commission's supervisory and enforcement framework for nonbank SDs. Start Printed Page 58567

As detailed in section II.F.1. above, PRA-designated UK nonbank SDs are subject to direct supervision by the PRA in its capacity of prudential regulator. The PRA has supervision, audit, and investigation powers with respect to the six PRA-designated UK nonbank SDs currently registered with the Commission.

The Commission's assessment of the PRA's supervisory programs included an evaluation of the PRA's authority to supervise PRA-designated UK nonbank SDs based on applicable UK laws and regulations, as discussed in section II.F.1. above. This evaluation included an assessment of the financial reporting that PRA-designated UK nonbank SDs are required to provide to the PRA, the PRA's ability to conduct examinations, including onsite inspections of PRA-designated UK nonbank SDs, and the PRA's ability to impose sanctions or take other action to address noncompliance with applicable laws and regulations. Based upon its evaluation, the Commission preliminarily determined that the relevant UK laws and regulations are comparable in purpose and effect to the CEA and Commission regulations, and that the PRA has appropriate power to supervise PRA-designated UK nonbank SDs for compliance with the UK PRA Capital Rules and UK PRA Financial Reporting Rules. The Commission further determined, based on applicable UK laws and regulations, that the PRA has the ability to sanction PRA-designated UK nonbank SDs for failing to comply with regulatory requirements. Specifically, as discussed in section II.F.1. above, the PRA has the power to impose sanctions on the PRA-designated UK nonbank SD if the firm's capital or liquidity fall below the applicable thresholds, [ 364 ] and may impose various requirements on PRA-designated UK nonbank SDs, including a requirement to hold additional capital if certain conditions are met. [ 365 ] The PRA may also withdraw a PRA-designated UK nonbank SD's authorization to operate if the firm no longer meets its minimum capital requirements. [ 366 ]

Furthermore, as discussed in this Comparability Determination, by issuing a Comparability Order, the Commission is not ceding its supervisory and enforcement authorities. PRA-designated UK nonbank SDs that are subject to a Comparability Order are registered with the Commission as SDs and are members of NFA, and, as such, are subject to the CEA, Commission regulations, and NFA membership rules and requirements. In this regard, PRA-designated UK nonbank SDs covered by a Comparability Order are required to directly provide the Commission with additional information upon the Commission's request to facilitate the ongoing supervision of such firms. [ 367 ] Further, section 17 of NFA's SD Financial Requirements rule provides that each SD member of NFA must file the financial, operational, risk management and other information required by NFA in the form and manner prescribed by NFA. [ 368 ] The ability to obtain information directly from PRA-designated UK nonbank SDs ensures that the Commission and NFA have access to the information necessary to monitor the financial condition of such firms and to assess the firms' compliance with applicable capital and financial reporting requirements. PRA-designated UK nonbank SDs covered by a Comparability Order remain subject to the Commission's examination and enforcement authority with respect to all elements of the CEA and Commission regulations, including capital and financial reporting. [ 369 ]

In addition, as detailed in section I.E. above, the conditions set forth in the Comparability Order reflect the fact that the Commission and NFA have a continuing obligation to conduct ongoing oversight, including potential examination, of PRA-designated UK nonbank SDs to ensure compliance with the Comparability Order and with relevant CEA requirements and Commission regulations. Specifically, the conditions require PRA-designated UK nonbank SDs to file directly with the Commission and NFA financial reports and notices that are comparable to the financial reports and notices filed by nonbank SDs domiciled in the U.S. In addition to requiring PRA-designated UK nonbank SDs to maintain current books and records reflecting all transactions, [ 370 ] the conditions further require each PRA-designated UK nonbank SD covered by the Comparability Order to file directly with the Commission and NFA: (i) monthly and annual financial reports;  [ 371 ] (ii) notice that the firm was informed by the PRA that it is not in compliance with the UK PRA Capital Rules and/or UK PRA Financial Reporting Rules;  [ 372 ] (iii) notice that the firm has experienced a decrease of 30 percent or more in its excess regulatory capital as compared to the last excess regulatory capital reported in filings with the Commission and NFA;  [ 373 ] (iv) notice that the firm has breached its combined capital buffer requirement and is required to file a capital conservation plan with the PRA;  [ 374 ] (v) notice that the firm has failed to maintain regulatory capital in the form of common equity tier 1 capital equal to or in excess of the U.S. dollar equivalent of $20 million;  [ 375 ] and (vi) notice that the firm has failed to maintain current financial books and records. [ 376 ] The Comparability Order further requires the Applicants to provide notice to the Commission of any material changes to the information submitted in the application, including, but not limited to, proposed and final material changes to the UK PRA Capital Rules or UK PRA Financial Reporting Rules and proposed and final material changes to the PRA's supervisory authority or supervisory regime over PRA-designated UK nonbank SDs. [ 377 ] The financial information and notices required to be filed directly with the Commission and NFA under the Comparability Order, and through the Commission's and NFA's direct authority to obtain additional information from PRA-designated UK nonbank SDs, will allow the Commission and NFA to conduct ongoing oversight of such firms to assess their overall safety and soundness.

Although Commission Regulation 23.106 does not condition the issuance of a Comparability Order on the Commission and the authority or authorities in the relevant foreign jurisdiction having entered into a formal MOU or similar arrangement, the Commission recognizes the benefit that such an arrangement may provide. Specifically, although Commission staff may engage directly with PRA-designated UK nonbank SDs to obtain information regarding their financial and operational condition, it may not be able to exchange and discuss such firm-specific information  [ 378 ] with the PRA or Start Printed Page 58568 reach shared expectations on procedures for conducting on-site examinations in the UK. [ 379 ] Therefore, Commission staff will continue its engagement with PRA staff to negotiate and finalize an MOU or similar arrangement to facilitate the joint supervision of PRA-designated UK nonbank SDs.

Based on the UK Application and the Commission's review of applicable UK laws and regulations, as well as the review of comments submitted in response to the Commission's request for comment on the UK Application and the proposed Comparability Determination and Comparability Order, the Commission finds that the UK PRA Capital Rules and the UK PRA Financial Reporting Rules, subject to the conditions set forth in the Comparability Order below, achieve comparable outcomes and are comparable in purpose and effect to the CFTC Capital Rules and CFTC Financial Reporting Rules. In reaching this conclusion, the Commission recognizes that there are certain differences between the UK PRA Capital Rules and CFTC Capital Rules and certain differences between the UK PRA Financial Reporting Rules and the CFTC Financial Reporting Rules. The Comparability Order is subject to conditions that are necessary to promote consistency in regulatory outcomes, or to reflect the scope of substituted compliance that would be available notwithstanding certain differences. In the Commission's view, the differences between the two rules sets are not inconsistent with providing a substituted compliance framework for PRA-designated UK nonbank SDs subject to the conditions specified in the Order below.

Furthermore, the Comparability Determination and Comparability Order are limited to the comparison of the UK PRA Capital Rules to the Bank-Based Approach contained within the CFTC Capital Rules. As noted previously, the Applicants have not requested, and the Commission has not performed, a comparison of the UK PRA Capital Rules to the Commission's NLA Approach or TNW Approach.

It is hereby determined and ordered, pursuant to Commodity Futures Trading Commission (“CFTC” or “Commission”) Regulation 23.106 ( 17 CFR 23.106 ) under the Commodity Exchange Act (“CEA”) ( 7 U.S.C. 1 et seq. ) that a swap dealer (“SD”) subject to the Commission's capital and financial reporting requirements under sections 4s(e) and (f) of the CEA ( 7 U.S.C. 6s(e) and (f) ), that is organized and domiciled in the United Kingdom (“UK”) and designated for prudential supervision by the UK Prudential Regulation Authority (“PRA”), may satisfy the capital requirements under section 4s(e) of the CEA and Commission Regulation 23.101(a)(1)(i) ( 17 CFR 23.101(a)(1)(i) ) (“CFTC Capital Rules”), and the financial reporting rules under section 4s(f) of the CEA and Commission Regulation 23.105 ( 17 CFR 23.105 ) (“CFTC Financial Reporting Rules”), by complying with certain specified requirements of the UK laws and regulations cited below and otherwise complying with the following conditions, as amended or superseded from time to time:

(1) The SD is not subject to regulation by a prudential regulator defined in section 1a(39) of the CEA ( 7 U.S.C. 1a(39) );

(2) The SD is organized under the laws of the UK and is domiciled in the UK;

(3) The SD is licensed as an investment firm in the UK and is designated for prudential supervision by the PRA (“PRA-designated UK nonbank SD”);

(4) The PRA-designated UK nonbank SD is subject to and complies with: Regulation (EU) No 575/2013 of the European Parliament and of the Council of 26 June 2013 on prudential requirements for credit institutions and amending Regulation (EU) No 648/2012 as restated and applicable in the UK (“UK CRR”), the provisions implementing the Directive 2013/36/EU of the European Parliament and of the Council of 26 June 2013 on access to the activity of credit institutions and the prudential supervision of credit institutions, amending Directive 2002/87/EC and repealing Directives 2006/48/EC and 2006/49/EC (“CRD”), including Capital Requirements Regulations 2013 and Capital Requirements (Capital Buffers and Macro-prudential Measures) Regulations 2014, Commission Delegated Regulation (EU) 2015/61 of 10 October 2014 to supplement Regulation (EU) No 575/2013 of the European Parliament and the Council with regard to liquidity coverage requirement for Credit Institutions (“Liquidity Coverage Delegated Regulation”), the provisions of the Banking Act 2009 and its secondary legislation related to the minimum requirement for own funds and eligible liabilities (“MREL”), and the rules of the PRA as reflected in the PRA Rulebook (collectively the “UK PRA Capital Rules”);

(5) The PRA-designated UK nonbank SD satisfies at all times applicable capital ratio and leverage ratio requirements set forth in Article 92 of UK CRR and the rules in PRA Rulebook, CRR Firms, Leverage Ratio—Capital Requirements and Buffers Part, Chapter 3 Minimum Leverage Ratio, the capital conservation buffer requirements set forth in PRA Rulebook, CRR Firms, Capital Buffers Part, and applicable liquidity requirements set forth in PRA Rulebook, CRR Firms, Liquidity Coverage Requirement—UK Designated Investment Firms Part and PRA Rulebook, CRR Firms, Liquidity (CRR) Part, and otherwise complies with the requirements to maintain a liquidity risk management program as required under PRA Rulebook, CRR Firms, Internal Liquidity Adequacy Assessment Part;

(6) The PRA-designated UK nonbank SD is subject to and complies with: Reporting (CRR) and Regulatory Reporting parts of the PRA Rulebook and the Companies Act 2006, Parts 15 and 16 (collectively and together with UK CRR, the “UK PRA Financial Reporting Rules”);

(7) The PRA-designated UK nonbank SD maintains at all times an amount of regulatory capital in the form of common equity tier 1 capital as defined in Article 26 of UK CRR, equal to or in excess of the equivalent of $20 million in United States dollars (“U.S. dollars”). The PRA-designated UK nonbank SD shall use a commercially reasonable and observable British pound/U.S. dollar exchange rate to convert the value of the pound-denominated common equity tier 1 capital to U.S. dollars;

(8) The PRA-designated UK nonbank SD has filed with the Commission a notice stating its intention to comply with the UK PRA Capital Rules and the UK PRA Financial Reporting Rules in lieu of the CFTC Capital Rules and the CFTC Financial Reporting Rules. The notice of intent must include the PRA-designated UK nonbank SD's representation that the firm is organized and domiciled in the UK, is a licensed investment firm designated for prudential supervision by the PRA, and Start Printed Page 58569 is subject to, and complies with, the UK PRA Capital Rules and UK PRA Financial Reporting Rules. A PRA-designated UK nonbank SD may not rely on this Comparability Order until it receives confirmation from Commission staff, acting pursuant to authority delegated by the Commission under Commission Regulation 140.91(a)(11) ( 17 CFR 140.91(a)(11) ), that the PRA-designated UK nonbank SD may comply with the UK PRA Capital Rules and UK PRA Financial Reporting Rules in lieu of the CFTC Capital Rules and CFTC Reporting Rules. Each notice filed pursuant to this condition must be submitted to the Commission via email to the following address: [email protected] ;

(9) The PRA-designated UK nonbank SD prepares and keeps current ledgers and other similar records in accordance with the PRA Rulebook, General Organisational Requirements Part, Rule 2.2 and Record Keeping Part, Rule 2.1 and 2.2, and conforming with the applicable accounting principles;

(10) The PRA-designated UK nonbank SD files with the Commission and with the National Futures Association (“NFA”) a copy of templates 1.1 (Balance Sheet Statement: assets), 1.2 (Balance Sheet Statement: liabilities), 1.3 (Balance Sheet Statement: equity), and 2 (Statement of profit or loss) of the financial reports (“FINREP”) that PRA-designated UK nonbank SDs are required to submit pursuant to PRA Rulebook, CRR Firms, Regulatory Reporting Part, Chapter 9 Regulatory Activity Group 3, Rule 9.2, and templates 1 (Own Funds), 2 (Own Funds Requirements) and 3 (Capital Ratios) of the common reports (“COREP”) that PRA-designated UK nonbank SDs are required to submit pursuant to PRA Rulebook, CRR Firms, Reporting (CRR) Part, Chapter 4 Reporting (Part Seven A CRR), Article 430 Reporting on Prudential Requirements and Financial Information, Rule 1. The FINREP and COREP templates must be provided with balances converted to U.S. dollars, using a commercially reasonable and observable British pound/U.S. dollar spot rate as of the date of the reports, and must be filed with the Commission and NFA within 35 calendar days of the end of each month. PRA-designated UK nonbank SDs that are registered as security-based swap dealers (“SBSDs”) with the U.S. Securities and Exchange Commission (“SEC”) must comply with this condition by filing with the Commission and NFA a copy of Form X-17A-5 (“FOCUS Report”) that the PRA-designated UK nonbank SD is required to file with the SEC or its designee pursuant to an order granting conditional substituted compliance with respect to Securities Exchange Act of 1934 Rule 18a-7. The copy of the FOCUS Report must be filed with the Commission and NFA within 35 calendar days after the end of each month in the manner, format and conditions specified by the SEC in Order Specifying the Manner and Format of Filing Unaudited Financial and Operational Information by Security-Based Swap Dealers and Major Security-Based Swap Participants that are not U.S. Persons and are Relying on Substituted Compliance with Respect to Rule 18a-7, 86 FR 59208 (Oct. 26, 2021);

(11) The PRA-designated UK nonbank SD files with the Commission and with NFA a copy of its annual audited accounts and strategic report (together, “annual audited financial report”) that are required to be prepared and published pursuant to Parts 15 and 16 of Companies Act 2006. The annual audited financial report may be reported in British pound. The annual audited financial report must be filed with the Commission and NFA on the earlier of the date the report is filed with the PRA or the date the report is required to be filed with the PRA pursuant to the UK PRA Financial Reporting Rules;

(12) The PRA-designated UK nonbank SD files Schedule 1 of appendix B to Subpart E of part 23 of the Commission's regulations ( 17 CFR 23 Subpart E —appendix B) with the Commission and NFA on a monthly basis. Schedule 1 must be prepared with balances reported in U.S. dollars, using a commercially reasonable and observable British pound/U.S. dollar spot rate as of the date of the report, and must be filed with the Commission and NFA within 35 calendar days of the end of each month. PRA-designated UK nonbank SDs that are registered as SBSDs must comply with this condition by filing with the Commission and NFA a copy of the FOCUS Report that they file with the SEC or its designee as set forth in Condition 10;

(13) The PRA-designated UK nonbank SD submits with each set of FINREP and COREP templates, annual audited financial report, and Schedule 1 of appendix B to Subpart E of part 23 of the Commission's regulations, a statement by an authorized representative or representatives of the PRA-designated UK nonbank SD that to the best knowledge and belief of the representative or representatives, the information contained in the reports, including the conversion of balances in the reports to U.S. dollars, is true and correct;

(14) The PRA-designated UK nonbank SD files a margin report containing the information specified in Commission Regulation 23.105(m) ( 17 CFR 23.105(m) ) (“Margin Report”) with the Commission and with NFA within 35 calendar days of the end of each month. The Margin Report's balances must be reported in U.S. dollars, using a commercially reasonable and observable British pound/U.S. dollar spot rate as of the date of the report;

(15) The PRA-designated UK nonbank SD files a notice with the Commission and NFA within 24 hours of being informed by the PRA that the firm is not in compliance with any component of the UK PRA Capital Rules or the UK PRA Financial Reporting Rules;

(16) The PRA-designated UK nonbank SD files a notice within 24 hours with the Commission and NFA if it fails to maintain regulatory capital in the form of common equity tier 1 capital as defined in Article 26 of UK CRR, equal to or in excess of the U.S. dollar equivalent of $20 million using a commercially reasonable and observable British pound/U.S. dollar exchange rate;

(17) The PRA-designated UK nonbank SD provides the Commission and NFA with notice within 24 hours of breaching its combined capital buffer requirement and being required to file a capital conservation plan with the PRA pursuant to PRA Rulebook, CRR Firms, Capital Buffers Part, Chapter 4 Capital Conservation Measures, Rule 4.4;

(18) The PRA-designated UK nonbank SD provides the Commission and NFA with notice within 24 hours if it is required by the PRA to maintain additional capital or additional liquidity requirements, or to restrict its business operations, or to comply with other requirements pursuant to Financial Services and Markets Act 2000, Part 4A or the Capital Requirements Regulation 2013, Regulation 35B;

(19) The PRA-designated UK nonbank SD files a notice with the Commission and NFA within 24 hours if it fails to maintain its MREL, if the PRA-designated UK nonbank SD is subject to such requirement as set forth by the Bank of England pursuant to the Banking Act 2009, section 3A and the Bank Recovery and Resolution (No. 2) Order 2014, Part 9;

(20) The PRA-designated UK nonbank SD files a notice with the Commission and NFA if it experiences a 30 percent or more decrease in its excess regulatory capital as compared to that last reported in the financial information filed pursuant to Condition 10. The notice filed with Commission and NFA must be filed within two business days of the firm experiencing the 30 percent or Start Printed Page 58570 more decrease in excess regulatory capital;

(21) The PRA-designated UK nonbank SD files a notice with the Commission and NFA within 24 hours if it fails to make or keep current the financial books and records;

(22) The PRA-designated UK nonbank SD files a notice with the Commission and NFA within 24 hours of the occurrence of any of the following: (i) a single counterparty, or group of counterparties under common ownership or control, fails to post required initial margin or pay required variation margin to the PRA-designated UK nonbank SD on uncleared swap and non-cleared security-based swap positions that, in the aggregate, exceeds 25 percent of the PRA-designated UK nonbank SD's minimum capital requirement; (ii) counterparties fail to post required initial margin or pay required variation margin to the PRA-designated UK nonbank SD for uncleared swap and non-cleared security-based swap positions that, in the aggregate, exceeds 50 percent of the PRA-designated UK nonbank SD's minimum capital requirement; (iii) the PRA-designated UK nonbank SD fails to post required initial margin or pay required variation margin for uncleared swap and non-cleared security-based swap positions to a single counterparty or group of counterparties under common ownership and control that, in the aggregate, exceeds 25 percent of the PRA-designated UK nonbank SD's minimum capital requirement; or (iv) the PRA-designated UK nonbank SD fails to post required initial margin or pay required variation margin to counterparties for uncleared swap and non-cleared security-based swap positions that, in the aggregate, exceeds 50 percent of the PRA-designated UK nonbank SD's minimum capital requirement. For purposes of the calculation, the PRA-designated UK nonbank SD's minimum capital requirement is the core capital requirement under the UK PRA Capital Rules, excluding capital buffers;

(23) The PRA-designated UK nonbank SD files a notice with the Commission and NFA of a change in its fiscal year-end approved or permitted to go into effect by the PRA. The notice required by this paragraph will satisfy the requirement for a nonbank SD to obtain the approval of NFA for a change in fiscal year-end under Commission Regulation 23.105(g) ( 17 CFR 23.105(g) ). The notice of change in fiscal year-end must be filed with the Commission and NFA at least 15 business days prior to the effective date of the PRA-designated UK nonbank SD's change in fiscal year-end;

(24) The PRA-designated UK nonbank SD or an entity acting on its behalf notifies the Commission of any material changes to the information submitted in the application for Comparability Determination, including, but not limited to, proposed and final material changes to the UK PRA Capital Rules or UK PRA Financial Reporting Rules and proposed and final material changes to the PRA's supervisory authority or supervisory regime over PRA-designated UK nonbank SDs; and

(25) Unless otherwise noted in the conditions above, the reports, notices, and other statements required to be filed by the PRA-designated UK nonbank SD with the Commission and NFA pursuant to the conditions of this Comparability Order must be submitted electronically to the Commission and NFA in accordance with instructions provided by the Commission or NFA.

It is also hereby determined and ordered that this Comparability Order becomes effective upon its publication in the Federal Register , with the exception of Conditions 14, 20, and 22, which will become effective 180 calendar days after publication of the Comparability Order in the Federal Register .

Issued in Washington, DC, on July 3, 2024, by the Commission.

Robert Sidman,

Deputy Secretary of the Commission.

The following appendices will not appear in the Code of Federal Regulations.

On this matter, Chairman Behnam and Commissioners Johnson, and Goldsmith Romero, Mersinger, and Pham voted in the affirmative. No Commissioner voted in the negative.

I support the Commission's approval of four comparability determinations and related orders finding that the capital and financial reporting requirements in Japan, Mexico, the European Union (France and Germany), and the United Kingdom (for swap dealers (SDs) designated for prudential supervision by the UK Prudential Regulation Authority (PRA)) are comparable to the Commission's capital and financial reporting requirements applicable to nonbank SDs. These are the first comparability determinations that the Commission has finalized for applications filed following the July 2020 adoption of its regulatory framework for substituted compliance for non-U.S. domiciled nonbank SDs. [ 1 ] There are currently 15 non-U.S. nonbank SDs that are eligible to comply with these conditional orders: three in Japan; three in Mexico; two in Germany and one in France for the EU; and six in the UK that are PRA-designated.

As part of the process leading to the Commission's final comparability determinations and orders, Commission staff engaged in a thorough analysis of each foreign jurisdictions' capital and financial reporting frameworks and considered the public comments received on the proposed determinations and orders. Based on those reviews, the Commission has determined that the respective foreign jurisdictions' rules are comparable in purpose and effect, and achieve comparable outcomes, to the CFTC's capital and financial reporting rules. Specifically, the Commission considered the scope and objectives of the foreign regulators' capital adequacy and financial reporting requirements; the ability of those regulators to supervise and enforce compliance with their respective capital and financial reporting requirements; and other facts or circumstances the Commission deemed relevant for each of the applications.

In certain instances, the Commission found that a foreign jurisdiction's rules impose stricter standards. In limited circumstances, where the Commission concluded that a foreign jurisdiction lacks comparable and comprehensive requirements on a specific issue, the Commission included a targeted condition designed to impose an equally stringent standard. The Commission has issued the final orders consistent with its authority to issue a comparability determination with the conditions it deems appropriate. These conditions aim to ensure that the orders only apply to nonbank SDs that are eligible for substituted compliance in these respective jurisdictions and that those non-U.S. nonbank SDs comply with the foreign country's capital and financial reporting requirements as well as certain additional capital, financial reporting, recordkeeping, and regulatory notice requirements. This approach acknowledges that jurisdictions may adopt unique approaches to achieving comparable outcomes. As a result, the Commission has focused on whether the applicable foreign jurisdiction's capital and financial reporting requirements achieve comparable outcomes to the corresponding Commission requirements for nonbank SDs, not whether they are comparable in every aspect or contain identical elements.

With these comparability determinations, the Commission fully retains its enforcement and examination authority as well as its Start Printed Page 58571 ability to obtain financial and event specific reporting to maintain direct oversight of nonbank SDs located in these four jurisdictions. The avoidance of duplicative requirements without a commensurate benefit to the Commission's oversight function reflects the Commission's approach to recognizing the global nature of the swap markets with dually-registered SDs that operate in multiple jurisdictions, which mandate prudent capital and financial reporting requirements. This is, however, an added benefit and not the Commission's sole justification for issuing these comparability determinations.

The comparability orders will become effective upon their publication in the Federal Register . For several order conditions, the Commission is granting an additional compliance period of 180 calendar days. To rely on a comparability order, an eligible non-U.S. nonbank SD must notify the Commission of its intention to satisfy the Commission's capital and financial requirements by substituted compliance and receive a Commission confirmation before relying on a determination.

I appreciate the hard work and dedication of the staff in the Market Participants Division over the past several years to propose and finalize these four determinations. I also thank the staff in the Office of the General Counsel and the Office of International Affairs for their support on these matters.

I support the Commodity Futures Trading Commission's (Commission or CFTC) issuance of four final capital and financial reporting comparability determinations and related orders (together, Final Comparability Determinations) for non-U.S. nonbank swap dealers (foreign nonbank SDs) and non-U.S. nonbank major swap participants (foreign nonbank MSPs) organized and domiciled in the United Kingdom (UK), the European Union (specifically, France and Germany), Mexico, and Japan. [ 1 ]

The Final Comparability Determinations allow eligible foreign nonbank SDs to satisfy certain capital and financial reporting requirements under the Commodity Exchange Act (CEA) and Commission regulations if they: (1) are subject to, and comply with, comparable capital and financial reporting requirements under the laws and regulations applicable in their home countries and (2) comply with the conditions enumerated in the applicable Final Comparability Determination. Under this conditional substituted compliance framework, foreign nonbank SDs in the relevant jurisdictions that comply with these conditions are deemed to be in compliance with the Commission's capital and financial reporting requirements.

Well-calibrated capital requirements create a cushion to absorb unexpected losses in times of market stress, and well-calibrated financial reporting requirements provide the Commission with information to monitor the business operations and financial condition of registered SDs. These tools are critical to managing systemic risk and fostering the stability of U.S. derivatives markets and the U.S. financial system. The Commission's substituted compliance framework addresses the need to promote sound global derivatives regulation while mitigating potentially duplicative cross-border regulatory requirements for non-U.S. market participants operating in our markets. Where the Commission permits substituted compliance, it must retain sufficient oversight, examination, and enforcement authority to ensure compliance with the foreign jurisdiction's laws and the conditions to substituted compliance.

Crucially, while these Final Comparability Determinations permit foreign nonbank SDs to comply with home country regulations in lieu of compliance with Commission regulations, the Commission is also imposing important guardrails to ensure continuous supervision of the operations and financial condition of the foreign SD.

For an example of the detrimental consequences of failing to adequately capitalize nonbank swap market participants, one need look no further than the 2008 global financial crisis. According to the U.S. Government Accountability Office, the crisis, which threatened the stability of the U.S. financial system and the health of the U.S. economy, may have led to $10 trillion in losses, including large declines in employment and household wealth, reduced tax revenues from lower economic activity, and lost economic output. [ 2 ] In response to the crisis, in 2010, the U.S. Congress passed the Dodd-Frank Wall Street Reform and Consumer Protection Act (the Dodd-Frank Act), which amended the CEA to create a new regulatory framework for swaps.

As amended, section 4s(e) of the CEA directs the Commission and prudential regulators to impose minimum capital requirements on SDs registered with the Commission. Section 4s(e) adopts separate approaches for the imposition of minimum capital requirements on bank and nonbank SDs. For bank SDs, prudential regulators are authorized to set the minimum capital requirements. For nonbank SDs, the Commission is authorized to set those requirements. The amended CEA also sets out financial reporting requirements for SDs. Under section 4s(f) of the CEA, registered SDs are required to make financial condition reports and other reports regarding transactions and positions as mandated by Commission regulations.

In 2020, the Commission adopted regulations implementing both the capital and financial reporting requirements for SDs, which were amended in 2024 (the Capital and Financial Reporting Rules). [ 3 ] The Capital and Financial Reporting Rules set minimum capital levels that nonbank SDs must maintain and financial reporting requirements that nonbank SDs must comply with, including filing periodic unaudited financial statements and an annual audited financial report. [ 4 ]

Like the U.S., many other nations adopted their own regulatory regimes to govern swaps markets in the aftermath of the financial crisis. Since then, regulators from around the world have endeavored to improve the resilience of swaps markets and establish a global set of standards on critical risk management issues, such as capital and financial reporting requirements. These efforts led to the development of the Principles for Financial Market Infrastructures, to which many jurisdictions, including our own, look for guidance. [ 5 ]

The Dodd-Frank Act amendments specifically address the cross-border application of the CFTC's swaps regime. Section 2(i) of the CEA establishes that the CEA's swaps provisions apply to foreign swaps activities that have a “direct and significant” connection to, or effect on, U.S. markets. In line with section 2(i) of the CEA, the Capital and Financial Reporting Rules set out a substituted compliance framework in Commission Regulation 23.106 for foreign nonbank SDs seeking to comply with the Commission's capital and financial reporting requirements.

The substituted compliance framework consists of comparability determinations that afford “due consideration [to] international comity principles” while being “consistent with . . . the Commission's interest in focusing its authority on potential significant risks to the U.S. financial system.”  [ 6 ] The determinations involve an assessment of the home-country requirements that is a principles-based, holistic approach, focusing on whether the applicable home-country requirements have comparable objectives and achieve comparable outcomes to the Commission's Capital and Financial Reporting Rules.

The Final Comparability Determinations will apply to 15 foreign nonbank SDs Start Printed Page 58572 currently registered with the Commission and subject to oversight by the UK Prudential Regulation Authority, the European Central Bank, the Mexican Comisión Nacional Bancaria y de Valores, and the Financial Services Agency of Japan. I commend staff for their hard work on the Final Comparability Determinations, including their work to thoroughly and thoughtfully analyze and address comments.

Importantly, while the Final Comparability Determinations permit foreign nonbank SDs in the relevant jurisdictions to comply with home country regulations in lieu of compliance with Commission regulations, there are numerous protections in place to ensure the Commission's ability to supervise on an ongoing basis the adequacy of the foreign nonbank SDs' compliance. The Final Comparability Determinations all include key conditions with which the foreign nonbank SDs must comply. For example, each of the Final Comparability Determinations requires that the foreign nonbank SDs provide monthly and annual financial reports to the Commission—and the Commission can request additional information as required to facilitate ongoing supervision. Each Final Comparability Determination also requires the foreign nonbank SDs to notify the Commission if adverse events occur, such as a significant decrease in excess regulatory capital, a significant failure of a counterparty to post required margin, or non-compliance with certain capital or financial reporting requirements. Finally, in recognition of the fact that a country's capital standards and financial reporting requirements may change over time, the Final Comparability Determinations require the foreign nonbank SDs to provide notice of material changes to the home country capital or financial reporting frameworks.

Moreover, the foreign nonbank SDs subject to these determinations are registered with the Commission and are members of the National Futures Association (NFA). Therefore, these entities are subject to the CEA, Commission regulations, and NFA membership rules, and each entity remains subject to Commission supervisory, examination and enforcement authority. As noted in the Final Comparability Determinations, if a foreign SD fails to comply with its home country's capital and financial reporting requirements, the Commission may initiate an action for a violation of the Commission's Capital and Financial Reporting Rules.

As I have previously noted, [ 7 ] it is important to recognize foreign market participants' compliance with the laws and regulations of their regulators when the requirements lead to an outcome that is comparable to the outcome of complying with the CFTC's corresponding requirements. Respect for partner regulators in foreign jurisdictions advances the Commission as a global standard setter for sound derivatives regulation and enhances market stability.

I thank the staff in the Market Participants Division for their hard work on these matters, particularly Amanda Olear, Tom Smith, and Lily Bozhanova.

I am pleased to support the order granting conditional substituted compliance in connection with certain capital and financial reporting requirements applicable to nonbank swap dealers subject to regulation by the United Kingdom Prudential Regulatory Authority (UK PRA) (UK Final Order). The UK Final Order, on balance, reflects an appropriate approach by the CFTC to collaboration with non-U.S. regulators that is consistent with IOSCO's 2020 report on Good Practices on Processes for Deference. [ 1 ]

I would like to thank Amanda Olear, Thomas Smith, Rafael Martinez, Liliya Bozhanova, Joo Hong, and Justin McPhee from the CFTC's Market Participants Division for their truly hard work on the UK Final Order and for addressing my concerns regarding the conditions for notice requirements. [ 2 ] I also thank the UK PRA for its assistance and support.

The CFTC's capital comparability determinations are the result of tireless efforts spanning over a decade since the global financial crisis. I commend the staff for working together with our regulatory counterparts around the world to promote regulatory cohesion and financial stability, and mitigate market fragmentation and systemic risk.

1.   7 U.S.C. 1 et seq. The CEA may be accessed through the Commission's website, www.cftc.gov .

2.   17 CFR Chapter I . Commission regulations may be accessed through the Commission's website, www.cftc.gov .

3.   Notice of Proposed Order and Request for Comment on an Application for Capital Comparability Determination Submitted on Behalf of Nonbank Swap Dealers Subject to Capital and Financial Reporting Requirements of the United Kingdom and Regulated by the United Kingdom Prudential Regulation Authority, 89 FR 8026 (Feb. 5, 2024) (“2024 Proposal”).

4.   7 U.S.C. 6s(e) .

5.  The term “prudential regulators” is defined in the CEA to mean the Board of Governors of the Federal Reserve System (“Federal Reserve Board”); the Office of the Comptroller of the Currency; the Federal Deposit Insurance Corporation; the Farm Credit Administration; and the Federal Housing Finance Agency. 7 U.S.C. 1a(39) .

6.  Subject to certain exceptions, the term “swap dealer” is generally defined as any person that: (i) holds itself out as a dealer in swaps; (ii) makes a market in swaps; (iii) regularly enters into swaps with counterparties as an ordinary course of business for its own account; or (iv) engages in any activity causing the person to be commonly known in the trade as a dealer or market maker in swaps. 7 U.S.C. 1a(49) .

The term “major swap participant” is generally defined as any person who is not an SD, and: (i) subject to certain exclusions, maintains a substantial position in swaps for any of the major swap categories as determined by the Commission; (ii) whose outstanding swaps create substantial counterparty exposure that could have serious adverse effects on the financial stability of the U.S. banking system or financial markets; or (iii) is a financial entity that: (a) is highly leveraged relative to the amount of capital it holds and that is not subject to capital requirements established by an appropriate Federal banking agency; and (b) maintains a substantial position in outstanding swaps in any major swap category as determined by the Commission. 7 U.S.C. 1a(33) .

7.   7 U.S.C. 6s(e)(2) .

8.   7 U.S.C. 6s(e)(1) and (2) .

9.   Margin and Capital Requirements for Covered Swap Entities, 80 FR 74840 (Nov. 30, 2015).

10.   Margin Requirements for Uncleared Swaps for Swap Dealers and Major Swap Participants, 81 FR 636 (Jan. 6, 2016).

11.   Capital Requirements of Swap Dealers and Major Swap Participants, 85 FR 57462 (Sept. 15, 2020). On April 30, 2024, the Commission amended the capital and financial reporting requirements to revise certain financial reporting obligations, among other changes. See Capital and Financial Reporting Requirements for Swap Dealers and Major Swap Participants, 89 FR 45569 (May 23, 2024). The amendments have limited impact on nonbank SDs covered by this order.

12.   7 U.S.C. 6s(f) .

13.   7 U.S.C. 6s(f)(1)(A) .

14.   85 FR 57462 .

15.   17 CFR 23.106 . Commission Regulation 23.106(a)(1) provides that a request for a Comparability Determination may be submitted by a non-U.S. nonbank SD or non-US nonbank MSP, a trade association or other similar group on behalf of its SD or MSP members, or a foreign regulatory authority that has direct supervisory authority over one or more non-US nonbank SDs or non-U.S. nonbank MSPs. However, Commission regulations also provide that any non-U.S. nonbank SD or non-U.S. nonbank MSP that is dually-registered with the Commission as a futures commission merchant (“FCM”) is subject to the capital requirements of Commission Regulation 1.17 ( 17 CFR 1.17 ) and may not petition the Commission for a Comparability Determination. 17 CFR 23.101(a)(5) and (b)(4) , respectively. Furthermore, substituted compliance is not available to non-U.S. bank SDs and non-U.S. bank MSPs with respect to their respective financial reporting requirements under Commission Regulation 23.105(p). Commission Regulation 23.105(p), however, permits non-U.S. bank SDs and non-U.S. bank MSPs that do not submit financial reports to a U.S. prudential regulator to file with the Commission a statement of financial condition, certain regulatory capital information, and Schedule 1 of Appendix C to Subpart E of Part 23 of the Commission's regulations prepared and presented in accordance with the accounting standards permitted by the non-U.S. bank SD's or non-U.S. bank MSP's home country regulatory authorities. 17 CFR 23.105(p)(2) .

16.   17 CFR 23.106(a)(3) .

17.   17 CFR 23.106(a)(3)(ii) . See also 85 FR 57462 at 57521.

18.   85 FR 57462 at 57521.

19.   17 CFR 23.106(a)(2) .

20.  The Commission provides the applicant with an opportunity to review for accuracy and completeness the Commission's description of relevant home country laws and regulations on which a proposed Comparability Determination and a proposed Comparability Order are based. The Commission relies on this review, and any corrections or feedback received, as part of the comparability assessment. A Comparability Determination and Comparability Order based on an inaccurate description of foreign laws and regulations may not be valid.

21.   17 CFR 23.106(a)(3) and 85 FR 57462 at 57520-57522.

22.  The Commission would conduct a similar analysis, adjusted as appropriate to account for regulatory distinctions, in performing a comparability assessment for foreign nonbank MSPs. Commission Regulation 23.101(b) requires a nonbank MSP to maintain positive tangible net worth. There are no MSPs currently registered with the Commission. 17 CFR 23.101(b) .

23.   17 CFR 23.106(a)(5) .

24.   17 CFR 23.106(a)(4)(i) .

25.  Notices must be filed in electronic form to the following email address: [email protected] .

26.   17 CFR 23.106(a)(4)(ii) and 17 CFR 140.91(a)(11) .

27.   17 CFR 23.106(a)(4)(ii) . Confirmation will be issued by MPD under authority delegated by the Commission. Commission Regulation 140.91(a)(11). 17 CFR 140.91(a)(11) .

28.   17 CFR 23.106(a)(4)(ii) .

29.   Id.

30.  Letter dated May 4, 2021 from Stephanie Webster, General Counsel, IIB, Steven Kennedy, Global Head of Public Policy, ISDA, and Kyle Brandon, Managing Director, Head of Derivatives Policy, SIFMA. The UK Application is available on the Commission's website at: https://www.cftc.gov/​LawRegulation/​DoddFrankAct/​CDSCP/​index.htm .

31.  As discussed in Section I.A. immediately below, the Commission has the authority to impose capital requirements on registered SDs that are not subject to regulation by a U.S. prudential regulator ( i.e., nonbank SDs).

32.  The Applicants also requested that the Commission determine that nonbank SDs licensed as investment firms and prudentially regulated by the UK Financial Conduct Authority (“FCA”) (“FCA-regulated UK nonbank SDs”) may satisfy certain capital and financial reporting requirements under the CEA by being subject to, and complying with, comparable capital and financial reporting requirements under UK laws and regulations. Due to the differences between the capital and financial reporting regimes applicable to PRA-designated UK nonbank SD and FCA-regulated UK nonbank SDs, the Commission anticipates assessing the comparability of the rules applicable to FCA-regulated UK nonbank SDs through a separate comparability determination.

33.  Article 3(1) and (2) of The Financial Services and Markets Act 2000 (PRA-regulated Activities) Order 2013.

34.   Id., Article 3(4).

35.   Id., Article 3(6).

36.  PRA, Statement of Policy, Designation of Investment Firms for Prudential Supervision by the Prudential Regulation Authority, December 2021, available here: https://www.bankofengland.co.uk/​-/​media/​boe/​files/​prudential-regulation/​statement-of-policy/​2021/​designation-of-investment-firms-for-prudential-supervision-by-the-pra-december-2021.pdf?​la=​en&​hash=​007EB17EDF2FA84714D372095F9E03627355776F .

37.   Id., at p. 5.

38.   Regulation (EU) No 575/2013 of the European Parliament and of the Council of 26 June 2013 on prudential requirements for credit institutions and amending Regulation (EU) No 648/2012 (“Capital Requirements Regulation” or “CRR”).

39.   Directive 2013/36/EU of the European Parliament and of the Council of 26 June 2013 on access to the activity of credit institutions and the prudential supervision of credit institutions, amending Directive 2002/87/EC and repealing Directives 2006/48/EC and 2006/49/EC (“Capital Requirements Directive” or “CRD”).

40.  The term “credit institution” is defined as an entity whose business consists of taking deposits and other repayable funds from the public and granting credits. CRR, Article 4(1), as applicable in the UK. For a reference to CRR provisions applicable in the UK, see infra note 50.

41.  The term “investment firm” is defined as an entity authorized under Directive 2014/65/EU of the European Parliament and of the Council of 15 May 2014 on markets in financial instruments and amending Directive 2002/92/EC and Directive 2011/61/EU (“Markets in Financial Instruments Directive” or “MiFID”), and whose regular business is the provision of one or more investment services to third parties and/or the performance of one or more investment-related activities on a professional basis, which includes dealing in derivatives for its own account. CRR, Article 4(1)(2) cross-referencing Article 4(1)(1) of MiFID.

42.   Consolidated Version of the Treaty on the Functioning of the European Union, OJ (C 326) 171, Oct. 26, 2012 (“TFEU”), Article 288.

43.   Id., Article 288 (stating that a directive is binding as to the result to be achieved upon each EU Member State to which the directive is addressed, and further provides, however, that each EU Member State elects the form and method of implementing the directive). In this connection, EU Member States were required to implement and start applying amendments to CRD, introduced by Directive (EU) 2019/878 of the European Parliament and of the Council of 20 May 2019 amending Directive 2013/36/EU as regards exempted entities, financial holding companies, mixed financial holding companies, remuneration, supervisory measures and powers and capital conservation measures (“CRD V”) by December 29, 2020. Some CRD V provisions were subject to delayed implementation deadlines of June 28, 2021 and January 1, 2022. CRD V, Article 2.

44.   Capital Requirements Regulations 2013, Statutory Instrument 2013 No. 3115 (“Capital Requirements Regulations 2013”).

45.   Capital Requirements (Capital Buffers and Macro-prudential Measures) Regulations 2014, Statutory Instrument 2014 No. 894 (“Capital Requirements (Capital Buffers and Macro-prudential Measures) Regulations 2014”).

46.  The PRA's rules (“PRA Rulebook”) are available here: https://www.prarulebook.co.uk/​ .

47.   See, An Act to Repeal the European Communities Act 1972 and make other provisions in connection with the withdrawal of the United Kingdom from the EU (2018 c.16) (“European Union (Withdrawal) Act 2018”).

48.  PRA, Policy Statement 21/21—The UK Leverage Framework, October 2021, available here: https://www.bankofengland.co.uk/​prudential-regulation/​publication/​2021/​june/​changes-to-the-uk-leverage-ratio-framework , and Policy Statement 22/21—Implementation of Basel standards: Final rules, October 2021, available here: https://www.bankofengland.co.uk/​prudential-regulation/​publication/​2021/​october/​implementation-of-basel-standards .

49.  Pursuant to the Financial Services and Markets Act 2023 (“FSMA 2023”), the UK revoked CRR and replaced it with: (i) PRA rules adopted under Section 144 of the Financial Services and Markets Act 2000 (“FSMA”) and (ii) UK regulations, adopted under Section 4 of FSMA 2023, restating CRR provisions.

50.  The UK CRR is available here: https://www.legislation.gov.uk/​eur/​2013/​575/​contents . The provisions that were incorporated in the PRA Rulebook are no longer part of UK CRR and appear instead in the PRA Rulebook.

51.   Commission Delegated Regulation (EU) 2015/61 of 10 October 2014 to supplement Regulation (EU) No 575/2013 of the European Parliament and the Council with regard to liquidity coverage requirement for Credit Institutions (“Liquidity Coverage Delegated Regulation”).

52.  PRA Rulebook, CRR Firms, Liquidity Coverage Requirement—UK Designated Investment Firms Part.

53.   Directive 2014/59/EU of the European Parliament and of the Council of 15 May 2014 establishing a framework for the recovery and resolution of credit institutions and investment firms and amending Council Directive 82/891/EEC, and Directives 2001/24/EC, 2002/47/EC, 2004/25/EC, 2005/56/EC, 2007/36/EC, 2011/35/EU, 2012/30/EU and 2013/36/EU, and Regulations (EU) No 1093/2010 and (EU) No 648/2012, of the European Parliament and of the Council. UK Application, p. 7.

54.  Banking Act 2009, Section 3A (4) and (4B); Bank Recovery and Resolution (No 2) Order 2014, Statutory Instrument No. 3348 (“Bank Recovery and Resolution (No 2) Order 2014”), Part 9.

55.   Commission Implementing Regulation (EU) 680/2014 of 16 April 2014 laying down implementing technical standards with regard to supervisory reporting of institutions according to Regulation (EU) No 575/2013 of the European Parliament and of the Council.

56.  UK Application, p. 24 and Responses to Staff Questions dated October 5, 2023.

57.  PRA Rulebook, CRR Firms, Reporting (CRR) Part.

58.  PRA Rulebook, CRR Firms, Regulatory Reporting Part.

59.  UK Application, p.7. Companies Act 2006, Part 15 and 16. The Companies Act 2006 is available here: https://www.legislation.gov.uk/​ukpga/​2006/​46/​contents .

60.  All six of the PRA-designated UK nonbank SDs currently registered with the Commission are also UK nonbank SBSDs.

61.  Section 15F(e)(1)(B) of the Exchange Act ( 15 U.S.C. 78o-10 ) directs the SEC to adopt capital rules for security-based swap dealers (“SBSDs”) that do not have a prudential regulator.

62.   Order Granting Conditional Substituted Compliance in Connection with Certain Requirements Applicable to Non-U.S. Security-Based Swap Dealers and Major Security-Based Swap Participants Subject to Regulation in the United Kingdom, 86 FR 43318 (July 30, 2021) (“Final UK Order”); Amended and Restated Order Granting Conditional Substituted Compliance in Connection with Certain Requirements Applicable to Non-U.S. Security-Based Swap Dealers and Major Security-Based Swap Participants Subject to Regulation in the Federal Republic of Germany; Amended Orders Addressing Non-U.S. Security-Based Swap Entities Subject to Regulation in the French Republic or the United Kingdom; and Order Extending the Time to Meet Certain Conditions Relating to Capital and Margin, 86 FR 59797 (Oct. 28, 2021) (“Amended UK Order,” together with the Final UK Order, “UK Order”); and Order Specifying the Manner and Format of Filing Unaudited Financial and Operational Information by Security-Based Swap Dealers and Major Security-Based Swap Participants that are not U.S. Persons and are Relying on Substituted Compliance with Respect to Rule 18a-7, 86 FR 59208 (Oct. 26, 2021) (“SEC Order on Manner and Format of Filing Unaudited Financial and Operational Information”).

63.  The conditioning of the UK substituted compliance order on UK nonbank SBSDs maintaining liquid assets in an amount that exceeds the UK nonbank SBSD's total liabilities by at least $100 million and by at least $20 million after applying certain deductions to the value of the liquid assets reflects that the SEC's capital rule for nonbank SBSDs is a liquidity-based requirement and that the SEC capital requirements are not based on the Basel standards. 17 CFR 240.18a-1(a)(1) (requiring a SBSD to maintain, in relevant part, net capital of $20 million or, if approved to use capital models, $100 million of tentative net capital and $20 million of net capital).

64.  2024 Proposal, 89 FR 8026 (Feb. 5, 2024).

65.   Id. Consistent with the process specified in Section I.B. above for conducting Comparability Determinations, the Commission provided the Applicants with an opportunity to review for factual accuracy and completeness the Commission's description of relevant UK laws and regulations on which the proposed Comparability Determination and proposed Comparability Order were based. The Commission has relied on the Applicants' review, and has incorporated feedback and corrections received from the Applicants. As previously noted, a Comparability Determination and Comparability Order based on an inaccurate description of foreign laws and regulations may not be valid.

66.   See 2024 Proposal at 8058-8061.

67.  NFA is a registered futures association (“RFA”) under Section 17 of the CEA ( 7 U.S.C. 21 ). Each SD registered with the Commission is required to be an NFA member. 17 CFR 170.16 . NFA, as an RFA, is also required by the CEA to adopt rules imposing minimum capital, segregation, and other financial requirements, as applicable, to its members, including SDs, that are at least as stringent as the Commission's minimum capital, segregation, and other financial requirements for such registrants, and to implement a program to audit and enforce such requirements. 7 U.S.C. 21(p) . Therefore, the Commission's proposed Comparability Order required PRA-designated UK nonbank SDs to file certain financial reports and notices with NFA so that it may perform oversight of such firms as required under Section 17 of the CEA. The Commission will refer to NFA in this Comparability Determination when referring to the requirements or obligations of an RFA.

68.   Id.

69.   Id. As described in the 2024 Proposal, the CFTC Capital Rules provide nonbank SDs with three alternative capital approaches: (i) the Tangible Net Worth Capital Approach (“TNW Approach”); (ii) the Net Liquid Assets Capital Approach (“NLA Approach”); and (iii) the Bank-Based Approach. See 2024 Proposal at 8031-8033, and 17 CFR 23.101 . The Bank-Based Approach is consistent with the Basel Committee on Banking Supervision's (“BCBS”) international framework for bank capital requirements (“BCBS framework” or “Basel standards”). The BCBS is the primary global standard-setter for the prudential regulation of banks and provides a forum for cooperation on banking supervisory matters. Institutions represented on the BCBS include the Federal Reserve Board, the European Central Bank, Deutsche Bundesbank, Bank of England, Bank of France, Bank of Japan, Banco de Mexico, and Bank of Canada. The BCBS framework is available at https://www.bis.org/​basel_​framework/​index.htm .

70.   See 2024 Proposal at 8035-8036.

71.  Letters from: Michael Ravnitzky (“Ravnitzky Letter”); Dennis M. Kelleher, Co-founder, President and CEO, and Cantrell Dumas, Director of Derivatives Policy, Better Markets (March 24, 2024) (“Better Markets Letter”); and Stephanie Webster, General Counsel, IIB, Steven Kennedy, Global Head of Public Policy, ISDA, and Kyle L. Brandon, Managing Director, Head of Derivatives Policy, SIFMA (March 24, 2024) (“Applicants' Letter”); Letter from William J. Harrington dated March 24, 2024 (“Harrington 03/24/2024 Letter”) and supporting material. The comment letters and related documents for the 2024 Proposal are available at: https://comments.cftc.gov/​PublicComments/​CommentList.aspx?​id=​7478 .

72.  Applicants' Letter at p. 2.

73.   Id. at p. 4.

74.   Id.

75.  Better Markets Letter at p. 3-5; Harrington 03/24/2024 Letter at p. 4 (asserting, as further discussed below, that the Commission should condition the Comparability Determination on a prohibition against PRA-designated UK nonbank SDs' entering into swap contracts with certain specified features).

76.  Better Markets Letter at p. 5.

77.   Id.

78.   See 2024 Proposal at 8036.

79.   85 FR 57462 at 57521.

80.   Interpretative Guidance and Policy Statement Regarding Compliance with Certain Swap Regulations, 78 FR 45292 (July 26, 2013) (“Guidance”).

81.  Guidance at 45343.

82.   Id.

83.   Id.

84.   See e.g., Comparability Determination for the European Union: Certain Entity-Level Requirements, 78 FR 78923 (December 27, 2013) at 78926.

85.   Margin Requirements for Uncleared Swaps for Swap Dealers and Major Swap Participants—Cross-Border Application of the Margin Requirements, 81 FR 34817 , 34836-34837 (May 31, 2016).

86.   Id.

87.   Id.

88.  Staff also reviewed various documents relevant to the proposed Comparability Determination and proposed Comparability Order published by the PRA.

89.  2024 Proposal at 8036-8058.

90.   Id. at 8057-8058.

91.  For a further discussion of the Commission's assessment of the PRA's supervision and enforcement powers, see Section II.F. below. In addition, in its policy statement discussing the forthcoming implementation of Basel 3.1 standards, the PRA noted that despite some adjustments to the international standards, the PRA considers that its policy and rules proposals align with the international framework. In this regard, the PRA expressed the view that alignment with international standards in turn supports the UK's competitiveness, including relative standing of the UK as a global financial center, by “strengthening key stakeholders' confidence in the UK banking system” and “assuring regulators in other jurisdictions of UK's authorities' commitment to robust standards.” See PRA, PS17/23—Implementation of the Basel 3.1 Standards Near-Final Part 1, December 12, 2023, available here: https://www.bankofengland.co.uk/​news/​2023/​december/​pra-publishes-first-of-two-policy-statements-for-basel-3-1-standards-implementation .

92.  Better Markets Letter at p. 5.

93.   Id.

94.   Id. at p. 4.

95.  Ravnitzky Letter at p. 6.

96.   Id.

97.   17 CFR 23.106(a)(5) , which provides that in issuing a Capital Comparability Determination, the Commission may impose any terms and conditions it deems appropriate, including certain capital adequacy and financial reporting requirements on swap dealers . . . (Emphasis added). Commission Regulation 23.106(a)(3) establishes the Commission's standard of review for performing a Comparability Determination and provides that the Commission may consider all relevant factors, including whether the relevant foreign jurisdiction's capital adequacy and financial reporting requirements achieve comparable outcomes to the Commission's corresponding capital adequacy and financial reporting requirements for SDs. 17 CFR 23.106(a)(3)(ii) .

98.   85 FR 57462 at 57520. See also Guidance at 45342-45344 and Comparability Determination for the European Union: Certain Transaction Level Requirements, 78 FR 78878 (December 27, 2013) at 78880.

99.   Comparability Determination for the European Union: Certain Transaction Level Requirements, 78 FR 78878 (December 27, 2013) at 78880.

100.  Guidance at 45343.

101.  The six criteria provide that the PRA-designated UK nonbank SD: (i) is not subject to capital rules of a U.S. prudential regulator (Condition 1); (ii) is organized and domiciled in the UK (Condition 2); (iii) is licensed as an investment firm and designated for prudential supervision by the PRA (Condition 3); (iv) is subject to the UK CRR, CRD provisions as implemented in the UK, the Liquidity Coverage Delegated Regulation, the provisions of the Banking Act 2009 and its secondary legislation related to the MREL, and the rules of the PRA as reflected in the PRA Rulebook (Condition 4); (v) satisfies at all times applicable UK CRR and PRA Rulebook capital ratios, leverage ratios, and capital conservation buffer ratios, and maintains a liquidity risk management program as required under the PRA Rulebook (Condition 5); and (vi) is subject to and complies with the UK financial reporting requirements that are part of the Commission's comparability assessment (Condition 6).

102.  The ten conditions require a PRA-designated UK nonbank SD to provide notice to the Commission in the event that the firm: (i) is informed by the PRA that the firm has failed to comply with any component of the UK PRA Capital Rules or UK PRA Financial Reporting Rules (Condition 15); (ii) fails to maintain common equity tier 1 capital denominated in GBP in an equivalent amount of at least $20 million (Condition 16); (iii) breaches its combined capital buffer requirement and is required to file a capital conservation plan with the PRA (Condition 17); (iv) is required by the PRA to maintain additional capital or additional liquidity (Condition 18); (v) fails to meet the required MREL (Condition 19); (vi) experiences a 30 percent or more decrease in its excess regulatory capital (Condition 20); (vii) fails to make or keep current financial books and records (Condition 21); (viii) fails to post or collect margin for uncleared swaps and non-cleared security-based swaps with one or more counterparties in amounts that exceed defined limits (Condition 22); (ix) changes its fiscal year-end date (Condition 23); and (x) is subject to material changes to the UK PRA Capital Rules, UK PRA Financial Reporting Rules, or the supervisory authority of the PRA (Condition 24).

103.  The two conditions provide that a PRA-designated UK nonbank SD must file with the Commission and NFA: (i) a copy of SEC Form X-17A-5 (“FOCUS Report”) that the PRA-designated UK nonbank SD files with the SEC or copies of certain financial reporting templates that the PRA-designated UK nonbank SD is required to submit to the PRA pursuant to PRA Rulebook rules, as applicable (Condition 10), and (ii) copies of its annual audited accounts and strategic report that are required to be prepared and published pursuant to Parts 15 and 16 of Companies Act 2006 (Condition 11).

104.  One of the administrative conditions provides that a PRA-designated UK nonbank SD must provide a notice to the Commission of its intent to comply with the Comparability Order and the UK PRA Capital Rules and UK PRA Financial Reporting Rules in lieu of the CFTC Capital Rules and CFTC Financial Reporting Rules. The notice must include the PRA-designated UK nonbank SD's representation that the firm is organized and domiciled in the UK, is a licensed investment firm designated for prudential supervision by the PRA, and is subject to and complies with the UK PRA Capital Rules and the UK PRA Financial Reporting Rules (Condition 8). The second administrative condition provides that a PRA-designated UK nonbank SD must file any documents with the Commission and NFA via electronic transmission (Condition 25).

105.  As the Commission stated in the 2024 Proposal, a non-U.S. nonbank SD that operates under a Comparability Order issued by the Commission remains subject to the Commission's examination and enforcement authority. Specifically, the Commission may initiate an enforcement action against a non-U.S. nonbank SD that fails to comply with its home-country capital adequacy and/or financial reporting requirements cited in a Comparability Order. See 2024 Proposal at 8029. See also 17 CFR 23.106(a)(4)(ii) , which provides that the Commission may examine all nonbank SDs, regardless of whether the nonbank SDs rely on substituted compliance, and that the Commission may initiate an enforcement action under the Commission's capital and financial reporting regulations against a non-U.S. nonbank SD that fails to comply with a foreign jurisdiction's capital adequacy and financial reporting requirements.

106.  Guidance at 45343.

107.   17 CFR 23.106(a)(5) .

108.  Better Markets at p. 10.

109.   Id.

110.   Id.

111.   Id. at p. 11.

112.  Condition 24 of the final Comparability Order requires a PRA-designated UK nonbank SD, or an entity acting on its behalf, to notify the Commission of any material changes to the information submitted in its application, including, but not limited to, proposed and final material changes to the UK PRA Capital Rules or UK PRA Financial Reporting Rules and proposed and final material changes to the PRA's supervisory authority or supervisory regime over PRA-designated UK nonbank SDs. The Commission notes that it made certain non-substantive, clarifying changes to the language of final Condition 24 as compared to proposed Condition 24.

113.  2024 Proposal at 8036 (n. 128).

114.  Harrington 03/24/2024 Letter at p. 4. Harrington also referenced the following two separate submissions to the Commission and noted that these submissions support the Harrington 03/24/2024 Letter: a letter dated October 20, 2022 (“Harrington 10/20/2022 Letter”), submitted in connection with the Commission's Notice of Proposed Order and Request for Comment on an Application for a Capital Comparability Determination From the Financial Services Agency of Japan, 87 FR 48092 , (August 8, 2022) and a letter dated August 28, 2023 (“Harrington 08/28/2023 Letter”), submitted in connection with the Commission's Notice of Proposed Order and Request for Comment on an Application for a Capital Comparability Determination Submitted on Behalf of Nonbank Swap Dealers Domiciled in the French Republic and Federal Republic of Germany and Subject to Capital and Financial Reporting Requirements of the European Union, 88 FR 41774 (June 27, 2023). Harrington 03/24/2024 Letter at p.7.

115.  William J. Harrington, Submission to the U.S. Securities and Exchange Commission Re: File No. S7-08-12 (Nov. 19, 2018) at p.8.

116.  For additional information on the legal mechanics of a flip clause, see Lehman Brothers Special Financing Inc v. Bank of America N.A., No. 18-1079 (2nd Cir. 2020).

117.  Harrington 03/24/2024 Letter at p. 8.

118.  Harrington 03/24/2024 Letter at p. 21 (noting that “[the CFTC margin requirements] render the flip-clause-contract commercially impracticable in the U.S.” and that “U.S. swap margin rules, including the CFTC swap margin rule, have greatly benefited U.S. persons by subduing financial sector credit exposures that might otherwise draw bailouts or other U.S. government support”).

119.  Harrington 03/24/2024 Letter at p. 25 (arguing that “U.K. and other non-U.S. swap margin and capital rules perpetuate the flip-clause-swap-contract by allowing [asset-backed securities] issuers, other structured debt issuers, banks, and swap dealers to under-resource their [respective] contract exposures via both exemptions from margin posting and see-no-evil capital rules that treat the contract as ` plain vanilla'.” )

120.   12 CFR 217.34 and 12 CFR 217.132 (indicating that nonbank SDs may recognize the risk-mitigating effects of financial collateral for collateralized derivatives contracts) and PRA Rulebook, CRR Firms, Counterparty Credit Risk Part, Article 276 and UK CRR, Article 285 (setting forth rules for the recognition and treatment of collateral in calculating the PRA-designated UK nonbank SD's counterparty credit risk exposure).

121.  Ravnitzky Letter at p. 4.

122.   See 2024 Proposal at 8036.

123.  Condition 24 of the final Comparability Order. The Commission notes that it made certain non-substantive, clarifying changes to the language of final Condition 24 as compared to proposed Condition 24.

124.   See 2024 Proposal at 8036. As stated in the 2024 Proposal, the Commission may also amend or supplement the final Comparability Order to address any material changes to the CFTC Capital Rules and CFTC Financial Reporting Rules, including rule amendments to capital rules of the Federal Reserve Board that are incorporated into the CFTC Capital Rules' Bank-Based Approach under Commission Regulation 23.101(a)(1)(i), that are adopted after the final Comparability Order is issued. See id., (n. 128). As noted in the 2024 Proposal, the Commission is aware that the PRA is considering changes to the UK PRA Capital Rules to implement Basel 3.1 standards. See PRA, PS17/23—Implementation of the Basel 3.1 Standards Near-Final Part 1, December 12, 2023, available here: https://www.bankofengland.co.uk/​news/​2023/​december/​pra-publishes-first-of-two-policy-statements-for-basel-3-1-standards-implementation . If the PRA proceeds with the implementation of the Basel 3.1 standards as proposed, the regulatory changes would be applicable after July 1, 2025 with a 4.5-year transitional period ending on January 1, 2030. The Commission will monitor progress on the PRA's proposed regulatory changes and may amend or supplement the Comparability Order. As noted, the Commission requires notification of any material changes to the UK PRA Capital Rules, including any Basel 3.1 implementing provisions.

125.   See 2024 Proposal at 8037.

126.  The BCBS's mandate is to strengthen the regulation, supervision, and practices of banks with the purpose of enhancing financial stability. See Basel Committee Charter available on the Bank for International Settlement website: www.bis.org/​bcbs/​charter.htm . See 2024 Proposal at 8037.

127.  2024 Proposal at 8039-8047.

128.   Id.

129.  2024 Proposal at 8039.

130.   Id.

131.   Id. at 8037.

132.   Id.

133.   Id.

134.   Id.

135.  Better Markets Letter at p. 15.

136.   Id. at p. 11.

137.   Id. at p. 16.

138.   Id. at p. 11.

139.   See 2024 Proposal at 8039.

140.   Id.

141.   Id.

142.   17 CFR 23.101(a)(1)(i) and 2024 Proposal at 8037-8038. The terms “common equity tier 1 capital,” “additional tier 1 capital,” and “tier 2 capital” are defined in the bank holding company regulations of the Federal Reserve Board. 12 CFR 217.20 .

143.   12 CFR 217.20(b) .

144.   12 CFR 217.20(c) .

145.   12 CFR 217.20(d) .

146.  Subordinated debt must meet requirements set forth in SEC Rule 18a-1d. Specifically, subordinated debt instruments must have a term of at least one year (with the exception of approved revolving subordinated debt agreements which may have a maturity term that is less than one year), and contain terms that effectively subordinate the rights of lenders to receive any payments, including accrued interest, to other creditors of the firm. 17 CFR 23.101(a)(1)(i)(B) and 17 CFR 240.18a-1d .

147.  2024 Proposal at 8038.

148.   Id. and UK CRR, Articles 26 and 28.

149.   Id.

150.   Id. and UK CRR, Articles 51-52.

151.   Id. and UK CRR, Article 63.

152.   See 2024 Proposal at 8039.

153.   Id.

154.   Id.

155.   Id.

156.   Id.

157.   17 CFR 23.101(a)(1)(i) . See also 2024 Proposal at 8039. The term “uncleared swap margin” is defined in Commission Regulation 23.100 to generally mean the amount of initial margin that a nonbank SD would be required to collect from each counterparty for each outstanding swap position of the nonbank SD. 17 CFR 23.100 . A nonbank SD must include all swap positions in the calculation of the uncleared swap margin amount, including swaps that are exempt or excluded from the scope of the Commission's uncleared swap margin regulations. A nonbank SD must compute the uncleared swap margin amount in accordance with the Commission's margin rules for uncleared swaps. 17 CFR 23.154 .

158.   17 CFR 23.101(a)(1)(i)(D) . See also 2024 Proposal at 8039. Commission Regulation 23.101(a)(1)(i)(D) sets forth one of the minimum thresholds that a nonbank SD must meet as the “the amount of capital required by a registered futures association.” As previously noted, NFA is currently the only entity that is registered with the Commission as a futures association. NFA has adopted the Commission's capital requirements as its own requirements, and has not adopted any additional or stricter minimum capital requirements. See NFA rulebook, Financial Requirements Section 18 Swap Dealer and Major Swap Participant Financial Requirements, available at nfa.futures.org.

159.   See 2024 Proposal at 8041-8042.

160.   Id. at 8042.

161.   Id.

162.   Id. at 8045.

163.   85 FR 57462 at 57492.

164.  2024 Proposal at 8045.

165.   Id. The Commission also noted that the six current PRA-designated UK nonbank SDs maintain common equity tier 1 capital in amounts in excess of the equivalent of $20 million based on financial filings made with the Commission. Id. (note 255).

166.  Better Markets Letter at p. 13.

167.   Id.

168.  Guidance at 45343.

169.   17 CFR 23.101(a)(1)(i)(B) .

170.   See generally 85 FR 57462 at 57530.

171.   See 2024 Proposal at 8046.

172.   Id. and UK CRR Articles 26, 28, 50-52, 61-63 and 92, and PRA Rulebook, CRR Firms, Capital Buffers Part, Chapter 2 Capital Conservation Buffer.

173.   See 2024 Proposal at 8046.

174.   Id.

175.   Id.

176.   Id. at 8040 and paragraph (3) of the definition of the term BHC equivalent risk-weighted assets in 17 CFR 23.100 .

177.   See 2024 Proposal at 8040, 17 CFR 1.17(c)(5) , and 17 CFR 240.18a-1(c)(1) .

178.   17 CFR 1.17(c)(5)(iii) .

179.   17 CFR 1.17(c)(5)(v) , referencing SEC Rule 15c3-1(c)(2)(vi) ( 17 CFR 240.15c3-1(c)(2)(vi) ).

180.   17 CFR 23.100 (definition of BHC equivalent risk-weighted assets ). As noted, a nonbank SD is required to maintain qualifying capital ( i.e., an aggregate of common equity tier 1 capital, additional tier 1 capital, and tier 2 capital) in an amount that equals or exceeds 8 percent of its risk-weighted assets. The regulations, however, require the nonbank SD to effectively maintain qualifying capital equal to or in excess of 100 percent of its market risk-weighted assets by requiring the nonbank SD to multiply its market-risk weighted assets by a factor of 12.5. For example, the market risk exposure amount for marketable equity securities with a current fair market value of $250,000 is $37,500 (market value of $250,000 × .15 standardized market risk factor). The nonbank SD is required to maintain regulatory capital equal to or in excess of full market risk exposure amount of $37,500 (risk exposure amount of $37,500 × 8 percent regulatory capital requirement equals $3,000; the regulatory capital requirement is then multiplied by a factor of 12.5, which effectively requires the nonbank SD to hold regulatory capital in an amount equal to at least 100 percent of the market risk exposure amount ($3,000 × 12.5 factor equals $37,500)).

181.   See 2024 Proposal at 8042.

182.   Id. and UK CRR, Article 326. As indicated in Article 326 of UK CRR, securitizations are treated as debt instruments for market risk requirements.

183.  2024 Proposal at 8042 and UK CRR, Article 351.

184.   Id.

185.  2024 Proposal at 8042 and UK CRR, Article 360.

186.  2024 Proposal at 8042 and UK CRR, Article 359-361.

187.  23.101(a)(1)(i)(B) and paragraph (1) of the definition of the term BHC equivalent risk-weighted assets in 17 CFR 23.100 . See also 2024 Proposal at 8040.

188.   12 CFR 217.32 . Lower credit risk factors are assigned to entities with lower credit risk and higher credit risk factors are assigned to entities with higher credit risk. For example, a credit risk factor of 0 percent is applied to exposures to the U.S. government, the Federal Reserve Bank, and U.S. government agencies ( 12 CFR 217.32(a)(1) ), and a credit risk factor of 100 percent is assigned to an exposure to foreign sovereigns that are not members of the Organization of Economic Co-operation and Development ( 12 CFR 217.32(a)(2) ). See also discussion in 2024 Proposal at 8040.

189.   12 CFR 217.33 . See also discussion in 2024 Proposal at 8040.

190.  2024 Proposal at 8043 and UK CRR, Articles 111 and 113(1).

191.  2024 Proposal at 8043 and UK CRR, Articles 114-122.

192.  2024 Proposal at 8043 and UK CRR, Articles 121(2) and 122(2).

193.   17 CFR 217.34 and 17 CFR 23.100 (defining the term BHC risk-weighted assets and providing that a nonbank SD that does not have model approval may use either CEM or SA-CCR to compute its exposures for OTC derivative contracts without regard to the status of its affiliate with respect to the use of a calculation approach under the Federal Reserve Board's capital rules). See also discussion in 2024 Proposal at 8040.

194.   12 CFR 217.34 .

195.   12 CFR 217.132(c) .

196.  2024 Proposal at 8043, UK CRR, Articles 92(3)(f), and PRA Rulebook, CRR Firms, Counterparty Credit Risk (CRR) Part, Chapter 3 Counterparty Credit Risk (Part Three, Title Two, Chapter Six CRR). As noted in the 2024 Proposal, PRA-designated UK nonbank SDs with smaller-sized derivatives business may also use a “simplified standardized approach to counterparty credit risk” or an “original exposure method” as simpler methods for calculating exposure values. PRA Rulebook, CRR Firms, Counterparty Credit Risk (CRR) Part, Chapter 3 Counterparty Credit Risk (Part Three, Title Two, Chapter Six CRR), Articles 281-282. To use either of these alternative methods, an entity's on-and off-balance sheet derivatives business must be equal to or less than 10 percent of the entity's total assets and GBP 260 million or 5 percent of the entity's total assets and GBP 88 million, respectively. PRA Rulebook, CRR Firms, Counterparty Credit Risk (CRR) Part, Chapter 3 Counterparty Credit Risk (Part Three, Title Two, Chapter Six CRR), Article 273a.

197.  PRA Rulebook, CRR Firms, Counterparty Credit Risk (CRR) Part, Chapter 3 Counterparty Credit Risk (Part Three, Title Two, Chapter Six CRR), Article 274 and 12 CFR 217.132(c) . See also discussion in 2024 Proposal at 8043.

198.  2024 Proposal at 8043 and UK CRR, Article 378 (indicating that if transactions in which debt instruments, equities, foreign currencies and commodities excluding repurchase transactions and securities or commodities lending and securities or commodities borrowing are unsettled after their delivery due dates, a PRA-designated UK nonbank SD must calculate the price difference to which it is exposed).

199.   Id. The price difference to which a PRA-designated UK nonbank SD is exposed is the difference between the agreed settlement price for an instrument ( i.e., a debt instrument, equity, foreign currency or commodity) and the instrument's current market value, where the difference could involve a loss for the firm. UK CRR, Article 378.

200.   17 CFR 23.100 (definition of BHC equivalent risk-weighted assets ), 12 CFR 217.38 and 12 CFR 217.136 .

201.  2024 Proposal at 8043 and UK CRR, Articles 381 and 382(1).

202.  UK CRR, Articles 383-384 and 12 CFR 217.132(e)(5) and (6) . Under the CFTC's Bank-Based Approach, nonbank SDs calculating their credit risk-weighted assets using the regulations in Subpart D of the Federal Reserve Board's Part 217 regulations do not calculate CVA of OTC derivatives instruments.

203.   See 2024 Proposal at 8040-8041 and 8043, respectively, for discussions of NFA and PRA model approvals. In discussing approval requirements for credit risk models as part of the general overview of the UK PRA Capital Rules, the Commission referred generally to counterparty credit risk exposures for “OTC derivatives transactions.” See 2024 Proposal at 8034-8035 (n. 115). For clarity, the Commission notes that the Internal Model Methodology for counterparty credit risk set out in UK CRR, Articles 283-294, can be used for the derivatives listed in Annex II of UK CRR, securities financing transactions, and long settlement transactions. PRA Rulebook, CRR Firms, Counterparty Credit Risk (CRR) Part, Article 273.

204.   See 2024 Proposal at 8046. For a discussion of the qualitative and quantitative requirements that models must meet under the CFTC Capital Rules and the UK PRA Capital Rules, see 2024 Proposal at 8040-8041 and 8043-8044, respectively. In discussing model approval conditions, the Commission noted that PRA-designated UK nonbank SDs were not permitted to use internal models to calculate counterparty credit risk amounts for large exposures. See 2024 Proposal at 8043 and 8044 (n. 217 and n. 237). The Commission notes that this statement is not correct with regard to securities financing transactions. PRA-designated UK nonbank SDs are allowed to use internal models to calculate exposure values for securities financing transactions. PRA Rulebook, CRR Firms, Large Exposures (CRR) Part, Chapter 4 Large Exposures (Part Four CRR), Article 390.

205.   See 2024 Proposal at 8046.

206.   Id.

207.   Id. and UK CRR, Article 92(3).

208.   Id. and 17 CFR 23.101(a)(1)(i) and 17 CFR 23.100 (definition of BHC equivalent risk-weighted assets ).

209.  Ravnitzky Letter at pp. 3-4.

210.   Id.

211.  As noted in the 2024 Proposal, the Commission is aware that the PRA is considering changes to the UK PRA Capital Rules to implement Basel 3.1 standards. If the PRA proceeds with the implementation of the Basel 3.1 standards as proposed, the regulatory changes would be applicable after July 1, 2025 with a 4.5-year transitional period ending on January 1, 2030. The Commission will monitor progress on the PRA's proposed regulatory changes and may amend or supplement the Comparability Order, as appropriate. 2024 Proposal at 8036 (n. 128).

212.  More specifically, in establishing the requirement that a nonbank SD must maintain a level of regulatory capital in excess of 8 percent of the uncleared swap margin amount associated with the firm's swap transactions, the Commission stated that the intent of the uncleared swap margin amount was to establish a method of developing a minimum amount of capital for a nonbank SD to meet all of its obligations as an SD to market participants, and to cover potential operational risk, legal risk and liquidity risk, and not just the risks of its trading portfolio. 85 FR 57462 at 57485.

213.   See 2024 Proposal at 8046-8047.

214.   Id.

215.   Id. at 8047 and UK CRR, Article 92(3).

216.   Id. More specifically, the UK PRA Capital Rules impose separate liquidity buffers and “stable funding” requirements designed to ensure that PRA-designated UK nonbank SDs can cover both long-term obligations and short-term payment obligations under stressed conditions for 30 days. PRA Rulebook, CRR Firms, Liquidity (CRR) Part, Chapter 4 Liquidity (Part Six CRR), Article 412-413. In addition, PRA-designated UK nonbank SDs are required to maintain robust strategies, policies, processes, and systems for the identification of liquidity risk over an appropriate set of time horizons, including intra-day. PRA Rulebook, CRR Firms, Internal Liquidity Adequacy Assessment Part.

217.   See 2024 Proposal at 8047. Specifically, Commission Regulation 23.600(b) requires each SD to establish, document, maintain, and enforce a system of risk management policies and procedures designed to monitor and manage the risks related to swaps, and any products used to hedge swaps, including futures, options, swaps, security-based swaps, debt or equity securities, foreign currency, physical commodities, and other derivatives. The elements of the SD's risk management program are required to include the identification of risks and risk tolerance limits with respect to applicable risks, including operational, liquidity, and legal risk, together with a description of the risk tolerance limits set by the SD and the underlying methodology in written policies and procedures. 17 CFR 23.600 .

218.  Better Markets Letter at p. 13.

219.   Id.

220.   Id.

221.  Applicants' Letter at p. 3.

222.   Id. at pp. 2-3. As discussed in the 2024 Proposal, the UK PRA Capital Rules impose a 3.35 percent leverage ratio floor on PRA-designated UK nonbank SDs that hold significant amounts of non-UK assets, as an additional element of the capital requirements. Specifically, a PRA-designated UK nonbank SD that has non-UK assets equal to or greater than GBP 10 billion is required to maintain tier 1 capital ( i.e., an aggregate of common equity tier 1 capital and additional tier 1 capital) equal to or in excess of 3.35 percent of the firm's on-balance sheet and off-balance sheet exposures, including exposures on uncleared swaps but excluding certain exposures to central banks, without regard to any risk-weighting. See 2024 Proposal at 8034 and PRA Rulebook, CRR Firms, Leverage Ratio (CRR) Part, Chapter 3 Leverage Ratio (Part Seven CRR), Article 429 et seq.

223.   85 FR 57462 at 57497.

224.   85 FR 57462 at 57485 and 57497.

225.   See 2024 Proposal at 8040 (referencing 85 FR 57462 ).

226.  2024 Proposal at 8047-8048.

227.   Id. and 17 CFR 23.105(d) and (e) .

228.   Id. and 17 CFR 23.105(d)(2) .

229.   Id. and 17 CFR 23.105(e)(4) .

230.   Id. at 8048 and 17 CFR 23.105(f) .

231.   Id.

232.  2024 Proposal at 8048, Regulation 23.105(l), and Schedule 1 of Appendix B to Subpart E of Part 23 (“Schedule 1”). 17 CFR 23.105(l) and 17 CFR Appendix B to Subpart E of Part 23. Schedule 1 includes a nonbank SD's holding of U.S Treasury securities, U.S. government agency debt securities, foreign debt and equity securities, money market instruments, corporate obligations, spot commodities, and cleared and uncleared swaps, security-based swaps, and mixed swaps in addition to other position information.

233.  2024 Proposal 8048 and schedules 2, 3 and 4, respectively, of Appendix B to Subpart E of Part 23.

234.  2024 Proposal 8048 and 17 CFR 23.105(k) and (l) , and schedules 2, 3 and 4 of Appendix B to Subpart E of Part 23.

235.  2024 Proposal 8048 and 17 CFR 23.105(m) .

236.   Id.

237.   Id.

238.   Id.

239.  2024 Proposal at 8048-8050.

240.  2024 Proposal at 8048-8049 and PRA Rulebook, CRR Firms, Reporting (CRR) Part, Chapter 4 Reporting (Part Seven A CRR), Rule 1.

241.  PRA Rulebook, CRR Firms, Reporting (CRR) Part, Chapter 6 Templates and Instructions.

242.  PRA Rulebook, CRR Firms, Reporting (CRR) Part, 5 Reporting Requirements, Chapter 3 Format and Frequency of Reporting on Own Funds, Own Funds Requirements.

243.  PRA Rulebook, CRR Firms, Reporting (CRR) Part, Chapter 6 Templates and Instructions, Annex I, Templates C 01.00 and C 02.00.

244.  PRA Rulebook, CRR Firms, Reporting (CRR) Part, Chapter 6 Templates and Instructions, Annex I, Template C 03.00.

245.  PRA Rulebook, CRR Firms, Reporting (CRR) Part, Chapter 6 Templates and Instructions, Annex I, Template C 02.00.

246.  A parent company ( i.e., “parent undertaking”) is defined in Companies Act 2006, Section 1162.

247.  PRA Rulebook, CRR Firms, Reporting (CRR) Part, Chapter 4 Reporting (Part Seven A CRR), Article 430, Rule 3. The International Accounting Standards Board is an independent, private-sector body that develops and approves IFRS.

248.  PRA Rulebook, CRR Firms, Reporting (CRR) Part, Chapter 6 Templates and Instructions, Templates 1.1., 1.2., and 1.3 at Annex III (for reporting according to IFRS) and Templates 1.1., 1.2., and 1.3 at Annex IV (for reporting according to national accounting frameworks).

249.  PRA Rulebook, CRR Firms, Reporting (CRR) Part, Chapter 6 Templates and Instructions, Template 2 at Annex III (for reporting according to IFRS) and Template 2 at Annex IV (for reporting according to national accounting frameworks).

250.  PRA Rulebook, CRR Firms, Reporting (CRR) Part, Chapter 6 Templates and Instructions, Template 8.1 at Annex III (for reporting according to IFRS) and Template 8.1 at Annex IV (for reporting according to national accounting frameworks).

251.  PRA Rulebook, CRR Firms, Reporting (CRR) Part, Chapter 6 Templates and Instructions, Template 8.2 at Annex III (for reporting according to IFRS) and Template 8.2. at Template 8.2 at Annex IV (for reporting according to national accounting frameworks).

252.  PRA Rulebook, CRR Firms, Reporting (CRR) Part, Chapter 6 Templates and Instructions, Template 46 at Annex III (for reporting according to IFRS) and Template 46 at Annex IV (for reporting according to national accounting frameworks).

253.  PRA Rulebook, CRR Firms, Reporting (CRR) Part, Chapter 6 Templates and Instructions, Templates 5.1 and 6.1 at Annex III (for reporting according to IFRS) and Templates 5.1 and 6.1 at Annex IV (for reporting according to national accounting frameworks).

254.  PRA Rulebook, CRR Firms, Reporting (CRR) Part, Chapter 6 Templates and Instructions, Template 10 at Annex III (for reporting according to IFRS) and Template 10 at Annex IV (for reporting according to national accounting frameworks).

255.  PRA Rulebook, CRR Firms, Reporting (CRR) Part, Chapter 6 Templates and Instructions, Template 13 at Annex III (for reporting according to IFRS) and Template 13 at Annex IV (for reporting according to national accounting frameworks).

256.  As indicated by the Applicants, the Regulatory Reporting Part of the PRA Rulebook applies to all PRA-designated UK nonbank SDs. See Responses to Staff Questions dated October 5, 2023.

257.  PRA Rulebook, CRR Firms, Regulatory Reporting Part, Chapter 9 Regulated Activity Group 3, Rule 9.2 (referencing Templates 1.1., 1.2., and 1.3 at Annex III and Templates 1.1., 1.2., and 1.3 at Annex IV of Chapter 6 of the Reporting (CRR) Part) and Rule 9.3.

258.  PRA Rulebook, CRR Firms, Regulatory Reporting Part, Chapter 9 Regulated Activity Group 3, Rule 9.2 (referencing Template 2 at Annex III and Template 2 at Annex IV of Chapter 6 of the Reporting (CRR) Part) and Rule 9.3.

259.  PRA Rulebook, CRR Firms, Regulatory Reporting Part, Chapter 9 Regulated Activity Group 3, Rule 9.2 and Rule 9.3.

260.   See Response to Staff Questions dated October 5, 2023. For the avoidance of doubt, as represented by the Applicants, the six PRA-designated UK nonbank SDs currently registered with the Commission are subject to the RAG 3 requirements in the Regulatory Reporting Part of the PRA Rulebook but are not subject the FINREP requirements set forth in Article 430(3) of the Reporting (CRR) Part of the PRA Rulebook. As such, the six PRA-designated UK nonbank SDs currently registered with the Commission are required to submit to the PRA only Templates 1 through 3 of FINREP.

261.  Companies Act 2006, Sections 393 to 414D and 475. Section 475 provides for an exemption from the audit requirement for certain entities ( i.e., “small companies”, qualifying “subsidiary companies” and “dormant companies”.) None of the six PRA-designated UK nonbank SD, however, falls into the exempt categories. See Responses to Staff Questions dated October 5, 2023.

262.  Companies Act 2006, Section 485 et seq.; see also PRA Rulebook, CRR Firms, Auditors Part, Rule 3 Auditors' Qualifications, and Rule 4 Auditors' Independence.

263.  PRA Rulebook, CRR Firms, Regulatory Reporting Part, Chapter 9 Regulatory Activity Group 3, Rules 9.1. and 9.4. The “accounting reference date” is determined in accordance with Section 391 of the Companies Act 2006 and depending on the firm's date of incorporation.

264.  Companies Act 2006, Section 441. The deadline for filing the annual audited financial report with the UK Registrar of Companies is nine months from the firm's accounting reference date for private companies and six months from the firm's accounting reference date for public companies. Id., Articles 442 (setting forth the filing deadlines by category of firm) and 391 (defining the terms “accounting reference period” and “accounting reference date”).

265.  Companies Act 2006, Sections 1080 and 1085. Information filed with the UK Registrar of Companies is available at: https://www.gov.uk/​government/​organisations/​companies-house .

266.  Companies Act 2006, Section 396.

267.   Id., Section 495.

268.   Id., Section 414C.

269.   Id., Section 496.

270.   Id.

271.   See 2024 Proposal at 8050 and UK Order. See also SEC Order on Manner and Format of Filing Unaudited Financial and Operational Information.

272.   See, SEC Order on Manner and Format of Filing Unaudited Financial and Operational Information.

273.   Id.

274.  2024 Proposal at 8050.

275.   Id.

276.   Id.

277.   Id. at 8051.

278.   Id. In the 2024 Proposal, the Commission proposed that the conversion of account balances from British pound to U.S. dollars would not be required to be subject to the audit of the independent auditor. A PRA-designated UK nonbank SD would be required report the exchange rate that it used to convert balances from British pound to U.S. dollars to the Commission and NFA as part of the financial reporting.

279.   Id.

280.   Id. The Commission noted that the UK PRA Financial Reporting Rules require PRA-designated UK nonbank SDs to submit the unaudited FINREP and COREP templates to PRA on a quarterly basis, whereas the CFTC Financial Reporting Rules contain a more frequent reporting requirement by requiring nonbank SDs that elect the Bank-Based Approach to file unaudited financial information with the Commission and NFA on a monthly basis. In emphasizing the importance of financial statement reporting requirements for the Commission's and NFA's oversight and the Commission's experience in monitoring the financial conditions of registrants through the receipt of monthly financial statements, the Commission proposed to condition the Comparability Order on a more frequent reporting submission. See 2024 Proposal at 8050-8051. The Commission also noted that PRA-designated UK nonbank SDs are required to submit the annual audited financial report to the PRA within 80 business days of the firm's accounting reference date. See PRA Rulebook, Regulatory Reporting Part, Rule 9.1.

281.  2024 Proposal at 8052. Schedule 1 includes a nonbank SD's holding of U.S Treasury securities, U.S. government agency debt securities, foreign debt and equity securities, money market instruments, corporate obligations, spot commodities, and cleared and uncleared swaps, security-based swaps, and mixed swaps in addition to other position information.

282.  2024 Proposal at 8052.

283.   Id.

284.   See Capital and Financial Reporting Requirements of Swap Dealers and Major Swap Participants, 89 FR 45569 (May 23, 2024).

285.  2024 Proposal at 8052.

286.   17 CFR 23.105(f) . Commission Regulation 23.105(f) requires a nonbank SD to attach to each unaudited and audited financial report an oath or affirmation that to the best knowledge and belief of the individual making the affirmation the information contained in the financial report is true and correct. The individual making the oath or affirmation must be a duly authorized officer if the nonbank SD is a corporation, or one of the persons specified in the regulation for business organizations that are not corporations.

287.  2024 Proposal at 8052.

288.  Commission Regulation 23.105(k) requires a nonbank SD that has obtained approval from the Commission or NFA to use internal capital models to submit to the Commission and NFA each month information regarding its risk exposures, including VaR, and requires certain credit risk exposure information from model and non-model approved firms. 17 CFR 23.105(k) . Commission Regulation 23.105(l) requires each nonbank SD to provide information to the Commission and NFA regarding its counterparty credit concentration for the 15 largest exposures in derivatives, a summary of its derivatives exposures by internal credit ratings, and the geographic distribution of derivatives exposures for the 10 largest countries in Schedules 2, 3, and 4, respectively. 17 CFR 23.105(l) .

289.  2024 Proposal at 8052-8053. As previously noted, however, the current six PRA-designated UK nonbank SDs will be required to include credit risk information set forth in Schedules 2-4 of Appendix B to Subpart E in the monthly FOCUS Report that the firms will be required to file with the Commission under Condition 10 of the final Comparability Order. In addition, as previously noted, each PRA-designated UK nonbank SD will be required to file Schedule 1 under Condition 12 of the final Comparability Determination.

290.   See 2024 Proposal at 8053 and NFA Financial Requirements, Section 17 — Swap Dealer and Major Swap Participant Reporting Requirements (“NFA Section 17 Rule”), available here: https://www.nfa.futures.org/​rulebooksql/​rules.aspx?​RuleID=​SECTION%2017&​Section=​7 , and Notice to Members — Monthly Risk Data Reporting for Swap Dealers (May 30, 2017) (“NFA Notice I-17-10”), available here: https://www.nfa.futures.org/​news/​newsNotice.asp?​ArticleID=​4817 .

291.   See 2024 Proposal at 8053 and FSMA, Part XI (indicating that the PRA has broad information gathering powers).

292.   See 2024 Proposal at 8053 and PRA Rulebook, CRR Firms, Reporting (CRR) Part, Chapter 6 Templates and Instructions, Annex I.

293.  Better Markets Letter at p. 14.

294.   Id. at p. 11.

295.   Id. at pp. 14-15.

296.   17 CFR 23.105(k) .

297.   17 CFR 23.105(k)(1) .

298.  Better Markets Letter at p.15.

299.  NFA Section 17 Rule, available here: https://www.nfa.futures.org/​rulebooksql/​rules.aspx?​RuleID=​SECTION%2017&​Section=​7 , and NFA Notice I-17-10, available here: https://www.nfa.futures.org/​news/​newsNotice.asp?​ArticleID=​4817 .

300.   See 2024 Proposal at 8053, NFA Section 17 Rule, and NFA Notice I-17-10.

301.  Applicants' Letter at p. 8.

302.   Id.

303.  CFTC Staff Letter No. 22-10, Extension of Time-Limited No-Action Position for Foreign Based Nonbank Swap Dealers domiciled in Japan, Mexico, the United Kingdom, and the European Union, issued by MPD on August 17, 2022. CFTC Staff Letter No. 22-10, which extended the expiration of CFTC Letter 21-20, provides that MPD would not recommend an enforcement action to the Commission if a non-U.S. nonbank SD covered by the letter, subject to certain conditions, complied with their respective home-country capital and financial reporting requirements in lieu of the Commission's capital and financial reporting requirements set forth in Commission Regulations 23.100 through 23.106, pending the Commission's determination of whether the capital and financial reporting requirements of certain foreign jurisdictions are comparable to the Commission's corresponding requirements.

304.  2024 Proposal at 8059.

305.  Furthermore, the Commission's approach to permitting PRA-designated UK nonbank SDs to maintain financial books and records, and to file financial reports and other financial information, prepared in accordance with local accounting standards is consistent with the SEC's final comparability determinations for non-U.S. SBSDs. German Order at 59812 and SEC Order on Manner and Format of Filing Unaudited Financial and Operational Information at 59219. Specifically, the SEC stated that the use of local reporting requirements will avoid non-U.S. SBSDs “having to perform and present two Basel capital calculations (one pursuant to local requirements and one pursuant to U.S. requirements).” SEC Order on Manner and Format of Filing Unaudited Financial and Operational Information at 59219. The SEC noted, in this regard, that the Basel standards are international standards that have been adopted in the U.S. and in jurisdictions where substituted compliance is available for capital under the SEC comparability determinations and that, therefore, requirements for how firms calculate capital pursuant to the Basel standards generally should be similar. Id. The Commission's approach to permitting PRA-designated UK nonbank SDs to maintain financial books and records, and file financial information, prepared in accordance with local accounting standards will also facilitate financial reporting by dually-registered PRA-designated UK nonbank SDs—UK nonbank SBSDs. In such case, dually-registered entities would not have to perform multiple calculations under different accounting standards or submit two different FOCUS Reports.

306.  CFTC Staff Letter No. 22-10, Extension of Time-Limited No-Action Position for Foreign Based Nonbank Swap Dealers domiciled in Japan, Mexico, the United Kingdom, and the European Union, August 17, 2022.

307.  2024 Proposal at 8053-8054 and 17 CFR 23.105(c) .

308.   17 CFR 23.105(c) .

309.   Id.

310.   See 2024 Proposal at 8053.

311.   Id. at 8054.

312.  See 2024 Proposal at 8054 and PRA Rulebook, CRR Firms, Capital Buffers Part, Chapter 4 Capital Conservation Measures, Rule 4.4. The combined capital buffer requirement is the total common equity tier 1 capital required to meet the sum of the capital conservation buffer and the institution-specific countercyclical capital buffer. PRA Rulebook, Capital Buffers Part, Chapter 1 Application and Definitions, Rule 1.2.

313.   Id.

314.   See 2024 Proposal at 8054 and PRA Rulebook, CRR Firms, Capital Buffers Part, Chapter 4 Capital Conservation Measures, Rules 4.4 and 4.5.

315.   See 2024 Proposal at 8054 and PRA Rulebook, CRR Firms, Capital Buffers Part, Chapter 4 Capital Conservation Measures, Rule 4.5.

316.   See 2024 Proposal at 8054 and Supervisory Statement SS6/14 Implementing Capital Buffers, Prudential Regulation Authority, January 2021 (“SS6/14”), available here: https://www.bankofengland.co.uk/​prudential-regulation/​publication/​2014/​implementing-crdiv-capital-buffers-ss .

317.   Id.

318.   See 2024 Proposal at 8054 and PRA Rulebook, CRR Firms, Notifications Part, Chapter 8 Specific Notifications, Rule 8.3.

319.  FSMA, Part 4A and Schedule 6.

320.   See 2024 Proposal at 8055.

321.   17 CFR 23.105(c)(1) and (2) .

322.  2024 Proposal at 8055.

323.   Id.

324.   Id. at 8056.

325.  Commission Regulation 23.105(c)(5) requires a nonbank SD to provide written notice to the Commission and NFA two business days prior to the withdrawal of capital by action of the equity holders if the amount of the withdrawal exceeds 30 percent of the nonbank SD's excess regulatory capital. 17 CFR 23.105(c)(5) .

326.  2024 Proposal at 8056.

327.  For comparison, see Commission Regulation 23.105(b), which similarly defines the term “current books and records” as used in the context of the Commission's requirements. 17 CFR 23.105(b) .

328.  2024 Proposal at 8056.

329.   Id.

330.   Id.

331.   Id.

332.   Id. at 8054-8057.

333.  Applicants' Letter at p. 5.

334.   Id.

335.   Id.

336.  Applicants' Letter at p. 6.

337.  For clarity, by “excess regulatory capital,” the Commission refers to the capital ratio by which the firm's capital exceeds the core capital ratio requirement of 8 percent of the firm's risk-weighted assets. For instance, if a firm maintains a capital ratio of 20 percent, its excess regulatory capital would be 12 percent. In this example, 30 percent of the excess regulatory capital would equal 3.6 percent.

338.   17 CFR 23.105(c)(4) .

339.  With regard to Condition 22, the Commission also notes, for clarity, that in proposing a notice condition based on thresholds of “required” margin, the Commission's intent was to set the notice trigger by reference to margin amounts that are legally required to be exchanged under the applicable margin requirements. To determine the applicable margin requirements, the Commission will consider the framework set forth in Commission Regulation 23.160. To the extent PRA-designated UK nonbank SDs intending to rely on the Comparability Order have inquiries regarding the scope of uncleared swap margin transactions to be monitored for purposes of complying with final Condition 22, MPD will discuss such inquiries with the PRA-designated UK nonbank SD during the confirmation process referenced in final Condition 8 of the Comparability Order.

340.   17 CFR 23.105(c)(3) .

341.  Applicants' Letter at p. 7. The Applicants indicated that in the context of proposed Condition 22, they understand the term “minimum capital requirement” to mean an amount equal to 8 percent of the PRA-designated UK nonbank SD's total risk exposure amount.

342.  Applicants' Letter at p. 8.

343.  Applicants' Letter at p. 8.

344.  2024 Proposal at 8057.

345.   Id.

346.   7 U.S.C. 21(p) .

347.  2024 Proposal at 8057.

348.  Commission Regulation 23.105(h) ( 17 CFR 23.105(h) ). See also 2024 Proposal at 8057.

349.   7 U.S.C. 6s(e) .

350.  2024 Proposal at 8057 and FSMA, Parts 4A, XI, and XIV.

351.   See 2024 Proposal at 8057 and PRA, The Prudential Regulation Authority's approach to banking supervision, July 2023, available at: https://www.bankofengland.co.uk/​prudential-regulation/​publication/​pras-approach-to-supervision-of-the-banking-and-insurance-sectors .

352.  2024 Proposal at 8057 and FSMA, Part 4A, Section 55M.

353.  PRA Rulebook, CRR Firms, Capital Buffers Part, Chapter 4 Capital Conservation Measures, Rule 4.3.

354.  Capital Requirements Regulations 2013, Regulation 35B and FSMA, Part XIV Disciplinary Measures (setting forth the PRA's disciplinary power with respect to all rules adopted under FSMA). The Applicants represented that “CRR rules” ( i.e., general PRA rules applying to CRR firms, including PRA-designated UK nonbank SDs) are adopted pursuant to FSMA, Part 9D, and as such the PRA has power to impose disciplinary measures in connection with these rules. See Response to Staff Questions dated October 5, 2023.

355.  FSMA, Parts 4A, Sections 55M and 55P, and Capital Requirements Regulation 2013, Regulation 35B.

356.  FSMA, Parts 4A and XIV.

357.  FSMA, Part 4A, Sections 55J-55K.

358.  Bank Recovery and Resolution (No. 2) Order 2014, Article 2 (defining “conditions for early intervention” in case of breach of UK CRR requirements or requirements derived from CRD) and Part 8 (laying down the procedure to be followed by the PRA to determine whether early intervention measures should be taken under FSMA). If additional requirements are met, it is also possible that the Bank of England, as the resolution authority, may assess the PRA-designated UK nonbank SD as “failing or likely to fail,” triggering a resolution action, which could occur even before the firm actually breached its minimum capital requirements. Banking Act 2009, Sections 4 to 83.

359.  Capital Requirements Regulation 2013, Section 34.

360.  2024 Proposal at 8058.

361.  2024 Proposal at 8029.

362.   Id. See also 17 CFR 23.106(a)(4)(ii) , which provides that all nonbank SDs, regardless of whether they rely on a Comparability Order or Comparability Determination, remain subject to the Commission's examination and enforcement authority.

363.   7 U.S.C. 21(p) .

364.  FSMA, Parts 4A and XIV.

365.  FSMA, Parts 4A, Sections 55M and 55P, and Capital Requirements Regulation 2013, Regulation 35B.

366.  FSMA, Part 4A, Sections 55J-55K.

367.   17 CFR 23.105(h) .

368.  NFA Section 17 Rule available at NFA's website: https://www.nfa.futures.org/​rulebooksql/​index.aspx .

369.   17 CFR 23.106(a)(4)(ii) .

370.  Condition 9 of the final Comparability Order.

371.  Conditions 10 and 11 of the final Comparability Order.

372.  Condition 15 of the final Comparability Order.

373.  Condition 20 of the final Comparability Order.

374.  Condition 17 of the final Comparability Order.

375.  Condition 16 of the final Comparability Order.

376.  Condition 21 of the final Comparability Order.

377.  Condition 24 of the final Comparability Order.

378.  The sharing of non-public information by CFTC staff would require assurances related to the use and treatment of such information in a manner consistent with Section 8(e) of the CEA, 7 U.S.C. 12(e) .

379.  For UK nonbank SDs regulated by the FCA, the Commission and the FCA are signatories to a supervisory MOU that covers information sharing and examinations. Memorandum of Understanding Concerning Cooperation and the Exchange of Information in the Context of Supervising Covered Firms (June 20, 2019).

1.   Capital Requirements of Swap Dealers and Major Swap Participants, 85 FR 57462 (Sept. 15, 2020). The Commission issued the final rule on July 24, 2020.

1.  Though the Final Comparability Determinations will apply to foreign nonbank MSPs in the relevant jurisdictions, there are no such MSPs currently registered with the Commission at this time. I will refer only to SDs herein.

2.  United States Government Accountability Office, Financial Regulatory Reform: Financial Crisis Losses and Potential Impacts of the Dodd-Frank Act (Jan. 2013), https://fraser.stlouisfed.org/​title/​gao-reports-testimonies-6136/​financial-regulatory-reform-622249 .

3.  Capital Requirements of Swap Dealers and Major Swap Participants, 85 FR 57462 (Sept. 15, 2020).

4.  The reporting requirements imposed on bank SD and bank MSPs were “more limited” “as the financial condition of these entities will be predominantly supervised by the applicable prudential regulator and subject to its capital and financial reporting requirements.” Id. at 57513. In May 2024, the Commission adopted amendments to the Capital and Financial Reporting Rules that codified two previously-issued staff letters providing interpretive guidance and no-action relief and made other technical amendments. 89 FR 45569 (May 23, 2024).

5.  Principles for Financial Market Infrastructures, Bank for International Settlements and International Organization of Securities Commissions (Apr. 2012), https://www.bis.org/​cpmi/​publ/​d101a.pdf .

6.  Cross-Border Application of the Registration Thresholds and Certain Requirements Applicable to Swap Dealers and Major Swap Participants, 85 FR 56924 , 56924 (Sept. 14, 2020).

7.  Kristin N. Johnson, Commissioner, CFTC, Combatting Systemic Risk and Fostering Integrity of the Global Financial System Through Rigorous Standards and International Comity (Jan. 24, 2024), https://www.cftc.gov/​PressRoom/​SpeechesTestimony/​johnsonstatement012424 ; Kristin N. Johnson, Commissioner, CFTC, Statement in Support of Notice and Order on EU Capital Comparability Determination (June 7, 2023), https://www.cftc.gov/​PressRoom/​SpeechesTestimony/​johnsonstatement060723c ; Kristin N. Johnson, Commissioner, CFTC, Statement in Support of Proposed Order and Request for Comment on Mexican Capital Comparability Determination (Nov. 10, 2022), https://www.cftc.gov/​PressRoom/​SpeechesTestimony/​johnsonstatement111022c ; Kristin N. Johnson, Commissioner, CFTC, Statement in Support of Proposed Order on Japanese Capital Comparability Determination (July 27, 2022), https://www.cftc.gov/​PressRoom/​SpeechesTestimony/​johnsonstatement072722c .

1.  IOSCO Report, “Good Practices on Processes for Deference” (June 2020), https://www.iosco.org/​library/​pubdocs/​pdf/​IOSCOPD659.pdf .

2.  Concurring Statement of Commissioner Caroline D. Pham Regarding Proposed Order and Request for Comment on an Application for a Capital Comparability Determination (Nov. 10, 2022), https://www.cftc.gov/​PressRoom/​SpeechesTestimony/​phamstatement111022 ; Statement of Commissioner Caroline D. Pham in Support of Proposed Order and Request for Comment on Comparability Determination for UK PRA Swap Dealer Capital and Financial Reporting Requirements (Jan. 24, 2024), https://www.cftc.gov/​PressRoom/​peechesTestimony/​phamstatement012424 .

[ FR Doc. 2024-15094 Filed 7-17-24; 8:45 am]

BILLING CODE 6351-01-P

  • Executive Orders

Reader Aids

Information.

  • About This Site
  • Accessibility
  • No Fear Act
  • Continuity Information

Information

  • Author Services

Initiatives

You are accessing a machine-readable page. In order to be human-readable, please install an RSS reader.

All articles published by MDPI are made immediately available worldwide under an open access license. No special permission is required to reuse all or part of the article published by MDPI, including figures and tables. For articles published under an open access Creative Common CC BY license, any part of the article may be reused without permission provided that the original article is clearly cited. For more information, please refer to https://www.mdpi.com/openaccess .

Feature papers represent the most advanced research with significant potential for high impact in the field. A Feature Paper should be a substantial original Article that involves several techniques or approaches, provides an outlook for future research directions and describes possible research applications.

Feature papers are submitted upon individual invitation or recommendation by the scientific editors and must receive positive feedback from the reviewers.

Editor’s Choice articles are based on recommendations by the scientific editors of MDPI journals from around the world. Editors select a small number of articles recently published in the journal that they believe will be particularly interesting to readers, or important in the respective research area. The aim is to provide a snapshot of some of the most exciting work published in the various research areas of the journal.

Original Submission Date Received: .

  • Active Journals
  • Find a Journal
  • Proceedings Series
  • For Authors
  • For Reviewers
  • For Editors
  • For Librarians
  • For Publishers
  • For Societies
  • For Conference Organizers
  • Open Access Policy
  • Institutional Open Access Program
  • Special Issues Guidelines
  • Editorial Process
  • Research and Publication Ethics
  • Article Processing Charges
  • Testimonials
  • Preprints.org
  • SciProfiles
  • Encyclopedia

remotesensing-logo

Article Menu

research instrument used in qualitative

  • Subscribe SciFeed
  • Recommended Articles
  • Google Scholar
  • on Google Scholar
  • Table of Contents

Find support for a specific problem in the support section of our website.

Please let us know what you think of our products and services.

Visit our dedicated information section to learn more about MDPI.

JSmol Viewer

Martian dust storms: reviews and perspective for the tianwen-3 mars sample return mission.

research instrument used in qualitative

1. Introduction

2. martian dust storms, 2.1. basic properties of dust storms, 2.2. driving mechanisms of dust storms, 2.3. impacts of dust storms, 3. detection methods for dust storms, 3.1. historical views on dust storms, 3.2. orbiter multicolor imaging, 3.3. orbiter infrared spectroscopy, 3.4. landers/rovers monitoring, 4. dust storm monitoring and predicting for tianwen-3, 4.1. statistical predictions.

  • The four zones are located in Utopia Planitia, Isidis Basin, Amazonis Planitia, and Chryse Planitia, respectively. Each zone covers a range of 18° in longitude and 15° in latitude, creating a 3 × 3 grid in the VDOD database;
  • The VDOD data in the grid during MY 24–35 (12 MYs) are extracted from the database to form a subset for each region, resulting in 108 samples for each zone (=3 × 3 × 12);
  • For each Martian day, the numbers of the six dust activity levels in the 108 samples are counted, and their probabilities are calculated;
  • Since Martian dust storms are seasonal, we extend the date to MY 40 and 41 according to the orbit of Mars;
  • Finally, the predicted results are presented in Figure 6 .
  • Before the southern spring equinox ( L S < 180°), the dust activity remains weak, generally below the safety line (VDOD < 1.0), occasionally exceeding 1.0 but with very short duration;
  • After the southern spring equinox ( L S > 180°), there is a significant increase in the probability of VDOD exceeding 1.0. The trend is particularly noticeable between L S of 200°~240° (i.e., 7 October to 11 December, 2029 in MY 40 and 25 August to 29 October, 2031 in MY 41), with the probability of VDOD exceeding 2.0 being greater than 30%. Dust activity generally peaks near L S of 210°~220°, and then begins to weaken and recover. During this period, it is important for operators to closely monitor the changing trend of dust activity;
  • After the southern summer solstice ( L S > 270°), dust activity experiences a significant weakening but remains at an active level. As shown in Figure 6 a, just before L S = 270°, the probabilities for Level 2 and Level 3 are both about 50%. Subsequently, dust activity will recover to quiet conditions and begin another seasonal trend.

4.2. Dust Storm Monitoring for Tianwen-3

  • Martian dust storm monitoring and early warning: Conducting large-field multicolor imaging of mid-low latitude dust storm activities on Mars to obtain information such as location, coverage and moving speed of dust storms on Mars.
  • Spatial and temporal distribution of atmospheric ozone on Mars: Conducting large-field ultraviolet imaging of the column content of middle- and low-latitude ozone on Mars to obtain its spatial and temporal distributions. This will also improve the accuracy of dust storm identification in combination with water-ice cloud identification in visible light channels.

5. Conclusions

Author contributions, data availability statement, acknowledgments, conflicts of interest.

  • Connerney, J.E.P.; Acuña, M.H.; Wasilewski, P.J.; Ness, N.F.; Rème, H.; Mazelle, C.; Vignes, D.; Lin, R.P.; Mitchell, D.L.; Cloutier, P.A. Magnetic lineations in the ancient crust of Mars. Science 1999 , 284 , 794–798. [ Google Scholar ] [ CrossRef ] [ PubMed ]
  • Jakosky, B.M. Atmospheric loss to space and the history of water on Mars. Ann. Rev. Earth Planet. Sci. 2021 , 49 , 71–93. [ Google Scholar ] [ CrossRef ]
  • Chaffin, M.S.; Kass, D.M.; Aoki, S.; Fedorova, A.A.; Deighan, J.; Connour, K.; Heavens, N.G.; Kleinböhl, A.; Jain, S.K.; Chaufray, J.-Y.; et al. Martian water loss to space enhanced by regional dust storms. Nat. Astronom. 2021 , 5 , 1036–1042. [ Google Scholar ] [ CrossRef ]
  • Dubinin, E.; Fraenz, M.; Pätzold, M.; McFadden, J.; Halekas, J.S.; DiBraccio, G.A.; Connerney, J.E.P.; Eparvier, F.; Brain, D.; Jakosky, B.M.; et al. The effect of solar wind variations on the escape of oxygen ions from Mars through different channels: MAVEN observations. J. Geophys. Res. Space Phys. 2017 , 122 , 11285–11301. [ Google Scholar ] [ CrossRef ]
  • Heavens, N.G.; Kleinböhl, A.; Chaffin, M.S.; Halekas, J.S.; Kass, D.M.; Hayne, P.O.; McCleese, D.J.; Piqueux, S.; Shirley, J.H.; Schofield, J.T. Hydrogen escape from Mars enhanced by deep convection in dust storms. Nat. Astronom. 2018 , 2 , 126–132. [ Google Scholar ] [ CrossRef ]
  • Lillis, R.J.; Brain, D.A.; Bougher, S.W.; Leblanc, F.; Luhmann, J.G.; Jakosky, B.M.; Modolo, R.; Fox, J.; Deighan, J.; Fang, X.; et al. Characterizing atmospheric escape from Mars today and through time, with MAVEN. Space Sci. Rev. 2015 , 195 , 357–422. [ Google Scholar ] [ CrossRef ]
  • Lundin, R.; Barabash, S.; Andersson, H.; Holmström, M.; Grigoriev, A.; Yamauchi, M.; Sauvaud, J.-A.; Fedorov, A.; Dudnik, E.; Thocaven, J.J.; et al. Solar wind-induced atmospheric erosion at Mars: First results from ASPERA-3 on Mars Express. Science 2004 , 305 , 1933–1936. [ Google Scholar ] [ CrossRef ]
  • Ramstad, R.; Barabash, S.; Futaana, Y.; Nilsson, H.; Wang, X.-D.; Holmström, M. The Martian atmospheric ion escape rate dependence on solar wind and solar EUV conditions: 1. Seven years of Mars Express observations. J. Geophys. Res. Planets 2015 , 120 , 1298–1309. [ Google Scholar ] [ CrossRef ]
  • Wei, Y.; Fraenz, M.; Dubinin, E.; Woch, J.; Lühr, H.; Wan, W.; Zong, Q.-G.; Zhang, T.L.; Pu, Z.Y.; Fu, S.Y.; et al. Enhanced atmospheric oxygen outflow on Earth and Mars driven by a corotating interaction region. J. Geophys. Res. 2012 , 117 , A03208. [ Google Scholar ] [ CrossRef ]
  • Barlow, N. Mars: An Introduction to Its Interior, Surface and Atmosphere ; Cambridge University Press: New York, NY, USA, 2008; pp. 163–186. [ Google Scholar ]
  • Leovy, C. Weather and climate on Mars. Nature 2001 , 412 , 245–249. [ Google Scholar ] [ CrossRef ]
  • Shirley, J.H. Solar system dynamics and global-scale dust storms on Mars. Icarus 2015 , 251 , 128–144. [ Google Scholar ] [ CrossRef ]
  • Keiffer, H.H.; Jakosky, B.M.; Snyder, C.W.; Matthews, M.S. Mars ; Univ. Ariz. Press: Tucson, AZ, USA, 1992. [ Google Scholar ]
  • Haberle, R.M.; Clancy, R.T.; Forget, F.; Smith, M.D.; Zurek, R.W. The Atmosphere and Climate of Mars ; Cambridge University Press: New York, NY, USA, 2007. [ Google Scholar ]
  • Encrenaz, T. The atmosphere of Mars as constrained by remote sensing. Space Sci. Rev. 2001 , 96 , 411–424. [ Google Scholar ] [ CrossRef ]
  • Smith, M.D. Spacecraft observations of Martian atmosphere. Ann. Rev. Earth Planet. Sci. 2008 , 36 , 191–219. [ Google Scholar ] [ CrossRef ]
  • Schmidt, F.; Way, M.J.; Costard, F.; Bouley, S.; Séjourné, A.; Aleinov, I. Circumpolar ocean stability on Mars 3 Gy ago. Proc. Natl. Acad. Sci. USA 2022 , 119 , e2112930118. [ Google Scholar ] [ CrossRef ] [ PubMed ]
  • Wei, Y.; He, F.; Fan, K.; Rong, Z.; Wang, Y. Preliminary predictions of the dust storm activity at the landing site of China’s Zhurong Mars rover in 2022. Chin. Sci. Bull. 2022 , 67 , 1938–1944. (In Chinese) [ Google Scholar ] [ CrossRef ]
  • Wang, Y.; Wei, Y.; Fan, K.; He, F.; Rong, Z.; Zhou, X.; Tan, N. The impact of dust storms on Mars surface rovers: Review and prospect. Chin. Sci. Bull. 2023 , 68 , 368–379. (In Chinese) [ Google Scholar ] [ CrossRef ]
  • Rong, Z.; Wei, Y.; He, F.; Gao, J.; Fan, K.; Wang, Y.; Cao, L.; Yan, L.; Ren, Z.; Zhou, X.; et al. The orbit schemes to monitor Martian dust storms: Benefits to China’s future mars missions. Chin. Sci. Bull. 2023 , 68 , 716–728. (In Chinese) [ Google Scholar ] [ CrossRef ]
  • James, P.B. Martian local dust storms. In Recent Advances in Planetary Meteorology ; Hunt, G., Ed.; Cambridge University Press: New York, NY, USA, 1985; pp. 85–100. [ Google Scholar ]
  • Martin, L.J.; Zurek, R.W. An analysis of the history of dust activity on Mars. J. Geophys. Res. 1993 , 98 , 3221–3246. [ Google Scholar ] [ CrossRef ]
  • Zurek, R.W.; Martin, L.J. Interannual variability of planet-encircling dust storms on Mars. J. Geophys. Res. 1993 , 98 , 3247–3259. [ Google Scholar ] [ CrossRef ]
  • Fernández, W. Martian dust storms: A review. Earth Moon Planet. 1997 , 77 , 19–46. [ Google Scholar ] [ CrossRef ]
  • Kahre, M.A.; Murphy, J.R.; Newman, C.E.; Wilson, R.J.; Cantor, B.A.; Lemmon, M.T.; Wolff, M.J. The Mars Dust Cycle. In The Atmosphere and Climate of Mars ; Haberle, R.M., Clancy, R.T., Forget, F., Smith, M.D., Zurek, R.W., Eds.; Cambridge University Press: New York, NY, USA, 2007; pp. 295–337. [ Google Scholar ]
  • Malin, M.C.; Cantor, B.A. MRO MARCI Weather Report for the week of 27 December 2021–2 January 2022. Malin Space Science Systems Captioned Image Release, 2022, MSSS-603. Available online: https://www.msss.com/msss_images/2022/01/05/ (accessed on 9 March 2024).
  • Malin, M.C.; Cantor, B.A.; Britton, A.W. MRO MARCI Weather Report for the week of 4 June 2018–10 June 2018, Malin Space Science Systems Captioned Image Release, 2018, MSSS-534. Available online: https://www.msss.com/msss_images/2018/06/13/ (accessed on 9 March 2024).
  • Cantor, B.A.; James, P.B.; Caplinger, M.; Wolff, M.J. Martian dust storms: 1999 Mars Orbiter Camera observations. J. Geophys. Res. 2001 , 106 , 23653–23688. [ Google Scholar ] [ CrossRef ]
  • Wang, H.; Richardson, M.I. The origin, evolution, and trajectory of large dust storms on Mars during Mars Year 24–30 (1999–2011). Icarus 2015 , 251 , 112–127. [ Google Scholar ] [ CrossRef ]
  • Kulowski, L.; Wang, H.; Toigo, A.D. The seasonal and spatial distribution of textured dust storms observed by Mars Global Surveyor Mars Orbiter Camera. Adv. Space Res. 2017 , 59 , 715–721. [ Google Scholar ] [ CrossRef ]
  • Guzewich, S.D.; Fedorova, A.A.; Kahre, M.A.; Toigo, A.D. Studies of the 2018/Mars Year 34 planet-encircling dust storm. J. Geophys. Res. Planets 2020 , 125 , e2020JE006700. [ Google Scholar ] [ CrossRef ]
  • Wang, H.; Saidel, M.; Richardson, M.I.; Toigo, A.D.; Battalio, J.M. Martian dust storm distribution and annual cycle from Mars daily global map observations. Icarus 2023 , 394 , 115416. [ Google Scholar ] [ CrossRef ]
  • Sánchez-Lavega, A.; del Río-Gaztelurrutia, T.; Hernández-Bernal, J.; Delcroix, M. The onset and growth of the 2018 Martian Global Dust Storm. Geophys. Res. Lett. 2019 , 46 , 6101–6108. [ Google Scholar ] [ CrossRef ]
  • Kass, D.M.; Kleinböhl, A.; McCleese, D.J.; Schofield, J.T.; Smith, M.D. Interannual similarity in the Martian atmosphere during the dust storm season. Geophys. Res. Lett. 2016 , 43 , 6111–6118. [ Google Scholar ] [ CrossRef ]
  • Montabone, L.; Forget, F.; Millour, E.; Wilson, R.J.; Lewis, S.R.; Cantor, B.; Kass, D.; Kleinböhl, A.; Lemmon, M.T.; Smith, M.D.; et al. Eight-year climatology of dust optical depth on Mars. Icarus 2015 , 251 , 65–95. [ Google Scholar ] [ CrossRef ]
  • Montabone, L.; Spiga, A.; Kass, D.M.; Kleinböhl, A.; Forget, F.; Millour, E. Martian year 34 column dust climatology from Mars Climate Sounder observations: Reconstructed maps and model simulations. J. Geophys. Res. Planets 2020 , 125 , e2019JE006111. [ Google Scholar ] [ CrossRef ]
  • Lemmon, M.T.; Guzewich, S.D.; McConnochie, T.; de Vicente-Retortillo, A.; Martínez, G.; Smith, M.D.; Bell, J.F., III; Wellington, D.; Jacob, S. Large dust aerosol sizes seen during the 2018 Martian global dust event by the Curiosity rover. Geophys. Res. Lett. 2019 , 46 , 9448–9456. [ Google Scholar ] [ CrossRef ]
  • Chen-Chen, H.; Pérez-Hoyos, S.; Sánchez-Lavega, A. Dust particle size, shape and optical depth during the 2018/MY34 Martian global dust storm retrieved by MSL Curiosity rover Navigation Cameras. Icarus 2021 , 354 , 114021. [ Google Scholar ] [ CrossRef ]
  • Clancy, R.T.; Wolff, M.J.; Christensen, P.R. Mars aerosol studies with the MGS TES emission phase function observations: Optical depths, particle sizes, and ice cloud types versus latitude and solar longitude. J. Geophys. Res. 2003 , 108 , 5098. [ Google Scholar ] [ CrossRef ]
  • Wolff, M.J.; Clancy, R.T. Constraints on the size of Martian aerosols from thermal emission spectrometer observations. J. Geophys. Res. 2003 , 108 , 5097. [ Google Scholar ] [ CrossRef ]
  • Vicente-Retortillo, Á.; Martínez, G.; Rennó, N.; Lemmon, M.T.; de la Torre Juarez, M. Determination of dust aerosol particle size at Gale crater using REMS UVS and Mastcam measurements. Geophys. Res. Lett. 2017 , 44 , 3502–3508. [ Google Scholar ] [ CrossRef ]
  • Fedorova, A.A.; Montmessin, F.; Rodin, A.V.; Korablev, O.I.; Maattanen, A.; Maltagliati, L.; Bertaux, J.-L. Evidence for a bimodal size distribution for the suspended aerosol particles on Mars. Icarus 2014 , 231 , 239–260. [ Google Scholar ] [ CrossRef ]
  • Bertaux, J.-L.; Korablev, O.; Perrier, S.; Quémerais, E.; Montmessin, F.; Leblanc, F.; Lebonnois, S.; Rannou, P.; Lefèvre, F.; Forget, F.; et al. SPICAM on Mars Express: Observing modes and overview of UV spectrometer data and scientific results. J. Geophys. Res. 2006 , 111 , E10S90. [ Google Scholar ] [ CrossRef ]
  • Guzewich, S.D.; Tiogo, A.D.; Richardson, M.I.; Newman, C.E.; Talaat, E.R.; Waugh, D.W.; McConnochie, T.H. The impact of a realistic vertical dust distribution on the simulation of the Martian General Circulation. J. Geophys. Res. Planet. 2013 , 118 , 980–993. [ Google Scholar ] [ CrossRef ]
  • Medvedev, A.S.; Yiğit, E.; Kuroda, T.; Hartogh, P. General circulation modeling of the Martian upper atmosphere during global dust storms. J. Geophys. Res. Planet. 2013 , 118 , 2234–2246. [ Google Scholar ] [ CrossRef ]
  • Wang, Y.; Chow, K.-C.; Xiao, J.; Wang, C.-F. Effect of dust particle size on the climate of Mars. Planet. Space Sci. 2021 , 208 , 105346. [ Google Scholar ] [ CrossRef ]
  • Hanel, R.; Conrath, B.; Hovis, W.; Kunde, V.; Lowman, P.; Maguire, W.; Pearl, J.; Pirraglia, J.; Prabhakara, C.; Schlachman, B.; et al. Investigation of the Martian environment by infrared spectroscopy on Mariner 9. Icarus 1972 , 17 , 423–442. [ Google Scholar ] [ CrossRef ]
  • Leovy, C.B.; Zurek, R.W.; Pollack, J.B. Mechanisms for Mars dust storms. J. Atmos. Sci. 1973 , 30 , 749–762. [ Google Scholar ] [ CrossRef ]
  • Gierasch, P.J.; Goody, R.M. A model of a Martian great dust storm. J. Atmos. Sci. 1973 , 30 , 169–179. [ Google Scholar ] [ CrossRef ]
  • Leovy, C.B.; Briggs, G.A.; Young, A.T.; Smith, B.A.; Pollack, J.B.; Shipley, E.N.; Wildey, R.L. The Martian atmosphere: Mariner 9 television experiment progress report. Icarus 1972 , 17 , 373–393. [ Google Scholar ] [ CrossRef ]
  • Cross, C.A. The heat balance of the Martian polar caps. Icarus 1971 , 15 , 110–114. [ Google Scholar ] [ CrossRef ]
  • Ryan, J.A.; Sharman, R.D.; Lucich, R.D. Local Mars dust storm generation mechanism. Geophys. Res. Lett. 1981 , 8 , 899–901. [ Google Scholar ] [ CrossRef ]
  • Ryan, J.A.; Lucich, R.D. Possible dust devils, vortices on Mars. J. Geophys. Res. 1983 , 88 , 11005–11011. [ Google Scholar ] [ CrossRef ]
  • Hess, S.L. Martian winds and dust clouds. Planet. Space Sci. 1973 , 21 , 1549–1557. [ Google Scholar ] [ CrossRef ]
  • Newman, C.E.; Lewis, S.R.; Read, S.R.; Forget, F. Modeling the Martian dust cycle, 1, Representations of dust transport processes. J. Geophys. Res. 2002 , 107 , 5123. [ Google Scholar ] [ CrossRef ]
  • Newman, C.E.; Lewis, S.R.; Read, S.R.; Forget, F. Modeling the Martian dust cycle, 2, Multiannual radiatively active dust transport simulations. J. Geophys. Res. 2002 , 107 , 5124. [ Google Scholar ] [ CrossRef ]
  • Basu, S.; Wilson, J.; Richardson, M.; Ingersoll, A. Simulation of spontaneous and variable global dust storms with the GFDL Mars GCM. J. Geophys. Res. 2006 , 111 , E09004. [ Google Scholar ] [ CrossRef ]
  • Kahre, M.A.; Murphy, J.R.; Haberle, R.M. Modeling the Martian dust cycle and surface dust reservoirs with the NASA Ames general circulation model. J. Geophys. Res. 2006 , 11 , E06008. [ Google Scholar ] [ CrossRef ]
  • Xiao, J.; Chow, K.C.; Chan, K. Dynamical processes of dust lifting in the northern mid-latitude region of Mars during the dust storm season. Icarus 2019 , 317 , 94–103. [ Google Scholar ] [ CrossRef ]
  • Forget, F.; Hourdin, F.; Fournier, R.; Hourdin, C.; Talagrand, O.; Collins, M.; Lewis, S.R.; Read, P.L.; Huot, J.P. Improved general circulation model for the Martian atmosphere from the surface to above 80 km. J. Geophys. Res. 1999 , 104 , 24155–24175. [ Google Scholar ] [ CrossRef ]
  • Petrosyan, A.; Galperin, B.; Larsen, S.E.; Lewis, S.R.; Määttänen, A.; Read, P.L.; Renno, N.; Rogberg, L.P.H.T.; Savijärvi, H.; Siili, T.; et al. The Martian atmospheric boundary layer. Rev. Geophys. 2011 , 49 , RG3005. [ Google Scholar ] [ CrossRef ]
  • Shirley, J.H.; Mischna, M.A. Orbit-spin coupling and the interannual variability of global-scale dust storm occurrence on Mars. Planet. Space Sci. 2017 , 139 , 37–50. [ Google Scholar ] [ CrossRef ]
  • Mischna, M.A.; Shirley, J.H. Numerical modeling of orbit-spin coupling accelerations in a Mars general circulation model: Implications for global dust storm activity. Planet. Space Sci. 2017 , 141 , 45–72. [ Google Scholar ] [ CrossRef ]
  • Newman, C.E.; Lee, C.; Mischna, M.A.; Richardson, M.I.; Shirley, J.H. An initial assessment of the impact of postulated orbit-spin coupling on Mars dust storm variability in fully interactive dust simulations. Icarus 2019 , 317 , 649–668. [ Google Scholar ] [ CrossRef ]
  • Shirley, J.H.; McKim, R.J.; Battalio, J.M.; Kass, D.M. Orbit-Spin Coupling and the Triggering of the Martian Planet-Encircling Dust Storm of 2018. J. Geophys. Res. Planets 2020 , 125 , e2019JE006077. [ Google Scholar ] [ CrossRef ]
  • Bertrand, T.; Wilson, R.J.; Kahre, M.A.; Urata, R.; Kling, A. Simulation of the 2018 global dust storm on Mars using the NASA Ames Mars GCM: A multitracer approach. J. Geophys. Res. Planets 2020 , 125 , e2019JE006122. [ Google Scholar ] [ CrossRef ]
  • Wu, Z.; Li, T.; Heavens, N.G.; Newman, C.E.; Richardson, M.I.; Yang, C.; Li, J.; Cui, J. Earth-like thermal and dynamical coupling processes in the Martian climate system. Earth Sci. Rev. 2022 , 229 , 104023. [ Google Scholar ] [ CrossRef ]
  • Zhou, X.; Wei, Y.; Wu, Z.; Ren, Z.; Tan, N.; Fan, S.; He, F.; Rong, Z.; Yan, L.; Wang, Y.; et al. Martian whole atmosphere model and dust activities: Review and prospect. Chin. Sci. Bull. 2024 , 69 , 1058–1067. (In Chinese) [ Google Scholar ] [ CrossRef ]
  • Fenton, L.K.; Geissler, P.E.; Haberle, R.M. Global warming and climate forcing by recent albedo changes on Mars. Nature 2007 , 446 , 646–649. [ Google Scholar ] [ CrossRef ]
  • Kass, D.M.; Schofield, J.T.; Kleinböhl, A.; McCleese, D.J.; Heavens, N.G.; Shirley, J.H.; Steele, L.J. Mars Climate Sounder observation of Mars’ 2018 global dust storm. Geophys. Res. Lett. 2020 , 47 , e2019GL083931. [ Google Scholar ] [ CrossRef ]
  • Guzewich, S.D.; Lemmon, M.; Smith, C.L.; Martínez, G.; de Vicente-Retortillo, Á.; Newman, C.E.; Baker, M.; Campbell, C.; Cooper, B.; Gómez-Elvira, J.; et al. Mars Science Laboratory observations of the 2018/Mars year 34 global dust storm. Geophys. Res. Lett. 2019 , 46 , 71–79. [ Google Scholar ] [ CrossRef ]
  • Haberle, R.M.; Pollack, J.B.; Barnes, J.R.; Zurek, R.W.; Leovy, C.B.; Murphy, J.R.; Lee, H.; Schaeffer, J. Mars atmospheric dynamics as simulated by the NASA Ames General Circulation Model: 1. The zonal-mean circulation. J. Geophys. Res. 1993 , 98 , 3093–3123. [ Google Scholar ] [ CrossRef ]
  • Wu, Z.; Li, T.; Zhang, X.; Li, J.; Cui, J. Dust tides and rapid meridional motions in the Martian atmosphere during major dust storms. Nat. Commun. 2020 , 11 , 614. [ Google Scholar ] [ CrossRef ] [ PubMed ]
  • Zhao, Y.; Zhong, L.; Yuan, R.; Zhao, C.; Li, R.; Wang, Y.; Lian, Y.; Richardson, M. Simulation of Martian dust effects on polar CO 2 ice caps and atmospheric circulation using the MarsWRF model. J. Geophys. Res. Planet. 2021 , 126 , e2021JE006937. [ Google Scholar ] [ CrossRef ]
  • Melnik, O.; Parrot, M. Electrostatic discharge in Martian dust storms. J. Geophys. Res. 1998 , 103 , 29107–29117. [ Google Scholar ] [ CrossRef ]
  • Farrell, W.M.; Kaiser, M.L.; Desch, M.D.; Houser, J.G.; Cummer, S.A.; Wilt, D.M.; Landis, G.A. Detecting electrical activity from Martian dust storms. J. Geophys. Res. 1999 , 104 , 3795–3801. [ Google Scholar ] [ CrossRef ]
  • Krauss, C.E.; Horanyi, M.; Robertson, S. Experimental evidence for electrostatic discharging of dust near the surface of Mars. New J. Phys. 2003 , 5 , 70. [ Google Scholar ] [ CrossRef ]
  • Merrison, J.; Jensen, J.; Kinch, K.; Mugford, R.; Nørnberg, P. The electrical properties of Mars analogue dust. Planet Space Sci. 2004 , 52 , 279–290. [ Google Scholar ] [ CrossRef ]
  • Michael, M.; Tripathi, S.N. Effect of charging of aerosols in the lower atmosphere of Mars during the dust storm of 2001. Planet Space Sci. 2008 , 56 , 1696–1702. [ Google Scholar ] [ CrossRef ]
  • Haider, S.A.; Sheel, V.; Smith, M.D.; Maguire, W.C.; Molina-Cuberos, G.J. Effect of dust storms on the D region of the Martian ionosphere: Atmospheric electricity. J. Geophys. Res. Space Phys. 2010 , 115 , A12336. [ Google Scholar ] [ CrossRef ]
  • McDunn, T.L.; Bougher, S.W.; Murphy, J.; Smith, M.D.; Forget, F.; Bertaux, J.-L.; Montmessin, F. Simulating the density and thermal structure of the middle atmosphere (80–130 km) of Mars using the MGCM–MTGCM: A comparison with MEX/SPICAM observations. Icarus 2010 , 206 , 5–17. [ Google Scholar ] [ CrossRef ]
  • Chaufray, J.Y.; Chaffin, M.S.; Deighan, J.; Jain, S.; Schneider, N.; Mayyasi, M.; Jakosky, B. Effect of the 2018 Martian global dust storm on the CO2 density in the lower nightside thermosphere observed from MAVEN/IUVS Lyman-alpha absorption. Geophys. Res. Lett. 2020 , 47 , e2019GL082889. [ Google Scholar ] [ CrossRef ]
  • Fang, X.; Ma, Y.; Lee, Y.; Bougher, S.; Liu, G.; Benna, M.; Mahaffy, P.; Montabone, L.; Pawlowski, D.; Dong, C.; et al. Mars dust storm effects in the ionosphere and magnetosphere and implications for atmospheric carbon loss. J. Geophys. Res. Space Phys. 2020 , 125 , e2019JA026838. [ Google Scholar ] [ CrossRef ]
  • Niu, D.D.; Cui, J.; Wu, S.Q.; Gu, H.; Cao, Y.-T.; Wu, Z.-P.; Wu, X.-S.; Zhong, J.-H.; Wu, M.-Y.; Wei, Y.; et al. Species-dependent response of the Martian ionosphere to the 2018 global dust event. J. Geophys. Res. Planets 2021 , 126 , e2020JE006679. [ Google Scholar ] [ CrossRef ]
  • Mukundan, V.; Thampi, S.V.; Bhardwaj, A.; Fang, X. Impact of the 2018 Mars global dust storm on the ionospheric peak: A study using a photochemical model. J. Geophys. Res. Planets 2021 , 126 , e2021JE006823. [ Google Scholar ] [ CrossRef ]
  • Stone, S.W.; Yelle, R.V.; Benna, M.; Lo, D.Y.; Elrod, M.K.; Mahaffy, P.R. Hydrogen escape from Mars is driven by seasonal and dust storm transport of water. Science 2020 , 370 , 824–831. [ Google Scholar ] [ CrossRef ]
  • Clancy, R.T.; Nair, H. Annual (perihelion-aphelion) cycles in the photochemical behavior of the global Mars atmosphere. J. Geophys. Res. Planets 1996 , 101 , 12785–12790. [ Google Scholar ] [ CrossRef ]
  • Clancy, R.T.; Wolff, M.J.; Lefèvre, F.; Cantor, B.A.; Malin, M.C.; Smith, M.D. Daily global mapping of Mars ozone column abundances with MARCI UV band imaging. Icarus 2016 , 266 , 112–133. [ Google Scholar ] [ CrossRef ]
  • Hvidberg, C.S.; Fishbaugh, K.E.; Winstrup, M.; Svensson, A.; Byrne, S.; Herkenhoff, K.E. Reading the climate record of the Martian polar layered deposits. Icarus 2012 , 221 , 405–419. [ Google Scholar ] [ CrossRef ]
  • Colaprete, A.; Barnes, J.; Haberle, R.; Hollingsworth, J.L.; Kieffer, H.H.; Titus, T.N. Albedo of the south pole on Mars determined by topographic forcing of atmosphere dynamics. Nature 2005 , 435 , 184–188. [ Google Scholar ] [ CrossRef ] [ PubMed ]
  • Golombek, M.P.; Warner, N.H.; Ganti, V.; Lamb, M.P.; Parker, T.J.; Fergason, R.L.; Sullivan, R. Small crater modification on Meridiani Planum and implications for erosion rates and climate change on Mars. J. Geophys. Res. Planets 2014 , 119 , 2522–2547. [ Google Scholar ] [ CrossRef ]
  • Farley, K.A.; Malespin, C.; Mahaffy, P.; Grotzinger, J.P.; Vasconcelos, P.M.; Milliken, R.E.; Malin, M.; Edgett, K.S.; Pavlov, A.A.; Hurowitz, J.A.; et al. In Situ Radiometric and Exposure Age Dating of the Martian Surface. Science 2013 , 343 , 1247166. [ Google Scholar ] [ CrossRef ] [ PubMed ]
  • Vasavada, A.R.; Chen, A.; Barnes, J.R.; Burkhart, P.D.; Cantor, B.A.; Dwyer-Cianciolo, A.M.; Fergason, R.L.; Hinson, D.P.; Justh, H.L.; Kass, D.M.; et al. Assessment of environments for mars science laboratory entry, descent, and surface operations. Space Sci. Rev. 2012 , 170 , 793–835. [ Google Scholar ] [ CrossRef ]
  • Team, R. Characterization of the Martian Surface Deposits by the Mars Pathfinder Rover, Sojourner. Science 1997 , 278 , 1765–1767. [ Google Scholar ] [ CrossRef ]
  • Rieder, R.T.; Economou, T.E.; Wänke, H.; Turkevich, A.; Crisp, J.; Brückner, J.; Dreibus, G.; McSween, H.Y., Jr. The Chemical Composition of Martian Soil and Rocks Returned by the Mobile Alpha Proton X-ray Spectrometer: Preliminary Results from the X-ray Mode. Science 1998 , 278 , 1771–1774. [ Google Scholar ] [ CrossRef ]
  • Lorenz, R.D.; Martínez, G.M.; Spiga, A.; Vicente-Retortillo, A.; Newman, C.E.; Murdoch, N.; Forget, F.; Millour, E.; Pierron, T. Lander and rover histories of dust accumulation on and removal from solar arrays on Mars. Planet. Space Sci. 2021 , 207 , 105337. [ Google Scholar ] [ CrossRef ]
  • Landis, G.A.; Jenkins, P.P. Measurement of the settling rate of atmospheric dust on Mars by the MAE instrument on Mars Pathfinder. J. Geophys. Res. 2000 , 105 , 1855–1857. [ Google Scholar ] [ CrossRef ]
  • Crisp, D.; Pathare, A.; Ewell, R.C. The performance of gallium arsenide/germanium solar cells at the Martian surface. Acta Astronaut. 2004 , 54 , 83–101. [ Google Scholar ] [ CrossRef ]
  • Staab, M.S.; Herman, J.A.; Reich, K.; Reich, K.; Sridhar, V.; Nelson, R.W. MER Opportunity dust-storm recovery operations and implications for future Mars surface missions. In Proceedings of the 2020 IEEE Aerospace Conference, Big Sky, MT, USA, 7–14 March 2020. [ Google Scholar ]
  • Yen, A.; Gellert, R.; Schröder, C.; Morris, R.V.; Bell, J.F., III; Knudson, A.T.; Clark, B.C.; Ming, D.M.; Crisp, J.A.; Arvidson, R.E.; et al. An integrated view of the chemistry and mineralogy of Martian soils. Nature 2005 , 436 , 49–54. [ Google Scholar ] [ CrossRef ] [ PubMed ]
  • Slipher, E.C. Mars—The Photographic Story ; Northland Press: Flagstaff, AZ, USA, 1962. [ Google Scholar ]
  • Horn, D.; McAfee, J.M.; Winer, A.M.; Herr, K.C.; Pimentel, G.C. The composition of the Martian atmosphere: Minor constituents. Icarus 1972 , 16 , 543–556. [ Google Scholar ] [ CrossRef ]
  • Cimino, G.; Clavin, W.M. Calibration and analysis of Mariner 7 infrared spectra. Bull. Am. Astron. Soc. 1996 , 28 , 1068. [ Google Scholar ]
  • Hord, C.W.; Barth, C.A.; Stewart, A.I.; Lane, A.L. Mariner 9 ultraviolet spectrometer experiment: Photometry and topography of Mars. Icarus 1972 , 17 , 433–456. [ Google Scholar ] [ CrossRef ]
  • Anderson, E.; Loevy, C. Mariner 9 television limb observations of dust and ice hazes on Mars. J. Atmos. Sci. 1978 , 35 , 723. [ Google Scholar ] [ CrossRef ]
  • Martin, T.Z.; Richardson, M.I. New dust opacity mapping from Viking infrared thermal mapper data. J. Geophys. Res. 1993 , 98 , 10941–10949. [ Google Scholar ] [ CrossRef ]
  • Biggs, G.A.; Baum, W.A.; Barnes, J. Viking Orbiter imaging observations of dust in the martian atmosphere. J. Geophys. Res. 1979 , 84 , 2795. [ Google Scholar ] [ CrossRef ]
  • Malin, M.C.; Danielson, G.E.; Ravine, M.A.; Soulanille, T.A. Design and development of the Mars Observer Camera. Int. J. Imaging Syst. Technol. 1991 , 3 , 76–91. [ Google Scholar ] [ CrossRef ]
  • Malin, M.C.; Danielson, G.E.; Ingersoll, A.P.; Masursky, H.; Veverka, J.; Ravine, M.A.; Soulanille, T.A. Mars Observer Camera. J. Geophys. Res. 1992 , 97 , 7699–7718. [ Google Scholar ] [ CrossRef ]
  • Christensen, P.R.; Bandfield, J.L.; Hamilton, V.E.; Ruff, S.W.; Kieffer, H.H.; Titus, T.N.; Malin, M.C.; Morris, R.V.; Lane, M.D.; Clark, R.L.; et al. Mars Global Surveyor Thermal Emission Spectrometer experiment: Investigation description and surface science results. J. Geophys. Res. 2001 , 106 , 23823–23871. [ Google Scholar ] [ CrossRef ]
  • Tomasko, M.G.; Doose, L.R.; Lemmon, M.; Smith, P.H.; Wegryn, E. Properties of dust in Martian atmosphere from the Imager on Mars Pathfinder. J. Geophys. Res. 1999 , 104 , 8987–9007. [ Google Scholar ] [ CrossRef ]
  • Smith, P.H.; Lemmon, M. Opacity of the Martian atmosphere measured by the Imager for Mars Pathfinder. J. Geophys. Res. 1999 , 104 , 8975–8985. [ Google Scholar ] [ CrossRef ]
  • Christensen, P.R.; Jakosky, B.M.; Kieffer, H.H.; Malin, M.C.; McSween, H.Y., Jr.; Nealson, K.; Mehall, G.L.; Silverman, S.H.; Ferry, S.; Caplinger, M.; et al. The Thermal Emission Imaging System (THEMIS) for the Mars 2001 Odyssey mission. Space Sci. Rev. 2004 , 110 , 85–130. [ Google Scholar ] [ CrossRef ]
  • Määttänen, A.; Listowski, C.; Montmessin, F.; Maltagliati, L.; Reberac, A.; Joly, L.; Bertaux, L.J. A complete climatology of the aerosol vertical distribution on Mars from MEx/SPICAM UV solar occultations. Icarus 2013 , 223 , 892–941. [ Google Scholar ] [ CrossRef ]
  • Bibring, J.-P.; Soufflot, A.; Berthé, M.; Langevin, Y.; Gondet, B.; Drossart, P.; Bouyé, M.; Combes, M.; Puget, P.; Semery, A.; et al. OMEGA: Observatoire pour la Mineralogie, l’Eau, les Glaces et l’Activité. Eur. Space Agency Spec. Pub. 2004 , 1240 , 37. [ Google Scholar ]
  • Gwinner, K.; Jaumann, R.; Hauber, E.; Hoffmann, H.; Heipke, C.; Oberst, J.; Neukum, G.; Ansan, V.; Bostelmann, J.; Dumke, A.; et al. The High Resolution Stereo Camera (HRSC) of Mars Express and its approach to science analysis and mapping for Mars and its satellites. Planet. Space Sci. 2016 , 126 , 93–138. [ Google Scholar ] [ CrossRef ]
  • Lemmon, M.T.; Wolff, M.J.; Bell, J.F., III; Smith, M.D.; Cantor, B.A.; Smith, P.H. Dust aerosol, clouds, and the atmospheric optical depth record over 5 Mars years of the Mars Exploration Rover mission. Icarus 2015 , 251 , 96–111. [ Google Scholar ] [ CrossRef ]
  • Bell, J.F., III; Squyres, S.W.; Herkenhoff, K.E.; Maki, J.N.; Arneson, H.M.; Brown, D.; Collins, S.A.; Dingizian, A.; Elliot, S.T.; Hagerott, E.C.; et al. Mars Exploration Rover Athena Panoramic Camera (Pancam) investigation. J. Geophys. Res. Planets 2003 , 108 , 8063. [ Google Scholar ] [ CrossRef ]
  • Chistensen, P.R.; Mehall, G.L.; Silverman, S.H.; Anwar, S.; Cannon, G.; Gorelick, N.; Kheen, R.; Tourville, T.; Bates, D.; Ferry, S.; et al. Miniature Thermal Emission Spectrometer for the Mars Exploration Rovers. J. Geophys. Res. Planets 2003 , 108 , 8064. [ Google Scholar ]
  • Bell, J.F., III; Wolff, M.J.; Malin, M.C.; Calvin, W.M.; Cantor, B.A.; Caplinger, M.A.; Clancy, R.T.; Edgett, K.S.; Edwards, L.J.; Fahle, J.; et al. Mars Reconnaissance Orbiter Mars Color Imager (MARCI): Instrument description, calibration, and performance. J. Geophys. Res. Planets 2009 , 114 , E08S92. [ Google Scholar ] [ CrossRef ]
  • Murchie, S.; Arvidson, R.; Bedini, P.; Beisser, K.; Bibring, J.-P.; Bishop, J.; Boldt, J.; Cavender, P.; Choo, T.; Clancy, R.T.; et al. Compact Reconnaissance Imaging Spectrometer for Mars (CRISM) on Mars Reconnaissance Orbiter (MRO). J. Geophys. Res. Planets 2007 , 112 , E05S03. [ Google Scholar ] [ CrossRef ]
  • McCleese, D.J.; Schofield, J.T.; Taylor, F.W.; Calcutt, S.B.; Foote, M.C.; Kass, D.M.; Leovy, C.B.; Paige, D.A.; Read, P.L.; Zurek, R.W. Mars Climate Sounder: An investigation of thermal and water vapor structure, dust and condensate distributions in the atmosphere, and energy balance of the polar regions. J. Geophys. Res. Planets 2007 , 112 , E05S06. [ Google Scholar ] [ CrossRef ]
  • Smith, P.H.; Tamppari, L.; Arvidson, R.E.; Bass, D.; Blaney, D.; Boynton, W.; Carswell, A.; Catling, D.; Clark, B.; Duck, T.; et al. Introduction to special section on the Phoenix Mission: Landing Site Characterization Experiments, Mission Overviews, and Expected Science. J. Geophys. Res. Planets 2008 , 113 , E00A18. [ Google Scholar ] [ CrossRef ]
  • Grotzinger, J.P.; Crisp, J.; Vasavada, A.R.; Anderson, R.C.; Baker, C.J.; Barry, R.; Blake, D.F.; Conrad, P.; Edgett, K.S.; Ferdowski, B.; et al. Mars Science Laboratory mission and science investigation. Space Sci. Rev. 2012 , 170 , 5–56. [ Google Scholar ] [ CrossRef ]
  • Vandaele, A.C.; Neefs, E.; Drummond, R.; Thomas, I.R.; Daerden, F.; Lopez-Moreno, J.-J.; Rodriguez, J.; Patel, M.R.; Bellucci, G.; Allen, M.; et al. Science objectives and performances of NOMAD, a spectrometer suite for the ExoMars TGO mission. Planet. Space Sci. 2015 , 119 , 233–249. [ Google Scholar ] [ CrossRef ]
  • Thomas, N.; Cremonese, G.; Ziethe, R.; Gerber, M.; Brändli, M.; Bruno, G.; Erismann, M.; Gambicorti, L.; Gerber, T.; Ghose, K.; et al. The Colour and Stereo Surface Imaging System (CaSSIS) for the ExoMars Trace Gas Orbiter. Space Sci. Rev. 2017 , 212 , 1897–1944. [ Google Scholar ] [ CrossRef ]
  • Korablev, O.; Montmessin, F.; Trokhimovskiy, A.; Fedorova, A.A.; Shakun, A.V.; Grigoriev, A.V.; Moshkin, B.E.; Ignatiev, N.I.; Forget, F.; Lefèvre, F.; et al. The Atmospheric Chemistry Suite (ACS) of Three Spectrometers for the ExoMars 2016 Trace Gas Orbiter. Space Sci. Rev. 2018 , 214 , 7. [ Google Scholar ] [ CrossRef ]
  • Maki, J.N.; Golombek, M.; Deen, R.; Abarca, H.; Sorice, C.; Goodsall, T.; Schwochert, M.; Lemmon, M.; Trebi-Ollennu, A.; Banerdt, W.B. The Color Cameras on the InSight Lander. Space Sci. Rev. 2018 , 214 , 105. [ Google Scholar ] [ CrossRef ]
  • Banfield, D.; Rodriguez-Manfredi, J.A.; Russell, C.T.; Rowe, K.M.; Leneman, D.; Lai, H.R.; Cruce, P.R.; Means, J.D.; Johnson, C.L.; Mittelholz, A.; et al. Insight Auxiliary Payload Sensor Suite (APSS). Space Sci. Rev. 2019 , 215 , 4. [ Google Scholar ] [ CrossRef ]
  • Bell, J.F., III; Maki, J.N.; Mehall, G.L.; Ravine, M.A.; Caplinger, M.A.; Bailey, Z.J.; Brylow, S.; Schaffner, J.A.; Kinch, K.M.; Madsen, M.B.; et al. The Mars 2020 Perseverance Rover Mast Camera Zoom (Mastcam-Z) Multispectral, Stereoscopic Imaging Investigation. Space Sci. Rev. 2021 , 217 , 24. [ Google Scholar ] [ CrossRef ]
  • Maurice, S.; Wiens, R.C.; Bernardi, P.; Caïs, P.; Robinson, S.; Nelson, T.; Gasnault, O.; Reess, J.-M.; Deleuze, M.; Rull, F.; et al. The SuperCam Instrument Suite on the Mars 2020 Rover: Science Objectives and Mast-Unit Description. Space Sci. Rev. 2021 , 217 , 47. [ Google Scholar ] [ CrossRef ]
  • Rodriguez-Manfredi, J.A.; de la Torre Juárez, M.; Alonso, A.; Apéstigue, V.; Arruego, I.; Atienza, T.; Banfield, D.; Boland, J.; Carrera, M.A.; Castañer, L.; et al. The Mars Environmental Dynamics Analyzer, MEDA. A Suite of Environmental Sensors for the Mars 2020 Mission. Space Sci. Rev. 2021 , 217 , 48. [ Google Scholar ] [ CrossRef ] [ PubMed ]
  • Jones, A.R.; Wolff, M.; Alshamsi, M.; Osterloo, M.; Bay, P.; Brennan, N.; Bryant, K.; Castleman, Z.; Curtin, A.; DeVito, E.; et al. The Emirates Exploration Imager (EXI) Instrument on the Emirates Mars Mission (EMM) Hope Mission. Space Sci. Rev. 2021 , 217 , 81. [ Google Scholar ] [ CrossRef ]
  • Meng, Q.; Wang, D.; Wang, X.; Li, W.; Yang, X.; Yan, D.; Li, Y.; Cao, Z.; Ji, Q.; Sun, T.; et al. High Resolution Imaging Camera (HiRiC) on China’s first Mars exploration Tianwen-1 mission. Space Sci. Rev. 2021 , 217 , 42. [ Google Scholar ] [ CrossRef ]
  • Yu, G.B.; Liu, E.H.; Liu, G.L.; Zhou, L.; Zeng, J.Z.; Chen, Y.P.; Zhou, X.D.; Zhao, R.J.; Zhu, S.Y. Moderate Resolution Imaging Camera (MoRIC) of China’s First Mars Mission Tianwen-1. Earth Planet. Phys. 2020 , 4 , 364–370. [ Google Scholar ] [ CrossRef ]
  • Yang, J.-F.; Liu, D.-W.; Xue, B.; Lyu, J.; Liu, J.-J.; Li, F.; Ren, X.; Ge, W.; Liu, B.; Ma, X.-L.; et al. Design and ground verification for Multispectral Camera on the Mars Tianwen-1 Rover. Space Sci. Rev. 2022 , 218 , 19. [ Google Scholar ] [ CrossRef ]
  • Li, B.; Yue, Z.; Qu, S.; Yao, P.; Fu, X.; Ling, Z.; Chen, S. Spatio-Temporal Analysis of Dust Storm Activity in Chryse Planitia Using MGS-MOC Observations from Mars Years 24–28. Universe 2021 , 7 , 433. [ Google Scholar ] [ CrossRef ]
  • Guha, B.K.; Gebhardt, C.; Young, R.M.B.; Wolff, M.J.; Montabone, L. Seasonal and diurnal variations of dust storms in Martian year 36 based on the EMM-EXI database. J. Geophys. Res. Planets 2024 , 129 , e2023JE008156. [ Google Scholar ] [ CrossRef ]
  • Wang, H.; Ingersoll, A.P. Martian clouds observed by Mars Global Surveyor Mars Orbiter Camera. J. Geophys. Res. 2002 , 107 , 5078. [ Google Scholar ] [ CrossRef ]
  • Battalio, M.; Wang, H. The Mars Dust Activity Database (MDAD): A comprehensive statistical study of dust storm sequences. Icarus 2021 , 354 , 114059. [ Google Scholar ] [ CrossRef ]
  • Wu, B.; Dong, J.; Wang, Y.; Rao, W.; Sun, Z.Z.; Li, Z.; Tan, Z.; Chen, Z.; Wang, C.; Liu, W.C.; et al. Landing Site Selection and Characterization of Tianwen-1 (Zhurong Rover) on Mars. J. Geophys. Res. Planets 2022 , 127 , e2021JE007137. [ Google Scholar ] [ CrossRef ]
  • Shirley, J.H.; McConnochie, T.H.; Kass, D.M.; Kleinböhl, A.; Schofield, J.T.; Heavens, N.G.; McCleese, D.J.; Benson, J.; Hinson, D.P.; Bandfield, J.L. Temperature and aerosol opacities of the Mars atmosphere at aphelion: Validation and inter-comparison of limb sounding profiles from MRO/MCS and MGS/TES. Icarus 2015 , 251 , 26–49. [ Google Scholar ] [ CrossRef ]
  • Laseigneur, Y.; Vincendon, M. OMEGA/Mars Express: A new Martian atmospheric dust hunter. Icarus 2023 , 392 , 115366. [ Google Scholar ] [ CrossRef ]
  • Smith, M.D.; Pearl, J.C.; Conrath, B.J.; Christensen, P.R. Mars Global Surveyor Thermal Emission Spectrometer (TES) observations of dust opacity during aerobraking and science phasing. J. Geophys. Res. 2000 , 105 , 9539–9552. [ Google Scholar ] [ CrossRef ]
  • Smith, M.D.; Bandfield, J.L.; Christensen, P.R.; Richardson, M.I. Thermal Emission Imaging System (THEMIS) infrared observations of atmospheric dust and water ice cloud optical depth. J. Geophys. Res. 2003 , 108 , 5115. [ Google Scholar ] [ CrossRef ]
  • Smith, M.D. Interannual variability in TES atmospheric observations of Mars during 1999-2003. Icarus 2004 , 167 , 148–165. [ Google Scholar ] [ CrossRef ]
  • Kleinböhl, A.; Schofield, J.T.; Kass, D.M.; Abdou, W.A.; Backus, C.R.; Sen, B.; Shirley, J.H.; Lawson, W.G.; Richardson, M.I.; Taylor, F.W.; et al. Mars Climate Sounder limb profile retrieval of atmospheric temperature, pressure and dust and water ice opacity. J. Geophys. Res. 2009 , 114 , E10006. [ Google Scholar ] [ CrossRef ]
  • Millour, E.; Forget, F.; Spiga, A.; Navarro, T.; Madeleine, J.B.; Montabone, L.; Pottier, A.; Lefevre, F.; Montmessin, F.; Chaufray, J.-Y.; et al. The Mars Climate Database (MCD version 5.2). In Proceedings of the European Planetary Science Congress 2015, Nantes, France, 27 September 2015. [ Google Scholar ]
  • Kleinböhl, A.; Spiga, A.; Kass, D.M.; Shirley, J.H.; Millour, E.; Montabone, L.; Forget, F. Diurnal variations of dust during the 2018 global dust storm observed by the Mars Climate Sounder. J. Geophys. Res. Planets 2020 , 125 , e2019JE006115. [ Google Scholar ] [ CrossRef ]
  • Vandaele, A.C.; Korablev, O.; Daerden, F.; Aoki, S.; Thomas, I.R.; Altieri, F.; López-Valverde, M.; Villanueva, G.; Liuzzi, G.; Smith, M.D.; et al. Martian dust storm impact on atmospheric H 2 O and D/H observed by ExoMars Trace Gas Orbiter. Nature 2019 , 568 , 521–525. [ Google Scholar ] [ CrossRef ]
  • Information Office of the State Council of the People’s Republic of China. China’s Spaceflight in 2021 ; People’s Publishing House: Beijing, China, 2022. (In Chinese) [ Google Scholar ]
  • Kleinböhl, A.; Schofield, J.T.; Abdou, W.A.; Irwin, P.G.J.; de Kok, R.J. A single-scattering approximation for infrared radiative transfer in limb geometry in the Martian atmosphere. J. Quant. Spectrosc. Radiat. Transfer. 2011 , 112 , 1568–1580. [ Google Scholar ] [ CrossRef ]
  • He, F.; Wei, Y.; Rong, Z.; Reng, Z.; Yan, L.; Tan, N.; Wang, Y.; Fan, K.; Zhou, X.; Gao, J. Monitoring methods for Martian dust storms. Chin. Sci. Bull. 2023 , 68 , 2046–2057. (In Chinese) [ Google Scholar ] [ CrossRef ]
  • Montabone, L.; Forget, F. On forecasting dust storms on Mars. In Proceedings of the 48th International Conference on Environmental Systems, Albuquerque, NM, USA, 8–12 July 2018. [ Google Scholar ]
  • Fan, K.; Wei, Y. Prospection of multi-spacecraft planetary exploration missions in upcoming future. Chin. Sci. Bull. 2021 , 66 , 4350–4353. (In Chinese) [ Google Scholar ] [ CrossRef ]

Click here to enlarge figure

MissionTypeInstrumentsYearsReferences
Mariner 6FlybyIR Spectrometer (IRS)1969[ , ]
Mariner 7FlybyIR Spectrometer (IRS)1969[ ]
Mariner 9OrbiterIR Spectrometer (IRS)
UV Spectrometer (UVS)
Television (TV)
1971~1972[ , , ]
Viking 1Orbiter
Lander
IR Thermal Mapper (IRTM)
Visual Imaging Subsystem (VIS)
Cameras
1976~1980
1976~1982
[ , ]
Viking 2Orbiter
Lander
IR Thermal Mapper (IRTM)
Visual Imaging Subsystem (VIS)
Cameras
1976~1978
1976~1980
[ , ]
Mars Global Surveyor
(MGS)
OrbiterThermal Emission Spectrometer (TES)
Mars Orbit Camera (MOC)
1997~2006[ , , ]
Mars PathfinderLander
Rover
Imager for Mars Pathfinder (IMP)
Atmospheric Structure Instrument
Navigation Camera (ASINC)
Alpha Proton X-Ray Spectrometer (APXS)
1997
1997
[ , , ]
Mars OdysseyOrbiterThermal Emission Imaging System (THEMIS)2004~present[ ]
Mars ExpressOrbiterSpectroscopy for the Investigation of the Characteristics of the Atmosphere of Mars (SPICAM)
Observatoire pour la Mineralogie, l’Eau, les Glaces et l’Activité (OMEGA)
High Resolution Stereo Camera (HRSC)
2003~present[ , , , ]
Mars Exploration Rovers:
Spirit
Opportunity
Rover
Rover
Panoramic Cameras (Pancam)
Miniature Thermal Emission
Spectrometer (Mini-TES)
Alpha Particle X-ray Spectrometer (APXS)
2004~2009
2004~2022
[ , , ]
Mars Reconnaissance Orbiter (MRO)OrbiterMars Color Imager (MARCI)
Compact Reconnaissance Imaging Spectrometer for Mars (CRISM)
Mars Climate Sounder (MCS)
2006~present[ , , ]
PhoenixLanderSurface Stereo Imager (SSI)2008~2009[ ]
Mars Science Laboratory
(MSL) Curiosity
RoverRover Environmental Monitoring
Station (REMS)
Mast Camera (Mastcam)
2012~present[ , ]
ExoMars 2016
Trace Gas Orbiter (TGO)
OrbiterNadir and Occultation for Mars
Discovery (NOMAD)
Color and Stereo Surface Imaging
System (CaSSIS)
Atmospheric Chemistry Suit (ACS)
2016~present[ , , ]
InsightLanderColor Cameras
Auxiliary Payload Sensor Suite
2018~2022[ , ]
PerseveranceRoverMastcam-Z
SuperCam
Mars Environmental Dynamics
Analyzer (MEDA)
2021~[ , , ]
HopeOrbiterEmirates Exploration Imager (EXI)2021~[ ]
Tianwen-1OrbiterHigh-Resolution Imaging Camera (HiRiC)
Medium Resolution Imaging Camera (MoRiC)
Multispectral Camera (MSCam)
2021~[ , , ]
LevelsVDOD ValuesActions
Level 1<0.7No restrictions.
Level 20.7 < VDOD < 1.0No restrictions but watch the VDOD closely.
Level 31.0 < VDOD < 2.0Scientific activities can only be carried out within reasonable limits during the day, with no overnight activities, and the operation team searches for possible parking points for rovers and protective measures for landers.
Level 42.0 < VDOD < 3.0Start driving to a parking point to safely ride out the dust storm, allowing ONLY essential activities (VDOD observation and battery control board history).
Level 53.0 < VDOD < 3.5Final drive to parking point, minimal activities ONLY (VDOD observation and battery control board history), begin ultra-high frequency (UHF) overflights per a couple of days, and wait for VDOD to drop.
Level 6VDOD > 3.5Minimal activities ONLY (VDOD observation and battery control board history), UHF overflights per couple of days, and wait for VDOD to drop.
The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

He, F.; Rong, Z.; Wu, Z.; Gao, J.; Fan, K.; Zhou, X.; Yan, L.; Wang, Y.; Wei, Y. Martian Dust Storms: Reviews and Perspective for the Tianwen-3 Mars Sample Return Mission. Remote Sens. 2024 , 16 , 2613. https://doi.org/10.3390/rs16142613

He F, Rong Z, Wu Z, Gao J, Fan K, Zhou X, Yan L, Wang Y, Wei Y. Martian Dust Storms: Reviews and Perspective for the Tianwen-3 Mars Sample Return Mission. Remote Sensing . 2024; 16(14):2613. https://doi.org/10.3390/rs16142613

He, Fei, Zhaojin Rong, Zhaopeng Wu, Jiawei Gao, Kai Fan, Xu Zhou, Limei Yan, Yuqi Wang, and Yong Wei. 2024. "Martian Dust Storms: Reviews and Perspective for the Tianwen-3 Mars Sample Return Mission" Remote Sensing 16, no. 14: 2613. https://doi.org/10.3390/rs16142613

Article Metrics

Article access statistics, further information, mdpi initiatives, follow mdpi.

MDPI

Subscribe to receive issue release notifications and newsletters from MDPI journals

IMAGES

  1. PPT

    research instrument used in qualitative

  2. PPT

    research instrument used in qualitative

  3. CHAPTER 3 RESEARCH METHODOLOGY Components of a research

    research instrument used in qualitative

  4. (PDF) Validation Instrument for Undergraduate Qualitative Research

    research instrument used in qualitative

  5. Qualitative Research: Definition, Types, Methods and Examples (2023)

    research instrument used in qualitative

  6. RESEARCH INSTRUMENTS FOR QUANTITATIVE AND QUALITATIVE RESEARCH

    research instrument used in qualitative

VIDEO

  1. Research Instrument 1

  2. Qualitative research instrument

  3. Validation Of Research Instruments

  4. Developing the Research Instrument/Types and Validation

  5. Development of Research Instrument in the field of Education/ Social Sciences (Part one )

  6. DEVELOPING QUALITATIVE & QUANTITATIVE RESEARCH INSTRUMENT IN EDU & SPECIAL EDU

COMMENTS

  1. (PDF) QUALITATIVE DATA COLLECTION INSTRUMENTS: THE MOST ...

    [email protected], 0246502881. Abstract. Deciding on the appropriate data collection instrument to use in capturing the needed. data to address a research problem as a novice qualitative ...

  2. 9 Best Examples of Research Instruments in Qualitative Research Explained

    Visual methods, such as photography, video recording, or drawings, can be used as qualitative research instruments. These methods allow participants to express their experiences and perspectives visually, providing rich and nuanced data. Visual methods can be particularly useful in studying topics related to art, culture, or visual communication.

  3. Qualitative Research: Data Collection, Analysis, and Management

    Qualitative research is used to gain insights into people's feelings and thoughts, which may provide the basis for a future stand-alone qualitative study or may help researchers to map out survey instruments for use in a quantitative study. It is also possible to use different types of research in the same study, an approach known as "mixed ...

  4. What Is Qualitative Research?

    Qualitative research involves collecting and analyzing non-numerical data (e.g., text, video, or audio) to understand concepts, opinions, or experiences. It can be used to gather in-depth insights into a problem or generate new ideas for research. Qualitative research is the opposite of quantitative research, which involves collecting and ...

  5. Planning Qualitative Research: Design and Decision Making for New

    While many books and articles guide various qualitative research methods and analyses, there is currently no concise resource that explains and differentiates among the most common qualitative approaches. We believe novice qualitative researchers, students planning the design of a qualitative study or taking an introductory qualitative research course, and faculty teaching such courses can ...

  6. How to use and assess qualitative research methods

    Abstract. This paper aims to provide an overview of the use and assessment of qualitative research methods in the health sciences. Qualitative research can be defined as the study of the nature of phenomena and is especially appropriate for answering questions of why something is (not) observed, assessing complex multi-component interventions ...

  7. PDF Selecting and Describing Your Research Instruments

    Advisor Consultation Checklist Use the checklist below to ensure that you consulted with your advisor during the key steps in the process of selecting and describing your research instruments. 1. _____ Read this checklist. 2. _____ Made an appointment for our first meeting to discuss the instrument selection. 3.

  8. Qualitative Research

    Qualitative Research. Qualitative research is a type of research methodology that focuses on exploring and understanding people's beliefs, attitudes, behaviors, and experiences through the collection and analysis of non-numerical data. It seeks to answer research questions through the examination of subjective data, such as interviews, focus ...

  9. PDF Qualitative Research

    Chapter 1 Qualitative Research 3. Still other definitions focus on the process and context of data collection: Qualitative research is a situated activity that locates the observer in the world. It consists of a set of interpretive, material practices that makes the world visible. These practices transform the world.

  10. Qualitative Methods Used to Generate Questionnaire Items: A Systematic

    However, since the introduction of recommendations for qualitative methods to generate items for questionnaires (Brod et al., 2009; Lasch et al., 2010; Patrick et al., 2011a, 2011b) and in the health research literature (Mays & Pope, 1995, 2000; Pope et al., 2000), we have lacked a comprehensive overview of current practices regarding the qualitative methods actually used in the development of ...

  11. Qualitative Research: Your Ultimate Guide

    Qualitative research methods can use structured research instruments for data collection, like: Surveys for individual views. A survey is a simple-to-create and easy-to-distribute qualitative research method, which helps gather information from large groups of participants quickly. Traditionally, paper-based surveys can now be made online, so ...

  12. PDF Research Instrument Examples

    What is a Research Instrument? A Research Instrument is a tool used to collect, measure, and analyze data related to your research interests. These tools are most commonly used in health sciences, social sciences, and education to assess patients, clients, students, teachers, staff, etc. A research instrument can include interviews, tests,

  13. PDF QUALITATIVE DATA COLLECTION INSTRUMENTS: THE MOST ...

    The Qualitative Research Instrument I find Most Challenging to Use Among the data collection instruments highlighted in the previous section, I find the documentary analysis guide, which is used ...

  14. 8 Essential Qualitative Data Collection Methods

    1. Interviews. One-on-one interviews are one of the most commonly used data collection methods in qualitative research because they allow you to collect highly personalized information directly from the source. Interviews explore participants' opinions, motivations, beliefs, and experiences and are particularly beneficial in gathering data on ...

  15. Methods of data collection in qualitative research: interviews and

    Qualitative research in dentistry This paper explores the most common methods of data collection used in qualitative research: interviews and focus groups. The paper examines each method in detail ...

  16. Qualitative Data Collection: What it is + Methods to do it

    Qualitative data is defined as data that approximates and characterizes. It can be observed and recorded. This data type is non-numerical in nature. This type of data is collected through methods of observations, one-to-one interviews, conducting focus groups, and similar methods. Qualitative data in statistics is also known as categorical data ...

  17. LibGuides: Research Methodologies: Research Instruments

    A research instrument is a tool you will use to help you collect, measure and analyze the data you use as part of your research. The choice of research instrument will usually be yours to make as the researcher and will be whichever best suits your methodology. There are many different research instruments you can use in collecting data for ...

  18. What is Qualitative Research?

    It can be used to gather in-depth insights into a problem or generate new ideas for research. It answers the hows and whys instead of how many or how much. It could be structured as a stand-alone study, purely relying on qualitative data or it could be part of mixed-methods research that combines qualitative and quantitative data.

  19. How to use and assess qualitative research methods

    Quality assessment. This paper aims to provide an overview of the use and assessment of qualitative research methods in the health sciences. Qualitative research can be defined as the study of the nature of phenomena and is especially appropriate for answering questions of why something is (not) observed, assessing complex multi-component ...

  20. What is a research instrument?

    A research instrument is a tool used to obtain, measure, and analyze data from subjects around the research topic. You need to decide the instrument to use based on the type of study you are conducting: quantitative, qualitative, or mixed-method. For instance, for a quantitative study, you may decide to use a questionnaire, and for a ...

  21. Questionnaire Design

    Revised on June 22, 2023. A questionnaire is a list of questions or items used to gather data from respondents about their attitudes, experiences, or opinions. Questionnaires can be used to collect quantitative and/or qualitative information. Questionnaires are commonly used in market research as well as in the social and health sciences.

  22. LibGuides: HOW to Find Research Instruments: Home

    ISBN: 9780203422991. Publication Date: 2003-12-08. Clear, accessible and practical, this guide introduces the first-time researcher to the various instruments used in social research. It assesses a broad range of research instruments - from the well-established to the innovative - enabling readers to decide which are particularly well suited to ...

  23. Home

    A research instrument is a tool used to collect, measure, and analyze data related to your subject. Research instruments can be tests, surveys, scales, questionnaires, or even checklists. To assure the strength of your study, it is important to use previously validated instruments! Getting Started. Already know the full name of the instrument ...

  24. (PDF) Qualitative data instruments and validity issues

    Department of Special Needs and Inclusive Education. Email: [email protected]. Title. Qualitative Data Collection Methods and Traditional Vs Alternative Standards for Judging. Quality of a ...

  25. Other Research Methods

    About CCQDER additional research methods. Cognitive interviewing is a popular method for evaluating survey questions.In addition to cognitive interviewing, Collaborative Center for Questionnaire Design and Evaluation Research (CCQDER) researchers can also use several other methods to evaluate questions.

  26. Unlocking potential: a qualitative exploration guiding the

    The exploratory qualitative study used semi-structured interviews with key opinion leaders, decision makers, facilitators, recipients, and frontline implementers, who had influence and involvement in the implementation of professional role substitution models. Data analysis was guided by the Consolidated Framework for Implementation Research ...

  27. Risk and threat assessment instruments for violent extremism: A

    Frontline law enforcement, police, and security personnel of various backgrounds have the challenging task to identify extremists who have a high risk for committing violent acts, describe driving risk trajectories, prioritize the use of scarce resources, and develop individualized risk management plans. In this line of work, risk and threat assessment instruments are frequently used to ...

  28. JMIR Public Health and Surveillance

    Background: Colorectal cancer (CRC) is the second most deadly form of cancer, inducing an estimated 1.9 million incidence cases and 0.9 million deaths worldwide in 2020. Despite the availability of screening tests, their uptake remains suboptimal. However, blood-based tests that look for signs of cancer-specific markers in the body are increasingly available as an alternative for more invasive ...

  29. Federal Register :: Order Granting Conditional Substituted Compliance

    Start Preamble AGENCY: Commodity Futures Trading Commission. ACTION: Order. SUMMARY: On February 5, 2024, the Commodity Futures Trading Commission issued a notice and request for comment on an application submitted by the Institute of International Bankers, International Swaps and Derivatives Association, and Securities Industry and Financial Markets Association requesting that the Commission ...

  30. Remote Sensing

    Dust storms play a crucial role in the climate system and the space environment of Mars, significantly impacting human exploration activities on the planet. The Martian dust storms exhibit significant regional, seasonal and interannual variations due to various controlling factors such as large-scale atmospheric circulation, varying solar radiation forcing, and Martial orbital and rotational ...