research expertise meaning

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• My Account Login
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Open access
• Published: 13 January 2020

Expertise in research integration and implementation for tackling complex problems: when is it needed, where can it be found and how can it be strengthened?

• Gabriele Bammer   ORCID: orcid.org/0000-0001-9098-0951 1 , 2 ,
• Michael O’Rourke   ORCID: orcid.org/0000-0002-4629-0811 3 ,
• Deborah O’Connell 4 ,
• Linda Neuhauser 5 ,
• Gerald Midgley 6 , 7 , 8 , 9 , 10 , 11 ,
• Julie Thompson Klein 12 , 13 ,
• Nicola J. Grigg   ORCID: orcid.org/0000-0002-7601-3866 4 ,
• Howard Gadlin 14 ,
• Ian R. Elsum   ORCID: orcid.org/0000-0002-0954-4800 15 , 16 ,
• Marcel Bursztyn   ORCID: orcid.org/0000-0002-2680-9145 17 , 18 ,
• Elizabeth A. Fulton 19 , 20 ,
• Christian Pohl 13 , 21 ,
• Michael Smithson 22 ,
• Ulli Vilsmaier 23 ,
• Matthias Bergmann 24 , 25 ,
• Jill Jaeger 26 ,
• Femke Merkx 27 ,
• Bianca Vienni Baptista 13 ,
• Mark A. Burgman 28 ,
• Daniel H. Walker 29 ,
• John Young 30 ,
• Hilary Bradbury 31 ,
• Lynn Crawford 32 , 33 , 34 ,
• Budi Haryanto   ORCID: orcid.org/0000-0001-5208-5980 35 ,
• Cha-aim Pachanee 36 ,
• Merritt Polk 37 &
• George P. Richardson 38  

Palgrave Communications volume  6 , Article number:  5 ( 2020 ) Cite this article

16k Accesses

81 Citations

68 Altmetric

Metrics details

• Complex networks
• Science, technology and society
• Social policy

Expertise in research integration and implementation is an essential but often overlooked component of tackling complex societal and environmental problems. We focus on expertise relevant to any complex problem, especially contributory expertise, divided into ‘knowing-that’ and ‘knowing-how.’ We also deal with interactional expertise and the fact that much expertise is tacit. We explore three questions. First, in examining ‘when is expertise in research integration and implementation required?,’ we review tasks essential (a) to developing more comprehensive understandings of complex problems, plus possible ways to address them, and (b) for supporting implementation of those understandings into government policy, community practice, business and social innovation, or other initiatives. Second, in considering ‘where can expertise in research integration and implementation currently be found?,’ we describe three realms: (a) specific approaches, including interdisciplinarity, transdisciplinarity, systems thinking and sustainability science; (b) case-based experience that is independent of these specific approaches; and (c) research examining elements of integration and implementation, specifically considering unknowns and fostering innovation. We highlight examples of expertise in each realm and demonstrate how fragmentation currently precludes clear identification of research integration and implementation expertise. Third, in exploring ‘what is required to strengthen expertise in research integration and implementation?,’ we propose building a knowledge bank. We delve into three key challenges: compiling existing expertise, indexing and organising the expertise to make it widely accessible, and understanding and overcoming the core reasons for the existing fragmentation. A growing knowledge bank of expertise in research integration and implementation on the one hand, and accumulating success in addressing complex societal and environmental problems on the other, will form a virtuous cycle so that each strengthens the other. Building a coalition of researchers and institutions will ensure this expertise and its application are valued and sustained.

Similar content being viewed by others

Process expertise in policy advice: Designing collaboration in collaboration

Giulia Molinengo, Dorota Stasiak & Rebecca Freeth

A tool for reflecting on research stances to support sustainability transitions

L. Hazard, M. Cerf, … P. Steyaert

Co-producing the science–policy interface: towards common but differentiated responsibilities

Timo Y. Maas, Annet Pauwelussen & Esther Turnhout

Introduction

‘Interdisciplinarity’ and ‘transdisciplinarity’ are widely heralded as key to research addressing complex societal and environmental problems, such as reducing the gap between rich and poor, combating illicit drug use, controlling spiralling health care costs and achieving sustainable social-ecological systems (Gibbons et al., 1994 ; Jacob, 2015 ; Ledford, 2015 ; National Academy of Sciences, National Academy of Engineering and Institute of Medicine, 2005 ). In these situations, the terms ‘interdisciplinarity’ and ‘transdisciplinarity’ are used generically to indicate that different strands of disciplinary and other knowledge (e.g., from policy makers and affected communities) need to be brought together and acted upon. Implicit, but largely unrecognised, is required expertise in (1) research integration to develop a more comprehensive understanding of the problem and possible ways to address it and (2) research implementation to improve the situation.

Poor understanding of expertise needed for research integration and implementation makes assessing interdisciplinarity and transdisciplinarity difficult at all levels, including tenure and promotion applications, funding proposals, outcomes of research projects, and outputs of inter- and transdisciplinary centres and other institutions (British Academy Working Group on Interdisciplinarity, 2016 ; Bursztyn and Drummond, 2014 ; Klein and Falk-Krzesinski, 2017 ; Lyall, 2019 ; McLeisch and Strang, 2016 ). For example, inadequate understanding of what interdisciplinarity involves and how to assess it may explain why interdisciplinary grant applications have lower success rates than discipline-based proposals (Bammer, 2016a ; Bromham et al., 2016 ; Reckling and Fischer, 2010 ).

It is tempting to blame reviewers for assessment problems, labelling them as hostile or ignorant. Instead, we argue that those researching complex societal and environmental problems must ensure that expertise in research integration and implementation is well articulated, accessible and useable. These tasks require a major effort, especially as defining ‘expertise’ is far from straight-forward. Our aim is to lay foundations for further work by exploring three questions:

When is expertise in research integration and implementation required?

Where can expertise in research integration and implementation currently be found?

What is required to strengthen expertise in research integration and implementation?

We open a discussion rather than being prescriptive and provide enough detail to give the ideas substance while inviting input and further development by others practising research integration and implementation.

Our starting point is that complex societal and environmental problems are generally investigated by teams made up of disciplinary experts and increasingly they include stakeholders affected by the problem, as well as those in a position to do something about it. We argue that some team members must have expertise in research integration and implementation to effectively harness the contributions of the full team. In this article, we start to tease out what that expertise entails.

In doing so, our focus is expertise that is not specifically about the problem being tackled, therefore we leave to one side understanding of the problem itself, be it climate change, organised crime or some other complex issue. Instead we are interested in the required expertise in research integration and implementation that is relevant to tackling any complex societal or environmental problem.

We explore three components of such expertise. Most of our focus is on contributory expertise, which Collins and Evans ( 2002 , 2007 ) define as the expertise required to make a substantive contribution to a field. We divide contributory expertise into ‘knowing-that’ and ‘knowing-how’ (Gobet, 2015 ). For research integration and implementation, ‘knowing-that’ involves understanding what is required to deal with complex societal and environmental problems in an integrated way, such as knowing to look for interconnections with other problems and to explore political, economic, historical and other circumstances. ‘Knowing-how’ involves knowing which methods or processes to use in a particular context, along with skills in those methods and processes, such as building a model to describe the problem, or processes for engaging decision-makers in discussing research results. Of course, we acknowledge that knowing-that and knowing-how are in practice inseparable; nevertheless, distinguishing them helps us illuminate critical aspects of expertise in research integration and implementation.

Interactional expertise is a second component of expertise in research integration and implementation. This is the ability to understand disciplines, professional practice and community experience without being trained in those disciplines or professions or having lived in those communities (Collins and Evans, 2007 ). Interactional expertise is required to work effectively and knowledgeably with a team.

Third, contributory and interactional expertise are often tacit, in which case their inputs to thought and action are difficult to access and identify reflexively (Collins and Evans, 2007 ). Collins and Evans argue that expertise becomes tacit through the process of achieving expert status, which involves an internalisation of knowledge and skills, along with a fluidity in applying them. In this article we are also interested in tacit expertise as a component of learning-by-doing, which is a common way of achieving expertise in research integration and implementation. As we describe below, many researchers find themselves in roles requiring integration and implementation and develop skills on the fly without paying much conscious attention to them.

In opening up this discussion on expertise, our target audience is the many researchers who investigate complex societal and environmental problems and who are interested in the integration and implementation role. We want to kick-start a process of understanding and building expertise in research integration and implementation that involves newcomers through to established researchers. This scope is essential both to improving action-oriented research on complex societal and environmental problems and to recognising and rewarding properly those who undertake the integration and implementation.

As an authorship group, we illustrate, at a small scale, the challenges that this article seeks to highlight and address. Despite our common interests in research integration and implementation, we have not found it easy to articulate our own expertise. Further, many of us were not aware of each other’s contributions until we came together as invitees at the 2013 First Global Conference on Research Integration and Implementation (Integration and Implementation Sciences, 2019a ). That conference made evident the extensive array of integration and implementation expertise that has been developed for tackling complex problems and how much more effective research could be if it could draw on the full range, rather than a partial selection. Figuring out how to address these challenges has motivated our work on this article.

We start by identifying research tasks that lie outside the remit of traditional disciplines and that require expertise in research integration and implementation. We then identify three realms where such expertise currently resides, demonstrating that expertise is highly fragmented both within and across these realms. Members of the authorship group play leading roles in each of these realms.

To overcome fragmentation and strengthen expertise, we propose building a shared knowledge bank of expertise. A knowledge bank would have several major benefits. It would strengthen relevant expertise by bringing together different ways in which research integration and implementation are conceived and put into practice. It would also make expertise more visible and accessible. Further, it would unite relevant individuals and groups enabling them to provide an authoritative voice to research policy makers and funders about properly recognising, valuing and evaluating the integration and implementation expertise required to deal with complex societal and environmental problems.

We highlight key challenges in developing a knowledge bank: compiling existing expertise, indexing and organising the expertise to make it widely accessible, and understanding and overcoming the core reasons for the existing fragmentation. We close by describing the potential for a virtuous cycle between establishing a knowledge bank of expertise in research integration and implementation and increasing success in tackling complex societal and environmental problems.

Question 1: when is expertise in research integration and implementation required?

Addressing complex societal and environmental problems requires specific expertise over and above that contributed by disciplines, but there is little formal recognition of what that expertise is or reward for contributing it to a research team’s efforts. Our focus in this section is on tasks that require expertise in research integration and implementation, along with an indication of what that expertise involves, both in working with discipline-based experts and stakeholders and in dealing with the complexity of the problems.

The end points of research integration and implementation are to:

develop a more comprehensive understanding of the problem, plus possible ways to address it, by integrating disciplinary and stakeholder perspectives, and

support implementation of that understanding into evidence-informed government policy, professional and community practice, business and social innovation and other measures.

How best to achieve these goals (e.g., sequentially or concurrently) and which perspectives and implementers to involve are part of the required expertise in research integration and implementation. Expertise is also required to manage different start points, in other words whether the research is initiated and defined by the researchers, implementers (e.g., government policy makers), stakeholders affected by the problem or a combination of these.

An indication of the expertise needed can be gained by considering the complex problem of illicit drug use. One important task is to identify—using an amalgamation of know-that contributory expertise and interactional expertise—relevant discipline-based researchers and stakeholders, each of whom has an important, but only partial, understanding of the problem. Useful disciplinary inputs may include knowledge about drug effects from pharmacologists, estimates of levels of use in the population from epidemiologists, impacts on property theft and other crime from criminologists, information about regulations and laws from legal experts, and analysis of how those laws came into being from historians. Additionally, contributions to understanding come from two main groups of stakeholders: those affected by the problem, such as illicit drug users, and professional groups dealing with the problem, such as treatment and other service providers, police officers, and policy makers. As well as identifying useful perspectives, expertise in research integration and implementation is required to integrate them (know-how expertise), which includes assessing where perspectives align and where they conflict, and finding a way through conflicts.

Expertise in research integration and implementation is also required to assess and combine suggestions for action, determine strengths and risks, and decide whether the suggestions need to be supplemented by new ideas elicited through processes to spark innovation. This generally needs contributory know-how expertise. Know-how expertise, complemented by interactional expertise, is also required to identify various implementation options (through government, business and/or civil society, and through policy and/or practice change), as well as suitable implementation pathways, which can range from effective communication strategies for presenting results to decision-makers to using co-creative processes with decision-makers from the outset.

Expertise in research integration and implementation also requires the ability to embrace the challenges posed by ‘wicked problems’ (Rittel and Webber, 1973 ), also referred to as ‘messes’ (Ackoff, 1974 ). These are core to what we refer to as ‘complex’. (We eschew the terms ‘wicked’ and ‘messes’ because they are difficult to translate into other languages.) Based on our experience and key literature (Ackoff, 1974 ; Churchman, 1970 ; Cilliers, 1998 ; Horn and Weber, 2007 ; Midgley, 2000 ; Rittel and Webber, 1973 ; Ulrich, 1983 ), we propose five particular challenges that complex problems present and that need specific expertise in order to be understood and managed, and again illustrate them using the problem of illicit drug use, drawing on Babor et al. ( 2018 ), Ritter et al. ( 2017 ) and Stevens ( 2010 ).

Delimiting the problem.

Know-that expertise is required to understand that complex real-world problems have no natural boundaries and that problems have many disparate causes, which are tangled and not easily apparent or readily inferred. For example, prevention of illicit drug use needs to account for, among other things, the legacy of childhood sexual abuse, influences of popular culture, youthful rebellion and peer pressure. Know-that expertise also includes understanding that (1) addressing a single aspect of the problem causes changes in other aspects and may lead to the emergence of new issues, (2) the problem and the system in which it is embedded evolve and (3) from both a research and an action perspective, everything cannot be dealt with, so artificial but necessary boundaries must be set. Know-how and interactional expertise are then required to draw out (from disciplinary and stakeholder subject matter experts) what the relevant interconnections are, what issues may emerge, what changes are likely, as well as to help set effective boundaries around the problem.

Managing contested problem definitions.

Know-that expertise is required to appreciate that the various parties involved in a complex societal or environmental problem have different ideas about the ‘real’ problem and its causes. For example, some see illicit drug use as a crime that results from the failure of individuals to take responsibility for adhering to laws meant to protect them. Others perceive laws as the heart of the problem, driving growth of organised crime and preventing a relatively innocuous activity from being controlled by social and cultural norms. Still others argue that illicit drug use results from a brain disorder that requires medical treatment. Know-that expertise entails understanding that definitional challenges are intrinsic to any complex problem and can only be effectively dealt with by understanding the history of conflict around the problem and its impact on the ability of groups with different perspectives to trust, listen to, and engage with each other. Know-how expertise, in turn, is needed to interact with different perspectives, to manage conflicts among them, and to provide an understanding of how they may affect decisions taken.

Managing critical, unresolvable unknowns.

Appreciating that it is not possible to know everything about a complex real-world problem is another dimension of know-that expertise. First, not everything that could be known will be investigated, because there is not enough research capacity, funding or interest to address every conceivable, and potentially important, question. Second, some critical issues cannot be researched effectively. For example, there are few feasible entry points for examining links among illicit drug use, organised crime and funding for terrorism. Third, interpretations of available information often conflict. Know-how expertise is then required to identify and chart a way of managing unknowns, so that they do not lead to adverse unintended consequences or nasty surprises.

Managing real-world constraints on ameliorating the problem.

Know-that expertise is required to appreciate that ideological, cultural, political, economic and other circumstances constrain how any complex real-world problem can be tackled, and also limit the influence of research-based evidence. In addition, options for moving forward are often hampered by current ways of managing the problem and may change the distribution of benefits and losses amongst the parties involved. Further, effectively addressing the problem often requires action across multiple poorly connected organisations. Know-that expertise therefore includes awareness of generic factors that play out in specific ways depending on the problem at hand. Such factors include the impact of laws and international treaties (which e.g., restrict options for action on illicit drugs), the importance of resources (where shifting resources, e.g., between law enforcement and health, can be challenging) and the necessity and difficulties of multi-sector collaboration (e.g., across law enforcement, health, social welfare and education). Know-that expertise is also required to appreciate that the multi-faceted circumstances in which a problem is embedded can make it resistant to change, as well as that those involved in dealing with the problem are likely to disagree about which constraints are open to modification. Know-how expertise, for its part, is required to find openings for doing things differently and to overcome resistance to change.

Appreciating and accommodating the partial and temporary nature of solutions.

Finally, know-that expertise is required to understand that no effort to tackle a complex real-world problem can take all aspects of complexity into account and that any way of moving forward will cause changes in interconnected problems, sacrifice a way of seeing the problem that some stakeholders want to preserve or even hold as non-negotiable, open the door to adverse unintended consequences and miss some real-world constraints. It also requires appreciation that the search is for best-possible or least-worst, rather than definitive, solutions. Know-how expertise is required to identify and address these limitations to understanding and action.

Given that complex societal and environmental problems are generally investigated by teams made up of disciplinary experts and stakeholders, an important question is ‘who in the team needs to have research integration and implementation expertise?’. We do not explore this question in detail here, noting that one of us (Bammer, 2013 ) has written about the advantages of developing a new discipline of Integration and Implementation Sciences (i2S) that would train experts in research integration and implementation as members of teams tackling complex problems.

In any case, identifying the range of expertise required and systematically considering how to include it in teams tackling complex societal and environmental problems will be an advance on the current shortcomings that stem from teams tending to rely on the happenstance of what their members know, are interested in, and consider to be important.

Question 2: where can expertise in research integration and implementation currently be found?

We have identified three major realms where expertise in research integration and implementation can be found and how they correspond (or not) to communities of researchers. First, some researchers apply specific approaches to tackling complex societal and environmental problems, such as interdisciplinarity, systems thinking, and action research. These approaches have coalesced around particular ways of understanding and operationalising research integration and implementation. Each community practising a specific approach is largely independent of the others.

Second, some researchers develop case-based experience without reference to specific approaches and, by moving from one problem to another, progressively build useful know-that and know-how expertise (often tacit) in research integration and implementation, along with interactional expertise. From time to time they may incorporate know-that and know-how developed by others into their practice. Unlike researchers using specific approaches, researchers drawing primarily on case-based experience tend not to be organised into communities around expertise in research integration and implementation, although they may be organised into communities around the problems of interest. We recognise that the communities using specific approaches to complex societal and environmental problems also work on cases (see for example Fulton et al. ( 2014 ) for a case using complex systems science and Neuhauser ( 2018 ) for a case using transdisciplinarity), but that is tangential to the point we make here.

The final source of expertise comes from researchers who investigate an element of research integration and implementation and who are not aligned with either of the other realms. We focus here on two examples—researchers interested in unknowns and those interested in innovation. In both cases, researchers come from various disciplinary and professional backgrounds. These examples differ in the strength of the associated community (measured by regular conferences and publishing in specific journals), which is weak in the case of unknowns and stronger for innovation. In both cases, interest in unknowns or innovation is not specifically focused on complex societal and environmental problems and the relevance of their insights to research integration and implementation may not be immediately obvious.

Of course, the three realms do not have hard boundaries and researchers may identify with different realms at different times in their careers. The point of identifying these three realms where expertise in research integration and implementation exists is to highlight both the existing fragmentation, as well as which veins need to be tapped into to draw together what is already available, especially in relevant know-that and know-how contributory expertise, and to illustrate where interactional expertise and tacit expertise are important.

Specific approaches

One rich source of insights into expertise in research integration and implementation can be found in what we call specific approaches. The lists in Box 1 come from a sub-group of the authors who drew on several centuries of combined experience and scholarship, as well as their roles in helping develop some of these approaches (marked with an ‘a’).

The first column in Box 1 records 14 approaches that provide a wide range of expertise across both research integration and implementation. These approaches include action research, integrated assessment and post-normal science. The ten approaches in the second column provide a subset of expertise in research integration and implementation. Some provide expertise in research implementation only (change management, impact assessment, impact evaluation, implementation science, K* and policy science). Others provide expertise across both research integration and implementation, but only for a specific set of activities, rather than the broad range of expertise provided by the approaches listed in the first column. In particular, three approaches provide expertise in decision making and/or dealing with risk (decision making under deep uncertainty, decision sciences and risk analysis) and one in collaboration (science of team science). Multiple related approaches are listed under one specific approach in each column, namely systems thinking and K*. Many other approaches encompass various schools of thinking as well, but these have not split into separate but related approaches.

While we have listed all the specific approaches we are aware of, we anticipate that this list is not complete and that there are elements which could be contested. Further, we have not aimed to be comprehensive in the cited references, but rather have provided a major work (occasionally more) as a starting point for those interested in learning about each approach.

Before proceeding, it is important to recognise that the terms ‘interdisciplinarity’ and ‘transdisciplinarity’ are used in two different ways in the literature. In the introduction we used them in the generic sense for any research that brings together different strands of disciplinary and other knowledge and supports action based on this improved understanding. Now, unless otherwise specified, we use the second connotation, which refers to specific research approaches with established canons of scholarly work. It is also worth noting that in its specific sense, the term transdisciplinarity is used in multiple ways, including the development of new theoretical paradigms (e.g., general system theory) and methodologies that transcend disciplinary boundaries; critiques of existing structures of knowledge; building new integrative frameworks and research strategies; and involving stakeholders in both research on complex problems or phenomena and the implementation of solutions. The definition we use is provided in Box 2 .

Specific approaches have at least one of the following characteristics:

they are associated with one or more professional associations or networks, and often with peer-reviewed journals and conferences; and

handbooks or other major academic works describe them.

Three examples, two from the first column in Box 1 and one from the second column, provide an illustration, while also demonstrating the different stages of development of these three approaches and the high level of fragmentation in the systems thinking community.

Systems thinkers, systems scientists and systems engineers have formed many international associations, including the Complex Systems Society; the IEEE (Institute of Electrical and Electronics Engineers) Systems Council; the IEEE Systems, Man and Cybernetics Society; the International Council on Systems Engineering; the International Federation for Systems Research; the International Society for Knowledge and Systems Sciences; the International Society for the Systems Sciences; the System Dynamics Society; and the World Organisation of Systems and Cybernetics. There are also numerous national bodies. Most societies run annual conferences. Journals include Cybernetics and Systems ; International Journal of General Systems ; International Journal of Knowledge and Systems Sciences ; System Dynamics Review; Systemic Practice and Action Research ; Systems ; and Systems Research and Behavioural Science . Major reference works include an encyclopaedia (François, 2004 ) and various volumes of classic and contemporary reprints covering the whole field (Beishon and Peters, 1972 ; Buckley, 1965 ; Emery, 1969 , 1981 ; Midgley, 2003 ).

Interdisciplinarians have formed the Association for Interdisciplinary Studies and Intereach (Interdisciplinary Integration Research Careers Hub). The Association for Interdisciplinary Studies publishes the journal Issues in Interdisciplinary Studies and runs an annual conference. The Oxford Handbook of Interdisciplinarity (Frodeman, 2017 ) is now in its second edition.

Proponents of the science of team science have formed an International Network for the Science of Team Science and manage a listserv and annual conference. The major reference, produced in the US, is a report by the National Research Council ( 2015 ).

A detailed list of professional associations and networks, along with their journals and conferences, is available for most specific approaches at Integration and Implementation Sciences ( 2019b ).

It is beyond the scope of this article to review these approaches. Instead, the aim of the lists in Box 1 is to demonstrate that there are many specific approaches that can provide know-that and know-how contributory expertise. Teasing out that expertise is a task for future research.

In Box 2 we describe examples of contributory expertise drawing on some of the specific approaches that we know best: interdisciplinarity, sustainability science, systemic intervention and transdisciplinarity. For each example of contributory expertise (e.g., the ‘three types of knowledge tool’ developed in transdisciplinary research) we also briefly describe when that expertise would be useful, referring back to the issues raised in addressing Question 1 (When is expertise in research integration and implementation required?). At this stage, we have not attempted to be systematic or detailed, but instead have aimed to highlight examples of expertise that we expect will resonate with our target audience. For this reason we focus on contributory expertise, except for Box 3 below. We also want to provide a sense of what compiling expertise will involve and that this compilation task is ripe for further work.

Case-based experience that is independent of specific approaches

Many researchers investigating complex real-world problems build their expertise by tackling particular problems without any real appreciation of the know-that and know-how offered by the specific approaches described above. This case-based expertise involves learning-by-doing and is generally augmented from project to project. Some codify the expertise they develop, while for others it remains largely tacit knowledge.

In Box 3 we provide brief descriptions of case-based expertise developed by members of the authorship group. This provides an opportunity to highlight both interactional expertise (usually tacit) and tacit contributory expertise, which are easier to identify in case-based experience than in the other realms. For each case we describe the aims of the research, along with the outcomes of the research integration and research implementation. We then highlight which disciplines and stakeholders were involved, requiring interactional expertise to effectively draw on their contributions. Finally, we highlight examples of the tacit contributory expertise developed in addressing the problem. That tacit expertise was made explicit through the process of writing this article.

As an aside, reviewing both Case 1 and Box 2 illustrates that researchers can move between realms. One of us (O’Connell) used the experience gained in case-based research to subsequently contribute to the development of sustainability science.

In Box 4 , we describe two examples of codified expertise developed from case-based experience. The first example is a compilation of collaboration methods largely based on lessons learnt in resolving conflicts among researchers at the US National Cancer Institute (Bennett et al., 2018 ). From this compilation we specifically describe a set of know-that understandings that can be used to underpin ‘pre-nuptial’ agreements for research collaborators. The second example comes from a wide array of projects undertaken by the RAPID (Research and Policy in Development) programme of the UK’s Overseas Development Institute, which led to the development of a toolkit for engaging and influencing policy (Young et al., 2014 ), from which we describe one know-how method (alignment, interest and influence matrix).

Elements of research integration and implementation: considering unknowns and enhancing innovation

Expertise for tackling complex societal and environmental problems also comes from investigations on specific elements of research integration and implementation undertaken by researchers who are not involved in developing specific approaches or case-based experience. Here, we review expertise developed in research on unknowns and on enhancing innovation as two examples. We particularly emphasise considering unknowns, which is an area that is generally poorly understood and under-researched (Smithson, 1989 ), despite its critical importance in dealing with complex problems. Enhancing innovation is also of major importance in finding new, creative ways for understanding and acting on complex societal and environmental problems. These are also areas in which two of our authorship group specialise. Of course, expertise in considering unknowns and enhancing innovation can also be developed by those involved in specific approaches and case-based experience, but this is tangential to the point we aim to make here.

In Box 5 , we provide two examples of know-that and three examples of know-how expertise developed by researchers considering unknowns. The first example of know-how is a compilation of strategies for accepting unknowns, while the other two are specific methods. In Box 6 , we present two examples of know-that expertise and one of know-how expertise developed by researchers considering innovation. In both boxes, we also indicate when this expertise is required in tackling complex societal and environmental problems, referring back to the matters raised in addressing Question 1 (When is expertise in research integration and implementation required?).

Making an inventory of expertise in each of the three realms is beyond the scope of this article, but, as we argue in the next section, is a first step in strengthening expertise in research integration and implementation. We have set out to provide enough examples to demonstrate that considerable expertise already exists and to provide an indication of the effort required for an effective compilation exercise. In both this and the previous section, we have also aimed to provide a sense of the extent and diversity of the expertise required for effective research integration and implementation. On the other hand, there is also considerable overlap in know-that and know-how expertise developed by different communities and groups, which also has to be dealt with in any compilation exercise.

Box 1 Specific approaches to tackling complex real-world problems, divided into those that provide wide-ranging expertise in both research integration and implementation and those that provide expertise in a subset of research integration and implementation activities

• a Approaches that members of the authorship group have helped develop

Box 2 Examples of expertise from four specific approaches: interdisciplinarity, sustainability science, systemic intervention and transdisciplinarity. Brief descriptions of the approaches are also provided, along with when the expertise described is useful in tackling complex societal and environmental problems

Box 3 three examples of cases tackling complex societal or environmental problems, which led to the development of tacit contributory and interactional expertise in research integration and implementation.

CASE 1 . Assessing the potential for biomass to provide sustainable bioelectricity and biofuels, and greenhouse gas emission reduction in Australia (2005 – 2016) ( Crawford et al., 2016 ; Farine et al., 2012 ; Hayward et al., 2015 ; O’Connell et al., 2009 )

Aim: (1) to provide credible quantification of benefits, sustainability impacts and opportunities of biofuels, (2) to assess a range of emerging technology options and (3) to provide reliable knowledge upon which industry and government could base their decisions.

Research integration outcomes: research findings were integrated across multiple value chains, multiple industry sectors and different types of emerging technology, aquatic and terrestrial production systems, time periods from current to future, different types of stakeholders, and local to global scales.

Research implementation outcomes: project results influenced positions of various stakeholders and their investments: the blueprint for aviation industry targets and commitments (Graham et al., 2011 ), national research and development plans for Australia (Bioenergy Research, Development and Extension Advisory Forum and Technical Working Groups, 2014 ; O’Connell and Haritos, 2010 ; O’Connell et al., 2007 ) and the international standard for sustainability ‘Sustainability Criteria for Bioenergy’ ISO 13065:2015 (International Standards Association ISO, 2015 ).

Interactional expertise was required to work with the disciplines of: agriculture, forestry, hydrology, greenhouse gas accounting, process engineering, chemistry, biochemistry, mathematics, economics.

Interactional expertise was required to work with stakeholders from: energy, agriculture and forestry sectors; aviation; companies and large global corporations; state and national governments; non-government organisations, including World Wildlife Fund and Australian Conservation Foundation; international governments; and various local communities.

Example of tacit know-that: Knowledge about the necessity to look for different delimitations and definitions of the problem being addressed among the multiple groups involved.

Example of tacit know-how: Skills to work with different stakeholder groups in different ways to support the research implementation. For example, changing the international standard involved attending standard-setting meetings and being hands-on in the trade-offs and wordsmithing involved.

CASE 2 . Reducing air pollution and improving health in Jakarta, Indonesia (2004 – 2009) (Haryanto, 2013 )

Aim : to provide scientific evidence about health impacts caused by air pollution and to develop a first-stage ‘academic draft’ of a provincial regulation for reducing air pollution.

Research integration outcomes: first comprehensive assessment of main air pollutant exposures and effects on human health in Indonesia.

Research implementation outcomes: the research underpinned a Jakarta provincial regulation on air quality and a decree issued by the Governor on indoor air quality.

Interactional expertise was required to work with the disciplines of: epidemiology, environmental health, pharmacology, public health nutrition, environmental engineering.

Interactional expertise was required to work with stakeholders from: Ministry of Environment, provincial health office, 20 private companies, Government of Jakarta, 28 elementary schools in Jakarta, Jakarta Metropolitan Office, private car and public transport commuters, international non-government organisations, and an international nutritional supplement company.

Example of tacit know-that: Knowledge about the role of provincial regulations and where research findings would be appropriate.

Example of tacit know-how: Skills in acting as a policy entrepreneur and seizing opportunities for effecting change.

CASE 3 . Developing policies on medical tourism in Thailand (2010, not documented)

Aim : to find an appropriate balance between competing private and public demands on health workers and other resources caused by ‘medical tourism’ in Thailand, as well as how the private and public health sectors can collaborate to improve the health of the Thai population. Medical tourism generates substantial revenue from foreign patients and is promoted by the private health sector with political support from the government. Significant concerns arise, however, about inadequate public health provision for the Thai population.

Research integration outcomes: evidence from several studies, expert opinion, and findings from a public hearing were combined to develop a resolution for the National Health Assembly.

Research implementation outcomes: the National Health Assembly adopted the resolution in December 2010 (Third National Health Assembly, 2010 ) and the cabinet of the Thai government endorsed it in April 2011.

Interactional expertise was required to work with the disciplines of: public health, economics, sociology, medicine.

Interactional expertise was required to work with stakeholders from: Ministry of Public Health, Regional and General Hospital Society, Rural Doctors Society, Private Hospital Association, health professional councils (medical, dental and nursing), University Hospital Networks—Thailand (medical schools), civil society organisations, Ministry of Commerce, relevant areas of the law, constituencies of the National Health Assembly.

Example of tacit know-that: Knowledge about the mechanics of the political process.

Example of tacit know-how: Skills to identify the powerful players who needed to be on board and to interact effectively with them in finding ways of responding to their concerns.

Box 4 Two examples of codified expertise from cased-based experience developed independently of specific approaches, plus when this expertise is helpful

Pre-nuptial agreement for research collaborators

Know-that expertise includes understanding that collaboration can be assisted by addressing expectations and areas of potential conflict at the outset, specifically considering (1) overall goals and outcomes; (2) who will do what, including how personnel decisions are made, supervision is provided and data are managed; (3) authorship and credit, including giving public presentations, responding to media inquiries and managing intellectual property; (4) contingencies and communication, including management of team communication, new collaborations and spin-off projects, departures or changes of key personnel and modification of the research agenda by new discoveries or unexpected outcomes; and 5) identifying and addressing conflicts of interest (Bennett et al., 2018 ).

This expertise is useful for: opening discussion about issues that could cause conflict in collaboration and setting in place proposed resolutions.

Alignment, interest and influence matrix

Know-how expertise includes being able to use a four-dimensional stakeholder mapping tool to identify with whom researchers should engage for their work to have policy impact. The first two dimensions—degree of alignment with the policy direction emerging from the research and degree of interest in the policy issue—sort stakeholders into those with whom the project should seek to collaborate, those to co-opt, those to challenge, and those who can be ignored. Leaving aside the last group, the two next steps in the process are to identify those with power and those with whom it is possible to engage directly (Mendizabal, 2010 ; Young et al., 2014 ).

This expertise is useful for: prioritising which stakeholders to work with and how to work with them.

Box 5 Examples of expertise in research integration and implementation specifically relevant to considering unknowns. Brief descriptions of when this expertise is useful are also provided

Social construction of ignorance

Know-that expertise includes appreciating that ignorance is usefully considered as socially constructed rather than somehow imposed by a complex universe (Smithson, 1989 ). Further, people may strategically construct and impose unknowns on others through, for example, intellectual suppression and ‘undone science’ (e.g., topics that are not funded; Hess, 2016 ) or by manufacturing doubt (Michaels, 2008 ; Oreskes and Conway, 2010 ).

This expertise is useful for: better appreciating the complexity of unknowns.

Recognising different kinds of unknowns

Know-that expertise recognises different kinds of unknowns and taxonomies of unknowns differentiating, for example, ‘what we are ignorant of’ from ‘what we choose to ignore’, or distinguishing ‘error’ from ‘vagueness’ (Smithson, 1989 ). Humans operate differently depending on the kind of unknown they are dealing with, so that addressing different unknowns requires different know-how strategies. For example, people are generally more averse to and distrustful of unknowns arising from conflicting information than unknowns arising from ambiguity or vagueness (Smithson, 1999 ).

Unknowns can also be differentiated into ‘known unknowns’ (where disciplines generally put their efforts), ‘unknown knowns’ (tacit knowledge) and ‘unknown unknowns’ (Kerwin, 1993 ). The last of these can be particularly hard to recognise and handle. Finally, unknowns can be approached from the perspective of a particular complex real-world problem, which can highlight unknowns at intersections of disciplines, unknowns that are not in the purview of any discipline but that concern stakeholders, and unknowns marginalised by power relations (Bammer, 2016a ).

This expertise is useful for: better appreciating the complexity of unknowns, especially those that may be critical and unresolvable.

Accepting unknowns

Know-how expertise includes strategies to accept that unknowns exist and to diminish the risk of adverse unintended consequences, or at least limit their impact. Examples include practicing adaptive management (Holling, 1978 ; Hughes et al., 2007 ), building in resilience (Biggs et al., 2015 ), adopting the precautionary principle (de Sadeleer, 2007 ; Kriebel et al., 2001 ) and differentiating high-risk from low-risk unknowns in strategic choice processes (Friend and Hickling, 2005 ). Acceptance strategies tend to prefer trust-based relations over contractual ones, and prioritise robustness and satisficing over optimisation, identifying options worth preserving (Smithson and Ben-Haim, 2015 ).

This expertise is useful for: managing critical, unresolvable unknowns.

Info-gap theory

Know-how expertise also includes facility with processes such as info-gap theory, which starts with a model for the situation, where some parameters or model structures are unknown. Estimates of the unknowns are then taken; they are assumed to be substantially wrong, and analysis involves determining how sensitive outcomes under the model are to the errors in these estimates (Ben-Haim, 2006 ).

Learning plan method

Know-how expertise involves ability to employ methods such as learning loops, which can be used to resolve unknowns and discover unforeseen unknowns. The first step is to define what to do next. Sources of unknowns are identified and categorised by the importance of resolving them immediately. One source of unknowns is selected, an assumption about the unknown formulated and a test of the assumption designed. The next step is to test the assumption. The third step is to evaluate the information and knowledge gained through the test, including whether any new unknowns emerged. This then informs the next iteration of the loop (Rice et al., 2008 ).

Box 6 Examples of expertise in research integration and implementation specifically relevant to enhancing innovation. Brief descriptions of when this expertise is useful are also provided

Differentiating adoption from use

Know-that expertise includes understanding that adoption differs from use. Adoption is the willingness and ability to take research results and embody them into something more broadly useable, such as a manufactured product, a policy or an implementation programme. Use focuses on the receiving ecosystem of people and organisations, who use what the adopter has created and thereby change their behaviour—i.e., do something new—indicating research impact. Adopters and users have different motivations, which must be recognised and taken into account by those developing research implementation expertise (Adner, 2012 ; Elsum, 2013 ).

This expertise is useful for: understanding the processes required to make change happen.

Innovation through searching for ideas, information and knowledge

Know-that expertise includes understanding that the search strategy used for ideas, information or knowledge will determine whether an innovation is incremental, involving minor changes to established ways of working, or radical, involving major, even fundamental, changes. Radical innovation enables previously intractable problems to be tackled, though it requires individuals and organisations to step outside their existing cognitive frames and/or search strategies (Dorst, 2015 ; Nicholas et al., 2013 ). Reframing and a broad search strategy increase the likelihood of being able to combine elements from distinctly different existing ways of doing things. Such recombination can be a source of novel ideas with high impact (Fleming, 2007 ; Savino et al., 2017 ).

This expertise is useful for: developing new ways for tackling complex societal and environmental problems.

Using analogy to spark innovation

Know-how expertise includes being able to use analogy to spark innovation. Effective use of analogy requires drawing on knowledge of solutions in a familiar domain and applying them to an unfamiliar domain. Unfamiliarity in the application domain reduces constraints arising from unconsciously sticking to mental models and problem-solving strategies that have been helpful in the past, which impedes sparking of highly novel solutions. Developing non-obvious analogies requires identification of deep similarities in relational characteristics of the domains. This is typically stimulated by a specific problem and involves a high level of abstraction (Gassmann and Zeschky, 2008 ).

Question 3: what is required to strengthen expertise in research integration and implementation?

Key to strengthening expertise in research integration and implementation is to make it readily identifiable and accessible. As little effort has gone into documenting such expertise, it is largely invisible and unrecognised. Further, as we have shown in the previous section, it is also highly fragmented. It is, therefore, currently much easier for researchers and teams to ‘reinvent the wheel’ by duplicating know-that and know-how than to find, build on and improve existing expertise.

Identifying expertise and overcoming fragmentation are therefore critical, requiring both an inventory and an organisational framework that promotes accessibility. Both requirements could be achieved by building a dynamic, shared knowledge bank and here we outline what is involved.

The scale of a knowledge bank would be considerably greater than a toolkit, and indeed, many toolkits would be included in the knowledge bank. In order to avoid consignment to a graveyard of integrative databases, atlases, and knowledge compendiums that have not gained traction, the knowledge bank requires wide-ranging support from the realms where expertise currently resides. Building a knowledge bank therefore also involves building a coalition of key communities and teams. Such a coalition will, through the process of building the knowledge bank, provide an authoritative voice about expertise in research integration and implementation, ensuring that it is properly valued. In turn this will improve assessments of integration and implementation in research tackling complex societal and environmental problems, including in tenure and promotion applications, funding proposals, outcomes of research projects, and outputs of inter- and transdisciplinary centres and other institutions.

We review three key challenges involved in building a knowledge bank: compiling existing expertise, indexing and organising the expertise to make it widely accessible, and understanding and overcoming the core reasons for the existing fragmentation. For each we also highlight some current-positive trends. The aim is to provide an indication of the effort that is required, especially from those in the three realms who need to be involved for a knowledge bank to be a long-term success.

Compiling existing expertise

Compiling existing expertise is a major task and we have aimed to lay foundations in this article by addressing three key issues:

defining expertise

identifying tasks requiring expertise in research integration and implementation

describing expertise, which also needs to incorporate illustrative examples and guidance for when the expertise is appropriate (relating back to the tasks requiring expertise).

In focusing on the third of these—describing expertise—we identify five major challenges. One immediate challenge is finding relevant expertise and illustrative case studies, many of which are not documented in either the published or grey literatures. Those that are documented are widely dispersed and not easy to locate.

Second, developing guidance to link expertise to tasks is hampered by lack of evidence about key aspects of most know-that and know-how, including strengths and weaknesses, effectiveness, or how well the expertise can be adapted to differing circumstances.

Third, elements of expertise in research integration and implementation will be characterised differently within and across the three realms. This is affirmed by our experience of working together as authors, where we found, for example, different framings of power relations and how they should be addressed. As well as identifying differences, it is essential to recognise when particular know-that or know-how expertise, as used by different groups, is essentially the same.

Fourth, those who currently have expertise in research integration and implementation come from very different backgrounds, so that expertise that is self-evident to some will be a revelation to others. For example, the importance of understanding and managing power relations will be obvious to those originally trained in the social sciences, but will be a topic some others rarely consider. Similarly, the possibility of non-linear relations between two causal factors will be self-evident to those originally trained in the physical sciences, but new to many others.

As hinted at in these examples, there are many areas where expertise in research integration and implementation intersects with expertise in existing disciplines. How this is dealt with in deciding what expertise to include in a compilation of know-that and know-how for research integration and implementation requires careful consideration.

Finally, the biggest challenge in compiling and assessing expertise, as well as writing guidance notes, is that no single individual or group has experience across the three realms, or even a significant subset. Instead, a more labour-intensive and time-consuming process is necessary, requiring individuals and groups from the different realms to work together globally to understand each other’s contributions, before being able to undertake the requisite sifting, assessing and provision of guidance.

It is beyond the scope of this article to do more than flag these matters. Nevertheless, we see the challenges as both intellectually exciting and practical, although far from straight-forward to manage.

Building on positive trends

Growing recognition of connections across different specific approaches is a positive trend, especially for identifying the broad range of expertise to be included in a knowledge bank. For example, the Oxford Handbook of Interdisciplinarity (Frodeman, 2017 ) includes information on transdisciplinarity, systems thinking, design science, team science, sustainability science, integration and implementation sciences, and innovation, along with its primary focus on interdisciplinarity. Similarly, the Sage Handbook of Action Research (Bradbury, 2015 ) provides chapters on systems thinking and integration and implementation sciences. In the same vein, the journal GAIA has published a series of Toolkits for Transdisciplinarity (Bammer, 2017 ) and is now issuing a series of Frameworks for Transdisciplinary Research that are drawing connections, not only to other specific approaches such as systems thinking, change management and integration and implementation sciences (e.g., Cabrera and Cabrera, 2018 on systems thinking), but also to dialogue and collaboration methods from case-based experience (e.g., Bammer, 2016b ; McDonald et al., 2009 ).

Indexing and organising the expertise and making it widely accessible

Compiling expertise is not enough. The knowledge bank needs to be organised in a way that makes expertise easy to find by a wide range of interested individuals, teams and communities of practice, as both contributors and users. Multiple entry points are required, providing access that is intuitive and welcoming, and that accommodates the different ways of understanding and tackling complex societal and environmental problems described in this article.

Finding an effective way to index expertise in the knowledge bank is key. This formidable challenge calls for collaboration with information scientists to build on lessons from existing knowledge banks and cyberinfrastructures, including Wikipedia ( 2019 ), Dryad Digital Repository ( 2019 ), Open Biological and Biomedical Ontology (OBO) Foundry ( 2019 ), Gene Ontology Resource ( 2019 ), as well as the Long-Term Ecological Research Network ( 2019 ) and the National Ecological Observatory Network ( 2019 ). In addition, Leonelli and Ankeny ( 2015 , p. 703) highlight a ‘tension between the stability and the flexibility of classificatory categories,’ which need both consistency over time and to be adapted and updated so that they ‘mirror the research practices and knowledge of their users’.

A host of other practical matters must also be resolved, including funding, maintaining long-term integrity and interoperability, establishing meta-data standards, encompassing burgeoning variations in nomenclature, determining who can contribute and how, providing credit and other incentives to contribute, and establishing standards for assessment of contributions. Again, how such challenges are addressed by existing successful repositories will be instructive. For example, Wikipedia ( 2019 ) is the best-known online reference website and provides lessons about a resource written collaboratively by volunteers. Its open authorship and editing policy mean new content can be easily created and all content can be kept up-to-date, but it also means accuracy, rigour and indexing can be inconsistent.

A positive trend to build on is the activities of Integration and Implementation Sciences (i2S; Bammer, 2013 ), which is testing ideas relevant to developing an ontology. The i2S frame consists of three domains: synthesis of disciplinary and stakeholder knowledge, understanding and managing diverse unknowns, and providing integrated research support for policy and practice change. It also addresses five questions: For what and for whom? Of what? How? Context? Outcome? These questions encompass specific knowledge and skills in integration and implementation, such as framing, modelling and co-production. In the aggregate, these components aim to provide a structure or ontology for indexing, codified in a repository of resources (Integration and Implementation Sciences, 2019c ).

An important caveat to organising expertise into a knowledge bank is the need for congruence with the challenges of dealing with complex problems. The multiple facets and interactions of complexity, and the creativity required to deal with them, cannot be ignored or downplayed in favour of indexing requirements. Indeed, the challenges of building a knowledge bank to strengthen expertise in research integration and implementation are likely to stimulate further development of information sciences towards systems that can deal effectively with complexity.

Understanding and overcoming the core reasons for the existing fragmentation

Overcoming fragmentation of expertise in research integration and implementation requires understanding why it exists and what forces maintain it. Separate explanations offer insights into fragmentation of expertise codified by various specific approaches and of expertise developed by case-based experience that is independent of specific approaches.

Specific approaches can be seen as small ‘tribes’ around particular ‘territories’, to use concepts and terms made popular by Becher ( 1989 ) in his analysis of the construction of boundaries that differentiate disciplines. By their nature, tribes organise around certain journals and conferences, and their members become reviewers for each other’s work. More broadly, members of a tribe are identified by ‘traditions, customs and practices, transmitted knowledge, beliefs, morals and rules of conduct, as well as their linguistic and symbolic forms of communication and the meanings they share’ (Becher, 1989 , p. 24). Tribal differences among specific approaches were evident in our interactions at the 2013 conference (Integration and Implementation Sciences, 2019a ) and in our deliberations as an authorship group.

As an aside, Becher’s insights can be further invoked to envision that building a knowledge bank is an exercise in reviewing a territory in order to formalise permeable boundaries ‘open to incoming and outgoing traffic’ (1989, p. 37) with the aim of rendering accessible all elements of expertise required to tackle complex societal and environmental problems. Indeed, it was the interaction of our individual ‘tribes’ that exposed our group to rich ‘intra-tribal’ knowledge and reinforced our joint commitment to developing a knowledge bank.

The fragmentation of expertise developed by case-based experience that is independent of specific approaches can be understood through the expositions on mode 2 research underpinned by transdisciplinarity (in its generic rather than specific sense) by Gibbons et al. ( 1994 ) and on interdisciplinarity (also in its generic sense) by the US National Academies (National Academy of Sciences, National Academy of Engineering and Institute of Medicine, 2005 ). Both publications emphasised that, at the time they were written, relevant expertise was diffused in a loose way, typically transmitted by experienced researchers moving to new problems rather than through institutionalised reporting in professional journals. This mode of diffusion continues to be typical among those focused on case-based experience, as described earlier.

Both forces of fragmentation—multiple small tribes and loose diffusion—are reinforced by a third force combining high-academic workload and publication pressure (Kinman, 2014 ; Kinman and Jones, 2003 ). This combination severely limits the time researchers have to look beyond their own tribe or accumulated experience to find useful know-that and know-how expertise, especially when doing so requires familiarity with a wide array of literature and multiple specialised ‘languages.’

At least three encouraging trends are overcoming fragmentation. First, ‘borrowing’ (i.e., taking concepts and methods from one discipline into another) has long been an acknowledged feature of interdisciplinary research (Klein, 1990 ). In addition, it occurs across specific approaches. For example, the interdisciplinary tool “Toolbox’ dialogue method to uncover research worldviews’ (see Box 2 ) has been incorporated into an online toolkit of co-production methods for transdisciplinary research (td-Net, 2019 ). Borrowing is also starting to link case-based experience with specific approaches. Population health research, for instance, is adapting ideas from complexity science (Long et al., 2018 ; Thompson et al., 2016 ).

A second trend is exemplified by the Integration and Implementation Insights blog ( 2019 ), which is a conduit for linking researchers, regardless of tribe or problem tackled, allowing them to share integration and implementation expertise in easy-to-read form. This is in line with the community building identified by Leonelli and Ankeny ( 2015 ) as an essential component of developing a large-scale repository.

A third trend builds on the long-standing practice of establishing dedicated centres, employing a range of disciplinary experts, to address complex societal and environmental issues. The heads of such organisations are forming an authoritative leadership group to ensure funders and research policy makers understand, value and support expertise in research integration and implementation (Network of Interdisciplinary and Transdisciplinary Research Organisations—Oceania, 2019 ; Palmer, 2018 ).

Final considerations

To address societal and environmental problems through the sustainable development goals, effective illicit drugs policy, action on spiralling health care costs and other initiatives, a better understanding is required of expertise in research integration and implementation, along with the ability to readily access that expertise. In this article we seek to rally those who have developed elements of that expertise around the tasks of defining expertise, making it visible and accessible, and organising to ensure that it is rewarded. What is required is a major effort by the relevant communities and teams. In this article we have sought to tread the fine line between proposing these ideas with enough detail to be comprehensible, while leaving them open to amendment by those who need to be involved in codifying expertise in research integration and implementation.

We have sought to lay foundations for the work of systematising expertise by discussing:

when expertise is needed in tackling complex societal and environmental problems

the realms where it can currently be found and who therefore needs to be represented

how expertise could be strengthened by being organised in a knowledge bank.

To realise the vision of codified expertise requires a virtuous cycle between developing an accessible, widely known, dynamic knowledge bank and successes based on improved research and action on complex societal and environmental problems (Fig. 1 ).

Mutual reinforcement (a virtuous cycle) between a knowledge bank to strengthen expertise in research integration and implementation and success in addressing complex societal and environmental problems.

In particular, building a knowledge bank to strengthen expertise in research integration and implementation has the potential to:

improve access to, and application of, the most effective know-that and know-how contributory expertise

provide more and new opportunities for building expertise, along with enhancing its quality by identifying gaps in expertise and assessment of strengths and weaknesses, effectiveness and adaptability

facilitate break-through innovation in tackling complex societal and environmental problems.

These goals share the common aim of achieving greater tractability and success in tackling complex societal and environmental problems. In turn, accumulating success, with well-worked through examples of how this rests on effective research integration and implementation, has the potential to:

increase demand for relevant expertise, along with interest in developing capacity and capability in research integration and implementation

increase support and justification for evaluation of expertise to improve its quality and provide a stronger evidence base for its appropriate use

build institutional support for using and further developing the knowledge bank, manifested in funding to sustain it, training for users, and recognition of expertise in research integration and implementation in faculty hiring and promotion processes.

However, none of this can happen without the effort to define, identify and gather expertise. Unless there is continual input into the knowledge bank, the feedback loop can become a stagnant trap, miring the system in its own inertia. Expertise stays limited, few opportunities for building expertise arise, quality remains low and no break-through innovations occur. As a result, few successes in addressing complex societal and environmental problems accumulate, so that there is no increased interest and demand for expertise, little justification for evaluation efforts and lack of institutional support.

Avoiding the stagnation trap and the graveyard for ideas that did not gain traction requires a large cohort of committed proponents, substantial funding and a process for building the knowledge bank guided by principles that frame pursuit of robust workable outcomes for complex problems, especially eliciting and respecting diverse perspectives, fostering curiosity, managing systems with many unknowns and supporting on-going learning. The aim is to make construction of a knowledge bank a fulfilling social process, building community and connections while supporting the community in becoming more skilled and responsive to challenges and change. In that respect, the knowledge bank and the community would co-evolve.

The long-term vision is an institutionalised knowledge bank that serves not only researchers, but also practitioners, policy makers, and other stakeholders concerned with improving research-based understanding and action on complex societal and environmental problems. In particular, the knowledge bank will be an essential resource about expertise for funders of research on complex societal and environmental problems, as well as scientific journal editors and reviewers who judge the quality of such research and its applications. Further, it will have widespread use in university teaching and continuing education. In summary, we envision building a major resource that is a recognised and accepted part of the global research environment.

Ackoff RL (1974) Redesigning the future. A systems approach to societal problems. Wiley, New York, NY

Google Scholar  

Ackoff RL (1981) Creating the corporate future. Wiley, New York, NY

Adner R (2012) The wide lens. Penguin, London

Aven T (2012) Foundations of risk analysis, 2nd edn. Wiley, Chichester

Book   MATH   Google Scholar  

Babor T, Caulkins J, Fischer B, Foxcroft D, Humphreys K, Medina-Mora M, Obot I, Rehm J, Reuter P, Room R, Rossow I, Strang J (2018) Drug policy and the public good, 2nd edn. Oxford University Press, Oxford

Book   Google Scholar  

Bammer G (2013) Disciplining interdisciplinarity: Integration and implementation sciences for researching complex real-world problems. ANU Press, Canberra, https://doi.org/10.22459/DI.01.2013

Bammer G (2016a) What constitutes appropriate peer review for interdisciplinary research? Palgrave Commun 2(16017). https://doi.org/10.1057/palcomms.2016.17

Bammer G (2016b) Toolkits for transdisciplinarity—Toolkit #3: dialogue methods for knowledge synthesis. GAIA 25(1):7. https://doi.org/10.14512/gaia.25.1.3

Article   Google Scholar  

Bammer G (2017) Eight toolkits for transdisciplinarity. Originally published in the journal GAIA. Available online at https://www.oekom.de/fileadmin/zeitschriften/gaia_Grafiken/GAIA_Flyer_Toolkits.pdf

Becher T (1989) Academic tribes and territories: Intellectual enquiry and the cultures of disciplines. The Society for Research into Higher Education & Open University Press, Milton Keynes

Beer S (1984) The viable system model: Its provenance, development, methodology and pathology. J Operational Res Soc 35:7–25

Beishon J, Peters G (1972) Systems behaviour. Harper and Row, London

Ben-Haim Y (2006) Info-Gap decision theory: Decisions under severe uncertainty, 2nd edn. Academic Press, London

Bennett LM, Gadlin H, Marchand C (2018) Collaboration and team science field guide, 2nd edn. National Institutes of Health Publication No. 18-7660. National Cancer institute, Bethesda, https://www.cancer.gov/about-nci/organization/crs/research-initiatives/team-science-field-guide

Bergmann M, Jahn T, Knobloch T, Krohn W, Pohl C, Schramm E (2012) Methods for transdisciplinary research. A primer for practice. Campus Verlag, Frankfurt

Bioenergy Research, Development & Extension Advisory Forum and Technical Working Groups (2014) Opportunities for primary industries in the bioenergy sector: National research, development and extension strategy. Rural Industries Research and Development Corporation, Canberra, https://www.agrifutures.com.au/wp-content/uploads/publications/11-079.pdf

Biggs R, Schlüter M, Schoon ML (2015) An introduction to the resilience approach and principles to sustain ecosystem services in social-ecological systems. In: Biggs R, Schlüter M, Schoon ML (eds) Principles for building resilience: Sustaining ecosystem services in social-ecological systems. Cambridge University Press, Cambridge, pp 1–31

Chapter   Google Scholar  

Bradbury H (ed) (2015) The SAGE handbook of action research, Third edn. Sage, Los Angeles

British Academy Working Group on Interdisciplinarity (2016) Crossing paths: Interdisciplinary institutions, careers, education and applications. The British Academy for the Humanities and Social Sciences, London, https://www.thebritishacademy.ac.uk/sites/default/files/Crossing%20Paths%20-%20Full%20Report.pdf

Bromham L, Dinnage R, Hua X (2016) Interdisciplinary research has consistently lower funding success. Nature 534(June):684–687

Article   ADS   PubMed   CAS   Google Scholar  

Buckley W (1965) Modern systems research for the behavioral scientist. Aldine, Chicago

Bursztyn M, Drummond J (2014) Sustainability science and the university: Pitfalls and bridges to interdisciplinarity. Environ Educ Res 20(3):313–332. https://doi.org/10.1080/13504622.2013.780587

Cabrera D, Cabrera L (2018) Four building blocks of systems thinking. Frameworks for transdisciplinary research #4. GAIA 27(2):200. https://doi.org/10.14512/gaia.27.2.3

Cairney P, Weible CM (2017) The new policy sciences: Combining the cognitive science of choice, multiple theories of context, and basic and applied analysis. Policy Sci 50(4):619–627. https://doi.org/10.1007/s11077-017-9304-2

Checkland P (1981) Systems thinking, systems practice. Wiley, Chichester

Churchman CW (1970) Operations research as a profession. Manag Sci 17:B37–53

Churchman CW (1979) The systems approach, 2nd edn. Dell, New York, NY

Cicmil SJK, Cooke-Davies TJ, Crawford LH, Richardson KA (2009) Exploring the complexity of projects: Implications of complexity theory for project management practice. Project Management Institute, Newtown Square, PA

Cilliers P (1998) Complexity and postmodernism: Understanding complex systems. Routledge, Abingdon

Clark WC, Dickson NM (2003) Sustainability science: The emerging research program. Proc Natl Acad Sci USA 100(14): 8059–8061

Article   ADS   CAS   Google Scholar  

Collins H, Evans R (2002) The third wave of science studies: Studies of expertise and experience. Soc Stud Sci 32:235–296

Collins H, Evans R (2007) Rethinking expertise. University of Chicago Press, Chicago

Crawford DF, O'Connor MH, Jovanovic T, Herr A, Raison RJ, O'Connell DA, Baynes T (2016) A spatial assessment of potential biomass for bioenergy in Australia in 2010, and possible expansion by 2030 and 2050. Glob Change Biol Bioenergy 8:707–722. https://doi.org/10.1111/gcbb.12295

Article   CAS   Google Scholar  

de Sadeleer N (ed) (2007) Implementing the precautionary principle: Approaches from the Nordic Countries, EU and USA. Earthscan, London

Dorst K (2015) Frame innovation. MIT Press, Cambridge

Dryad Digital Repository (2019) [online] Available at: http://datadryad.org/ [Accessed 12 Mar 2019]

Eccles MP, Armstrong D, Baker P, Cleary K, Davies H, Davies S, Glasziou P, Ilott I, Kinmonth A-L, Leng G, Logan S, Marteau T, Michie S, Rogers H, Rycroft-Malone J, Sibbald B (2009) An implementation research agenda. Implement Sci 4:18, http://www.implementationscience.com/content/4/1/18

Elsum I (2013) Tackling integrative applied research: Lessons from the management of innovation (Commentary). In: Bammer G (ed) Disciplining interdisciplinarity: Integration and implementation sciences for researching complex real-world problems. ANU Press, Canberra, pp. 431–439

Emery FE (1969) Systems thinking. Volume 1. Penguin, Harmondsworth

Emery FE (1981) Systems thinking. Volume 2. Penguin, Harmondsworth

Farine DR, O'Connell DA, Raison RJ, May BM, O'Connor MH, Crawford DF, Herr A, Taylor JA, Jovanovic T, Campbell PK, Dunlop MIA, Rodriguez LC, Poole ML, Braid AL, Kriticos D (2012) An assessment of biomass for bioelectricity and biofuel, and for greenhouse gas emission reduction in Australia. Glob Change Biol Bioenergy 4:148–175

Fleming L (2007) Breakthroughs and the long tail of innovation. MIT Sloan Manag Rev 49:69–74

Flood RL, Jackson MC (eds) (1991) Critical systems thinking: Directed readings. Wiley, Chichester

Flood RL, Romm NRA (eds) (1996) Critical systems thinking: Current research and practice. Plenum, New York, NY

Forrester JW (1961) Industrial dynamics. MIT Press, Cambridge MA

Fortune J, Peters G (1995) Learning from failure: The systems approach. Wiley, Chichester

François C (2004) International encyclopedia of systems and cybernetics, 2nd edn. K G Saur, Muenchen

Frantzeskaki N, Hölscher K, Holman IP, Pedde S, Jaeger J, Kok K, Harrison PA (2019) Transition pathways to sustainability in greater than 2°C climate futures of Europe. Regional Environ Change 19:777–789. https://doi.org/10.1007/s10113-019-01475-x

Friend J, Hickling A (2005) Planning under pressure: The strategic choice approach, 3rd edn. Routledge, London

Frodeman R (2017) (ed in chief), Klein JT, Pacheco RCS (associate eds) The Oxford handbook of interdisciplinarity. 2nd edn. Oxford University Press: Oxford

Fulton EA, Smith ADM, Smith DC, Johnson P (2014) An integrated approach is needed for ecosystem based fisheries management: Insights from ecosystem-level management strategy evaluation. PLoS ONE 9:e84242. https://doi.org/10.1371/journal.pone.0084242

Article   ADS   PubMed   PubMed Central   CAS   Google Scholar  

Funtowicz SO, Ravetz JR (1993) Science for the post-normal age. Futures 25(Sept):739–755

Gassmann O, Zeschky M (2008) Opening up the solution space: The role of analogical thinking for breakthrough product innovation. Creativity Innov Manag 17:97–106

Gene Ontology Resource (2019) The gene ontology resource. [online] Available at: http://geneontology.org/ [Accessed 18 Apr 2019]

Gertler PJ, Martinez S, Premand P, Rawlings LB, Vermeersch CMJ (2016) Impact evaluation in practice, 2nd edn. Inter-American Development Bank and World Bank, Washington, DC, http://www.worldbank.org/ieinpractice

Gibbons M, Limoges C, Nowotny H, Schwartzman S, Scott P, Trow M (1994) The new production of knowledge. The dynamics of science and research in contemporary societies. Sage, London

Gobet F (2015) Understanding expertise: A multidisciplinary approach. Palgrave, London

Gorddard R, Colloff MJ, Wise RM, Ware D, Dunlop M (2016) Values, rules and knowledge: Adaptation as change in the decision context. Environ Sci Policy 57:60–69

Graham P, Reedman L, Rodriguez L, Raison J, Braid A, Haritos V, Brinsmead T, Hayward J, Taylor J, O'Connell D (2011) Sustainable aviation fuels road map: Data assumptions and modelling. CSIRO, Australia

Hall AD (1962) A methodology for systems engineering. Van Nostrand, Princeton

Haryanto B (2013) Applying the I2S framework to air pollution and health in Indonesia (Commentary). In: Bammer G (ed) Disciplining interdisciplinarity: Integration and implementation sciences for researching complex real-world problems. ANU Press, Canberra, pp. 389–395

Hayward JA, O'Connell DA, Raison RJ, Warden AC, O'Connor MH, Murphy HT, Booth TH, Braid AL, Crawford DF, Herr A, Jovanovic T, Poole ML, Prestwidge D, Raisbeck-Brown N, Rye L (2015) The economics of producing sustainable aviation fuel: a regional case study in Queensland, Australia. Global Change Biology. Glob Change Biol Bioenergy 7:497–511. https://doi.org/10.1111/gcbb.12159

Hess DJ (2016) Undone science. Social movements, mobilized publics, and industrial transitions. MIT Press, Cambridge MA

Hirsch Hadorn G, Hoffmann-Riem H, Biber-Klemm S, Grossenbacher-Mansuy W, Joye D, Pohl C, Wiesmann U, Zemp E (eds) (2008) Handbook of transdisciplinary research. Springer, New York, NY

Holling CS (1978) Adaptive environmental assessment and management. John Wiley & Sons, Chichester

Horn RE, Weber RP (2007) New tools for resolving wicked problems: mess mapping and resolution mapping processes. http://www.strategykinetics.com/files/New_Tools_For_Resolving_Wicked_Problems.pdf

Hughes TP, Gunderson L, Folke C, Baird A, Bellwood D, Berkes F, Crona B, Helfgott A, Leslie H, Norberg J, Nystrom M, Olsson P, Österbloom H, Scheffer M, Schuttenberg H, Steneck RS, Tengö M, Troell M, Walker B, Wilson J, Worm B (2007) Adaptive management of the Great Barrier Reef and the Grand Canyon World Heritage Areas. Ambio 36:586–592

Article   PubMed   Google Scholar  

Integration and Implementation Insights (2019) [online] Available at: http://i2Insights.org [Accessed 18 Apr 2019]

Integration and Implementation Sciences (2019a) First i2S conference [online] Available at: http://i2s.anu.edu.au/what-i2s/first-i2s-conference [Accessed 18 Apr 2019]

Integration and Implementation Sciences (2019b) Professional associations and networks. [online] Available at: http://i2s.anu.edu.au/resources/associations_networks [Accessed 18 Apr 2019]

Integration and Implementation Sciences (2019c) Resources. [online] Available at: http://i2s.anu.edu.au/resources [Accessed 18 Apr 2019]

International Standards Association (ISO); 2015 ISO 13065:2015 Sustainability criteria for bioenergy. [online] Available at: https://www.standards.org.au/standards-catalogue/international/iso-slash-tmbg/iso-13065-colon-2015 [Accessed 18 Apr 2019]

Jacob WJ (2015) Interdisciplinary trends in higher education. Palgrave Commun 1:15001. https://doi.org/10.1057/palcomms.2015.1

Jahn T, Bergmann M, Keil F (2012) Transdisciplinarity: Between mainstreaming and marginalization. Ecol Econ 79:1–10. https://doi.org/10.1016/j.ecolecon.2012.04.017

Johnson MP (ed) (2012) Community-based operations research: Decision modeling for local impact and diverse populations. Springer, New York, NY

Kerwin A (1993) None too solid: medical ignorance. Knowl: Creation, Diffus, Utilisation 15(2):166–185

Kinman G (2014) Doing more with less? Work and wellbeing in academics. Somatechnics 4:219–235

Kinman G, Jones F(2003) ‘Running up the down escalator': Stressors and strains in UK academics Qual High Educ 9:21–38. https://doi.org/10.1080/13538320308162

Klein JT (1990) Interdisciplinarity: History, theory and practice. Wayne State University Press, Detroit

Klein JT (2010) Creating interdisciplinary campus cultures. Jossey Bass and the Association of American Colleges and Universities, San Francisco

Klein JT, Falk-Krzesinski HJ (2017) Interdisciplinary and collaborative work: Framing promotion and tenure practices and policies. Res Policy 46(6):1055–1061

Kleindorfer PR, Kunreuther HC, Schoemaker PJH (1993) Decision sciences: An integrative perspective. Cambridge University Press, Cambridge

Kriebel D, Tickner J, Epstein P, Lemons J, Levins R, Loechler EL, Quinn M, Rudel R, Schettler T, Stoto M (2001) The precautionary principle in environmental science. Environ Health Perspect 109:871–876

Article   PubMed   PubMed Central   CAS   Google Scholar  

Ledford H (2015) Team science. Nature 525(Sept):308–311

Leonelli S, Ankeny RA (2015) Repertoires: how to transform a project into a research community. Bioscience 65(7):701–708. https://doi.org/10.1093/biosci/biv061

Long KM, McDermott F, Meadows GN (2018) Being pragmatic about healthcare complexity: our experiences applying complexity theory and pragmatism to health services research. BMC Med 1:94. https://doi.org/10.1186/s12916-018-1087-6

Long Term Ecological Research (LTER) Network (2019) [online] Available at: https://lternet.edu/ [Accessed 18 Apr 2019]

Loorbach D, Frantzeskaki N, Avelino F (2017) Sustainability transitions research: transforming science and practice for societal change. Annu Rev Environ Resour 42:599–626. https://doi.org/10.1146/annurev-environ-102014-021340

Lyall C (2019) Being an interdisciplinary academic: How institutions shape university careers. Palgrave Macmillan, Cham

Marchau VAWJ, Walker WE, Bloemen PJTM, Popper SW (eds) (2019) Decision making under deep uncertainty: From theory to practice. Springer, Cham, https://link.springer.com/book/10.1007/978-3-030-05252-2

Maru Y, O’Connell D, Grigg N, Abel N, Cowie A, Stone-Jovicich S, Butler J, Wise R, Walker B, Belay M, Fleming A, Meharg S, Meyers J (2017) Making ‘resilience’, ‘adaptation’ and ‘transformation’ real for the design of sustainable development projects: piloting the Resilience, Adaptation Pathways and Transformation Assessment (RAPTA) framework in Ethiopia. CSIRO, Canberra

McDonald D, Bammer G, Deane P (2009) Research integration using dialogue methods. ANU Press, Canberra, https://doi.org/10.22459/RIUDM.08.2009

McLeisch T, Strang V (2016) Evaluating interdisciplinary research: the elephant in the peer-reviewers’ room. Palgrave Commun 2:16055. https://doi.org/10.1057/palcomms.2016.55

Mendizabal E (2010) The Alignment, Interest and Influence Matrix (AIIM) toolkit. Guidance Note. Research and Policy in Development (RAPID), Overseas Development Institute (ODI), London, https://www.odi.org/publications/5288-stakeholder-engagement-stakeholder-analysis-aiim-alignment-interest-influence-matrix-roma

Michaels D (2008) Doubt is their product: How industry's assault on science threatens your health. Oxford University Press, Oxford

Midgley G (2000) Systemic intervention: philosophy, methodology, and practice. Kluwer Academic/Plenum Publishers, New York, NY

Midgley G (2003) Systems thinking (Volumes I–IV). Sage, London

Midgley G, Ochoa-Arias AE (eds) (2004) Community operational research: OR and systems thinking for community development. Kluwer Academic/Plenum Publishers, New York, NY

Miser HJ, Quade ES (eds) (1985) Handbook of systems analysis: overview of uses, procedures, applications and practice. Elsevier, New York, NY

Miser HJ, Quade ES (eds) (1988) Handbook of systems analysis: Craft issues and procedural choices. Wiley, New York, NY

Mitchell M (2009) Complexity: a guided tour. Oxford University Press, Oxford

MATH   Google Scholar  

National Academy of Sciences, National Academy of Engineering & Institute of Medicine (2005) Facilitating interdisciplinary research. National Academies Press, Washington DC

National Ecological Observatory Network (2019) [online] Available at: http://www.neoninc.org [Accessed 18 Apr 2019]

National Research Council (2015) Enhancing the effectiveness of team science. National Academies Press, Washington, DC

Nauheimer H (1997) The change management toolbook. A collection of tools, methods and strategies. Open access pdf at http://www.change-management-toolbook.com

Network of Interdisciplinary and Transdisciplinary Research Organisations–Oceania (2019) [online] Available at https://nitro-oceania.net/ [Accessed 26 Aug 2019]

Neuhauser L (2018) Practical and scientific foundations of transdisciplinary research and action. In: Fam D, Neuhauser L, Gibbs P (eds) Transdisciplinary theory, practice and education: the art of collaborative research and collective learning Springer, Cham, pp. 25–38

Nicholas J, Ledwith A, Bessant J (2013) Reframing the search space for radical innovation. Res Technol Manag 56:27–35

O'Connell D, Haritos VS (2010) Conceptual investment framework for biofuels and biorefineries research and development. Biofuels 1:201–216

O'Connell D, Haritos V, Graham S, Farine D, O'Connor M, Batten D, May B, Raison J, Braid A, Dunlop M (2007) Bioenergy, bioproducts and energy: a framework for research and development. Rural Industries Research and Development Corporation, Canberra

O'Connell D, Braid A, Raison J, Handberg K, Cowie A, Rodriguez L, George B (2009) Sustainable production of bioenergy: A review of global bioenergy sustainability frameworks and assessment systems. Rural Industries Research and Development Corporation, Canberra

O’Connell D, Braid A, Raison J, Hatfield-Dodds S, Wiedmann T, Cowie A, Littleboy A, Clark M (2013) Navigating sustainability: measurement, evaluation and action. CSIRO, Australia, https://www.betterevaluation.org/sites/default/files/Navigating%20sustainability%20report.pdf

Open Biological and Biomedical Ontology (OBO) Foundry (2019) [online] Available at: http://www.obofoundry.org/ [Accessed 18 Apr 2019]

Oreskes N, Conway E (2010) Merchants of doubt. How a handful of scientists obscured the truth on issues from tobacco smoke to global warming. Bloomsbury, London

O’Rourke M, Crowley S (2013) Philosophical intervention and cross-disciplinary science: The story of the Toolbox Project. Synthese 190:1937–1954

Palmer L (2018) Meeting the leadership challenges for interdisciplinary environmental research. Nat Sustainability 1:330–333. https://doi.org/10.1038/s41893-018-0103-3

Pohl C, Hirsch Hadorn G (2007) Principles for designing transdisciplinary research. Proposed by the Swiss Academies of Arts and Sciences. Oekom, Munich

PNAS Proceedings of the National Academy of Sciences of the United States of America (2019) Sustainability science. [online] Available at http://sustainability.pnas.org/ [Accessed 26 Aug 2019]

Reckling FJ, Fischer C (2010) Factors influencing approval probability in Austrian Science Fund (FWF) decision-making procedures—FWF stand-alone projects programme, 1999 to 2008. Available at SSRN (Social Science Research Network): https://doi.org/10.2139/ssrn.1725985

Rice M, O’Connor G, Pierantozzi R (2008) Implementing a learning plan to counter project uncertainty. MIT Sloan Manag Rev 49(2):54–62

Richardson GP, Andersen DF (2010) Systems thinking, mapping, and modeling for group decision and negotiation. In: Eden C, Kilgour DN (eds) The handbook for group decision and negotiation. Springer, Dordrecht, pp. 313–324

Rittel HWJ, Webber MM (1973) Dilemmas in a general theory of planning. Policy Sci 4:155–169

Ritter A, King T, Lee N eds. (2017) Drug use in Australian society, 2nd edn. Oxford University Press, South Melbourne

Rotmans J, Van Asselt M (1996) Integrated assessment: growing child on its way to maturity. An editorial essay. Climatic Change 34:327–336

Article   ADS   Google Scholar  

Savino T, Petruzzelli A, Albino V (2017) Search and recombination process to innovate: A review of the empirical evidence and a research agenda. Int J Manag Rev 19:54–75

Shaxson L, with Bielak A, Ahmed I, Brien D, Conant B, Fisher C, Gwyn E, Klerkx L, Middleton A, Morton S, Pant L, Phipps D (2012) Expanding our understanding of K* (KT, KE, KTT, KMb, KB, KM, etc.) A concept paper emerging from the K* conference held in Hamilton, Ontario, Canada, April 2012. United National University Institute of Water, Environment and Health: Hamilton, Ontario. https://assets.publishing.service.gov.uk/media/57a08a6e40f0b649740005ba/KStar_ConceptPaper_FINAL_Oct29_WEBsmaller.pdf

Simon H (1996) The sciences of the artificial, 3rd edn. MIT Press, Cambridge MA

Smithson M (1989) Ignorance and uncertainty: Emerging paradigms. Springer Verlag, New York, NY

Smithson M (1999) Conflict aversion: preference for ambiguity vs. conflict in sources and evidence. Organ Behav Hum Decis Process 79:179–198

Article   PubMed   CAS   Google Scholar  

Smithson M, Ben-Haim Y (2015) Reasoned decision making without math? Adaptability and robustness in response to surprise. Risk Anal 35:1911–1918

Article   PubMed   PubMed Central   Google Scholar  

Sterman JD (2000) Business dynamics. Systems thinking and modeling for a complex world. McGraw-Hill Higher Education, Boston

Stevens A (2010) Drugs, crime and public health: The political economy of drug policy. Routledge-Cavendish, London

Taha HA (2017) Operations research: An introduction, 10th edn. Pearson, Harlow

Td-Net (Network for transdisciplinary research) (2019) Methods and tools for co-producing knowledge. [online] Available at: https://naturalsciences.ch/topics/co-producing_knowledge [Accessed 18 Apr 2019]

Therival R, Wood G (eds) (2018) Methods of environmental and social impact assessment, 4th edn. Routledge, New York, NY

Third National Health Assembly (2010) NHA3/Resolution 8 Medical hub policy. [online] Available at: https://en.nationalhealth.or.th/wp-content/uploads/2017/09/Medical-Hub-Resolution.pdf (English) https://www.samatcha.org/sites/default/files/document/0304-Res-3-4.pdf (Thai) [Accessed 18 Apr 2019]

Thompson DS, Fazio X, Kustra E, Patrick L, Stanley D (2016) Scoping review of complexity theory in health services research. BMC Health Serv Res 16:87. https://doi.org/10.1186/s12913-016-1343-4

Toolbox Dialogue Initiative (2019) [online] Available at: http://tdi.msu.edu/ [Accessed 18 Apr 2019]

Ulrich W (1983) Critical heuristics of social planning: a new approach to practical philosophy. Haupt, Berne

Vennix JAM (1996) Group model building: facilitating team learning using system dynamics. Wiley, Chichester

Vilsmaier U, Brandner V, Engbers M (2017) Research in-between: The constitutive role of cultural differences in transdisciplinarity. The Transdisciplinary J Eng Sci 8:169–179

Wikipedia (2019) Wikipedia: About. [online] Available at: https://en.wikipedia.org/wiki/Wikipedia:About [Accessed 18 Apr 2019]

Young J, Shaxson L, Jones H, Hearn S, Datta A, Cassidy C (2014) Rapid outcome mapping approach: A guide to policy engagement and influence. Overseas Development Institute, London, https://www.odi.org/sites/odi.org.uk/files/odi-assets/publications-opinion-files/9011.pdf

Download references

Acknowledgements

This article has its origins in the 2013 First Global Conference on Research Integration and Implementation (Integration and Implementation Sciences 2019a), which brought together the authors and specifically drew on all three realms of expertise. The conference was supported by the Australian Research Council Centre of Excellence on Policing and Security and The Australian National University. Co-conferences in Germany, The Netherlands and Uruguay were, respectively, sponsored by Leuphana University of Lueneburg; the Centre for Innovation at Campus The Hague, Leiden University; and the Espacio Interdisciplinario, Universidad de la República. Contributions by Robyn Mildon, Dean Fixsen, Alison Ritter and Noshir Contractor at the conference informed the development of this article and valuable feedback on earlier versions was received from L. David Brown, Sharon Friel, Cynthia Mitchell, Christian Roth and Judith Sutz. Assistance was provided by Peter Deane and David McDonald on aspects of the literature, as well as Erin Walsh on the figure.

Author information

Authors and affiliations.

Research School of Population Health, The Australian National University, Canberra, ACT, Australia

Gabriele Bammer

Institute for Advanced Sustainability Studies, Potsdam, Germany

Center for Interdisciplinarity and Department of Philosophy, Michigan State University, East Lansing, MI, USA

Michael O’Rourke

CSIRO Land and Water, Canberra, ACT, Australia

Deborah O’Connell & Nicola J. Grigg

Health Research for Action Center, School of Public Health, University of California, Berkeley, CA, USA

Linda Neuhauser

Centre for Systems Studies, Business School, University of Hull, Hull, UK

Gerald Midgley

School of Innovation, Design and Engineering, Mälardalen University, Eskilstuna, Sweden

Victoria Business School, Victoria University of Wellington, Wellington, New Zealand

School of Political and Social Sciences, University of Canterbury, Christchurch, New Zealand

School of Agriculture and Food Sciences, University of Queensland, Brisbane, QLD, Australia

Informatics Department, Faculty of Technology, Linnaeus University, Växjö, Sweden

Department of English, Wayne State University, Detroit, MI, USA

Julie Thompson Klein

Transdisciplinarity Lab, Department of Environmental Systems Science, ETH Zurich, Zurich, Switzerland

Julie Thompson Klein, Christian Pohl & Bianca Vienni Baptista

Retired, Bethesda, MD, USA

Howard Gadlin

Research School of Management, The Australian National University, Canberra, ACT, Australia

Ian R. Elsum

Centre for Transformative Innovation, Swinburne University of Technology, Melbourne, VIC, Australia

Center for Sustainable Development, University of Brasília, Brasília, Brazil

Marcel Bursztyn

INCT/Odisseia—Observatory of Socio-environmental Dynamics, Brasília, Brazil

CSIRO Oceans and Atmosphere, Hobart, TAS, Australia

Elizabeth A. Fulton

Centre for Marine Socioecology, University of Tasmania, Hobart, TAS, Australia

td-net, Swiss Academies of Arts and Sciences, Bern, Switzerland

Christian Pohl

Research School of Psychology, The Australian National University, Canberra, ACT, Australia

Michael Smithson

Institute of Philosophy and Sciences of Art and Methodology Center, Leuphana University of Lueneburg, Lueneburg, Germany

Ulli Vilsmaier

Institute for Social-Ecological Research, Frankfurt, Germany

Matthias Bergmann

Institute for Ethics and Transdisciplinary Sustainability Research, Leuphana University of Lueneburg, Lueneburg, Germany

Independent Scholar, Vienna, Austria

Jill Jaeger

Kenniscocreatie, Onderzoek & Advies, The Hague, The Netherlands

Femke Merkx

Centre for Environmental Policy, Imperial College London, London, UK

Mark A. Burgman

Australian Centre for International Agricultural Research, Canberra, ACT, Australia

Daniel H. Walker

International Network for Advancing Science and Policy (INASP), Oxford, UK

Action Research Plus Foundation (AR+), Portland, OR, USA

Hilary Bradbury

Faculty of Engineering, The University of Sydney, Sydney, NSW, Australia

Lynn Crawford

School of Management, Cranfield University, Cranfield, UK

Faculty of Society and Design, Bond University, Gold Coast, QLD, Australia

Department of Environmental Health, University of Indonesia, Depok, West Java, Indonesia

Budi Haryanto

International Health Policy Program, Ministry of Public Health, Nonthaburi, Thailand

Cha-aim Pachanee

School of Global Studies, University of Gothenburg, Gothenburg, Sweden

Merritt Polk

Rockefeller College of Public Affairs and Policy, University at Albany, State University of New York, Albany, NY, USA

George P. Richardson

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Gabriele Bammer .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Bammer, G., O’Rourke, M., O’Connell, D. et al. Expertise in research integration and implementation for tackling complex problems: when is it needed, where can it be found and how can it be strengthened?. Palgrave Commun 6 , 5 (2020). https://doi.org/10.1057/s41599-019-0380-0

Download citation

Received : 20 May 2019

Accepted : 27 November 2019

Published : 13 January 2020

DOI : https://doi.org/10.1057/s41599-019-0380-0

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

This article is cited by

Putting grant/d terms to work: from promise to practice in inter- and transdisciplinary research projects.

• Stina Powell
• Sofie Joosse

Discover Sustainability (2024)

The Urban Circularity Assessment Framework (UCAF): a Framework for Planning, Monitoring, Evaluation, and Learning from CE Transitions in Cities

• Fedra Vanhuyse

Circular Economy and Sustainability (2023)

Expert and non-expert at the same time: knowledge integration processes and dynamics in interdisciplinary teamwork

• Annemarie Horn
• Eduardo Urias
• Marjolein B. M. Zweekhorst

Sustainability Science (2023)

Convergence Research as a ‘System-of-Systems’: A Framework and Research Agenda

• Lisa C. Gajary
• Shalini Misra

Minerva (2023)

Different perspectives on a common goal? The Q-method as a formative assessment to elucidate varying expectations towards transdisciplinary research collaborations

• Verena Radinger-Peer
• Elisabeth Schauppenlehner-Kloyber
• Katharina Gugerell

Sustainability Science (2022)

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Encyclopedia of the Sciences of Learning pp 1238–1240 Cite as

• Tamara van Gog 2  
• Reference work entry

254 Accesses

1 Citations

Excellence ; Experience ; Expert performance ; Expert-novice research

According to Webster’s online dictionary, an expert is someone “having, involving, or displaying special skill or knowledge derived from training or experience.” The term expertise is often used in a relative sense in educational research. In expert-novice research, an expert is someone who has the knowledge required to perform a certain task, distinguishing him or her from a novice who is not able to perform that task. This task, however, can still be a relatively simple one in the domain. On a domain level, expertise may also refer to the quantity and quality of experience in a domain, with different stages being distinguished on the route from novice to expert. Finally, expertise can be defined in terms of exceptional performance in a domain; expert performance research investigates the consistently superior performance of individuals who excel at representative tasks within that domain.

Theoretica...

This is a preview of subscription content, log in via an institution .

Buying options

• Available as PDF
• Read on any device
• Instant download
• Own it forever
• Available as EPUB and PDF
• Durable hardcover edition
• Dispatched in 3 to 5 business days
• Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Alexander, P. A. (2003). The development of expertise: The journey from acclimation to proficiency. Educational Researcher, 32 , 10–14.

Article   Google Scholar  

Chi, M. T. H. (2006). Two approaches to the study of experts’ characteristics. In K. A. Ericsson, N. Charness, P. Feltovich, & R. R. Hoffman (Eds.), The Cambridge handbook of expertise and expert performance (pp. 21–30). Cambridge, UK: Cambridge University Press.

Google Scholar  

Chi, M. T. H., Glaser, R., & Farr, M. (Eds.). (1988). The nature of expertise . Hillsdale, NJ: Erlbaum.

Dreyfus, S. E., & Dreyfus, H. L. (1980). A five-stage model of the mental activities involved in directed skill acquisition . Berkeley, CA: Operations Research Center, University of California.

Ericsson, K. A., & Lehmann, A. C. (1996). Expert and exceptional performance: Evidence of maximal adaptations to task constraints. Annual Review of Psychology, 47 , 273–305.

Ericsson, K. A., Krampe, R. T., & Tesch-Römer, C. (1993). The role of deliberate practice in the acquisition of expert performance. Psychological Review, 100 , 363–406.

Ericsson, K. A., Charness, N., Feltovich, P., & Hoffman, R. R. (Eds.). (2006). The Cambridge handbook of expertise and expert performance . Cambridge, UK: Cambridge University Press.

Kalyuga, S., & Sweller, J. (2005). Rapid dynamic assessment of expertise to improve the efficiency of adaptive e-learning. Educational Technology Research and Development, 53 , 83–93.

Download references

Author information

Authors and affiliations.

Institute of Psychology, Erasmus University Rotterdam, 1738, Rotterdam, 3000 DR, The Netherlands

Dr. Tamara van Gog

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Tamara van Gog .

Editor information

Editors and affiliations.

Faculty of Economics and Behavioral Sciences, Department of Education, University of Freiburg, 79085, Freiburg, Germany

Norbert M. Seel

Rights and permissions

Reprints and permissions

Copyright information

© 2012 Springer Science+Business Media, LLC

About this entry

Cite this entry.

van Gog, T. (2012). Expertise. In: Seel, N.M. (eds) Encyclopedia of the Sciences of Learning. Springer, Boston, MA. https://doi.org/10.1007/978-1-4419-1428-6_95

Download citation

DOI : https://doi.org/10.1007/978-1-4419-1428-6_95

Publisher Name : Springer, Boston, MA

Print ISBN : 978-1-4419-1427-9

Online ISBN : 978-1-4419-1428-6

eBook Packages : Humanities, Social Sciences and Law

Share this entry

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Publish with us

Policies and ethics

• Find a journal
• Track your research

• More from M-W
• To save this word, you'll need to log in. Log In

Definition of expertise

• proficiency

Examples of expertise in a Sentence

These examples are programmatically compiled from various online sources to illustrate current usage of the word 'expertise.' Any opinions expressed in the examples do not represent those of Merriam-Webster or its editors. Send us feedback about these examples.

Word History

borrowed from French, going back to Middle French, from espert, expert expert entry 2 + -ise -ice

1868, in the meaning defined at sense 2

Phrases Containing expertise

• area of expertise

Dictionary Entries Near expertise

Cite this entry.

“Expertise.” Merriam-Webster.com Dictionary , Merriam-Webster, https://www.merriam-webster.com/dictionary/expertise. Accessed 18 Apr. 2024.

Kids Definition

Kids definition of expertise, more from merriam-webster on expertise.

Nglish: Translation of expertise for Spanish Speakers

Britannica English: Translation of expertise for Arabic Speakers

Subscribe to America's largest dictionary and get thousands more definitions and advanced search—ad free!

Can you solve 4 words at once?

Word of the day.

See Definitions and Examples »

Get Word of the Day daily email!

Popular in Grammar & Usage

Your vs. you're: how to use them correctly, every letter is silent, sometimes: a-z list of examples, more commonly mispronounced words, how to use em dashes (—), en dashes (–) , and hyphens (-), absent letters that are heard anyway, popular in wordplay, 10 words from taylor swift songs (merriam's version), a great big list of bread words, the words of the week - apr. 12, 10 scrabble words without any vowels, 12 more bird names that sound like insults (and sometimes are), games & quizzes.

• Skip to primary navigation
• Skip to main content
• Skip to primary sidebar
• Skip to footer

Global Cognition

What is expertise and how can you develop it.

by Louise Rasmussen updated September 13, 2021

Everybody’s an expert these days. Pest Control Expert, Plumbing Expert, Weather Expert, and so on.

But what is expertise, really?

A common conception is that experts know more and perform better than those around them.

From this standpoint, expertise refers to the knowledge and skills that distinguish top performers from novices and less proficient people.

There is a sense in which expertise is relative. That’s the key idea that makes the movie Idiocracy so entertaining. In it, an ordinary Joe who is completely average on every measurable dimension is transported to a future where humanity has devolved to the lowest common denominator. Suddenly Joe is the resident expert on everything.

One convenient indicator often used in cognitive field research to decide whether someone is an expert is whether or not other people in their area say they are one.

Other quick and easy signs are: ‘has a lot of knowledge and experience,’ ‘has an advanced degree or certification,’ ‘is always (or almost always) right,’ and ‘can solve very difficult problems.’

These indicators hold up reasonably well in applied studies. However, researchers continue exploring the nature of expertise to unpack it further. An important result of these studies is that there are different kinds of expertise. And this has implications for how to go about building it.

Two Kinds of Expertise

K. Anders Ericsson and Giyoo Hatano are two scientists who study expertise. Their research has led them in slightly diverging directions. As a result, they also give different advice about how you would go about building expertise.

Ericsson is a well-known cognitive psychologist who has studied learning and expertise for decades. His definition focuses on consistency.

In a discussion of the Superior Performance of Experts in Current Directions in Psychological Science , Ericsson and Ward define expertise as the thinking and qualities that lead to consistently superior performance.

This is very much in line with our definition above.

Hatano and Inagaki expanded on this notion of expertise in their studies. These researchers noticed that there seemed to be two kinds of expertise.

First, they described one form of expertise in much the same way as Ericsson and others. They called this routine expertise. The routine experts could consistently solve the problems, but relied on routine procedures they had used many times before.

However, they also found that in groups of recognized experts, some appeared to be even more expert than others.

They discovered that what set these ‘experts among the experts’ apart was their ability to not just solve problems, but solve them in new ways by inventing new procedures and strategies. Hatano and Inagaki called what these experts had adaptive expertise .

According to Hatano you know you have adaptive expertise when you can perform with understanding. You know you have understanding if you can do the following:

• you can explain why certain strategies or procedures work and others don’t
• you can distinguish between appropriate and inappropriate ways to modify strategies
• you can change your strategies in response to changes in the environment

How do You Get Expertise?

The two different ways of thinking about expertise lead to different ideas about how to go about building it.

Practice, Seek Feedback, Analyze

Thinking about expertise as consistently superior performance has led Ericsson to recommend a training regimen that includes lots and lots of repetitions of tasks and activities. According to Ericsson, you can build expertise through:

• Practice: Your goal when practicing should be to concentrate deeply and perform just a little bit better than last time
• Feedback: When you complete a task or problem, seek feedback about the accuracy.
• Analysis: When you’re not practicing, study past moves or solutions—your own or those of accomplished experts

Following this recipe you should gradually (over the course of 10,000 hours) get to the point where you can perform the skill at a superior level consistently.

Practice, Explain, Modify

Hatano and Inagaki recommend a similar approach (lots of practice) but with important modifications.

• Practice: Your objective when practicing skills should be to discover what happens when you apply a variety of strategies.
• Explain: Each time you apply a strategy and observe an outcome you should try to explain why it worked, or didn’t.
• Modify: When practicing the skill again, either change some aspect of the task or problem or your strategy for approaching it.

The key to Hatano’s approach to building expertise is that you continue to seek out new problems that challenge your current state of skills and knowledge.

Pick Your Poison, I Mean, Practice

Ericsson’s ideas about how to build expertise seem appropriate for building what Hatano calls routine expertise . Especially if you’re trying to build expertise in physical skills, like tennis or ballet, or areas with manageable problem spaces, like chess.

In many domains the problem spaces are much more open and dynamically changing. New diseases, new economic crises, new weather patterns are constantly emerging. To tackle these kinds of problems you need adaptive expertise.

Even though Ericsson and Hatano agree that lots and lots of practice is needed they have different ideas about how you should practice. Both approaches seem like they would help you get better at whatever it is you’re practicing. But, before you start solving endless numbers of calculus problems or examining unending arrays of chest x-rays, you may want to think about what kind of expertise you’re hoping to develop. That way you can adopt a practice strategy that will help you get there.

Ericsson, K., & Ward, P. (2007). Capturing the Naturally Occurring Superior Performance of Experts in the Laboratory: Toward a Science of Expert and Exceptional Performance Current Directions in Psychological Science, 16 (6), 346-350. DOI: 10.1111/j.1467-8721.2007.00533.x

Hatano, G., & Inagaki, K. (1984). Two courses of expertise Research and Clinical Center for Child Development Annual Report, 6 , 27-36.

Study Smarter

Build your study skills with thinker academy.

About Louise Rasmussen

Dr. Louise Rasmussen is an applied cognitive psychologist. She works to advance cultural competence in demanding environments through research, training, and assessment.

Reader Interactions

June 10, 2016 at 12:40 am

My needs require both routine and adaptive expertise. I’m an actor and an aspiring FOREX speculator. As an actor I have to react to other actors who will never give the same performance twice, and for that matter, neither will I. Also, the audience will always will be different, even if the same people fill the house every night. The financial markets can turn at various markets, though there are numerous ways to predict when change will occur. I can learn to recognize patterns through routine expertise, but to put pattern recognition into practice, knowing where to put my entry, target and stop loss, require adaptive expertise.

December 4, 2017 at 7:27 pm

Thanks Louise for comparing these approaches and citing our paper (Ericsson & Ward, 2007). Look out for an upcoming paper in a special section of JARMAC on adaptive skill by Ward et al. (2018) where we look at this issue more closely.

December 6, 2017 at 9:27 am

Thank you for your comment Paul! Great to hear from you. We’ll be very interested to read your upcoming paper–and will certainly keep an eye out!

• Save Your Ammo
• Publications

GC Blog Topics

• Culture & Communication
• Thinking & Deciding
• Learning Skills
• Learning Science

Online Courses

• Thinker Academy
• Study Skills Course
• For Parents
• For Teachers

• Cambridge Dictionary +Plus

Meaning of expertise in English

Your browser doesn't support HTML5 audio

• I've been in this job for 30 years , and I've picked up a good deal of expertise along the way.
• Software is not really my area of expertise.
• You place too much reliance on her ideas and expertise.
• He is known for his expertise in the history of British economics .
• The database is simple to use and requires no expertise at all.
• accomplishment
• functional skills
• functioning
• prestidigitation
• proficiency
• pyrotechnic
• pyrotechnics
• workmanship

expertise | American Dictionary

Expertise | business english, examples of expertise, collocations with expertise.

These are words often used in combination with expertise .

Click on a collocation to see more examples of it.

Translations of expertise

Get a quick, free translation!

Word of the Day

have your hands full

to be so busy that you do not have time to do anything else

Binding, nailing, and gluing: talking about fastening things together

Learn more with +Plus

• Recent and Recommended {{#preferredDictionaries}} {{name}} {{/preferredDictionaries}}
• Definitions Clear explanations of natural written and spoken English English Learner’s Dictionary Essential British English Essential American English
• Grammar and thesaurus Usage explanations of natural written and spoken English Grammar Thesaurus
• Pronunciation British and American pronunciations with audio English Pronunciation
• English–Chinese (Simplified) Chinese (Simplified)–English
• English–Chinese (Traditional) Chinese (Traditional)–English
• English–Dutch Dutch–English
• English–French French–English
• English–German German–English
• English–Indonesian Indonesian–English
• English–Italian Italian–English
• English–Japanese Japanese–English
• English–Norwegian Norwegian–English
• English–Polish Polish–English
• English–Portuguese Portuguese–English
• English–Spanish Spanish–English
• English–Swedish Swedish–English
• Dictionary +Plus Word Lists
• English    Noun
• American    Noun
• Business    Noun
• Collocations
• Translations
• All translations

Add expertise to one of your lists below, or create a new one.

{{message}}

Something went wrong.

There was a problem sending your report.

An official website of the United States government

Here’s how you know

Official websites use .gov A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS A lock ( Lock A locked padlock ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

• Science and Technology Directorate

Interoperability is Key to Effective Emergency Communications

During National Public Safety Telecommunicators Week, we’re sharing updates on S&T efforts focused on getting first responders the information they need quickly.

When it comes to communicating emergency information to and among first responders, interoperability is a problem. In some cases, emergency responders cannot talk to some parts of their own agencies—let alone communicate with agencies in neighboring cities, counties, or states. And when time is of the essence, the results can be catastrophic. But there are other factors that can impede response, and we are keenly focused on addressing each of these with technological solutions.

The 9/11 Commission Report speaks at great length about the issues the lack of interoperability caused. As a result of the Commission Report, there was a significant reorganization of response capabilities, which included the creation of the Department of Homeland Security (DHS) and, soon after, the Science and Technology Directorate (S&T). We've been on the case ever since, working for and with responders to better understand and deliver on their technology needs.

While all these organizations are working to find a solution, we have multiple efforts underway to support new technologies to help correct for these gaps. For instance, our First Responder Capability portfolio and Technology Centers work with responders across the country on communications solutions. But the challenges are formidable, as jurisdictions manage their own technology across 6,000 911 call centers nationwide.

Wireless: The Wave of the Future

Let’s face it. The future of communications is going to be wireless, and that extends to emergency communications, too.

S&T has been sponsoring research across a number of areas, based on findings in the S&T “Study on Mobile Device Security” Report, which concluded that targeted research and development (R&D) could inform standards to improve security and resilience of critical mobile communications networks. As a result, S&T’s Mobile Security R&D Program established the Secure and Resilient Mobile Network Infrastructure (SRMNI) project and has efforts underway to establish standards for secure voice and video capability for communications across the 3G, 4G, and 5G networks.

Last year, interagency discussions were held that included S&T, Cybersecurity & Infrastructure Security Agency, and the U.S. Department of Defense, among others, to identify lab testing requirements for 5G Emergency Communications interoperability. Then, in early spring of 2024, S&T and MITRE demoed new features in the new 5G ecosystem critical to DHS components and first responder use cases and continued to conduct engineering analysis and lab-based research to identify potential gaps. Research will be ongoing.

So, we are trending forward but are still working on helping aid improvements across traditional networks.

Connectivity is Key

As it stands, CAD-to-CAD (computer-aided dispatch) communications are the key to interoperability and resilience between government agencies responding to emergencies. Once the 911 call or text is answered, the information is sent to CAD, which is used to send the right resource to the right location. Public safety agencies have different CAD systems that don’t always efficiently share information. The result is ineffective and costly interoperable issues across communications systems.

In 2021, S&T funded a successful CAD-to-CAD interoperability pilot project run by the Integrated Justice Information Systems (IJIS) Institute to apply a single standard across municipalities to achieve interoperability. This pilot was successful in testing this theory by applying specifications across three localities – two in New Hampshire and one in Vermont, all three reliant on an InfoCAD™ environment hosted in the Amazon GovCloud. Through testing of two use cases – a three- or four-alarm fire and a medical emergency – IJIS demonstrated a viable solution in a live environment.  

Our Office of Mission and Capability Support will be conducting market research within the next few months on CAD-to-CAD Interoperability Compliance / Conformance testing. This upcoming effort is part of a five-year research & development portfolio under S&T’s Critical Infrastructure Security & Resilience Research (CISRR) Program, which is funded by the Infrastructure Investment and Jobs Act (IIJA) of 2021. The objective is to build upon the previous work done by the SRMNI project to establish interoperability functional specifications and develop a model for wide-scale implementation of these standards.

Location, Location, Location

Out-of-date Voice-over-IP (VoIP) phone numbers, connected to the Internet by design, are another issue that can create emergence response delays. The Federal Communications Commission (FCC) requires VoIP telephone service providers to maintain a subscriber’s verified street address as a dispatchable location to the 911 community. If a call is placed for emergency services and there is a lack of cellular coverage, the VoIP address should serve as backup. But these addresses aren’t being updated when moves are made.

The result is first responders routed to the wrong place during emergencies. Our Small Business Innovation Research (SBIR) program released a solicitation in 2023 calling for a solution to help identify whether a call to 911 is coming from a different location than the registered location. We will have more information available on this later this spring, but the aim is to better enable VoIP service providers to provide a valid, dispatchable address.

By helping to advance CAD-to-CAD interoperability testing, seeking a solution to assist with address accuracy through VoIP, and planning for mobile interoperability solutions of the future, we at S&T are hopeful that we can help support first responders and the telecommunicators that assist them get services to the people that need them more efficiently.

Learn more about other S&T efforts to help provide technology solutions to improve emergency response communications .

• Science and Technology
• Emergency Communication
• Public Safety
• First Responders

Site Search

• About ARPA-E
• Team Directory
• ARPA-E History
• Annual Reports
• Budget Requests
• Apply For Funding
• Authorization
• View Active Programs
• Search Our Programs
• Search Individual Projects
• Interactive Project Map
• Exploratory Topics
• The SCALEUP Program
• OPEN Programs
• Project Guidance
• ARPA-E Technology-to-Market
• Technology Commercialization
• External Engagement Model
• Investor Updates
• ARPA-E News & Media
• Press Releases
• ARPA-E Disruptors
• Publications
• ARPA-E Events
• Energy Innovation Summit
• Careers at ARPA-E
• Job Opportunities
• Life at ARPA-E
• ARPA-E FAQs
• General Questions
• Current Funding Opportunities
• Closed Funding Opportunities

Press Release

U.S. Department of Energy Announces $28 Million to Decarbonize Domestic Iron and Steel Production

WASHINGTON, D.C. — In support of President Biden’s Investing in America agenda, the U.S. Department of Energy (DOE) today announced $28 million in funding to 13 projects across 9 states to advance zero-process-emission ironmaking and ultra-low life cycle emissions steelmaking. The transformative technologies funded through this program would be the first to meet both emissions and cost parity goals, meaning the new, transformative concepts must be cost competitive with existing technologies. The teams announced today—managed by DOE’s Advanced Research Projects Agency-Energy (ARPA-E) under the Revolutionizing Ore to Steel to Impact Emissions (ROSIE) program —support the Biden-Harris Administration’s goals to reduce harmful, climate-change fueling emissions and imports of iron and steel products.   “Iron and steel production are among the most difficult industrial sectors to decarbonize, which is why ARPA-E is laser focused on accelerating game-changing technological breakthroughs to lower emissions from these critical sectors,” said ARPA-E Director Evelyn N. Wang. “Today’s announcement will help the nation achieve President Biden’s ambitious clean energy and net-zero goals while also reinforcing America’s global leadership in clean manufacturing for generations to come.” The iron and steel industry accounts for approximately 7% of global greenhouse gas (GHG) emissions and 11% of global carbon dioxide (CO2) emissions. By 2050, global iron and steel demand is projected to rise as much as 40%. This projected growth underscores the importance of lowering emissions from this industry. Current blast furnace technologies—responsible for approximately 70% of global iron and steel GHG emissions—require carbon for heat, chemistry, and structure, making the process particularly difficult to decarbonize. The ROSIE projects selected today seek to revolutionize not just the iron or steelmaking process, but the entire supply chain from ore to final steel production. The following teams will work to develop and demonstrate novel technologies that produce iron-based products from iron-containing ores and alternative feedstocks without process emissions in the ironmaking step:

• Argonne National Laboratory (Lemont, IL) will further develop a microwave-powered hydrogen plasma rotary kiln process for reducing iron ore that would eliminate carbon dioxide emissions from the ironmaking process. Argonne’s method has the potential to reduce carbon dioxide emissions arising from ironmaking by 35% compared to the blast furnace process when using today’s grid and by 88% when using a future low-carbon grid, while also reducing the cost of making hot rolled steel. (Award amount: $3,066,221)
• Blue Origin (Los Angeles, CA) will use an “Ouroboros” system that produces high-purity ferro-silicate pig iron from low-quality iron ores using molten oxide electrolysis (MOE) with zero direct process greenhouse gas emissions. Blue Origin will leverage and transfer the MOE expertise developed for lunar applications toward novel, terrestrial iron making approaches. The approach could reduce greenhouse gas emissions from the terrestrial ironmaking industry and clean up mine tailing storage facilities across the country. (Award amount: $1,109,422)
• Electra (Boulder, CO) will develop a process for producing iron at the temperature of a cup of coffee using unconventional feedstocks and a process involving two electrochemical cell stacks. If successful, the project will produce iron for use in green steel with 80% less greenhouse gas emissions at half the cost of existing traditional fuel-based processes. (Award amount: $2,874,596)
• Form Energy (Somerville, MA) will leverage its patent-pending breakthrough to directly produce iron powders from alkaline iron ore slurries in a first-of-a-kind powder-to-powder process. Using domestically available iron ore feedstocks, the process has the potential to produce greenhouse gas emission-free iron at cost parity with today’s carbon-intensive ironmaking methods. (Award amount: $1,000,000)
• Georgia Institute of Technology (Atlanta, GA) will work on a method to produce net-shaped engineered lattice structures and cellular structures of alloy steels by solid-state direct reduction of extruded structures. Several potential markets for the use of structural steels—where lightweighting and stiffness are most highly valued—include aerospace, military, and civilian aircraft, as well as automotive structural components. (Award amount: $2,843,274)
• Limelight Steel (Oakland, CA) will convert iron ore into iron metal using a laser furnace without emitting carbon dioxide at lower cost than a blast furnace. The process leverages semiconductor laser diodes, which enable new temperature and pressure ranges to reduce high- and low-grade iron ore fines into molten iron metal. Limelight estimates that their technology would reduce energy consumption of steelmaking by 46% and emissions by 81%. (Award amount: $2,910,346)
• Pennsylvania State University (State College, PA)  will develop an efficient, productive, and reliable electrochemical process for the economical reduction of iron ore at temperatures below 600°C without direct greenhouse gas emissions. The approach of a metallic anode protected by a solid metal oxide would overcome many of the challenges of anodic degradation that have hindered historical progress in this area. A host of electrolytes will be investigated while processing mixed Fe(II) and Fe(III) ores and simultaneously addressing ore impurities.  (Award amount: $760,000)
• Phoenix Tailings (Woburn, MA) will utilize an ore-to-iron production process using the arc generated from an air-gapped electrode to electrolyze the molten oxide electrolyte powered by clean electricity. Molten oxide electrolysis is a promising alternative to conventional approaches, but until now has required anode materials that are either consumable or prohibitively expensive. (Award amount: $1,000,000)
• Tufts University (Medford, MA) will develop a method to directly reduce iron ore concentrates with ammonia, eliminating all direct process emissions from the ironmaking step, as well as emissions that result from baking iron ore with clay to make hard pellets. By using low-grade ores, bypassing the pellet-hardening step, and lowering melting costs, this new approach to ammonia-based reduction would reduce the cost of domestic steel while decreasing total steel emissions by greater than 60%. (Award amount: $2,924,514)
• University of Minnesota (Minneapolis, MN)  will work on a fully electrified microwave hydrogen plasma process to replace blast furnace technology. The technology will use blast furnace and direct reduction grade iron ore concentrates, eliminating the emissions associated with the pelletization, sintering, and coke-making steps in the conventional blast furnace process.  (Award amount: $2,820,071)
• University of Nevada (Las Vegas, NV) will develop technology to use electrowinning to convert pulverized iron ore into pure iron that is deposited on a cathode. The goal is to create a laboratory-scale prototype of an impeller-accelerated reactor that maintains the production of one kilogram per hour of over 95% pure iron for 100 hours. (Award amount: $2,102,353)
• University of Utah (Salt Lake City, UT)  will advance a hydrogen-reduction melt-less steelmaking technology. The proposed process has the potential to drastically reduce energy consumption by eliminating several high-energy steps in traditional iron and steelmaking and is conducted at substantially lower temperatures than conventional methods. This approach is projected to decrease energy use by at least 50% in the production of steel mill products and up to 90% in creating near-net-shape steel components. (Award amount: $3,479,082)
• Worcester Polytechnic Institute (Worcester, MA)  will focus on manufacturing technologies for low carbon electrolyzed iron powder to be used in iron-silicon electrical steel. The work could revolutionize iron production by replacing the traditional carbothermic process while significantly reducing energy usage, greenhouse gas emissions, and cost. (Award amount: $1,241,919)

Access project descriptions for the teams announced today on the  ARPA-E website . If successful, novel ironmaking technologies meeting the metrics set forth by ROSIE will enable a reduction of U.S. emissions by over 65 million metric tonnes CO2 emitted annually (approximately 1% of U.S. emissions) and global emissions by over 2.9 gigatonnes annually (5.5% of global emissions).

ARPA-E advances high-potential, high-impact clean energy technologies across a wide range of technical areas that are strategic to America's energy security. Learn more about these efforts and ARPA-E's commitment to ensuring the United States continues to lead the world in developing and deploying advanced clean energy technologies. 

Selection for award negotiations is not a commitment by DOE to issue an award or provide funding. Before funding is issued, DOE and the applicants will undergo a negotiation process, and DOE may cancel negotiations and rescind the selection for any reason during that time.

Press and General Inquiries: 202-287-5440 [email protected]

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• HHS Author Manuscripts

Understanding Clinical Expertise: Nurse Education, Experience, and the Hospital Context

Matthew d. mchugh.

1 Center for Health Outcomes and Policy Research, University of Pennsylvania School of Nursing, Philadelphia, PA

Eileen T. Lake

2 Secondary Faculty, Department of Sociology, University of Pennsylvania

Clinical nursing expertise is central to quality patient care. Research on factors that contribute to expertise has focused largely on individual nurse characteristics to the exclusion of contextual factors. To address this, we examined effects of hospital contextual factors and individual nurse education and experience on clinical nursing expertise in a cross-sectional analysis of data from 8,611 registered nurses. In a generalized ordered logistic regression analysis, the composition of the hospital staff, particularly the proportion of nurses with at least a bachelor of science in nursing degree, was associated with significantly greater odds of a nurse reporting a more advanced expertise level. Our findings suggest that, controlling for individual characteristics, the hospital context significantly influences clinical nursing expertise.

Clinical nursing expertise is fundamental to quality of care. Research on the foundations of expertise has focused on individual characteris tics—particularly a nurse's years of experience and level of education. Debate continues about the respective contributions of experience and education to expertise. A notable gap in this debate that we examine is the influence of hospital contextual factors on an individual nurse's expertise. These contextual factors include the educational and experience levels of a nurse's coworkers as well as the nursing practice environment.

We capitalized on a large multi-level secondary data set of nurses in hospitals to examine individual nurses and their contexts. We hypothesized that contextual factors affect expertise above and beyond individual factors. Our study contributes evidence about the theoretical relationships between individual and contextual factors and expertise that may guide efforts to improve nurses' expertise.

As background for the study, in this manuscript we first outline the theoretical framework used to structure this investigation and then review the literature related to expertise in nursing. We then outline the relationships between expertise and the three principal concepts examined in this study—experience, education, and the nurse practice environment.

Theoretical Framework

We developed a multi-level framework ( Fig. 1 ) to describe how clinical expertise develops through both individual and organizational factors. Through this framework, derived from the sociology of organizations ( Blau, 1960 ; Shortell & Getzen, 1979 ), we suggest that the organization provides a context that influences individual outcomes. The organizational context includes the educational and experiential composition of the staff and the nurse practice environment. A contextual approach is particularly useful for understanding organizational phenomena where individuals are nested within larger systems such as nurses within hospitals. In our framework the individual characteristics of the nurse, such as education level and years of experience, influence an individual nurse's clinical expertise. Additionally, the context in which the nurse practices may help to establish a culture of professional nursing that encourages, values, and provides opportunities for the development of clinical nursing expertise.

Conceptual framework outlining relationships between individual nurse characteristics, hospital contextual features of nursing and clinical nursing expertise.

Contextual effects have long been of interest in sociological and educational research where, for example, individual student achievement was thought to be influenced by the achievement level of the class and school of a student ( Kreft & de Leeuw, 1998 ). Generally, contextual effects occur when the collective properties of individuals (e.g., nurses) in an organization like a hospital have an effect on an outcome (e.g., expertise) over and above the effect of individual characteristics ( Blalock, 1984 ). Books and Prysby (1988) outlined three potential mechanisms of contextual effects: (a) social interaction with like-minded others, (b) conformity to prevailing norms, and (c) information flow patterns. Although we did not test the specific mechanism, we expected that more educated and experienced staff and a more professional work environment would provide opportunities for ongoing learning and feedback, thereby functioning to promote the development of nurse expertise through these mechanisms.

Contextual effects like those examined in this investigation can be described as compositional or structural ( Books & Prysby, 1988 ). Compositional effects are usually measured as the aggregate mean of an individual phenomenon. In this study, education and experience were examined both as individual-level variables (the education and experience of a nurse) and compositional variables (the mean education and experience for all nurses in a hospital). By contrast, structural variables were derived from individual-level data and represent more complex aggregate measures. In our study, the practice environment was a structural variable derived as a complex measure from nurse ratings of their environment; it was not compositional because it was not an aggregate of nurses' individual characteristics. Based on this framework, we hypothesized that working in a hospital with higher mean levels of education and experience and a more professional practice environment would have a contextual effect on an individual nurse's expertise level even after controlling for the individual nurses' level of education and experience.

Literature Review

We defined clinical expertise as a hybrid of practical and theoretical knowledge, based on Benner (1984) . Clinically expert nurses are distinguished from their colleagues by their often intuitive ability to efficiently make critical clinical decisions while grasping the whole nature of a situation. Expertise influences nurses' clinical judgment and quality of care and develops when a nurse tests and refines both theoretical and practical knowledge in actual clinical situations ( Benner, 1984 ).

Benner (1984) also detailed the acquisition of nursing expertise and proposed five possible expertise levels: novice, advanced beginner, competent, proficient, and expert. Nurses at the novice stage are still in nursing school. Nurses at the advanced beginner stage use learned procedures and rules to determine what actions are required for the immediate situation. Competent nurses are task-oriented and deliberately structure their work in terms of plans for goal achievement. Competent nurses can respond to many clinical situations but lack the ability to recognize situations in terms of an overall picture. Proficient nurses perceive situations as a whole and have more ability to recognize and respond to changing circumstances. Expert nurses recognize unexpected clinical responses and can alert others to potential problems before they occur. Experts have an intuitive grasp of whole situations and are able to accurately diagnose and respond without wasteful consideration of ineffective possibilities. Because of their superior performance, expert nurses are often consulted by other nurses and relied upon to be preceptors. Although most nurses will progress to the competent level of expertise, many will not become experts ( Benner, 1984 ).

Experience and expertise

Experience and expertise are related but different concepts. We define experience, also based on Benner (1984) , as both time in practice and self-reflection that allows preconceived notions and expectations to be confirmed, refined, or disconfirmed in real circumstances. Merely encountering patient conditions and situations is not experience; rather, experience involves nurses reflecting on encountered circumstances to refine their moment-to-moment decision making at an unconscious, intuitive level, ( Benner, 1984 ; Benner & Tanner, 1987 ; Simmons, Lanuza, Fonteyn, Hicks, & Holm, 2003 ).

Experience is a necessary but not sufficient condition for expertise and not all experienced nurses are experts ( Christensen & Hewitt-Taylor, 2006 ; Ericsson, Whyte, & Ward, 2007 ). For example, Benner (1984) noted that a number of years on the job in the same or similar situations may create competence; however, the passage of time and occurrence of events and interactions does not automatically confer expert status. As Benner stated, there is a discontinuity or leap between expertise at the competent level and expertise at the proficient and expert levels. One potential explanation for this discontinuity is that years of experience may provide fluidity and flexibility but not the complex reflexive thinking that has been hypothesized to be an important component of clinical nursing expertise ( Bobay, 2004 ).

Few quantitative studies have been able to capture both the temporal and transactional nature of experience, and these studies have been limited to measuring experience in terms of years in practice. Young, Lehrer, and White (1991) found that nurses with more experience reported performing more complex functions than those with less experience. In a recent study of five hospitals, Bobay, Gentile, and Hagle (2009) found that years of experience were associated with expertise. However, Kovner and Schore (1998) did not find such a relationship.

The majority of the research on how experience contributes to expertise is at the individual nurse level; however, experience has also been assessed as a contextual variable. This latter work has primarily focused on the influence of aggregate experience on patient and nurse outcomes. For example, Aiken, Clarke, Cheung, Sloane, and Silber (2003) assessed the influence of the mean years of experience among nurses on surgical patient mortality in 168 hospitals. They found that the mean experience level was not a significant predictor of mortality. In a study that used the patient care unit as the level of analysis, researchers found that a higher proportion of nurses with ≥ 5 years of experience was associated with fewer medication errors and lower patient fall rates ( Blegen, Vaughn, & Goode, 2001 ). Similarly, Clarke, Rockett, Sloane, and Aiken (2002) examined the effect of the mean nurse experience level at the hospital level on nurse needlestick injuries; they concluded that a low mean experience level was associated with more near-miss needlestick incidents. In another study of nursing outcomes, Kanai-Pak, Aiken, Sloane, and Poghosyan (2008) found that the odds of high burnout, job dissatisfaction and poor-to-fair quality of care were twice as high in hospitals with 50% inexperienced nurses (i.e., nurses with less than 4 years experience) versus those with 20% inexperienced nurses.

Education and expertise

Education influences expertise by providing a theoretical and practical knowledge base that can be tested and refined in actual situations ( Dreyfus & Dreyfus, 1996 ). Didactic learning alone cannot generate clinical expertise, and one distinguishing aspect of nursing education is a focus on clinical learning. Benner (2004) suggested that hands-on learning is at the heart of good clinical judgment. Mentored clinical learning situations in both classrooms and practice sites offer critical opportunities for nurses to apply and integrate theoretical knowledge with actual events ( Field, 2004 ). A sound educational foundation expedites the acquisition of skills through experience ( Benner, 1984 ). Without background knowledge, nurses risk using poor judgment and lack the tools necessary to learn from experience.

Theory and principles enable nurses to ask the right questions to hone in on patient problems to provide safe care and make good clinical decisions. Bonner's (2003) research on nephrology nurses showed expert and non-expert nurses differed based on types of learning opportunities (both formal and informal) rather than years of experience. In a literature review on the relationship between nursing education and practice, Kovner and Schore (1998) reported mixed findings regarding whether and in what ways bachelor of science in nursing (BSN) prepared nurses' skills and abilities differ from those of associate degree and diploma-prepared nurses.

The collective education level of staff may impart a unique contribution to the development of expertise in the clinical setting. Few researchers have focused directly on the aggregate educational composition of the staff with whom a nurse practices as a factor affecting individual clinical nursing expertise. There are, however, examples where researchers have examined the relationship of contextual variables including education at the hospital level, to outcomes. For example, Aiken et al. (2003) found that the proportion of BSN-prepared nurses in a hospital was associated with lower surgical patient mortality and failure to rescue. In the same study, mean years of experience in a hospital was not associated with outcomes and did not alter the relationship between education and outcomes. Aiken et al. (2003) hypothesized that the effect of education was due, in part, to better critical thinking and clinical judgment skills associated with BSN preparation. Estabrooks, Midodzi, Cummings, Ricker, and Giovannetti (2005) also found that the proportion of BSN-prepared nurses in a hospital was associated with lower patient mortality. In a report on two studies, Blegen et al. (2001) found no association between the nursing unit's proportion of BSN-prepared nurses and patient falls and mixed-results for the association with medication errors.

Nurse practice environment and expertise

Many expert nurses leave hospital practice due to negative working conditions ( Orsolini-Hain & Malone, 2007 ). The nurse practice environment may offer a modifiable avenue through which nurse managers and administrators can cultivate nursing expertise and attract and retain nurse experts. Benner (1984) noted that the most skilled clinical nursing performance can be attained in a supportive environment where clinical learning with colleagues from all levels of expertise takes place. Organizations that facilitate a professional nursing practice environment foster clinical autonomy, support the continued education and advancement of nurses, increase the opportunity for shared experience and knowledge with physician colleagues, and provide support for professional decision making and action ( Lake & Friese, 2006 ). In one study of the nurse practice environment and expertise, researchers surveyed 103 nurses in two military hospitals ( Foley, Kee, Minick, Harvey, & Jennings, 2002 ). The investigators measured the practice environment with the Revised Nursing Work Index (NWI-R; Aiken & Patrician, 2000 ). Expertise was measured by the Manifestation of Early Recognition instrument, a 16-item scale based on the concepts of clinical expertise ( Minick, 2003 ). The results indicated significant, positive although modest correlations between nursing expertise and two of the three reported NWI-R subscales: control over practice and collaborative relationships between nurses and physicians.

Overall, our understanding of the relationship between contextual factors and expertise is limited. Thus, we examined the effects of both hospital contextual factors and individual nurse education and experience on clinical nursing expertise. We hypothesized that contextual factors would affect expertise over and above individual factors.

We conducted a secondary analysis of cross-sectional data from a 1999 statewide survey of registered nurses (RNs) in Pennsylvania ( Aiken, Clarke, Sloane, Sochalski, & Silber, 2002 ) to explore the relationship between individual nurse characteristics and hospital contextual factors and their association with nursing expertise. The sample for this analysis included acute care staff nurses ( n = 9,445) working in Pennsylvania acute care hospitals ( n = 182). To obtain stable and reliable estimates, data were limited to hospitals with survey data from at least 15 staff nurses ( Lake & Friese, 2006 ). The average number of respondents per hospital was 86 (range 15–225). Nurse responses were aggregated to calculate the measure of the nurse practice environment and also to create hospital-level measures of education and experience, using the mean as a measure of central tendency. Institutional review board approval for research with human subjects was obtained for the studies that generated these data ( Aiken et al., 2002 ).

The measures of the variables of interest, including the outcome of individual nurse expertise, independent variables at the nurse level, independent variables at the hospital level, and additional descriptive variables are described below.

The dependent variable was nurse-reported level of expertise. Based on Benner's (1984) work, the survey asked nurses to identify their level of expertise as one of the following: Advanced Beginner, Competent, Proficient, or Expert. The responses to this item were categorized as a four-category ordered variable. Nurses at the novice stage are generally still in nursing school and were not included in our study. In unpublished work by one of the authors ( Lake, 2002a ), nurses' self-reported expertise was strongly correlated with assessments by colleagues and supervisors.

Content validity questions were developed based on two characteristics that Benner (1984) suggested identify expert nurses: (a) how frequently the nurse was selected as a preceptor; and (b) how often the nurse was consulted by other nurses for clinical judgment. The response categories were Never , Rarely , Occasionally , and Frequently .

Independent Variables: Individual Nurse Measures

Nurses reported their highest nursing education degree as diploma, associate's degree (ADN), BSN degree, or master of science in nursing (MSN) degree. The categories were collapsed into less than a BSN (equal to zero) or BSN or higher (equal to one) for analysis. We chose to dichotomize education by BSN status to reflect the importance of the BSN in policy recommendations ( American Association of Colleges of Nursing, 2000 ; Benner, Sutphen, Leonard, Day, & Shulman, 2010 ). To ensure that there were not different relationships between diploma or ADN level of education and expertise, we estimated alternative models with each category of education and obtained equivalent findings.

Individual-level nurse experience was analyzed on a continuous scale of years. The data for this variable were drawn from responses to the survey question, “How many years have you worked as an RN?”

Independent Variables: Contextual Measures

Mean education.

Mean education was calculated as the group mean of a binary (zero/one) variable indicating an individual nurse's having a BSN degree or higher. This variable was equivalent to the proportion of RNs in each hospital with a BSN degree or higher.

Mean experience

Mean nurse experience in each hospital was measured by calculating the mean of the years of experience among the nurses practicing in each hospital. The data for this variable were drawn from responses to the survey question, “How many years have you worked as an RN?”

Nurse practice environment

The Practice Environment Scale of the Nursing Work Index (PES-NWI) was used to measure the professional nursing practice environment of each hospital ( Lake, 2002b ). The PES-NWI was adopted by the National Quality Forum (2004) as a national voluntary consensus standard for nursing-sensitive care.

The PES-NWI consists of 31 items in five subscales that characterize the domains of professional nursing practice environments ( Lake, 2007 ). All five subscales were used in this study. The nurses rated each item on a scale of 1 ( strongly disagree ) to 4 ( strongly agree ) to indicate whether the feature was “present in the current job.” The subscales are: (a) Nurse Participation in Hospital Affairs; (b) Staffing and Resource Adequacy; (c) Nursing Foundations for Quality of Care; (d) Nurse Manager Ability, Leadership, and Support of Nurses; and (e) Collegial Nurse/Physician Relations. The subscale score was the average of the subscale item responses. The potential score ranged from 1 to 4 with higher scores indicating more agreement that the subscale items were present in the current job. With a theoretical midpoint of 2.5, values above 2.5 indicated general agreement that the characteristics measured by the scales were present; values below 2.5 indicated disagreement. Agreement that characteristics were present was interpreted to indicate a favorable assessment of a domain of the practice environment as measured by the subscale. Hospitals were sorted into a three-level classification (“favorable,” “mixed,” and “unfavorable”) according to how many subscales were assessed favorably by their nurses ( Lake & Friese, 2006 ). Hospitals rated favorably on 0 or 1 subscale were classified as unfavorable, on 2 or 3 as mixed, and on 4 or 5 as favorable. The three-level categorization has been favored for use in research ( Aiken, Clarke, Sloane, Lake, & Cheney, 2008 ; Aiken et al., 2010 ; Friese, Lake, Aiken, Silber, & Sochalski, 2008 ; Kutney-Lee et al., 2009 ; Patrician, Shang, & Lake, 2010 ). We estimated alternative models with the continuous form of the variable and obtained equivalent findings.

The subscales exhibited high reliability at the individual and hospital levels. Internal consistency at the individual level was high (α ≥ .80) for all subscales in our sample. All hospital-level subscale measures were highly internally consistent (α = .86–.93).

Nurse characteristics were explored in descriptive and bivariate analyses. For the purposes of regression modeling, the individual nurse variables were centered on the grand mean—the mean for all the nurses in the sample was subtracted from each individual nurse's score for both the education and experience variables. Centering the individual-level variables and expressing them as a deviation from the grand mean removed correlation and concern with multi-collinearity ( Kreft & de Leeuw, 1998 ). Using the grand-mean-centered variables allowed for an assessment of a within-group effect. We then interpreted within-group effects, which represented the expected difference in log-odds of being in the next highest category of expertise between two nurses working in the same hospital who differ by one unit in an individual-level independent variable (education or experience). Analyzing the hospital-level group means for education and experience allowed for interpretation of a contextual effect. The contextual effect was interpreted as the difference in expertise between two nurses who have the same experience and education but who work in hospitals differing by one unit mean experience or one unit mean education. In other words, these contextual effects explained the association between expertise and the educational and experiential composition of the nursing staff with whom a nurse practiced.

We conducted robust, generalized ordered logistic regression to evaluate the association of nurse and hospital contextual factors with a four-category measure of individual nurses' expertise. Robust regression allows for the analysis of data clustered at an organizational level, such as nurses in hospitals, while addressing error variance arising from intraclass correlation of the data ( Huber, 1967 ; Lake, 2006 ; White, 1980 ).

Ordered logistic regression is appropriate with an ordered categorical-dependent variable such as our nurse expertise variable ranging from Advanced Beginner to Expert. An advantage of the generalized ordered logistic model, unlike the ordered logistic model, is that it is not limited by the parallel regression assumption ( Williams, 2006 ). That is, it does not constrain the parameter estimates to be constant across each of the four expertise groups. An assessment of the Brandt test ( Long & Freese, 2006 ) suggested that the parallel slopes assumption would be inappropriate for our data.

We considered and tested both robust and hierarchical linear regression as modeling approaches. The robust versus hierarchical linear models yielded roughly equivalent results in terms of significance and direction of effects. We chose ordered general robust regression because it handled the general problem of correlated observations while still accurately modeling the ordered nature of the dependent variable without being limited by the parallel regression assumption. All presented analyses were conducted using Stata (Version 10).

The generalized ordered logistic model was fitted,

where each Y ij was the probability that an individual nurse i was in the k th category out of the M = 4 potential categories of expertise, X ij 1 was the grand-centered deviation ( X ij 1 − X ̄ . 1 ) of individual nurse experience, X ij 2 was the grand-centered deviation ( X ij 2 − X ̄ . 11 ) of the dummy variable for having a BSN degree or higher, X ̄ . j 1 was the hospital mean of nurse experience, X ̄ . j 2 was the hospital proportion with BSN degree or higher, Z j was the vector of the dummy variables for hospital practice environment category (favorable, mixed, with unfavorable as the reference category). The model predicted the probability of being in the next highest category of expertise. Positive coefficients indicated that higher values on the explanatory variable made it more likely that the respondent would be in a higher category of expertise than the current one. Negative coefficients indicated that higher values on the explanatory variable increased the likelihood of being in the current or a lower category.

The data were cleaned and the final data set ( n = 8,611) contained no missing data on the analytic variables. The nurse respondents were 94% female with an average age of 39 years. The respondents reported an average of 13.2 years nursing experience. Thirty-eight percent of the nurses held a BSN degree.

The mean hospital-level nursing experience was 13.6 years. The higher hospital-level mean as compared to the overall mean indicates that nurse experience is not evenly distributed across hospitals. RNs comprised 74% of the licensed nursing staff in the hospitals. Most hospitals (66%) were classified as having mixed practice environments, 21% were classified as having unfavorable environments, and 13% were classified as having favorable environments.

Details of nurses' self-reported level of expertise are displayed in Table 1 . The majority of nurses (58%) rated themselves as Proficient, followed in descending proportional order by Competent (20%), Expert (16%), and Advanced Beginner (6%). Nurses' categorization of themselves into the four categories of expertise was correlated with the validation questions of “how frequently a nurse was selected as preceptor” ( r s = .34, p < .001) and “how frequently a nurse was consulted by other nurses for clinical judgment on a difficult clinical problem” ( r s = .42, p < .001).

Note . All numbers are expressed as percentages.

Table 2 displays the distribution of expertise across levels of individual nurse education. Spearman's rank-order correlations demonstrate a weak but significant association between nurse's highest degree (diploma, ADN, BSN, or MSN) and expertise classification ( r s = .03, p < .001). Nurses with an MSN reported the highest level of expertise, followed in descending order by Diploma, BSN, and ADN nurses. The high level of expertise reported by Diploma nurses suggests an experience effect as Diploma nurses had an average of 17.7 years of experience, just behind those with an MSN ( M = 18.9). Nurses with a BSN had an average of 10.9 years of nursing experience, and ADN-prepared nurses had the least experience ( M = 9.5). Individual nurse years of experience were significantly positively correlated with expertise ( r s = .48, p < .001).

Note . ADN, Associate's Degree in Nursing; BSN, Bachelor of Science Degree in Nursing; MSN, Master of Science Degree in Nursing.

Table 3 displays the generalized ordered logistic regression coefficients and corresponding odds ratios indicating the effects of nurse characteristics and organizational contextual effects on individual nurse expertise. The coefficients represent the change in log-odds of being in the next highest category of expertise related to one unit change in the predictor or independent variable with other variables held at their mean. The odds ratios indicate the odds of being in the next highest category of expertise related to one unit change in the predictor or independent variable with other variables held at their mean.

At the lower end of expertise (Advanced Beginner vs. all higher categories), the odds of being in a category higher than Advanced Beginner were 1.89 times greater for each additional year of individual nurse experience. However, the effect of individual experience diminishes at the higher end of expertise. Although still significant, the odds of being in the Expert category versus the lower categories were only 1.11 times greater for each additional year of individual nurse experience holding the other variables at their mean. The contextual effect of the mean level of experience at the hospital was not significant at any level.

Education was significant at both the individual and hospital levels. Nurses with a BSN were more likely to report higher expertise levels. Similarly, there was a contextual effect of education. Nurses practicing in hospitals with a higher proportion of BSN nurses were more likely to report higher levels of expertise. This effect was more pronounced at more advanced levels of expertise, as evidenced by the non-significant findings for the Advanced Beginner versus Competent, Proficient, and Expert level but highly significant findings at higher levels (e.g., Advanced Beginner and Competent vs. Proficient and Expert; and Advanced Beginner, Competent, and Proficient, vs. Expert). The practice environment was not significantly associated with clinical nursing expertise.

The contextual effect of a higher proportion of BSN-educated colleagues implies that altering the educational composition of the staff has implications for an individual nurse's level of expertise. As shown in Table 4 , if the proportion of BSN-prepared nurses in a hospital increased from 25% to 65%, the predicted probability of an average nurse in an average hospital reporting being an expert increased from .10 to .16.

Note . BSN, bachelor of science in nursing degree.

The study provides the first multi-level, multi-hospital evidence showing that the composition of a hospital's staff, particularly the aggregate level of education, contributes to clinical nurse expertise independent of individual education and experience level. Our findings also confirm the evidence from prior, smaller-scale studies showing that individual nurse level of education and years experience are related to clinical nursing expertise ( Bobay et al., 2009 ; Bonner, 2003 ).

Unexpectedly, we did not find support for our hypothesized association between a professional practice environment as measured by the PES-NWI and nursing expertise. This is surprising in light of the considerable literature suggesting the importance of the professional nurse practice environment measured by the PES-NWI on patient outcomes ( Aiken et al., 2008 ; Friese et al., 2008 ; Kutney-Lee et al., 2009 ; Vahey, Aiken, Sloane, Clarke, & Vargas, 2004 ). Given that the hospital sample demonstrated sufficient variation in practice environments, we do not attribute the lack of association to restricted range of the independent variable.

One potential explanation for the finding of no association between the practice environment and expertise is that the PES-NWI may not capture all facets of the practice environment relevant to the development of clinical nursing expertise. The American Association of Colleges of Nursing (AACN) has specified eight Hallmarks of the Professional Practice Environment (2002) , which are a comprehensive set of characteristics that permit “nurses to practice to their full potential” (p. 298). In a review of measures of the practice environment, Lake (2007) identified that although the PES-NWI was best instrument across criteria of theoretical relevance, ease of use, and dissemination, the measure does not cover all eight of the AACN hallmarks. The PES-NWI should be supplemented with four domains—autonomy, recognition/advancement of nurse preparation and expertise, professional development, and supportive relationships with peers—to cover the full spectrum of practice environment measurement. Recognition/advancement of nurse preparation and expertise is one potential domain where the PES-NWI may not assess how the practice environment advances clinical nursing expertise. The missing domain of professional development and supportive relationships with peers may also contain important factors associated with precepting and peer feedback that may enhance expertise.

As a possible alternative to the PES-NWI, a hospital's American Nurses Credentialing Center Magnet Recognition Program status could be used as an overarching contextual factor; however, none of the hospitals in our sample had achieved formal Magnet recognition at the time of data collection. We note that Foley et al. (2002) detected a significant association between the practice environment and expertise in nurse-level bivariate analyses. A key difference between that study and ours may be that both Foley et al.'s independent and dependent variables were a nurse's own ratings of the practice environment and expertise, introducing the potential for correlation due to the same source (i.e., the nurse). In our study, the independent variable (practice environment) was a structural contextual variable measured at the hospital level, and the dependent variable (expertise) was the nurse's own individual rating.

Our study also did not find a contextual effect of hospital experience. Although this seems reasonable given research showing no link between aggregate experience and patient outcomes ( Aiken et al., 2003 ), one limitation may be our one-dimensional measure of experience as a number of years. Benner (1984) noted that experience depends not only on the passage of time but also on the availability of actual situations through which a nurse can refine, elaborate, or disconfirm knowledge. A limitation of our study that is also common throughout the literature is the use of a measure of experience based solely on time.

Our findings may inform nurse executive strategies for shaping the composition of their staff to maximize the expertise of individual nurses. Executives can favor individual characteristics (a BSN or more experience) through recruitment or retention and advancement. Clinical advancement programs based on expertise and the use of expert nurses as clinical preceptors and educators may also augment the overall expertise within a hospital ( Moore, 2008 ). Hospital executives can institute policies favoring hiring of experienced nurses or nurses with a BSN through salary structures that differentiate based on these factors. They can also support completion of the BSN among existing nursing staff by providing tuition reimbursement.

We have shown that although an individual nurse's education level and years experience both influence his or her level of expertise, gains in the probability of an individual nurse being an expert can also be achieved through having a more educated nursing staff overall. To ensure that there is an adequate pool of nurses for hospitals to draw from, particularly to replace retiring experienced nurses, federal policy changes should be directed at preparing BSN-prepared nurses. Expanding the pool of available BSN nurses could be promoted through increasing nursing school program capacity (including faculty), tuition support, targeting of underrepresented groups, and promoting pathways to a BSN for RNs and licensed practical nurses (LPNs).

The current study had notable limitations. The cross-sectional design cannot establish causal relationships. The dependent variable of expertise is self-reported by nurses; although we provide some evidence of content validity, methods of measuring expertise other than self-report should be considered in future research. Our data only represent Pennsylvania nurses and hospitals and the findings may have limited generalizability. The data are from 1999 and some of the variables may have shifted over time. However, data from the National Sample Survey of Registered Nurses show that the percentage of RNs in the US with a BSN as their highest educational preparation has been relatively stable (32.7% in 2000 to 34.2% in 2004; U.S. Department of Health and Human Services, 2006 ). Additionally, although experience has increased somewhat, the increase is consistent with the aging of the nursing workforce nationally.

Our findings highlight important issues for nurse researchers going forward. Alternatives to self-reported expertise, such as peer or manager assessments, should be explored. Multidimensional measures that assess factors such as care coordination, clinical assessment and management, or relationships with patients may help to develop a more robust conceptualization of nurse expertise. Future research might consider individual trajectories of expertise and identify mechanisms influencing expertise longitudinally. A related research endeavor would develop a more nuanced measure of experience that might capture time as well as the nature of the clinical situations to which the nurse is exposed.

Future researchers should also examine the relationship between expertise and patient outcomes. Factors theoretically related to expertise have been associated with patient outcomes. For example, researchers have found the proportion of staff nurses with a BSN degree ( Aiken et al., 2003 ; Estabrooks et al., 2005 ; Tourangeau et al., 2007 ), nurse experience ( Blegen et al., 2001 ; Kendall-Gallagher & Blegen, 2009 ), and the nursing practice environment ( Aiken et al., 2008 ; Friese et al., 2008 ) to be significant predictors of patient outcomes. One pathway through which these factors may affect outcomes is the expertise level of nurses. Evidence illuminating this pathway will help complete the causal chain from nurse characteristics and environments to patient outcomes. This evidence can then guide the development of interventions to improve both expertise and outcomes. Clarifying the relationship between patient outcomes and clinical nursing expertise would also enlighten the current state of nursing science on factors such as staffing which largely treat RNs as equivalent in expertise, that is, a nurse is a nurse.

This investigation provides the first multi-hospital study of nursing expertise and its relationship to individual-level education and experience as well as hospital contextual characteristics. Our study did not identify the ideal nurse staffing composition to maximize expertise because ideal staffing is unique to each hospital. Our findings suggest, however, that both individual level and hospital contextual factors have important effects on expertise and must be considered when making human resource decisions.

Acknowledgments

This study was supported by the National Institute of Nursing Research (T32-NR-007104; P30-NR-005043; R01-NR-004513; Aiken, PI; and K01-NR00166; Lake, PI) and the Agency for Healthcare Research and Quality (K08--HS-017551; McHugh, PI). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institute for Nursing Research or the Agency for Healthcare Research and Quality. The authors thank the anonymous reviewers, Associate Editor, and Editor for their thoughtful reviews of and suggestions for this manuscript as well as Robert Lucero PhD, MPH, RN for his contributions to early drafts of this manuscript.

• Aiken LH, Clarke SP, Cheung RB, Sloane DM, Silber JH. Educational levels of hospital nurses and surgical patient mortality. JAMA. 2003; 290 :1617–1623. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Aiken LH, Clarke SP, Sloane DM, Lake ET, Cheney T. Effects of hospital care environment on patient mortality and nurse outcomes. Journal of Nursing Administration. 2008; 38 :223–229. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Aiken LH, Clarke SP, Sloane DM, Sochalski J, Silber JH. Hospital nurse staffing and patient mortality, nurse burnout, and job dissatisfaction. JAMA. 2002; 288 :1987–1993. [ PubMed ] [ Google Scholar ]
• Aiken LH, Patrician PA. Measuring organizational traits of hospitals: The Revised Nursing Work Index. Nursing Research. 2000; 49 :146–153. [ PubMed ] [ Google Scholar ]
• Aiken LH, Sloane DM, Cimiotti JP, Clarke SP, Flynn L, Seago JA, et al. Implications of the California Nurse Staffing Mandate for Other States. Health Services Research. 2010 doi: 10.1111/j.1475-6773.2010.01114.x. Advance online publication. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• American Association of Colleges of Nursing. The baccalaureate degree in nursing as minimal preparation for professional practice. Dec 12, 2000. Retrieved April 21, 2010, from http://www.aacn.nche.edu/Publications/positions/baccmin.htm . [ PubMed ]
• American Association of Colleges of Nursing. Hallmarks of the professional nursing practice environment. Journal of Professional Nursing. 2002; 18 :295–304. [ PubMed ] [ Google Scholar ]
• Benner P. From novice to expert: Excellence and power in clinical nursing practice. Menlo Park, CA: Addison-Wesley; 1984. [ Google Scholar ]
• Benner P. Using the Dreyfus Model of Skill Acquisition to describe and interpret skill acquisition and clinical judgment in nursing practice and education. The Bulletin of Science, Technology and Society. 2004; 24 :188–199. [ Google Scholar ]
• Benner P, Sutphen M, Leonard V, Day L, Shulman LS. Educating nurses: A call for radical transformation. San Francisco: Jossey-Bass; 2010. [ Google Scholar ]
• Benner P, Tanner C. Clinical judgment: How expert nurses use intuition. American Journal of Nursing. 1987; 87 :23–31. [ PubMed ] [ Google Scholar ]
• Blalock HM. Contextual-effects models: Theoretical and methodological issues. Annual Review of Sociology. 1984; 10 :353–372. [ Google Scholar ]
• Blau PM. Structural effects. American Sociological Review. 1960; 25 :178–193. [ Google Scholar ]
• Blegen MA, Vaughn TE, Goode CJ. Nurse experience and education: Effect on quality of care. Journal of Nursing Administration. 2001; 31 :33–39. [ PubMed ] [ Google Scholar ]
• Bobay KL. Does experience really matter? Nursing Science Quarterly. 2004; 17 :313–316. [ PubMed ] [ Google Scholar ]
• Bobay K, Gentile DL, Hagle ME. The relationship of nurses' professional characteristics to levels of clinical nursing expertise. Applied Nursing Research. 2009; 22 :48–53. [ PubMed ] [ Google Scholar ]
• Bonner A. Recognition of expertise: An important concept in the acquisition of nephrology nursing expertise. Nursing & Health Sciences. 2003; 5 :123–131. [ PubMed ] [ Google Scholar ]
• Books J, Prysby C. Studying contextual effects on political behavior: A research inventory and agenda. American Politics Quarterly. 1988; 16 :211–238. [ Google Scholar ]
• Christensen M, Hewitt-Taylor J. From expert to tasks, expert nursing practice redefined? Journal of Clinical Nursing. 2006; 15 :1531–1539. [ PubMed ] [ Google Scholar ]
• Clarke SP, Rockett JL, Sloane DM, Aiken LH. Organizational climate, staffing, and safety equipment as predictors of needlestick injuries and near-misses in hospital nurses. American Journal of Infection Control. 2002; 30 :207–216. [ PubMed ] [ Google Scholar ]
• Dreyfus HL, Dreyfus SE. The relationship of theory and practice in the acquisition of skill. In: Benner PA, Tanner CA, Chesla CA, editors. Expertise in nursing practice: Caring, clinical judgment, and ethics. New York: Springer; 1996. [ Google Scholar ]
• Ericsson KA, Whyte J, Ward P. Expert performance in nursing: Reviewing research on expertise in nursing within the framework of the expert-performance approach. Advances in Nursing Science. 2007; 30 :E58–E71. [ PubMed ] [ Google Scholar ]
• Estabrooks CA, Midodzi WK, Cummings GG, Ricker KL, Giovannetti P. The impact of hospital nursing characteristics on 30-day mortality. Nursing Research. 2005; 54 :74–84. [ PubMed ] [ Google Scholar ]
• Field DE. Moving from novice to expert—The value of learning in clinical practice: A literature review. Nurse Education Today. 2004; 24 :560–565. [ PubMed ] [ Google Scholar ]
• Foley BJ, Kee CC, Minick P, Harvey SS, Jennings BM. Characteristics of nurses and hospital work environments that foster satisfaction and clinical expertise. Journal of Nursing Administration. 2002; 32 :273–282. [ PubMed ] [ Google Scholar ]
• Friese CR, Lake ET, Aiken LH, Silber JH, Sochalski J. Hospital nurse practice environments and outcomes for surgical oncology patients. Health Services Research. 2008; 43 :1145–1163. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Huber PJ. The behavior of maximum likelihood estimates under non-standard conditions. Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability; Berkeley, CA: University of California Press; 1967. pp. 221–233. [ Google Scholar ]
• Kanai-Pak M, Aiken LH, Sloane DM, Poghosyan L. Poor work environments and nurse inexperience are associated with burnout, job dissatisfaction and quality deficits in Japanese hospitals. Journal of Clinical Nursing. 2008; 17 :3324–3329. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Kendall-Gallagher D, Blegen MA. Competence and certification of Registered Nurses and safety of patients in intensive care units. American Journal of Critical Care. 2009; 18 :106–113. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Kovner CT, Schore J. Differentiated levels of nursing work force demand. Journal of Professional Nursing. 1998; 14 :242–253. [ PubMed ] [ Google Scholar ]
• Kreft IGG, de Leeuw J. Introducing multilevel modeling. London: Sage; 1998. [ Google Scholar ]
• Kutney-Lee A, McHugh MD, Sloane DM, Cimiotti JP, Flynn L, Neff DF, et al. Nursing: A key to patient satisfaction. Health Affairs. 2009; 28 :w669–w677. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Lake ET. Development of survey measures of clinical nursing expertise. Paper presented at the 14th Annual Scientific Session of the Eastern Nursing Research Society.2002a. [ Google Scholar ]
• Lake ET. Development of the practice environment scale of the Nursing Work Index. Research in Nursing & Health. 2002b; 25 :176–188. [ PubMed ] [ Google Scholar ]
• Lake ET. Multilevel models in health outcomes research. Part II: Statistical and analytic issues. Applied Nursing Research. 2006; 19 :113–115. [ PubMed ] [ Google Scholar ]
• Lake ET. The nursing practice environment: Measurement and evidence. Medical Care Research and Review. 2007; 64 :104S–122S. [ PubMed ] [ Google Scholar ]
• Lake ET, Friese CR. Variations in nursing practice environments: Relation to staffing and hospital characteristics. Nursing Research. 2006; 55 :1–9. [ PubMed ] [ Google Scholar ]
• Long JS, Freese J. Regression models for categorical dependent variables using Stata. 2nd. College Station, TX: Stata Press; 2006. [ Google Scholar ]
• Minick P. Manifestations of early recognition: A measure of nursing expertise. In: Strickl OI, DiIorio C, editors. Measurement of nursing outcomes. New York: Springer; 2003. pp. 294–303. [ Google Scholar ]
• Moore ML. Preceptorships: Hidden benefits to the organization. Journal for Nurses in Staff Development. 2008; 24 :E9–E15. [ PubMed ] [ Google Scholar ]
• National Quality Forum. National Voluntary Consensus Standards for Nursing-Sensitive Care: An initial performance measure set—A consensus report (No NQFCR-08-04) Washington, DC: 2004. [ Google Scholar ]
• Orsolini-Hain L, Malone RE. Examining the impending gap in clinical nursing expertise. Policy, Politics, & Nursing Practice. 2007; 8 :158–169. [ PubMed ] [ Google Scholar ]
• Patrician PA, Shang J, Lake ET. Organizational determinants of work outcomes and quality care ratings among Army Medical Department registered nurses. Research in Nursing and Health. 2010; 33 :99–110. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Shortell SM, Getzen TE. Measuring hospital medical staff organizational structure. Health Services Research. 1979; 14 :97–110. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Simmons B, Lanuza D, Fonteyn M, Hicks F, Holm K. Clinical reasoning in experienced nurses. Western Journal of Nursing Research. 2003; 25 :701–719. [ PubMed ] [ Google Scholar ]
• Tourangeau AE, Doran DM, McGillis Hall L, O'Brien Pallas L, Pringle D, Tu JV, et al. Impact of hospital nursing care on 30-day mortality for acute medical patients. Journal of Advanced Nursing. 2007; 57 :32–44. [ PubMed ] [ Google Scholar ]
• U.S. Department of Health and Human Services. The registered nurse population: Findings from the March 2004 National Sample Survey of Registered Nurses. Washington, DC: Health Resources and Services Administration, Bureau of Health Professions; 2006. [ Google Scholar ]
• Vahey DC, Aiken LH, Sloane DM, Clarke SP, Vargas D. Nurse burnout and patient satisfaction. Medical Care. 2004; 42 :II57–II66. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• White H. A heteroskedasticity-consistent covariance matrix estimator and a direct test for heteroskedasticity. Econometrica. 1980; 48 :817–838. [ Google Scholar ]
• Williams R. Generalized ordered logit/partial proportional odds models for ordinal dependent variables. The Stata Journal. 2006; 6 :58–82. [ Google Scholar ]
• Young WB, Lehrer EL, White WD. The effect of education on the practice of nursing. Image: The Journal of Nursing Scholarship. 1991; 23 :105–108. [ PubMed ] [ Google Scholar ]