Introduction

Teamwork is crucial for medical teams. Research has shown that teamwork is strongly related to clinical performance [1], while teamwork failures can have devastating consequences for patients. At the same time, training teams leads to better clinical outcomes [2,3]. Teamwork is also often the central focus of healthcare simulation training, most prominently in the form of crew resource management programmes [4].

Despite its importance, educators and researchers often struggle to effectively measure teamwork in simulation settings. Two main reasons contribute to this challenge. First, an underlying framework and clear definitions of what should be measured are often missing. Second, many educators lack methodological knowledge about how teamwork can be assessed systematically.

The goal of this article is to provide a practical overview of methods for measuring teamwork in healthcare simulation. While we cannot provide a comprehensive account of all possible methods and analytical considerations, we focus on the three most common approaches: survey measures, systematic behavioural observation (SBO) and Behaviourally Anchored Rating Scales (BARS). We also refer to further literature that can guide the choice of a specific method.

Relevance to healthcare simulation

The economist Peter Drucker famously stated: ‘You can’t manage what you don’t measure’. This insight is highly relevant for simulation-based education. Teamwork and interprofessional collaboration are often the primary focus of simulation training; yet surprisingly little emphasis is placed on systematically and reliably measuring these aspects during training.

We identify four reasons why measuring teamwork can offer significant benefits. First, teamwork assessments help reveal training needs by making teamwork visible. They highlight specific strengths and weaknesses, which in turn enable educators to tailor debriefings and future training sessions more effectively. Second, systematic measurement can enhance the quality of debriefings. Data from surveys, observations or rating scales provide objective reference points to enrich discussions. In particular, BARS offer concrete behavioural examples of effective teamwork that can be directly incorporated into training. Third, measurement allows educators to monitor progress over time. Repeated assessments can show whether a team improves across training cycles or whether a specific intervention has been successful. Finally, appropriate teamwork measures support research and benchmarking. They enable comparisons across teams, hospitals or training centres and facilitate scientific studies on the effectiveness of simulation-based education.

Defining what to measure

The first step in measuring teamwork is to decide which aspects are most relevant. The term teamwork is often used loosely in the literature, and clear definitions are sometimes lacking. To provide structure, we draw on the input–process–output (IPO) model of team performance, which describes how inputs (e.g. knowledge, team composition [5]) are transformed into outcomes through interdependent actions within the team [6]. In healthcare simulation, the focus is typically on these interdependent actions – so-called team processes. It is useful, however, to distinguish team processes from emergent states. Team processes refer to verbal (e.g. communication) and behavioural (e.g. coordination) interactions among team members that help organize and carry out clinical tasks, transforming individual contributions into effective collective performance aimed at a shared goal, such as high-quality patient care. In contrast, emergent states describe the shared cognitive, motivational and emotional characteristics of a team that develop and change over time, depending on the team’s context, interactions and experiences. Unlike team processes, which focus on how members interact, emergent states reflect the qualities of the team. Examples include psychological safety, the shared belief that it is safe to speak up [7] or team cohesion, a shared bond and commitment among team members that reflects their attraction to the team and their motivation to achieve its goals together.

This distinction is important because it directly affects which measurement methods are suitable. Emergent states, by definition, are internal to team members and cannot be directly observed. To measure them, surveys and self-reports are necessary. Team processes, however, involve observable actions and communication, making them accessible to external observers who can code and quantify them.

In the following, we introduce three common approaches for measuring teamwork: survey measures, SBO and BARS. We also provide guidance on how to apply these methods effectively in healthcare simulation. Table 1 provides a summary of the three methods, including advantages, limitations and examples.

Table 1:

Overview of three teamwork measurement approaches

Aspect	Survey measures	Systematic behavioural observation	Behaviourally anchored rating scales
Purpose	Capture team members’ perceptions, attitudes or experiences	Capture observable behaviours and interactions in real time	Assess the quality of observable teamwork behaviours
Data source	Team members (individuals’ perception aggregated to team level)	Trained observers that rate individual and/or team behaviours	Trained raters/observers (usually directly referred to team level constructs)
Data type	Subjective self-assessed ratings	Behavioural frequencies, sequences or categories	Judgement-based ratings anchored in behavioural exemplars
Timing of assessment	After a simulation (post-scenario questionnaires) or training (post-training questionnaire)	During the simulation (real-time coding or video analysis)	During or immediately after the simulation (performance assessment)
Advantages	Easy to administer, can cover a broad spectrum of psychological constructs (attitudes, behaviour, emergent states etc.), efficient for large samples and longitudinal designs	Rich behavioural detail, real-time data, high ecological validity because the behaviour of interest is directly observed	Combines objectivity with evaluative judgement, provides structured feedback, shared performance language and benchmarking
Limitations	Response bias, limited behavioural validity	Labour-intensive, requires coding schemes, can only capture behaviour that is visible from the outside	Time-consuming to develop and validate, requires training, can only evaluate behaviour that is visible from the outside
Example	Psychological safety [33], Teamwork [11]	Coding speaking up behaviour during a simulated case [34]	ANTS (Anaesthetists’ Non-Technical Skills), NOTECHS [25]

Across all three approaches, measurement quality is a central concern. Regardless of whether teamwork is assessed through self-report surveys, SBO or BARS, the resulting data must be both reliable and valid to support meaningful interpretation. Reliability refers to the consistency of measurement: a method should produce similar results when applied under comparable conditions, such as across repeated simulations or different raters observing the same team [8]. Validity concerns whether a measure actually captures the aspect of teamwork it is intended to assess [8,9]. This requires clear conceptual definitions, appropriate operationalization of constructs or behaviours and careful attention to potential sources of bias, such as rater subjectivity, recall bias or social desirability. Without adequate reliability and validity, observed differences in teamwork may reflect measurement artefacts rather than true differences in team functioning. For this reason, considerations of measurement quality are essential when selecting and applying any teamwork assessment method in healthcare simulation.

Survey measures

Surveys are a common tool to assess teamwork [10].  They consist of several questions, called items, that together measure different specific teamwork-related concepts such as psychological safety or team cohesion. Each of these concepts is known as a construct, and each item represents one small aspect of it [9]. When combined, the items form a scale that provides an overall score for how strongly a team or individual demonstrates that construct. Although surveys can, in principle, be completed by external raters, we use the term ‘survey’ here exclusively to refer to self-reported questionnaire measures that capture team members’ subjective perceptions, attitudes or experiences.

Most teamwork constructs are emergent states that develop through interaction and are shared among team members. Because these states or climate variables cannot be observed directly, surveys rely on the perceptions of the team members themselves. For example, to assess cohesion, each member might rate statements such as ‘Our team works together as one unit’. These individual responses are valuable because teamwork is built from the shared experiences and perceptions of the people who form the team.

Once all members have completed the survey, their responses can be combined to represent the team as a whole. The simplest way to do this is to calculate the team’s average score, which reflects the group’s shared view. Educators can also look at differences within the team (e.g. standard deviation). Large differences in how team members respond may suggest that not everyone experiences teamwork in the same way. Both the average score and the variation between members can offer useful insights for training and debriefing.

Examples of established survey tools to measure teamwork in healthcare are the self-assessment teamwork tool (SATT) [11] or TeamSTEPPS [12]. Many survey items are phrased to capture general perceptions of teamwork (e.g. ‘Team members communicate clearly and effectively with one another’). When surveys are used in simulation settings, however, it is necessary to adapt the frame of reference of each item. This is a crucial step to ensure construct validity, particularly when the aim is to assess teamwork during a specific scenario rather than general team functioning [13]. In such cases, items should explicitly reference the simulation episode. For example, the item above can be adapted to: ‘During the last scenario, team members communicated clearly and effectively with one another’. This contextualization should be applied consistently across all items to ensure a shared and unambiguous reference point.

The same principle applies to measures of emergent states. For instance, an item from Amy Edmondson’s psychological safety scale [14] – ‘Members of this team are able to bring up problems and tough issues’ – can be adapted to a simulation context as: ‘During the scenario, members of this team were able to bring up problems and tough issues’. When selecting which aspects of teamwork to measure, it is also important to consider whether the construct is likely to change within the observed time frame. Teamwork behaviours, such as communication, can reasonably be expected to improve across repeated simulation scenarios and debriefings (e.g. from scenario one to scenario three). In contrast, emergent states such as psychological safety are typically more stable within the same team and context and are therefore less likely to fluctuate substantially across closely spaced scenarios, as they usually require more time and repeated experiences to change.

Systematic behavioural observation

SBO is a powerful method for studying real-time team processes, particularly in dynamic environments such as acute care. It involves systematically counting and recording the occurrence of specific, predefined behaviours during a simulation or debriefing, for example, instances of speaking up, coordination or reflection. To do this, researchers use a coding scheme that defines each behaviour in clear, observable terms. Such a scheme can be adapted from existing frameworks or developed specifically for the context under study [17]. Coding schemes can also be hierarchical, meaning that coders first identify who performs the behaviour (e.g. nurse, resident) and then specify what behaviour was shown. Using this structured approach, observers capture and code behaviours, typically from video or audio recordings, to quantify how interaction patterns unfold over time [18]. Compared with survey measures, systematic observation allows researchers to assess actual, context-embedded behaviour rather than perceptions or retrospective judgements. For example, the Co-ACT framework quantifies explicit and implicit coordination patterns in acute care teams [19] and the TuRBO system captures in-action reflection cycles during ongoing care episodes [20]. Together, these approaches demonstrate how SBO enables researchers to examine complex, emergent team processes such as coordination or reflection directly from behaviour in context.

In practice, SBO requires several design decisions concerning what to code (for example, reflection, coordination or leadership behaviours), how to code (for example, continuous versus interval coding) and how to train coders [18]. In interval coding, the observation period is divided into fixed time windows (for example, 10 seconds), and raters record whether a behaviour occurred within each interval (for example, ‘Behaviour A’ in interval 1: yes/no). In continuous event coding, by contrast, each behaviour is coded exactly when it occurs and marked with a time stamp (for example, ‘Behaviour A’ at 11 minutes 22 seconds). The choice of method depends on the level of detail needed and the study’s purpose. Video recordings allow for richer and more complex coding schemes because coders can rewatch sequences, clarify ambiguities and capture overlapping behaviours. Live coding, in contrast, is more practical in training settings but typically requires a smaller number of clearly defined categories to observe to remain manageable.

Behaviourally anchored rating scales

Building on the distinction above between survey methods that capture self-rated perceptions and SBO that records what teams actually do or say, BARS translate those observations into an assessment of how well teamwork behaviours are performed. Like systematic observation, an external rater observes and evaluates behaviour, but instead of simply counting its occurrence, BARS require a qualitative judgement of performance (for example, 1 = poor to 5 = excellent). Rather than asking for a general rating such as ‘good’ or ‘poor’, BARS provide concrete behavioural examples – called anchors – at each scale point that illustrate what low, moderate and high levels of performance look like. These anchors improve interrater consistency, make evaluations more transparent and support targeted feedback.

Widely used BARS frameworks such as Anaesthetists’ Non-Technical Skills (ANTS) [25,26]  and Non-Technical Skills for Surgeons (NOTECHS) [27,28] illustrate the approach: they define core dimensions (e.g. teamwork, decision-making, situation awareness, task management) and offer anchored descriptors (e.g. ‘Team members participate actively in checking procedures’) that help faculty judge the quality of behaviours during scenarios and debriefings. Because anchors make expectations explicit, BARS are especially useful for guiding debriefs (‘what high-quality teamwork looked like here’) and for monitoring improvement across sessions. BARS complement surveys and SBO by offering a practical, reliable way to convert observed actions into standardised judgements of teamwork competence.

Choosing the right level of assessment: Team or individual?

An important yet often overlooked question when measuring teamwork is at what level the assessment should take place: the team or the individual. The choice depends on the purpose of the evaluation and the underlying theoretical assumptions about what the intervention or measurement aims to capture.

When evaluating a team-level intervention (e.g. a simulation-based team training), the appropriate level of analysis is the team, because it is expected that the intervention has an impact on the whole team. Here, data are aggregated across team members, and comparisons (e.g. pre–post training) are made using team-level scores (i.e. one teamwork score per team). Consequently, the sample size corresponds to the number of teams rather than individual participants. This approach reflects the assumption that teamwork phenomena, such as shared mental models and coordination, are emergent properties that exist at the team level rather than as individual traits or behaviours.

In contrast, when training or measurement focuses on individual skills, attitudes or behaviours, the unit of analysis should be the individual. For example, individual speaking-up behaviour, such as how often or how effectively a single team member voices concerns or challenges decisions during a simulation, can be assessed for each participant separately. In this case, pre–post comparisons are conducted at the individual level, with the number of participants representing the sample size.

However, because individuals are nested within teams, their responses are not statistically independent; members of the same team often influence each other. For instance, an individual’s performance might differ depending on whether they work in a highly experienced or an unfamiliar team, although the individual’s skills remain the same. Similarly, individual speaking-up behaviour may vary across team contexts: the same person may voice concerns more readily in a psychologically safe team than in a hierarchical or unfamiliar team. As a result, two individuals working in the same team are likely to show more similar levels of speaking-up behaviour or performance than two individuals from different teams, even when their individual skills are comparable. To account for this dependency, multilevel analytical techniques such as hierarchical linear modelling (HLM) or mixed-effects models should be used, as these explicitly partition variance between the individual and team levels [32].

The issue of levels is particularly relevant for survey measures, since each team member typically completes the questionnaire individually. However, similar considerations apply to SBO and BARS. In systematic observation, for example, coders can use a hierarchical scheme that records both who performs a behaviour and what the behaviour is, thereby allowing analyses at both the individual and team levels. Alternatively, observers can focus on collective team behaviour without distinguishing between members, resulting in team-level data. Likewise, most existing BARS instruments in healthcare are designed to evaluate teamwork as a collective performance (e.g. ‘the team reevaluated its procedures’), but it is also possible to adapt them to assess individual contributions using the same anchored scales (e.g. ‘Person A reevaluated the procedures’).

Selecting the right method for simulation research and training

Choosing the most suitable method to measure teamwork depends on what aspect of teamwork you want to understand and how the results will be used. Survey measures are ideal for capturing how team members feel about their teamwork, such as their sense of trust in the team, cohesion or psychological safety. SBO is best when the focus is on what teams do in practice, allowing researchers or educators to record and quantify real behaviours as they unfold during a simulation (e.g. speaking up behaviour). BARS bridge these two perspectives by combining observation with evaluative judgement, offering a structured way to assess the quality of observed teamwork behaviours.

No single approach is superior in every situation. Instead, each method provides a different lens for understanding teamwork. Educators may use surveys to explore how team members experience collaboration, while researchers might combine BARS with behavioural observation to gain a richer picture of interaction patterns and performance. Using several methods together can provide the most complete insight into team functioning and learning.

Ultimately, the method should match both the purpose of the measurement and the resources available. A short survey may be most practical for routine evaluation, whereas a detailed observational study can offer a deeper understanding in research projects. The overview in Table 1 should help educators and researchers choose the right approach. Whichever method is chosen, the key is to apply it systematically, ensure measurement quality and use the results to strengthen teamwork training and, ultimately, patient care.

Suggestions for further reading

•Brauner E, Boos M, Kolbe M, editors. The Cambridge handbook of group interaction analysis. Cambridge: Cambridge University Press. 2018.

•Fletcher G, Flin R, McGeorge P, Glavin R, Maran N, Patey R. Anaesthetists’ Non-Technical Skills (ANTS) system: handbook v1.0. Aberdeen: University of Aberdeen. 2012. Available from: https://research.abdn.ac.uk/wp-content/uploads/sites/14/2019/03/ANTS-Handbook-2012-1.pdf [Accessed 29 May 2026].

•Flin R, Patey R. Non-technical skills for anaesthetists: developing and applying ANTS. Best Practice & Research Clinical Anaesthesiology. 2011;25(2):215–227.

•González-Romá V, Hernández A. Conducting and evaluating multilevel studies: recommendations, resources, and a checklist. Organizational Research Methods. 2022;25(5):795–825.

•Tschan F, Zimmermann J, Semmer NK. Rules for coding scheme development. In: Brauner E, Boos M, Kolbe M, editors. The Cambridge handbook of group interaction analysis. Cambridge: Cambridge University Press. 2018. pp. 191–207.