Figure 1Selection process for entrustable professional activities (EPAs) submitted to the first round of voting.
DOI: https://doi.org/https://doi.org/10.57187/s.3592
Postgraduate medical training curricula must ensure that junior physicians moving towards unsupervised practice are proficient in all the essential domains of their profession and, therefore, adequately prepared to deliver competent care [1, 2]. In the past two decades, competency-based medical education (CBME) has progressively become the principal educational means of achieving these goals [1]. CBME provides training outcomes in terms of essential competencies derived from an analysis of societal and patient needs[3]. Although competencies clearly describe medical training outcomes, they represent a challenge in assessment. Competencies describe personal attributes and are not easily observable in the workplace [4, 5]. Therefore, entrustable professional activities (EPAs) were developed as a useful complement to CBME [6, 7]. EPAs represent concrete daily clinical tasks that trainee physicians should be able to handle with increasing autonomy and that require the appropriate integration of multiple competencies [8, 9]. EPAs facilitate assessment processes because they are observable [10, 11], which is why clinical supervisors and educators have shown a growing interest in them since their introduction as a concept by ten Cate in 2005 [12–14]. Efforts to define EPAs for training programmes remain ongoing in various contexts, with each speciality or medical school moving at its own pace.
Published guidelines providing best practices on how to describe EPAs [9, 15] and select them for comprehensive training programmes are new [16–20]. Most of the literature on EPAs was published before these guidelines and lacked, for example, details on the approaches used to establish lists of EPAs [14]. These approaches appear relatively heterogeneous across training programmes. A recent scoping review found that high-level evidence-based research on the validity of EPAs was globally lacking [14].
Once we were aware of the challenges involved in the process of selecting entrustable professional activities, and considering the existing recommendations and guidelines, we designed a strict, step-by-step approach to establishing a list of entrustable professional activities for postgraduate training in general internal medicine (GIM) in Switzerland.
To overcome the limitations of previous studies and provide evidence of validity at each step of the process, we also relied on the expertise of specialists in medicine, consensus process technology and medical education.
The Swiss Society of General Internal Medicine (SSGIM) is the largest medical speciality society in Switzerland, with 8000 members. The SSGIM is the result of the 2015 merger between the former societies of internal medicine and general (primary care) medicine; it thus covers internal medicine practised in both ambulatory and hospital settings. The postgraduate training programme for general internal medicine lasts five years, structured as three years of “primary training” followed by two years of elective “secondary training”. Primary training involves at least two years of residency in a hospital general internal medicine department. Secondary training is organised into modules, with a broad range of disciplines and settings for trainees to choose from according to whether they wish to complete their training as a hospital internist or a primary care physician [21]. The programme also requires junior physicians to undergo training in different healthcare institutions.
In 2019, the SSGIM funded a project to develop a list of EPAs for postgraduate training in general internal medicine. This article describes the multi-step approach used to produce a list of quality-verified candidate EPAs, with a special focus on the national consensus process.
The methodological framework used to select our entrustable professional activities comprised four major steps. First, we performed a systematic literature review to identify all the published lists of EPAs for postgraduate general internal medicine training programmes. Second, we conducted a series of focus group discussions to complement the literature review and add missing EPAs specific to the local context. Third, we submitted a list of EPAs resulting from the two initial steps to a nationwide consensus process [22]. Fourth, we analysed the resulting consensus list using standard EPA quality criteria [23].
The project team drafted a preliminary list of potential participants, representing the postgraduate education committee of SSGIM and the educational directors of various university and non-university hospitals and outpatient clinics of the three main linguistic regions. People on the list were invited to participate in the project and propose other participants. In this way, a purposive sample of physicians and medical educators specifically involved in postgraduate education and clinical supervision – trained and board-certified as Swiss GIM specialists – were invited by email to participate in the focus groups and consensus process. Senior physicians of different ages, with various levels of experience, from chief residents to postgraduate training programme directors and members of the SSGIM’s medical directorate, were eligible. We invited female and male physicians from across Switzerland’s German-, French- and Italian-speaking regions. We also considered the types of activities that those physicians did across different general internal medicine settings (large vs small institutions and hospital vs outpatient clinic vs private practice).
The methodological details and results of our systematic review are described in a previous publication [24]. We aimed to identify all the available lists of entrustable professional activities developed internationally for postgraduate training in general internal medicine. We followed the PRISMA guidelines for conducting and reporting our systematic review. Search strategies, including keywords and databases, were built with the help of an expert librarian. We extended our search to grey literature to be as exhaustive as possible. The study selection and data extraction were conducted by two authors independently. The methodological quality of the included articles was assessed by the same two authors, using validated tools.
The objectives of our focus group were to discuss which professional activities were the most important in general internal medicine and identify candidate entrustable professional activities specific to Swiss contexts. We conducted four focus groups between 15 January and 22 January 2020, two involving physicians from inpatient settings and two involving physicians from outpatient settings, with two moderated in French and two in German.
We developed a focus group document to guide discussions (supplementary file 1, available for download as a separate file at https://doi.org/10.57187/s.3592). After reminding the participants about the concept and definition of EPAs, we presented each group with an initial list of 24 EPAs selected from the preliminary literature review. These formed the basis for discussions within our framework, built around three main issues. First, participants were asked to discuss the appropriateness and relevance of the selected EPAs. Second, they were asked to identify any additional EPA relevant to their practice. Third, with a view to the evolution of medical practice, they were asked to envision new EPAs that might become relevant in the future.
The focus groups lasted two hours and were moderated by two physicians specialising in medical education (MM/DG or MM/SF). A methodologist (VP) managed the overall focus group schedule and took field notes to facilitate the subsequent transcription of the discussions. At recruitment, the physicians received an information sheet containing the research project’s aims and signed a written agreement that their data could be used anonymously in the analysis. Twenty-eight general internal medicine physicians agreed to participate, of whom half were female, 16 were working in hospital settings (nine from French- and seven from German-speaking regions) and were 12 working in ambulatory settings (eight from French- and four from German-speaking regions). Among those working in hospital settings, nine were chief residents, and the remaining seven were attending physicians.
The focus groups were audio-recorded and transcribed verbatim. The participants were assigned numbers to ensure their anonymity. Qualitative content analysis [25] was performed using MAXQDA Pro 2020 software (VERBI Software GmbH, Berlin, Germany). Themes and broad categories were inductively derived from the content analysis of the verbatim transcriptions, and these were used in the next step of studying EPAs. We then deductively extracted concepts useful for deriving future EPAs.
Our nationwide consensus process used a modified RAND appropriateness method (RAM) with two rounds of rating. RAM is an internationally recognised technique that uses a systematic approach and a panel of experts to classify policies or practices according to their appropriateness [22, 26]. It combines features of the Delphi and nominal group techniques [27]. The Delphi technique’s stages include anonymous iterative rounds of postal or email questionnaires, the collection of individual and group feedback between rounds, and a summary of the findings [22]. The nominal group technique consists of structured face-to-face interactions that allow discussion and debate; it is a particularly useful method for brainstorming and finding consensus [22]. The RAM involves participants responding to a traditional Delphi questionnaire. During the subsequent face-to-face meeting, participants discuss the results of the first round, and finally, they participate individually in a second round of voting on the Delphi questionnaire’s items [28].
We combined the list of EPAs discovered via our systematic review with those revealed through the focus group analysis. These steps generated a final list of 247 candidate EPAs that were submitted for rating (figure 1).
Figure 1Selection process for entrustable professional activities (EPAs) submitted to the first round of voting.
Twenty-eight of the 34 experts invited to participate in the first round of individual ratings of the EPAs did so between June and July 2021. They comprised 10 female and 18 male experts; 13 from French-speaking regions, three from Italian-speaking regions and 12 from German-speaking regions; and 16 working in hospital settings and 12 working in ambulatory settings. Six of the 28 participating experts were chief residents, representing recently trained physicians able to reflect a learner’s perspective. Each of the 247 EPAs presented was to be rated using six statements (see table 1). The experts were asked to rate each candidate EPA according to their level of agreement with each statement on a 9-point Likert scale.
Table 1Statements about the candidate entrustable professional activities (EPAs) submitted for rating.
| 1. The proposed EPA is relevant for hospital practice. | |
| 2. The proposed EPA is relevant for ambulatory practice. | |
| 3. Full entrustment should be expected during: | the first part of the training (1st–3rd PGY*). |
| the second part of the training (4th–5th PGY*). | |
| 4. The maximum level of entrustment expected at the end of the postgraduate general internal medicine training should be: | level 4: performed with at-distance supervision. |
| level 5: no supervision needed and able to supervise junior colleagues. | |
* PGY: postgraduate training year
Fourteen experts who participated in round one agreed to participate in round two, but only 11 eventually took part during a face-to-face panel meeting held in Bern on 23 November 2021. They comprised four female and seven male experts; three from French-, two from Italian- and six from German-speaking regions; and five working in hospital settings and six working in ambulatory settings. Two of the 11 participating experts were chief residents.
The panel received a detailed summary of the first round’s results. Round two discussed and rated only those entrustable professional activities whose relevance or level of entrustment had proven uncertain in round one or for which significant disagreement had existed between raters. EPAs for which a clear consensus existed on relevance or level of entrustment (details in next section) were not discussed or submitted to the second round of rating. The 11 panellists did not re-rate when full entrustment was expected (1st–3rd PGY vs 4th–5th PGY) because round one’s results were clear and coherent. The EPAs presented at this face-to-face meeting were first discussed in depth and then rated individually and privately.
Following the RAM guidelines [26], each rating was categorised based on the median of the experts’ ratings (1–3 = inappropriate; 4–6 = uncertain; 7–9 = appropriate) and the level of intra-panel disagreement. Disagreement on categorisation was considered to exist when at least one third of the ratings ranged from 1 to 3 and another third ranged from 7 to 9. In cases of disagreement, ratings were categorised as uncertain, irrespective of the median.
We calculated medians and interquartile ranges for the overall results but also for subgroups of experts (hospital vs ambulatory practice and German- vs French- or Italian-speaking regions). To identify differences between these subgroups, we used Fisher’s exact test to compare rating category proportions and the Mann–Whitney test to compare rating distributions. Statistical analyses were performed with STATA17® (StataCorp. 2021. Stata Statistical Software: Release 17. College Station, TX, USA: StataCorp LLC.).
Considering the article by Taylor et al.[29], we added a final step to our framework, aimed at evaluating whether the quality and structure of the selected EPAs aligned with the existing EPA standards. Three authors experienced with the concept of EPAs (MM/DG/SF) independently assessed each selected EPA using five EQual criteria (1. This EPA has a clearly defined beginning and end; 2. This EPA is independently executable to achieve a defined clinical outcome; 3. This EPA is specific and focused; 4. This EPA is observable in process; 5. This EPA is measurable in outcome) [23]. EPAs that failed to meet one or more of these criteria were excluded. Disagreements were resolved through discussion.
This study was performed following the ethical standards stated in the Declaration of Helsinki. All participants provided their written informed consent. Under institutional and national regulations, our study was deemed exempt from formal ethical approval.
The review identified 16 entrustable professional activity lists established for general internal medicine postgraduate training programmes, 15 of which were established in English-speaking countries (the US, Canada, Australia and the UK). These lists yielded 308 candidate EPAs, which we further classified into six domains (care and management of the general adult population; care and management of specific populations or needs; care coordination and collaboration; management and leadership; healthcare quality, education and research activities; miscellaneous) [24]. Since several lists of EPAs included in the systematic review were published before the definition of specific guidelines [9], EPAs were, when needed, reformulated to better match recently recommended styles and wording [9] After the reformulation or elimination of redundant EPAs, 204 final candidate EPAs were presented for rating.
The deductive focus group analysis retrieved 256 themes, 60 of which were selected because they were formulated as professional activities; 17 were subsequently removed because they were already on the list obtained from the systematic review. Forty-three candidate EPAs were finally grouped into the six categories proposed by Valding et al. [24] and added for rating.
These steps generated a final list of 247 candidate EPAs in general internal medicine (figure 1) that were included in the first round of rating (supplementary file 2, available for download as a separate file at https://doi.org/10.57187/s.3592).
The complete dataset of the aggregate and subgroup analyses of the RAM is accessible on the open-source repository Zenodo (https://doi.org/10.5281/zenodo.10463141).
After the first round of rating, 159 of 247 EPAs (64.3%) were considered appropriate (median score ≥7/9) for postgraduate training in general internal medicine in Switzerland, for both the hospital and ambulatory tracks. Thirty-one EPAs (12.5%) were agreed to be inappropriate (median score <4), with 22 deemed not relevant for the hospital GIM track and nine deemed not relevant for the ambulatory GIM track. This left 49 EPAs (19.8%) of uncertain relevance (median score 4–6). Intra-panel disagreements were very rare and involved the relevance of four EPAs (1.6%) and the level of entrustment of two EPAs. The second round of rating thus included 83 (34%) EPAs.
After two rounds of rating, the expert panel agreed that 225 candidate EPAs (91%) presented were appropriate for the postgraduate training of general internal medicine in Switzerland and 22 were inappropriate. Disagreement persisted for only two EPAs. Most of the 22 EPAs considered inappropriate concerned the care of patients with specific needs, such as pregnant women with obstetric, perinatal or delivery problems or the care of newborns and children. Supplementary file 3 (available for download as a separate file at https://doi.org/10.57187/s.3592) shows details of the RAM process results. To make the table easier to understand, we organised the 225 candidate EPAs into seven domains and 26 contexts, adapted from those proposed by Valding et al. [24].
Of the 225 EPAs selected, 180 (80%) were considered relevant both for the ambulatory and hospital postgraduate training tracks, 27 (12%) were considered relevant only for the ambulatory track and 18 (8%) were considered relevant only for the hospital track (supplementary file 3). Regarding levels of entrustment, the expert panel agreed that by the end of postgraduate training, full entrustment (level 5) should be expected for most (83.6%) of the EPAs selected. The full entrustment of 21 (9.3%) should be achieved earlier, during the first part of postgraduate training (supplementary file 4, available for download as a separate file at https://doi.org/10.57187/s.3592). By the end of postgraduate training, a maximum entrustment of level 4 (able to be performed with supervision at a distance) was expected for 37 EPAs (16.4%) (supplementary file 3).
Analysis of the ratings of different physician subgroups (linguistic regions and work settings) showed very high rates of agreement. For example, only nine EPAs (4%) were assigned to different categories of appropriateness by physicians working in hospital or ambulatory settings (supplementary file 5, available for download as a separate file at https://doi.org/10.57187/s.3592).
After independently analysing the 225 EPAs using the EQual criteria [23], the authors agreed to exclude a further 69 EPAs that did not fulfil these standard quality criteria. Most of the excluded items involved either competencies (e.g. “Respecting patients’ autonomy in decision-making”) or activities not having a clear beginning or end (e.g. “Sharing uncertainties with colleagues” or “Developing a personal learning plan for future practice”). The following list shows the final list of 156 quality-checked candidate EPAs (121 common; 16 hospital medicine only; 19 family medicine only).
EQual criteria considered:
(h) = specific to hospital physician track; (p) = specific to primary care physician track
1. Diagnosing internal medicine conditions during pregnancy
2. Investigating internal medicine conditions during pregnancy
3. Managing internal medicine conditions during pregnancy
4. Diagnosing internal medicine conditions after pregnancy
5. Investigating internal medicine conditions after pregnancy
6. Recognising emergent obstetrical conditions and referring to specialists as appropriate
7. Providing pre-conception care (p)
The work of determining a list of entrustable professional activities in general internal medicine by using an explicit process of finding consensus is not new [24]. Nevertheless, to date, most published lists of EPAs have been created from scratch by local expert groups, with only limited descriptions of the methodologies used [14]. Additionally, most of those lists were developed before the 2015 publication of a reference standard for describing EPAs [8, 9, 30] and the 2021 recommendations for developing valid lists of EPAs to describe professional practice [29]. Young et al. highlighted the importance of adopting methods for developing EPAs that would ensure descriptions of activities that were valid for the profession [31]. Ensuring construct validity during the development of EPAs is essential to ensuring the quality of the assessments that trainees will undergo based on those EPAs [29, 32]. Thus, a number of the existing lists of EPAs do not meet those quality criteria [24] and are of limited utility for practice groups preparing or revising their lists of EPAs.
To the best of our knowledge, the present work is among the first to give a detailed description of the process of developing a list of EPAs and providing evidence of validity at each step, as proposed by Taylor et al. [29]. To achieve this, our innovative contribution has been to bring a range of experts into the project, both from the medical field and from the fields of medical education, consensus approach, and qualitative and quantitative analysis.
The participants in the focus groups and consensus process were selected to ensure their representativeness across all aspects of the profession and limit selection bias [29]. The identification of a preliminary list of candidate EPAs was achieved by integrating two different approaches, including the results of a systematic review of international literature. A consensus on the candidate EPAs was achieved using the RAM method, which combines the nominal group technique and a modified Delphi methodology [29, 33]. The nominal group technique’s appropriateness for identifying candidate EPAs [34] and the high inter-rater reliability demonstrated by the modified Delphi process consensus strengthen the validity of our results.
Most of the EPAs selected by the expert panel (80%) were found to be appropriate for both the hospital and ambulatory postgraduate training tracks. Subgroup analyses confirmed good agreement between clinical supervisors active in hospital and ambulatory settings and between Switzerland’s three major linguistic regions. Only 4% of the EPAs were given very divergent ratings. This was not surprising since Switzerland’s postgraduate training programme in general internal medicine requires every internist to do some training in both inpatient and outpatient structures, enabling all to develop a good understanding of the professional activities specific to both settings.
The small number of EPAs judged inappropriate may be surprising, however. Given that most of the EPAs originated from a systematic review – which mainly included lists of EPAs developed in English-speaking countries (the US, Canada, the UK and Australia) – we might have expected more to be deemed unsuitable for Switzerland. This result may suggest common roots and practices in general internal medicine, at least among these countries, and the potential to generalise our findings beyond a particular national context. On the other hand, this result may also reflect the broad range of professional activities within general internal medicine, a medical speciality whose embodiment of holistic patient care is part of its philosophy and specificity. Other, more organ-focused specialities, such as cardiology, have opted to describe EPAs based on the diseases that they treat [35], but this is unsuitable for general internal medicine.
Our expert panel considered that only a minority of the professional activities in general internal medicine (10%) should be fully entrustable to junior physicians during the first part of their training. This likely reflects the challenges in deciding precisely when a trainee is ready to care for patients without supervision. Entrustment decisions depend on not only the trainee’s experience but also their supervisor’s personality and the nature and complexity of the activity under observation [36]. Moreover, determining the right time to give that trust implies a trade-off between two potential risks: entrusting a junior physician too early may negatively affect patient safety. Conversely, a supervisor’s excessive caution may delay the development of autonomy and negatively impact the learning curve of junior physicians [37]. Our expert panel’s low propensity to grant full entrustment during the first part of postgraduate training may originate from their consideration of these factors. At an early stage of implementation of a new training and assessment strategy, this caution seems appropriate. Avoiding strict rules on full entrustment will allow more real-world flexibility for junior physicians, their supervisors and their training institutions.
One limitation of our study is that some entrustable professional activities submitted to the rating process did not completely match the proper criteria defining an EPA. EPAs are a recent concept, and how physicians and medical educators understand the underlying construct is still evolving. Several of the lists of EPAs included in our systematic review were developed before the quality criteria for describing EPAs were published [23]. This is why our final selection used the EQual criteria [23]. We believe that this final step improved the validity, usefulness and generalisability of our results.
Since the quality criteria to assess EPAs are now better understood and recognised, it might be more efficient to apply them to the process of EPA development earlier than we did (e.g. before submitting candidate EPAs to a panel for consensus).
Although our list of EPAs in general internal medicine is well-aligned with other lists published in the Global North [24], our results may not be generalisable to countries with different types of healthcare systems and needs or in which GIM does not include ambulatory practice. Therefore, we have highlighted those EPAs that are solely relevant to ambulatory practice.
We have already described our study’s strengths. Its explicit design enabled us to demonstrate the validity of each step in our process and aligned with Taylor et al.’s five-step framework for delivering valid descriptions of professional practice [29]. Moreover, integrating a systematic review with a RAM methodology to establish a national consensus proved particularly appropriate in a field where the scientific literature remains limited [26].
This research project’s results provide a valid and consensual new list of revised candidate entrustable professional activities [29] for postgraduate training programmes in general internal medicine. Some of these EPAs, such as “Providing care for patients with end-stage diseases”, “Conducting an interdisciplinary ward round”, “Managing transitions of care” and “Performing medical handovers”, could also be relevant to other medical specialities or healthcare professions.
Nevertheless, because this list was developed using the results of a systematic literature review and the analysis of focus groups, it will not be directly implementable as such into a training programme. This is because of the large number of candidate EPAs obtained and the variability in formulations, with some describing concrete activities and others seeming vaguer. Guidelines suggest limiting lists for a single programme to a maximum of 20–30 EPAs [9]. Therefore, programme committees will be forced to find a balance between a manageable number and the breadth of EPAs [9]. This could be a challenge for holistic disciplines such as general internal medicine, but it will be possible to achieve this balance by aggregating similar concrete EPAs around a broader title or specifying broad EPAs using smaller (“nested”) activities [9]. Building on this list, a group of internists and medical education specialists, mandated by the Postgraduate Education Committee of the Swiss Society of General Internal Medicine (SSGIM), is currently working on selecting, aggregating and elaborating a final list of EPAs that will meet current recommendations for fully described EPAs [9, 20]. Although they are the outcomes of the personal choices of the authors, their proposal to aggregate EPAs under 26 broader headings (table 2) is just one example of how it will be possible to end up with fewer EPAs by combining those relating to similar or interconnected activities.
We believe that the clinical breadth of our candidate EPAs, together with the detailed description of our methodology, could serve as an interesting and useful starting point from which medical education specialists or clinicians could develop or revise applicable lists of EPAs, particularly for postgraduate training programmes in either general internal medicine or family medicine.
The entire project on defining candidate entrustable professional activities in general internal medicine was supported financially by the Foundation of the Swiss Society of General Internal Medicine (the SSGIM Foundation)”.
All authors have completed and submitted the International Committee of Medical Journal Editors form for disclosure of potential conflicts of interest. No potential conflict of interest related to the content of this manuscript was disclosed.
1. Frank JR, Snell LS, Cate OT, Holmboe ES, Carraccio C, Swing SR, et al. Competency-based medical education: theory to practice. Med Teach. 2010;32(8):638–45.
2. Sabatino, F., Training of health care professionals must change, says Pew Commission director. Hospitals, 1991. 65(4): p. 50, 52.
3. Frank JR, Mungroo R, Ahmad Y, Wang M, De Rossi S, Horsley T. Toward a definition of competency-based education in medicine: a systematic review of published definitions. Med Teach. 2010;32(8):631–7.
4. Taylor DR, Park YS, Smith CA, Karpinski J, Coke W, Tekian A. Creating Entrustable Professional Activities to Assess Internal Medicine Residents in Training: A Mixed-Methods Approach. Ann Intern Med. 2018 May;168(10):724–9.
5. Witteles RM, Verghese A. Accreditation Council for Graduate Medical Education (ACGME) Milestones-Time for a Revolt? JAMA Intern Med. 2016 Nov;176(11):1599–600.
6. Ten Cate O, Scheele F. Competency-based postgraduate training: can we bridge the gap between theory and clinical practice? Acad Med. 2007 Jun;82(6):542–7.
7. Holmboe ES, Sherbino J, Long DM, Swing SR, Frank JR. The role of assessment in competency-based medical education. Med Teach. 2010;32(8):676–82.
8. Ten Cate O. Nuts and bolts of entrustable professional activities. J Grad Med Educ. 2013 Mar;5(1):157–8.
9. Ten Cate O, Taylor DR. The recommended description of an entrustable professional activity: AMEE Guide No. 140. Med Teach. 2021 Oct;43(10):1106–14.
10. Ten Cate O, Young JQ. The patient handover as an entrustable professional activity: adding meaning in teaching and practice. BMJ Qual Saf. 2012 Dec;21 Suppl 1:i9–12.
11. Carraccio C, Englander R, Holmboe ES, Kogan JR. Driving Care Quality: Aligning Trainee Assessment and Supervision Through Practical Application of Entrustable Professional Activities, Competencies, and Milestones. Acad Med. 2016 Feb;91(2):199–203.
12. Schumacher DJ, Turner DA. Entrustable Professional Activities: Reflecting on Where We Are to Define a Path for the Next Decade. Acad Med. 2021 Jul;96 7S:S1–5.
13. Ten Cate O. Entrustability of professional activities and competency-based training. Med Educ. 2005 Dec;39(12):1176–7.
14. Shorey S, Lau TC, Lau ST, Ang E. Entrustable professional activities in health care education: a scoping review. Med Educ. 2019 Aug;53(8):766–77.
15. Ten Cate O, Young JQ. The patient handover as an entrustable professional activity: adding meaning in teaching and practice. BMJ Qual Saf. 2012 Dec;21 Suppl 1:i9–12.
16. Carraccio C, Englander R, Gilhooly J, Mink R, Hofkosh D, Barone MA, et al. Building a Framework of Entrustable Professional Activities, Supported by Competencies and Milestones, to Bridge the Educational Continuum. Acad Med. 2017 Mar;92(3):324–30. 10.1097/ACM.0000000000001141
17. de Graaf J, Bolk M, Dijkstra A, van der Horst M, Hoff RG, Ten Cate O. The Implementation of Entrustable Professional Activities in Postgraduate Medical Education in the Netherlands: Rationale, Process, and Current Status. Acad Med. 2021 Jul;96 7S:S29–35. 10.1097/ACM.0000000000004110
18. Wagner LM, Dolansky MA, Englander R. Entrustable professional activities for quality and patient safety. Nurs Outlook. 2018;66(3):237–43.
19. Hauer KE, Boscardin C, Fulton TB, Lucey C, Oza S, Teherani A. Using a Curricular Vision to Define Entrustable Professional Activities for Medical Student Assessment. J Gen Intern Med. 2015 Sep;30(9):1344–8.
20. Hennus MP, Jarrett JB, Taylor DR, Ten Cate O. Twelve tips to develop entrustable professional activities. Med Teach. 2023 Jul;45(7):701–7.
21. Training IS. Programme de formation postgraduée du 1er janvier 2022- Specialiste en Médecine Interne Générale. 2022 [cited 2022 23 december]; Available from: https://www.siwf.ch/files/pdf7/aim_version_internet_f.pdf
22. Humphrey-Murto S, Varpio L, Gonsalves C, Wood TJ. Using consensus group methods such as Delphi and Nominal Group in medical education research. Med Teach. 2017 Jan;39(1):14–9.
23. Taylor DR, et al. EQual, a Novel Rubric to Evaluate Entrustable Professional Activities for Quality and Structure. Acad Med, 2017. 92(11S Association of American Medical Colleges Learn Serve Lead: Proceedings of the 56th Annual Research in Medical Education Sessions): p. S110-S117 DOI:
24. Valding B, Monti M, Junod Perron N, Frick S, Jaques C, Nendaz M, et al. Entrustable professional activities for residency in general internal medicine: a systematic review. Swiss Med Wkly. 2022 Nov;152(4344):40002.
25. P., D., Conversation Analysis. 2006: Sage Publishing Ltd.
26. Fitch K, et al. The RAND/UCLA appropriateness method user’s manual. Rand Corp Santa Monica CA; 2001.
27. Black N, Murphy M, Lamping D, McKee M, Sanderson C, Askham J, et al. Consensus development methods: a review of best practice in creating clinical guidelines. J Health Serv Res Policy. 1999 Oct;4(4):236–48.
28. McMillan SS, King M, Tully MP. How to use the nominal group and Delphi techniques. Int J Clin Pharm. 2016 Jun;38(3):655–62.
29. Taylor D, Park YS, Smith C, Cate OT, Tekian A. Constructing Approaches to Entrustable Professional Activity Development that Deliver Valid Descriptions of Professional Practice. Teach Learn Med. 2021;33(1):89–97.
30. Ten Cate O, Chen HC, Hoff RG, Peters H, Bok H, van der Schaaf M. Curriculum development for the workplace using Entrustable Professional Activities (EPAs): AMEE Guide No. 99. Med Teach. 2015;37(11):983–1002.
31. Young JQ, Hasser C, Hung EK, Kusz M, O’Sullivan PS, Stewart C, et al. Developing End-of-Training Entrustable Professional Activities for Psychiatry: Results and Methodological Lessons. Acad Med. 2018 Jul;93(7):1048–54. 10.1097/ACM.0000000000002058
32. Marceau M, Gallagher F, Young M, St-Onge C. Validity as a social imperative for assessment in health professions education: a concept analysis. Med Educ. 2018 Jun;52(6):641–53.
33. Waggoner J, Carline JD, Durning SJ. Is There a Consensus on Consensus Methodology? Descriptions and Recommendations for Future Consensus Research. Acad Med. 2016 May;91(5):663–8. 10.1097/ACM.0000000000001092
34. Touchie C, De Champlain A, Pugh D, Downing S, Bordage G. Supervising incoming first-year residents: faculty expectations versus residents’ experiences. Med Educ. 2014 Sep;48(9):921–9.
35. Tanner FC, Brooks N, Fox KF, Gonçalves L, Kearney P, Michalis L, et al.; ESC Scientific Document Group. ESC Core Curriculum for the Cardiologist. Eur Heart J. 2020 Oct;41(38):3605–92.
36. Hauer KE, Ten Cate O, Boscardin C, Irby DM, Iobst W, O’Sullivan PS. Understanding trust as an essential element of trainee supervision and learning in the workplace. Adv Health Sci Educ Theory Pract. 2014 Aug;19(3):435–56.
37. Sterkenburg A, Barach P, Kalkman C, Gielen M, ten Cate O. When do supervising physicians decide to entrust residents with unsupervised tasks? Acad Med. 2010 Sep;85(9):1408–17.
The supplementary files 1-5 are available for download as a separate file at https://doi.org/10.57187/s.3592.