Entrustable professional activities for residency in general internal medicine: a systematic review

DOI: https://doi.org/ 10.57187/smw.2022.40032

Introduction

Physicians supervising medical students and residents have shown a growing interest in the concept of entrustable professional activities (EPAs) since Ten Cate introduced this term in 2005 [1–3]. EPAs are concrete tasks performed in day-to-day clinical work [2, 4]. These activities are described as "entrustable" because medical trainees must gain their supervisors’ trust before being permitted to accomplish them independently [2].

EPAs were developed as a necessary complement to competency-based medical education [5, 6]. Whereas competencies describe physicians’ characteristics, EPAs represent concrete clinical tasks that require proficiency in several competencies [7]. As a complement to competencies, EPAs facilitate assessment processes because they are easily observable [8, 9]. Therefore, EPAs help clarify the expectations for both trainees and supervisors [5]. Finally, EPAs highlight the critical issue of entrustment and thereby the appropriate level of supervision needed in clinical environments [10–12].

Defining the EPAs that trainee doctors should have mastered by the end of their residency is essential to setting clear expectations about their future autonomous practice. Numerous residency programmes in various disciplines, including general internal medicine, have begun defining which EPAs need to be mastered [13–19]. Defining a particular residency programme’s EPAs is a multistep process, usually requiring an initial selection of EPAs by a panel of experts and then the formation of a consensus among them [20]. To facilitate this process for general internists, we aimed to collect all the available lists of EPAs defined for residency programmes in general internal medicineand synthesise them into a useful reference.

Methods

This systematic review had the following research question: “Which entrustable professional activities have been defined for residency programmes in general internal medicine?” The study protocol was registered and published online on the PROSPERO website(CRD42021169755). There was no deviation from this protocol. We followed the PRISMA guideline recommendations for reporting systematic reviews [21].

Data sources and searches

The present systematic review searched the following databases: Medline Ovid SP, Embase.com, Education Resource Information Center (ERIC), Web of Science, and ProQuest Dissertations andTheses A&I. We based our search on the following keywords: (1) "entrustable professional activities" and (2) "postgraduate education" or "training" or "residency" (appendix 1). The search algorithms, developed with the help of a medical librarian, are shown in appendix 2.

We also performed complementary searches introducing the same keywords on Google Scholar. For practicality and relevance, we limited the search to the first 200 results yielded by Google Scholar. We hand-searched literature in two leading medical education journals (Medical EducationandMedical Teacher)the abstract books of several medical education conferences and medical societies’ recommendations. The details of our grey literature search are also available in appendix 2.

The search extended from January 2005, when Ten Cate published his first article on EPAs [2], to February 2020, when the initial search was performed. Due to the COVID-19 pandemic, the review has been delayed, requiring a complementary search we performed in April 2022, covering the period between the initial search and April 2022. We gathered all the references using EndNote, version 20.1 (Clarivate Analytics).

Study selection

Inclusion criteria

Although our focus was on residency programmes for general internal medicine, we knew this wasprobably not a uniform specialty across healthcare systems internationally. We therefore considered multiple existing definitions of general internal medicine[22–25] and chose to define it as the medical discipline responsible for the holistic care of patients in both inpatient and outpatient settings. We included every article meeting our definition of general internal medicine. This is why our inclusion criteria extended more broadly to residency programmes in internal medicineand family medicine (FM). Only articles in English, Frenchor German were included. The last inclusion criterion was the availability of an EPA list.

Exclusion criteria

We excluded articles concerning healthcare professionals other than medical doctors, focusing on undergraduate medical education or addressing only a limited clinical activity (e.g., patient handovers[26]) rather than a full residency programme. In addition, we excluded articles using terms other than EPAs for their proposed lists (e.g., lists of “competencies in practice” or “observable professional activities”). If the authors of such articles had not explicitly mentioned that their term was a synonym for EPAs, we considered that their list reflected a different concept and excluded the source. Finally, publications that used previously published lists of EPAs without further work were not included in the study.

Two authors (BV and DG) performed every phase of the study selection process independently. After the removal of duplicates, the titles and abstracts of the retrieved articles were screened using Rayyan QCRI website [27]. Full-text articles were obtained when needed to help decide on final inclusion. BV and DG resolved disagreements through discussion. Finally, a forward and backward citation search was performed on the included articles to identify any possible additional records.

Data extraction and quality assessment

Two authors (BV and DG) independently extracted the following data from the articles: year of publication, country, language, type of residency programme (e.g., the medical discipline as designated by the publication’s authors), practice setting for the EPAs (e.g., EPAs developed for outpatient practice only, inpatient practice only, or both), number of EPAs on the list, methods used to develop that list and methods authors used to assess the quality of each EPA on the list (e.g., the EQual protocol). BV and DG resolved any disagreements concerning data extraction through discussion. A final, common version of the extraction form was established.

BV and DG independently assessed the methodological quality of the process leading to each article’s EPA list but not the EPAs themselves. This assessment followed the guidelines of the Quality Assessment Tool for Studies with Diverse Designs (QATSDD), a 16-item tool used in healthcare services research and dedicated to the quality assessment of studies with diverse designs [28]. QATSDD was previously used by Kerth, O’Dowd and Pinilla in their respective systematic reviews concerning EPAs [16, 20, 29]. We used the specifically developed AACODS tool to critically appraise the quality of grey literature [30]. The AACODS acronym represents each section of the tool—Authority, Accuracy, Coverage, Objectivity, Date and Significance—with each section including from 1 to 12 closed questions. To synthesise our quality assessment of the reports in the grey literature, we converted the AACODS results into a score (with 0 to 1 point scored for each section).

The mean of the two raters’ scores was then computed for each item of both tools. Finally, a total score was computed for each article. The inter-rater reliability was calculated using an intraclass correlation coefficient (ICC (2,k))[31].

Data synthesis and analysis

We could not perform a meta-analysis because of the types of studies examined; rather, we planned a qualitative approach to the data synthesis, with a thematic categorisation of the available EPAs. Categories were not defined in advance, and we favoured an inductive approachto creating themes.

Our broader inclusion criteria extended to internal medicineand familymedicine residency programmes, for the previously mentioned reasons. We planned to conduct a secondary analysis of our data to consider each residency programme’s exact title or designation (e.g., family or internal medicine). To do so, we would simply refer to the designation used by the EPA lists’ authors.

Results

Description of the articles

Our research algorithms found 1664 records in selected databases and an additional 249 in our grey literature search, making a total of 1913. This number was reduced to 1132 after the removal of duplicates, but only 14 articles fitted our criteria [13, 32–44] (fig. 1). Another reference [45] was found after a forward and backward citation search from the articles selected. The complementary search performed in April 2022 found 1379 records, that were reduced to 547 after duplicate removal. Out of those, only one new article fitted our criteria [46].

The list of selected articles is available in table 1. The articles were numbered from 1–16, allowing us to number each EPA easily using a four-digit code: in EPA 0534, for example, the first two digits represent article number 05, and the last two digits represent the EPA’s position in its particular list.

All 16 articles were available in English, with most from North America (USA [n = 6], Canada [n = 6]), two from Australia, one from the UK and one from Iran. The oldest articles were publishedin 2012, and most (n = 9) were published between 2017 and early 2021. Half of the articles (n = 8) described residency programmes identified as family medicineby their authors; the other half (n = 8) were identified as internal medicineprogrammes.

Thirteen of the 16 articles described the methods used to define their EPAs [13, 32–41, 43, 46]. They referred to literature reviews or previous work completed on the matter. One article used a scoping review, but none reported the completion of a systematic literature review. All 13 articles described involvement of an expert panelin the definition of the EPAs.

The methodological quality assessment of the articles included showed that the literature had a medium level of quality (QATSDD mean score 30.6, standard deviation [SD] 5.3; maximum possible tool score 48) (AACODS mean score 3.9, SD 0.2; maximum possible tool score 6). The scores for each item are available in a supplementary table (appendix 3).

The highest scoring items for QATSDD were the description of the objectives of the studies, the description of the research settings and the fit between the research question and the method used (items 2, 3 and 10). The two items that scored highest for AACODS were the assessment of bias and the reference to current content (items O and D). The lowest scoring items in the QATSDD test were the evidence for the sample size (e.g., size of the expert panel), the assessment of reliability and the evidence of user’s involvement in the design (item 4, 14 and 15). The lowest scoring items in AACODS were the accuracy (e.g., not peer-reviewed, no stated methodology) and the coverage(e.g., no clear question of research, no clear limit) (items A2 and C). The reliability was assessed between the reviewers using an ICC (ICC (2,k)) [31], and is considered good for the QATSDD score (0.89) and moderate for the AACODS score (0.69).

Lists of EPAs

The analysis of the 16 lists yielded a totalof 395 different EPAs. The mean number of EPAs per list was 25 (median25 EPAs). The number of EPAs in the lists created for FM residency programmes (median 25 EPAs) is slightly higher than the number of EPA in the lists created for internalmedicine residency programmes (median 21 EPAs).

EPA classification

A total of 395 EPAs were retrieved from the 16 different lists.Using an iterative approach, we were able to organise 308 of them into a three-level categorisation of 6 domains, 14 themes and 24 subthemes. Our categorisation was not predefined; rather, it developed inductively from the EPAs themselves, thus providing a more meaningful synthesis. Thesix domains were: (1) care and management of the general adult population (n = 103 EPAs); (2) care and management of patients with specific needs (n = 67 EPAs); (3) care coordination and communication (n = 52 EPAs); (4) management and leadership (n = 21 EPAs); (5) healthcare quality, education, and research (n = 41 EPAs); and (6)miscellaneous (n = 24 EPAs). The themes and subthemes are detailed in table 2. The full list of EPAs is available as supplementary information.

Eighty-seven EPAs, originating from two articles [38, 43], were not included in our three-level categorisation because of their disease-based nature (e.g., managing patients with headache,joint pain or chronic asthma). These EPAs had been written in a style too different from the other lists to be included in the final categorisation.

Finally, we examined the representation of the 14 themes across IM and FM residency programmes separately. Results are available in the appendix 4.

Discussion

To the best of our knowledge, this systematic review is the first to include the EPAs defined for residency programmes in general internal medicine. We identified 16 articles meeting our inclusion criteria, published from2012 onwards; their 16 different lists provided a total of 395 EPAs. Although this seems a large number, many expressed similar concepts with different wording or specific details. We found it useful to categorise these heterogeneous descriptions using a pragmatic, inductive approach. Categorisation resulted in five main domains: (1) care and management of the general adult population; (2) care and management of patients with specific needs; (3) care coordination and communication; (4) management and leadership; and (5) healthcare quality, education, and research. There was also a sixth, broader domain called miscellaneous. The EPAs were further divided into 14 themes and 24 subthemes.

Two lists of EPAs [38, 43] were based on specific diseases, contrasting with the 14 other lists that referred to daily clinical tasks rather than the cause of a patient’s consultation. A disease-based approach for EPAs does exist, but it seems more suitable for specialties with a focused scope of practice (e.g., cardiology [14]) rather than more holistic disciplines (e.g., GIM). In addition, using disease-based approaches seems to increase the total number of EPAs required to properly encompass professional practice, as demonstrated in Shaughnessy et al. [38], where the number of EPAs (n = 76) was well beyond the average number found in the lists in the literature [4]. An argument can be made about the generic nature of some EPAs gathered in this review (e.g.,manage care of patients with acute common diseases across multiple care (EPA 0101)). However, each EPA should come with a full description or specification, adding precious details about what is finally covered by a given EPA [7]. In addition, a list of EPAs does not stand alone in a training programme. They usually supplement a number of competences in the paradigm of competency-based medical education. EPAs can also supplement a list of relevant clinical contexts, as it is done in the framework for undergraduate medical training in Switzerland with “situation as starting points” [47].

It is worth noting that all the articles included came from only five countries, with 75% from North America. Those countries appear leaders in publishing EPAs for postgraduate training in general internal medicine; however, our research algorithm was limited to three languages and likely missed lists that were not translated.

The articles’ overall level of methodological quality could be considered moderate. The QATSDD quality scores identified a particular lack of rigour in the constitution of expert panels and in the justifications for sample sizes. Low quality scores were also obtained for user involvement, namely the residents themselves, who were rarely consulted during the process of defining the EPAs. The quality scores for assessing the grey literature using ACCODS were undermined by either a lack of accuracy in the method itself or in the way in which the method was described.

For assessing the quality of EPAs themselves, only one team of authors [48] had used a validated tool – the EQual protocol assessment score [49]; others had relied on their experts’ opinions. This could be explained simply by the fact that quality assessment tools for EPAs were only developed recently and thus published after half of the articlesin this systematic review. This underscores the need to include this last,critical, quality verification step in any approach to developing EPAs. We were unable to assess each of the 395EPAs found in our literature review because we would have had to have been fully informed about each list’s professional context.Indeed, one EQual protocol item pertains to how important the task is to the profession [49]. Finally, for authors undertaking future efforts to define EPAs for a given educational programme, it is important to know that a validated assessment tool exists [49] and that recommendations on formulating EPAs have been published [4].

This review was limited by the relatively small amount of literature available on residency programmes using EPAs in general internal medicine, even though it is one of the most studied specialties [20]. As already mentioned,our three-language research algorithm may have missed lists that have not been translated into any of them. A final limitation was our commitment to focus solely on EPAs, which therefore excluded work that has been undertaken on very similar concepts, such as “capabilities in practices” (CiPs). Our rationale was to avoid concepts that were deliberately named anything other than entrustable professional activities because they were meant to entail some type of nuance from the original concept (i.e., EPAs).

Regarding this review’s strengths, it is, to the best of our knowledge, the first systematic review on residency programmes using EPAs in general internal medicine. It is also the first review of EPAs in postgraduate medical education that provides a thematic categorisation of EPAs drafted for a particular specialty. It will give readers a useful tool with which to begin the development of their own lists of EPAs for residency programmes in general internal medicine. Future authors will be able to use this review and adapt its large number of EPAs to the context of their own clinical practice.

Conclusions

Entrustable professional activities (EPAs)have become an important complement to competency-based medical education. Since general internal medicineprogrammes, like those of most other medical disciplines, have embarked on the complex, multi-step process of defining listsof EPAs for postgraduate training, it is important to be able to make sense of all the efforts already invested in this process internationally. Our systematic review not only collected 16 lists of EPAs developed for general internal medicineresidency programmes but also produced a useful reference document with a unique, thematic categorisation of those EPAs.

Acknowledgement

This study was part of a larger research project on EPAs in general internal medicinethat has been granted funding by the Swiss Society of General Internal Medicine Foundation (SSGIM). The SSGIM was not involved in either the design, construction, or reporting for this study.

We would like to thank Pedro Marques Vidal, Marie-Claude Audétat, and Sarah Cairo for their time, advice and experience with systematic reviews.

References

1. Schumacher DJ, Turner DA. Entrustable Professional Activities: Reflecting on Where We Are to Define a Path for the Next Decade. Acad Med. 2021 Jul;96(7S 7s):S1–5. https://doi.org/10.1097/acm.0000000000004097 https://doi.org/10.1097/ACM.0000000000004097 

2. ten Cate O. Entrustability of professional activities and competency-based training. Med Educ. 2005 Dec;39(12):1176–7. https://doi.org/10.1111/j.1365-2929.2005.02341.x 

3. Shorey S, Lau TC, Lau ST, Ang E. Entrustable professional activities in health care education: a scoping review. Med Educ. 2019 Aug;53(8):766–77. https://doi.org/10.1111/medu.13879 

4. Ten Cate O, Chen HC, Hoff RG, Peters H, Bok H, van der Schaaf M. Curriculum development for the workplace using Entrustable Professional Activities (EPAs): AMEE Guide No. 99. Med Teach. 2015;37(11):983–1002. https://doi.org/10.3109/0142159X.2015.1060308 10.3109/0142159X.2015.1060308</p> https://doi.org/10.3109/0142159X.2015.1060308 

5. ten Cate O, Scheele F. Competency-based postgraduate training: can we bridge the gap between theory and clinical practice? Acad Med. 2007 Jun;82(6):542–7. https://doi.org/10.1097/ACM.0b013e31805559c7 

6. Holmboe ES, Sherbino J, Long DM, Swing SR, Frank JR. The role of assessment in competency-based medical education. Med Teach. 2010;32(8):676–82. https://doi.org/10.3109/0142159x.2010.500704 https://doi.org/10.3109/0142159X.2010.500704 

7. Ten Cate O. Nuts and bolts of entrustable professional activities. J Grad Med Educ. 2013 Mar;5(1):157–8. https://doi.org/10.4300/JGME-D-12-00380.1 

8.    ten Cate O, Young JQ. The patient handover as an entrustable professional activity: adding meaning in teaching and practice. Bmj Quality & Safety. 2012;21:9-12. doi: https://doi.org/10.1136/bmjqs-2012-001213. PubMed PMID: WOS:000311419400003. 

9. Carraccio C, Englander R, Holmboe ES, Kogan JR. Driving Care Quality: Aligning Trainee Assessment and Supervision Through Practical Application of Entrustable Professional Activities, Competencies, and Milestones. Acad Med. 2016 Feb;91(2):199–203. https://doi.org/10.1097/ACM.0000000000000985 

10. Ten Cate O, Hart D, Ankel F, Busari J, Englander R, Glasgow N, et al.; International Competency-Based Medical Education Collaborators. Entrustment Decision Making in Clinical Training. Acad Med. 2016 Feb;91(2):191–8. https://doi.org/10.1097/ACM.0000000000001044 

11. Chen HC, van den Broek WE, ten Cate O. The case for use of entrustable professional activities in undergraduate medical education. Acad Med. 2015 Apr;90(4):431–6. https://doi.org/10.1097/ACM.0000000000000586 

12. Ten Cate O, Chen HC. The ingredients of a rich entrustment decision. Med Teach. 2020 Dec;42(12):1413–20. https://doi.org/10.1080/0142159x.2020.1817348 https://doi.org/10.1080/0142159X.2020.1817348 

13. Taylor DR, Park YS, Smith CA, Karpinski J, Coke W, Tekian A. Creating Entrustable Professional Activities to Assess Internal Medicine Residents in Training: A Mixed-Methods Approach. Ann Intern Med. 2018 May;168(10):724–9. https://doi.org/10.7326/M17-1680 

14. Tanner FC, Brooks N, Fox KF, Gonçalves L, Kearney P, Michalis L, et al.; ESC Scientific Document Group. ESC Core Curriculum for the Cardiologist. Eur Heart J. 2020 Oct;41(38):3605–92. https://doi.org/10.1093/eurheartj/ehaa641 

15. Leipzig RM, Sauvigné K, Granville LJ, Harper GM, Kirk LM, Levine SA, et al. What is a geriatrician? American Geriatrics Society and Association of Directors of Geriatric Academic Programs end-of-training entrustable professional activities for geriatric medicine. J Am Geriatr Soc. 2014 May;62(5):924–9. https://doi.org/10.1111/jgs.12825 

16. Kerth JL, van Treel L, Bosse HM. The Use of Entrustable Professional Activities in Pediatric Postgraduate Medical Education: A Systematic Review. Acad Pediatr. 2021; https://doi.org/10.1016/j.acap.2021.07.007 

17. Hart D, Franzen D, Beeson M, Bhat R, Kulkarni M, Thibodeau L, et al. Integration of Entrustable Professional Activities with the Milestones for Emergency Medicine Residents. West J Emerg Med. 2019 Jan;20(1):35–42. https://doi.org/10.5811/westjem.2018.11.38912 

18. de Graaf J, Bolk M, Dijkstra A, van der Horst M, Hoff RG, Ten Cate O. The Implementation of Entrustable Professional Activities in Postgraduate Medical Education in the Netherlands: Rationale, Process, and Current Status. Acad Med. 2021 Jul;96(7S 7s):S29–35. https://doi.org/10.1097/acm.0000000000004110 https://doi.org/10.1097/ACM.0000000000004110 

19. Moll-Khosrawi P, Ganzhorn A, Zollner C, Schulte-Uentrop L. Development and validation of a postgraduate anaesthesiology core curriculum based on Entrustable Professional Activities: a Delphi study. GMS J Med Educ. 2020;37(5):Doc52. Epub 2020/09/29. doi: https://doi.org/10.3205/zma001345. PubMed PMID: 32984511; PubMed Central PMCID: PMCPMC7499458. 

20. O’Dowd E, Lydon S, O’Connor P, Madden C, Byrne D. A systematic review of 7 years of research on entrustable professional activities in graduate medical education, 2011-2018. Med Educ. 2019 Mar;53(3):234–49. https://doi.org/10.1111/medu.13792 

21. Moher D, Liberati A, Tetzlaff J, Altman DG; PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. J Clin Epidemiol. 2009 Oct;62(10):1006–12. https://doi.org/10.1016/j.jclinepi.2009.06.005 

22. Friedman RH, Pozen JT, Rosencrans AL, Eisenberg JM, Gertman PM. General internal medicine units in academic medical centers: their emergence and functions. Ann Intern Med. 1982 Feb;96(2):233–8. https://doi.org/10.7326/0003-4819-96-2-233 

23. Kramer MH, Akalin E, Alvarez de Mon Soto M, Bitterman H, Ferreira F, Higgens C, et al.; Working Group on Professional Issues in Internal Medicine. Internal medicine in Europe: how to cope with the future? an official EFIM strategy document. Eur J Intern Med. 2010 Jun;21(3):173–5. https://doi.org/10.1016/j.ejim.2010.03.007 

24. Internal Medicine vs. Family Medicine: American College of Physicians; [cited 2020 07.09.2020]. Available from: https://www.acponline.org/about-acp/about-internal-medicine/career-paths/medical-student-career-path/internal-medicine-vs-family-medicine 

25. Ghali WA, Greenberg PB, Mejia R, Otaki J, Cornuz J. International perspectives on general internal medicine and the case for “globalization” of a discipline. J Gen Intern Med. 2006 Feb;21(2):197–200. https://doi.org/10.1111/j.1525-1497.2005.00289.x 

26. ten Cate O, Young JQ. The patient handover as an entrustable professional activity: adding meaning in teaching and practice. BMJ Qual Saf. 2012 Dec;21 Suppl 1:i9–12. https://doi.org/10.1136/bmjqs-2012-001213 

27. Ouzzani M, Hammady H, Fedorowicz Z, Elmagarmid A. Rayyan-a web and mobile app for systematic reviews. Syst Rev. 2016;5(1):210. Epub 2016/12/07. doi: https://doi.org/10.1186/s13643-016-0384-4. PubMed PMID: 27919275; PubMed Central PMCID: PMCPMC5139140. 

28. Sirriyeh R, Lawton R, Gardner P, Armitage G. Reviewing studies with diverse designs: the development and evaluation of a new tool. J Eval Clin Pract. 2012 Aug;18(4):746–52. https://doi.org/10.1111/j.1365-2753.2011.01662.x 

29. Pinilla S, Lenouvel E, Strik W, Kloppel S, Nissen C, Huwendiek S. Entrustable Professional Activities in Psychiatry: A Systematic Review. Acad Psychiatry. 2019; https://doi.org/10.1007/s40596-019- 01142-7 https://doi.org/10.1007/s40596-019-01142-7 

30. Tyndall J, University F. The AACODS checklist designed to enable evaluation and critical appraisal of grey literature 2010. Available from: https://dspace.flinders.edu.au/xmlui/bitstream/handle/2328/3326/AACODS_Checklist.pdf;jsessionid=A10 EBD54545A54357AA2DB8BA7694851?sequence=4. 

31. Koo TK, Li MY. A Guideline of Selecting and Reporting Intraclass Correlation Coefficients for Reliability Research. J Chiropr Med. 2017;16(4):346-. doi: https://doi.org/10.1016/j.jcm.2017.10.001. PubMed PMID: WOS:000426972000011. 

32. Caverzagie KJ, Cooney TG, Hemmer PA, Berkowitz L. The development of entrustable professional activities for internal medicine residency training: a report from the Education Redesign Committee of the Alliance for Academic Internal Medicine. Acad Med. 2015 Apr;90(4):479–84. https://doi.org/10.1097/ACM.0000000000000564 

33. Chang A, Bowen JL, Buranosky RA, Frankel RM, Ghosh N, Rosenblum MJ, et al. Transforming primary care training--patient-centered medical home entrustable professional activities for internal medicine residents. J Gen Intern Med. 2013;28(6):801-9. Epub 2012/09/22. doi: https://doi.org/10.1007/s11606-012- 2193-3. PubMed PMID: 22997002; PubMed Central PMCID: PMCPMC3663955. https://doi.org/10.1007/s11606-012-2193-3 

34. Hauer KE, Kohlwes J, Cornett P, Hollander H, Ten Cate O, Ranji SR, et al. Identifying entrustable professional activities in internal medicine training. J Grad Med Educ. 2013;5(1):54-9. Epub 2014/01/10. doi: https://doi.org/10.4300/JGME-D-12-00060.1. PubMed PMID: 24404227; PubMed Central PMCID: PMCPMC3613318. 

35. Quraishi S, Wade W, Black D. Development of a GMC aligned curriculum for internal medicine including a qualitative study of the acceptability of ‘capabilities in practice’ as a curriculum model. Future Healthc J. 2019 Oct;6(3):196–203. https://doi.org/10.7861/fhj.2018-0016 

36. Schultz K, Griffiths J, Lacasse M. The Application of Entrustable Professional Activities to Inform Competency Decisions in a Family Medicine Residency Program. Acad Med. 2015 Jul;90(7):888–97. https://doi.org/10.1097/ACM.0000000000000671 

37. Shah R, Melvin L. Cavalcanti RBJCJoGIM. EPAs for the Ambulatory Internist in Translation: Findings from a Canadian Multi-Center Survey. 2019;14(3):9–15. 

38. Shaughnessy AF, Sparks J, Cohen-Osher M, Goodell KH, Sawin GL, Gravel J, Jr. Entrustable professional activities in family medicine. J Grad Med Educ. 2013;5(1):112-8. Epub 2014/01/10. doi: https://doi.org/10.4300/JGME-D-12-00034.1. PubMed PMID: 24404237; PubMed Central PMCID: PMCPMC3613294. 

39. Soran C, Laponis R, Summerville S, Thompson V, Eastburn A, O’Sullivan P, et al. Identifying Entrustable Professional Activities for Internal Medicine Residents in Ambulatory Continuity Practice. J Gen Intern Med. 2019; https://doi.org/10.1007/s11606-019-05430-8 

40. Valentine N, Wignes J, Benson J, Clota S, Schuwirth LW. Entrustable professional activities for workplace assessment of general practice trainees. Med J Aust. 2019 May;210(8):354–9. https://doi.org/10.5694/mja2.50130 

41. Association of Family Medicine Residency Directors. Entrustable Professional Activities (EPAs) for family medicine : Overview. 2015 [Accessed 2020 May 18] Available from: https://www.afmrd.org/d/do/P1887 

42. University of Calgary. Department of Family Medicine Residency Program Entrustable Professional Activities (“EPA’s) 2017 June [Accessed 2020 January 10]. Available from: https://www.calgaryfamilymedicine.ca/residency/index.php/program-info/curriculum?id=354 

43. Saltis T, Starling C, Regan C. Using trust in assessment A trial of Entrustable Professional Activities (EPA’s) in GP Training. 2015. 

44. University of Calgary - Departement of Family Medicine - Competency Framework. University of Manitoba. Rady Faculty of Health Sciences; 2018 June [Accessed 2020 January 10]. Available from: http://umanitoba.ca/faculties/health_sciences/medicine/units/family_medicine/media/Competency_Framework_2018-19.pdf 

45. Royal College of Physicians and Surgeons of Canada. Entrustable Professional Activities for General Internal Medicine. 2019. 

46. Dehghani Poudeh M, Mohammadi A, Mojtahedzadeh R, Yamani N. Entrustability levels of general internal medicine residents. BMC Med Educ. 2021;21(1):185. PubMed PMID: 33766005; PubMed Central PMCID: PMCPMC7995576. https://doi.org/10.1186/s12909-021-02624-9 

47. Michaud PA, Jucker-Kupper P, The P; The Profiles Working Group. The “Profiles” document: a modern revision of the objectives of undergraduate medical studies in Switzerland. Swiss Med Wkly. 2016 Feb;146(0506):w14270. https://doi.org/10.4414/smw.2016.14270 

48. Soran C, Laponis R, Summerville S, Thompson V, Eastburn A, O'Sullivan P, et al. Identifying Entrustable Professional Activities for Internal Medicine Residents in Ambulatory Continuity Practice. J Gen Intern Med. 2020;35(6):1917-9. Epub 2019/10/23. doi: https://doi.org/10.1007/s11606-019-05430-8. PubMed PMID: 31637647; PubMed Central PMCID: PMCPMC7280367. 

49. Taylor DR, Park YS, Egan R, Chan MK, Karpinski J, Touchie C, et al. EQual, a Novel Rubric to Evaluate Entrustable Professional Activities for Quality and Structure. Academic Medicine. 2017;92(11S Association of American Medical Colleges Learn Serve Lead: Proceedings of the 56th Annual Research in Medical Education Sessions):S110-S7. PubMed PMID: 29065031. https://doi.org/10.1097/ACM.0000000000001908 

Appendices: Supplementary information

The appendices are available in the pdf version of the article.