Threats to patient safety in the primary care office: concerns of physicians and nurses

DOI: https://doi.org/10.4414/smw.2012.13601

Introduction

Errors in health care have gained increasing attention since the Institute of Medicine published its report “To Err Is Human” in 1999 [1]. In a recent multinational survey study 11.4% of the adult population in Switzerland reported an experience of medical or medication error in the past year [2]. Chart review studies of adverse events have been conducted in several countries [3–6]. Less research has been conducted in the outpatient care setting but the available studies suggest that patients are at considerable risk as well. The incidence of medical errors in primary care ranges from 5 to 80 per 100000 consultations [7]. This impressively large range reflects the heterogeneity of study designs, data assessment methods, reference units, and outcome measures (e.g., errors, adverse events, incidents). Preventable adverse drug events are frequent among patients in outpatient care [8, 9]. Gurwitz et al. conducted a cohort study of all Medicare enrollees treated in a group practice and used multiple methods to collect data on drug-related incidents. They report an overall rate of adverse drug events among older patients in the ambulatory setting of 50.1/1,000 person-years, of which 28% were considered preventable [10]. Other studies relied on staff self-report of incidents or errors: Staff members’ incident reports in the UK yielded an error report rate of 75/1000 patient contacts in outpatient care [11]. In a similar study conducted in the US, errors and preventable adverse events were reported after 24% of outpatient visits [12].

Research on patient safety in primary care has focused on the identification and assessment of adverse events in outpatient settings, either by chart review or by self-reports of medical staff. However, little is known about primary care professionals’ concerns about risks to patient safety in their offices. These professionals’ perceptions of threats to safety may differ from observed occurrences of adverse events, their preventability and potential harm. For example, physicians and nurses in primary care may be concerned about confusion of vaccines and engage in preventive efforts, even though they had never experienced this event. Professionals may also be less reluctant to report perceived threats because it does not require disclosing errors or incidents in their own office. Professionals’ subjective concerns and risk perceptions are a valuable source of information that may help to develop strategies for patient safety in primary care close to their needs. Health care professionals are more likely to engage in improvement activities if these match their perceptions of threats and safety problems. Similarly, contrasting risk perceptions with true incidences of adverse events or errors may support learning activities and raise awareness towards risks that are not psychologically salient. The main aim of this study was to identify threats to patient safety in the primary care office from the perspective of health care staff. We examined threats to patient safety, primary care professionals' were personally most concerned about. As safety of primary care is highly interprofessional we studied safety concerns of both physicians and nurses and investigated differences between both groups.

Methods

Design

As part of a larger cross-sectional study, primary care physicians and nurses working in outpatient offices were surveyed about patient safety [13]. We assessed safety climate in the offices as well as frequency of and harm associated with safety incidents during the preceding 12 months. For a list of 23 possible incidents, respondents were asked to indicate the frequency of their occurrence in the office and the severity of last occurrences of the incident. The incidents were organised along the care continuum in the chapters diagnostic process, medication, other therapeutic and preventive measures including interventional procedures, patient encounter and information, organisation and work flow in the office, cooperation with other providers, and storage of drugs and materials. Exact definitions of frequency and severity were provided. Background information on the respondent and their working environment was obtained as well. An iterative pre-test was conducted and the survey adapted subsequently. For this analysis, we focused on one particular task in the survey. Participants were asked to name 3 threats to patient safety in their office they personally would prioritise for “elimination” if this would be possible (called “threats” hereinafter). Respondents were invited to name those threats they were most concerned about, even if their elimination would prove difficult in practice. A text box with lines numbered 1–3 was provided for responses. Such free-listing tasks have a long tradition in anthropology and are useful to understand how a concept (here: threats) is perceived by studying the cognitive salience, or prominence, of items [14–16].

Sample

All primary care physicians organised in four physician networks in Switzerland were sent the questionnaire together with a cover letter and pre-paid envelope. The sample consists of all primary care physicians (n = 627 physicians) formally organised in four large physician networks and included 472 offices. The physician networks were selected because they cover a broad range of office types and regions in the German-speaking part of Switzerland including two metropolitan areas, smaller cities as well as rural regions. As there is no register of nurses in Switzerland, each physician received two survey sets. The physician was instructed to pass the set labeled “nurse” to one nurse according to the alphabetic position of the first letter of the last names of all nurses working in the office. Nurses of each participating office were sent a letter (addressed to “the nurses working in the office”) to inform about the study and explain how nurses will receive the questionnaire. The survey was completely anonymous. A reminder was sent after two weeks to the entire sample. Return of the questionnaire was regarded as informed consent.

Data analysis

Analysis of the data followed a two-step procedure. First, the verbatim reports of threats were analysed under a “theme-identification” content-analysis framework [17–20]. A 10% random sample of citations was drawn to develop a hierarchical category system inductively by two researchers with expertise in patient safety and qualitative research (KG, DS). We followed an entirely inductive approach and developed categories very close to the material rather than applying a pre-existing classification system of adverse events. Categories were grouped in hierarchical order posthoc [21]. As a consequence of categorising material bottom-up, we accepted that threats that may refer to similar risks occur at different levels and groups of the hierarchy. For example, an unspecified citation “errant documentation” would be categorised as “documentation” whereas “errant documentation of medication” would be subsumed under “medication, documentation”. Two coders independently applied this system and coded a second random sample of 10% of descriptions. After discussion and refinement of the category system, the final coding scheme was applied to the entire sample. The Kappa statistic was calculated to assess interrater reliability. Areas of disagreement were discussed and resolved by consensus.

In a second step, a quantitative assessment of the coded threats was conducted. The unit of analyses was the individual statement of threat not the respondent providing this statement. Sum scores for each category were calculated to combine frequency of threats with “priority weights”, i.e., rank position on the numbered free list. The reversed coded priority numbers were used as rank weights (no citation = 0, priority 3 = weight 1, priority 2 = weight 2, priority 1 = weight 3). To ease interpretation, a priority index was computed that normalises the sum score a threat obtained relative to the theoretical maximum sum score, i.e., if a threat would have been cited and assigned highest priority by all responders: [priority index = (observed sum score – theoretical minimum sum score) / (theoretical maximum sum score – theoretical minimum sum score)]. This index equals 1 if all respondents cited a particular threat and assigned it highest priority and approaches 0 if a threat was described by only one respondent who assigned lowest priority. Chi-square tests, Wilcoxon rank sum tests (for unpaired data) and logistic regression were conducted to test for associations of threats and assigned priorities with occupational group (physician vs. nurse). Statistical analyses were done with STATA v12 [22].

Results

Of the 1260 invited individuals, 630 returned the survey (50% response rate). 391 (219 physicians and 172 nurses) of these 630 respondents described at least one threat to safety (62% item response, 31% response rate). Details of responders are provided in table 1. In total, 936 threats to patient safety were described by responders. There was large variability in the content and in the level of specification and detail provided by responders. Many threats described precise situations, themes and constellations, e.g., “confusion of vaccine [trademark] in children”, “interaction between [drug A] and [drug B]” or “patient falls in the office because of stairs at the entrance to the waiting area”. Other statements were less specific, such as “medication errors”, “wrong diagnosis”, or “failure in communication”. The hierarchical category system included 29 coded categories, organized in 5 main themes (fig. 1). Agreement of coders in assignment of the 29 categories was good (kappa = 0.87, CI = 0.86–0.87). Several patient safety threats were cited in different contexts and with different levels of specification. For example, “confusion” was described as an unspecified risk, as confusion of drugs in the context of medication errors (e.g., sound alikes), and explicitly as misidentification of patients and records in the context of organising office processes.

Figure 1

Category system for coding citations of priority threats to patient safety.

Figure 2

Frequency of cited threats by top-level category and occupational group (n = 936 threats). Numbers on top of bars indicate the mean priority assigned to that category (higher numbers indicate higher priority).

Table 2 presents frequencies of threats, average priority weights (i.e., rank on the numbered freetext list), sum scores and the priority index by category. More than every second threat cited by responders related to medical processes (53%). 28% of citations related to drug therapy and medication errors, 11% to medical procedures performed in the office (e.g., IV administration, vaccination). Several threats were described only occasionally but were on average assigned high priority, e.g., safe office rooms and dosing of drugs. Contrary, organisation of office processes was described in 6% of responses but was assigned low priority. Based on the priority index (i.e., the normalised sum score) the most important threats to patient safety are medication in general and interactions in particular, triage by nurses, and time pressure.

There were considerable differences in frequency of mentions by occupational groups (fig. 2, Chi2 = 119.1, df28, p <0.001). Threats relating to medical processes and limits to individual performance were more likely to be cited by physicians. Less likely to be described as threats by nurses compared to physicians were: errors in diagnosis (OR = 0.21, CI 0.09–0.47, p <0.001), drug interactions (OR = 0.10, CI 0.04–0.25, p <0.001) and compliance of patients (OR = 0.28, CI 0.08–0.96, p = 0.044). More likely to be cited as threat by nurses were: X-rays in the office (OR = 3.34, CI 1.04–10.71, p = 0.043), other procedures in the office (OR = 2.69, CI 1.14–6.35, p = 0.024), confusion of patients or records (OR = 3.28, CI 1.55–6.94, p = 0.002), hygiene and infections (OR = 3.21, CI1.12–9.19, p=0.030), safety of office rooms (OR = 6.70, CI 1.46–30.73, p = 0.014), and confidentiality of patient information (OR = 7.38, CI 1.63–33.50, p = 0.010). Significant differences between professional groups were observed in the priority assigned to only 3 threats: Triage (mean_nurse= 2.37 mean_physicians= 1.94, z = –2.48, p = 0.013), diagnosis (mean_nurse = 1.43 mean_physicians= 2.12, z = 2.24, p = 0.025), and vaccination (mean_nurse = 1.96 mean_physicians = 2.46, z = 2.25, p = 0.024).

With the respondent as unit of analyses, 57% of individual responders specified at least one medication related threat and 24% described at least one procedure related threat (X-ray, vaccination, emergencies, other). The eight categories medication (general), interactions, diagnosis, stress, vaccination, triage, confusion of patients and hygiene covered the top priority threats of 48.9% of physicians and 42.6% of nurses.

Discussion

Summary of main findings

In this study we report threats to patient safety in the primary care office perceived by primary care physicians and nurses. To the authors’ knowledge, this study is the first to provide such data for Switzerland. The threats professionals are most concerned about relate to 29 categories. They show a large degree of variability and include both errors and contributing factors. For example, stress, time pressure and limits to concentration as a contributing factor to errors were described by a considerable fraction of physicians and nurses but would probably not be detected in chart review of actual errors. Contrary to other approaches, we did not examine actual incidents in the offices but asked participants to list safety threats they would prioritise for elimination. Our approach reveals information on the perception and psychological salience of different types of risks, and includes a judgment on preferences for improvement. Knowledge on priorities towards more safety is important to get professionals involved in safety improvement activities and to design interventions that meet the needs and concerns of primary care professionals. Safety of medication in general, triage by nurses and drug interactions in particular were cited most frequently. The perception of physicians and nurses that medication errors are a serious threat to patient safety is in some accordance with chart-review. Contrary, triage is rarely identified explicitly as a safety problem. We also observed important differences between physicians and nurses. Physicians were more likely to be concerned about diagnostic errors and drug interactions while safety of office rooms, confusions and misidentification, and safety of medical procedures are more likely to be reported by nurses.

Strengths and limitations

To the authors’ knowledge this is the first study that employed a guided free list task to assess professionals’ subjective safety concerns rather than asking for actual errors or offering a pre-defined list of risks to select from. Using the open-ended format we collected more diverse answers individuals give spontaneously and unbiased from presented response sets. While we do not know the underlying reasons why specific threats were cited it seems likely that salience of threat is some function of frequency of occurrence, potential for harm, risk aversion towards specific threats (some types of risks may be more acceptable than others), and proximity to personal duties and work environment. Asking for concerns rather than actual incidents has the advantage that we identified those threats to patient safety that may cause adverse events but are often prevented by additional efforts. For example, many physicians and nurses engage in (re-)collecting missing information during the patient encounter to overcome incomplete patient records. Kostopoulou and Delaney used an open-ended question format to study adverse events in the primary care office and developed a taxonomy of cognitive and system factors [23]. Contrary to this study, we asked participants for the subjective importance of threats.

The analysis of frequency and ordering of threats reveals that there are risks that were perceived only by a small minority of responders, but are listed on first position by virtually all who cited that threat indicating high salience of this item in some subgroups (e.g., medication dosing, safe office rooms). Relying on the threats most frequently cited may thus not be a sufficient strategy given the variability of safety issues in primary care. Our study is also novel in that we simultaneously surveyed physicians and nurses working in the same offices and obtained comparable response rates in both groups. Our results therefore shed some light on important differences in perceived risks between these groups though the statistical analysis is limited by the relatively low number of cites per threat. We found that nurses are alerted about safety within the office, e.g., prevention of infections and injuries such as falls. Nurses often directly observe patient flows, behaviours and incidents in the office physicians may not be aware of.

We acknowledge the limitations of our study: First, we sampled physicians and nurses organised in physician networks. The generalisability of our results is therefore unclear. Unfortunately, no data is available to allow comparison of our sample with national or regional data of offices on relevant characteristics. However, the heterogeneity both within and between the participating networks is large. Some require their members to engage in patient safety (e.g., incident reporting). Others are predominantly administrative alliances. Thus, we can be quite confident that our sample represents are large variety of primary care providers rather than a small homogeneous group. This is also indicated by the variability in characteristics and work environments of responders.

Second, we followed an entirely inductive approach to the qualitative data and did not use existing classifications, such as the taxonomy developed by Dovey et al. [24]. While taxonomies commonly ask for “errors in the office”, in our study we studied threats to patient safety [11, 25, 26]. As a consequence our category system of threats includes risks, adverse events, errors, contributing factors, and root causes. The categories are not mutually exclusive. We did not apply existing taxonomies to our data because these were mainly developed based on physicians’ reports of errors (not nurses’) and as not to lose detail and cognitive prominence of particular threats: For example, triaging patients would probably be classified as “Office Administration – Appointments”: Errors in managing appointments for healthcare” in the Dovey et al. taxonomy (Dovey 2002). Only 10 out of 284 reports (3.5%) related to this category. Contrary, in our study 7.2% of threats related to triage, not including other appointment errors. Subsuming specific, explicit threats in broad categories bears the risk of losing the detail required for targeting and developing interventions attractive for those involved.

Third, the qualitative free list task was preceded by the quantitative assessment of incidents in terms of frequency and harm in our survey. This approach has its pros and cons: The presentation of potential incidents certainly affected responses to the free list task. However, respondents were also guided to broaden their evaluation and consider the entire range of safety problems. Thus, they had reflected on the frequency and outcome of incidents before voicing their concerns. Indeed, many concerns submitted by respondents in the free list task were not even explicitly mentioned in the list of incidents, e.g., safety of office rooms and triage.

Finally, our quantitative approach to the qualitative data has its limitations. Whether the “order of mention” can be interpreted as rank is a matter of discussion. However, as subjects completed an ordered list the task in our study was not completely unaided. Free-listing tasks are commonly used to study the psychological salience of concepts or domains [27–30]. Different methods for combining frequency and order of mention in free list tasks exist [31, 32]. We used the (normalised) sum score relative to a hypothetical maximal consensus for easier interpretation and because the average number of threats in our sample was rather small. Other relevant metrics could be the median order of items, the items with a minimum of first positions, or time to mention (when responses are obtained verbally). We also calculated Smiths’ salience index but the resulting relative ordering of themes was nearly identical [33]. However, the assumptions underlying our manipulation should be borne in mind when interpreting the results.

Comparison with existing literature

In our study, concerns for medication related threats compose the largest group of citations. This seems to mirror the true prevalence of this type of errors obtained in incident reporting data [11, 26, 34]. However, in a recent study conducted among family physician offices in Canada, the top four types of self-reported incidents were documentation (41.4%), medication (29.7%), clinical administration (18.7%) and clinical process (17.5%) [35]. Contrary, only 6% of threats were explicitly related to documentation in our study. This suggests considerable differences between reported incidents and perceived threats. The difference between the two results could be used to raise awareness about documentation as a major root cause for safety incidents. Time pressure as a contributing factor was identified as a threat to safety in 5.6% of cites. Thus, time pressure is not a high-salience threat and its impact may be underestimated by physicians and nurses. Amalberti and Brami report that disease tempo, office tempo, patient tempo, out-of-office coordination tempo, and GP’s access to knowledge tempo contributed considerably to incidents that led to malpractice claims [36].

The significance attached to the identification of emergent and urgent conditions (triage) as a major threat is an important result of our study. Descriptions of triage were provided by 20% of nurses and 15% of physicians. Triage as a threat to patient safety is rarely documented explicitly in adverse event studies in primary care but has gained increasing attention more recently. Smits et al. report that 9 out of 27 incidents (33%) identified in record review of out-of-hours primary care patients were related to triage [37]. Giesen et al. report that triage nurses underestimated the level of urgency in 19% of telephone contacts [38]. Telephone triage nurses in Norway correctly classified 82% of acute and 74% of urgent cases based on written scenarios [39]. Our study adds to this evidence that nurses and physicians themselves perceive triage as a major threat to patient safety. Triaging patients for appointment is a difficult and complex task in which safety, efficiency, and physicians’ and patients’ preferences need to be sensitively balanced. In addition, nurses often get no feedback about the quality of their triage performance thus limiting the opportunities to learn from errors. Regular trainings in triage as well as clear communication between physicians and nurses about triage, e.g., asking back, may help to increase safety.

Implications for future research or clinical practice

Our results provide important insight into primary care professionals’ perspectives on threats to patient safety. Physicians and nurses are concerned about many diverse risks to patient safety in their offices. As participants named the threats they would prioritise for “elimination” our results are useful to design safety activities in primary care that could be relevant to both physicians and nurses. Our data show that 8 categories cover the top concerns of nearly 50% of nurses and physicians. Concentrating efforts on these concerns and providing support to office based teams in improving these safety aspects is likely to motivate many professionals to engage in patient safety. Involving both physicians and nurses in detection and analysis of risks in medical offices is thus an important strategy to improve collaboration and quality of care. Our experience is that discussion of safety issues with health care professionals based on their own concerns rather than focusing on errors alone is a good starting point and often more fruitful for the discussion of potential improvements. Contrasting differences between perceived threats and actual errors or contributing factors could be useful for raising awareness towards these risks. Future research is required to illuminate the relation between risk perceptions, prioritisation of threats, and experienced incidents. Qualitative methods like interviewing or focus groups may be useful to gain a deeper understanding of why some threats are cognitively salient. Based on the results obtained in this study, research has been started to assess the safety of triage in primary care, using a combination of qualitative and quantitative methods.

Acknowledgments:The authors thank all physicians and nurses who participated in the study and completed the survey. Advice by several experts on prior versions of the survey is highly appreciated. Support by the FMH is acknowledged.

References

  1 Institute of Medicine. To err is human. Building a safer health system. Washington, DC: National Academy Press; 2000.

  2 Schwappach DL. Frequency of and predictors for patient-reported medical and medication errors in Switzerland. Swiss Med Wkly. 2011;141:w13262.

  3 Zegers M, de Bruijne MC, Wagner C, Hoonhout LHF, Waaijman R, Smits M, et al. Adverse events and potentially preventable deaths in Dutch hospitals: results of a retrospective patient record review study. Qual Saf Health Care. 2009;18(4):297–302.

  4 Aranaz-Andres JM, Aibar-Remon C, Vitaller-Murillo J, Ruiz-Lopez P, Limon-Ramirez R, Terol-Garcia E, et al. Incidence of adverse events related to health care in Spain: results of the Spanish National Study of Adverse Events. J Epidemiol Community Health. 2008;62(12):1022–9.

  5 Soop M, Fryksmark U, Koster M, Haglund B. The incidence of adverse events in Swedish hospitals: a retrospective medical record review study. Int J Qual Health Care. 2009;21(4):285–91.

  6 Vincent C, Neale G, Woloshynowych M. Adverse events in British hospitals: preliminary retrospective record review. BMJ. 2001;322(7285):517–9.

  7 Sandars J, Esmail A. The frequency and nature of medical error in primary care: understanding the diversity across studies. Fam Pract. 2003;20(3):231–6.

  8 Thomsen LA, Winterstein AG, Sondergaard B, Haugbolle LS, Melander A. Systematic Review of the incidence and characteristics of preventable adverse drug events in ambulatory care. Ann Pharmacother. 2007;41(9):1411–26.

  9 Miller GC, Britth HC, Valenti L. Adverse drug events in general practice patients in Australia. Med J Aust. 2006;184(7):321–4.

10 Gurwitz JH, Field TS, Harrold LR, Rothschild J, Debellis K, Seger AC, et al. Incidence and preventability of adverse drug events among older persons in the ambulatory setting. JAMA. 2003;289(9):1107–16.

11 Rubin G, George A, Chinn DJ, Richardson C. Errors in general practice: development of an error classification and pilot study of a method for detecting errors. Qual Saf Health Care. 2003;12(6):443–7.

12 Elder NC, Meulen MV, Cassedy A. The identification of medical errors by family physicians during outpatient visits. Ann Fam Med. 2004;2(2):125–9.

13 Gehring K, Schwappach DLB, Battaglia M, Buff R, Huber F, Sauter P, et al. Frequency of and harm associated with safety incidents in the primary care office (under review). Am J Manag Care. 2012.

14 Weller SC. Cultural consensus theory: applications and frequently asked questions. Field Methods. 2007;19(4):339–68.

15 Bousfield WA, Barclay WD. The relationship between order and frequency of occurrence of restricted associative responses. J Exp Psychol. 1950;40(5):643–7.

16 Bernard HR. Research methods in anthropology: qualitative and quantitative approaches. 4 ed. New York: Altamira Press; 2006.

17 Mayring P. Qualitative content analysis. Qualitative Sozialforschung / Forum: Qualitative Social Research 2000 [cited 2007 Nov 26];1(2)Available from: URL: http://www.qualitative-research.net/fqs-texte/2-00/2-00mayring-e.htm

18 Sandelowski M. Whatever happened to qualitative description? Res Nurs Health. 2000;23(4):334–40.

19 Graneheim UH, Lundman B. Qualitative content analysis in nursing research: concepts, procedures and measures to achieve trustworthiness. Nurs Edu Today. 2004;24(2):105–12.

20 Hsieh HF, Shannon SE. Three approaches to qualitative content analysis. Qual Health Res 2005;15(9):1277–88.

21 Elo S, Kyngas H. The qualitative content analysis process. J Adv Nurs. 2008;62(1):107–15.

22 StataCorp. Stata Statistical Software: Release 12. College Station, TX: Stata Corporation; 2011.

23 Kostopoulou O, Delaney B. Confidential reporting of patient safety events in primary care: results from a multilevel classification of cognitive and system factors. Qual Saf Health Care. 2007;16(2):95–100.

24 Dovey SM, Meyers DS, Phillips RL, Jr., Green LA, Fryer GE, Galliher JM, et al. A preliminary taxonomy of medical errors in family practice. Qual Saf Health Care. 2002;11(3):233–8.

25 Pace WD, Fernald DH, Harris DM, Dickinson LM, Araya-Guerra R, Staton EW, et al. Developing a taxonomy for coding ambulatory medical errors: A report from the ASIPS collaborative. Advances in patient safety: From research to implementation.Rockville,MD: Agency for Healthcare Research and Quality (US); 2005.

26 Makeham MA, Dovey SM, County M, Kidd MR. An international taxonomy for errors in general practice: a pilot study. Med J Aust. 2002;177(2):68–72.

27 Tishelman C, Lövgren M, Broberger E, Hamberg K, Sprangers MAG. Are the most distressing concerns of patients with inoperable lung cancer adequately assessed? A mixed-methods analysis. J Clin Oncol. 2010;28(11):1942–9.

28 Lindqvist O, Tishelman C, Hagelin CL, Clark JB, Daud ML, Dickman A, et al. Complexity in non-pharmacological caregiving activities at the end of life: An international qualitative study. PLoS Med. 2012;9(2):e1001173.

29 Fennell D, Liberato ASQ, Zsembik B. Definitions and patterns of CAM use by the lay public. Complementary Therapies in Medicine. 2009;17(2):71–7.

30 Bennett I, Switzer J, Aguirre A, Evans K, Barg F. “Breaking it down”: Patient-clinician communication and prenatal care among African American women of low and higher literacy. The Annals of Family Medicine. 2006;4(4):334–40.

31 Sutrop U. List Task and a Cognitive Salience Index. Field Methods. 2001;13(3):263–76.

32 Libertino L, Ferraris D, Lopez Osornio MM, Hough G. Analysis of data from a free-listing study of menus by different income-level populations. Food Quality and Preference. 2012;24(2):269–75.

33 Smith JJ. Using ANTHOPAC 3.5 and a spreadsheet to compute a free-list salience index. Field Methods. 1993;5(3):1–3.

34 Hoffmann B, Beyer M, Rohe J, Gensichen J, Gerlach FM. “Every error counts”: a web-based incident reporting and learning system for general practice. Qual Saf Health Care. 2008;17(4):307–12.

35 O’Beirne M, Sterling PD, Zwicker K, Hebert P, Norton PG. Safety incidents in family medicine. BMJ Quality & Safety. 2011;20(12):1005–10.

36 Amalberti R, Brami J. Tempos management in primary care: a key factor for classifying adverse events, and improving quality and safety. BMJ Quality & Safety 2011.

37 Smits M, Huibers L, Kerssemeijer B, de Feijter E, Wensing M, Giesen P. Patient safety in out-of-hours primary care: a review of patient records. BMC Health Serv Res. 2010;10(1):335.

38 Giesen P, Ferwerda R, Tijssen R, Mokkink H, Drijver R, van den Bosch W, et al. Safety of telephone triage in general practitioner cooperatives: do triage nurses correctly estimate urgency? Qual Saf Health Care. 2007;16(3):181–4.

39 Hansen EH, Hunskaar S. Telephone triage by nurses in primary care out-of-hours services in Norway: an evaluation study based on written case scenarios. BMJ Quality & Safety. 2011;20(5):390–6.