Negligible impact of highly patient-specific decision support for potassium-increasing drug-drug interactions – a cluster-randomised controlled trial

DOI: https://doi.org/10.4414/smw.2019.20035

Background

Drug-drug interactions (DDIs) are an important cause of adverse drug events and may thereby increase morbidity, mortality and costs [1–3]. Since recognised DDIs are often preventable, clinical decision support (CDS) systems have been developed to automatically detect DDIs and inform the healthcare providers [4]. However, as a result of insufficient consideration of patient-specific parameters and variable clinical significance, providers may perceive automated notifications as annoying and override them [5]. In fact, it has been shown that with a high number of automated CDS messages the likelihood of overriding increases (‘alert fatigue’), which bears the risk that providers miss critical notifications [6]. To improve the acceptance and effectiveness of CDS notifications, various approaches have been proposed:

• focusing on high-priority DDIs [7]
• displaying context information in notifications such as current laboratory values [8]
• patient data-driven filtering/suppressing of improper notifications [9]
• displaying notifications at the right time, when adverse event becomes likelier [10]
• removing notifications when the triggering conditions are no longer met [11]

Whereas CDS interventions usually feature only a few of these approaches, we designed a concept hereafter described that combined all features (a–e) to reduce the number of CDS notifications and thereby minimise alert fatigue.

Among frequently observed DDIs are potassium-increasing DDIs [12, 13], which are of high clinical significance because hyperkalaemia can induce potentially fatal cardiac arrhythmias [14]. Potassium-increasing DDIs are the most frequent high-priority DDIs at our institution [15]. In our previous analyses of inpatients’ electronic health record (EHR) data, we identified patient- and physician-related risk factors for hyperkalaemia [16] in order to develop patient-specific models for calculating the risk for hyperkalaemia over time during potassium-increasing DDIs [10]. Based on the model with the highest specificity, we designed and implemented CDS notifications to improve care in potassium-increasing DDIs.

The purpose of this cluster-randomised controlled clinical trial was to determine the impact of our innovative CDS feature combination on both, process improvements and occurrences of hyperkalaemia.

Methods

Setting, design, subjects and study period

The study was conducted at the University Hospital Zurich, a tertiary care academic medical centre in Switzerland, which has approximately 850 beds and manages nearly 40,000 hospital stays per year.

The study was designed as a hospital-wide cluster-randomised controlled trial. Patients were grouped into 29 clusters, and each cluster was a clinical unit for inpatients. Allocation of the clinical units to either the intervention or the control group was based on restricted replacement randomisation [17].

All inpatients receiving potassium-increasing DDIs were included in the study if they were treated in units where medications were ordered via computerised provider order entry (CPOE). Intensive care units did not use CPOE during the study period, and patient stays on these units were therefore not considered. Outpatient visits were excluded.

The present CDS intervention was first activated on 7 January 2014, and the study period lasted 1 year. No other automated CDS was addressing DDIs before or during the study.

The Cantonal Ethics Committee Zurich (KEK-ZH-Nr. 2013-0507), which is the Institutional Review Board (IRB) of the University Hospital Zurich, approved the study and patient consent was waived.

Randomisation procedure

Before the study started, we used the following procedure to balance the random allocation of the clinical units to the two study groups.

We retrospectively computed for each study group (i) the total number of potassium-increasing DDIs and (ii) the proportion of prolonged potassium monitoring intervals, defined as the number of intervals >72 hours divided by the total number of potassium measurement intervals. Importantly, the latter calculation was based on patients with potassium-increasing DDIs and only considered the monitoring intervals as long as the respective combination of drugs was active, i.e., as long as the DDI lasted. The matching was then performed as follows: We accepted the random allocation of the clinical units if the intervention-to-control group ratios based on (i) and (ii) were each within the range 2/3 to 3/2 (i.e. maximum difference of 50% for the number of DDIs and delayed monitoring, respectively). This constraint was applied to control for the frequency of potassium-increasing DDIs and insufficient potassium monitoring, thereby establishing a balanced allocation of the clinical units to the

• intervention group: (14 units: abdominal surgery
• control group (15 units: angiology

^a Surgical units, ^b internal medicine and related units.

The study team used the software R, version 3.4.0 (R Foundation for Statistical Computing, Vienna, Austria) for the randomisation, and enrolled/assigned the clusters to the intervention and control groups by parameterising the algorithm displaying notifications. Eventually, all included patients were automatically enrolled and allocated to the intervention and control groups depending on the unit where they stayed.

The electronic health record system on site

Since the stepwise introduction of CPOE at the University Hospital Zurich from 2009 to 2010, inpatient care is comprehensively documented and managed by a vendor EHR system (Kisim, Cistec AG, Zurich, Switzerland). All pharmacological therapies, other treatments and diagnostic procedures are ordered via CPOE on all wards except for intensive care units (ICUs). The system offers expandable CDS functionality with optional consideration of medication and laboratory data.

DDI knowledge base and potassium-increasing drugs

The regularly updated medication knowledge base galdat/hospINDEX (distributed to customers within the Swiss market by HCI Solutions AG, Bern, Switzerland), derived from ABDATA Pharma-Daten Service (Werbe- und Vertriebsgesellschaft Deutscher Apotheker, Eschborn, Germany), is automatically loaded as part of the EHR backend and integrated into the frontends of the EHR system at our institution. This knowledge base is available to clinicians and pharmacologists for reference [15] and was used in this study to identify potassium-increasing DDIs. Of note, all DDIs mentioned in this study should be considered as potential DDIs that did not necessarily induce adverse events.

The knowledge base tiers DDIs into six levels [18]. The tiers 1 to 3 indicate serious [19] DDIs: (i) contraindicated, (ii) contraindicated as precaution, (iii) monitoring or adaptation required. Only these serious potassium-increasing DDIs were considered by the investigated CDS intervention and in the analyses of this study, whereas the remaining tiers were excluded: (iv) monitoring or adaption in the case of risk factors, (v) monitoring as a precaution, (vi) no action required. In the context of this DDI study, drugs that can induce hyperkalaemia on their own, such as potassium supplements, were only considered in combination with another drug, according to the DDI definitions of tiers 1 to 3.

Potassium-increasing drugs were defined as drugs involved in serious potassium-increasing DDIs according to the knowledge base. Among those drugs were angiotensin converting-enzyme inhibitors (ACE inhibitors), angiotensin antagonists (angiotensin-receptor blockers), direct renin inhibitors, immunosuppressive agents (calcineurin inhibitors), potassium-sparing diuretics (aldosterone receptor antagonists and epithelial sodium channel blockers), potassium supplements and trimethoprim.

The algorithm displaying electronic notifications

Electronic notifications were displayed only in the EHRs of patients hospitalised in the units of the intervention group, identically visible to physicians and nurses, and they were displayed only as long as the triggering conditions were met. The display of the notifications was suppressed in the control group. Nevertheless, data from both study groups were consistently collected in the background.

At onset of and during potassium-increasing DDIs the algorithm checked whether a current serum potassium value was available, and if so, whether the potassium level was elevated. Otherwise, the algorithm checked if a potassium level measurement had been ordered. Three types of electronic notifications could be displayed as a result of these checks:

• Reminders:
  • If the most recent potassium measurement was more than 48 hours previously and no monitoring had been planned, the algorithm displayed a reminder that suggested ordering a serum potassium measurement.
• Warnings:
  • If the current serum potassium value was ≥4.9 mEq/l, the CDS warned against the increasing risk of hyperkalemia.
• Alerts:
  • If the current serum potassium value was ≥ 5.5 mEq/l, a notification was displayed alerting the provider about a hyperkalemic state of the patient [20].

Reminders, warnings and alerts each appeared as a non-interruptive red bar in the top section of the EHR (a screenshot of a similar CDS notification bar has been published elsewhere [21]). By clicking on this bar, a pop-up window was displayed that notified the provider of the reason for the intervention, the prescribed potassium-increasing DDI, the most recent serum potassium value, the creatinine and the glomerular filtration rate, and it also included suggestions how to proceed. The preferred action could directly be initiated by a single click on one of these suggested options. Two additional buttons referred to all identified DDIs, and to further general information. Although all notifications were visible to both physicians and nurses, only physicians had the ability to choose options in the pop-up windows.

Primary and secondary endpoints

The primary endpoint for assessing the impact of the intervention was the frequency of prolonged potassium-monitoring intervals during potassium-increasing DDIs, defined as periods without any potassium level measurement lasting longer than 72 hours. When a potassium measurement was ordered, the monitoring interval was immediately reduced to this point in time (shortening the monitoring period). Also, when a DDI was discontinued, the monitoring interval was immediately reduced to the timepoint of DDI discontinuation.

The secondary endpoints of the study included the mean length in hours of potassium monitoring intervals during DDIs, the frequency of hyperkalaemic patient states during DDIs, the frequency of potassium-increasing DDIs ordered for patients with hyperkalaemia, the mortality related to hyperkalaemia (within 48 hours after occurrence of hyperkalaemia), and finally provider responses to the CDS notifications in the intervention group (which were automatically assigned in the control group merely for comparison purposes).

Sample size calculation, data processing and statistics

Based on a retrospective analysis of EHR data, we estimated that the proportion of prolonged monitoring intervals could be reduced in the intervention group from 6% to 5%. With a type I error rate of 0.05 and a power of 80%, we calculated a sample size of 6426 monitoring intervals for each study group. The study period of one year additionally addressed potential concerns about seasonal influences [22].

Routinely collected EHR data were extracted from the clinical data warehouse, exported to raw data files, which in turn were imported into a separate database management system and processed using structured query language (SQL) statements.

Statistics were computed using the software R, version 3.4.0 (R Foundation for Statistical Computing, Vienna, Austria). Group comparisons of continuous variables were performed using the Wilcoxon rank-sum test. Group comparisons of proportions were performed using the Fisher’s exact test. Hereafter, p-values of ≤0.05 are considered to be statistically significant.

Results

We analysed data of 34,253 inpatient stays, as shown in table 1. These patients received 922,121 medications during the study period, resulting in a total of 4861 potassium increasing DDIs.

Clinical endpoints

The analysis of the primary endpoint showed that the frequency of insufficiently monitored DDI periods was not reduced by the monitoring reminders. In the intervention group, 4.66% (451 of 9686) potassium-monitoring periods lasted >72 h, compared with the control group, where 4.06% (249 of 6140) prolonged monitoring periods were found. Independent of the study groups, we found that patients treated in internal medicine and related units had prolonged monitoring intervals in 4.32% of cases and those in surgical units in 3.50%.

However, a statistically significant difference was observed for one secondary endpoint. The mean length of the potassium monitoring intervals during DDIs was slightly shorter in the intervention group (22.9 h) compared with the control group (23.7 h; p <0.001).

No benefit was found with respect to other secondary endpoints. In the intervention group, during 50 of 2804 observed potassium-increasing DDI periods (1.78%) a serum potassium value ≥5.5 mEq/l was measured at least once. In the control group, during 27 of 2057 potassium-increasing DDI periods (1.31%) one or more serum potassium values ≥5.5mEq/l were detected (p = 0.20). In the intervention group, seven potassium-increasing DDIs (0.25%) were ordered in the presence of hyperkalaemia, in the control group ten (0.49%; p = 0.22).

In the intervention group 22 patients died during the study, in the control group 14; however, none of these fatalities were related to hyperkalaemia.

CDS notifications, provider responses and events

The providers in the intervention group could acknowledge the notification or enter override reasons, whereas all results of the control group had to be evaluated automatically, based on electronically available data recorded after a notification had been suppressed. For comparison purposes, the respective data from the intervention group were analogously evaluated.

Table 2 reports the number of CDS notifications by type and categorised provider responses. In a relatively small proportion of cases the notifications were triggered again after the providers announced plans to monitor potassium or to discontinue the DDI but did not implement their intentions. For instance, providers ordered 412 potassium levels and announced further measurements; however, the lack of implementation caused the display of 38 additional reminders summing up to 450.

In the evaluation of events recorded, the most notable difference between the two study groups were the higher numbers of reminders and warnings in the intervention group, explainable by an increased number of medication orders inducing more potassium-increasing DDIs in the intervention group (cf. table 1).

Patient-specific notifications

Based on the number of displayed notifications and the number of potassium-increasing DDIs in the intervention group (2804) we calculated ratios to illustrate the intended reduction of the number of notifications.

A total of 869 reminders to monitor potassium were displayed, representing a ratio of 1 per 3.2 potassium-increasing DDIs, 356 notifications warned against the risk of a hyperkalaemia (1 per 7.9 DDIs), and 62 hyperkalaemia alerts were triggered (1 per 45.2 DDIs). These ratios compare favorably to a hypothetical (but common) concept of displaying notifications during order entry for each potassium-increasing DDI. If the 62 episodes of hyperkalaemia were the direct result of the 2804 potassium-increasing DDIs in the intervention group, then the concept of displaying notifications during order entry would have had a positive predictive value of only 2.2%. This is in line with a retrospective analysis previously performed at our institution [10].

Discussion

We implemented an innovative, comprehensive and highly patient-specific alert concept to improve safety during potassium-increasing DDIs and to minimise the risk of overriding and alert fatigue. Although the electronic notifications had no significant impact on clinical endpoints, we observed a statistically significant reduction of the length of potassium monitoring intervals. However, this modest reduction of the time to monitor the potassium serum level from 23.7 to 22.9 hours may have improved the process – at best – and is of very limited clinical significance. All other endpoints were not influenced as intended. Our cluster-randomised controlled trial showed no impact on poorly monitored DDIs with monitoring intervals longer than 72 hours and we were unable to reduce the number of hyperkalaemia occurrences. We therefore question the effectiveness of CDS interventions against DDIs, which has also been questioned by others [24].

Duke at al. [8] performed a similar study in the outpatient setting to investigate “context enhanced” DDI alerts for patients at risk of hyperkalaemia. To our knowledge, this has been the only study so far that also targeted a broad range of DDIs increasing a patient’s serum potassium, based on the automated detection of “six class interactions (combinatorial of ACE inhibitors, angiotensin receptor blockers, potassium-sparing diuretics, and potassium supplements)”. This context-enhanced type of DDI alert, if displayed to a provider of the intervention group, included information on the triggering DDI, and the current potassium and creatinine levels. Importantly, in their study the control group was also notified of potassium-increasing DDIs but without additional information on laboratory measurements. The study by Duke et al. [8] was unable to show any improvements of the providers’ adherence and concluded that context-enhanced alerts were not superior to the usual ones. In our study, however, the notifications were completely withheld from the control group.

The literature on automated monitoring reminders distinguishes between synchronous and asynchronous notifications [25]. There are published studies showing successful interventions synchronous to the ordering process [26] – sometimes considered as “corollary orders” [27] – as well as of asynchronous monitoring reminders [25]. Some of these publications do mention a form of monitoring reminders targeted at potassium levels; however, one did not report on related outcomes [27], and the other investigated alerts of low potassium levels during treatment with digoxin [25]. Therefore, the comparability with our study is limited. Our reminders and alerts were non-interruptive and were asynchronously displayed, but – depending on potassium levels electronically available at the time of ordering – notifications may have been displayed before or immediately after completing the order. As opposed to non-interruptive designs, interruptive CDS has been linked to patient safety concerns; for instance, one group reported that CDS against warfarin trimethoprim-sulfamethoxazole DDIs caused unacceptable treatment delays [28, 29].

In comparison to published recommendations [30], our CDS notifications featured various assets, such as

• consideration of current laboratory values in the CDS logic to reduce the alert burden and minimise the risk for alert fatigue,
• non-interruptive notification bar linking to information window that is accessible on demand,
• distinct colours for each notification type within the respective information windows of reminders, warnings, and alerts, indicating seriousness,
• presentation of interacting drug pair and patient-specific context including laboratory values,
• concise instructions with minimal text, signal words and provision of potential clinical consequences,
  • suggested options (e.g., order potassium measurement) directly available from the information window,
• further details available via links,
• possibility to enter override reasons which were routinely collected,
• and further patient-specific DDI checks – for all potential DDIs featured in the comprehensive knowledge base used – could be triggered on demand [15],

while other recommended features were lacking, i.e.,

• there was no explicit provision of specific evidence for the seriousness of the DDI or the underlying mechanism,
• none of the CDS types was interruptive, even when hyperkalemia was detected,
• our asynchronous notifications may have been displayed shortly after the time of decision making (but stayed visible a lengthy period),
• providers were unable to turn notifications permanently off, and no “snooze” function was available,
• presentation of notifications did not vary by clinician type,
• notifications could not be forwarded by a single click to others,
• and patients could access neither information of their EHR, nor on CDS notifications via patient portals.

In summary, we used an innovative, highly patient-specific combination of CDS features intended to improve adherence to electronic notifications and to reduce overriding and minimise the risk for alert fatigue. Among the features was a functionality that displayed the triggering potassium-increasing DDI, the most recent serum potassium value and the glomerular filtration rate. The intervention targeted the most frequent high-priority DDI at our institution. The hospital-wide study was conducted at a large tertiary care academic medical centre, a leading teaching hospital in Switzerland that comprehensively manages inpatient care by a state-of-the-art EHR system.

Randomisation procedures for hospital-wide controlled trials investigating clinical decision support are challenging. The standard approach of randomising individual patients carries the risk of contamination due to the fact that the physician will receive notifications for some patients but not for all. On the one hand, the effect of notifications received for patients in the intervention group might influence the reasoning and actions of a physician when caring for patients of the control group, potentially having an impact on the control group as well. On the other hand, providers might feel unreasonably confident that they would receive notifications by the system for any patient, despite a potential allocation of a patient to the control group, which might introduce patient safety risks. We therefore randomised the 29 clinical units. Hereby, contamination was limited to situations where a physician in charge asked for advice from a consultant of a unit allocated to the other study arm. In order to ensure the best possible comparability of the two study groups in regard of our endpoints, a balanced cluster randomisation was used. However, this balancing method does not necessarily prevent uneven distributions of all patient and study parameters between the two groups.

Our study has additional limitations that need to be taken into account in interpreting the results. It was a single centre trial and only inpatient data were available, whereas approaches for outpatient follow-up were not considered and therefore patient outcomes after discharge remain unknown. Some work related to that specific area has recently been done by Saito et al. [31]. Furthermore, we processed large amounts of routinely collected EHR data but we did not additionally perform chart reviews to identify potential adverse drug events (ADEs) induced by hyperkalaemic states that might have been insufficiently addressed by our approaches. However, the only endpoint that differed between the study groups as intended showed a negligible impact of our intervention, and no meaningful difference in the ADE incidences would be expected as a consequence of a 0.8 hours (3.4%) shorter monitoring interval. Finally, comparing the in-hospital mortality between the study groups lacked any consideration of confounders; however, none of the patients died because of hyperkalaemia during the study.

It should be noted that we measured mean monitoring periods of <24 hours, indicating an already high quality of care in the context of the considered DDIs. This contributes to the inherent methodological challenge arising from a relatively low incidence of serious potassium homeostasis disorders and resulting ADEs induced by potassium-increasing DDIs. It appears that the potential to improve patient safety was limited despite a high DDI frequency.

Although there is a broad range of evidence on the ability of CDS to improve processes, only a few results are available that support the viewpoint that CDS also improves patient outcomes [32, 33]. Our study particularly challenges the usefulness of automated notifications about potassium-increasing DDIs despite a low alert burden. In this context, we increasingly question the justification for both (i) direct costs due to research, development and maintenance of the information technology and (ii) indirect expenses due to the burden providers experience in handling clinically insignificant notifications that are to be appropriately overridden [5, 34]. Ultimately, even if potassium-increasing DDIs were to clearly increase mortality due to serious hyperkalaemic states with fatal cardiac arrhythmias, CDS could still not guarantee clinical effectiveness, which is hampered by high override rates [5]. The need for outcome studies investigating the true clinical impact of DDIs has recently been advocated [35], and therefore, future research should focus on CDS targeting evidence-based clinical consequences that are frequent and serious, as opposed to frequent and perhaps serious DDIs. Additional considerations for future studies include qualitative research and focus group discussions on human factors and usability, testing populations with important comorbidities such as diabetes and renal failure, combination outcomes that take into account all of the providers’ options to handle clinical issues and different CDS types, and randomised controlled trials with the patients or the providers as units of randomisation.

In conclusion, our asynchronous, highly patient-specific, innovative CDS feature combination was unsuccessful in improving patient safety in terms of the most frequent high-priority DDI at our institution.

Acknowledgements

Parts of the data were shown during a presentation at the eHealth Summit 2015 in Bern, Switzerland, and an abstract was presented at the AMIA Annual Meeting 2016 in Washington, D.C., USA. We thank Dominik Kamm (Cistec AG, Zurich, Switzerland) for the programming of the reminder algorithm.

References

1Institute of Medicine (US) Committee on Quality of Health Care in America. To Err is Human: Building a Safer Health System [Internet]. Kohn LT, Corrigan JM, Donaldson MS, editors. Washington (DC): National Academies Press (US); 2000 [cited 2017 Aug 16]. Available from: http://www.ncbi.nlm.nih.gov/books/NBK225182/  

2 Dechanont S , Maphanta S , Butthum B , Kongkaew C . Hospital admissions/visits associated with drug-drug interactions: a systematic review and meta-analysis. Pharmacoepidemiol Drug Saf. 2014;23(5):489–97. doi:.https://doi.org/10.1002/pds.3592  

3 Moura CS , Acurcio FA , Belo NO . Drug-drug interactions associated with length of stay and cost of hospitalization. J Pharm Pharm Sci. 2009;12(3):266–72. doi:.https://doi.org/10.18433/J35C7Z  

4 Nabovati E , Vakili-Arki H , Taherzadeh Z , Saberi MR , Medlock S , Abu-Hanna A , et al. Information Technology-Based Interventions to Improve Drug-Drug Interaction Outcomes: A Systematic Review on Features and Effects. J Med Syst. 2017;41(1):12. doi:.https://doi.org/10.1007/s10916-016-0649-4  

5 van der Sijs H , Aarts J , Vulto A , Berg M . Overriding of drug safety alerts in computerized physician order entry. J Am Med Inform Assoc. 2006;13(2):138–47. doi:.https://doi.org/10.1197/jamia.M1809  

6 Cho I , Slight SP , Nanji KC , Seger DL , Maniam N , Fiskio JM , et al. The effect of provider characteristics on the responses to medication-related decision support alerts. Int J Med Inform. 2015;84(9):630–9. doi:.https://doi.org/10.1016/j.ijmedinf.2015.04.006  

7 Phansalkar S , Desai AA , Bell D , Yoshida E , Doole J , Czochanski M , et al. High-priority drug-drug interactions for use in electronic health records. J Am Med Inform Assoc. 2012;19(5):735–43. doi:.https://doi.org/10.1136/amiajnl-2011-000612  

8 Duke JD , Li X , Dexter P . Adherence to drug-drug interaction alerts in high-risk patients: a trial of context-enhanced alerting. J Am Med Inform Assoc. 2013;20(3):494–8. doi:.https://doi.org/10.1136/amiajnl-2012-001073  

9 Tilson H , Hines LE , McEvoy G , Weinstein DM , Hansten PD , Matuszewski K , et al. Recommendations for selecting drug-drug interactions for clinical decision support. Am J Health Syst Pharm. 2016;73(8):576–85. doi:.https://doi.org/10.2146/ajhp150565  

10 Eschmann E , Beeler PE , Schneemann M , Blaser J . Developing strategies for predicting hyperkalemia in potassium-increasing drug-drug interactions. J Am Med Inform Assoc. 2017;24(1):60–6. doi:.https://doi.org/10.1093/jamia/ocw050  

11 Beeler PE , Kuster SP , Eschmann E , Weber R , Blaser J . Earlier switching from intravenous to oral antibiotics owing to electronic reminders. Int J Antimicrob Agents. 2015;46(4):428–33. doi:.https://doi.org/10.1016/j.ijantimicag.2015.06.013  

12 Zwart-van Rijkom JEF , Uijtendaal EV , ten Berg MJ , van Solinge WW , Egberts ACG . Frequency and nature of drug-drug interactions in a Dutch university hospital. Br J Clin Pharmacol. 2009;68(2):187–93. doi:.https://doi.org/10.1111/j.1365-2125.2009.03443.x  

13 Sánchez Muñoz-Torrero JF , Barquilla P , Velasco R , Fernández Capitan MC , Pacheco N , Vicente L , et al. Adverse drug reactions in internal medicine units and associated risk factors. Eur J Clin Pharmacol. 2010;66(12):1257–64. doi:.https://doi.org/10.1007/s00228-010-0866-6  

14 Gennari FJ . Disorders of potassium homeostasis. Hypokalemia and hyperkalemia. Crit Care Clin. 2002;18(2):273–88, vi. doi:.https://doi.org/10.1016/S0749-0704(01)00009-4  

15 Beeler PE , Eschmann E , Rosen C , Blaser J . Use of an on-demand drug-drug interaction checker by prescribers and consultants: a retrospective analysis in a Swiss teaching hospital. Drug Saf. 2013;36(6):427–34. doi:.https://doi.org/10.1007/s40264-013-0022-1  

16 Eschmann E , Beeler PE , Kaplan V , Schneemann M , Zünd G , Blaser J . Patient- and physician-related risk factors for hyperkalaemia in potassium-increasing drug-drug interactions. Eur J Clin Pharmacol. 2014;70(2):215–23. doi:.https://doi.org/10.1007/s00228-013-1597-2  

17 Schulz KF , Grimes DA . Generation of allocation sequences in randomised trials: chance, not choice. Lancet. 2002;359(9305):515–9. doi:.https://doi.org/10.1016/S0140-6736(02)07683-3  

18 Hansten PD , Horn JR , Hazlet TK . ORCA: OpeRational ClassificAtion of drug interactions. J Am Pharm Assoc (Wash). 2001;41(2):161–5. doi:.https://doi.org/10.1016/S1086-5802(16)31244-X  

19 Scheife RT , Hines LE , Boyce RD , Chung SP , Momper JD , Sommer CD , et al. Consensus recommendations for systematic evaluation of drug-drug interaction evidence for clinical decision support. Drug Saf. 2015;38(2):197–206. doi:.https://doi.org/10.1007/s40264-014-0262-8  

20 Uijtendaal EV , Zwart-van Rijkom JEF , van Solinge WW , Egberts TCG . Frequency of laboratory measurement and hyperkalaemia in hospitalised patients using serum potassium concentration increasing drugs. Eur J Clin Pharmacol. 2011;67(9):933–40. doi:.https://doi.org/10.1007/s00228-011-1028-1  

21 Beeler PE , Eschmann E , Schumacher A , Studt J-D , Amann-Vesti B , Blaser J . Impact of electronic reminders on venous thromboprophylaxis after admissions and transfers. J Am Med Inform Assoc. 2014;21(e2):e297–303. doi:.https://doi.org/10.1136/amiajnl-2013-002225  

22 Dowell SF , Ho MS . Seasonality of infectious diseases and severe acute respiratory syndrome-what we don’t know can hurt us. Lancet Infect Dis. 2004;4(11):704–8. doi:.https://doi.org/10.1016/S1473-3099(04)01177-6  

23 Indermitte J , Burkolter S , Drewe J , Krähenbühl S , Hersberger KE . Risk factors associated with a high velocity of the development of hyperkalaemia in hospitalised patients. Drug Saf. 2007;30(1):71–80. doi:.https://doi.org/10.2165/00002018-200730010-00007  

24 Bryant AD , Fletcher GS , Payne TH . Drug interaction alert override rates in the Meaningful Use era: no evidence of progress. Appl Clin Inform. 2014;5(3):802–13. doi:.https://doi.org/10.4338/ACI-2013-12-RA-0103  

25 Galanter WL , Polikaitis A , DiDomenico RJ . A trial of automated safety alerts for inpatient digoxin use with computerized physician order entry. J Am Med Inform Assoc. 2004;11(4):270–7. doi:.https://doi.org/10.1197/jamia.M1500  

26 Raebel MA , Lyons EE , Chester EA , Bodily MA , Kelleher JA , Long CL , et al. Improving laboratory monitoring at initiation of drug therapy in ambulatory care: a randomized trial. Arch Intern Med. 2005;165(20):2395–401. doi:.https://doi.org/10.1001/archinte.165.20.2395  

27 Overhage JM , Tierney WM , Zhou X-H , McDonald CJ . A randomized trial of “corollary orders” to prevent errors of omission. J Am Med Inform Assoc. 1997;4(5):364–75. doi:.https://doi.org/10.1136/jamia.1997.0040364  

28 Beeler PE , Bates DW , Hug BL . Clinical decision support systems. Swiss Med Wkly. 2014;144:w14073.  

29 Strom BL , Schinnar R , Aberra F , Bilker W , Hennessy S , Leonard CE , et al. Unintended effects of a computerized physician order entry nearly hard-stop alert to prevent a drug interaction: a randomized controlled trial. Arch Intern Med. 2010;170(17):1578–83. doi:.https://doi.org/10.1001/archinternmed.2010.324  

30 Payne TH , Hines LE , Chan RC , Hartman S , Kapusnik-Uner J , Russ AL , et al. Recommendations to improve the usability of drug-drug interaction clinical decision support alerts. J Am Med Inform Assoc. 2015;22(6):1243–50. doi:.https://doi.org/10.1093/jamia/ocv011  

31 Saito Y , Yamamoto H , Nakajima H , Takahashi O , Komatsu Y . Incidence of and risk factors for newly diagnosed hyperkalemia after hospital discharge in non-dialysis-dependent CKD patients treated with RAS inhibitors. PLoS One. 2017;12(9):e0184402. doi:.https://doi.org/10.1371/journal.pone.0184402  

32 Lobach D , Sanders GD , Bright TJ , Wong A , Dhurjati R , Bristow E , et al. Enabling health care decisionmaking through clinical decision support and knowledge management. Evid Rep Technol Assess (Full Rep). 2012;(203):1–784.  

33 Wong A , Amato MG , Seger DL , Rehr C , Wright A , Slight SP , et al. Prospective evaluation of medication-related clinical decision support over-rides in the intensive care unit. BMJ Qual Saf. 2018;27(9):718–24. doi:.https://doi.org/10.1136/bmjqs-2017-007531  

34 Nanji KC , Seger DL , Slight SP , Amato MG , Beeler PE , Her QL , et al. Medication-related clinical decision support alert overrides in inpatients. J Am Med Inform Assoc. 2018;25(5):476–81. doi:.https://doi.org/10.1093/jamia/ocx115  

35 Bykov K , Gagne JJ . Generating Evidence of Clinical Outcomes of Drug-Drug Interactions. Drug Saf. 2017;40(2):101–3. doi:.https://doi.org/10.1007/s40264-016-0496-8