a Division of General Internal Medicine & Primary Care, Brigham and Women’s Hospital, Boston, MA, USA
b Harvard Medical School, Boston, MA, USA
c Research Center for Medical Informatics, University Hospital Zurich, Switzerland
d Partners Healthcare Systems, Inc., Wellesley, MA, USA
e Basel University Hospital, Switzerland
Clinical decision support (CDS) systems link patient data with an electronic knowledge base in order to improve decision-making and computerised physician order entry (CPOE) is a requirement to set up electronic CDS. The medical informatics literature suggests categorising CDS tools into medication dosing support, order facilitators, point-of-care alerts and reminders, relevant information display, expert systems and workflow support. To date, CDS has particularly been recognised for improving processes. CDS successfully fostered prevention of deep-vein thrombosis, improved adherence to guidelines, increased the use of vaccinations, and decreased the rate of serious medication errors. However, CDS may introduce errors, and therefore the term “e-iatrogenesis” has been proposed to address unintended consequences. At least two studies reported severe treatment delays due to CPOE and CDS. In addition, the phenomenon of “alert fatigue” – arising from a high number of CDS alerts of low clinical significance – may facilitate overriding of potentially critical notifications. The implementation of CDS needs to be carefully planned, CDS interventions should be thoroughly examined in pilot wards only, and then stepwise introduced. A crucial feature of CPOE in combination with CDS is speed, since time consumption has been found to be a major factor determining failure. In the near future, the specificity of alerts will be improved, notifications will be prioritised and offer detailed advice, customisation of CDS will play an increasing role, and finally, CDS is heading for patient-centred decision support. The most important research question remains whether CDS is able to improve patient outcomes beyond processes.
Key words: decision support systems; clinical; health IT; medicine
The volume, complexity and dynamics of clinical information are a challenge for physicians and other health professionals. Clinical decision support (CDS) systems help to tackle this challenge. A CDS system links patient data with a knowledge base to generate information and suggestions that help providers improve the health care they deliver [1–3]. A knowledge base is a database that stores knowledge in a suitable form depending on its use .
There are several definitions of CDS in the literature showing the wide range of functionality included in these systems. Shortliffe defined CDS in the following words: “A medical decision-support system is any computer programme designed to help health professionals make clinical decisions” . According to Sim et al. CDS is defined as “software that is designed to be a direct aid to clinical decision-making, in which the characteristics of an individual patient are matched to a computerised clinical knowledge base and patient-specific assessments or recommendations are then presented to the clinician or the patient for a decision” .
Computerised physician order entry (CPOE) means that physicians order online – and it is linked with CDS. The combination of CPOE and CDS has proven to be very effective for reducing the frequency of potential adverse drug events .
Technically, CDS software incorporates the generic steps input, processing, and output: (i.) The patient-specific data are entered by health professionals involved in the care, (ii.) processed and linked to knowledge stored in a data base, and (iii.) notifications are communicated back to the clinicians .
CDS approaches pursuing various objectives have been developed and evaluated . For instance, CDS successfully fostered the prevention of deep-vein thrombosis , improved the adherence to glucose regulation guidelines for critically ill patients , increased the use of vaccinations and other preventive measures , significantly decreased the rate of serious medication errors , and one system identified drug-induced thrombocytopenia in an intensive care unit .
CDS types and functions
Basic actions of CDS systems include alerting, reminding, critiquing (rejecting orders), interpreting, predicting, diagnosing, assisting and suggesting [13, 14].
Shortliffe distinguished three basic CDS types: (i.) CDS for information management to provide information on patients or medical knowledge, (ii.) focusing attention such as alerts, and (iii.) patient-specific consultation in order to receive recommendations and customised information on a single patient . More than twenty years later Wright et al.  studied and summarised CDS taxonomies and categorised them into the following six types: (i.) “medication dosing support”, (ii.) “order facilitators”, (iii.) “point-of-care alerts/reminders”, (iv.) “relevant information display”, (v.) “expert systems”, and (vi.) “workflow support” (see table 1). Furthermore, these authors discriminated front-end and back-end capabilities in CDS. Front-end capabilities in CDS are described as “intervention types available to end-users created using specific clinical knowledge bases and application logic” such as drug-drug interaction checks, whereas back-end capabilities are summarised as “discrete system capabilities such as alert triggers, available data input elements, and end-user notification methods” .
Wright et al.  described “medication dosing support” as tools that support drug order decisions such as choosing a dose while adjusting it to the patient’s renal and hepatic functions. “Order facilitators” support the ordering physician in offering templates with order sets, such as for the diagnosis and treatment of myocardial infarctions or common daily drug orders. “Point-of-care alerts/reminders” may show a drug-drug interaction, remind the ordering physician of a due HbA1c check in a diabetic patient or alert health staff of critical laboratory values. “Relevant information display” may show patient specific information such as showing potassium levels when ordering digoxin, displaying relevant information such as renal and hepatic function as well as body surface area when ordering chemotherapy and sporting cost of drugs and laboratory tests. “Expert systems” support physicians by offering complex decision support combining patient characteristics with other electronically available data. Examples are differential diagnostic suggestions, antibiotic therapy suggestions and prognostic models. “Workflow support” encompasses tools such as process templates such as for patients being transferred to the intensive care unit or medication reconciliation functions .
|Table 1: Overview of CDS types and functions, adapted from Wright et al. .|
|1. “Medication Dosing support”||Suggests routine dosages as well as dose reductions in patients with loss of renal function.|
|2. “Order facilitators”||Template for admission orders in pneumonia and order sets.|
|3. “Point-of-care alerts/reminders”||Shows drug-drug interactions, warns against too high levels of potassium, and reminds of intravenous to oral switch of antibiotics.|
|4. “Relevant information display”||Offers potassium level when ordering digoxin, renal and hepatic functions display, and body surface when ordering chemotherapy.|
|5. “Expert systems”||Diagnostic and treatment decision support and planning, e.g. suggestion of an antibiotic therapy in consideration of blood culture results and local bacterial resistance patterns.|
|6. “Workflow support”||Medication reconciliation of current drug therapies in patients transferred, admitted or discharged.|
Impact of CDS
Excellent decisions due to CDS are expected to lead to higher patient safety with better treatment quality, less adverse events and reduced costs.
There are many different applications that support clinical physician decisions and a few have been found to improve outcomes. For instance, a recent study by Bourgeois et al. presented a CDS intervention for acute respiratory illnesses in children which reduced antibiotic use, although the net impact was modest . Furthermore, CDS may improve patient safety and especially reduce serious medication errors . This is of particular importance, since three out of four adverse drug events (ADEs) have been shown to be preventable, and ADEs are common in tertiary referral hospitals as well as in community hospitals . A review by Wolfstadt et al. mentions that 50% of the identified studies showed that CPOE combined with CDS reduces the incidence of ADEs significantly . Similar results were published by Kaushal et al. showing that CDS can reduce medication error rates substantially . In a survey of six community hospitals in the greater Boston area, 9 out of 10 adverse drug events were preventable and all of these could potentially have been prevented with a CDS system including renal dose checking . Furthermore, recommended therapies are being followed, if the information is delivered at the point of care and no physician response is needed .
There is good evidence that CDS improves performance around prevention . As an example, CDS systems that alert physicians regarding the need for venous thromboembolism prophylaxis have proven to be very successful. The study by Kucher at al. showed a significant reduction of symptomatic and asymptomatic deep-vein thrombosis due to CDS alerts . Others found that (i.) electronic thromboprophylaxis reminders have a sustained effect [22, 23], (ii.) reminders after both admissions and transfers have a significant impact on the awareness of thromboembolism prevention , and (iii.) the timing of CDS needs to be addressed in order to thoroughly evaluate such interventions [22, 24].
Alerts should provide knowledge gain without too many workflow interruptions. Otherwise they may cause “alert fatigue” (cf. next chapter),regarding drug-drug interactions for example , without reaching any patient safety goals . Wipfli and Lovis discussed points to consider designing and installing alerts .
Some aspects of CDS impact need additional assessment. There is still insufficient data on the influence of CDS on management of patients with multiple chronic diseases, clinician workload, length of stay of hospitalised patients, mortality and economic impact, warranting further studies in these and other areas [8, 28].
The Institute of Medicine’s book “To Err is Human”  advocated the use of CPOE in combination with CDS to increase patient safety. However, it is clear that health information technology can also create new safety issues and some have expressed doubts about whether CDS improves patient outcomes [30–32] – we believe that it does, but its impact varies substantially by content domain, and it does not necessarily produce improvements in any specific area.
Koppel et al.  identified and observed 22 situations where CPOE facilitated medication errors, such as misleading display of dose ranges. As a consequence the authors suggested to (i.) emphasise workflow more than CPOE in reducing medication risks, (ii.) evaluate the implemented CPOE system, (iii.) fix CPOE problems as soon as possible, (iv.) consider and investigate various causes of medication errors, and (v.) progressively improve quality while “recognising that all changes generate new error risks” .
An article published by Han et al.  reported a significant increase in the mortality of children after a commercial CPOE system had been implemented (adjusted odds ratio 3.28). In essence, the authors explained the increased mortality by delayed administration of critical medications due to mal-performance of the CPOE system – which apparently required an additional physician solely entering orders, led to complicated communication between health professionals, and decreased the time of physicians and nurses at the bedside .
That publication raised a number of comments: Some readers acknowledged the importance of this contribution to the literature in the field, but also, they recommended to carefully plan CPOE and CDS implementations as well as thoroughly examine systems and functions only in pilot wards, before stepwise introducing them [31, 35, 36]. One group highlighted the reported policy changes (removal of crucial drugs from location of care, interdiction of prescribing before patient’s arrival, pharmacy received orders not before approval by a nurse) contributing to critical administration delays, which were independent of the technology . Changing the policies during or shortly before a CPOE implementation process should be avoided . However, the reportedly inadequate performance of the network and CPOE system, and the missed preparation of electronic order sets have been criticised [35, 37]. The latter was one of the most serious omissions in the view of Jacobs et al. (38), and this group also emphasised that – as long as the pharmacist accessed the electronic patient chart – other health professionals were “locked out,”  in that they were unable to enter additional orders.
Furthermore, some commentators highlighted that the implementation of computerisation in the clinical setting is complex and inevitably changes the workflows [35, 36]. However, it is crucial that health professionals are always able to care for the patient, including the possibility of using paper orders [35, 36]. It has been stated that it is unreasonable to implement CPOE in only six days [31, 35] as described . In contrast, such implementations have been estimated to require from 1 to 3 years [36, 39]. Finally, Han et al. , among many others including the Institute of Medicine, suggested establishing an independent board that ensures the safety of health information technology (HIT), similar to safety arrangements in the aviation industry , and in fact, such a group is now being set up in the U.S.
Another group  observed and interviewed health professionals of five U.S. hospitals and identified a large number of issues due to CPOE and particularly CDS, including additional workload and too many false positive alerts. As a consequence, Weiner et al.  introduced the term “e-iatrogenesis”.
False positive alerts increase the risk of alert fatigue . This psychological phenomenon results from high numbers of clinically insignificant alerts that consume time and mental energy, potentially causing overrides of highly important notifications . However, by increasing the specificity of electronic notifications, the risk of alert fatigue will be minimised. For instance, Hulgan et al.  presented a sophisticated algorithm that detected drip-fed patients – before electronically suggesting the oral route for intravenous quinolones.
Strom et al.  evaluated interruptive alerts in a randomised controlled trial: The aim of the CDS intervention was to avoid a potentially harmful drug-drug interaction between warfarin and trimethoprim-sulfamethoxazole. It was possible to override these hard-stop alerts (i) by entering the indication would be Pneumocystis carinii pneumonia or (ii) by calling the pharmacy. On the one hand, the intervention had a significant impact on prescription behaviour, but on the other hand, the institutional review board terminated the study early because of inacceptable treatment delays in four patients due to the interruptive alerts. A high proportion of all known drug interactions are of low clinical significance . Not so the interaction between warfarin and trimethoprim-sulfamethoxazole, which increases the bleeding risk substantially . Strom et al.  implemented a strong CDS intervention, though, regardless of whether the patients were already stabilised on warfarin (adding trimethoprim-sulfamethoxazole increases bleeding risk), or whether they already received trimethoprim-sulfamethoxazole and warfarin administration was initiated (little risk for bleeding) . Horn concluded that “a reminder to appropriately monitor the patient’s INR would have mitigated the risk of the interaction while avoiding the unintended consequences” . However, a hard-stop alert may not be the best option to avoid the outlined drug-drug interaction, because there are patients who urgently need both drugs – whereas absolute contraindications (e.g. isotretinoin during pregnancy) warrant strong interventions .
Implementation of CDS: points to consider
Computerised CDS has been investigated for several decades [49, 50]. The trend away from the “Greek oracle” approach of diagnostic decision support systems to a more supporting kind of software was accompanied by improving hardware and philosophical changes regarding the human-computer interaction (the physician outgrew the role of a passive observer) . More recently, it has been realised that the way CDS is implemented and presented is critical for a system to succeed (see table 2), and that time consumption is a major factor determining failure .
Kawamoto et al.  analysed publications on electronic and non-electronic decision support systems and presented four CDS features that contributed to improved clinical practice: (i) “provide decision support automatically as part of clinician workflow”, (ii) “deliver decision support at the time and location of decision making”, (iii) “provide actionable recommendations”, and (iv) “use a computer to generate the decision support”.
Notifications should be specific and fit the context when displayed, in order to minimise overriding . However, besides patient-specificity, physician-related factors should also be considered while designing interventions , and the understanding of human factors have been recognised to be important to the acceptance of CDS . Software needs to be tailored to suit the users of different target groups among health professionals .
There is a need for governance when implementing and employing CDS systems. Wright et al.  offer a comprehensive summary with six recommendations outlined in table 3. It is important to realise the impact CDS may have on clinical practice as well as surrounding clinical expert systems. Accordingly, a governance concept followed up on by a group of specialists in clinical as well as technical aspects is warranted. Therefore, Eppenga et al. suggest evaluating CDS systems repeatedly, in order to review, adjust and improve algorithms .
|Table 2: Summary of the 10 commandments for effective clinical decision support by Bates et al. .|
|1||“Speed Is Everything”||Speed is the most important feature of a clinical information system.|
|2||“Anticipate Needs and Deliver in Real Time”||Clinical decision support should anticipate needs, e.g. suggest dose adjustments when the kidney function worsens.|
|3||“Fit into the User’s Workflow”||CDS should be integrated with practice, e.g. displaying prescription recommendations right at the time when the physician is in the process of ordering the specific drug.|
|4||“Little Things Can Make a Big Difference”||The usability of a system is crucial. CDS should be designed to allow for intuitively working with the system.|
|5||“Recognize that Physicians Will Strongly Resist Stopping”||Suggestions not to carry out a specific action are frequently ignored by users. Offering alternatives can help to mitigate this problem, however, monitoring the overrides and face-to-face conversations may be required.|
|6||“Changing Direction Is Easier than Stopping”||Systems can be designed to lead users in the right direction. For instance, if free text entries hamper CDS functions, the system should display check boxes providing likely choices, in addition to the free text field.|
|7||“Simple Interventions Work Best”||Comprehensive guidelines usually don't fit on a single screen. However, computerised guidelines are more likely to be adopted in routine practice if they display the key points only, on a single screen.|
|8||“Ask for Additional Information Only When You Really Need It”||CDS has been shown to lose effectiveness if the computer demands additional data input.|
|9||“Monitor Impact, Get Feedback, and Respond”||Over-alerting increases the risk of alert fatigue and important notifications might be ignored. Therefore, when computerised interventions are implemented they should be evaluated early, corrected and improved.|
|10||“Manage and Maintain Your Knowledge-based Systems”||It is important to keep the front-end (user interface of CDS) and the back-end (algorithms and knowledge base) up to date. This process involves the consideration of new medical knowledge.|
|Table 3: Governance recommendations for CDS, according to Wright et al. .|
|1. “Prioritise the order of development for new CDS and delegate content development to specialised working groups”|
|2. “Consider the potential impact of new CDS on existing clinical information systems”|
|3. “Develop tools to monitor CDS inventory, facilitate updates, and ensure continuity”|
|4. “Implement procedures for assessing the impact of changes and additions to CDS system’s own the system’s own functionality”|
|5. “Provide multiple robust channels for user feedback and the dissemination of systems-related information to end users”|
|6. “Develop tools for ongoing monitoring of CDS interventions“|
Outlook and future research
O’Connor et al.  published an overview and outlook of CDS in diabetes care, and some points discussed in this article are generalisable to other future CDS developments. For instance, CDS should suggest therapeutic options whenever it warns against problems, individual customisation of CDS will play an increasing role, CDS should be designed to save time, and CDS is moving toward patient-centred decision support . In fact, CDS directed not only to health professionals but also to the patient seems to be a powerful approach as concluded in two recent meta-analyses [59, 60]. This is an important result that needs to be considered while developing novel CDS interventions.
Furthermore, O’Connor et al. hope that – within the next 10 years – the compatibility issues across different electronic health records will be solved, CDS implementations will provide “validated transparent clinical algorithms”, CDS will be able to prioritise recommendations, and genetic markers will be part of the input data considered by the systems .
Features identified to be crucial in successful CDS interventions are “provide decision support automatically as part of clinician workflow”, “deliver decision support at the time and location of decision making”, “provide actionable recommendations”, “use a computer to generate the decision support” , and “integration with charting or order entry system, promotion of action rather than inaction, no need for additional clinician data entry, justification of decision support via research evidence, local user involvement, and provision of decision support results to patients as well as providers” . Studies are needed to prospectively investigate the individual role and importance of these features one by one .
It has repeatedly been stated [30–32, 58, 59, 61] that study results, regarding whether CDS actually improves clinical outcomes, are still scarce. The limited power of available studies is an important reason for this knowledge gap [24, 48]. Murphy  proposed cohort studies as a means to investigate the long-term outcomes of CDS on morbidity and mortality. Quality measures and financial incentives might additionally boost the meaningful implementation and effectiveness of CDS .
It is likely that knowledge implemented in CDS tools will be shared more widely in the future, by uploading knowledge to repositories, by disclosing the specifications of algorithms, and by sharing executable modules (cf. www.opencds.org) [48, 62]. The knowledge management of CDS is a novel research field with great promise . It includes the use of specialised management software, considers online tools and online collaborations, the dissemination of CDS knowledge, and knowledge generation by data mining.
Hongsermeier et al.  mention the need for protection of intellectual property invested in CDS in order to foster sharing of CDS knowledge. However, it is difficult to patent intellectual property of the knowledge integrated in CDS. Additionally another aspect is that – while health care providers are considered to be the “learned intermediaries” deciding to follow CDS recommendations or not – the liability principles in terms of CDS and its developers are still not well established . Finally, the policies on premarket review and regulations on CDS are – if not obscure – in progress . A comparison of the CDS regulations in the U.S. and the E.U. has been published by Andersson .
Implementation of customisation regarding alerts for providers and regarding information for patients is complex , however, the user profiles of the target groups should be extended, for example personal options such as ‘‘remind me in one week’’ and ‘‘don’t show this message again’’ could play a role . Researchers should always consider the impact of both the back-end (algorithms and knowledge base) and front-end (user interface) of CDS interventions . Sophisticated algorithms allow for increasing the specificity of notifications [44, 69, 70], and in turn, decrease the risk for alert fatigue . In addition, the design of the human-computer interface should consider human factors principles, and the extent to which it does so will affect performance [54, 62, 71].
CDS systems – whether commercial or home-grown – need to be carefully addressed in an on-going way. There is no doubt that CDS specialists will increasingly be a wanted human resource . Furthermore, the more patient-specific and clinically significant the content of CDS notifications is, the more users will accept and value computerised recommendations. CDS works best when providers are led to make it easy to do the right thing, rather than through approaches in which – after a lot of ordering work is done – critiques are provided at the end. Overall, much of the benefit which is realised from electronic health records comes from the embedded CDS, and it is essential to do a good job with it.
Funding / potential competing interests: Patrick E. Beeler: Dr. Beeler was supported by the Swiss National Science Foundation. David W. Bates: No funding. Balthasar L. Hug: No funding. Patrick E. Beeler: No competing interests stated.
David W. Bates: Dr. Bates is a co-inventor on Patent No. 6029138 held by Brigham and Women’s Hospital on the use of decision support software for medical management, licensed to the Medicalis Corporation. He holds a minority equity position in the privately held company Medicalis which develops web-based decision support for radiology test ordering. He serves on the board for SEA Medical Systems, which makes intravenous pump technology. He is on the clinical advisory board for Zynx, Inc., which develops evidence-based algorithms, and Patient Safety Systems, which provides a set of approaches to help hospitals improve safety. He consults for EarlySense, which makes patient safety monitoring systems. He receives equity and cash compensation from QPID, Inc, a company focused on intelligence systems for electronic health records. Dr. Bates’ financial interests have been reviewed by Brigham and Women’s Hospital and Partners HealthCare in accordance with their institutional policies.
Balthasar L. Hug: No competing interests stated.
Correspondence: Balthasar L. Hug, MD, MBA, MPH, Division of Internal Medicine, 4 Petersgraben, CH-4031 Basel, Switzerland, balthasar.hug[at]usb.ch
1 Wyatt J, Spiegelhalter D. Field trials of medical decision-aids: potential problems and solutions. Proceedings / the Annual Symposium on Computer Application [sic] in Medical Care Symposium on Computer Applications in Medical Care. 1991:3–7.
2 Sim I, Gorman P, Greenes RA, et al. Clinical decision support systems for the practice of evidence-based medicine. Journal of the American Medical Informatics Association: JAMIA. 2001;8(6):527–34.
3 Osheroff JA, Teich JM, Middleton B, Steen EB, Wright A, Detmer DE. A roadmap for national action on clinical decision support. Journal of the American Medical Informatics Association: JAMIA. 2007;14(2):141–5.
4 Mylopoulos J, Chaudhri V, Plexousakis D, Shrufi A, Topologlou T. Building knowledge base management systems. The VLDB Journal. 1996;5:238–63.
5 Shortliffe EH. Computer programs to support clinical decision making. JAMA: the journal of the American Medical Association. 1987;258(1):61–6.
6 Bates DW, Leape LL, Cullen DJ, et al. Effect of computerized physician order entry and a team intervention on prevention of serious medication errors. JAMA: the journal of the American Medical Association. 1998;280(15):1311–6.
7 Berner ES. Clinical Decision Support Systems: State of the Art. http://healthit.ahrq.gov/sites/default/files/docs/page/09–0069–EF_1.pdf, accessed 4 May 2014.
8 Bright TJ, Wong A, Dhurjati R, et al. Effect of Clinical Decision-Support SystemsA Systematic Review. Ann Intern Med. 2012;157(1):29–43.
9 Kucher N, Koo S, Quiroz R, et al. Electronic alerts to prevent venous thromboembolism among hospitalized patients. N Engl J Med. 2005;352(10):969–77.
10 Rood E, Bosman RJ, van der Spoel JI, Taylor P, Zandstra DF. Use of a computerized guideline for glucose regulation in the intensive care unit improved both guideline adherence and glucose regulation. Journal of the American Medical Informatics Association: JAMIA. 2005;12(2):172–80.
11 Dexter PR, Perkins S, Overhage JM, Maharry K, Kohler RB, McDonald CJ. A computerized reminder system to increase the use of preventive care for hospitalized patients. N Engl J Med. 2001;345(13):965–70.
12 Harinstein LM, Kane-Gill SL, Smithburger PL, Culley CM, Reddy VK, Seybert AL. Use of an abnormal laboratory value-drug combination alert to detect drug-induced thrombocytopenia in critically Ill patients. J Crit Care. 2012;27(3):242–9.
13 Randolph AG, Haynes RB, Wyatt JC, Cook DJ, Guyatt GH. Users’ Guides to the Medical Literature: XVIII. How to use an article evaluating the clinical impact of a computer-based clinical decision support system. JAMA: the journal of the American Medical Association. 1999;282(1):67–74.
14 Pryor TA. Development of decision support systems. Int J Clin Monit Comput. 1990;7(3):137–46.
15 Wright A, Sittig DF, Ash JS, et al. Development and evaluation of a comprehensive clinical decision support taxonomy: comparison of front-end tools in commercial and internally developed electronic health record systems. Journal of the American Medical Informatics Association: JAMIA. 2011;18(3):232–42.
16 Bourgeois FC, Linder J, Johnson SA, Co JP, Fiskio J, Ferris TG. Impact of a computerized template on antibiotic prescribing for acute respiratory infections in children and adolescents. Clinical pediatrics. 2010;49(10):976–83.
17 Bates DW, Gawande AA. Improving safety with information technology. N Engl J Med. 2003;348(25):2526–34.
18 Hug BL, Witkowski DJ, Sox CM, et al. Adverse Drug Event Rates in Six Community Hospitals and the Potential Impact of Computerized Physician Order Entry for Prevention. J Gen Intern Med. 2009 Nov 6.
19 Wolfstadt JI, Gurwitz JH, Field TS, et al. The effect of computerized physician order entry with clinical decision support on the rates of adverse drug events: a systematic review. J Gen Intern Med. 2008;23(4):451–8.
20 Kaushal R, Shojania KG, Bates DW. Effects of computerized physician order entry and clinical decision support systems on medication safety: a systematic review. Arch Intern Med. 2003;163(12):1409–16.
21 Hug BL, Witkowski DJ, Sox CM, et al. Occurrence of adverse, often preventable, events in community hospitals involving nephrotoxic drugs or those excreted by the kidney. Kidney Int. 2009;76(11):1192–8.
22 Beeler PE, Kucher N, Blaser J. Sustained impact of electronic alerts on rate of prophylaxis against venous thromboembolism. Thromb Haemost. 2011;106(4):734–8.
23 Lecumberri R, Marques M, Diaz-Navarlaz MT, et al. Maintained effectiveness of an electronic alert system to prevent venous thromboembolism among hospitalized patients. Thromb Haemost. 2008;100(4):699–704.
24 Beeler PE, Eschmann E, Schumacher A, Studt JD, Amann-Vesti B, Blaser J. Impact of electronic reminders on venous thromboprophylaxis after admissions and transfers. J Am Med Inform Assoc. 2014 Mar 26.
25 van der Sijs H, Aarts J, Vulto A, Berg M. Overriding of drug safety alerts in computerized physician order entry. Journal of the American Medical Informatics Association: JAMIA. 2006;13(2):138–47.
26 Phansalkar S, van der Sijs H, Tucker AD, et al. Drug–drug interactions that should be non-interruptive in order to reduce alert fatigue in electronic health records. J Am Med Inform Assoc. 2012 September 25, 2012.
27 Wipfli R, Lovis C. Alerts in clinical information systems: building frameworks and prototypes. Stud Health Technol Inform. 2010;155:163–9.
28 Payne TH, Bates DW, Berner ES, et al. Healthcare information technology and economics. J Am Med Inform Assoc. 2012 July 10, 2012.
29 Kohn L. To err is human: an interview with the Institute of Medicine’s Linda Kohn. Jt Comm J Qual Improv. 2000;26(4):227–34.
30 Berger RG, Kichak JP. Computerized physician order entry: helpful or harmful? Journal of the American Medical Informatics Association: JAMIA. 2004;11(2):100–3.
31 Gesteland PH, Nebeker JR, Gardner RM. These are the technologies that try men's souls: common-sense health information technology. Pediatrics. 2006;117(1):216–7.
32 Hemens BJ, Holbrook A, Tonkin M, et al. Computerized clinical decision support systems for drug prescribing and management: a decision-maker-researcher partnership systematic review. Implement Sci. 2011;6:89.
33 Koppel R, Metlay JP, Cohen A, et al. Role of computerized physician order entry systems in facilitating medication errors. JAMA. 2005;293(10):1197–203.
34 Han YY, Carcillo JA, Venkataraman ST, et al. Unexpected increased mortality after implementation of a commercially sold computerized physician order entry system. Pediatrics. 2005;116(6):1506–12.
35 Rosenbloom ST, Harrell FE, Jr., Lehmann CU, Schneider JH, Spooner SA, Johnson KB. Perceived increase in mortality after process and policy changes implemented with computerized physician order entry. Pediatrics. 2006;117(4):1452–5; author reply 5–6.
36 Sittig DF, Ash JS, Zhang J, Osheroff JA, Shabot MM. Lessons from “Unexpected increased mortality after implementation of a commercially sold computerized physician order entry system”. Pediatrics. 2006;118(2):797–801.
37 Longhurst C, Sharek P, Hahn J, Sullivan J, Classen D. Perceived increase in mortality after process and policy changes implemented with computerized physician order entry. Pediatrics. 2006;117(4):1450–1; author reply 5–6.
38 Jacobs BR, Brilli RJ, Hart KW. Perceived increase in mortality after process and policy changes implemented with computerized physician order entry. Pediatrics. 2006;117(4):1451–2; author reply 5–6.
39 Payne TH, Hoey PJ, Nichol P, Lovis C. Preparation and use of preconstructed orders, order sets, and order menus in a computerized provider order entry system. Journal of the American Medical Informatics Association: JAMIA. 2003;10(4):322–9.
40 Han YY, Carcillo JA, Venkataraman ST, et al. In Reply. Pediatrics. 2006;117(4):1455–6.
41 Campbell EM, Sittig DF, Ash JS, Guappone KP, Dykstra RH. Types of unintended consequences related to computerized provider order entry. Journal of the American Medical Informatics Association: JAMIA. 2006;13(5):547–56.
42 Weiner JP, Kfuri T, Chan K, Fowles JB. “e-Iatrogenesis”: the most critical unintended consequence of CPOE and other HIT. Journal of the American Medical Informatics Association: JAMIA. 2007;14(3):387–8; discussion 9.
43 Peterson JF, Bates DW. Preventable medication errors: identifying and eliminating serious drug interactions. J Am Pharm Assoc (Wash). 2001;41(2):159–60.
44 Hulgan T, Rosenbloom ST, Hargrove F, et al. Oral quinolones in hospitalized patients: an evaluation of a computerized decision support intervention. J Intern Med. 2004;256(4):349–57.
45 Strom BL, Schinnar R, Aberra F, et al. Unintended effects of a computerized physician order entry nearly hard-stop alert to prevent a drug interaction: a randomized controlled trial. Arch Intern Med. 2010;170(17):1578–83.
46 Glasheen JJ, Fugit RV, Prochazka AV. The risk of overanticoagulation with antibiotic use in outpatients on stable warfarin regimens. J Gen Intern Med. 2005;20(7):653–6.
47 Horn JR. Hard-stops for drug interactions. Arch Intern Med. 2011;171(7):706; author reply -7.
48 Bates DW. CPOE and clinical decision support in hospitals: getting the benefits: comment on “Unintended effects of a computerized physician order entry nearly hard-stop alert to prevent a drug interaction”. Arch Intern Med. 2010;170(17):1583–4.
49 Morrell J, Podlone M, Cohen SN. Receptivity of physicians in a teaching hospital to computerized drug interaction monitoring and reporting system. Med Care. 1977;15(1):68–78.
50 Miller RA. Medical diagnostic decision support systems – past, present, and future: a threaded bibliography and brief commentary. Journal of the American Medical Informatics Association: JAMIA. 1994;1(1):8–27.
51 Bates DW, Kuperman GJ, Wang S, et al. Ten commandments for effective clinical decision support: making the practice of evidence-based medicine a reality. Journal of the American Medical Informatics Association: JAMIA. 2003;10(6):523–30.
52 Kawamoto K, Houlihan CA, Balas EA, Lobach DF. Improving clinical practice using clinical decision support systems: a systematic review of trials to identify features critical to success. BMJ. 2005;330(7494):765.
53 Eschmann E, Beeler PE, Kaplan V, Schneemann M, Zund G, Blaser J. Patient- and physician-related risk factors for hyperkalaemia in potassium-increasing drug-drug interactions. Eur J Clin Pharmacol. 2014;70(2):215–23.
54 Phansalkar S, Zachariah M, Seidling HM, Mendes C, Volk L, Bates DW. Evaluation of medication alerts in electronic health records for compliance with human factors principles. Journal of the American Medical Informatics Association: JAMIA. 2014 Apr 29.
55 Beeler PE, Eschmann E, Rosen C, Blaser J. Use of an on-demand drug-drug interaction checker by prescribers and consultants: a retrospective analysis in a Swiss teaching hospital. Drug safety: an international journal of medical toxicology and drug experience. 2013;36(6):427–34.
56 Wright A, Sittig DF, Ash JS, et al. Governance for clinical decision support: case studies and recommended practices from leading institutions. J Am Med Inform Assoc. 2011;18(2):187–94.
57 Eppenga WL, Derijks HJ, Conemans JMH, Hermens WAJJ, Wensing M, De Smet PAGM. Comparison of a basic and an advanced pharmacotherapy-related clinical decision support system in a hospital care setting in the Netherlands. J Am Med Inform Assoc. 2012;19(1):66–71.
58 O’Connor PJ, Desai JR, Butler JC, Kharbanda EO, Sperl-Hillen JM. Current status and future prospects for electronic point-of-care clinical decision support in diabetes care. Curr Diab Rep. 2013;13(2):172–6.
59 Lobach D, Sanders GD, Bright TJ, et al. Enabling health care decisionmaking through clinical decision support and knowledge management. Evidence report/technology assessment. 2012(203):1–784.
60 Roshanov PS, Fernandes N, Wilczynski JM, et al. Features of effective computerised clinical decision support systems: meta-regression of 162 randomised trials. BMJ. 2013;346:f657.
61 Murphy EV. Clinical Decision Support: Effectiveness in Improving Quality Processes and Clinical Outcomes and Factors That May Influence Success. Yale J Biol Med. 2014;87(2):187–97.
62 Sittig DF, Wright A, Osheroff JA, et al. Grand challenges in clinical decision support. J Biomed Inform. 2008;41(2):387–92.
63 Sittig DF, Wright A, Simonaitis L, et al. The state of the art in clinical knowledge management: an inventory of tools and techniques. Int J Med Inform. 2010;79(1):44–57.
64 Hongsermeier T, Maviglia S, Tsurikova L, et al. A legal framework to enable sharing of Clinical Decision Support knowledge and services across institutional boundaries. AMIA Annual Symposium proceedings / AMIA Symposium AMIA Symposium. 2011;2011:925–33.
65 Kesselheim AS, Cresswell K, Phansalkar S, Bates DW, Sheikh A. Clinical decision support systems could be modified to reduce “alert fatigue” while still minimizing the risk of litigation. Health affairs. 2011;30(12):2310–7.
66 Thompson BM, Brooke MJ. FDA's approach to clinical decision support software: A brief summary. http://www.fiercehealthit.com/special-reports/fdas-approach-clinical-decision-support-software-brief-summary, accessed 28 August 2014.
67 Andersson C. Comparing EU and US Approaches to Regulating Clinical Decision Support Software. http://www.emdt.co.uk/article/comparing-eu-and-us-approaches-regulating-clinical-decision-support-software-brief-summary-e, accessed 28 August 2014.
68 Bottiger Y, Laine K, Andersson ML, et al. SFINX-a drug-drug interaction database designed for clinical decision support systems. Eur J Clin Pharmacol. 2009;65(6):627–33.
69 Seidling HM, Klein U, Schaier M, et al. What, if all alerts were specific – estimating the potential impact on drug interaction alert burden. Int J Med Inform. 2014;83(4):285–91.
70 Seidling HM, Schmitt SP, Bruckner T, et al. Patient-specific electronic decision support reduces prescription of excessive doses. Quality & safety in health care. 2010;19(5):e15.
71 Coleman JJ, van der Sijs H, Haefeli WE, et al. On the alert: future priorities for alerts in clinical decision support for computerized physician order entry identified from a European workshop. BMC medical informatics and decision making. 2013;13:111.
Published under the copyright license
“Attribution – Non-Commercial – NoDerivatives 4.0”.
No commercial reuse without permission.