Diagnostic diversity – an indicator of institutional and regional healthcare quality

Brutsche Martina, Rassouli Franka, Gallion Haraldb, Kalra Sanjayc, Roger Veronique Ld, Baty Florenta a Lung Centre, Cantonal Hospital St Gallen, Switzerland b Department of Medical Coding, Cantonal Hospital St Gallen, Switzerland c Department of Pulmonary and Critical Care Medicine, Mayo Clinic, Rochester, Minnesota, USA d Department of Internal Medicine, Division of Cardiovascular Diseases, Mayo Clinic, Rochester, Minnesota, USA


Introduction
Measuring healthcare quality is essential to optimise the effectiveness of healthcare delivery in a rapidly changing healthcare environment, but is challenging and at times controversial.There is a plethora of quality indicators [1] from self-reported health status [2] to mortality rates in different age groups and conditions that are applied to evaluate care across regions [3] and institutions [4].These measures are affected by a complex interplay between availability of resources, and socioeconomic and healthcare factors [5].Measuring and aligning the contribution of different healthcare determinants is important, albeit challenging [6].The United States-based Aligning Forces for Quality (AF4Q)-program, the largest of its kind worldwide reaching 12.5% of the US population at a cost of 300 Mio US dollars, analysed the effect of institutional alignment and networking on 144 quality outcomes [7].The authors concluded that the AF4Q initiative had less impact than expected.The multitude of unweighted outcomes to be optimised, and the overweight of reimbursement effects on the development of a healthcare landscape [8] may have diluted the impact of the intervention.The publication of annual institutional mortality rates and other quality indicators is imposed on institutions in different industrial countries [9].Such publications seem to stimulate quality improvement activities at the hospital level.However, the effect of public reporting on effectiveness, safety and patient-centeredness remains uncertain [10].Such publications rather become public relations and market factors of a nonvalidated nature in the hands of nonprofessionals [11].
We propose an alternative indicator of healthcare quality based on diagnostic diversity -the diversity of diagnostic codes attributed to inpatient medical care.Diagnostic diversity and its demographic implications have been investigated in a recent study from Schuster and colleagues [12].It should be noticed that the term "diagnostic diversity" has also been used independently in sociological science.In this unrelated context, the term diversity refers to differences in the perception and interpretation of diagnoses in various sociodemographic groups of the population [13].
Adequate disease management strategies and treatments are dependent on the timely identification of an exact diagnosis beforehand [14].Diagnostic imprecision or errors, on the other hand, lead to suboptimal disease management and treatment -especially in rare diseases [15].Diagnostic errors are increasingly recognised as important contributors to preventable morbidity and mortality [16].In the United States, they are estimated to occur in up to 15% of all clinical encounters, affect 12 million adults annually and lead to permanent damage or death in nearly 160,000 patients each year [17].Thus, the earlier and the more precisely the diagnosis is refined, the greater the chance for a better outcome, as well as a better allocation of healthcare resources.
The variety of diagnosed diseases established by an institution or accumulated in a healthcare region can be measured with a diversity index.In a hypothetical, ideal situation where all diseases would be diagnosed correctly, a high degree of diagnostic diversity can be expected (fig 1).The degree of diagnostic diversity that can be achieved by modern medicine is dependent on the diagnostic efforts and quality, and is increasing with new diagnostic developments.Thus, the diagnostic diversity index (DDI) can also be seen as an indicator of diagnostic precision -the higher the degree of diagnostic diversity, the better the diagnostic precision provided.
We compared the DDI of the International Classification of Disease, 10th revision, German Modification (ICD-10-GM) codes in healthcare regions and hospitals from a nationwide database of all hospitalised cases in Switzerland from 2009 to 2015.

Health service landscape in Switzerland
Switzerland has one of the best European health services, according to the European Health Consumer Index 2015 [18].It is divided in 26 cantons of different surface and population size, which represent independent healthcare regions (HCRs).These HCRs are serviced by 292 acute care in-patient institutions.An overview of the Swiss healthcare system is provided in table 1.A total of five university hospitals and several larger cantonal hospitals can be considered as quaternary and tertiary healthcare institutions, respectively.As restrictions to extracantonal healthcare apply, patients are mostly treated within a HCR.If triggered either by a referring physician or patient preference, patients can also be treated outside of their HCR at increased costs, in specific well-defined situations (basic health insurance) or more broadly with private insurance (approximately 15% of the population).

Hospitalisation database
The hospitalisation dataset used in this analysis was provided by the Swiss Federal Office for Statistics.It includes medical information about all hospitalisations in Switzerland since 1998.In this database, the patient information is fully anonymised.No written informed consent was given by patients, who were unidentifiable owing to the anonymisation.The data belong to the Swiss Federal Office for Statistics (Bundesamt für Statistik, Neuchâtel, Switzerland), which provides regulated access to the data for research purposes [19,20].The database included geographical and temporal information (patient's area of residence, HCR [canton] of institution, year and month of hospitalisation, length of hospital stay), as well as age at admission and reason/type of discharge (including death).Hospitalisations within and outside of the area of residency were considered.Thus we could quantify the percentage of hospitalisations taking place outside of the HCR of residency.Unique anonymised institution numbers for the calculation of annual caseload were available.The patients' list of diagnoses included one main diagnosis as well as up to 50 additional diagnoses coded using ICD-10-GM codes [21].The ICD-10-GM coding was uniformly used throughout the study period.For the current publication, the timespan of the analysis was restricted to 2009-2015.Data prior 2009 were not included in order to keep the analysed dataset as homogenous as possible.In addition, only Swiss residents were considered [22].

Statistical analysis
The diagnostic diversity was measured using the Shannon diversity index [23] defined as follows: with D the number of diagnoses (ICD-10-GM codes), ln the natural logarithm and p i the proportional abundance of the i th diagnosis.The Shannon diversity index, originally developed in information theory, is one of the most commonly used diversity indices.It is widely used in ecological science, but has also been previously reported as a promising measurement in health service research [12].In our analysis, the Shannon diversity index accounts for both the abundance and the evenness of the ICD-10-GM at the HCR or institution level.It assumes that all ICD-10-GM codes are equal and therefore treats equally the most abundant as well as the rarest codes.This index has a lower bound of 0 but no upper bound.It increases logarithmically, proportionally to the diversity of codes employed.Numerical simulations were performed to evaluate the effect of misdiagnoses in the DDI.We used nonparametric bootstrapping to resample the ICD-10-GM codes by removing an increasing proportion of ICD-10-GM codes in a prevalence-dependent fashion.More specifically, we used this procedure to randomly remove rare ICD-10-GM codes replacing them by randomly chosen more common codes.This resulted in an artificial decrease of the diagnostic diversity and an increase in the rate of misdiagnoses.We arbitrarily defined the minimum clinically important difference (MCID) as a 10%-misdiagnosis, i.e., 1 in 10 patients seen on the ward leaves the hospital with an imprecise or misdiagnosis.This cut-off was chosen due to its clinical relevance.The diagnostic performance was considered sufficient when an institution/HCR has a DDI above this cut-off.A second cut-off was set at 20% misdiagnosis and used for illustration purposes in some analyses.MCID is level-specific (e.g.HCR, institution) and is different within each ICD-10-GM code category.In a sub-analysis, chapters I to XIV (A-N) of the ICD-10-GM were analysed separately (codes O-Z being used for specific purposes including pregnancy or various injuries).All analyses were done using the R statistical software [24] including the following extension packages: vegan [25], ade4 [26].

ICD-10-GM utilisation and DDI benchmark
Between 2009 and 2015, 9,325,326 hospitalisation cases were reported.A total of 16,435 ICD-10-GM codes were used in the study period.Each hospitalisation case was coded with on average 4.2 ICD-10-GM codes.The 10% most frequently used ICD10-GM codes (1644 codes) were used in 87% (8,094,114 cases) of all hospitalisation cases.The Swiss-wide DDI (ICD-10-GM chapters A-N) was 7.18.The Swiss-wide DDI for individual chapters A to N, as well the MCID-threshold are given in table 2. Chapters A, B (infections causes bacteria, viruses) and E (endocrine diseases) had a relatively low diagnostic diversity (DDI below 4), whereas diseases of the musculoskeletal system and connective tissue had the highest diagnostic diversity (DDI above 5).The effect of misdiagnoses on the DDI was assessed by numerical simulation (fig.2).The relationship between the rates of misdiagnosis on the DDI shows a linear decrease in the lower misclassification rates, and a more abrupt decrease in the higher misclassification rates.The inclusion of 10% of misdiagnoses resulted in a decrease of 0.12 point in the DDI.According to this simulation, a DDI of <7.06 would represent the occurrence of >10% misdiagnoses in the observed case sample (MCID).A second MCID cut-off set at 20% misdiagnoses was also considered.A further drop of 0.14 points in the DDI was observed when applying a MCID 20% threshold.

Type of hospital Number of institutions
Primary hospitals (university hospitals) 5

Specialised clinics 83
Original article

Characterisation of acute in-patient medical care institutions in Switzerland
There was relevant DDI heterogeneity among hospitals in Switzerland, where the DDI was strongly associated with caseload (fig.3).Simulated 10% and 20% misdiagnosis thresholds applied to each of the 14 ICD-10-GM chapters (A-N) allowed Swiss hospitals to be categorised into four distinct groups (fig.4): High-volume hospitals with an all-over comprehensive diagnostic performance (n = 10; university / large cantonal hospitals with an average caseload of 36,000 hospitalisations per year), High-to mid-volume hospitals with a gradient of extensive to relevant basic diagnostic performance in most chapters (n = 88; most cantonal / regional hospitals, with an average caseload of 8330 hospitalisations per year), Low-volume specialised hospitals with a high diagnostic performance in a single chapter (n = 48; with an average caseload of 2,481 hospitalisations per year), and Low-volume hospitals with inadequate diagnostic performance in all chapters (n = 146; with an average caseload of 689 hospitalisations per year).The 146 low-volume hospitals had an estimated misdiagnosis rate of >20% in all chapters and contributed to 8% (710,188 out of 9,325,326) of all hospitalisations in Switzerland.

Residential health care regions and health regions' institutions
The DDI observed in the 26 Swiss HCRs (fig.5, upper right panel) also showed relevant heterogeneity and strong associations with ICD-10-GM code utilisation and caseload, and hence size of the HCR (fig.5, left panels).There was a strong association of DDI and ICD-10-GM-code utilisation (fig.5, inlet within upper-left panel).In smaller HCRs <40% of possible ICD-10-GM codes were utilised by HCR institutions in the observation period.Institutions of larger HCRs with a higher DDI utilised >60% of the ICD-10-GM codes.All but one HCR, with a population >363,522, had a DDI above the 7.06 MCID threshold.In smaller HCRs (population <363,522), the DDI was mostly below the MCID threshold and there was an increasing difference between the DDI observed in the general population and the DDI provided by the HCR institution.The population of such HCRs more often sought healthcare ser-  vices in another HCR.The lower diagnostic performance in small HCRs could partially be compensated for by consumption of health services outside the HCR (fig.5, lower right panel).In smaller HCRs, up to 62% of residents were treated outside their HCR, whereas around 90% of residents of larger HCRs (among those including university hospitals) were exclusively treated within their HCR of residency.This additional diagnostic performance of external healthcare institutions could increase, but in most cases not entirely correct, a reduced diagnostic performance of HCRs concerned (fig.5, lower left panel).

Statement of principal findings
We found significant spatial and institutional heterogeneity of the DDI calculated from ICD-10-GM codes for acute inpatient medical care in Switzerland.Because of the natural diversity of human beings, a higher DDI is more likely to approach the true biological diagnostic diversity of a population (fig. 1 -working hypothesis).Although it is impracticable to verify the correctness of diagnoses of each of the hospital files [16], the fact that some institutions rarely coded low-frequency conditions and, thus, had a low DDI, points to diagnostic limitations.Thus, regions and institutions with less diagnostic diversity are more likely to under-or misdiagnose a significant proportion of patients, missing the chance of an optimal alignment between health condition and treatment -with the potential for worse outcome.

Implications for clinicians and policy makers
There was a significant interaction between caseload and DDI.Acute medical care institutions with an annual caseload of >7217 cases per year had a good chance to have an acceptable DDI and, thus, deliver full diagnostic quality.In Switzerland, however, 50% of 292 hospitals deliver inadequate diagnostic precision (i.e., did not reach MCID in any A-N category).These, mostly small, institutions were treating only 8% of all hospital cases.Extrapolated from the DDI, as many as 6 out of 10 patients are under-or misdiagnosed in certain Swiss low-volume institutions (<10th percentile).The effect of institutional caseload and outcome has already been shown for different diseases and surgical interventions [4].Our findings allow an intuitive alignment between higher diagnostic precision -probably associated with improved outcome -and caseload.
In Switzerland, the cantons represent fairly closed HCRs, limiting reimbursement for healthcare consumption outside of the residential HCR.Although varying significantly between HCRs, in 2015 only 19% of patients profited from cross-cantonal border healthcare [27].HCRs with a population of >363,522 were likely to deliver a sufficient DDI and, thus, a good diagnostic performance for their residents.We could not identify a diagnostic diversity-driven upper limit for the population size of a healthcare region.
In line with the paradigm of the importance of caseload, smaller HCRs, i.e., cantons in Switzerland, may do better to contract with other HCRs rather than invest in their own low-volume higher-care institutions.
It is mandatory for Swiss residents to have basic health insurance coverage giving them access to all levels of healthcare.Approximately 3.2% of the Swiss population receives social support from the state, slightly more in urban agglomerations [28].There are practically no relevant disparities between communities and regions in socioeconomic factors that impact on healthcare utilisation and, thus, on our results.In Switzerland, the ICD-10-GM codes have been the nationwide basis for drug-related group (DRG) reimbursement since 2009, assuring a level of attention to guarantee data quality.On the other hand, the association with reimbursement might impact on the ICD-10-GM coding process [29].As the financial pressure applies to all coding institutions, it is likely that reimbursement-driven coding trends related to specific ICD-10-GM-codes would be mutually balanced out [8].Extensive up-coding would, if it ever occurs, negatively impact diagnostic diversity.The calculation of DDI using the Shannon diversity index proved to be unaffected by socio-culturo-linguistic factors and the numerical caseload in the range observed in this manuscript.
Patients with a severe health condition requiring hospitalisation expect and deserve high-level care competence [30].
Institutions with sufficient caseload and, as a consequence, delivering primary as well as higher-level healthcare, are shown to reach sufficient upfront diagnostic precision.Institutional caseload and size of HCRs should, thus, more consequently be the basis for a rational healthcare planning.The calculation of the DDI from ICD-10-codes is simple and complements the information derived from other quality indicators as it sheds a light on the fitness of the institutional interplay between primary and specialised medical in-patient care.
Furthermore, in some specific conditions, such as hospitalisations due to pneumonia, we found that different subgroups of ICD-10 codes (implying different treatments) were more frequently used in larger hospitals than smaller ones (appendix 1).This, in turn, might impact on the patient management.

Limitations
Limitations in the use of the DDI include the fact that not all institutions have the same professionalism in terms of a coding unit.On the other hand, since the introduction of a new reimbursement system in the Swiss health care, starting on 1 January 2012, and based on DRGs, the level of coding in all Swiss institutions has been uniformly improved.
Another limitation comes from the fact that the institutions in the hospitalisation database were anonymously coded, which made detailed institution-oriented analyses difficult.Also, there might be some more-or-less systematic referral of particular patient groups to larger institutions.
No gold standard for correct diagnosis exists in the hospitalisation database.Misdiagnosis and its link with DDI could only be assessed empirically using numerical simulations.
Finally, in Switzerland only hospitalised cases are systematically recorded in a nation-wide fashion.Therefore, our analysis is limited to hospitalised cases.

Figure 1 :
Figure 1: Diagnostic diversity as a measure of overall diagnostic performance -study hypothesis.It can be expected that the diseases occurring in a population reflect the biological diversity of humans with their diverse genetic make-up and different exposures.Thus, it can be assumed that a high diversity of diagnoses established in a health institution or region is an indicator of good overall diagnostic performance.

Figure 2 :
Figure 2: Diagnostic diversity index (DDI) benchmark -numerically simulated introduction of an increasing number of misdiagnoses and its effect observed on the DDI.The overall Swiss-wide DDI level (before simulation) is represented by a grey dashed line.The simulation of 10% of misdiagnoses -defined as MCID 10% -resulted in a DDI decrease of 0.12 points or 7.06 (blacked dashed line).A second cut-off set at 20% misdiagnosis (MCID 20%) is also represented (lower horizontal blacked dashed line).

Figure 3 :
Figure 3: Association between the diagnostic diversity index and the institutions' caseload.The 10% misdiagnosis threshold (minimum clinically important difference; MCID) is represented by a horizontal dashed line.Local regression (loess) representing a smooth curve through the set of data points is depicted.In general, institutions with an annual caseload >7217 hospitalisation cases (intersection point where the loess regression crosses the 10% misdiagnosis diagnostic diversity index [DDI] threshold) were more likely to have an adequate DDI.

Figure 4 :
Figure 4: Heat map of the diagnostic fitness (DDI) of acute medical care hospitals within 14 ICD-10-GM chapters (A-N).For each chapter the 10% (minimum clinically important difference; MCID) and 20% misdiagnosis thresholds were established by simulation.In the case of a DDI >10% and >20% threshold the respective bar is shown in dark and light grey for each chapter for each of the hospitals.The number of hospitalisation cases per institution is coded in different bullet sizes on the right side (bigger indicating a larger caseload).This view allows four groups of acute medical care hospitals to be distinguished (see text).

Figure 5 :
Figure 5: Diagnostic diversity index (DDI) in Swiss health care regions.The upper left panel displays the relationship between the size of the healthcare region (population) and the diagnostic diversity index generated by healthcare regions' institutions (inset including the representation of the diagnostic diversity as a function of the percentage of ICD-10-GM codes used).The minimum clinically important difference (MCID) of 10% misdiagnosis is represented using a dashed line.The intersection point where the MCID 10% misdiagnosis and the loess regression curve cross defines the cut-off of sufficient diagnostic diversity.In general, health care regions with a population size >363,522 were likely to have an adequate DDI.The upper right panel provides choropleth representation of the DDI generated by the institutions of each healthcare region.The lower left panel shows the relationship between the size of the healthcare regions (population) and the DDI of the region's residents.The lower right panel depicts the relationship between the DDI generated by healthcare region's institutions and the percentage of hospitalisation outside of health care regions.The dashed line represents the DDI threshold of 10% misclassification.The health care regions (Swiss cantons) are coded as follows: AG = Aargau; AI = Appenzell Innerrhoden; AR = Appenzell Ausserrhoden; BE = Bern; BL = Basel-Landschaft; BS = Basel-Stadt; FR = Fribourg; GE = Geneva; GL = Glarus; GR = Graubünden; JU = Jura; LU = Luzern; NE = Neuchatel; NW = Nidwalden; OW = Obwalden; SG = St.Gallen; SH = Schaffhausen; SO = Solothurn; SZ = Schwyz; TG = Turgau; TI = Ticino; UR = Uri; VD = Vaud; VS = Valais; ZG = Zug; ZH = Zürich.

Table 1 :
Categorisation of inpatient institutions in Switzerland in 2015.

Table 2 :
Description of the ICD-10-GM categories.
The Swiss-wide code usage, i.e. the number of times ICD-10-GM codes have been used within each category is provided as absolute number and percentage, as well as the diagnostic diversity index (DDI) and the minimum clinically important difference (MCID) (10% and 20% misdiagnosis rate) are reported.NA = not applicableOriginal articleSwissMed Wkly.2018;148:w14691 Swiss Medical Weekly • PDF of the online version • www.smw.chPublished under the copyright license "Attribution -Non-Commercial -No Derivatives 4.0".No commercial reuse without permission.See http://emh.ch/en/services/permissions.html.