Assessment of three frequently used blood glucose monitoring devices in clinical routine

BACKGROUND: Self-monitoring of blood glucose plays an important role in the management of diabetes and has been shown to improve metabolic control. The use of blood glucose meters in clinical practice requires sufficient reliability to allow adequate treatment. Direct comparison of different blood glucose meters in clinical practice, independent of the manufactures is scarce. We, therefore, aimed to evaluate three frequently used blood glucose meters in daily clinical practice. METHODS: Capillary blood glucose was measured simultaneous using the following glucose meters: Contour® (Bayer Diabetes Care, Zürich, Switzerland), Accu-Chek® aviva (Roche Diagnostics, Rotkreuz, Switzerland), FreeStyle® lite (Abbott Diabetes Care, Baar, Switzerland). The reference method consisted of the HemoCue® Glucose 201+ System (HemoCue® AB, Ängelholm, Sweden) with plasma conversion. The devices were assessed by comparison of the Mean Absolute Relative Differences (MARD), the Clarke Error Grid Analysis (EGA) and the compliance with the International Organization of Standardization criteria (ISO 15197:2003). RESULTS: Capillary blood samples were obtained from 150 patients. MARD was 10.1 ± 0.65%, 7.0 ± 0.62% and 7.8 ± 0.48% for Contour®, Accu-Chek® and Free-Style®, respectively. EGA showed 99.3% (Contour®), 98.7% (Accu-Chek®) and 100% (Free-Style®) of all measurements in zone A and B (clinically acceptable). The ISO criteria were fulfilled by Accu-Chek® (95.3%) and FreeStyle® (96%), but not by Contour® (92%). CONCLUSIONS: In the present study the three glucose meters provided good agreement with the reference and reliable results in daily clinical routine. Overall, the FreeStyle® and Accu-Chek® device slightly outperformed the Contour® device.


Introduction
Self measurement of blood glucose (SMBG) is an essential element in the treatment of diabetic patients.It provides immediate information on blood glucose (BG) values to patients and health care professionals and enables them to optimise glucose control thereby reducing the risk for complications [1,2].Several studies documented the clinical advantage of SMBG in type 1 diabetes patients [3][4][5][6][7] and there is rising evidence of similar benefits in patients with type 2 diabetes [8][9][10].Owing to the increasing prevalence of diabetic patients [11] and the beneficial effects of blood glucose measurement in this population one can expect SMBG to be even more widely used in the future.This highlights the importance of providing reliable measurements of SMBG-devices used in clinical routine.Accuracy of devices can be assessed with regard to right or wrong therapeutic decision based on a given blood glucose value obtained from an SMBG-device.This is achieved by using the error grid analysis (EGA), which divides a scatter plot comparing reference values and SMBG values into five regions of different clinical significance/interpretation.A more technical and mathematical approach to determine accuracy as well as precision is to calculate the relative deviation compared to an accepted reference method [12].Furthermore the compliance with the requirements determined by the International Organization for Standardization (ISO 15197:2003 [13]) is an accepted measure for the quality of clinically used SMBG devices.Independently whatever approach of analysis is used the underlying principle is a good agreement between the reference and the SMBG under investigation.In clinical practice patients are often confronted with a lack of information on reliability of their blood glucose measurements.Whether the degree of deviation is within an established range is often unclear, since the manufacturers' specification on accuracy and precision is usually based on assessments with so called "golden lots".This includes test strips known to have a particularly high accuracy.Therefore, we prospectively assessed three frequently used glucose meters in clinical practice.Importantly, the corresponding manufactures did not financially support this evaluation.

Study design and patients
This was a prospective single centre study performed at the Diabetes Section of the Inselspital, the University Hospital of Bern, Switzerland between October 2009 and September 2010.The study followed the guidelines of good clinical practice, the Swiss health laws and the ordinance on clinical research.Each participant gave informed consent, and the study was approved by the local ethics committee in Bern, Switzerland.A total of 150 participants attending the outpatient clinic for routine control were enrolled in this clinical study.Inclusion criteria were diabetes mellitus or impaired glucose tolerance according to current guidelines [14] and age of more than 18 years.

Blood glucose measurements
Capillary blood samples were always collected by the same experienced person.This procedure was chosen in order to minimise user dependant pre-analytical bias.The area of measurement was disinfected before pricking and the first drop of blood was removed with a sterile tissue.All the measurements were carried out with one single subsequent drop from the same area.The drop was applied to the stripes of the following four devices in random order: Contour ® (Bayer Diabetes Care, Zürich, Switzerland), Accu-Check ® aviva (Roche Diagnostics, Rotkreuz, Switzerland), Free-Style ® lite (Abbott Diabetes Care, Baar, Switzerland) and the HemoCue ® Glucose 201+ System (HemoCue ® AB, Ängelholm, Sweden), which uses the glucose dehydrogenase method and was defined as the reference method in this study [15].For statistical analysis plasma equivalent glucose readings were used.In case of the HemoCue® we applied plasma conversion by multiplying the values with 1.12.The three devices under investigation directly reported plasma values.The test strips were stored in the original vials at 21-25 °C and 25-50% atmospheric moisture.For coding purpose of the Accu-Chek ® device we used the coding chip delivered with the test strip box; Contour ® and Free-Style ® did not require coding.

Statistics
To assess agreement with the reference the mean absolute relative deviation (MARD, absolute value of the difference between SMBG readings and reference divided by the reference, expressed as percentage) of the three devices was calculated [12].To account for differences according to glucose range, a sub analysis of MARD-values below and above 75 mg/dl (4.16 mmol/l) was performed.Furthermore, agreement between the SMBG values and the reference values was evaluated using Bland-Altman plots displaying the difference between SMBG readings and references versus their mean values [16] [13]).The ISO standard for SMBG devices suggests that at least 95% of the SMBG values should fall within ±20% deviation compared to the reference for glucose values ≥75 mg/dl (4.16 mmol/l) and within ±15 mg/dl (0.83 mmol/l) for values <75 mg/dl (4.16 mmol/l), since in low glucose ranges a small absolute deviation would result in a comparably high relative deviation.Values are expressed as mean ± SEM, unless otherwise specified.Continuous variables were tested for significant differences by two-tailed t tests or repeated measure ANOVA.Bonferroni-correction was applied to correct for multiple comparison.Contingency tables were analysed for statistical significance using the χ 2 or Fisher's exact test.A p-value <0.05 was considered statistically significant.All analysis was performed using Microsoft Excel 2011 and Stata 12.0 (Stata Corporation, College Station, TX, USA).

Principle findings
In this study we evaluated three frequently used SMBGdevices using exclusively commercially available test strips.From a clinical point of view all three devices showed sufficient agreement with the reference method.The mean relative difference (MARD) was 7% (Accu-Chek ® ), 8% (Free-Style ® ) and 10% (Contour ® ), respectively.Only three out of 450 measurements showed clinically significant deviation, which could possibly result in an inadequate therapeutic decision according to the EGA.The Accu-Chek ® and Free-Style ® device, which revealed significantly lower MARD values than the Contour ® device, met the ISO minimum criteria, whereas the Contour ® device was just below the required limit.

Mean absolute relative deviation
In the overall analysis MARD values in our study were comparable with recent studies, corroborating the validity and results of our analysis [19,20].The slightly lower MARD values of Accu-Chek ® , Free-Style ® and Contour ® in the analysis of Tack and colleagues might be explained by the exclusion of certain values or patients not meeting the predefined requirements, whereas in our analysis all obtained values were included.Furthermore they used duplicate measurements, which might additionally improve MARD values [20].Intriguingly MARD values in the glucose range above 75 mg/dl (4.16 mmol/l) were significantly lower compared to the hypoglycaemic range for the Accu-Chek ® and Free-Style ® device, whereas for the Contour ® device no such differences were found.The performance in the range below 75 mg/dl (4.16 mmol/l) was similar for all three devices.These findings suggest a better performance of Accu-Chek ® and Free-Style ® in the euglycaemic and hyperglycaemic ranges compared to low blood glucose levels.

Bland-Altman and error grid analysis
The results obtained from the Bland-Altman plot show that the deviation of all three devices in the low blood glucose ranges mainly results from overestimation of blood glucose values, whereas in the hyperglycaemic range glucose levels tended to be underestimated in relation to the Hemo-Cue as the reference method.This may, therefore, result in an underreporting of hypoglycaemia, which in the case of hypoglycaemia unawareness might have clinical consequences.Nevertheless only three of the 450 measurement values were allocated in Zone D of the EGA, which may result in a wrong therapeutic decision.The remaining values would lead to a similar therapeutic decision according to the EGA.The values in zone A ranged from 94% (Contour ® ) to 96% (Accu-Chek ® and Free-Style ® ), which is comparable to a recent report by Tack and colleagues, who also used commercially available test strips [20].A study performed by Freckman et al. showed somewhat higher values in zone A, which might be explained by the use of supreme test stripe lots [21].Interpreting the results of EGA, it has to be kept in mind that deviations of more than 20% seem to be unacceptable from an analytical point of view, but can clinically be acceptable (see Zone B of EGA), because treatment decisions are not negatively affected by this error.

International Organisation of Standardization criteria (ISO 15197:2003)
Accu-Chek ® and Free-Style ® met the ISO criteria (13) of more than 95% of all readings within a deviation of ±20% in the range above 75 mg/dl (4.16 mmol/l) and ±15 mg/ dl (0.83 mmol/l) in the range below 75 mg/dl (4.16 mmol/ l).In contrast, Contour ® showed only 92% of all readings within the ISO limits.Similar results were obtained in a recent study by Tack and colleagues (20).When applying tighter limits of ±5% in the range above 75 mg/dl (4.16 mmol/l) and ±5 mg/dl (0.28 mmol/l) in the range below 75 mg/dl (4.16 mmol/l) as suggested by the American Diabetes Association [18,22] none of the three devices would comply with the requirements in either ranges.In contrast to the ISO criteria we found significant differences between the three devices, when applying tighter limits.Only the Accu-Chek ® device showed more than 50% of all readings within the limits suggested by the ADA.Since the error grid analysis showed in only 0.7% of all measurements clinical meaningful deviation it remains questionable whether the application of the tight ADA limits is reasonable in routine clinical practice.Nevertheless, the obvious superiority for the Accu-Chek ® device when it comes to tighter limits, suggests the application of this device in case of usage of the SMBG devices in situations where a reliability as high as possible is absolutely mandatory (as for instance when SMBG devices are used for calibration of continuous glucose monitoring systems).

Interpretation of glucose readings and sources of error
Although the overall mean deviation is acceptable (7-10%) and the devices showed a good clinical performance with regard to the EGA, it must be kept in mind, that we found maximal deviations of up to 60%.Therefore, it is important that patients are critical regarding the plausibility of their measurements and repeat the procedure if concerns arise about the correctness of a measurement.Furthermore, thorough instruction of the patients is absolutely mandatory to reduce user dependent sources of error.Besides adherence to the device manual and recommended measurement condition (i.e.temperature and altitude range), patients should wash and dry their hands before blood sampling, wipe off the first drop of blood and store the strips as indicated by the manufacturer.Contamination of the hands with glucose can lead to pseudohyperglycaemia [23], whereas wet or sweaty hands could lead to a dilution effect with false  [12,24].In addition, alternate site testing (forearm) can lead to glucose concentration differences compared to finger prick testing, because of a time lag in case of rapid blood glucose changes [25].

Strength and limitations
The strength of the present study is the independency from the manufacturers and thus reduction of bias by so called golden lots.Furthermore all measurements were performed with capillary blood from a single finger prick and analysed in a standardised procedure.All blood samples were collected by the same experienced person.Nevertheless, we have to acknowledge several limitations: (i) Since the present study was not industry funded we could only include a comparatively small number of patients.(ii) As reference method we used the HemoCue ® Glucose 201+ system with conversion to plasma levels.This decision was based on a recent report that compared HemoCue ® with the Yellow Springs Instrument (YSI 2300 STAT; YSI).This study suggests that HemoCue ® and the YSI 2300 STAT, which has been used as reference in previous reports assessing SMBG devices in a clinical routine setting [20,21], can be used interchangeably for research and clinical purposes [15].Furthermore, using HemoCue ® offered the possibility of performing all measurements immediately after finger pricking and with the same capillary drop of blood.(iii) Reliability of the measurements is particularly important for the assessment of blood glucose values in the lower range.
This study provides only limited data about the performance of the glucose meters in the lower range.We can, therefore, not exclude the fact that the results might be slightly different with a higher incidence of blood glucose values in the hypoglycaemic range.(iv) Since an experienced person performed the blood sampling the present data reflect mainly the technical performance of the devices.Although, if the measurements are carried out according to the manufacture's recommendation -and usually the patients are instructed to do so -these data are also valid in the clinical setting.

Conclusion
From a clinical point of view the three evaluated glucose meters provide sufficient agreement with the reference to warrant a save diabetic therapy when test strips and devices are obtained through regular supply channels.Overall, the Accu-Chek ® and Free-Style ® tended to slightly outperform the Contour ® device.Provided that the measurement is carried out correctly, patients can rely on the results of all three devices, since they assure correct clinical decision making.Nevertheless patients should keep in mind that a mean deviation of about 7-10% must be expected.

Figure 1 Error
Figure 1Error Grid Analysis to evaluate clinical significance of measuring inaccuracies of the Contour ® (A), the Accu-Chek ® (B) and the Free-Style ® (C) device by plotting the SMBG readings against the reference (HemoCue ® Glucose 201+ System)

Figure 2 Bland-
Figure 2 Bland-Altman diagram depicting deviation between SMBG readings of the Contour ® (A), the Accu-Chek ® (B), the Free-Style ® (C) and the reference values (HemoCue ® Glucose 201+ System) plotted against their means.The solid line represents the linear fit model of the corresponding data points.Horizontal lines show the mean difference between SMBG readings and reference values (solid), as well as the corresponding 95%-limits of agreement (i.e.±2 SD; dashed lines).

Figure 1 Error
Figure 1Error Grid Analysis to evaluate clinical significance of measuring inaccuracies of the Contour ® (A), the Accu-Chek ® (B) and the Free-Style ® (C) device by plotting the SMBG readings against the reference (HemoCue ® Glucose 201+ System)

Figure 2 Bland-
Figure 2Bland-Altman diagram depicting deviation between SMBG readings of the Contour ® (A), the Accu-Chek ® (B), the Free-Style ® (C) and the reference values (HemoCue ® Glucose 201+ System) plotted against their means.The solid line represents the linear fit model of the corresponding data points.Horizontal lines show the mean difference between SMBG readings and reference values (solid), as well as the corresponding 95%-limits of agreement (i.e.±2 SD; dashed lines).

Table 1 :
Summary of Clarke Error Grid Analysis Data are number of SMBG readings in each zone (% of total readings).Zone A, glucose levels within ±20% of those found with the reference method leading to clinically correct treatment decisions; Zone B, deviation between device under investigation and reference method is more than 20% but without serious clinical consequences; Zone C, SMBG values would result in unnecessary treatment (overcorrecting acceptable glucose levels); Zone D represents a failure to detect and treat abnormal blood glucose levels; Zone E is an erroneous treatment zone, in which treatment decisions would be unsuitable (confusing hypo-and hyperglycemia).

Table 2 :
Summary of SMBG readings within different accuracy ranges Data are number of SMBG readings within pre-specified deviation limits in the range below and above 75 mg/dl (4.16 mmol/l) and number of SMBG readings meeting the ISO 15197:2003 accuracy criteria (defined as ±15 mg/dl in the glucose range <75 mg/dl and ±20% in the glucose range ≥75 mg/dl) and the ADA criteria (defined as ±5 mg/ dl in the glucose range <75 mg/dl and ±5% in the glucose range ≥75 mg/dl).Swiss Medical Weekly • PDF of the online version • www.smw.ch PDF of the online version • www.smw.chlow values.There are several additional factors leading to under-or overestimation of blood glucose values including incorrect storage of the strips, temperature, altitude, medication, haematocrit or sample volume