Feasibility, acceptability and construct validity of EQ-5D in systemic sclerosis

DOI: https://doi.org/10.4414/smw.2016.14394

Introduction

Systemic sclerosis is a chronic systemic disease of unknown aetiology characterised by skin fibrosis, Raynaud’s phenomenon and variable involvement of the musculoskeletal, respiratory, cardiovascular, renal and gastrointestinal systems [1]. Patients affected by systemic sclerosis must cope with a progressive disabling condition often characterised by pain, disfigurement, feelings of helplessness and impairment in the ability to perform basic activities [2–4], affecting health-related quality of life (HRQoL) [5–8]. Furthermore, patients with systemic sclerosis often also suffer from psychological impairments, such as depression and anxiety, which further impact on their HRQoL [9, 10].

The assessment of HRQoL is a cornerstone in the management of patients with systemic sclerosis [5]. A recent study of an international cohort demonstrated that patients with systemic sclerosis may perceive that Raynaud’s phenomenon and gastrointestinal involvement have the greatest impact on their HRQoL, differently from physicians, who consider heart and lung involvement to be the most important manifestations [6]. Other symptoms, such as pain and itch [7] or hand disability, should also not be overlooked [4, 11]. The impact of HRQoL on patients affected by systemic sclerosis is severe and it is surprising how often, in clinical practice, the assessment of HRQoL is neglected in these patients. One of the reasons could be related to psychometric or practical limitations of the available instruments. Patient-reported outcome measures (PROMs) aim to inform healthcare professionals and policy decision makers about the patient’s experience with his/her state attributable to having / receiving a diagnosis for a disease, the effects of treatments received, coping with or recovery from the disease and the related HRQoL [5]. Like many other diseases, systemic sclerosis affects individuals with varying levels of education, languages, and socioeconomic and cultural backgrounds. Therefore, the use of PROMs in daily routine should be easy and well accepted, by both patients and their healthcare professionals. Of course, PROMs have to be valid and reliable for the target population.

Broadly speaking, among the instruments available for the assessment of HRQoL we can distinguish generic and disease-specific types [12, 13]. Generic instruments are applicable to a wide range of populations defined by age and sex, type and severity of disease, language, and culture. Disease-specific instruments have been developed for several conditions, with the aim to obtain specific information on the target condition. However, comparison of HRQoL is difficult or impossible between different diseases. The short form-36 (SF-36) [14] and the EQ-5D [15] are among the most frequently used generic instruments worldwide. The SF-36 is a very useful self-administered questionnaire [14]; however, in our experience, completion by the patients and analysis of the data collected are time-consuming in clinical practice. EQ-5D is a generic questionnaire intended to be preference-based, simple and quickly completed, providing a multi-dimensional description and measure of health [15]. It can be used to generate a utility index, which when multiplied by the duration of time corresponding to a given health state generates quality-adjusted life years (QALYs). EQ-5D has been widely applied in many nonrheumatic diseases, as well as in several rheumatic diseases such as rheumatoid arthritis and psoriatic arthritis [16–18]. For economic evaluations, health technology assessments and decision making especially, the use of generic preference-based measures has been recommended by bodies such as the National Institute for Health and Care excellence (https://www.nice.org.uk/process/pmg9/chapter/1-foreword). In Italy, national-regional health authorities (DGR n. VIII / 7856 of 30.07.2008) and national health economic associations have increased their interest in these measures for the assessment of health technologies and interventions [19]. However, because the EQ-5D contains only a few questions, it could be considered simplistic and not capable of fully assessing individuals’ HRQoL. The health assessment questionnaire (HAQ) is another widely used questionnaire, which was designed to measure functional ability or disability in rheumatic diseases [20]. It is quickly completed by the patients; however, by definition, it does not investigate the psychological aspect of HRQoL.

Against this background, our specific objectives were to assess the feasibility, acceptability and construct validity of EQ-5D as compared with SF-36 and HAQ in patients with systemic sclerosis, in a multicentre study.

Methods

Patients and setting

We recruited consecutive patients referring to out-patient or in-patient clinics of three rheumatology centres (G. Pini Institute, Ist. Auxologico Italiano and Niguarda Hospital) in Milan, Italy. The study was approved by the ethics committee, and informed consent was obtained from all patients.

Patients fulfilled both the 1980 American Rheumatism Association criteria [21] and the 2013 classification criteria for systemic sclerosis [22]. The former requires the presence of a major criterion – skin thickening proximal to the metacarpophalangeal joints – and two or more of the following minor criteria: (1) sclerodactyly; (2) digital pitting scars of fingertips or loss of substance of the distal finger pad; and (3) bilateral basilar pulmonary fibrosis. The latter consist of eight items (skin thickening of the fingers of both hands extending proximal to the metacarpophalangeal joints, skin thickening of the fingers [either puffy fingers or sclerodactyly], fingertip lesions [either digital tip ulcers or fingertip pitting scars], teleangiectasia, abnormal nailfold capillaries, pulmonary arterial hypertension and/or interstitial lung disease, Raynaud’s phenomenon, systemic sclerosis-related autoantibodies (any of anti-centromere, anti-topoisomerase I/Scl-70, anti-RNA polymerase III) and requires a total score ≥9 [22]. Antinuclear antibodies (ANAs), anti-centromere and anti-topoisomeraseI-Scl-70 antibodies were tested as described previously [23]. Pulmonary fibrosis was defined on lung computed tomography imaging, pulmonary hypertension from Doppler echocardiography, arrhythmias from 24-hour electrocardiogram Holter monitoring, and oesophageal involvement from barium X-ray examination.

HRQoL instruments

HRQoL was assessed with three generic instruments – EQ-5D, SF-36 and HAQ – which were given to the patients to be self-completed. These questionnaires were chosen for their ability to assess both physical and psychological components of health, and because they are widely used and recognised for their informative power, allowing comparisons of perceived health within and between different clinical conditions and within the general population.

The EQ-5D is a standardised, self-administered questionnaire that consists of two main parts: the first part generates a health profile (EQ-5D profile), based on a descriptive system that defines health in terms of five dimensions: mobility, self-care, usual activities and pain/discomfort are related to the physical component of health; the anxiety/depression domain is related to the mental component of health. Each dimension has three response categories corresponding to “no problem”, “some problems” and “extreme problems / unable to do”. The second part of the questionnaire consists of a visual analogue scale (EQ-5D VAS), which measures the overall HRQoL with a score ranging from 0 (worst imaginable health status) to 100 (best imaginable health state). The health status captured by EQ-5D refers to the day of completion [24].

The SF-36 questionnaire has demonstrated good validity and reliability in systemic sclerosis [25, 26] and, together with the EQ-5D, is considered the gold standard for HRQoL assessment in many diseases [27]. It consists of 36 questions and eight domains based on both physical and emotional components: physical functioning, role-physical and bodily pain are related more to the physical component of health; social functioning, role-emotional and mental health are related more to the mental component of health; energy/vitality and general health relate to both components. The eight domains can be further grouped into two summary scores, one specific to physical health (physical component summary score: PCS) and the other to mental health (mental component summary score: MCS) [28, 29]. The questions refer to the previous 4 weeks. The score for each question is summed and transformed into a 0–100 scale (with 0 and 100 corresponding to the minimum and maximum levels, respectively, of HRQoL).

The HAQ questionnaire is a self-administered generic instrument that measures physical disability in 20 questions focusing on both upper and lower extremities. The questions are collapsed into eight domains, all relating to the physical component of health: dressing/grooming, arising, eating, walking, hygiene, reaching, gripping and activities. The results of the eight subscales and the composite disability score are scored from 0 (no difficulties) to 3 (unable to do). A visual analogue scale (VAS) for pain is used to assess the presence and severity of pain and consists of a horizontal VAS that is scored from 0 (no pain) to 100 (severe pain). The HAQ has a 7 day recall period [20]. It is extensively used to evaluate disability in systemic sclerosis, where it has demonstrated good construct validity and reliability [5, 25, 30]. Additional information regarding analysis and scoring of the HAQ instruments is available at http://aramis.stanford.edu.

Data analysis

The study sample was described according to the main sociodemographic and clinical characteristics, and HRQoL data. In particular, the responses obtained with the EQ-5D profile were converted into a utility index using Italy-specific social tariffs [31]. Also, the HRQoL of systemic sclerosis patients was compared with the HRQoL of the general population based on the results of a recent study assessing the population reference HRQoL in Italy with use of the EQ-5D [32]. Because age and sex are recognised to be generally associated with HRQoL in any population or group of persons, the results of this comparison were stratified according to the following age classes (consistent with the study by Scalone et al. 2015 [32]): 18–35, 36–45, 46–55, 56–65, 66–75 and ≥76 years. As far as sex is concerned, we compared the HRQoL of patients with systemic sclerosis, 94% of whom were female in this study, with that of the female general population. HRQoL was reported as the frequency of respondents declaring some or severe problems in each domain.

All the results were reported as absolute and relative frequency for categorical data, and mean, standard deviation, median, 25th and 75th percentiles (interquartile range, IQR) for continuous variables.

Feasibility was measured in terms of the proportion of patients needing assistance to complete the questionnaire and the percentage of missing data in at least one domain per questionnaire. Acceptability was estimated in terms of patients’ comments on or complaints about the questionnaires.

We examined construct validity of EQ-5D based on a priori hypotheses for convergent and divergent validity. Convergent validity is related to how strongly conceptually equivalent domains of different tools correlate with each other, whereas the opposite is expected between conceptually dissimilar domains. Owing to the characteristics of the variables of the domains, we analysed the correlations using the nonparametric Spearman-rank correlation coefficients between single and summary domains of EQ-5D vs SF-36 and EQ-5D vs HAQ. We were interested in investigating the strength of correlation, if this existed, according to the guidelines provided by Cohen et al. [33]: coefficients from 0.10 to 0.29 were considered low, those between 0.30 and 0.49 were considered moderate, and correlations of 0.50 or above were considered high.

We identified similar and dissimilar domains and indices between the EQ-5D and the SF-36, and between the EQ-5D and the HAQ according to the meaning of each domain and index reported in their official sources [14, 34]. We hypothesised that the EQ-5D domains relating to the physical component of health (mobility, self-care, usual activities and pain/discomfort), the VAS and the utility score correlated moderately or highly with the physical domains of the SF-36 (physical functioning, role-physical, bodily pain and physical component summary score) and all the domains of the HAQ. We also hypothesised that the HAQ pain VAS highly correlated with the pain/discomfort domain of the EQ-5D. We hypothesised that the anxiety/depression (EQ-5D) moderately or highly correlated with mental health, role emotional and the mental component summary score (SF-36). No correlation was expected between anxiety/depression and any HAQ domain. Vitality and general health on the SF-36 were expected to correlate moderately with all the EQ-5D domains, because of their particular capability to capture both physical and mental components of HRQoL. Low correlations were expected between the other domains not considered conceptually similar. This approach has been used successfully to analyse the construct validity of the EQ-5D with other standard, widely used generic HRQoL instruments [35]. In order to simplify the interpretation of the results, we presented correlation coefficients for the domains or scales that produced negative coefficients as positive values due to scores in opposite directions.

SPSS (version 23, IBM) was used to perform all analyses. A two-tailed p-value of <0.05 was considered significant.

Results

One hundred and nineteen adult patients with systemic sclerosis, median age 63 years (IQR 53–70 years), 94.1% female, were recruited. Median duration of Raynaud’s phenomenon was 10 years (IQR 6–16 years). ANAs were positive in 98.3% of patients; anti-topoisomeraseI-Scl-70 and anti-centromere antibodies were present in 27.7% and 49.6% of patients, respectively. The demographic and clinical characteristics of the patients are summarised in table 1.

Detailed HRQoL results are reported in table 2 and in figures 1 and 2. The total median scores obtained with the three instruments were 42.5 (PCS) and 48.2 (MCS) from the SF-36, 60.0 (EQ-VAS) and 0.81 (EQ-utility index) from the EQ-5D, and 0.5 (total score) and 36.5 (VAS pain score) from the HAQ.

The patients generally showed reduced levels of HRQoL in comparison with the general Italian female population [32] (table 3). In particular, the frequency of patients reporting some or severe problems in each domain of the EQ-5D profile was higher than the levels reported by the female general population in all but one age range: the 18- to 35-year-old patients apparently had no worse levels of HRQoL than their counterparts in the general population. However, only two patients were included in this subgroup; hence we do not consider this result to be reliable. Nevertheless, the median and mean VAS and utilities were lower in all the subgroups, including the 18- to 35-year-old patients.

Regarding acceptability, we did not receive from the patients any comment or complaint during or after the completion of the questionnaires. Regarding feasibility, the percentage of patients needing assistance to complete any HRQoL questionnaire was in total 2.5% (three patients). Responses were missing in one or more domains in 2.5% of the EQ-5D, in 7.6% of the HAQ and in 39% of the SF-36 questionnaires.

In the comparison between EQ-5D and SF-36 (table 4), we found moderate correlation coefficients (0.35–0.48) for EQ-5D mobility, self-care, pain/discomfort and the SF-36 PCS domain, and low correlation coefficients (0.05–0.21) between the same domains and the SF-36 MCS domain. EQ-5D usual activities moderately correlated (0.31) with MCS and highly correlated (0.71) with PCS. In contrast, EQ-5D anxiety/depression moderately correlated with SF-36 MCS (0.44), whereas the correlation coefficient with PCS was low (0.23). Furthermore, EQ-5D mobility correlated strongly with SF-36 physical functioning (0.63) and with bodily pain (0.54), weakly with mental health (0.19) and moderately with the other SF-36 single domains (0.31–0.41). EQ-5D self-care weakly correlated with SF-36 mental health, vitality and social functioning (0.18–0.26) and moderately with the other single SF-36 domains (0.32–0.37). Usual activities correlated moderately with general health, vitality and mental health (0.45–0.47), and strongly with the other domains (0.57–0.66). Pain/discomfort correlated strongly with SF-36 bodily pain (0.53) and weakly to moderately with the other domains (0.21–0.39). Anxiety/depression correlated strongly with social functioning and mental health (0.58 and 0.60, respectively), and weakly or moderately with the other single domains (0.20–0.48). The EQ-5D VAS and utility index, which measure and value overall health, were moderately or highly correlated with almost all of the SF-36 single and composite domains. EQ-VAS only weakly correlated with social functioning, mental health and MCS (0.27).

In the comparison between EQ-5D and HAQ (table 5), most of the correlations were moderate to strong between the EQ-5D mobility, self-care, usual activities, pain/discomfort, VAS and utility and all the HAQ single domains, the VAS pain and the total score. The highest coefficients were between self/care and dressing/grooming (0.68), self/care and HAQ total score (0.71), and HAQ total score and EQ-5D utility index (0.77). Unexpectedly, we found moderate correlations between HAQ pain VAS and pain/discomfort. Anxiety/depression of the EQ-5D was found to be weakly correlated with all the HAQ domains (all the coefficients were ≤0.23).

Discussion

The results of this study showed that the use of the EQ-5D is feasible, well accepted and suitable for clinical practice for the assessment of HRQoL in patients with systemic sclerosis. The structure of EQ-5D includes both physical and mental health status, as in the case of SF-36. However, compared with SF-36, EQ-5D can be completed quickly and its score can be simply interpreted by physicians.

In this study we found that the domains and indexes of the EQ-5D generally provided useful information on the general health of the patients, including both the physical component of health, which affects these patients with several disabilities and pain, and the mental component of health, related with the psychological impact attributable to everyday living with the disease and with the treatment necessary to manage it.

The comparison of the EQ-5D with the more frequently used SF-36 and HAQ questionnaires is interesting, as the results showed that all the domains of the latter two questionnaires were, overall, covered by the EQ-5D. Furthermore, the HAQ, which is a good questionnaire because it focuses on some specific aspects of daily activities and physical disability, does not include the mental component of health, which is a drawback in the assessment of health in patients with systemic sclerosis, who report compromised levels of mental health in comparison with the general population, as confirmed in other studies [9]. The SF-36 is more complete than the HAQ, but includes 36 questions on general health, which can be too many for some individuals, probably even more so for those who may have serious difficulties in writing because of typical symptoms affecting the fingers as in the case of patients with systemic sclerosis. In our study, we found a much lower percentage of missing data in the EQ-5D than in the HAQ and SF-36. This difference was probably influenced by the order of completion: the patients received the EQ-5D first, then the HAQ and finally the SF-36. However, these results showed that completion of the EQ-5D is feasible for these patients, and suggested that many questions or long questionnaires negatively affected the quantity of data collected and, probably, their quality. In fact, we can see that the percentage of missing data was already higher in the HAQ, given second, and even higher in the SF-36.

We found many correlation coefficients with expected strength. However, we also found a number of correlations whose values are different from those expected according to the official meanings assigned to the domains of each questionnaire. These results deserve some interpretation specific for systemic sclerosis. In particular, we found several unexpected moderate correlations such as those between EQ-5D mobility and role emotional, and EQ-5D anxiety/depression and role physical, and some unexpectedly low correlations such as EQ-5D anxiety/depression with vitality. These can be explained by the characteristics of the disease, in which physical and emotional aspects are peculiarly intertwined: patients with systemic sclerosis often experience depression due to their physical appearance (skin thickening of face, hands and arms), or painful physical complications (digital ulcers), which are also responsible for their limitation in everyday activities, both physical and social [9, 10]. The low correlation between EQ-5D pain/depression and the SF-36 domain vitality, which is expected to correlate with both physical and mental components of HRQoL, might be also surprising. However, the vitality domain is very specific and relates to “feeling energetic and full of pep, or worn out and tired” in SF-36 domain definitions [14]. One possible explanation is that patients with systemic sclerosis often suffer from lung disease, which may imply exertional dyspnoea and fatigue.

EQ-5D has been proposed as a measure for HRQoL and particularly for health utility in clinical trials in systemic sclerosis [35]. EQ-5D has also been used for HRQoL assessment in patients with primary Reynaud phenomenon [36]. Finally, EQ-5D is one of the most used tools for the calculation of QALYs [37].

Several studies in different clinical fields, such as schizophrenia [39], beta-thalassemia major [40] and haemophilia [41], in which both the EQ-5D and the SF-36 were used simultaneously, have shown that they provide similar information about individuals’ health, suggesting that the use of the EQ-5D as a generic instrument is sufficient and probably more efficient than the use of a longer generic questionnaire such as the SF-36. HAQ, which has been designed to be used in a rheumatology setting, seems to be more specific, since it contains domains related to activities that can be compromised in subjects with physical disabilities, and hence it can be viewed as a complementary tool. For instance, reaching, eating and arising in the HAQ are also contained in three domains of EQ-5D (mobility, self-care and usual activities), but not at the same level of detail. The domains that we considered conceptually equivalent in the EQ-5D and the HAQ correlated with each other, and those that we considered dissimilar did not correlate: in particular, anxiety/depression in the EQ-5D did not correlate at all with any of the HAQ domains.

The strength of our study is that although EQ-5D has already been validated in systemic sclerosis [38], to the best of our knowledge, this is the first study that compares its performance with the popular HAQ and SF-36.

One limitation of this study is that we did not assess acceptability with a specific set of questions, and this could be the subject of a future study. A further limitation is that we could not compare the feasibility of the EQ-5D with the other instruments, as we did not arrange the order of completion in a way that allowed the measurement of this property. In any case, based on our results we can conclude that use of the EQ-5D is feasible and that the submission of many questions to the patients can be counterproductive in terms of the quantity and, perhaps, the quality of data provided. Finally, we did not assess other psychometric properties of the EQ-5D in patients with systemic sclerosis, such as external reliability and its responsiveness to real changes in HRQoL.

In conclusion, the results of this study show that the EQ-5D is a suitable instrument for the routine assessment of HRQoL in patients with systemic sclerosis, because it is feasible, well accepted and provides a complete picture of overall health, despite the reduced number of items (five domains and one VAS). Furthermore, because it was designed to evaluate health, it also provides a utility index that can be used for the estimation of the QALY index that is advised for cost-utility analyses necessary to identify the most efficient treatment.

EQ-5D, used alone or in combination with HAQ, could be the most efficient tool, both for the patients with systemic sclerosis and their physicians, for the assessment of patients’ perceived health in clinical routine practice.

In the future, it might be useful to research one or two bolt-on domains in the EQ-5D, in order to obtain a reduced but not less informative battery of HRQoL instruments, by reducing those domains (e.g. mobility and walking, self-care and hygiene) that are probably redundant if the EQ-5D is used with the HAQ.

References

1 McMahan ZH, Hummers LK. Systemic sclerosis – challenges for clinical practice. Nat Rev Rheumatol. 2012;9(2):90–100. doi:http://dx.doi.org/10.1038/nrrheum.2012.191.

2 Haythornthwaite JA, Heinberg LJ, McGuire L. Psychologic factors in scleroderma. Rheum Dis Clin North Am. 2003;29(2):427–39. doi:http://dx.doi.org/10.1016/S0889-857X(03)00020-6.

3 Malcarne VL, Greenbergs HL. Psychological adjustment to systemic sclerosis. Arthritis Care Res. 1996;9(1):51–9. doi:http://dx.doi.org/10.1002/art.1790090110.

4 Ingegnoli F, Boracchi P, Ambrogi F, Gualtierotti R, Galbiati V, Meroni PL. Hand impairment in systemic sclerosis: association of different hand indices with organ involvement. Scand J Rheumatol. 2010;39(5):393–7. doi:http://dx.doi.org/10.3109/03009741003629028.

5 Pellar RE, Tingey TM, Pope JE. Patient-Reported Outcome Measures in Systemic Sclerosis (Scleroderma). Rheum Dis Clin North Am. 2016;42(2):301–16. doi:http://dx.doi.org/10.1016/j.rdc.2016.01.003.

6 Frantz C, Avouac J, Distler O, Amrouche F, Godard D, Kennedy AT, et al. Impaired quality of life in systemic sclerosis and patient perception of the disease: A large international survey. Semin Arthritis Rheum. 2016;46(1):115–23. doi:http://dx.doi.org/10.1016/j.semarthrit.2016.02.005.

7 Racine M, Hudson M, Baron M, Nielson WR. Canadian Scleroderma Research G. The Impact of Pain and Itch on Functioning and Health-Related Quality of Life in Systemic Sclerosis: An Exploratory Study. J Pain Symptom Manage. 2016;52(1):43–53. doi:http://dx.doi.org/10.1016/j.jpainsymman.2015.12.314.

8 Almeida C, Almeida I, Vasconcelos C. Quality of life in systemic sclerosis. Autoimmun Rev. 2015;14(12):1087–96. doi:http://dx.doi.org/10.1016/j.autrev.2015.07.012.

9 Thombs BD, van Lankveld W, Bassel M, Baron M, Buzza R, Haslam S, et al. Psychological health and well-being in systemic sclerosis: State of the science and consensus research agenda. Arthritis Care Res (Hoboken). 2010;62(8):1181–9. doi:http://dx.doi.org/10.1002/acr.20187.

10 Nguyen C, Ranque B, Baubet T, Bérezné A, Mestre-Stanislas C, Rannou F, et al.; Groupe Français de Recherche sur la Sclérodermie. Clinical, functional and health-related quality of life correlates of clinically significant symptoms of anxiety and depression in patients with systemic sclerosis: a cross-sectional survey. PLoS One. 2014;9(2):e90484. doi:http://dx.doi.org/10.1371/journal.pone.0090484.

11 Ingegnoli F, Galbiati V, Boracchi P, Comi D, Gualtierotti R, Zeni S, et al. Reliability and validity of the Italian version of the hand functional disability scale in patients with systemic sclerosis. Clin Rheumatol. 2008;27(6):743–9. doi:http://dx.doi.org/10.1007/s10067-007-0785-9.

12 Guyatt GH, Feeny DH, Patrick DL. Measuring health-related quality of life. Ann Intern Med. 1993;118(8):622–9. doi:http://dx.doi.org/10.7326/0003-4819-118-8-199304150-00009.

13 Dowie J. Decision validity should determine whether a generic or condition-specific HRQOL measure is used in health care decisions. Health Econ. 2002;11(1):1–8. doi:http://dx.doi.org/10.1002/hec.667.

14 Ware JE, Jr, Sherbourne CD. The MOS 36-item short-form health survey (SF-36). I. Conceptual framework and item selection. Med Care. 1992;30(6):473–83. doi:http://dx.doi.org/10.1097/00005650-199206000-00002.

15 Brooks R. EuroQol: the current state of play. Health Policy. 1996;37(1):53–72. doi:http://dx.doi.org/10.1016/0168-8510(96)00822-6.

16 Olivieri I, Cortesi PA, de Portu S, Salvarani C, Cauli A, Lubrano E, et al.; PACE Working Group. Long-term costs and outcomes in psoriatic arthritis patients not responding to conventional therapy treated with tumour necrosis factor inhibitors: the extension of the Psoriatic Arthritis Cost Evaluation (PACE) study. Clin Exp Rheumatol. 2016;34(1):68–75.

17 Ferreira PL, Gonçalves SP, Ferreira LN, Pereira LN, Antunes P, Gouveia N, et al. Assessing quality of life of self-reported rheumatic patients. Rheumatol Int. 2016;36(9):1265–74. doi:http://dx.doi.org/10.1007/s00296-016-3517-0.

18 Wallman JK, Eriksson JK, Nilsson JA, Olofsson T, Kristensen LE, Neovius M, et al. Costs in Relation to Disability, Disease Activity, and Health-related Quality of Life in Rheumatoid Arthritis: Observational Data from Southern Sweden. J Rheumatol. 2016;43(7):1292–9. doi:http://dx.doi.org/10.3899/jrheum.150617.

19 Fattore G. Proposta di linee guida per la valutazione economica degli interventi sanitari in Italia. [A proposal for guidelines for the economic evaluation of health interventions in Italy.] PharmacoEconomics Italian Research Articles. 2009;11(2):83–93. doi:http://dx.doi.org/10.1007/BF03320660.

20 Fries JF, Spitz P, Kraines RG, Holman HR. Measurement of patient outcome in arthritis. Arthritis Rheum. 1980;23(2):137–45. doi:http://dx.doi.org/10.1002/art.1780230202.

21 Masi AT. Preliminary criteria for the classification of systemic sclerosis (scleroderma). Subcommittee for scleroderma criteria of the American Rheumatism Association Diagnostic and Therapeutic Criteria Committee. Arthritis Rheum. 1980;23(5):581–90. doi:http://dx.doi.org/10.1002/art.1780230510.

22 van den Hoogen F, Khanna D, Fransen J, Johnson SR, Baron M, Tyndall A, et al. 2013 classification criteria for systemic sclerosis: an American College of Rheumatology/European League against Rheumatism collaborative initiative. Arthritis Rheum. 2013;65(11):2737–47. doi:http://dx.doi.org/10.1002/art.38098.

23 Ingegnoli F, Gualtierotti R, Orenti A, Schioppo T, Marfia G, Campanella R, et al. Uniphasic Blanching of the Fingers, Abnormal Capillaroscopy in Nonsymptomatic Digits, and Autoantibodies: Expanding Options to Increase the Level of Suspicion of Connective Tissue Diseases beyond the Classification of Raynaud’s Phenomenon. J Immunol Res. 2015;2015:371960. doi:http://dx.doi.org/10.1155/2015/371960.

24 Dolan P. Modeling valuations for EuroQol health states. Med Care. 1997;35(11):1095–108. doi:http://dx.doi.org/10.1097/00005650-199711000-00002.

25 Khanna D, Furst DE, Clements PJ, Park GS, Hays RD, Yoon J, et al.; Relaxin Study Group; Scleroderma Clinical Trials Consortium. Responsiveness of the SF-36 and the Health Assessment Questionnaire Disability Index in a systemic sclerosis clinical trial. J Rheumatol. 2005;32(5):832–40.

26 Hudson M, Thombs BD, Steele R, Panopalis P, Newton E, Baron M; Canadian Scleroderma Research Group. Health-related quality of life in systemic sclerosis: a systematic review. Arthritis Rheum. 2009;61(8):1112–20. doi:http://dx.doi.org/10.1002/art.24676.

27 Callahan LF. The History of Patient-Reported Outcomes in Rheumatology. Rheum Dis Clin North Am. 2016;42(2):205–17. doi:http://dx.doi.org/10.1016/j.rdc.2016.01.012.

28 Apolone G, Mosconi P. The Italian SF-36 Health Survey: translation, validation and norming. J Clin Epidemiol. 1998;51(11):1025–36. doi:http://dx.doi.org/10.1016/S0895-4356(98)00094-8.

29 Ware JE, Jr, Kosinski M, Bayliss MS, McHorney CA, Rogers WH, Raczek A. Comparison of methods for the scoring and statistical analysis of SF-36 health profile and summary measures: summary of results from the Medical Outcomes Study. Med Care. 1995;33(4, Suppl):AS264–79.

30 Poole JL, Steen VD. The use of the Health Assessment Questionnaire (HAQ) to determine physical disability in systemic sclerosis. Arthritis Care Res. 1991;4(1):27–31. doi:http://dx.doi.org/10.1002/art.1790040106.

31 Scalone L, Cortesi PA, Ciampichini R, Belisari A, D’Angiolella LS, Cesana G, et al. Italian population-based values of EQ-5D health states. Value Health. 2013;16(5):814–22. doi:http://dx.doi.org/10.1016/j.jval.2013.04.008.

32 Scalone L, Cortesi PA, Ciampichini R, Cesana G, Mantovani LG. Health related quality of life norm data of the Italian general population: results using the EQ-5D-3L and EQ-5D-5L instruments. Epidemiology, Biostatistics and Public Health. 2015;12(3):e11457-1–15. doi:http://dx.doi.org/10.2427/11457.

33 Cohen J. Statistical power analysis for the behavioural sciences. Hillside. NJ: Lawrence Earlbaum Associates. 1988.

34 Fox-Rushby J, Selai C. What concepts does the EQ-5D measure? Intentions and interpretations. The measurement and valuation of health status using EQ-5D: A European Perspective: Springer; 2003. p. 167–82.

35 Scalone L, Tomasetto C, Matteucci MC, Selleri P, Broccoli S, Pacelli B, et al. Assessing quality of life in children and adolescents: development and validation of the Italian version of the EQ-5D-Y. Ital J Public Health. 2011;8(4):331–41. doi:http://dx.doi.org/10.2427/5679.

36 De Angelis R, Salaffi F, Grassi W. Health-related quality of life in primary Raynaud phenomenon. J Clin Rheumatol. 2008;14(4):206–10. doi:http://dx.doi.org/10.1097/RHU.0b013e31817a2485.

37 Stamuli E. Health outcomes in economic evaluation: who should value health? Br Med Bull. 2011;97(1):197–210. doi:http://dx.doi.org/10.1093/bmb/ldr001.

38 Gualtierotti R, Scalone L, Ingegnoli F, Cortesi P, Lubatti C, Zeni S, et al. Health related quality of life assessment in patients with systemic sclerosis. Reumatismo. 2010;62(3):210–4. Italian.

39 Cortesi PA, Mencacci C, Luigi F, Pirfo E, Berto P, Sturkenboom MC, et al. Compliance, persistence, costs and quality of life in young patients treated with antipsychotic drugs: results from the Cometa study. BMC pychiatry. 2013:13(98):2-16.

40 Scalone L, Mantovani LG, Krol M, Rofail D, Ravera S, Bisconte MG, et al. Costs, quality of life, treatment satisfaction and compliance in patients with beta-thalassaemia major undergoing iron chelation therapy: the ITHACA study. Curr Med Res Opin. 2008;24(7):1905-17.

41 Scalone L, Mantovani LG, Mannucci PM, Gringeri A; COCIS Study Investigators. Quality of life is associated to the orthopaedic status in haemophilic patients with inhibitors. Haemophilia. 2006;12(2):154-62.