Evaluation of Work Capacity in Switzerland: a Survey among Psychiatrists about Practice and Problems

QUESTIONS UNDER STUDY: In Switzerland, psychiatric evaluations of work capacity for determining a person's eligibility for disability benefits are being criticised for a lack of transparency and high inter-rater variability. The aims of this study were to learn about the current practice of psychiatrists, to explore possible sources for lack of transparency and variability, and to contrast practice with current professional guidance. METHODS: A national online-survey among psychiatrists who performed five or more evaluations of work capacity per year. Based on discussions with experts and a literature review, we structured questions focusing on reporting on work capacity, the description of a claimant's previous job, and measures of quality assurance. RESULTS: A total of 129 psychiatrists responded (31% of estimated 412 eligible psychiatrists). The majority reported using instructions of the insurers (77%), peer consulting (65%) and process guidelines (51%). They expressed a claimant's work capacity as free text and percentage work capacity (49%), percentage only (23%), or free text only (14%). A total of 13% used instruments to document work capacity. Psychiatrists considered three different interpretations of percentage work capacity as equally applicable. A job description was regarded as mandatory to determine work capacity by 90% but only 26% received it and found it mostly deficient. CONCLUSIONS: The transparency and reliability of Swiss psychiatrists' conclusions on a claimant's work capacity may be reduced by unsystematic reporting, variable interpretation of the percentage work capacity, lack of a detailed job description and insufficient quality control. Education , engagement of insurers and new guidelines might be effective means of implementing improvements.


Introduction
Claimants for a work disability benefit need to undergo an independent medical evaluation (IME) to establish their work capacity.In Switzerland, IMEs are conducted by medical specialists (experts).They may work in their own practice or in a hospital where they treat patients, or they may work in specialised evaluation centres or insurance organisations.IMEs consist of assessments and lead to reports that serve insurers with key information to grant disability benefits [1,2].Swiss insurers typically ask medical specialists to evaluate a claimant's work capacity in both his last job and in suitable alternative work.Work capacity may be obvious in claimants with severe diseases, such as heart failure or cancer, but is often more challenging to determine in claimants with other disorders such as psychiatric disorders.Difficulties arise in assessing the diagnosis [3][4][5] and the remaining functional capacity [3][4][5][6][7][8][9][10][11][12], and in relating functional capacity to work requirements [5,[9][10][11][12].In recent years, claimants, insurers, lawyers and authorities have raised major concerns about the quality of IMEs that provoke legal procedures that are costly both in monetary terms and in human suffering [13,14].The Swiss Federal Court concluded that current reports of evaluations lack explicitness and transparency in the decision-making process, and lack reliability in the conclusions [15].Evidence supports the concerns [3][4][5][6][7][8][9][10][11][12].In one study, 22 German psychiatrists evaluated the work capacity of a claimant with recurrent depression on the basis of a video-taped interview.One-third each attributed a work capacity of <3 hours, 3 to 6 hours or >6 hours per day [4]; a result as good as throwing a dice.In a Danish study, 11 specialists in social medicine agreed poorly on the work capacity of 8 claimants with somatic and psychiatric disorders (kappa 0.33, worse in psychiatric claimants) [12].Not surprisingly, medical experts have urgent calls for research to sort out the sources of such disagreement and find ways to reduce it [16].Little research has explored possible sources of inter-rater variability.Two Dutch studies hypothesised that the variability may be related to different assessment routines of medical experts [5,10].Training and quality control of IMEs vary considerably among Western countries [17,18].In addition, variability in reporting may affect transparency and reliability of IMEs.A recent survey among social insurers in 12 European countries [19] draws a heterogeneous picture: Instruments to report on work capacity are in use in Iceland, the Netherlands, Sweden and the United Kingdom.Germany applies a semi-structured approach, requesting the psychiatrist to report on body function, activities and environmental factors.Belgium, Switzerland, Finland, France, Italy, Norway and Slovenia do not require a formal structure for reporting.In order to address the concerns about transparency and reliability we need to know the current practice and existing deficits.Given the lack of empirical information about current practice in Switzerland, we performed a nationwide survey.For practical reasons, we focused this survey, and a subsequent research programme (www.unispital-basel.ch/asim/RELY), on psychiatric disorders.These constitute a major and growing proportion of claims for disability allowance in Switzerland and elsewhere [1,20].We aimed to learn about the psychiatric experts' current practice in assessing and, in particular, reporting on work capacity, and to identify potential sources for lack of transparency and high variability in IMEs that may be amenable to improvement.We contrasted the findings with the current professional guidance and compared psychiatrists in specialised IME centres to other psychiatrists.

Study population and survey administration
Our study population were psychiatrists who had conducted a minimum of five IMEs of work capacity during the previous year.We distributed the survey through the mailing list of the Swiss Society of Psychiatry and Psychotherapy (SGPP) to its 1,678 members.An invitation letter from the presidents of the SGPP and the Swiss Society of Insurance Psychiatry informed participants about the purpose of the survey, time to completion (15 minutes), confidentiality of the data and our intention to publish the results.We sent a reminder after three weeks.

Development of the questionnaire and review of current guidance
We identified critical issues in current evaluation practice in three iterative steps.Firstly, we discussed potential issues within our study group consisting of Swiss experts in the field, psychologists and methodologists.Secondly, we reviewed the literature about Swiss IMEs, including news-paper and juridical articles, topped up by empirical studies we could identify in the literature, without performing a systematic review.Thirdly, we studied existing Swiss guidance [1,5,13,14,[21][22][23][24][25][26][27].Three topics seemed to be of high relevance to the Swiss community: reporting of work capacity, the description of a claimant's previous job and measures of quality assurance applied in practice.Based on these three topics, we developed a questionnaire.The study group iteratively discussed versions of the questionnaire in order to focus on issues that are amenable to improvement, to develop a clear terminology and to keep the questionnaire concise.Finally, we included questions about various ways to report work capacity (free-text, instruments, percentage and various combinations) and potential interpretations of a percentage work capacity.We asked whether psychiatrists regularly report not only their own judgment on work capacity, but also the claimant's self-perception.We inquired about the use of measures to standardise the evaluation, such as a description of the current job, process guidelines, reporting forms, reporting instruments, and the use of scientific evidence to support the judgments and the importance of such measures to the psychiatrist.Furthermore, we ascertained the psychiatrists' training and activities in continuous education.Where applicable, we structured response options on a four-point scale.We programmed the survey on www.surveygizmo.comusing verification of data entry, randomised order of response options, and prevention of duplicate participation.Three psychiatric experts pre-tested the survey for clarity and time to completion.The appendix shows the original questions in German and French, and an English translation.

Analysis
We analysed both complete and incomplete questionnaires unless they contained only demographic information.In order to estimate the response rate we had to approximate the size of our study population, as the number of psychiatrists who performed five or more evaluations of work capacity per year is unknown.Based on discussions within the study group, we assumed that 50% of our study population would hold a voluntary certification for IME that is offered by the Swiss Society of Insurance Medicine.Currently, 206 psychiatrists are certified, corresponding to an estimated population size of 412.For the following variables we hypothesised that psychiatrists who work in insurance organisations or evaluation centres, differ from psychiatrists who primarily treat patients in their practices or in hospitals: Work experience (numerical, Q4), reporting work capacity through free text / percentage / instrument (all dichotomous, Q8), meanings of percentage work capacity (all dichotomous, Q10 + 12), number of education activities (numerical, Q13), frequency of feedback from insurer (dichotomised, Q20), and quality of this feedback (dichotomised, Q21).We used the Wilcoxon signed rank test with continuity correction for numerical variables that were not normally distributed and the Fisher's exact test for dichotomous / dichotomised variables.We tested two sided with an alpha of 0.05 without correction for multiple testing.We used statistical software R [28].

Characteristics of respondents
A total of 129 of the responding psychiatrists qualified for the study (fig.1).The estimated response rates for individual questions varied from 31% (129/412) to 22% (91/ 412).Table 1 summarises the characteristics of the psychiatrists.

Reporting of work capacity
Table 2 shows modes of reporting work capacity.A total of 13% (14/111) of the psychiatrists used one or more instruments, such as the MINI-ICF-APP [29] (11/14), the Global Assessment of Functioning [30] (8/14), or a checklist We asked the psychiatrists two times to indicate all interpretations of percentage work capacity that they regarded as applicable (a) "in principle", and (b) "in a recent case".
We offered five options that we had identified through discussion in our group (table 3).Regarding "in principle", they approved an average of 4 (median, interquartile range [IQR] 3 to 5) interpretations.No clear preference for one single interpretation emerged.Rather, three interpretations -"restrictions of activities", "working hours per day", and "productivity" -ranked equally with approval rates around 80%. Regarding "in a recent case", they selected on average 2 (median, IQR 1 to 3) interpretations.Again, no clear preference emerged (table 3).Most psychiatrists approved productivity (60%) followed by "working hours per day" (48%), "working hours per week" (46%), and "restriction of activities" (41%).

Job description
Most psychiatrists (74%, 70/94) rarely received a description of the claimant's last workplace.If so, 67% (56/83) were dissatisfied or highly dissatisfied with its quality.The provision of a detailed job description by the insurers was considered as mandatory information by 90% (84/93).

Measures for quality control
When asked about professional guidance and quality control, psychiatrists referred to reporting instructions provided by the insurers (77%), their use of peer consultation (65%), or formal supervision (40%).Half of the psychiatrists (51%) regularly employed process guidelines, and one third (32%) regularly referenced scientific literature in their reports (table 4).
In the two years prior to the survey, 85% (84/99) of psychiatrists attended educational activities related to IMEs (median frequency 3, IQR 1 to 6).Of those, 68% (57/84) experienced the educational meetings as being of high or very high quality.A total of 63% (63/100) of psychiatrists welcomed the idea of compulsory continuing educational activities, while 29% (29/100) opposed it and 8% were indifferent (8/100).Almost all psychiatrists (92%, 82/89) wanted the insurers to always provide feedback about the final administrative decision on the claimant's work capacity.However, most psychiatrists (78%; 74/93) rarely received such feedback and if so, they were dissatisfied with its quality (75%; 43/ 57).

Group comparisons
Psychiatrists employed by assessment centres or insurance organisations (n = 28) attended more educational activities

Original article
Swiss Med Wkly.2013;143:w13890 related to IMEs compared to practice, or hospital-based psychiatrists (p = 0.001, median 6 versus 2).In addition, they indicated more often that the percentage work capacity in a recent case meant productivity (p = 0.006, OR = 3.5).Group differences were not significant in the other pre-specified variables.

Current guidance
Six Swiss professional societies or opinion leaders in insurance medicine published guidance on how to perform and report independent general and psychiatric evaluations [21-23, 26, 31, 32].Table 5 compares the recommendations of these publications: Four publications instruct experts to report the claimant's work capacity as a percentage [21,22,31,32] and three publications refer to the Mini-ICF-APP as an optional tool for reporting functional capacity [22,23].One publication stresses the need for comprehensive information about job requirements [21].

Discussion
We found that reporting on work capacity by psychiatrists is hindered by procedural (interpretation of and reporting about work capacity) and structural deficits (use of job description and quality control).

Interpretation and reporting of work capacity
Guidance consistently requested experts to express a claimant's work capacity as percentage, and indeed, 81% of psychiatrists reported work capacity exclusively as a percentage, or in combination with free text or an instrument.None of the publications reflected on the advantages and disadvantages of using this percentage, its different options of interpretation, or alternative ways to document work capacity.In our survey, where we had offered five different interpretations of percentage work capacity, the experts indicated all interpretations to be sensible and expressed a high or very high approval to each of four interpretations.The lack of a congruent understanding of percentage work capacity, a key concept in the Swiss evaluation of work capacity, seems an important source of poor transparency and variability in the decisions about work capacity.
Reporting tools can help to standardise process and content of an IME.However, instruments documenting work ability have not yet found their place in current Swiss psychiatric evaluations of work capacity.Only 13% of psychiatrists reported the regular use of an instrument, such as the Mini-ICF-APP [29] and / or the Global Assessment of Functioning [30].The low uptake by psychiatric experts seems to mirror both the lack of instruments that have been validated in the context of evaluation of work capacity [23] and the low importance that the professional recommendations attach to these tools.The professional guidance that referred  to instruments, did so in an unassertive manner and mentioned these only as one option to report on activity and participation deficiencies [22,23] (table 5).

Job description
Most psychiatrists found it mandatory to have an informative job description in order to relate a claimant's deficits and remaining abilities to his or her last work.However, they rarely received this from the insurer, and if provided, its quality was regarded as insufficient.Current guidance hardly comments on the job description.Only the most recent interdisciplinary guidance [21] demands unambiguously that the medical experts need to know the claimant's work requirements.

Quality control
The majority of psychiatric experts follow specific education (85%), and adhere to reporting instructions from the insurers (77%) and guidelines (51%).The widespread acceptance of these measures indicates that they may be suitable for addressing the deficits that we identified in this study.Although we did not explore the content of education programmes and instructions from insurers, there may be high potential for improvement if we assume that specific instructions on reporting and use of job profiles may be equally vague as in current guidance (table 5).
A proportion of 65% psychiatrists regularly uses peer consulting.Although we did not explore the psychiatrists understanding of peer consulting, which may range from a side comment in a conversation to a formal exchange, this indicates that a majority is interested in consolidating their own judgments through the professional consensus.In con-trast, few psychiatrists use supervision.This may be explained by the longstanding experience of most experts (median age between 51 and 60 years).A total of 32% of the psychiatrist referenced literature on a regular basis.This seems promising considering the lack of research in the field.

Strength and limitations
We identified critical issues in current psychiatric IMEs of work capacity and generated the questionnaire using an iterative approach with input from literature and experts.Its nationwide administration captures key elements of the practice of psychiatrists across Switzerland.We contrasted the critical issues with published guidance on performing IMEs.This enabled us to identify inconsistent or vague recommendations that may cause variability in the current practice of psychiatric experts.
Our study has limitations.An estimated response rate of 31% cautions against overconfident generalising of the findings.We could not compare the characteristics of our sample with the study population as sufficient information about the latter is lacking.We did not cover the entire assessment process and other deficits may be relevant as well, although we did not find these in preparing the survey.We did not go into detail about quality control.Furthermore, it is uncertain as to what degree reporting of work capacity, description of a claimant's previous job, and measures of quality control actually contribute to poor transparency and inter-rater variability.Although being of high relevance for the community, empirical evidence seems to be completely lacking.We conducted a survey that essentially draws on respondents' opinions and memory of their practice.This may therefore be biased.Ideally, the practice of reporting could be verified in a random sample of IME reports.We addressed Swiss psychiatrists, but evidence suggests that similar problems exist in other disciplines and in other countries [6,9].In addition, we did not address other important stakeholders such as insurers, patients, judges, and lawyers who may have a different perspective from psychiatric experts.We will address these stakeholders in a subsequent survey.

Conclusions
Swiss psychiatrists have heterogeneous routines of assessing and reporting on work capacity in IMEs.Regular use of reporting instruments is uncommon and the key concept of expressing work capacity as a percentage lacks a universally accepted interpretation.In addition, psychiatrists miss a detailed job description and feedback from insurers.These deficiencies may reduce the transparency of decision-making and contribute to the variability in the reports.The impact of these deficits needs to be established.Education, engagement of insurers, and new professional guidelines might be effective means of implementing intended improvements.

Figures (large format)
Recruitment of psychiatric experts.

Table 2 :
Current practice of reporting on work capacity through text, instrument(s), percentage or combinations.

Table 3 :
Interpretations of percentage work capacity.

Table 4 :
Activities for quality control used by psychiatric experts.
Swiss Medical Weekly • PDF of the online version • www.smw.ch

Table 5 :
Published guidance on psychiatric and general medical evaluations of work capacity.

or societies Health conditions Description of last job for evaluating work capacity Instruments to standardize evaluation of work capacity Percentage to report work capacity Swiss Society of Psychiatry and Psychotherapy + Swiss Society of Insurance Psychiatry 2012 [22]
Swiss Medical Weekly • PDF of the online version • www.smw.ch