Primary care research – trade-off between representativeness and response rate of GP teachers for undergraduates

DOI: https://doi.org/10.4414/smw.2012.13537

Introduction

Low response rates are common in primary care research [1–5]. In contrast to clinical study settings, the target population of surgery-based physicians is often difficult to observe. Especially among general practitioners (GPs), lack of time or interest in the topic under study may induce low response rates [6]. In this case, the respondents’ representativeness was investigated using publicly available data [7].

The aim of the underlying study was to investigate the distribution and use of rapid diagnostic tests among German general practitioners. The questionnaire response rate was relatively low (32.87%). We found a proportionally much lower response rate among the participating GPs than among university associated GPs. These GP teachers for undergraduates (GPTUs) are involved in the medical curriculum of Leipzig Medical School where students complete an internship at a GP’s surgery to acquire work experience. GPTUs’ remuneration is not substantial (EUR 250 per student). The educational activity is not very time-consuming.

The higher response rate among the GPTUs raised the question whether the creation of a panel of these GPTUs for further research would be beneficial in furnishing more reliable results.

The objective of the study was therefore to examine the representativeness of all respondents in comparison with all GPs in the federal state of Saxony. To investigate the option of a panel of GPTUs, the representativeness of the responding GPTUs was of particular interest.

Methods

Sample and survey

The GPs were randomly selected from the members’ register of the Kassenärztliche Vereinigung Sachsen (KVS) [8]. This is the most valid and up-to-date register of physicians in Saxony. The KVS is a subdivision of the National Association of Statutory Health Insurance Physicians (Kassenärztliche Bundesvereinigung, KBV) in the federal state of Saxony. The membership list contained a total of 1,808 GPs in 2009. The GPTUs (n = 96 in 2009) had to be excluded to prevent duplications during the selection process. The resulting list of 1,712 GPs was sorted alphabetically and numbered consecutively. Then 244 numbers from the range of 1 to 1712 (approximately one in seven GPs) were randomly selected using commercial statistical software. These 244 GPs were contacted by mail.

The list of GPTUs was also sorted alphabetically and numbered consecutively. For the mail survey, 48 numbers (one in two GPs) were randomly selected from the range of 1 to 96 using commercial statistical software.

In 2009 we used a three-page questionnaire containing questions on the awareness, use and assessment of relevance of a total of 27 different rapid tests. Each GP was contacted once. The GPs received no incentives for participation in the survey.

To examine the representativeness of our sample compared with the underlying basic population of all GPs in Saxony we assessed socio-demographic (age, sex) and subject-specific characteristics (years in practice, academic degree, additional qualification, practice structure) using a public data set. Additional qualifications are officially tested and certified qualifications, for example diabetology, geriatrics or occupational medicine.

The data set was provided by the Kassenärztliche Bundesvereinigung (KBV, National Association of Statutory Health Insurance Physicians), specifically for the purpose of the study [9]. The key date of the set was 31 December 2009. In accordance with the German data privacy act, the data set was pooled; it included absolute numbers or, if applicable, the mean and the standard deviation. No individual data records of the GPs were available. For that reason some characteristics, such as age and years in practice, of those who did not respond to the questionnaire were not publicly available.

To verify the option of creating a panel of GPTUs for further research, the current group of GPTUs in 2011 (n = 118; since 2009, a further 22 GPs have became GPTUs) was also compared with the data set provided by the KBV (2009).

The characteristics of the GPTUs were mandatorily recorded during the signing of the contract between the GP and the university.

Statistical analysis

As the first step, differences between the respondents and all GPs in Saxony regarding sex, academic degree, additional qualifications and practice structure were investigated using the Chi² goodness of fit-test. Differences regarding age and years in practice were examined using the one sample two-sided t-test.

As the second step the responding GPTUs were compared with all GPs in Saxony in the same way. Additionally, the responding GPTUs were compared with all non-GPTU respondents.

Finally, the representativeness of the complete group of GPTUs (including those who did not respond) in comparison with all GPs in Saxony was assessed. All statistical analyses were performed with SPSS 18.0 for Windows.

Results

A total of 292 randomly selected GPs (48 GPTUs and 244 other GPs) were notified by mail. Due to incorrect address data, six letters were returned. Of the 286 GPs who were finally notified, 94 responded (response rate: 32.87%).

As the first step, differences between all respondents and all GPs in Saxony were investigated. The respondents’ socio-demographic characteristics are shown in table 1. There were no significant differences between the respondents and all GPs in Saxony in gender ratio (p = 0.851). The mean age of the respondents differed from that of all GPs in Saxony statistically significantly (p = 0.004). The respondents tended to be younger although the difference was small (50.46 vs. 53.29 years).

The respondents’ characteristics according to the subject-specific characteristics are shown in table 1. There were significant differences between the respondents and all GPs in Saxony in academic degree (p = 0.008), additional qualifications (p <0.001), and practice structure (p = 0.029). The respondents more often had an academic degree; also, they had more additional qualifications and tended more to work in a group practice than all GPs in Saxony. However, there were no significant differences between the respondents and all GPs in Saxony regarding years of practice (p = 0.922).

All in all, the respondents were not representative with regard to the variables age, academic degree, additional qualification and practice structure. In comparison, they could be regarded as representative for the variables sex and years in practice.

As the second step, only the group of responding GPTUs was investigated in comparison with all GPs in Saxony. The results were similar to the first step (all respondents vs. all GPs in Saxony) as can be seen in table 2. According to the variables academic degree (p = 0.036) and additional qualifications (p = 0.017) the participating GPTUs tended to be significantly better qualified than the mean of all GPs in Saxony. However, no significant differences were found with regard to the variables age (p = 0.127), sex (p = 0.600), years in practice (p = 0.521), and practice structure (p = 0.292). According to this, no significant differences were found between the responding GPTUs and the non-GPTU respondents (data not shown).

Further analysis of the responding GPTUs showed that the response rate among this group (64.58%, n = 31) was much higher than among other GPs (26.47%, n = 63). On this account, the complete group of GPTUs (n = 118 in the year 2011) was tested for representativeness by comparing all GPTUs with all GPs in Saxony according to the same socio-demographic and subject-specific characteristics.

However, it appeared that this group is likewise not representative, as can be seen in table 3. According to the variables academic degree (p <0.001) and additional qualifications (p = 0.038) the GPTUs were significantly better qualified than the mean.

The mean age of all GPTUs differed from that of all GPs in Saxony statistically significantly (p = 0.014). The GPTUs tended to be younger, but the difference was small (51.04 vs. 53.29 years). No significant differences were found in relation to the variables sex (p = 0.333), years in practice (p = 0.236), and practice structure (p = 0.069).

In conclusion it can be said that neither the respondents nor the GPTUs are a representative sample of the GPs in Saxony.

Discussion

In our study, the respondents were not a representative sample, as they were more highly qualified than the mean of all GPs in Saxony. Higher response rates were found among the GP teachers for undergraduates (GPTUs), but this group tended to be younger and more highly qualified than the mean of all GPs in Saxony.

Strengths and limitations

To the best of our knowledge this is the first study to focus on characteristics and representativeness of university-associated GP teachers. Furthermore, the option of creating a panel of GPs to overcome low response rates in primary care research has not yet been investigated.

To examine the option of creating a panel of GPTUs, the current group of GPTUs in 2011 (n = 118) was compared to the data set provided by the National Association of Statutory Health Insurance Physicians (KBV, 2009). This is a limitation of the study due to possible changes in the target population (the GPs in Saxony). However, no newer data set is available.

Some other characteristics of the respondents such as ethnicity or remoteness of the practice might have been interesting for the research question. Furthermore, other studies suggested that remoteness does not exert a relevant influence on response rates or response bias [10, 11]. Unfortunately we could not include further characteristics in the analysis because the data set offered only limited information. In this context, the lack of detailed information in the pooled data set prevented analysis of non-response bias.

The GPs were only contacted once by mail. In this context a mail reminder or phone call could have resulted in a higher response rate.

Scientific context

As already mentioned, low response rates are common in primary care research when using mail surveys [1, 4, 7, 14–16].

In general, mail surveys of professionals can be difficult to conduct compared to surveys conducted in the general population. Professionals such as physicians are not simply members of the general population; they need special incentives and information on the study to secure their cooperation [17, 18]. A review of response rates published in medical journals showed that response rates of surveys among physicians are 14% lower than those of non-physician surveys [14]. An important reason could be lack of time in a busy practice. GPs must prioritise their workload and will probably balance the perceived value of the survey against the time it takes [17, 19].

On the other hand, the perceived low salience of the study might also play a role [6]. In a study on British GPs, significantly different response rates resulted from two identically designed surveys on different issues. The design, the formats, the methods and the target population were similar, so solely the content might have influenced the response rate [20].

Lack of time in particular might have played a role for the non-responding GPs, as the questionnaire was relatively extensive (three questions for each test on a total of 27 different rapid tests). A Cochrane Review on methods to increase response rates to postal questionnaires found that the questionnaire length has a substantial impact on the response rate [12]. In a study on non-response bias it was found that every additional page in a survey caused a significant decline in the response rate [21].

When the response rate is low, the respondents’ representativeness is to be investigated by the use of publicly available data [7]. There is no necessary link between a low response rate and a response bias; in principle surveys with low response rates can also provide a representative sample [15, 21] while, vice versa, a high response rate does not necessarily provide a representative sample [14, 16]. If the respondents constitute a representative sample, bias caused by structural differences between the respondents and the target population can be ruled out.

In this context, particular caution is necessary when the respondents are GPTUs. The response rate among this group was significantly higher than among other GPs, but GPTUs did not constitute a representative sample. The GPTUs were obviously more highly qualified than the mean, especially where academic degrees and additional qualifications are involved (table 2). However, the higher qualification finding may be a trait of all respondents because no significant differences were found between responding GPTUs and responding non-GPTUs. It may be assumed that there is a risk of a trade-off between high response rates and the respondents’ representativeness. These findings correspond to the results of other studies investigating response bias in primary care research, where respondents tend to be more highly qualified [1, 7, 22, 23].

In contrast to all GPs in Saxony, the respondents tended more to work in a group practice. This finding corresponds to the results of the Swiss FIRE project (Family medicine ICPC-Research using Electronic medical records) where in the main the participants worked in group practices [18]. It is conceivable that the more flexible organisational structures of a group practice afford more resources for research activities in addition to patient care.

But what are the reasons for the substantial differences between the GPTUs and the target population (the GPs in Saxony) in particular? Some studies show that GPs involved in teaching undergraduates respond significantly more often to surveys than the mean [24–26].

The GPTUs are possibly more interested in clinical research and more likely to cooperate because they engage in teaching students in addition to their extensive work in practice [26]. However, teaching students may also have some other effects on the GPTUs. Two qualitative studies on the effects of teaching undergraduates on GPs found that teaching leads to a higher motivation and a broadened professional horizon. The GPTUs must reflect on what they do and the students’ questions force them to keep up to date with recent developments through literature research [27, 28]. These effects may encourage the interest in clinical research and thus probably explain the higher response rates of GPTUs.

In our study the GPTUs had an academic degree significantly more often than the mean. It is possible that the personal experience of the research process leads to sympathy for and interest in current research projects [29].

Finally, some organisational aspects probably also play an important role in answering university mail surveys. The group of GPTUs surveyed has a contractual agreement with the university on teaching of students that provides some kind of commitment. Additionally, on a more personal level, the department periodically arranges personal meetings for the GPTUs, including introduction of the department’s new research projects. This personal relationship is very important in recruiting physicians as participants in research [30].

In this context it is possible that GPTUs may have been reminded of the survey by colleagues at the university, and this could have been an influence on the response rate additional to the personal characteristics of the GPTUs. This implicit influence on surveying (which cannot be ruled out) is inevitable, unintentional and not measurable.

Conclusions

In our specific case, care must be taken when creating a panel of GPTUs for clinical research because, despite high response rates, the results of such studies probably cannot be generalised. On the other hand, the responding GPTUs were similar to the other respondents; therefore, recruiting them for surveys may be acceptable. We have to consider that it is difficult to achieve representativeness in a group of reluctant professionals.

Generally, the creation of a panel of GPTUs is possible, but it requires investigation of its representativeness. If the panel is not representative, there remains the option of creating a stratified sample according to the target population.

Acknowledgements:The authors would like to thank Mr. Hannes Munz (KBV, National Association of Statutory Health Insurance Physicians) for his helpful support. The data set was provided by the KBV and is based on the Bundesarztregister (German Federal Registry of Physicians).

References

  1 Cockburn J, Campbell E, Gordon JJ, Sanson-Fisher RW. Response bias in a study of General Practice. Fam Pract. 1988;5(1):18–23.

  2 Creavin ST, Creavin AL, Mallen CD. Do GPs respond to postal questionnaire surveys? A comprehensive review of primary care literature. Fam. Pract. 2011;28:461–7.

  3 Crouch S, Robinson P, Pitts M. A comparison of general practitioner response rates to electronic and postal surveys in the setting of the National STI Prevention Program. Aust N Z J Public Health. 2011;35:187–9.

  4 Guadagnoli E, Cunningham S. The effects on nonresponse and late response on a survey of physician attitudes. Evaluation & the Health Professions. 1989;12:318–28.

  5 Bergk V, Gasse C, Schnell R, Haefeli W. Mail surveys: Obsolescent model or valuable instrument in general practice research? Swiss Med Wkly. 2005;189–91.

  6 VanGeest J, Johnson T, Welch V. Methodologies for improving response rates in surveys of physicians. Evaluation & the Health Professions. 2007;30:303–21.

  7 Barclay S, Todd C, Finlay I, Grande G, Wyatt P. Not another questionnaire! Maximizing the response rate, predicting non response and assessing non response bias in postal questionnaire studies of GPs. Fam Pract. 2002;19(1):105–11.

  8 List of members. Kassenärztliche Vereinigung Sachsen (KVS, Association of Statutory Health Insurance Physicians of Saxony) c2009 – [cited 2009]. Available from: http://www.kvs-sachsen.de/mitglieder/

  9 Unpublished data provided by the National Association of Statutory Health Insurance Physicians (KBV), personal communication with Mr. Hannes Munz, 17.03.2011.

10 Bjertnaes O, Garrat A, Botten G. Nonresponse bias and cost-effectiveness in a Norwegian survey of family physicians. Evaluation & the Health Professions. 2008;31:65–80.

11 McLaren B, Shelley J. Response rates of Victorian general practitioners to a mailed survey on miscarriage: randomised trial of a prize and two forms of introduction to the research. Aust N Z J Public Health. 2000;24:360–4.

12 Edwards PJ, Roberts IG, Clarke MJ, DiGuiseppi C, Wentz R, Kwan I, et al. Methods to increase response rates to postal questionnaires (Review). The Cochrane Library 2009, Issue 2.

13 Chmiel C, Bhend H, Senn O, Zoller M, Rosemann T. The FIRE project. A milestone for research in primary care in Switzerland. Swiss Med Wkly. 2011,140:w13142.

14 Asch D, Jedrziewski M, Christakis N. Response rates to mail surveys published in medical journals. J Clin Epidemiol. 1997;50:1129–36.

15 Robertson J, Walkom E, McGettigan P. Response rates and representativeness: a lottery incentive improves physician survey return rates. Pharmacoepidemiol Drug Saf. 2005;14:571–7.

16 Templeton L, Deehan A, Taylor C, Drummond C, Strang J. Surveying general practitioners: does a low response rate matter? Br J Gen Pract. 1997;47:91–4.

17 Sudman S. Mail surveys of reluctant professionals. Evaluation Review. 1985;9(3):349–60.

18 Flanigan TS, McFarlane E, Cook S 2008. Conducting survey research among physicians and other medical professionals – A review of current literature. Presented at the AAPOR 2008; May 2008; New Orleans, LA, USA.

19 Kaner EFS, Haighton CA, McAvoy BR. “So much post, so busy with practice – so, no time”: A telephone survey of general practitioners’ reasons for not participating in postal questionnaire surveys. Br J Gen Pract 1998;48:1067–9.

20 Rashidian A, van der Meulen J, Russell I. Differences in the contents of two randomized surveys of GPs’ prescribing intentions affected response rates. J Clin Epidemiol. 2008;61:718–21.

21 McFarlane E, Olmsted MG, Murphy J, Hill CA. Nonresponse bias in a mail survey of physicians, Evaluation & the Health Professions. 2007;30(2):170–85.

22 Cartwright A. Professionals as responders: variations in and effects of response rates to questionnaires, 1961–77. Br Med J. 1978;2:1419–21.

23 Stocks N, Gunnell D. What are the characteristics of general practitioners who routinely do not return postal questionnaires: a cross sectional study. J Epidemiol. & Community Health 2000;54:940–1.

24 Giveon S, Kahan E, Kitai E. Factors associated with family physicians' involvement in research in Israel. Acad Med. 1997;72 (5):388–90.

25 Hummers-Pradier E, Scheidt-Nave C, Martin H, Heinemann S, Kochen M, Himmel W. Simply no time? Barriers to GPs’ participation in primary health care research. Fam Pract. 2008;25(2):105–12.

26 Silagy C, Carson N. Factors affecting the level of interest and activity in primary care research among general practitioners. Fam Pract. 1989;6(3):173–6.

27 Hartley S, MacFarlane F, Gantley M, Murray E. Influence on general practitioners of teaching undergraduates: qualitative study of London general practitioner teachers. Br Med J. 1999; 319:1168–71.

28 Grant A, Prout H, Hawthorne KT, Jones L, Houston H. Some effects of teaching undergraduate medical students on general practitioner thinking and learning. Educ Prim Care. 2010;21:97–104.

29 Morris, CJ, Cantrill, JA, Weiss, MC. GP survey response rate: a miscellany of influencing factors. Fam Pract. 2001;18(4):454–6.

30 Borgiel AE, Dunn EV, Campbell LT, MacDonald PJ, Evensen MK, Bass MJ, et al. Recruiting family physicians as participants in research. Fam Pract. 1989;6(3):168–72.