Confused by clinical research methods? Get a clue!
Thomas Perneger
Clinical research used to be simple, a relaxing diversion after a long day at the surgery. A doctor reviewed the charts of all three patients in his or her practice who suffered from an exotic ailment, computed a proportion or two (“a third died”), took photographs of a Petri dish and of a face rash, added a black bar to cover the eyes of Exhibit 2 – done.
That was then. Today researchers are like wedding planners – they obsess about inclusion criteria, cover up adverse events before anyone notices, lose sleep over budget overruns, and miss out on all the fun. Readers of medical journals are not spared. Articles are full of jargon, and it’s not medical Latin like in the old days. Especially maddening are statistical concepts, disguised as single letters (p, r, kappa), cute metaphors (neural network, bootstrap), or outright lies (normality, significance, error). It’s no wonder doctors only read GetWithIt™ updates anymore! To remedy this situation, Swiss Medical Weekly proposes here an interactive learning tool that will turn you into an instant expert in current research methodology.
To proceed, consider each clue, close your eyes, and let free associations flow. Soon a statistical or epidemiological phrase will occur to you. Do not over-think it. For instance, if the clue is “Final ranking at a beauty pageant?” a valid answer is “Miss classification”. Get it? Misclassification! Don’t worry about sloppy spelling or awkward grammar. Click on “show answer” to verify that you got it right. If you are unfamiliar with the scientific concept, read the comment (if you don’t get the joke, you are on your own).
Let’s play!
1. Final ranking at a beauty pageant
Miss classification
(misclassification) Error in the measurement of a risk factor or a disease, which typically weakens the observed association (misclassification bias)
2. Blueprint for an airport luggage scanner
Case-control design
Type of study in which patients who have a disease are compared to controls who don't
3. Mindset of a helpful doctor
Intention to treat
Analysis of a clinical trial that includes all participants as initially randomised (regardless of actual treatment received)
4. Possibility of your crazy uncle dropping by
Relative risk
Ratio of risks in the exposed and the unexposed
5. Editing a captain's entry
Log transformation
Application of the logarithm function
6. Sigmund Freud's memoirs
Meta-analysis
Statistical synthesis of the results of several studies that address the same research question
7. Ultimate goal of rehabilitation?
Normal function
Bell-shaped probability density distribution; synonym of Gaussian distribution
8. Slip-up by an evil nerd?
Mean square error
In analysis of variance, the sum of squared differences between observed and predicted values divided by the degrees of freedom
9. They are raised at a nudist camp
Odds of exposure
Odds are a function of a proportion, namely the proportion p divided by its complement (1-p). Exposure is synonym of risk factor.
10. Scale that goes from 0 (sane) to 10 (Hannibal Lecter)
Psycho metric
(psychometric) Related to classic methods for the measurement of unobserved attributes (often psychological variables)
11. Choosing Ronaldo over Messi
Forward selection
Method for selecting covariates for a multivariable model, which starts with the best univariable predictor, then adds the second best, etc.
12. Ask what you will be charged for water or electricity
Cost utility
Type of economic analysis where costs incurred are related to health utility gains
13. Requirement for a double date
Two by two table
Common tool in epidemiology; e.g. tabulation of exposure by outcome
14. Outfitting the troops
Uniform distribution
Distribution that assigns equal probability (or density) to all values on a given interval
15. Telling a friend that she was right to get a divorce?
Split-half validation
Validation method where a model is built on one half of the data and then applied to the other half
16. Urinalysis report
p-values
Probabilities of the observed result or more extreme results under the null hypothesis
17. They don’t make the news
Censored events
In time-to-event analysis, observations for which the event of interest has not occurred at the time of analysis
18. Recycling facility for plastics
Composite endpoint
Occurrence of any of several events that are considered equally relevant and therefore combined in a clinical trial
19. Cindy Crawford trying to save on groceries, e.g.
Parsimonious model
Statistical model that includes as few covariates as possible while explaining as much as possible of the outcome
20. It’s all a small window needs
Single blind
A clinical trial in which the patients do not know which treatment they receive
21. List of favourite neighbourhood watering holes
Bar chart
Graph that represents by the height of a bar a characteristic of interest (mean, percentage, etc.) for several subgroups
22. Hanging the flag upside down, e.g.
Standard error
Level of uncertainty of an estimator (i.e., a value obtained from the data) due to the limited sample size
23. Telephone contact of the health insurance department that preapproves health care expenditures
Number needed to treat
Measure of treatment effectiveness; the number of persons that must be treated in order to avoid one clinically relevant event
24. Immature behaviour by the planning committee
Logistic regression
Modelling method used for binary outcome variables
25. Pretending that you have won at roulette, e.g.
Monte Carlo simulation
Computer-based method that explores a phenomenon through the simulated behaviour of repeated samples
26. Iconoclast on drugs
High outlier
Unusually high value of a variable
27. Inside squabble
Within-group variance
In analysis of variance, variability that is not attributable to group membership
28. Dropping a four-letter bomb?
F-test
Statistical test used in analysis of variance
29. Two qualities of an effective parental admonishment
Sensitivity and specificity
Ability of a test to correctly identify the diseased (sensitivity) and the non-diseased (specificity)
30. Ultimate pluck
Confidence limit
Upper or lower bound of a confidence interval, which is the range of values of a parameter that are compatible with the observed data
31. When nobody comes to the fashion show
Discrete event modelling
Simulation technique of a complex process that can lead to one or several distinct events
32. Proportion of medical students who choose psychiatry?
Shrinkage parameter
In empirical Bayes estimation, the relative weight given to the overall mean when it is combined with the local estimate
33. Probability of winning the Ig Nobel Prize, e.g.
p <0.001
Low p-value that typically leads to the rejection of the null hypothesis
34. Neatly stacked stolen goods, presented in court
Pyramid of evidence
Hierarchy of scientific evidence based on the trustworthiness of the methods used (meta-analysis of randomised trials at the top and expert opinion at the bottom)
35. What differentiates solitary confinement, county jail and house arrest
Degrees of freedom
Number of independent bits of information that are used to compute a statistic or estimate a model
36. What a wide-receiver should engage in before the pass
Safety monitoring
Procedures for detecting threats to patient safety in a clinical trial
37. Enduring battery life?
Patient autonomy
Ethical principle that gives patients ultimate say in decisions that affect them, including participation in research
38. What the Civil Rights Act of 1964 aimed to do
Recall bias
Systematic error that can affect case-control studies, because the cases may remember their exposure history differently from the controls
39. Parliamentary committee
Representative sample
Sample drawn through a process that guarantees fair representation of the underlying population in the long run (e.g., at random)
40. First date, essentially
Interaction test
Statistical procedure that examines the effect of a variable as a function of another; e.g. a test to compare the effectiveness of a treatment in men versus women
41. Theory of evolution?
Adaptive design
Design of a clinical trial that incorporates future decisions based on accumulated data
42. Like someone who orders Bud Light when Westmalle is on tap
Tripel blind
(triple blind) Triple blinding ensures that the patient, the researcher, and the data analyst are unaware of treatment allocation
43. Important cricket game
Test of significance
Procedure for choosing between two hypotheses (typically “the treatment doesn’t work” vs “the treatment works”) based on observed data
44. Hence the saying “traduttore, traditore”
Translational trials
Studies that test whether basic science advances are applicable to patient care
45. Someone as crazy as you or me
Normal deviate
Normally distributed statistic or observation expressed in standard deviation units
46. Result of neglecting one’s workouts
Marginal conditioning
Treating marginal distributions as fixed when the association between two variables is examined; e.g. the row and column totals are considered fixed when a kappa statistic is computed
47. Tongue tied abbot
Non-informative prior
In Bayesian statistics, the a priori distribution of a parameter that reflects ignorance of its true value
48. Dilemma of the schizophrenic patient?
Multiplicity problem
Proliferation of “significant” results due to type 1 errors when a large number of tests is performed
49. Democracy?
Law of large numbers
Principle that describes the average behaviour of a procedure when it is repeated many times
50. Concession in Amazonia
Forest plot
Graph used in meta-analysis representing the results of several studies, with their confidence intervals, plus the pooled effect
51. Lead a working group at ISO
Direct standardisation
Method for removing confounding, where rates of disease observed in strata of the confounder (e.g., in age groups) are applied to a reference population
52. The custom of putting your boss on your papers, e.g.
Research protocol
Document that describes in detail all procedures to be used in a study
53. Possible causes of mistaken fraternity?
Alpha and beta errors
Type 1 and type 2 errors. The former is the probability of rejecting the null hypothesis when it is true, the latter the probability of accepting the null hypothesis when the alternate is true
54. Its rejection in science leads to acceptance in Nature (and vice versa)
Null hypothesis
Hypothesis of no association or no effect that the researcher hopes to reject.
55. Pissing contest, often
Non-inferiority test
Test of the hypothesis that a treatment is less effective than another by a pre-specified amount
56. Ceremony on the golf course
Link function
Mathematical function applied to the expected value of the outcome variable in generalised linear models; e.g. logarithm function applied to the number of events in log-linear models
57. Prince Philip saying “I do”, e.g.
CONSORT statement
Guideline for the reporting of randomised controlled trials
58. Getting stood up, e.g.
N-of-1 trial
Trial where two treatments are alternated at random within a single patient
59. Ironman deity
Goodness of fit
Measure of the adequacy of a model for a given dataset
60. The sinking of Noah’s ark?
Pairwise deletion
Method for dealing with missing data when correlations are obtained for >2 variables, where each correlation is computed on all available pairs
61. Replying to personal ads on a dating website
Disclosure of interests
Authors’ disclosures of financial or other links with private entities that may bias scientific conclusions
62. Flipping the USB key to make it fit in the slot, e.g.
Two-sided test
Statistical test that leads to rejection of the null hypothesis if the data deviate from the null value in either direction
63. When you wonder if the waiter understood what people wanted to drink next
Second order uncertainty
In modelling, uncertainty about the values of parameters (whereas first order uncertainty concerns values of individual observations)
64. The pride of aesthetes, and the bane of oppressed minorities
Discrimination
Ability of a diagnostic or predictive model to correctly categorise patients with and without a characteristic (e.g., a specific disease)
65. Fil__ng in _he bl_nks
Imputation of missing data
Replacement of missing data by plausible values
66. Checking the authenticity of a wooden relic during the Inquisition
Cross validation
(cross-validation) Method for verifying the properties of a predictive model
67. The number of times you have been fined for speeding or for disorderly conduct
Citation count
The number of times an article is cited in the scientific literature
68. Double take?
Peer review
Evaluation of research articles or protocols by experts in the field to aid the decision to publish the paper or to fund the study
69. What an effective doctor strives for
Good clinical practice
(Good Clinical Practice) Regulations for clinical trials established by the International Conference on Harmonization
70. Plan B?
Residual plot
In regression modelling, differences between observed and predicted values, plotted against predicted values
71. The long and the short of it?
Range of values
Lowest and highest values of a variable
Thank you for playing along!
Disclosure statement
No financial support and no other potential conflict of interest relevant to this article was reported.
Thomas Perneger, MD, PhD
Division of clinical epidemiology
Geneva University Hospitals
6 rue Gabrielle-Perret-Gentil
CH-1211 Geneva