Surgical Trials and Tribulations

Mohit Bhandari, Associate Professor of Surgery at McMaster University and Canada Research Chair in Musculoskeletal Outcomes presented "Surgical Trials (and Tribulations)" at the Hospital for Sick Children. He opened with a quote from Donald Gannon "where facts are few, experts are many".

He paid homage to Alvin Feinstein, David Sackett, Jim Wright and Ben Alman, and to the McMaster clinical epidemiology group Clarity, headed by Gordon Guyatt. The Clarity Group fosters evidence based medicine - "the conscientious use of current best evidence for making clinical decisions." The pillars of EBM are conscientiousness, current best evidence and decision making. EBM has been described as the best advance in medicine in the past 166 years and forms the basis for president Obama's Current Effectives Research program (CER), intended to guide the health care reforms currently underway in United States.

David Sackett devised the five levels of evidence. The highest level -1 -randomized control trials, 2- prospective trials, 3 -case control trials, 4- retrospective case studies and 5 - opinion. Only 10% of surgical trials are randomized trials and 56% of reports in the surgical literature are level 4 retrospective case studies. Drugs have a very organized pathway into practice, characterized by three phases and generally requiring thousands of patients in phase 3 before they are approved for general use. In sharp contrast devices can enter into the market under the 5-10K requirement of the US Food and Drug Administration. "If the device is deemed substantially equivalent to a currently approved device, it may be marketed immediately in the same regulatory category". So, acceptable experimental biomechanics gives direct access to the market for new devices. Not surprisingly, many of devices are subsequently pulled from the market as problems or failures are encountered. A recent example is the metal on metal hip replacement.

Randomized trials and metanalysis of randomized trials are increasing in the orthopaedic literature. However, there are a number of problems. Many randomized trials do not report whether the randomization was concealed from the operating surgeons to prevent biased entry of patients to secure a particular treatment assignment. There is a substantial and important difference in providers' skills. For example, in the randomized trial of hip fractures, residents did most of the non-replacement operations and consultants performed most of the replacements, resulting in a differential expertise bias. Blinding and loss to follow-up - which can be substantial, are rarely reported.

Nine of ten orthopaedic randomized trials are underpowered to detect significant effects. This is a sample size problem. Deveraux (1) has developed a thoughtful discussion of "p value fragility" related to minor changes in the incidence of measured events in small trials. The bone morphogenic protein study which showed a very significant 59 percent risk reduction would lose its significance if only three events were switched from the test group to the control group. To eliminate "this implausibility factor", large trials are required where two or three switches would not affect the p value. To reduce p value fragility, Mo recommends networks with a large number of hospitals in a large number of countries, for example the tibial fracture study conducted by 205 surgeons in 29 countries (2).

Funders are now stating that "we only want trials that will change practice" This puts an unmanageable burden on investigators to have the preliminary data from a large trial in order to qualify for funding to begin a large trial.

Industry drives innovation. 80% of orthopaedic surgeons have industry ties. The bulk of the money for research is currently going to clinical research organization- globally $18 billion, an amount that is increasing by 14% annually. It is important that these funds be captured by academic centers as Muhammad Mamdani is trying to do at the Li Ka Shing Knowledge Centre for the University of Toronto.

Mohit Bhandari

In order to conduct trials with large numbers of participants it is important for investigators to include China and India where there are large populations (40 % of the world population) and frequent events. In India there are road accidents every three minutes and highway deaths every ten minutes. The challenge is that infrastructure is lacking in these countries. It is difficult to conduct a trauma trial if there is no 911, and no trauma network or registry.

There are ethical issues associated with carrying out trials in developing or lower income countries. For example bone morphogenic protein, studied extensively in South Africa, was simply too expensive and essentially unavailable to South African patients following the FDA approval of the biologic for fracture use. One of the criteria for an ethical trial is the requirement that the agent proven clinically effective by the trial should be available to participants following the trial. For example, Dr. Bhandari discussed an ongoing multinational RCT evaluating the use of high pressure irrigation and soap in open fractures - two simple and inexpensive therapies that could be applied globally if proven effacious. He concluded with a recommendation that we "think big", not limit our trials to North America or Europe and emphasize China and India as we expand the use of randomized trials. During the discussion period the problem of surgeons reluctant to admit personal equipoise was raised. "I can't tell my patients that I don't know what I am doing". The solution recommended is discussion of clinical equipoise, the lack of a settled opinion in the informed medical community as to the best available treatment - despite an individual surgeon's personal preference.

The problem of academic credit was raised by Jack Langer. It is difficult to convince 205 surgeons that they will advance their careers by participating in a trial. This is an issue that universities must confront. Promotions committees have insufficient understanding and appreciation of the role of collaboration. PubMed now uses electronic listing of collaborators, and group authorship is very slowly gaining acceptance. Participation in committees other than the writing committee should be given more weight in the academic setting.

(1) Devereaux, P J; Bhandari, Mohit; Clarke, Mike; Montori, Victor M; Cook, Deborah J; Yusuf, Salim; Sackett, David L; Cina, Claudio S ; Walter, S D; Haynes, Brian; Schunemann, Holger J; Norman, Geoffrey R.; Guyatt, Gordon H. Need for expertise based randomised controlled trials, BMJ 2005, 330(2):88

(2) Bhandari M, Guyatt G, Tornetta P, Schemitsch EH, Swiontkowski M, Sanders D, Walter SD., Randomized trial of reamed and unreamed intramedullary nailing of tibial shaft fractures. Study to Prospectively Evaluate Reamed Intramedullary Nails in Patients with Tibial Fractures; J Bone Joint Surg Am. 2008 Dec; 90(12):2567-78.