Untitled Page

Sabermetrics and Surgical Outcomes

Justin Dimick
Justin Dimick

Justin Dimick, a former American Greco-Roman varsity wrestling champion at Cornell, is currently practicing general and minimally invasive surgery the University of Michigan. He trained in epidemiology at Dartmouth and completed his medical and surgical training at Johns Hopkins University. His research is conducted at the University of Michigan’s Centre for Healthcare Outcomes and Policy.

At December’s University Rounds on Surgical Epidemiology and Statistics, Justin used a picture of the University of Michigan football stadium filled with over a 100,000 people to drive home the point that there over 100,000 deaths each year following surgery in the United States. He asked our audience “if surgical performance follows a bell-shaped curve, which of you is in the lower half of the curve?” He asks this question whenever he speaks; it universally elicits a negative response. Like the 95% of teenage drivers who consider themselves to be in the top 5% in driving performance, we tend to overestimate ourselves. At the current time, there is great interest and attention focused on the outcomes of surgery.

The National Surgical Quality Improvement Program (NSQIP) - the American College of Surgeons Program (Surgical Spotlight, Winter 2010, pp. 14-15 or http:// www.surgicalspotlight.ca/Article.aspx?ver=Winter_201 0&f=KoCaterpillarGraphs) on quality improvement, teaches that “the road to quality improvement is paved with DATA.” Justin used the popular movie “Money Ball”to illustrate how important is to use the right data to improve outcome. The manager of the Oakland Athletics, Billy Beane (played by Brad Pitt) was able to bring his team into the playoffs regularly, and establish an amazing record with the lowest budget in the major leagues ~ 40 million dollars per year. He outmanaged the New York Yankees despite their annual budget of 226 million dollars. Beane relied on sabermetrics - the mathematical and statistical analysis of baseball records originated by Bill James. James recognized that the ability to get on base was what scored runs and won games, not the usual metrics used to choose and recruit players, such as running speed, fielding dexterity, and batting average. Similarly, the usual metrics for measuring or predicting surgical outcomes may not be using the right data. The commonly accepted formula is: Severity of illness + quality of care + random error = outcome.

But severity of illness as a risk factor is overemphasized in Justin’s view. Though predictive of complications, it is the ability to rescue from complications, not their incidence, that determines hospital mortality. That is why teams, rather than individual surgeons, account for lower mortality at large volume hospitals. The individual surgeon proved to be an important factor in only a few operations, like carotid endarterectomy. The team proved more influential in most operations like pancreatectomy. These findings were recently published in a landmark article in the New England Journal of Medicine. They clarify and emphasize the critical importance of the quality of care.

A second helpful contribution to improving outcomes was a simplification of the burdensome documentation process of care. In a recent analysis of the NISQUIP data, Dimick and his colleagues found that they could simplify their data collection, as five variables gave as robust a correlation with outcome as the 20 variables currently collected in the NISQUIP program.

Alice Wei asked about analyzing the rescue process. Justin responded that this is a complex component which is currently being analyzed using qualitative and quantitative methods. For example, what is the impact on outcome of adding of a rapid response team? David Latter asked whether mortality was overused. It can be accurate, but impoverished in that it does not address some of the significant goals of surgery. Dimick responded that mortality had proven to be very unhelpful in analysis in bariatric surgery, as there were only two deaths in 6000 analyzed cases. The complication rate is far more significant. Gail Darling asked about the analysis of length of stay, which proved to be much greater following esophagectomy in high volume hospitals. This was a reflection of the rescue process at low volume hospitals, which probably led to earlier deaths with less prolonged rescue procedures.

Both Dimick in Michigan and David Urbach in Ontario are analyzing ‘what goes on in the operating room’ as a factor in outcome. Perioperative antibiotics and other nonsurgical factors have been well analyzed. Technical proficiency and innate skill are probably high leverage components, but these are inadequately studied. The Michigan group is using the skills lab to assess innate surgical skill and videotaping operating room cases to assess technical proficiency. They use an OSATS scale (developed by Richard Reznick and his colleagues) to assess skill of the operating room team. They find that the details of the operation that are the usual metrics, such as stapled versus hand -sewn anastomoses have not proven to be potent factors.

M.M.

(1) Ghaferi, A.,John D. Birkmeyer, J., Dimick, J.Variation in Hospital Mortality Associated with Inpatient Surgery; N Engl J Med 2009; 361:1368-1375




Skip Navigation Links