How to avoid the ‘shazam’ trap in sports medicine

– Written by Boris Gojanovic and François Fourchet, Switzerland

Who has fallen for that therapeutic modality, you know, the one for which you know there is no real evidence of effectiveness? There may even be some pretty good evidence that it is ineffective (although most probably harmless). Most of us, of course. Sports and exercise medicine (SEM) practitioners try to apply rigorous science as often as possible in clinical practice, even when facing extreme demands from our patient-athletes. But sticking to evidence-based treatments is not always easy.
Authors’ disclaimer: we have applied and suggested therapies, for which we are pretty convinced that the available evidence is either non-existent or non-supportive.

Rolf Dobelli is a novelist, thinker and entrepreneur. In his book The Art of Thinking Clearly¹, he pinpoints the assumptions, biases and illusions that shape the way we think and make decisions. We all fall for these cognitive biases as we make decisions, whether for ourselves or when working with our patient-athletes. The further you delve into his 99 short chapters, each describing one of these phenomena, as a healthcare provider you start connecting the dots between what our patient-athletes search for, do and expect, and what we are willing to try, offer, recommend or advise. Sports medicine is an area where borders are ill-defined, lines blurred. Science can be extremely specific, but also looks like a big piece of Swiss cheese in the practical setting: looks and tastes yummy, but can be hard to digest, and is filled with holes that we happily ignore and mix with the good bits. This leaves the door open to many biases known to behavioural psychologists for a long time.

In this article, we will look at some of the cognitive and systematic flaws that can cloud our judgment, raise the public’s and the athlete’s expectations to irrational levels and ultimately obscure the art of treating clearly. Throughout the paper, we will refer to the terms used in Dobelli’s treatise, succumbing in the process to the mother of all biases, confirmation bias.

TO SHAZAM

If you work with athletes, you work with superheroes. They achieve what was deemed impossible years ago. They keep pushing the limits (and your limits) and they defy business as usual as we know it. In the 1940s, Bill Batson’s alter ego, Shazam (Captain Marvel), summoned extraordinary powers to fight against evil (Figure 1).

As SEM professionals we often feel compelled, or are expected to produce shazam-like effects, to restore physical performance capacity as quickly as possible (if not immediately). Sometimes these demands extend to chasing ‘marginal gains’ to make the difference between yesterday’s loss and tomorrow’s win. This would constitute a ‘shazam act’. Although it may be possible on very rare occasions (think benign symptomatic tachycardia that is converted through the Valsalva manoeuvre), most often it leads to disappointment and frustration for both the SEM professional and the patient-athlete.

People who shazam have much in common with those who excel in the art of quackery, charlatanism or snake-oil sales, relying on similar mechanisms: they are full of scientific terms and will quote references, putting on a mask of benevolence and camouflaging under a cloak of science. But maybe the flipside is that these people are not being disingenuous. Perhaps they are deeply convinced that they are providing an outstanding service by treading the waters others are too timid to approach. Maybe they interpret science differently? Perhaps the science was delivered in an unclear way, fuelling enthusiasm to develop new health and performance theories and therapies.

In our SEM shazam plot, there are three main characters: The Good, the Bad and the Ugly (Figure 2). Each part can be played alternatively by three protagonists: the SEM professional, the demanding patient-athlete and ‘Big Media’.

HOW SEM PRACTITIONERS TRY TO STAY AFLOAT

The bedrock of SEM is passion. Most professionals have some form of personal connection to the sporting world. We tend to be involved in sports we are closer to and relationships with sporting organisations and athletes may have personal components beyond the professional ones. SEM has developed from a field-based discipline into a scientific domain in its own right, although often the rules of the field take precedent for many arguable reasons. Concussion management provides many examples for this².

How can we navigate the troubled waters between fast, field-guided action and reasoned evidence-based therapy to avoid the many traps? Most agree that SEM’s role is to protect athlete health, restore optimal performance capacity and help athletes achieve their full potential in a safe, fair and legal way. For that purpose, SEM practitioners are ready to dedicate extra time, be extremely available and act faster than medically necessary. Working in sports requires adaptability, reactivity, innovation and creativity. One can argue that science does too, except the time frames are very different. Science takes a longer path full of codes, whereas the sporting setting needs all of this here and now³.

Take an athlete with worsening, symptomatic Achilles tendinopathy. Major competitions are coming up. Load management is a priority, but it is a ‘boring’ course of action. The practitioner feels compelled to do more for the athlete. This is called action bias (another example of action bias is the football goalkeeper defending a penalty kick. He or she will usually always jump to one side, even though he or she could stay in the middle of the goal – the likelihood of stopping the kick is the same. But if the ‘keeper did not move one way or the other, he or she would be perceived as lazy).  Coming back to the athlete with Achilles tendinopathy; your institution has recently acquired a shockwave therapy machine (availability bias) and your head of department has successfully treated Achilles tendon pain with it (authority bias). Not everyone has the device at hand, so a form of scarcity error will be at play (the rarity of the device makes it appear more effective than it is). When ‑ or if ‑ the athlete recovers and performs well, the clinician will gain satisfaction and maybe fame along the way (self-serving bias). Sometimes we have spent time, effort and money learning new therapeutic techniques that may not be validated by rigorous science. Yet we apply them nonetheless, falling prey to effort justification and sunk-cost fallacy.

On the other hand, the strict application of evidence-based principles or guidelines can also lead to problems. Single case studies may not be valid for the situation at hand. Meta-analyses may be misleading if they pool data from biased sources⁴. Applying evidence can be difficult. Example: a new study shows positive effects of a supplement to reduce the severity of upper respiratory tract infections (publication bias always present), so the practitioner starts prescribing it to athletes. A few years later, the same supplement is debunked and shown to be detrimental as it negatively impacts muscle adaptation to training. Science was applied in the decision making process, but the athlete probably did not benefit from the science.

Applying new scientific findings ahead of the field or simply making the latest reported advances available to athletes seems obvious. After all, it is about those marginal gains, and provided we respect the 'primum non nocere' principle, this course of action might seem appropriate. But the problem is, when we do this, we contribute to the hype and raise expectations with limited room for backpedalling.

TIPS FOR SEM PRACTITIONERS

1. Don’t overstate the effect of a single treatment (single-cause fallacy).
2. Make sure you have your bases covered with the most likely beneficial course of action before using sparingly (if at all) the slight added effect of the latest novelty.
3. Follow Warren Buffet’s advice⁵: find your circle of competence and stick to it. The rest belongs to your network that completes the expertise you need to treat clearly.

PATIENTS, ATHLETES AND EXPECTATIONS

We mentioned athletes are superheroes. They are trained to believe they can overcome all obstacles and prevail through the sheer power of their determination. Of course history teaches us that most athletes will not reach the pinnacle of their sport, however hard they try. They must, however, obliterate this reality to adopt an optimistic faith in success. Managing expectations at all levels is crucial. From this flows the necessity of an egocentric attitude, literally: centred on themselves, in praise of their individuality. This will open the door to many quacks. It becomes easy to develop a rhetoric of invincibility through cure-all remedies or specially-developed formulas that address the specific needs of the individual. There are many potential cognitive errors here: It’ll-get-worse-before-it-gets-better fallacy – predicated in this way, the quack cannot lose (consultants make their living on this one), scarcity error, fundamental attribution error, and one which should not be underestimated, affect heuristics (deliver the advice with empathy and compliments to get most bang for your snake oil buck). Patients may also adopt similar thinking. Recent developments in medicine, from genetics to personalised medicine reinforce the expectation of ‘anything’s possible’.

Fuelled by media hype, the demand for novel therapies is fast-growing. Platelet-rich plasma becomes a must-have commodity (Figure 3) and the more expensive it is, the greater the belief (placebo effect?) it will work. Hyperbolic discounting is at play (‘I want it now, no matter what’). If the treatment fails, cognitive dissonance comes to the rescue (reinterpreting negative outcomes positively).

When treatments are combined, often the salience effect will surface: an overreached athlete takes a couple of weeks of rest and starts to take spirulina supplements. He recovers his energy and performance level 2 weeks later; is it the rest or the fancy trending supplement that made all the difference? It may be harmless at first sight, but salience is also at play when more dramatic therapeutic choices are made. For example, in the presence of subacromial pain syndrome or medial meniscopathy, surgery has been advocated for a long time, despite the fact that conservative treatment is most often effective, cheaper and obviously less invasive. The only thing is, the scalpel is attractive⁶. Salience effect again.

Medicine has entered the age of shared decision-making, where practitioners are compelled to disclose, explain, share and discuss. This is where patients can be susceptible to information bias. Too much information kills information and decision fatigue ensues, potentially turning patients over to people who have simpler (or simplistic) explanations and therapeutic options. These solutions will entail some level of fallacy of single cause (e.g. the correction of foot pronation as the cure for all lower limb – and more – maladies) (see Breakout box 1).

TIPS FOR SEM PRACTITIONERS

1. Sharing information is good, but cut to the chase – understand what the patient expects and work from there.
2. Make the good treatment sound just as appealing as the shazam therapy.
3. Empathise and don’t make the mistake of the conjunction fallacy (too many details in a story make it less likely to be true).

READ WITH A CRITICAL EYE. DON’T BELIEVE THE HYPE!

First of all, take any headline that ends with a question mark (e.g. can X cure cancer? or can Y heal muscle tears faster?). Now answer systematically, no. You will find this to be correct most of the time. Betteridge’s law of headlines can help us identify an oversold story: characterised by weak to non-existent facts, an oversold story is built on tenuous hypotheses. This is sensationalism. But it does sell well, since we are all likely to continue reading – searching for the magic bullet. Oversold stories are an increasingly common problem, as readers of any media tend to spend less and less time on a piece, usually reading on a mobile device. Catching the eye and capturing the ever shorter attention span requires sensationalism.

In oversold stories, the halo effect will be at play: if it glitters, it must be gold! Key giveaways are often innovation, technology, ancient roots and celebrity endorsement, among others – the whole package used by advertising professionals. If you dig into the claims, you will find that they are usually unsubstantiated, but the damage has been done. The distorted message sticks as we all hope that the headline carries some truth. Unfortunately, the stories behind the headlines tend to die quickly, losing relevance as soon as the next issue of the media outlet arrives, due to the necessity to continually report ‘new’ news. This is the news illusion.

Second, we have seen in recent years the emergence of new conditions that seem pushed into the headlines. Diseases that were non-existent or rarely present before now seem to be at the front of everyone’s mind. Take lactose or gluten intolerance, for example. All of a sudden, we are confronted with an epidemic of digestive issues, which make for beautiful stories by mixing the best elements in any health saga: demonisation of food processing (and ‘Big Food’), the nostalgia of a lost paradise past where food was ‘natural’ and therefore ‘healthy for the gut’ and anecdotes of extreme longevity or stamina. Social proof becomes a problem, as more and more people embrace these theories and apply the (most likely) unnecessary dietary restrictions. Athletes are not immune to this⁷. And of course, the media may be influenced by the development of a vast, profitable (and ever-increasing) industry behind special-requirements food products.

According to Caulfield, an author and health law expert, the “rhetoric of revolution is everywhere, […] stem cell, microbiome, nanotech, genomic, personalised medicine revolution”⁸. We highly recommend reading his books on the topic of health messages. Science and sports are popular topics in the media and the combination makes for a good sell. It also comes as no surprise that scientists can fall prey to the appeal of widespread and instantaneous (although often short-lived) fame, by reporting small findings in a hyperbolic way. They may be encouraged to do so by their institution’s media department and editing/distortion might happen along the way.

TIPS FOR SEM PRACTITIONERS

1. Don’t read the news. But if you do, before you make up your own mind, assume the answer to headlines is NO.
2. Don’t use hyperbole to overstate your research findings, even when media experts nudge you in that direction.

A CALL TO BE CONSCIOUS OF THE POTENTIAL FOR THINKING ERRORS

We have tried to illustrate some of the major cognitive errors that cloud human reasoning, leading to suboptimal decisions and erroneous conclusions. When foul play is involved, the Bad and the Ugly show up and quackery takes over. Joint cracking, vitamin popping, snake oil, hair analysis, immune boosters, detoxification and balance restoration are on the menu. A large industry and the media contribute to spreading the beliefs of treatment benefits and social networks magnify the message using powerful storytelling. The promoters of these treatments use all the cognitive mechanisms in their bag of tricks, but now you know how to spot them.

However, it is often more difficult to spot our own cognitive errors as SEM clinicians when we apply some therapeutic modalities (vitamin D, extracorpeal shockwave therapy, platelet-rich plasma, various supplements, to name but a few). Figure 3 describes the recipe for Shazam therapies.

Awareness of these cognitive mechanisms allows the SEM practitioner to thoughtfully reflect on the optimal course of action, while avoiding decision paralysis. Some can then be put to good use, like the illusion of control – the false belief that we can influence the course of something over which we have no clear sway ‑ or affect heuristics. We can do this by making sure the athlete benefits from all established therapeutic techniques, while being aware of the power hidden in communication skills. For most practitioners, this is second nature. Although we may not be fully aware of all mechanisms, we apply some ‘remedies’ in the best interest of our patient-athletes, in an effort to incarnate 'The Good' practitioner.

With regard to science and the urge to stick to it, it may be worth recognising a few pitfalls. First, orthopaedics and sports medicine sometimes suffers from a lack of sound and valid science to base everyday decisions on. As Lohmander and Roos point out, “clinical impressions can be deceiving”, and we need to start recognising the major reasoning flaws and interpret the available science correctly, applying it when applicable⁹.

How then, do we deal with Ioannidis’ statement that “most published research findings are false¹⁰”? One way is to remember that the absence of evidence is not evidence of absence¹¹. Case reports give ideas, which become small observational studies, which, in turn, may become state-of-the-art intention-to-treat valid clinical trials. While research is important, the fast-paced action of the sporting world continues and success will be achieved regardless of the results of fancy scientific trials. The SEM field thrives on the passionate dedication of its actors, who go well beyond sports medicine staff, to include coaches, psychologists, trainers and athletes themselves. They all come up with innovative and creative solutions. These solutions may not all be scientifically supported, but they certainly tend to get a pass in the real athletic world and contribute to the art of treating.

To conclude, we leave the last word to a great philosopher of science, Karl Popper, who theorised about what differentiates science from pseudo-science: “the criterion of the scientific status of a theory is its falsifiability, or refutability, or testability”. Good science can be tried and proved wrong by better science, whereas any pre-emptory decree (Shazam!) cannot be considered scientifically true.

Boris Gojanovic M.D.

Health and Performance Medical Director

Hôpital de La Tour

La Tour Sports Medicine

Swiss Olympic Medical Center

Meyrin, Switzerland

Attending Associate Physician

Sports Medicine

Lausanne University and Hospital

Lausanne, Switzerland

François Fourchet P.T., Ph.D.

Hôpital de La Tour

La Tour Sports Medicine

Swiss Olympic Medical Center

Meyrin, Switzerland

Contact: boris.gojanovic@latour.ch

References

1. Dobelli R. The Art of Thinking Clearly.New York: Harper 2013. p. 384 .
2. Jenkins G. Smith 'snake dance' prompts change to concussion trial. ESPN Sportsmedia Ltd 2013. Available from: http://en.espn.co.uk/scrum/rugby/story/196939.html [Accessed August 2016].
3. McCall A, Davison M, Carling C, Buckthorpe M, Coutts AJ, Dupont G. Can off-field 'brains' provide a competitive advantage in professional football? Br J Sports Med 2016; 50:710-712.
4. Weir A, Rabia S, Ardern C. Trusting systematic reviews and meta-analyses: all that glitters is not gold! Br J Sports Med 2016; 50:1100-1101.
5. Buffett WE. 1996 Chairman's letter. Berkshire Hathaway Inc 1997. Available from: www.berkshirehathaway.com/letters/1996.html. [Accessed August 2016].
6. Jevne J. The sexy scalpel: unnecessary shoulder surgery on the rise. Br J Sports Med 2015; 49:1031-1032.
7. Lis DM, Fell JW, Ahuja KD, Kitic CM, Stellingwerff T. Commercial hype versus reality: our current scientific understanding of gluten and athletic performance. Curr Sports Med Rep 2016; 15:262-268.
8. Caulfield T. When we hype our science, discoveries are diminished. The Globe and Mail Inc 2016. Available from: www.theglobeandmail.com/opinion/when-we-hype-our-science-discoveries-are-diminished/article29984407/ [Accessed July 2016].
9. Lohmander LS, Roos EM. The evidence base for orthopaedics and sports medicine. BMJ 2015; 350:g7835.
10. Ioannidis JP. Why most published research findings are false. PLoS Med 2005; 2:e124.
11. Altman DG, Bland JM. Absence of evidence is not evidence of absence. BMJ 1995; 311:485.
12. Krabak BJ, Hoffman MD, Millet GY, Chimes GP. Barefoot running. PM R 2011; 3:1142-1149.
13. Gunderman RB, Sistrom C. Avoiding errors in reasoning: an introduction to logical fallacies. AJR Am J Roentgenol 2006; 187:W469-W471.