Do Umbrellas cause the Rain?

By whitneykf

Join us for an exploration of why COVID medical data was so confusing and tools that you can use to interpret medical testing and data. With guest Dr. Natalie Alexander, we cover the politicization of medical data,  understanding what is behind medical testing, not mistaking treatment for prevention, the hierarchy of studies and more!

no handout

As a medical community we rely heavily on evidence-based medicine (aka the studies said so) but we don’t talk about the limitations of studies much. Like the power of placebo, the myth of the p value, or how many studies aren’t applicable to clinical practice. To highlight this, we explore what happened with the pandemic.

Why was COVID data such a crazy mess?

  • Politicization of Medical data.
    It became about what party you support rather than actual proof and that was not allowed to be questioned. Censorship of studies, deleting youtube and facebook posts and even researchers getting fired made everyone afraid to speak out
  • Reliance on anecdotal evidence and/or case reports.
    Not the best sources of information as easily tainted by bias.
  • Rumors reported as facts.
    Media reported theories and editorials as facts to get views because most folks don’t fact check them. Go ahead, I dare you, click on those “study links”
  • Thinking treatment = prevention. (it doesn’t)
    hydroxychloroquine or other touted treatments may work or may not, but giving it as prevention is making one too many assumptions. We don’t give everyone antibiotics to prevent UTIs or Ebola– if you remember giving everyone antibiotics would be worse. Risk vs benefits not balanced.
  • Uncertainty.
    Our unwillingness to function without guidelines or a “right answer” increased the hysteria around COVID. When medical providers panic or falter, everyone around you panics. We must remember we are leaders and communities look to us to lead the way by example.
  • Confounding factors.
    Humans are complex bags of biology and its often hard to account for differences that might have made all the results change. Like how much sleep that person got, what their diet was, whether they actually remembered to take all the medications, their genetic receptors, and sometimes they could have just gotten better with time and the medication didn’t make a difference at all.

Hierarchy of research studies:

Think critically about what is behind the study numbers?

Behind those study numbers:

  • Are politics – motivating interpretations and what is being reported to you
  • Are statistics- which can be skewed or changed midway through
  • Are humans- who could have biases or make simple mistakes
  • Are academic jobs – which needs papers to publish (its why there are so many and may not be relevant) and constantly seek the Myth of the p value= p < .05 means statistically significant but often not even close to clinically significant or something we could use in clinical practice daily
  • Are funding – which can twist motivations and results
    • Pharma co who have a vested interest or government $ very limited
    • Not a money maker to compare generics
    • Who even friggin knows re: herbals, OTC stuff
  • Are poor study designs- which can lead to wrong conclusions
    • Hierarchy of studies (see below)
    • Who or what is being studied? Animals or petri dishes or humans?  Does it represent the patient population we would use this medication on? (is it all white men? is it all the same age adults? pediatric vs pregnant vs races vs genders)
    • What is being compared? New thing to standard of care or to a placebo? 
    • What is the goal/outcome the study is measuring and is that relevant to your practice? 
    • RAMBO method
  • Are tests- which may or may not be good quality

What makes a good medical test?

There are 3 components:

  • Sensitivity  – false positives because we want NEGATIVE result to be infalliable = a good SCREENING test should be sensitive because we don’t want to miss anyone so we can live with a few false positives but we want to catch every true positive / no false negatives
  • Specificity  – false negatives because we want POSITIVE result to be infalliable = a good CONFIRMATORY test should be specific because we want to know whether the thing we are testing for is actually there or not
  • Standardization – if we test you 3x are all the results the same? Aka reliability. How much can I TRUST the test result?

Did you like this?  Similar podcast topics:  How to think not what to think.

Trusted Sources

We know you can’t spend all your time looking at studies so you have to have some trusted sources. These are ours (but check yourself every now and then, especially if its controversial) 

Share on social media:

Share on facebook
Facebook
Share on twitter
Twitter
Share on reddit
Reddit
Share on email
Email