Question. You read about a possible new acne treatment, how do you decide whether to try it or not? Maybe you are like most people and you go online to read user reviews and experiences? In this post I’ll share with you 3 reasons why you should take those reviews with a grain of salt.
This all started when a reader emailed me asking about urine therapy – he had read my About Me page where I listed it among the things I had unsuccessfully tried. I told him I think it’s useless, devoid of reality as science understands it, and that I don’t recommend anyone wasting time with it.
He then replied the following:
That’s unfortunate. I hope you don’t get upset if I ask, but were you doing it correctly? E.g., you’re supposed to use aged (about a week old) mid-stream morning urine, and you apply it after exfoliating your skin and you don’t rinse it off & leave it on overnight. Did you follow these steps? I whole heartily trust your info because you’re a skin guru and you’re intelligent, but urine therapy has gotten rave reviews that i can’t quite ignore.
Let’s take a moment to examine the underlying assumptions here. This person has read good reviews of urine therapy and takes them more or less at face value. The assumption is that because there are so many good reviews online they can’t all be wrong and therefore there must be something to urine therapy. Therefore me not getting good results with it means I didn’t do it correctly.
Note that I’m not writing this to mock the person. The assumptions he makes come naturally to humans, and I’ve been guilty of making the same countless times.
There’s no doubt that user reviews can be very helpful, but they can also be very misleading – especially if you don’t know how to read them. In this post we’ll go over 3 reasons to be skeptical of user reviews for exotic natural acne treatments.
People are often wrong
Yes, I know that’s a horribly arrogant thing to say. And I’m not pretending that I know better than the people writing the reviews.
Let me tell you a story of what happened to me a while back. I was convinced that sodium laureth sulfate (SLS) caused my scalp acne. Countless studies showed it’s a strong skin irritant – in fact, it’s used in studies to provoke skin irritation. And here I was like an idiot applying it on my scalp every day!
So I switched to a natural, SLS-free shampoo. And you know what happened? My scalp started getting better. It didn’t clear completely but I could see a definitive improvement. I even wrote a post blaming SLS.
Fast forward some time and my scalp acne came back – despite the definitive lack of SLS.
This is a good example of the confirmation bias fallacy. As humans we like to be right, and we tend to pay more attention to information that confirms what we already believe (or want to believe). We also tend to ignore and discount information that challenges our beliefs – i.e. shows we are not right.
That’s exactly what happened to me during the SLS fiasco. I was primed to believe SLS was the culprit and I saw everything through that lens.
Confirmation bias alone makes user reviews highly unreliable, and combining it with the next point makes things exponentially worse.
Reviews are just a snapshot in time
Many pro-athletes believe they will be jinxed, or cursed, if they appear on the Sports Illustrated cover. There’s even a name for this urban legend, The Sports Illustrated Cover jinx.
What really happens is that SI cover features athletes and teams who did exceptionally well, which, by definition, is an exception to their normal performance. The SI cover jinx shows a statistical phenomena known as regression to mean; plainly put things return to normal.
Acne, like most health problems, waxes and wanes over time – without any apparent reason. You can see how this could create tons of ‘false positive’ user reviews, especially when combined with the above discussed confirmation bias.
People often try new things when their acne is worse than normal. Confirmation bias ensures that improvements in acne are attributed to the new treatment – even if it’s just normal fluctuation. Seeing that the new, natural treatment is working, the person gets excited and shares the excitement with the world, i.e. a review/testimonial gets written.
I believe that this explains all those stories where a person complains that a treatment worked initially but then stopped working, they often claim that acne became resistant to it.
You can find positive reviews for anything
Keeping in mind what we have discussed so far it shouldn’t come as a surprise that you can find positive reviews for absolutely anything. Even for things that don’t do anything. Here are a few examples.
At the turn of the 20th Century American doctor Albert Adams was selling machines (called The Dynomizer). He claimed the machines diagnosed and treated almost any illness. His machines were widely used, with millions of people swearing by their effectiveness. After he died the machines were opened and found to contain random machine parts. In short, the machines did nothing and yet thousands of practitioners believed into them and used the machines to diagnose people.
And let’s not forget the radium craze in early 1900’s.
When Marie Curie discovered radium in 1898 no one was really aware of the dangers of radiation, in fact it was hailed as a miracle cure for any number of ailments from tuberculosis to sagging skin to impotence. It would only be many experiments later that the risk would become apparent. In the meanwhile, a radium craze that swept through France, America, Germany and Italy had people convinced that radium held miraculous health benefits. Just about every product, including chocolates, beauty creams and toothpaste, was made and marketed as radioactive.
Radium was added to drinking waters, one of which claimed that drinkers are guaranteed to “sparkle with energy”, while another brand claimed to be “the most radioactive water in the world!” Convinced that it’s a miracle cure, people couldn’t get enough of radium. Of course, it was later discovered that radium is extremely dangerous and countless people suffered horrific radiation poisoning. Yet, that didn’t prevent people from believing radium is the best thing ever.
For modern examples, you only need to look at acupuncture and homeopathy. Both of which have been conclusively shown to be no better than placebo, but this hasn’t prevented thousands of people swearing they are effective.
Question, how do you tell the difference between a positive review for a real treatment vs. one for an illusory treatment? You can’t, and that’s the real problem.
Now that we understand how unreliable user reviews can be, let’s look at a better way to evaluate new treatment options.
How to evaluate new acne treatment options
In medicine there’s a concept called biological plausibility. In simple terms it considers how well a treatment fits to our current understanding of how the human body works. Treatments that work through existing cellular pathways and mechanisms are considered plausible. Similarly, treatments that rely on unproven/undiscovered mechanisms (meridians, memory of water, etc.) are considered implausible.
Every effective treatment I’ve talked about on this site works through the established hormones/inflammation mechanism I talked about here. Eating less carbs and sugar works because they reduce insulin load on the body. Topical and supplemental antioxidants work because they counter oxidative stress.
Similarly, I was able to almost completely eliminate my scalp acne with supplements that balance gut bacteria and heal the gut. No study has ever shown those supplements work against acne, but knowing how they affect the gut I was able to make an educated guess they could also help my acne – and they did.
I dismissed urine therapy because of utter lack of biological plausibility. Nobody has ever shown that drinking or applying urine on your face affects the known mechanisms that lead to acne.
So when you look at the reviews for urine therapy you have to consider which is more plausible. That it works through some new and yet to be discovered pathway (highly unlikely), or that the reviews reflect known human cognitive biases (likely).
So next time you consider some new acne treatment, pay less attention to user reviews and more attention the ‘how it works’ section. Treatments that affect hormones (androgens, insulin, IGF-1), inflammation or the gut are at least plausible. Be very skeptical of treatments that supposedly work through some other way, and be especially vary of anything that claims to get rid of acne by detoxifying or purifying. Detoxification is nonsense and toxins don’t cause acne.
Learning from other people’s experience can be helpful but can just as well lead you astray. The human brain comes bundled with countless cognitive biases that make it difficult to assess complicated and emotionally-charged issues.
In the world of Harry Potter magic may be able to heal, but in the real world treatments must work through existing cellular mechanisms. When considering exotic and alternative treatments, pay attention to claims of how the treatment works.
Scientific research has linked hormones and inflammation to acne. Treatments that claim to affect the two are at least plausible. Be very skeptical of treatments that supposedly work through some other mechanisms.
What do you think? How do you approach new treatments you haven’t tried?