The Problem with Google's Health Knowledge Graphs

Earlier this year, Google changed how it presented health search results. It added a new box to its search results it calls a “Knowledge Graph.”

Apparently this new product came about because a Google product manager had a hard time finding information about a concussion, using — you guessed it — Google. Believing that health information is different than all other information people search for, Google decided to start becoming a health information publisher instead of a search engine.

And when you get into the publishing business, well, you better know what you’re doing. Can a search engine company also offer vetted health information you can trust?

The answer is unclear.

The new health Knowledge Graphs, rolled out earlier this year, briefly describe the condition or disease (and even more information if you click on “More about this condition”). They also offer a brief description of the symptoms associated with the condition, as well as its treatments. These Knowledge Graphs are displayed inconsistently in Google’s search results, and may contain incomplete information or a “summary” of symptoms, rather than that actual, full list of symptoms for a condition.1

Google Tells You Depression is a Caused by a “Chemical Imbalance”

If you type in “depression symptoms,” you’ll get a colorful illustration that is eye-catching. It says, “Chemical (neurotransmitter) imbalance leads to depression” and features a woman’s head with a brain cutaway lit up (apparently they’re supposed to be neurotransmitters).

But as Dr. Ron Pies, Clinical Professor of Psychiatry, Tufts University School of Medicine and a regular contributor to Psych Central noted here back in 2011, the chemical imbalance theory was never taken very seriously by those who understand brain chemistry. After three decades of research into this theory with little result, no serious brain researcher believes that depression is caused by a “chemical imbalance” in the brain.

The actual fact is that we do not know what causes depression. That simple but important fact is not to be found in the Google Knowledge Graph on depression.

Yet the first thing you see when you look up this disorder on the world’s largest search engine is that depression is caused by a “chemical imbalance.” How could they get something so very basic so very wrong?

Google First Denies It’s an Issue, Then Claims 11 Doctors Reviewed It

Depression is medically treatableA spokesperson for Google, Jason Freidenfelds, responded to my query asking for more information about how this health information is developed and vetted. But first he wanted to argue with me that since the U.S. National Institutes of Health (NIH) believes the chemical imbalance theory, it must be true.2 Umm, thanks…?

Here’s how Google described the process to us:

We put the information (text and images) through a rigorous review process — each fact reviewed and validated by an average of 11.1 doctors, the Mayo Clinic giving final review to ensure accuracy, and artwork commissioned from licensed medical illustrators.

He ignored a followup question asking whether 11 doctors actually reviewed this particular illustration, so I turned next to the Mayo Clinic. The Mayo Clinic is Google’s primary partner in this effort, and so I would think that the buck stops with them. Mayo Clinic’s own page on the causes of depression also makes clear that we do not know what causes depression, but we do have a number of theories. None of them boil down to a “chemical imbalance” causing depression.

Mayo: You’ve Got Dysregulated Brain Function!

Mayo’s response was harder to come by in a timely fashion. The only thing I could get from them was a comment from Dr. Mark Frye, chair of psychiatry at Mayo Clinic:

What I can say to you is brain function is dysregulated in depression — whether it be biological, chemical, neurobiological remains to be further delineated but at some point, there is some overlap in these term. I think we can also say that depression, like hypertension and other chronic medical conditions, is a stress sensitive illness and yes, environmental impact can contribute to depression so our approach has been to try and address of be mindful of both.

Ah, the good ole “dysregulated” brain problem — just another way of rephrasing the “chemical imbalance” theory. Funny thing is, we don’t know what a “regulated” or balanced brain looks like. And it’s not just me who’s pointed out this problem. That’s the same view held by Dr. Pies, too:

As I have written on several occasions, to establish an “imbalance” in brain chemicals, we would need to know what the “correct” or optimal “balance” is, in the brain — and that would mean knowing the “right” proportion of dozens (if not hundreds) of different brain chemicals — not just serotonin, dopamine, or norepinephrine. We clearly don’t have that kind of information and may never have it. That said, there are, in my estimation, hundreds of convincing studies showing that, in persons with carefully-diagnosed major depressive disorder (MDD), various kinds of brain abnormalities occur significantly more frequently than in control subjects.

This does not necessarily mean the person’s depression was caused by these abnormalities — it’s conceivable that the person became depressed owing primarily to job loss or a marriage break up, which led, secondarily, to alterations in brain function. Or, the arrow of causality could have been reversed: the person may have lost her job because she developed a biologically-based depression that impaired her ability to work.3

In other words, “dysregulation” is psychobabble pseudo-scientific speak for saying a brain that is depressed may not look like a person’s brain that isn’t depressed. This image of a brain being “dysregulated” is just as likely caused by depression as it is the result of brain dysfunction itself.

When pressed to answer whether the Mayo Clinic believed in the “chemical imbalance” theory of depression, the spokesperson helpfully remarked:

The graphic is clearly a simplified attempt at communicating a concept and obviously should not be interpreted as a comprehensive medical definition.

Umm, okay. Except that we know via research that imagery online carries far more weight than words do.

Exactly How Powerful Is that One Graphic?

I decided to find out exactly how powerful that one image is by conducting a little survey of my own, using both Psych Central users and some fine randomized workers from Mechanical Turk.4

Over the course of nearly 48 hours, I garnered 593 respondents who either completed the task on Mechanical Turk or took the survey here on Psych Central.

I asked people one simple question when viewing the Google search results for “depression symptoms”: what are 3 things you learned about depression from that page? I gave respondents a choice of 10 different (randomized ordered) answers — all of which appeared on the page. Therefore, people are answering based upon what most likely caught their attention.

Here are the results:

  • A chemical imbalance leads to depression – 15.35%
  • A symptom of depression is loss of interest in daily activities – 14.91%
  • A symptom of depression is hopelessness – 14.91%
  • Depression is “very common” – 14.47%
  • Depression is medically treatable – 14.04%
  • A symptom of depression is appetite or weight changes – 7.46%
  • Symptoms of depression include self-loathing, anger or irritability – 7.02%
  • Not everyone who is depressed experiences every symptom of depression – 6.14%
  • A symptom of depression is difficulty concentrating – 3.07%
  • A symptom of depression is difficulty in remembering details – 2.63%

The number one statement people most often responded to was the statement that appears in the illustration about depression. The number one takeaway from looking up depression symptoms in Google is misleading, simplistic information about the disorder.

Also rounding out the top 5 were two other statements that appear only in the Google Knowledge Graph — neither of which are particularly helpful. Saying that depression is “very common” appears to minimize the potential severity of the disorder experienced by many who have it. Saying that depression is “medically treatable” completely negates the psychological (hello, psychotherapy anyone?), holistic, environmental and self-help treatments available for it — you know, those that are drug-free and not conducted in a traditional medical setting.

Three out of the five most remembered components about depression appeared in the Google Knowledge Graph. And they were all faulty, simplistic and/or misleading.

The findings from this quick survey illustrate what Google already knows from their own in-house research — these Knowledge Graphs are powerful tools that catch people’s attention. That’s why if they employ illustrations or graphics, they must be absolutely correct; there is no room for error.

Why Getting Health Information Right Matters

Few serious brain scientists, psychologists or psychiatrists continue to believe the chemical imbalance theory of depression today. And as Dr. Pies pointed out to me5, in materials published by the Mayo Clinic itself and organizations like the American Psychiatric Association, you won’t find the use of “the simplistic and misleading phrase, “chemical imbalance.” They lead off by telling us this simple, plain fact — we don’t know what causes depression — but we have a number of theories.

While everyone was happy to give me excuses in defending Google’s incorrect illustration, nobody was willing to simply say, “Hey, you’re right — the causes of depression aren’t really known. Besides that, it appears the causes are actually kind of complex and nuanced. This is not information we should try and be transmitting via a simplistic and misleading illustration.”

The answer remains unclear about whether Google’s “Knowledge Graphs” are a good thing or not, given this episode. When asked about this misleading information, both Google and Mayo denied anything was problematic with the illustration. This denial — even when presented with facts to the contrary by independent experts and research — was disturbing, suggesting misinformation may not be corrected.

Regardless of the causes of depression, it’s important to keep in mind that depression is treatable. And not just “medically treatable” as Google would have you believe, as Dr. Pies notes:

The good news is, we can intervene in such a cascade in several different and effective ways, including “talk therapy”, medication, and even exercise. Often, these treatment modalities are complementary, and have preferential effects on different components of the MDD; e.g., psychotherapy may alleviate guilt and low self-esteem, while antidepressant medication may help with sleep and agitation.

I agree. I hope folks at Google and the Mayo Clinic see this entry in order to update and correct the faulty misinformation they’ve put in front of the eyeballs of millions of people. Despite having been reviewed by “11.1 doctors,” it seems evident that Google and Mayo still have a few wrinkles to work out of this feature.

Knowledge Graphs are large
Knowledge Graphs aren’t small or hard to notice either —
they can take up nearly half of the top part of the search results page.

Footnotes:

  1. In the interests of full disclosure, Knowledge Graphs also have the potential to impact the traffic of independent health publishers like Psych Central, although we’ve yet to see any such impact. []
  2. For the evidence, he pointed me to a MedlinePlus page written — not by the NIH — but by one of their health information service providers, ADAM. Now while ADAM makes a fine health encyclopedia, I think they got this point wrong. I don’t think the reviewers of this article critically reviewed this aspect of their article. []
  3. Personal correspondence, August 17, 2015. []
  4. Social scientists regularly use this service to collect anonymous survey data, and it’s relatively randomized and representative. I used the Mechanical Turk workers to verify the data from the Psych Central survey, since I know that a survey of Psych Central readers is bound to be weighted toward people who are more sensitive to mental health concerns. []
  5. Personal correspondence, August 17, 2015. []