I respect physicians and yes, even most surgeons. They do great work and are the primary caregivers for most of the population seeking treatment for common mental health concerns like depression and anxiety. But sometimes they don’t know what they’re talking about. It concerns me because so many people see doctors as authoritative — no matter what their specialty. Many people’s ears perk up when a doctor speaks.
On Thursday, David Volpi, M.D. decided to share with Huffington Post readers the results of a bunch of survey data — released back in March — about the use of technology and mental health symptoms.
But unlike the researchers, Dr. Volpi doesn’t carefully distinguish between cause and effect and appears to jump to the conclusion that heavy technology use is causing these symptoms (rather than the other way around — that people who are more prone to such problems, or that have them already, simply turn to technology to use to feel more emotionally connected with others and relieve some of those symptoms).
That’s one reason that I believe docs like Dr. Volpi should stick to surgery rather than disseminating psychological research results to the public. But wait, there’s more…
I enjoy when physicians and others share their opinions and insights with ordinary folks about the medical literature, research, and more. I enjoy it a little less when they do so outside their field of expertise, or disseminate findings without any kind of context.
The Multiple Problems with This Research
Nowhere in Dr. Volpi’s 947 word essay does he mention, for instance, that the research he’s quoting is from a student’s thesis. A thesis, while potentially important work, is not the equivalent of peer-reviewed research that appears in a medical or psychology journal. Anyone can write a thesis; not everyone can get a study published.
Dr. Volpi doesn’t mention that survey research is the weakest sort of psychological research there is. Because it’s so weak, it’s generally used just for exploratory purposes — to explore a certain possible link between many variables, for instance. Survey research can only provide you data about what you’re specifically measuring — and nothing about what you failed to measure. What this means is that if you haven’t identified all the possible confounds or alternative variables that might account for the data relationship, you may be only looking at a small (and potentially inconsequential) part of the bigger picture.
He also fails to mention this survey was conducted exclusively on people from Sweden. Now, I love people from Sweden — they are some of the nicest people on Earth. However, they are not representative of people around the world, nor my home country, the United States.
But what about the study itself? Was it any good?
It certainly appeared good — detailed and extensive in its scope and data analysis. But something is off about its primary components (Study III and Study IV). We can see the problem starting right here:
Only about one quarter of the women and 41% percent of the men (in both cohorts) were categorized as reporting neither stress, sleep disturbances, nor symptoms of depression, at baseline.
Put another way, 60 percent of men and 75 percent of women in these studies — at the baseline measurement — reported either serious stress, sleep disturbances, or symptoms of depression. These are numbers that are significantly larger than we would expect to see. It’s a red flag about the researcher’s choices of measurements, or the population selected (or both).
Looking at the questionnaires administered, we find one potential answer. The researchers didn’t use existing measures to study depression, sleep and stress. They adopted existing measures into their own unique measure, and defined arbitrary cutoffs for what determined “depression.”
Here’s an example of the problem. Other researchers typically use a 20- or 21-item assessment to measure depression.1 How many items did the present research use? Just one (but divided into two items).
You can imagine such a reduction is going to increase the number of people who have “depression” in the study because it isn’t going to have very good specificity.2
Confirmatory Bias in Action
This is what happens when you read a news release about a study that only confirms your own beliefs and biases. This is what psychologists call, not surprisingly, confirmatory bias (or myside bias), and is a rampant problem among professionals and researchers of all stripes.
Which Dr. Volpi then nicely demonstrates in his essay, suggesting this is just one more solid data-point that we can add to this “growing and serious public health hazard:”
I tend to think that the relationship between technology and stress, sleep disorders and depression has more to do with the overuse of technology in our society, especially among young people. If you’re a parent like I am, than you know firsthand how difficult it can be to get children to turn off the computer or put down their phone and stop texting so you can, just maybe, have a real conversation.
This is a growing and serious public health hazard that should be acknowledged and addressed by both the medical community and technology industry.
Uh, okay… But no, I don’t think it needs to be either acknowledged or addressed. Because the research hasn’t demonstrated causal links between technology and these concerns the way some docs and some researchers believe it has.
This particular area of psychology research is littered with weak, correlational studies. When physicians like Dr. Volpi do a quick read and then write an essay about it on HuffPo, it only reinforces this kind of research. And makes the message all the more confusing to readers trying to figure out how much we actually know on this subject.
So Dr. Volpi, I’ll make you a deal. I won’t write about the world of ears, noses and throats (what an otolaryngology surgeon specializes in) if you keep out of the world of psychological research. Thanks!
Read the full article: Heavy Technology Use Linked to Fatigue, Stress and Depression in Young Adults.
- In case you were wondering, you can probably use a measure of no less than 10 items to accurately assess depression, according to other research. [↩]
- The researchers even knew this was a problem going into the study, but argued by using 2 items instead of just one, it’ll help the problem; no mention was made of the fact that this would not be standard procedure in most research examining depression. [↩]