Some mainstream media outlets over this past weekend told us “How Your Smartphone Can Detect Bipolar Disorder.” Based upon new research, one researcher claims to reliably detect changes in mood in people with bipolar disorder.
This must be some fantastic, robust study in which to generalize from, given how diverse the population of people with bipolar disorder is. Can smartphones really do that reliable a job of detecting mood changes in people with bipolar disorder?
Let’s find out.
The new study employed the same kind of passive smartphone monitoring techniques we’ve discussed previously here, utilizing the built-in GPS and accelerometer to detect movement (assuming the individual keeps their phone on their person most of the time), and secondly by conducting speech analysis on phone calls.
Yes, it’s a given that you basically have to give up a lot of your privacy in order for this passive monitoring to work. Right now, you’re giving it up to researchers. But if one of these apps became commercialized, you’d be giving it up to a company.
What the new researcher (Osmani, 2015) found isn’t surprising, given past research in this area:
Activity and location data together gave a good indication of the patient mood but more impressively, accurately predicted a change in this mood 94 percent of the time. And combining this with an analysis of patient phone calls increased the predictive success to over 97 percent. “Almost all changes were detected with almost no false alarms,” says Osmani.
These are just amazing statistics. Almost too good to be true… Which suggests maybe something was a little too perfect about the study.
Is This Newsworthy?
It’s not clear that media like MIT Technology Review understood this is not exactly something new (since no context was given in the article). But the largest problem with the new study is one barely mentioned in the article: it’s buried in the second-to-last paragraph, about the small sample size of the current study. “For example, it covers only 12 patients over 12 weeks.” Yet, even that part’s not entirely true.
While the study started with 12 participants, 2 patients withdrew and another 2 patients’ data was simply not used because they “didn’t experience a change of [mood] state” during the 12 week period the study was conducted. That means the data analysis come from a mere 8 people. Eight.
That’s a pretty big reason to not even report on this study at all. It’s a small pilot study that goes into the pile of other small studies in this area. For instance, we first began reporting on this smartphone passive monitoring capability back in 2012. We updated our take on it last year.
What I said then in relation to a University of Michigan study which had just been published:
It’s a good start, but as I said two years ago, we need much larger studies to determine if this stuff really has any long-term value.
That study had 6 patients. This new one had 12 (or 8 whose data was actually used for the analysis). We’re making progress, but it’s very small progress. More importantly, these small sample sizes don’t provide the kind of power needed in order to generalize from their findings. And it’s not at all clear that researchers are drawing on one another’s work, since Osmani didn’t cite the previous University of Michigan research in his work.
This area of research remains an interesting one to explore with a fair amount of potential. But the only mood the current study reliably detected a change is people getting more and more depressed — not someone becoming hypomanic or manic. Nobody started the study depressed and then moved on to a manic or hypomanic state. So it’s only proven one side of the bipolar equation.
Researchers need to go one big step further before anyone should even consider such an app for passive mood-state monitoring. The next research in this area needs to be a large-scale study that examines both people with depression and bipolar disorder, and has a large enough N to make it more robust research. (I say both groups should be the target, because I’m not convinced these monitoring apps detect elevated moods as well as they do depressed moods.)
I’ll end with what I said a year and a half ago:
I still believe these miniature computers we’re all carrying around to use for simple things like texting and phone calls could be leveraged in ways that we’re only beginning to scratch the surface of. We’re making progress in this area, but it seems slow-going despite the vast technological power we now have available.
We’ve taken a lot of first steps in this area of research. It’s now time to take the larger (and more rigorous) second and third steps to proven such technology is robust and generalizable.
Osmani, V. (2015). Smartphones in Mental Health: Detecting Depressive and Manic Episodes (PDF). IEEE Pervasive Computing.