The Trouble with Small Psychological Studies
If your research result isn’t interesting, new and unique, few media or news outlets are likely to pick up on it and turn it into a popular news story.
That puts an unhealthy emphasis on publishing shorter, easier-to-read and understand research studies whose sample sizes may simply be too small to generalize to the wider, more diverse population.
New research suggests this “bite-size science” may be hurting research more than most people realize — even the researchers themselves.
The authors of the new article are psychologists Marco Bertamini of the University of Liverpool and Marcus Munafò of the University of Bristol.
They define “bite-size science” as research papers based on one or a few studies and small samples.
“We’re not against [being concise or short and to the point],” says Bertamini. “But there are real risks in this trend toward shorter papers. The main risk is the increased rates of false alarms that are likely to be associated with papers based on less data.”
The article calls into question and refutes several claimed advantages of shorter papers.
For instance, proponents of short research papers say they’re easier to read. Perhaps, say the authors, but more articles mean more to keep up with, more reviewing and editing — not less work.
Proponents also note that the more citations an author has in the research literature, the more influence the authors gain. The current researchers agree, but note that two short papers do not equal twice the scientific value of a longer one. Indeed, they might add up to less.
The reason: The smaller the experimental sample the greater the statistical deviations — that is, the greater the inaccuracy of the findings. There are no statistical procedures that can make up for too small a sample size.
The results are sometimes flukes, with a bias toward false positives –the researchers finding something that appears to be a new or exciting research result, but which a larger study will determine is not true after all.
Stricter word limits in journal articles also mean cutting important details about previous research. The new results sound not only surprising, but also new and interesting.
“A bit of ignorance helps in discovering ‘new’ things,” suggest the authors.
These surprising, novel results are exactly what editors find exciting and newsworthy and what even the best journals seek to publish, say the authors. The mainstream media pick up the “hot” stories.
And then the wrong results proliferate.
“Scientists are skeptics by training,” says Bertamini. But the trend toward bite-sized science leaves no time or space for that crucial caution. And that, argue the authors, is antithetical to good science.
The new article appears in January’s Perspectives on Psychological Science.
Source: Perspectives on Psychological Science
News Editor, P. (2018). The Trouble with Small Psychological Studies. Psych Central. Retrieved on November 25, 2020, from https://psychcentral.com/news/2011/12/29/the-trouble-with-small-psychological-studies/33037.html