advertisement
Home » News » The Trouble with Small Psychological Studies

The Trouble with Small Psychological Studies

The Trouble with Small Psychological StudiesIf your research result isn’t interesting, new and unique, few media or news outlets are likely to pick up on it and turn it into a popular news story.

That puts an unhealthy emphasis on publishing shorter, easier-to-read and understand research studies whose sample sizes may simply be too small to generalize to the wider, more diverse population.

New research suggests this “bite-size science” may be hurting research more than most people realize — even the researchers themselves.

The authors of the new article are psychologists Marco Bertamini of the University of Liverpool and Marcus Munafò of the University of Bristol.

They define “bite-size science” as research papers based on one or a few studies and small samples.

“We’re not against [being concise or short and to the point],” says Bertamini. “But there are real risks in this trend toward shorter papers. The main risk is the increased rates of false alarms that are likely to be associated with papers based on less data.”

The article calls into question and refutes several claimed advantages of shorter papers.

For instance, proponents of short research papers say they’re easier to read. Perhaps, say the authors, but more articles mean more to keep up with, more reviewing and editing — not less work.

Proponents also note that the more citations an author has in the research literature, the more influence the authors gain. The current researchers agree, but note that two short papers do not equal twice the scientific value of a longer one. Indeed, they might add up to less.

The reason: The smaller the experimental sample the greater the statistical deviations — that is, the greater the inaccuracy of the findings. There are no statistical procedures that can make up for too small a sample size.

The results are sometimes flukes, with a bias toward false positives –the researchers finding something that appears to be a new or exciting research result, but which a larger study will determine is not true after all.

Stricter word limits in journal articles also mean cutting important details about previous research. The new results sound not only surprising, but also new and interesting.

“A bit of ignorance helps in discovering ‘new’ things,” suggest the authors.

These surprising, novel results are exactly what editors find exciting and newsworthy and what even the best journals seek to publish, say the authors. The mainstream media pick up the “hot” stories.

And then the wrong results proliferate.

“Scientists are skeptics by training,” says Bertamini. But the trend toward bite-sized science leaves no time or space for that crucial caution. And that, argue the authors, is antithetical to good science.

The new article appears in January’s Perspectives on Psychological Science.

Source: Perspectives on Psychological Science

The Trouble with Small Psychological Studies

Psych Central News Editor

The Psych Central News Editor is a member of the Psych Central staff with background in journalism and mental health reporting. This is a rotating position, but is filled with a qualified reporter with appropriate psychology and mental health news experience.

APA Reference
News Editor, P. (2018). The Trouble with Small Psychological Studies. Psych Central. Retrieved on September 15, 2019, from https://psychcentral.com/news/2011/12/29/the-trouble-with-small-psychological-studies/33037.html
Scientifically Reviewed
Last updated: 8 Aug 2018
Last reviewed: By a member of our scientific advisory board on 8 Aug 2018
Published on Psych Central.com. All rights reserved.