Contradicted and Initially Stronger Effects in Highly Cited Clinical Research

So you know all that research published and trumpeted in the nightly news (and all over the Net) about the latest health findings? Well, 32% of that research is subsequently discovered not to be so strong, or downright wrong:

Of 49 highly cited original clinical research studies, 45 claimed that the intervention was effective. Of these, 7 (16%) were contradicted by subsequent studies, 7 others (16%) had found effects that were stronger than those of subsequent studies, 20 (44%) were replicated, and 11 (24%) remained largely unchallenged. Five of 6 highly-cited nonrandomized studies had been contradicted or had found stronger effects vs 9 of 39 randomized controlled trials (P = .008). Among randomized trials, studies with contradicted or stronger effects were smaller (P = .009) than replicated or unchallenged studies although there was no statistically significant difference in their early or overall citation impact. Matched control studies did not have a significantly different share of refuted results than highly cited studies, but they included more studies with “negative” results.

This is, of course, one of the points of research — to have the results replicated (or not) to discover if the results are robust, and therefore, that treatment decisions should be made upon them. You simply can’t do that unless you do the initial research and publish the results somewhere (like in a peer-reviewed journal).

Via JAMA.

Dr. John Grohol is the CEO and founder of Psych Central. He is an author, researcher and expert in mental health online, and has been writing about online behavior, mental health and psychology issues -- as well as the intersection of technology and human behavior -- since 1992. Dr. Grohol sits on the editorial board of the journal Cyberpsychology, Behavior and Social Networking and is a founding board member and treasurer of the Society for Participatory Medicine.

Like this author?
Catch up on other posts by John M. Grohol, PsyD (or subscribe to their feed).

Comments

This post currently has no comments. You can read the comments or leave your own thoughts on our new comments page.

Trackbacks

    Last reviewed: By John M. Grohol, Psy.D. on 13 Jul 2005
    Published on PsychCentral.com. All rights reserved.