Large, Randomized Studies
Large, randomized studies that draw from diverse populations and include relevant, appropriate control groups are considered the “gold standard” in research. So why aren’t they done more often? Such large studies, often done at multiple geographic locations, are very expensive to run because they include dozens of researchers, research assistants, statisticians, and other professionals as well as hundreds, and sometimes thousands, of subjects or participants. But the findings from such research are robust and can be generalized to others far more easily, so their value to research is important.
Large studies are not immune to problems found in other kinds of research. It’s just that the problems tend to have a much smaller effect, if there are any, since the number of subjects is so large and mixed (heterogeneous). When properly designed and using accepted statistical analyses, large research studies provide both individuals and professionals with solid findings that they can act upon.
A literature review is pretty much what it describes. Virtually all peer-reviewed, published research includes what might be called a “mini literature review” in its introduction. In this section of a study, the researchers review previous studies to put the current study into some context. “Research X found 123, Research Y found 456, so we hope to find 789.”
Sometimes, however, the number of studies in a particular area of study is so large and covers so many results that it’s difficult to understand exactly what our understanding is at the moment. To help give researchers a better understanding and context for future research, a literature review may be conducted and published as its own “study.” This will basically be a comprehensive, large-scale review of all studies in a particular area published within the past 10 or 20 years. The review will describe the research efforts, expand on specific findings, and may draw some general conclusions that can be gleaned from such a global review. These reviews are usually fairly subjective and are mainly for other professionals. Their use to the general public is limited and they almost never produce new findings of interest.
A meta-analysis is similar to a literature review in that it seeks to examine all previous research in a very specific topic area. However, unlike a literature review, a meta-analytic study takes the review one important step further – it actually pulls together all of the previous study’s data and analyzes it with additional statistics to draw global conclusions about the data. Why bother? Because so much research is published in many fields that it’s virtually impossible for an individual to draw any solid conclusions from the research without such a global review that pulls together all that data and statistically analyzes it for trends and solid findings.
The key to meta-analytic studies is to understand that researchers can alter the results of such a review by being particular (or not very particular) about the kinds of studies they include in their review. If, for instance, the researchers decide to include non-randomized studies in their review, they will often get different findings than if they hadn’t included them. Sometimes researchers will require certain statistical procedures to have been performed in order for the study to be included, or certain data thresholds to be met (e.g., we’ll only examine studies that had more than 50 subjects). Depending upon what criteria researchers choose to include in their meta-analysis, it will effect the results of the meta-analysis.
Meta-analytic studies, when done properly, are important contributions to our scientific knowledge and understanding. When a meta-analysis is published, it generally acts as a new foundation for other studies to build upon. It also synthesizes a great deal of previous knowledge into a more digestable Chunk of Knowledge for everyone.