Why is generalization and replication in research so important

The field of social psychology was traumatized in when several Dutch researchers were found to have published fraudulent data. This type of misbehavior is also detected by failures of replication. An example of a fraud detected by replication failures is the "spotted mice" scandal at the prestigious Sloan-Kettering Institute in the early s. A research scientist was working under pressure to produce successful skin grafts from black-furred mice to white-furred mice.

In all previous experiments, such grafts failed to succeed because the host animal rejected the foreign tissue. This was important research, because organ rejection commonly occurs after transplants. The scientist who faked the skin graft was riding in an elevator early one morning with a batch of his white-furred mice and a permanent ink marking pen. The reports of a successful transplant surprised other scientists. They tried to replicate the experiment, and they could not.

In this case, after failed replications, the scientist confessed what he had done, so the mysterious results were explained. The consequences of fraud are devastating for a scientist, leading to dishonor and usually to the loss of job and career.

Scientists know any important result will be subjected to attempts at replication. This provides a powerful incentive for honesty among researchers.

Students are taught that a research report should include all necessary details to permit replication. However, it is often impossible to find all relevant details about how research is conducted in a published report of research. Sometimes, to carry out an exact replication, one must contact earlier researchers to learn details of a procedure. Gasparikova-Krasnec and Ging found that researchers were generally cooperative in providing information needed for replications.

A month's wait was normally all that was required. Researchers typically realize that double-checking surprising results is important to science. A failed replication may have a stimulating effect on a field of research. Replication failures inspire new studies to figure out why an attempt to use the "same" procedures led to different results.

A fine-grained analysis of the experimental procedures may reveal some key details that were different, when comparing the original study to the replication. If a replication fails, but the original researchers believe their original finding is correct, they will suggest ways to tighten up controls or other procedures to improve the chances of a successful replication.

They hope the results will come back if another replication is attempted with improved techniques. On some occasions, replication failures continue. False claims—including those that start as honest mistakes—produce a distinctive pattern during successive attempts at replication: the effects get smaller and smaller as more replications are conducted.

This happened, for example, in the case of cold fusion: a desktop apparatus was said to produce fusion energy. In psychology, it happened with cardiac conditioning : claims that heart rates could be altered directly through conditioning procedures. Diminishing effects with repeated replications occur not because an actual effect is disappearing, but because scientists are eliminating errors with better controls, as they make additional attempts at replication.

A solid, scientific finding will gain more support as people continue to test it. A false lead or quack science claim will become less solid as people continue to test it.

Gasparikova-Krasnec, M. Experimental replication and professional cooperation. American Psychologist, 41 , Prev page Page top Chapter Contents Next page. Don't see what you need? Psych Web has over 1, pages, so it may be elsewhere on the site. Do a site-specific Google search using the box below. Book Table of Contents Chapter Contents Prev page Next page The importance of replication Not just definitions but individual acts of experimentation must be checked for reliability.

What are common reasons a replication fails? In other words, what is the study attempting to measure? When replicating earlier researchers, experimenters will follow the same procedures but with a different group of participants. So what happens if the original results cannot be reproduced?

Does that mean that the experimenters conducted bad research or that, even worse, they lied or fabricated their data? In many cases, non-replicated research is caused by differences in the participants or in other extraneous variables that might influence the results of an experiment.

For example, minor differences in things like the way questions are presented, the weather, or even the time of day the study is conducted might have an unexpected impact on the results of an experiment.

Researchers might strive to perfectly reproduce the original study, but variations are expected and often impossible to avoid. In , a group of researchers published the results of their five-year effort to replicate different experimental studies previously published in three top psychology journals.

The results were less than stellar. As one might expect, these dismal findings caused quite a stir. So why are psychology results so difficult to replicate?

Writing for The Guardian , John Ioannidis suggested that there are a number of reasons why this might happen, including competition for research funds and the powerful pressure to obtain significant results. There is little incentive to retest, so many results obtained purely by chance are simply accepted without further research or scrutiny. The project authors suggest that there are three potential reasons why the original findings could not be replicated.

The Nobel Prize-winning psychologist Daniel Kahneman has suggested that because published studies are often too vague in describing methods used, replications should involve the authors of the original studies in order to more carefully mirror the methods and procedures used in the original research. While some might be tempted to look at the results of such replication projects and assume that psychology is rubbish, many suggest that such findings actually help make psychology a stronger science.

Human thought and behavior is a remarkably subtle and ever-changing subject to study, so variations are to be expected when observing diverse populations and participants. Some research findings might be wrong, but digging deeper, pointing out the flaws, and designing better experiments helps strengthen the field.

Ever wonder what your personality type means? Sign up to find out more in our Healthy Mind newsletter. Replications in psychology research: How often do they really occur? Perspectives on Psychological Science. Estimating the reproducibility of psychological science.

Kahneman D. A new etiquette for replication. Social Psychology. Your Privacy Rights. To change or withdraw your consent choices for VerywellMind. At any time, you can update your settings through the "EU Privacy" link at the bottom of any page.

These choices will be signaled globally to our partners and will not affect browsing data. We and our partners process data to: Actively scan device characteristics for identification.

I Accept Show Purposes. The replicated results were a false negative.