• In true experiments, researchers manipulate an independent variable with treatment and comparison condition(s) and exercise a high degree of control (especially through random assignment to conditions).
As we have noted, although many everyday activities (such as altering the ingredients of a recipe) might be called experiments, we would not consider them “true” experiments in the sense in which experimentation has been dis- cussed in this textbook. Analogously, many “social experiments” carried out by the government and those that are conducted by company officials or educa- tional administrators are also not true experiments. A true experiment is one that leads to an unambiguous outcome regarding what caused an event. True experiments exhibit three important characteristics:
1 In a true experiment some type of intervention or treatment is implemented. 2 True experiments are marked by the high degree of control that an ex- perimenter has over the arrangement of experimental conditions, assignment of participants, systematic manipulation of independent variables, and choice of dependent variables. The ability to assign participants randomly to experi- mental conditions is often seen as the most critical defining characteristic of the true experiment (Judd, Smith, & Kidder, 1991).
3 Finally, true experiments are characterized by an appropriate compari- son. Indeed, the experimenter exerts control over a situation to establish a proper comparison to evaluate the effectiveness of a treatment. In the simplest of experimental situations, this comparison is one between two comparable groups that are treated exactly alike except for the variable of interest.
When the conditions of a true experiment are met, any differences in a de- pendent variable that arise can logically be attributed to the differences between levels of the independent variable. There are differences, however, between true experiments done in natural settings and experiments done in a laboratory. A few of the most important differences are described in Box 10.1.
Obstacles to Conducting True Experiments in Natural Settings
• Researchers may experience difficulty obtaining permission to conduct true
experiments in natural settings and gaining access to participants.
• Although random assignment is perceived by some as unfair because it
may deprive individuals of a new treatment, it is still the best way and fairest way to determine if a new treatment is effective.
Experimental research is an effective tool for solving problems and answer- ing practical questions. Nevertheless, two major obstacles often arise when we try to carry out experiments in natural settings. The first problem is obtaining permission to do the research from individuals in positions of authority. Un- less they believe that the research will be useful, school board presidents and government and business leaders are unlikely to support research financially or otherwise. The second, and often more pressing, obstacle to doing experi- ments in natural settings is the problem of access to participants. This problem can prove especially troublesome if participants are to be randomly assigned to either a treatment group or a comparison group.
Random assignment to conditions appears unfair at first—after all, random assignment requires that a potentially beneficial treatment be withheld from some participants. Suppose that a new approach to the teaching of foreign languages was to be tested at your college or university. Suppose further that, when you went to register for your next semester’s classes, you were told that you would be randomly assigned to one of two sections taught at the time you selected—one section involving the old method and one involv- ing the new method. How would you react? Your knowledge of research methods tells you that the two methods must be administered to comparable groups of students and that random assignment is the best way to ensure such comparability. Nonetheless, you might be tempted to feel that random assignment is not fair, especially if you are assigned to the section using the old (old-fashioned?) method. Let’s take a closer look at the fairness of random assignment.
Threats to Internal Validity Controlled by True Experiments
• Threats to internal validity are confounds that serve as plausible alternative
explanations for a research finding.
• Major classes of threats to internal validity include history, maturation,
testing, instrumentation, regression, subject attrition, selection, and additive effects with selection.
Problems That Even True Experiments May Not Control
• Threats to internal validity that can occur in any study include
contamination, experimenter expectancy effects, and novelty effects.
• Contamination occurs when information about the experiment is
communicated between groups of participants, which may lead to resentment, rivalry, or diffusion of treatment.
• Novelty effects occur when people’s behavior changes simply because an
innovation (e.g., a treatment) produces excitement, energy, and enthusiasm.
• Threats to external validity occur when treatment effects may not be
generalized beyond the particular people, setting, treatment, and outcome of the experiment.
No comments:
Post a Comment