QUASI-EXPERIMENTS-Research Methodology

• Quasi-experiments provide an important alternative when true experiments
are not possible.
• Quasi-experiments lack the degree of control found in true experiments;
most notably, quasi-experiments typically lack random assignment.
• Researchers must seek additional evidence to eliminate threats to internal
validity when they do quasi-experiments rather than true experiments.
• The one-group pretest-posttest design is called a pre-experimental design
or a bad experiment because it has so little internal validity.

A dictionary will tell you that one definition of the prefix quasi- is “resem- bling.” Quasi-experiments involve procedures that resemble those of true ex- periments. Generally speaking, quasi-experiments include some type of intervention or treatment and they provide a comparison, but they lack the degree of control found in true experiments. Just as randomization is the hall- mark of true experiments, so lack of randomization is the hallmark of quasi- experiments. As Campbell and Stanley (1966) explain, quasi-experiments arise when researchers lack the control necessary to perform a true experiment.
Quasi-experiments are recommended when true experiments are not fea- sible. Some knowledge about the effectiveness of a treatment is more desir- able than none. The list of possible threats to internal validity that we reviewed earlier can be used as a checklist in deciding just how good that knowledge is. Moreover, the investigator must be prepared to look for additional kinds of evi- dence that might rule out a threat to internal validity that is not specifically con- trolled in a quasi-experiment. For example, suppose that a quasi-experiment does not control for history threats that would be eliminated by a true experi- ment. The investigator may be able to show that the history threat is implau- sible based on a logical analysis of the situation or based on evidence provided by a supplementary analysis. If the investigator can show that the history threat is implausible, then a stronger argument can be made for the internal validity of the quasi-experiment. Researchers must recognize the specific shortcomings of quasi-experimental procedures, and they must work like detectives to provide whatever evidence they can to overcome these shortcomings. As we begin to consider the appropriate uses of quasi-experiments, we need to acknowledge that there is a great difference between the power of the true experiment and that of the quasi-experiment. Before facing the problems of interpretation that result from quasi-experimental procedures, the researcher should make every effort possible to approximate the conditions of a true experiment.
Perhaps the most serious limitation researchers face in doing experiments in natural settings is that they are frequently unable to assign participants randomly to conditions. This occurs, for instance, when an intact group is singled out for treatment and when administrative decisions or practical considerations prevent randomly assigning participants. For example, children in one classroom or school and workers at a particular plant represent intact groups that might receive a treat- ment or intervention without the possibility of randomly assigning individuals to conditions. If we assume that behavior of a group is measured both before and after treatment, such an “experiment” can be described as follows:
O1 X O2
where O1 refers to the first observation of a group, or pretest, X indicates a treat- ment, and O2 refers to the second observation, or posttest.
This one-group pretest-posttest design represents a pre-experimental design or, more simply, may be called a bad experiment. Any obtained difference be- tween the pretest and posttest scores could be due to the treatment or to any of several threats to internal validity, including history, maturation, testing, and instrumentation threats (as well as experimenter expectancy effects and nov- elty effects). The results of a bad experiment are inconclusive with respect to the effectiveness of a treatment. Fortunately, there are quasi-experiments that improve upon this pre-experimental design.
The Nonequivalent Control Group Design
• In the nonequivalent control group design, a treatment group and a comparison group are compared using pretest and posttest measures.
●If the two groups are similar in their pretest scores prior to treatment but differ in their posttest scores following treatment, researchers can more confidently make a claim about the effect of treatment.
• Threats to internal validity due to history, maturation, testing,
instrumentation, and regression can be controlled in a nonequivalent control group design.


Nonequivalent Control Group Design: The Langer and Rodin Study
• Quasi-experiments often assess the overall effectiveness of a treatment that has many components; follow-up research may then determine which components are critical for achieving the treatment effect.
Langer and Rodin (1976) hypothesized that environmental changes associ- ated with old age contribute, in part, to feelings of loss, inadequacy, and low self-esteem among the elderly. Of particular importance is the change that occurs when elderly persons move into a nursing home. Although they usually care for the elderly quite adequately in physical terms, nursing homes often provide what Langer and Rodin call a “virtually decision-free” environment. The elderly are no longer called on to make even the simplest decisions, such as what time to get up, whom to visit, what movie to watch, and the like. In a nurs- ing home, many or most of these everyday decisions are made for the elderly, leaving them with little personal responsibility and choice.
To test the hypothesis that the lack of opportunity to make personal decisions contributes to the psychological and even the physical debilitation sometimes seen in the elderly, Langer and Rodin carried out a quasi-experiment in a Connecticut nursing home. The independent variable was the type of responsibility given to two groups of nursing home residents. One group was informed of the many decisions they needed to make regarding how their rooms were arranged, visit- ing, care of plants, movie selection, and so forth. These residents were also given a small plant as a gift (if they decided to accept it) and told to take care of it as they wished. This was the responsibility-induced condition. The second group of resi- dents, the comparison group, was also called together for a meeting, but instruc- tions for this group stressed the staff’s responsibility for them. These residents also received a plant as a gift (whether they chose to have one or not) and were told the nurses would water and care for the plants for them.
Residents of the nursing home had been assigned to a particular floor and room on the basis of availability, and some residents had been there for a long time. As a consequence, randomly assigning residents to the two responsibility groups was impractical—and probably undesirable from the administration’s point of view. Therefore, the two sets of responsibility instructions were given to residents on two different floors of the nursing home. These floors were cho- sen, in the words of the authors, “because of similarity in the residents’ physi- cal and psychological health and prior socioeconomic status, as determined from evaluations made by the home’s director, head nurses, and social worker” (Langer & Rodin, 1976, p. 193). The floors were randomly assigned to one of the two treatments. In addition, questionnaires were given to residents 1 week before and 3 weeks after the responsibility instructions. The questionnaires con- tained items that related to “how much control they felt over general events in their lives and how happy and active they felt” (p. 194). Furthermore, staff members on each floor were asked to rate the residents, before and after the experimental manipulation, on such traits as alertness, sociability, and activity. The investigators also included a clever posttest measure of social interest by holding a competition that asked participants to guess the number of jelly beans in a large jar. Residents entered the contest if they wished by simply filling out a piece of paper giving their estimate and name. Thus, there were a number of dependent variables to assess the residents’ perceptions of control, happiness, activity, interest level, and so forth pretest and posttest measures showed that the residents in the responsibility- induced group were generally happier, more active, and more alert following the treatment than were residents in the comparison group. Behavioral mea- sures such as frequency of movie attendance also favored the responsibility- induced group, and, although 10 residents from this group entered the jelly bean contest, only 1 resident from the comparison group participated! The investiga- tors point to possible practical implications of these findings. Specifically, they suggest that some of the negative consequences of aging can be reduced or reversed by giving the elderly the opportunity to make personal decisions and to feel competent.
Before turning to the specific limitations associated with this design, let us call your attention to another feature of the Langer and Rodin study, one that characterizes many experiments in natural settings. The treatment in the Langer and Rodin study actually had several components. For example, residents in the treatment group were encouraged by the staff to make decisions about a num- ber of different things (e.g., movies, rooms, etc.), and they were offered a plant to take care of. The experiment evaluated, however, the treatment “package.” That is, the effectiveness of the overall treatment, not individual components of the treatment, was assessed. We only know (or at least we assume based on the evidence) that the treatment with all its components worked; we don’t necessarily know whether the treatment would work with fewer components or whether one component is more critical than others.
Research in natural settings is often characterized by treatments with many components. Moreover, the initial goal of such research is often to assess the overall effect of the treatment “package.” Finding evidence for an overall treat- ment effect, therefore, may be only the first stage in a research program if we want to identify the critical elements of a treatment. There may be practical as well as theoretical benefits to such identification. On practical grounds, should research reveal that only some of the treatment’s features are critical to pro- duce the effect, then perhaps the less critical features could be dropped. This may make the treatment more cost-effective and more likely to be adopted and carried out. From a theoretical standpoint, it is important to determine whether components of the treatment specified by a theory as being critical are, indeed, the critical components. When you hear about research showing an overall treatment effect you might think about how additional research could reveal what specific components are critical to the treatment’s effect.
Sources of Invalidity in the Nonequivalent Control Group Design
• To interpret the findings in quasi-experimental designs, researchers examine the study to determine if any threats to internal validity are present.
• The threats to internal validity that must be considered when using the
nonequivalent control group design include additive effects with selection, differential regression, observer bias, contamination, and novelty effects.
• Although groups may be comparable on a pretest measure, this does not
ensure that the groups are comparable in all possible ways that are relevant to the outcome of the study.
According to Cook and Campbell (1979), the nonequivalent control group design generally controls for all major classes of potential threats to internal validity except those due to additive effects of (1) selection and maturation,
(2) selection and history, (3) selection and instrumentation, and (4) those due to differential statistical regression. We will explore how each of these poten- tial sources of invalidity might pose problems for Langer and Rodin’s interpre- tation of their findings. We will then explain how Langer and Rodin offered both logical argument and empirical evidence to refute the possible threats to the internal validity of their study. We will also examine how experimenter bias and problems of contamination were controlled. Finally, we will comment briefly on challenges of establishing external validity that are inherent in the nonequivalent control group design.
An important initial finding in Langer and Rodin’s study was that the resi- dents in the two groups did not differ significantly on the pretest measures. It would not have been surprising to find a difference between the two groups before the treatment was introduced because the residents were not randomly assigned to conditions. Even when pretest scores show no difference between groups, however, we cannot assume that the groups are “equivalent” (Campbell & Stanley, 1966). We will explain why we cannot conclude that the groups are equivalent in the discussion that follows.


The Issue of External Validity
• Similar to internal validity, the external validity of research findings must
be critically examined.
• The best evidence for the external validity of research findings is replication
with different populations, settings, and times.