Module 1, Chapter 3: Quasi-experimental Designs

Another research design that is eligible for
review under WWC Group Design Standards is a quasi-experimental design, or QED. Quasi-experimental design is one of those
terms that means different things to different people. The WWC has a specific definition. A quasi-experimental design is a study design
that has at least two distinct, nonoverlapping groups and did not use a purely random process
to form the groups. A simple QED identifies a group of units that
received an intervention and compares that group to a group of units that did not receive
the intervention. Researchers can choose these groups in many
ways, which we will discuss next. Because the groups are not the result of random
assignment, the QED approach does not provide any guarantee the groups are similar on observed
characteristics. For these reasons, studies that use a QED
must demonstrate equivalence. Because of this, QEDs cannot receive the WWC’s
highest rating. In fact, the highest rating a QED can receive
is Meets WWC Group Design Standards With Reservations. So QEDs, RCTs with high attrition, and RCTs
with compromised random assignment all must demonstrate equivalence. As I mentioned previously, the WWC considers
any study that uses nonrandomly formed groups a QED. There are many ways study authors might form
groups. Authors may use convenience samples, which
are composed of participants for whom a study author was able to collect data. For example, an author may focus the analysis
on a group of classrooms in the same school. Although not all convenience samples make
good comparison groups, right now we’re just focusing on whether this is an eligible
design, and it is. Sometimes studies use groups formed for another
purpose. For example, two districts might plan to implement
a new curriculum. A study might designate those districts as
the intervention group, and choose two other districts to form the comparison group. Other studies might use a national or district
average as a comparison group and that is an acceptable design for a QED as well, so
long as they don’t include the intervention group as part of that average. We also see studies that use specific statistical
techniques to form a comparison group that matches the intervention group on baseline
characteristics. Baseline refers to a point in time before
the intervention occurred. The WWC also considers these acceptable QEDs. It doesn’t matter whether the groups were
formed before the intervention occurred, or retroactively after the study collected outcomes. The highest rating a QED can receive is Meets
WWC Group Design Standards With Reservations. This rating means that the WWC has a lower
degree of confidence that the intervention caused the observed effect, relative to an
RCT. However, to receive this rating, the study
must demonstrate that the groups were equivalent on baseline characteristics. Again, we’ll discuss baseline equivalence
in the Baseline Equivalence Module, which is Module 3. QEDs that do not demonstrate equivalence receive
the Does Not Meet WWC Group Design Standards rating.

Tags:, ,

Add a Comment

Your email address will not be published. Required fields are marked *