statistical analysis for quasi experimental design

Home » Quasi-Experimental Research Design – Types, Methods

Quasi-Experimental Research Design – Types, Methods

Table of Contents

Quasi-experimental research design is a widely used methodology in social sciences, education, healthcare, and other fields to evaluate the impact of an intervention or treatment. Unlike true experimental designs, quasi-experiments lack random assignment, which can limit control over external factors but still offer valuable insights into cause-and-effect relationships.

This article delves into the concept of quasi-experimental research, explores its types, methods, and applications, and discusses its strengths and limitations.

Quasi-Experimental Design

Quasi-experimental research design is a type of empirical study used to estimate the causal relationship between an intervention and its outcomes. It resembles an experimental design but does not involve random assignment of participants to groups. Instead, groups are pre-existing or assigned based on non-random criteria, such as location, demographic characteristics, or convenience.

For example, a school might implement a new teaching method in one class while another class continues with the traditional approach. Researchers can then compare the outcomes to assess the effectiveness of the new method.

Key Characteristics of Quasi-Experimental Research

No Random Assignment: Participants are not randomly assigned to experimental or control groups.
Comparison Groups: Often involves comparing a treatment group to a non-equivalent control group.
Real-World Settings: Frequently conducted in natural environments, such as schools, hospitals, or workplaces.
Causal Inference: Aims to identify causal relationships, though less robustly than true experiments.

Purpose of Quasi-Experimental Research

To evaluate interventions or treatments when randomization is impractical or unethical.
To provide evidence of causality in real-world settings.
To test hypotheses and inform policies or practices.

Types of Quasi-Experimental Research Design

1. non-equivalent groups design (negd).

In this design, the researcher compares outcomes between a treatment group and a control group that are not randomly assigned.

Example: Comparing student performance in schools that adopt a new curriculum versus those that do not.
Limitation: Potential selection bias due to differences between the groups.

2. Time-Series Design

This involves repeatedly measuring the outcome variable before and after the intervention to observe trends over time.

Example: Monitoring air pollution levels before and after implementing an industrial emission regulation.
Variation: Interrupted time-series design, which identifies significant changes at specific intervention points.

3. Regression Discontinuity Design (RDD)

Participants are assigned to treatment or control groups based on a predetermined cutoff score on a continuous variable.

Example: Evaluating the effect of a scholarship program where students with test scores above a threshold receive funding.
Strength: Stronger causal inference compared to other quasi-experimental designs.

4. Pretest-Posttest Design

In this design, outcomes are measured before and after the intervention within the same group.

Example: Assessing the effectiveness of a training program by comparing employees’ skills before and after the training.
Limitation: Vulnerable to confounding factors that may influence results independently of the intervention.

5. Propensity Score Matching (PSM)

This method pairs participants in the treatment and control groups based on similar characteristics to reduce selection bias.

Example: Evaluating the impact of online learning by matching students based on demographics and prior academic performance.
Strength: Improves comparability between groups.

Methods of Quasi-Experimental Research

1. data collection.

Surveys: Collect information on attitudes, behaviors, or outcomes related to the intervention.
Observations: Document changes in natural environments or behaviors over time.
Archival Data: Use pre-existing data, such as medical records or academic scores, to analyze outcomes.

2. Statistical Analysis

Quasi-experiments rely on statistical techniques to control for confounding variables and enhance the validity of results.

Analysis of Covariance (ANCOVA): Controls for pre-existing differences between groups.
Regression Analysis: Identifies relationships between the intervention and outcomes while accounting for other factors.
Propensity Score Matching: Balances treatment and control groups to reduce bias.

3. Control for Confounding Variables

Because randomization is absent, quasi-experimental designs must address confounders using techniques like:

Matching: Pair participants with similar attributes.
Stratification: Analyze subgroups based on characteristics like age or income.
Sensitivity Analysis: Test how robust findings are to potential biases.

4. Use of Mixed Methods

Combining quantitative and qualitative methods enhances the depth of analysis.

Quantitative: Statistical tests to measure effect size.
Qualitative: Interviews or focus groups to understand contextual factors influencing outcomes.

Applications of Quasi-Experimental Research

1. education.

Assessing the impact of new teaching methods or curricula.
Evaluating the effectiveness of after-school programs on academic performance.

2. Healthcare

Comparing outcomes of different treatment protocols in hospitals.
Studying the impact of public health campaigns on vaccination rates.

3. Policy Analysis

Measuring the effects of new laws or regulations, such as minimum wage increases.
Evaluating the impact of urban planning initiatives on community health.

4. Social Sciences

Studying the influence of community programs on crime rates.
Analyzing the effect of workplace interventions on employee satisfaction.

Strengths of Quasi-Experimental Research

Feasibility: Can be conducted in real-world settings where randomization is impractical or unethical.
Cost-Effectiveness: Often requires fewer resources compared to true experiments.
Flexibility: Accommodates a variety of contexts and research questions.
Generates Evidence: Provides valuable insights into causal relationships.

Limitations of Quasi-Experimental Research

Potential Bias: Lack of randomization increases the risk of selection bias.
Confounding Variables: Results may be influenced by external factors unrelated to the intervention.
Limited Generalizability: Findings may not apply broadly due to non-random group assignment.
Weaker Causality: Less robust in establishing causation compared to randomized controlled trials.

Steps to Conduct Quasi-Experimental Research

Define the Research Question: Clearly articulate what you aim to study and why a quasi-experimental design is appropriate.
Identify Comparison Groups: Select treatment and control groups based on the research context.
Collect Data: Use surveys, observations, or archival records to gather pre- and post-intervention data.
Control for Confounders: Employ statistical methods or matching techniques to address potential biases.
Analyze Results: Use appropriate statistical tools to evaluate the intervention’s impact.
Interpret Findings: Discuss results in light of limitations and potential confounding factors.

Quasi-experimental research design offers a practical and versatile approach for evaluating interventions when randomization is not feasible. By employing methods such as non-equivalent groups design, time-series analysis, and regression discontinuity, researchers can draw meaningful conclusions about causal relationships. While these designs may have limitations in controlling bias and confounding variables, careful planning, robust statistical techniques, and clear reporting can enhance their validity and impact. Quasi-experiments are invaluable in fields like education, healthcare, and policy analysis, providing actionable insights for real-world challenges.

Cook, T. D., & Campbell, D. T. (1979). Quasi-Experimentation: Design and Analysis Issues for Field Settings . Houghton Mifflin.
Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and Quasi-Experimental Designs for Generalized Causal Inference . Houghton Mifflin.
Creswell, J. W. (2018). Research Design: Qualitative, Quantitative, and Mixed Methods Approaches . Sage Publications.
Bryman, A. (2016). Social Research Methods . Oxford University Press.
Babbie, E. (2020). The Practice of Social Research . Cengage Learning.

About the author

Muhammad Hassan

Researcher, Academic Writer, Web developer

Questionnaire – Definition, Types, and Examples

Exploratory Research – Types, Methods and...

Focus Groups – Steps, Examples and Guide

Explanatory Research – Types, Methods, Guide

Basic Research – Types, Methods and Examples

Case Study – Methods, Examples and Guide

UCCS Community

Current Students
Faculty Staff
Alumni & Friends
Parents & Families

Schools and Colleges

College of Business
College of Education
College of Engineering and Applied Science
College of Letters, Arts & Sciences
College of Public Service
Graduate School
Helen and Arthur E. Johnson Beth-El College of Nursing and Health Sciences

Effect Size Calculators

Dr. Lee A. Becker

Content, Part 1
Content, Part 2
Research Tools

Statistical Analysis of Quasi-Experimental Designs:

I. apriori selection techniques.

Content, part II

I. Overview

Random assignment is used in experimental designs to help assure that different treatment groups are equivalent prior to treatment. With small n 's randomization is messy, the groups may not be equivalent on some important characteristic.

In general, matching is used when you want to make sure that members of the various groups are equivalent on one or more characteristics. If you are want to make absolutely sure that the treatment groups are equivalent on some attribute you can use matched random assignment.

When you can't randomly assign to conditions you can still use matching techniques to try to equate groups on important characteristics. This set of notes makes the distinction between normative group matching and normative group equivalence. In normative group matching you select an exact match from normative comparison group for each participant in the treatment group. In normative group equivalence you select a comparison group that has approximately equivalent characteristics to the treatment group.

II. Matching in Experimental Designs: Matched Random Assignment

In an experimental design, matched random sampling can be used to equate the groups on one or more characteristics. Whitley (in chapter 8) uses an example of matching on IQ.

The Matching Process

Note: Tx = Treatment group, Ctl = Control Group.

Analysis of a Matched Random Assignment Design

If the matching variable is related to the dependent variable, (e.g., IQ is related to almost all studies of memory and learning), then you can incorporate the matching variable as a blocking variable in your analysis of variance. That is, in the 2 x 3 example, the first 6 participants can be entered as IQ block #1, the second 6 participants as IQ block #2. This removes the variance due to IQ from the error term, increasing the power of the study.

The analysis is treated as a repeated measures design where the measures for each block of participants are considered to be repeated measures. For example, in setting up the data for a two-group design (experimental vs. control) the data would look like this:

The analysis would be run as a repeated measures design with group (control vs. experimental) as a within-subjects factor.

If you were interested in analyzing the equivalence of the groups on the IQ score variable you could enter the IQ scores as separate variables. An analysis of variance of the IQ scores with treatment group (Treatment vs. Control) as a within-subjects factor should show no mean differences between the two groups. Entering the IQ data would allow you to find the correlation between IQ and performance scores within each treatment group.

One of the problems with this type of analysis is that if any score is missing then the entire block is set to missing. None of the performance data from Block 4 in Table 2 would be included in the analysis because the performance score is missing for the person in the control group. If you had a 6 cells in your design you would loose the data on all 6 people in a block that had only one missing data point.

I understand that Dr. Klebe has been writing a new data analysis program to take care of this kind of missing data problem.

SPSS Note

The SPSS syntax commands for running the data in Table 2 as a repeated measures analysis of variance are shown in Table 3. The SPSS syntax commands for running the data in Table 2 as a paired t test are shown in Table 4.

III. Matching in Quasi-Experimental Designs: Normative Group Matching

Suppose that you have a quasi-experiment where you want to compare an experimental group (e.g., people who have suffered mild head injury) with a sample from a normative population. Suppose that there are several hundred people in the normative population.

One strategy is to randomly select the same number of people from the normative population as you have in your experimental group. If the demographic characteristics of the normative group approximate those of your experimental group, then this process may be appropriate. But, what if the normative group contains equal numbers of males and females ranging in age from 6 to 102, and people in your experimental condition are all males ranging in age from 18 to 35? Then it is unlikely that the demographic characteristics of the people sampled from the normative group will match those of your experimental group. For that reason, simple random selection is rarely appropriate when sampling from a normative population.

The Normative Group Matching Procedure

Determine the relevant characteristics (e.g., age, gender, SES, etc.) of each person in your experimental group. E.g., Exp person #1 is a 27 year-old male. Then randomly select one of the 27 year-old males from the normative population as a match for Exp person #1. Exp person #2 is a 35 year-old male, then randomly select one of the 35 year-old males as a match for Exp person #2. If you have done randomize normative group matching then the matching variable should be used as a blocking factor in the ANOVA.

If you have a limited number of people in the normative group then you can do caliper matching . In caliper matching you select the matching person based a range of scores, for example, you can caliper match within a range of 3 years. Exp person #1 would be randomly selected from males whose age ranged from 26 to 27 years. If you used a five year caliper for age then for exp person #1 you randomly select a males from those whose age ranged from 25 to 29 years old. You would want a narrower age caliper for children and adolescents than for adults.

This procedure becomes very difficult to accomplish when you try to start matching on more than one variable. Think of the problems of finding exact matches when several variables are used, e.g., an exact match for a 27-year old, white female with an IQ score of 103 and 5 children.

Analysis of a Normative Group Matching Design

The analysis is the same as for a matched random assignment design. If the matching variable is related to the dependent variable, then you can incorporate the matching variable as a blocking variable in your analysis of variance.

III. Matching in Quasi-Experimental Designs: Normative Group Equivalence

Because of the problems in selecting people in a normative group matching design and the potential problems with the data analysis of that design, you may want to make the normative comparison group equivalent on selected demographic characteristics. You might want the same proportion of males and females, and the mean age (and SD) of the normative group should be the same as those in the experimental group. If the ages of the people in the experimental group ranged from 18 to 35, then your normative group might contain an equal number of participants randomly selected from those in the age range from 18 to 35 in the normative population.

Analysis of a Normative Group Equivalence Design

In the case of normative group equivalence there is no special ANOVA procedure as there is in Normative Group Matching. In general, demographic characteristics themselves rarely predict the d.v., so you haven’t lost anything by using the group equivalence method.

A Semantic Caution

The term "matching" implies a one-to-one matching and it implies that you have incorporated that matched variable into your ANOVA design. Please don’t use the term "matching" when you mean mere "equivalence."