On Selection Bias Magnitudes
Date
2014-05
Authors
Najab, Julius
Journal Title
Journal ISSN
Volume Title
Publisher
Abstract
Selection bias remains the most prominent threat to validity in social and behavioral sciences. Non-equivalence between groups prior to an intervention reduces our ability to evaluate or infer intervention effects. Some methodologists argue that the effects due to selection bias may be estimated and subtracted from observed effects. If the estimate and subtract method were tenable then social scientists might be able to better understand past, present and future findings by employing this relatively simple procedure. Unfortunately, despite its prominence, selection bias remains largely unknown with respect to its magnitude of effect. The current dissertation aimed to do two things to facilitate the estimate and subtract method. First, I estimated the mean effect for selection bias effects in two different domains. The purpose for the different domains was to ensure that the estimates derived in one domain generalize into at least one other domain. Second, I used a resampling procedure to estimate the distribution of possible effect sizes due to selection bias. The sampling distribution allowed me to estimate the probability of any effect - at least according to the current study and, more importantly, to introduce a method that other researchers may employ in future studies similar to this study. Both aims were met by experimentally manipulating a study to produce selection bias effects. My overall aim was to demonstrate that an experimental procedure to manipulate, estimate, and model selection bias was both possible and fruitful. Through this demonstration, I encourage other researchers to consider an experimental approach to better understanding threats to validity.
Description
Keywords
Quantitative psychology and psychometrics, Quasi-experiment, Selection bias, Validity