Document Type
Dissertation
Degree
Doctor of Philosophy
Major
Psychology, Industrial and Organizational
Date of Defense
4-15-2020
Graduate Advisor
Stephanie Merritt, PhD
Co-Advisor
Garett Foster, PhD
Committee
John Meriac, PhD
Cody Ding, PhD
Abstract
Research and development of modern psychometric methods such as item response theory have drastically changed the way we understand and carry out the measurement of psychological constructs. Despite this, there has been relatively little adoption by psychological researchers to incorporate these methods into their research. While multiple explanations are surely valid, one oft stated reason is the large sample size requirements of these methods. The sample size requirements of item response theory are needed so that effective estimation of item parameters can be carried out. In an attempt to make these modern measurement methods more accessible and feasible to psychological researchers, this study investigated the extent to which subject matter experts and trained novices could effectively rate the location parameter of items to use as starting parameters in the item parameter estimation process. Rather than starting with random values, as is the default approach, starting with more accurate item locations was hypothesized to result in just as accurate item parameters that do not require typical sample sizes for these models. A pseudo-simulation process was carried out to estimate parameter recovery at various sample sizes when using SME and trainee ratings of item locations as starting parameters. Results suggest that while SMEs and trainees were not able to perfectly align item location parameters with statistical estimations, person estimates derived when using these as starting parameters yielded quite similar results to the parameters from the default MML procedure. Similar results were uncovered across sample sizes. Additionally, as sample size decreased from 500 to 200, recovery results became less stable indicating that even with SME and trainee estimates of item location used as starting parameters, sample size issues still remained when estimating item parameters.
Recommended Citation
McKenna, Michael, "Can Ratings of Item Location Enhance Statistical Item Parameter Estimation? Extending the Feasibility of Unfolding IRT Models" (2020). Dissertations. 927.
https://irl.umsl.edu/dissertation/927