Scrutinizing the Basis of Originality in Divergent Thinking Tests: On the Measurement Precision of Response Propensity Estimates
Publication Date
10-29-2019
Document Type
Article
Organizational Units
Morgridge College of Education, Research Methods and Information Science, Research Methods and Statistics
Keywords
Divergent thinking, Item response theory, Originality, Reliability, Sample size
Abstract
Background
The originality of divergent thinking (DT) production is one of the most critical indicators of creative potential. It is commonly scored using the statistical infrequency of responses relative to all responses provided in a given sample.
Aims
Response frequency estimates vary in terms of measurement precision. This issue has been widely overlooked and is addressed in the current study.
Sample and method
Secondary data analysis of 202 participants was performed. A total of 900 uniquely identified responses were generated on three DT tasks and subjected to a 1‐parameter logistic model with a response as the unit of measurement which allowed for the calculation of response‐level conditional reliability (and marginal reliability as an overall summary of measurement precision).
Results
Marginal reliability of response propensity estimates ranged from .62 to .67 across the DT tasks. Unique responses in the sample (the basis for the classic uniqueness scoring) displayed the lowest conditional reliability (across tasks: ≈ .50). Reliability increased nonlinearly as a function of both the frequency of occurrence predicted by the model (conditional reliability) and sample size (conditional and marginal reliability).
Conclusions
This study indicates that the common practice of frequency‐based originality scoring with typical sample sizes (e.g., N = 100 to N = 200) yields unacceptable levels of measurement precision (i.e., in particular for highly original responses). We further offer recommendations to mitigate the lack of measurement precision of frequency‐based originality scores for DT research.
Publication Statement
Copyright held by author or publisher. User is responsible for all copyright compliance.
Recommended Citation
Forthmann, Boris, Paek, Sue Hyeon, Dumas, Denis, Barbot, Baptiste, & Holling, Heinz. (2020). Scrutinizing the basis of originality in divergent thinking tests: On the measurement precision of response propensity estimates. British Journal of Educational Psychology, 90(3), 683-699. doi: 10.1111/bjep.12325.