Scrutinizing the Basis of Originality in Divergent Thinking Tests: On the Measurement Precision of Response Propensity Estimates

Publication Date

10-29-2019

Document Type

Article

Organizational Units

Morgridge College of Education, Research Methods and Information Science, Research Methods and Statistics

Keywords

Divergent thinking, Item response theory, Originality, Reliability, Sample size

Abstract

Background

The originality of divergent thinking (DT) production is one of the most critical indicators of creative potential. It is commonly scored using the statistical infrequency of responses relative to all responses provided in a given sample.

Aims

Response frequency estimates vary in terms of measurement precision. This issue has been widely overlooked and is addressed in the current study.

Sample and method

Secondary data analysis of 202 participants was performed. A total of 900 uniquely identified responses were generated on three DT tasks and subjected to a 1‐parameter logistic model with a response as the unit of measurement which allowed for the calculation of response‐level conditional reliability (and marginal reliability as an overall summary of measurement precision).

Results

Marginal reliability of response propensity estimates ranged from .62 to .67 across the DT tasks. Unique responses in the sample (the basis for the classic uniqueness scoring) displayed the lowest conditional reliability (across tasks: ≈ .50). Reliability increased nonlinearly as a function of both the frequency of occurrence predicted by the model (conditional reliability) and sample size (conditional and marginal reliability).

Conclusions

This study indicates that the common practice of frequency‐based originality scoring with typical sample sizes (e.g., N = 100 to N = 200) yields unacceptable levels of measurement precision (i.e., in particular for highly original responses). We further offer recommendations to mitigate the lack of measurement precision of frequency‐based originality scores for DT research.

Publication Statement

Copyright held by author or publisher. User is responsible for all copyright compliance.

This document is currently not available here.


Share

COinS