WP 2021-01 Measuring Intelligence with Confidence

AUTHORS: Glenn W. Harrison, Don Ross and J. Todd Swarthout

ABSTRACT: The measurement of intelligence should identify and measure the subjective confidence that an individual has that a response to a test question is correct. Existing measures that use multiple-choice answers constrain themselves to measuring the response that reflects modal beliefs, using surveys with no extrinsic financial incentive for a truthful response. Apart from concerns with the potential for hypothetical bias, since intrinsic motivation is unobservable, no measures of confidence are elicited. We rectify both issues, and show that each matters for the measurement of intelligence. We use a canonical test for fluid or analytic intelligence, the Raven Advanced Progressive Matrices test. We demonstrate that extrinsic financial incentives matter for accuracy of the test measure: incentivized subjects score more highly. We then demonstrate that awareness of the confidence of responses, and being able to express them in a properly incentivized manner, plays an important role in understanding intelligence measures. For example, women tend to do worse than men in traditional intelligence measures using the Raven test, but do much better than men when allowing for incentived measures of confidence.