Bias-aware versus bias-blind confidence in humans and machines

Abstract

Confidence evaluates the likely accuracy of a current decision. However, to be maximally informative about accuracy, confidence judgments should incorporate information about ones broader decision tendencies, such as their propensity to favor specific alternatives. We distinguish bias-aware confidence, which considers such tendencies, from bias-blind confidence, which relies only on evidence available on the current trial. To adjudicate between bias-aware and bias-blind confidence, we identified a signature of bias-aware confidence: the down-weighting of confidence for alternatives that a participant is biased toward. We then used a large dataset (N = 200) spanning 4- and 8-choice digit-classification tasks to show that humans reliably exhibit this signature of bias-aware confidence. This effect was reduced under speed pressure and could not be explained by guessing. In contrast to the human results, artificial neural networks (ANNs) trained for object recognition lacked this signature of bias- aware confidence. Importantly, augmenting ANNs with a metacognitive module that allows confidence to take the networks biases into account led to the emergence of human-like bias- aware confidence. These findings show that human confidence incorporates not only information from the current trial but also longer-term decision tendencies, and that this capacity - absent in standard ANNs - can be conferred through specialized metacognitive mechanisms.