Using influence measures to test normative use of probability density information derived from a sample

Abstract

Bayesian decision theory (BDT) is frequently used to model normative performance in perceptual, motor, and cognitive decision tasks where the outcome of each trial is a reward or penalty that depends on the subjects actions. The resulting normative models specify how decision makers should encode and use information about uncertainty and value - step by step - in order to maximize their expected reward. When prior, likelihood, and posterior are probabilities, the Bayesian computation requires only simple arithmetic operations: addition, etc. We focus on visual cognitive tasks where Bayesian computations are carried out not on probabilities but on (1) probability density functions and (2) these probability density functions are derived from samples. We break the BDT model into a serries of computations and test human ability to carry out each of these computations in isolation. We test three necessary properties of normative use of pdf information derived from a sample - accuracy, additivity and influence. Influence measures allows us to assess how much weight each point in the sample is assigned in making decisions and allows us to compare normative use (weighting) of samples to actual, point by point. We find that human decision makers violate accuracy and additivity systematically but that the cost of failure in accuracy or additivity would be minor in common decision tasks. However, a comparison of measured influence for each sample point with normative influence measures demonstrates that the individuals use of sample information is markedly different from the predictions of BDT. We demonstrate that the normative BDT model takes into account the geometric symmetries of the pdf while the human decision maker does not. A heuristic model basing decisions on a single extreme sample point provided a better account for participants data than the normative BDT model. Author SummaryBayesian decision theory (BDT) is used to model human performance in tasks where the decision maker must compensate for uncertainty in order to to gain rewards and avoid losses. BDT prescribes how the decision maker can combine available data, prior knowledge, and value to reach a decision maximizing expected winnings. Do human decision makers actually use BDT in making decisions? Researchers typically compare overall human performance (total winnings) to the predictions of BDT but we cannot conclude that BDT is an adequate model for human performance based on just overall performance. We break BDT down into elementary operations and test human ability to execute such operations. In two of the tests human performance deviated only slightly (but systematically) from the predictions of BDT. In the third test we use a novel method to measure the influence of each sample point provided to the human decision maker and compare it to the influence predicted by BDT. When we look at what human decision makers do - in detail - we find that they use sensory information very differently from what the normative BDT observer does. We advance an alternative non-Bayesian model that better predicts human performance.