Accession Number ADA567393
Title Modeling Exploration and Exploitation in Structured Environments.
Publication Date Jun 2010
Media Count 18p
Personal Author M. Steyvers M. D. Lee
Abstract In bandit problems, a decision-maker chooses repeatedly between a set of alternatives. They get feedback after every decision, either recording a reward or a failure. They also know that each alternative has some fixed unknown probability of providing a reward when it is chosen. The goal of the decision-maker is to obtain the maximum number of rewards over all the trials they complete. Bandit problems provide an interesting formal setting for studying the balance between exploration and exploitation in decision-making. In early trials, it makes sense to explore different alternatives, searching for those with the highest reward rates. In later trials, it makes sense to exploit those alternatives known to be good, by choosing them repeatedly. How exactly this balance between exploration and exploitation should be managed, and should be influenced by factors such as the distribution of reward rates, the total number of trials, and so on, raises basic questions about adaptation, planning, and learning in intelligent systems. This research project completed a series of inter-related lines of bandit problem research that improved our understanding of human and optimal sequential decision making using bandit problems.
Keywords Bandit problems
Cognitive science
Decision theory
Group dynamics
Heuristic methods
Heuristic models
Individual differences
Mathematical models

Source Agency Non Paid ADAS
NTIS Subject Category 92B - Psychology
57T - Psychiatry
72E - Operations Research
Corporate Author California Univ., Irvine. Dept. of Cognitive Sciences.
Document Type Technical report
Title Note Final rept. 1 Jan 2007-31 Mar 2010.
NTIS Issue Number 1309
Contract Number FA9550-09-1-0082

Science and Technology Highlights

See a sampling of the latest scientific, technical and engineering information from NTIS in the NTIS Technical Reports Newsletter

Acrobat Reader Mobile    Acrobat Reader