An animal’s estimate of time intervals is variable. When time intervals are an important feature of a task, the animal should account for this variability in order to maximize reward. Recent research has suggested that humans and mice nearly maximize rewards per trial given their own timing variability. In three experiments, this result is extended to reward rate maximization (rewards per unit of time) in the Differential Reinforcement of Low Rates (DRL) task. In the first experiment, rats were shown to nearly maximize reward rate in the DRL task, but only when the DRL schedule was short (less than 28 seconds). In the second experiment, Gaussian variability was added to the response times. Humans modified their response times in a way that maximized reward rate, but rats reacted in a way that prioritized speed. In the third experiment, it was shown that rats adjust quickly to a new DRL schedule, sometimes in a few responses. Rats were faster to adjust if the new DRL schedule was shorter than the old schedule, compared to the when the new schedule was longer than the old schedule. The amount that the schedule changed did not determine the speed of adjustment. In all experiments, even though rats nearly maximized reward rate, the rats tended to respond earlier than optimal. This finding should be explored in future research.
Freestone, David M.,
"TIMING UNCERTAINTY AND REWARD RATE MAXIMIZATION"
Cognitive Sciences Theses and Dissertations.
Brown Digital Repository. Brown University Library.