Timezone: »

 
Poster
Temporal Difference Based Actor Critic Learning - Convergence and Neural Implementation
Dotan Di Castro · Dima Volkinshtein · Ron Meir

Mon Dec 08 08:45 PM -- 12:00 AM (PST) @
in Posters »

Actor-critic algorithms for reinforcement learning are achieving renewed popularity due to their good convergence properties in situations where other approaches often fail (e.g., when function approximation is involved). Interestingly, there is growing evidence that actor-critic approaches based on phasic dopamine signals play a key role in biological learning through the cortical and basal ganglia. We derive a temporal difference based actor critic learning algorithm, for which convergence can be proved without assuming separate time scales for the actor and the critic. The approach is demonstrated by applying it to networks of spiking neurons. The established relation between phasic dopamine and the temporal difference signal lends support to the biological relevance of such algorithms.

[ PDF] 

Author Information

Dotan Di Castro (Technion, Israel Institute of Technology)
Dima Volkinshtein
Ron Meir (Technion)

More from the Same Authors