Workshop Report

Sloan/Swartz Center for Theoretical Neurobiology, Salk Institute

Swartz Center for Computational Neuroscience, UCSD

Rancho Santa Fe, California

The goal of this workshop was to bring together leading experts from neuroscience, psychology and behavioral economics along with students to explore a new approach to understanding brain function and behavior. This was the first time many of these researchers had met each other and the discussions at the workshop led to new collaborations and major new research projects.

There were 17 faculty at the workshop, 12 students and postdoctoral fellows, and one observer from Science Magazine.

Visiting Participants:

*Students and Postdoctoral Fellows

Local Participants:

Mike Arnold (Salk)*

Roger Bingham (UCSD)

Pat Churchland (UCSD)

JR Duann (UCSD)*

David Eagleman (Salk)*

Luca Finelli (UCSD)*

Jean-Marc Fellous (Salk)*

Tzyy-Ping Jung (UCSD)*

John Reynolds (Salk)

Observer:

Gilbert Chin (Science)

There were18 long talks and ample time for discussion during the three day workshop. The first day was held at the Salk Institute and was open to the La Jolla community as an afternoon mini-symposium. The following two days took place at the Rancho Santa Fe Inn, a nearby hotel that was secluded from the campus to facilitate interactions between participants.

The feedback we received from the workshop was stellar. Several of the students said that it was the best meeting they had ever attended and that it had a major impact on their career direction.

Friday, May 16-Salk Institute (de Hoffmann Auditorium)

12:00 Noon - Lunch (Parker Room, Salk)

Afternoon Chair: Francis Crick -- Salk

1:30 PM - Wolfram Schultz - Cambridge, England

"Outcome coding in brain reward centers"

2:30 PM - Peter Dayan - UC London

"Dopamine in Evaluation and Evocation."

3:30 PM - Break

4:00 PM - Paul Glimcher - NYU

"Neuroeconomics"

6:00 PM - Dinner (Piatti, La Jolla Shores)

Saturday, May 17-Inn at Rancho Santa Fe (Croquet Cottage)

8:00 AM - Breakfast (Croquet Cottage)

Morning Chair: Terrence Sejnowski -- Salk

9:00 AM - Bill Newsome - Stanford

"Matching behavior and the neural representation of experienced value"

10:00 AM - Break

10:15 AM - Barry Richmond - NIMH

"Biology of the relative value of work vs reward in reward schedules"

11:15 AM - Break

11:30 AM - Allison Doupe (UCSF)

"Social context and variability of neuronal firing in the

songbird basal ganglia"

12:30 PM - Lunch - (Croquet Lawn)

Afternoon Host: Scott Makeig

2:00 PM - Sebastian Seung - MIT

"Optimizing with hedonistic synapses and neurons"

3:00 PM - Excursion

Torrey Pines Reserve

Mt. Soledad

Scripps Reserve

6:00 PM - Dinner (Sbicca, Del Mar)

Evening Chair: Tzyy-Ping Jung

8:00 PM - Colin Camerer - Caltech

"Game theory for neuroscientists (and vice versa)"

Sunday May 18-Inn at Rancho Santa Fe

8:00 AM - Breakfast (Croquet Cottage)

Morning Chair: John Reynolds

9:00 AM - Michael Kearns - University of Pennsylvania

"Network Representations for Economic Interaction"

10:00 AM - Break

10:15 AM - Read Montague - Baylor

"Neural Valuation Responses: models and fMRI results"

11:15 AM - Break

11:30 AM - John Allman - Caltech

"The Spindle Neurons of Frontoinsular (FI) and Anterior Cingulate Cortex (ACC):

A Recently Evolved Class of Neurons Related to Risk, Reward and Error"

12:30 PM - Lunch - (Croquet Lawn)

Sunday May 18-Inn at Rancho Santa Fe

Afternoon Chair: Geoff Boynton

2:00 PM - XJ Wang - Brandeis

"Cortical microcircuit mechanisms of decision-making"

3:00 PM - Break

3:30 PM - Terrence Sejnowski - Salk

"Regulation of attention and working memory by coherence"

4:30 PM - Scott Makeig - UCSD

"Human brain dynamic consequences of reward and punishment"

7:00 PM - Banquet

Banquet Speaker: Pat Churchland

Précis of the Workshop

Brains evolved in an uncertain world, a world that is probabilistic and nonstationary. Food that is plentiful one day may be gone the next; new events occur, such as a financial setback, that may affect many decisions. What brain systems are involved in adapting to these changes? There is growing evidence that interactions between the prefrontal cortex and the basal ganglia are central to adaptive responses and that shifts in behavioral strategies are mediated by the dopamine neuromodulatory projections into the striatum and the prefrontal cortex.

Dopamine neurons form an ancient system that is found in all vertebrates and a similar system based on octopamine, a related biogenic amine, is founding insects. Dopamine neurons in the ventral tegmental area (VTA) and the substantia nigra pars compacta, (SNc) project widely throughout the striatum and cerebral cortex and are known to carry information about predicted reward. Wolfram Schultz showed, for example, in recordings from these areas in monkeys during learning, that the dopamine cells in the VTA respond initially to the reward itself - typically juice - but gradually over time begin to respond to sensory stimuli that are associated with the reward, as the response to the reward itself extinguishes. With more learning, earlier and earlier sensory stimuli that predict the reward begin to stimulate the dopamine neurons.

Most recordings from neurons in the monkey cortex have focused on the abstract sensory information that they encode, or the actions that they elicit. Only recently has it been discovered that many cortical neurons also respond to properties of the reward. For example, Barry Richmond showed that the neurons in the anterior cingulate cortex respond not to the features of the sensory stimulus but to the meaning that the stimulus has for expected reward. Bill Newsome found neurons in the posterior parietal cortex that are strongly modulated by expected reward. These results suggest that a major function of the cerebral cortex has been ignored, and in particular that the cortex may be as concerned with representing reward contingencies as it is to representing properties of the physical world.

At the same time that these experimental advances were being made, a major new theoretical understanding of reinforcement learning has taken place that explains many of the experimental results. Peter Dayan and Read Montague showed that temporal-difference learning can account for the dopamine responses found by Wolfram Schultz in the VTA, and Montague has used functional MRI to observe similar responses in humans. He found that the activity for rewarding stimuli in both the nucleus accumbens and medial orbitofrontal cortex was greatest when stimuli were unpredictable. Sebastian Seung presented a general approach to reinforcement learning knows as the hedonistic neuron and showed how it could be applied to bird-song learning. Allison Doupe presented new results on how motivational factors, perhaps mediated by dopamine, influenced the degree of correlation between the spike trains between areas of the bird brain that are responsible for bird song.

XJ Wang reported that delay period activity observed in prefrontal cortical neurons could be modeled in the type of recurrent neural networks found in cortex, and that decision making could take place when integrated activity in these networks reached a threshold level. Terrence Sejnowski showed that the synchrony observed between spikes in a population of neurons, which are known to be modulated by attention, could have a major impact on the responses of neurons downstream. Scott Makeig showed evidence for large-scale coherent activity in the cortex that is linked to errors in performance. In particular, he has found that theta activity in the 6-8 Hz band originating in the anterior cingulate becomes phase-locked at the time of the error. These findings suggest that not only do rewards affect the firing rates of neurons, but also affect the degree of coherence within neural populations and between different brain regions.

Paul Glimcher reported that neurons in the prefrontal cortex respond according to the expected reward and that the responses are consistent with those predicted by classical game theory. In particular, the monkey responds in precisely the way that predicted by John Nash, who received a Nobel Prize in economics for proving the existence of these equilibria in certain types of games. Colin Camerer is an expert in game theory and showed how these approaches could be applied to studying humans in economic decision making, such as ultimatum bargaining games, and Michael Kearns extended the classical results to a wider range of games. Paul Glimcher argued that economic theory may provide an alternative to the traditional model of the brain and behavior.