Learning

QOTD

Kylie Dale Can people ever condition themselves to permanently change their behaviors? A. Yes B. No C. Sometimes D. I'm not sure

Jacqueline Turner Do you think it is more difficult to learn a physical task, such as sport, or an abstract concept, such as the biology of a cell? A. Sports are way easier for me B. Biology and school just come naturally C. Both are about the same difficulty D. Depends on the sport or the school subject E. I'm just not really sure

Margaret Sheahan, Tamar Hedeshian Was there ever a time where you truly tried to learn something but couldn't? a) yes b) no c) Maybe

Margaret Sheahan, Tamar Hedeshian In principle, should you be able to learn anything (e.g., quantum physics or whatever) if you tried hard enough? a) yes b) no c) Maybe

Steven Hoernicke Do you believe that violent behvior is mostly influenced by violence in media (ie video games, movies, shows)? A.Yes B.No C. Violent behavior is influence by another factor more

The Big Questions / Issues Learning is the most important feature of the human brain: we learn almost everything!  The textbook barely scratches the surface..  In part because… it’s complicated… and unsettled How does dopamine-based reinforcement learning work?  Role of dopamine in the basal ganglia  Key dopamine lesson: expectations vs. outcomes

What Learns? Amazing fact: we know exactly what part of individual neurons learns.

What Changes?? 10

Gettin’ AMPA’d 11

Synapses Change Strength (in response to patterns of activity) 12

Which Way? 13 Low Ca = “long term depression” – synapse gets weaker High Ca = “long term potentiation” – synapse gets stronger

Learning Rules Across the Brain 14 + = has to some extent … +++ = defining characteristic – definitely has - = not likely to have … = definitely does not have Learning SignalDynamics

Learning happens where it’s used (memory => processing) Basal ganglia: learning what actions (not) to use - based on reward / punishment (operant) Cerebellum: learning to perfect actions - based on error signals (e.g., feeling awkward) Neocortex: learning how to see, hear, speak, reach, act, socialize… everything! Hippocampus: learning snapshots of everything

Basal Ganglia and Action Selection 16

Release from Inhibition 17

Reinforcement Learning: Dopamine 18 CS = Tone R = Juice drop Classical conditioning happens in dopamine

Basal Ganglia Operant Learning (Frank, 2005…; O’Reilly & Frank 2006) 19 Dopamine burst = do more of what you just did (Law of Effect) Dopamine dip = do less of what you just did (bad outcome!) -> Classical conditioning drives operant conditioning!!

Dopamine Lessons Dopamine = Outcome – Expectation Should you just always have low expectations, so even low outcomes seem good?? I try hard to avoid hearing anything about movies

What about Neocortex?? How does all the actual important learning take place??

Umm, It’s Complicated…

Floating Threshold = Medium Term Synaptic Activity (Error-Driven) 23 dW = Outcome – Expectation = s - m

Where do the Targets Come From?

Latent Learning Humans exhibit massive amount of “latent learning” in neocortex and hippocampus Only a tiny bit is ever expressed in behavior  Much of it is evident in rich, elaborate dreams  Or when people sit down and write novels..

Wade Myers if almost all the information we know is learned, is there such thing as an original idea? A. Yes B. No