The human brain often recalls past memories (seemingly) unprompted. As we go throughout our day, we have spontaneous flashes of memory from our lives. While this spontaneous conjuration of memories has long been of interest to neuroscientists, AI research company DeepMind recently published a paper detailing how an AI of theirs replicated this strange pattern of recall.
The conjuration of memories in the brain, neural replay, is tightly linked with the hippocampus. The hippocampus is a seahorse-shaped formation in the brain that belongs to the limbic system, and it is associated with the formation of new memories, as well as the emotions that memories spark. Current theories on the role of the hippocampi (there is one in each hemisphere of the brain), state that different regions of the hippocampus are responsible for the handling of different types of memories. For instance, spatial memory is believed to be handled in the rear region of the hippocampus.
As reported by Jesus Rodriguez on Medium, Dr. John OKeefe is responsible for many contributions to our understanding of the hippocampus, including the hippocampal place cells. The place cells in the hippocampus are triggered by stimuli in a specific environment. As an example, experiments on rats showed that specific neurons would fire when the rats ran through certain portions of a track. Researchers continued to monitor the rats even when they were resting, and they found that the same patterns of neurons denoting a portion of the maze would fire, although they fired at an accelerated speed. The rats seemed to be replaying the memories of the maze in their minds.
In humans, recalling memories is an important part of the learning process, but when trying to enable AI to learn, it is difficult to recreate the phenomenon.
The DeepMind team set about trying to recreate the phenomenon of recall using reinforcement learning. Reinforcement learning algorithms work by getting feedback from their interactions with the environment around them, getting rewarded whenever they take actions that bring them closer to the desired goal. In this context, the reinforcement learning agent records events and then plays them back at later times, with the system being reinforced to improve how efficiently it ends up recalling past experiences.
DeepMind added the replaying of experiences to a reinforcement learning algorithm using a replay buffer that would playback memories/recorded experiences to the system at specific times. Some versions of the system had the experiences played back in random orders while other models had pre-selected playback orders. While the researchers experimented with the order of playback for the reinforcement agents, they also experimented with different methods of replaying the experiences themselves.
There are two primary methods that are used to provide reinforcement algorithms with recalled experiences. These methods are the imagination replay method and the movie replay method. The DeepMind paper uses an analogy to describe both of the strategies:
Suppose you come home and, to your surprise and dismay, discover water pooling on your beautiful wooden floors. Stepping into the dining room, you find a broken vase. Then you hear a whimper, and you glance out the patio door to see your dog looking very guilty.
As reported by Rodriguez, the imagination replay method doesnt record the events in the order that they were experienced. Rather, a probable cause between the events is inferred. The events are inferred based on the agents understanding of the world. Meanwhile, the movie replay method stores memories in the order in which the events occurred, and replays the sequence of stimuli spilled water, broken vase, dog. The chronological ordering of events is preserved.
Research from the field of neuroscience implies that the movie replay method is integral to the creation of associations between concepts and the connection of neurons between events. Yet the imagination replay method could help the agent create new sequences when it reasons by analogy. For instance, the agent could reason that if a barrel is to oil as a vase is to water, a barrel could be spilled by a factory robot instead of a dog. Indeed, when DeepMind probed further into the possibilities of the imagination replay method, they found that their learning agent was able to create impressive, innovative sequences by taking previous experiences into account.
Most of the current progress being made in the area of reinforcement learning memory is being made with the movie strategy, although researchers have recently begun to make progress with the imagination strategy. Research into both methods of AI memory can not only enable better performance from reinforcement learning agents, but they can also help us gain new insight into how the human mind might function.
View post:
DeepMind Discovers AI Training Technique That May Also Work In Our Brains - Unite.AI
- Unlocking Flow: The Neuroscience of Creative Bliss - Neuroscience News - April 15th, 2024 [April 15th, 2024]
- Revolutionizing Glioblastoma Treatment - Neuroscience News - April 15th, 2024 [April 15th, 2024]
- Decoding spontaneous thoughts from the brain via machine learning - EurekAlert - April 15th, 2024 [April 15th, 2024]
- Reducing Toxic AI Responses - Neuroscience News - April 15th, 2024 [April 15th, 2024]
- Advancing the neuroscience of human pregnancy - Nature.com - April 15th, 2024 [April 15th, 2024]
- Angela Bryan Awarded Hazel Barnes Prize | Psychology and Neuroscience - University of Colorado Boulder - April 15th, 2024 [April 15th, 2024]
- How Old Are You in Your Head? The Neuroscience of Subjective Age Reveals How to Stay Mentally Young - Inc. - April 15th, 2024 [April 15th, 2024]
- Biohaven Showcases Innovative Neuroscience Portfolio with 20 Presentations at the 2024 American Academy of ... - PR Newswire - April 15th, 2024 [April 15th, 2024]
- Those Who Use Willpower Deemed More Trustworthy - Neuroscience News - April 15th, 2024 [April 15th, 2024]
- How Sex and Gender Shape Our Cognition - Neuroscience News - April 15th, 2024 [April 15th, 2024]
- War's Toll on the Brain: Widespread PTSD and Anxiety Among Ukrainians - Neuroscience News - April 15th, 2024 [April 15th, 2024]
- Paints and Pesticides Linked to ALS Risk - Neuroscience News - April 15th, 2024 [April 15th, 2024]
- AI STORIES: A New Vision for AI and Narratives - Neuroscience News - April 15th, 2024 [April 15th, 2024]
- Neuroscience Breakthrough Unveils How We Learn and Remember - SciTechDaily - April 15th, 2024 [April 15th, 2024]
- Pregnancy's Toll: Accelerated Aging in Young Mothers - Neuroscience News - April 15th, 2024 [April 15th, 2024]
- Learning and Memory Formation's Molecular Basis - Neuroscience News - April 15th, 2024 [April 15th, 2024]
- Acetaminophen in Pregnancy Not Linked to Autism, ADHD Risk - Neuroscience News - April 15th, 2024 [April 15th, 2024]
- Childhood Loneliness Linked to Later Psychosis - Neuroscience News - April 15th, 2024 [April 15th, 2024]
- Vigil Neuroscience, Inc. (NASDAQ:VIGL) Expected to Post Q1 2024 Earnings of ($0.58) Per Share - Defense World - March 29th, 2024 [March 29th, 2024]
- International Brain Research Organisation (IBRO) Diversity Grants 2024 Opportunity Desk - Opportunity Desk - March 29th, 2024 [March 29th, 2024]
- Hereditary Alzheimer's Transmitted Via Bone Marrow Transplants - Neuroscience News - March 29th, 2024 [March 29th, 2024]
- Household Chemicals Linked to Brain Health Risks - Neuroscience News - March 29th, 2024 [March 29th, 2024]
- Cheers to Longevity: Couples Who Drink Together, Live Longer - Neuroscience News - March 29th, 2024 [March 29th, 2024]
- Sleeplessness Makes You Feel Up To Ten Years Older - Neuroscience News - March 29th, 2024 [March 29th, 2024]
- Aging Brain Cells Have Prolonged Death Process - Neuroscience News - March 29th, 2024 [March 29th, 2024]
- Anxiety Drives Wishful Thinking to Risky Levels - Neuroscience News - March 29th, 2024 [March 29th, 2024]
- Prolonged Progestogen Use Linked to Brain Tumor Risk - Neuroscience News - March 29th, 2024 [March 29th, 2024]
- The Genetic Secrets of Neuron Formation - Neuroscience News - March 29th, 2024 [March 29th, 2024]
- Vigil Neuroscience Reports Fourth Quarter and Full Year 2023 Financial Results and Provides Business Update - Yahoo Finance - March 29th, 2024 [March 29th, 2024]
- Big research, little time: Medical neuroscience student wins 3 Minute Thesis finals - Dal News - March 29th, 2024 [March 29th, 2024]
- The new NeuroAI - Nature.com - March 29th, 2024 [March 29th, 2024]
- Exploring Aphantasia: The Mind Without a Mental Picture - Neuroscience News - March 29th, 2024 [March 29th, 2024]
- Altered Brain pH Linked to Cognitive Disorders - Neuroscience News - March 29th, 2024 [March 29th, 2024]
- Emotion vs. Reason: Rethinking Decision-Making - Neuroscience News - March 29th, 2024 [March 29th, 2024]
- DNA Damage and Inflammation Key to Memory Formation - Neuroscience News - March 29th, 2024 [March 29th, 2024]
- Embracing Neurodiversity: Beyond Stigma to Strength - Neuroscience News - March 29th, 2024 [March 29th, 2024]
- Devil in the Details: The Visual World of Prosopometamorphopsia - Neuroscience News - March 29th, 2024 [March 29th, 2024]
- Rockefeller Neuroscience Institute receives $2M grant for ultrasound drug addiction treatments - Health Imaging - March 29th, 2024 [March 29th, 2024]
- Empathy Begins in Infancy - Neuroscience News - March 29th, 2024 [March 29th, 2024]
- Mapping Brain Activity with Brain-Computer Interface Technology - Neuroscience News - March 29th, 2024 [March 29th, 2024]
- Want Better Mental Focus? Neuroscience Just Found This Simple, Joyful Activity Helps Your Brain Concentrate - Inc. - March 29th, 2024 [March 29th, 2024]
- The neuroscience of groove: Why certain rhythms make us want to dance - PsyPost - March 29th, 2024 [March 29th, 2024]
- High Rates of Suicidal Ideation in Rural Black Men - Neuroscience News - March 29th, 2024 [March 29th, 2024]
- Harmony in Chaos: How BPD Influences Music Tastes - Neuroscience News - March 29th, 2024 [March 29th, 2024]
- WVU Rockefeller Neuroscience Institute receives $2M Grant - WDTV - March 29th, 2024 [March 29th, 2024]
- Study Links Diet, Diabetes, and Alzheimers - Neuroscience News - March 29th, 2024 [March 29th, 2024]
- Neuroscience and Society Series: Aligning Science with the Public's Values - The Hastings Center - March 29th, 2024 [March 29th, 2024]
- Hardwiring Your Brain: The Neuroscience Of Behaviour Change | TheHealthSite.com - TheHealthSite - March 29th, 2024 [March 29th, 2024]
- Refined Carb Intake's Effect on Facial Attractiveness - Neuroscience News - March 11th, 2024 [March 11th, 2024]
- Linking Childhood Adversity to Adult Mental Health - Neuroscience News - March 11th, 2024 [March 11th, 2024]
- Overnight neuronal plasticity and adaptation to emotional distress - Nature.com - March 11th, 2024 [March 11th, 2024]
- Less Obesity and Depression in NPD Individuals - Neuroscience News - March 11th, 2024 [March 11th, 2024]
- Unlocking Creative Flow: How the Brain Enters the Zone - Neuroscience News - March 11th, 2024 [March 11th, 2024]
- Self-Control, Not Impulsivity, Paves the Way to Power - Neuroscience News - March 11th, 2024 [March 11th, 2024]
- 3D Maps Reveal Molecular Complexities of the Brain - Neuroscience News - March 11th, 2024 [March 11th, 2024]
- Loneliness Linked to Personality Disorders - Neuroscience News - March 11th, 2024 [March 11th, 2024]
- Gen Z's Climate Anxiety: A Call for Action and Hope - Neuroscience News - March 11th, 2024 [March 11th, 2024]
- Decoding Emotions: Beyond Senses in the Human Brain - Neuroscience News - March 11th, 2024 [March 11th, 2024]
- Brain Circuit Balances Speech and Breath - Neuroscience News - March 11th, 2024 [March 11th, 2024]
- Schizophrenia and Aging Share Brain Changes - Neuroscience News - March 11th, 2024 [March 11th, 2024]
- Monitoring Spinal Cord Activity During Surgery in Real-Time - Neuroscience News - March 11th, 2024 [March 11th, 2024]
- Scientists team up with jazz musicians to reveal the neuroscience of creative flow - PsyPost - March 11th, 2024 [March 11th, 2024]
- Peer Pressure Persists Through Adulthood - Neuroscience News - March 11th, 2024 [March 11th, 2024]
- Mary Bartlett Bunge, 92, Dies; Pioneer in Spinal Injury Treatment - The New York Times - March 11th, 2024 [March 11th, 2024]
- Rethinking Reminiscence: Theater's Ageism Dilemma Unveiled - Neuroscience News - March 11th, 2024 [March 11th, 2024]
- PTSD: Virtual Reality and Magnetic Stimulation Provide Relief - Neuroscience News - March 11th, 2024 [March 11th, 2024]
- The Partisan Morality Divide: When Politics Shapes Right and Wrong - Neuroscience News - March 11th, 2024 [March 11th, 2024]
- Biological Sciences Professor Terrence Sejnowski Wins Brain Prize - University of California San Diego - March 11th, 2024 [March 11th, 2024]
- Pioneering work in computational and theoretical neuroscience is awarded the world's largest brain research prize - afp.com - March 11th, 2024 [March 11th, 2024]
- Substance Abuse Beyond Dopamine and Impulsivity - Neuroscience News - March 11th, 2024 [March 11th, 2024]
- Pioneering work in computational and theoretical neuroscience is awarded the world's largest brain research prize - Business Wire - March 11th, 2024 [March 11th, 2024]
- Dal study of football players sheds light on how concussions affect the brain - Yahoo Canada Sports - March 11th, 2024 [March 11th, 2024]
- Hebrew University's Haim Sompolinsky awarded prestigious Brain Prize for pioneering neuroscience research - The Times of Israel - March 11th, 2024 [March 11th, 2024]
- Harvard neuroscientist Haim Sompolinsky awarded Brain Prize - EurekAlert - March 11th, 2024 [March 11th, 2024]
- Rhythm in the Brain: Music Exposure Influences Rhythmic Interpretation - Neuroscience News - March 11th, 2024 [March 11th, 2024]
- Physicist Haim Sompolinsky first Israeli to win largest brain science research prize - The Times of Israel - March 11th, 2024 [March 11th, 2024]
- Prestigious 2024 Brain Prize awarded to Hebrew University's Prof. Haim Sompolinsky by Lundbeck Foundation - EurekAlert - March 11th, 2024 [March 11th, 2024]
- Pioneering work in computational and theoretical neuroscience is awarded the world's largest brain research prize - EurekAlert - March 11th, 2024 [March 11th, 2024]
- Role of AI in Neuroscience Research and Understanding of the Human Brain - Medriva - February 13th, 2024 [February 13th, 2024]
- Healis selected to present at key biotechnology and neuroscience investor conferences in Amsterdam, New York, - EIN News - February 13th, 2024 [February 13th, 2024]