An artificial intelligence could gauge what characters and locations people were viewing in the film (500) Days of Summer based on the activity of their neurons.
Movie characters and locations can be deciphered by monitoring the activity of people’s neurons while they watch a film, a discovery that could help us understand how the brain stores memories as they are being made.
 |
| A viewer’s brain activity told scientists when characters in the film (500) Days of Summer were on screen Collection Christophel / Alamy |
A deep brain structure called the hippocampus, which is critical for memory, contains neurons that encode the identity of objects, such as tools or buildings. The cells involved then respond to stimuli depicting those objects. But a lot of our knowledge about this comes from studying people while they view isolated images, rather than responding to changing scenes.
To learn more, Franziska Gerken at the Technical University of Munich in Germany and her colleagues looked at 29 people with epilepsy who had electrodes implanted into their brains to monitor for seizures before they underwent surgery for their condition. The researchers used this opportunity to record the activity of nearly 2300 neurons in and around each individual’s hippocampus while they watched the film (500) Days of Summer.
They then used an artificial intelligence model to analyse how these neural responses differed according to specific features of the film, such as characters and locations, and visual categories, like scene transitions.
When the participants rewatched the film, the team could tell the content they were viewing by looking at their neuronal responses.
The researchers found that neurons in all the regions in and around the hippocampus responded to the film’s features and categories, but activity in some regions reflected these more than others. For example, the activity of visually responsive neurons in a region called the parahippocampal gyrus was linked to scene transitions and locations, while the activity of cells in an adjacent region called the amygdala became activated with the presence of the film’s main characters.
Further analysis revealed that although several thousand cells were activated in response to seeing Summer, the female protagonist, her presence on screen could only be gauged from the activity of 500 neurons. But there was no consistent pattern of activity linked to Summer’s presence. Each time she appeared on screen, a different subset of cells became active.
This suggests that large numbers of neurons are activated when a movie character appears, depending on the context, the researchers say in their paper. But when it comes to remembering the character’s presence, this information is stored in smaller, changing subsets of neurons, rather than the entire neuronal population.
Rodrigo Quian Quiroga at the University of Leicester in the UK says the results are “very solid” and “a good step forwards to understanding how the memory system functions in real-life situations”.
“Neuroscience is full of experiments that are far from real-life memory functioning, so it’s valuable to analyse single neuron responses to different movie characters and scenes to disentangle the contribution of different neuron populations to encoding and storing real memories,” he says.
Reference:
bioRxiv DOI: 10.1101/2024.06.13.598791