HAAM Index / Film + mind · July 4, 2026 · 13 min read
The Film Is an Experiment
How neuroscience uses films to study attention, memory, emotion, narrative, and shared experience, from fMRI and eye tracking to the CognitiveNeuro&Movies group at NCKU.
Traditional experiments often isolate one image, one word, or one decision. A film does the opposite. It coordinates faces, voices, movement, music, space, expectation, memory, and social meaning across time.
That complexity once made movies look too messy for controlled science. It now makes them useful. A film is a repeatable stream of events. Every participant can receive the same frames and sounds at the same moments, while the experience remains closer to ordinary perception than a sequence of disconnected laboratory flashes.
The field is often called neurocinematics, but its purpose is broader than helping studios optimise scenes. Movies let scientists study the brain during continuous, emotionally meaningful, socially rich experience. Film becomes both an object of study and an instrument for studying human cognition.
A movie is one of the rare scientific instruments that can deliver the same complex world to many minds, frame by frame.
A local example in Tainan
CognitiveNeuro&Movies at NCKU
At National Cheng Kung University, the CognitiveNeuro&Movies group creates a bridge between people who think through film and people who think through experiments. It grows from the NCKU context around the course 認知神經科學與電影 (Rènzhī shénjīng kēxué yǔ diànyǐng), Cognitive Neuroscience and Movies.
NCKU's Department of Psychology explicitly places the course inside an applied programme connecting mind and brain science with real-world fields. The university's Mind Research and Imaging Center provides the larger technical setting: MRI, MR-EEG, visual and auditory stimulus systems, response boxes, and analysis support.
The group matters as an educational model because the most interesting questions do not belong to one discipline. A neuroscientist may know how to measure intersubject correlation. A filmmaker may know that a reaction begins before the cut, through blocking, sound, withheld information, and genre expectation. Film studies can explain why an edit matters. Neuroscience can test when and for whom its effects appear.
The value is not using science to certify a film. It is learning how to turn cinematic intuition into a researchable question without flattening the artwork.
Enter the movie lab
Every instrument sees a different film.
Select a method to see its signal, useful questions, and blind spots. No single measurement can explain an audience by itself.
01 / fMRI
Changes in blood oxygenation across the brain
Maps activity across the whole brain with relatively precise spatial detail.
What it can reveal
Which brain systems follow faces, motion, language, emotion, memory, social inference, and the longer structure of a narrative.
Questions it can support
- Do viewers' brains respond in similar ways at the same moments?
- Which scenes are later remembered?
- How does narrative context change the meaning of the same image?
What it cannot prove alone
The signal is indirect and slow. A brief cut or sound can happen much faster than the measured blood response.
From scene to signal
How a film becomes an experiment
The film does not enter the lab as a mysterious whole. Researchers turn its formal decisions into timed variables, then connect those variables to several layers of evidence.
01
Choose a film question
Start with a cinematic operation: a cut, a camera move, a face, a subtitle, a sound cue, a reversal, a suspense interval, or a change in narrative point of view.
02
Create a comparison
Researchers might compare an edited and unedited sequence, alter the soundtrack, scramble scenes, remove context, change subtitles, or compare audiences with different experience or traits.
03
Turn the film into a timeline
Every frame, cut, line, sound, face, movement, event boundary, rating, and participant response can be timestamped so that the movie and the measurements share one clock.
04
Record several kinds of evidence
Brain imaging can be combined with gaze, pupil size, heart rate, skin conductance, breathing, memory tests, scene descriptions, and moment-by-moment emotion ratings.
05
Compare people and moments
Analysis can ask which moments produce similar responses across viewers, where groups diverge, which film features predict a signal, and which responses are associated with later memory or understanding.
06
Return to the film
The final task is interpretation. A neural pattern is not a theory of cinema by itself. It becomes useful when connected back to editing, sound, performance, genre, culture, spectatorship, and lived experience.
Research examples
Six ways scientists have used movies to make the invisible measurable
2004
Science
A film can synchronize activity across different brains
Uri Hasson and colleagues recorded fMRI while participants watched part of The Good, the Bad and the Ugly. Even during free viewing, many cortical responses followed a shared time course across viewers. The study introduced intersubject correlation as a way to let a complex film provide its own experimental timeline.
The same movie can align parts of perception and cognition without asking everyone to press a button after every shot.
Read the study2008
Projections
Different films exert different amounts of attentional control
The Neurocinematics paper compared responses to several kinds of moving-image material. Carefully directed sequences produced more widespread intersubject correlation than less structured footage, showing that cinematic form can change how tightly an audience is guided through time.
Synchrony is not a quality score. It is evidence that a work is making viewers process some moments in more similar ways.
Read the study2008
Neuron
Shared responses can predict which moments enter memory
A later study combined intersubject correlation with a subsequent-memory test. Moments that were successfully remembered were associated with stronger shared responses in parts of the brain involved in processing and memory formation.
Researchers can connect the unfolding film to what survives after the screen goes dark.
Read the study2015
Cortex
Editing changes the neural map produced by the same performance
Researchers showed viewers two audiovisual versions of the same dance and examined how edits changed intersubject correlation. The work made editing itself an experimental variable rather than treating a finished movie as an indivisible stimulus.
A cut does not only join images. It reorganizes the timing and distribution of audience processing.
Read the study2024
Scientific Data
The same movie can connect single neurons, eye movements, memory, and fMRI
A multimodal dataset recorded 20 clinical participants watching an eight-minute excerpt of Alfred Hitchcock's Bang! You're Dead. It includes intracranial signals, eye tracking, recognition memory, and fMRI from 11 of the same participants, allowing direct comparison across measurement scales.
The movie becomes a common coordinate system across methods that normally live in separate experiments.
Read the study2025
Frontiers in Psychology
VR editing can be tested through gaze, attention, and reported experience
A study with 42 participants compared unedited, hard-cut, and dissolve-transition versions of a VR movie using eye tracking and questionnaires about emotion, immersion, and cognitive load. It shows how film research is moving beyond flat screens into spatial media.
New formats create new editing problems, but they can still be studied through timed behavior and experience.
Read the studyThe most revealing result is not always where viewers become synchronized. It may be where a film permits their minds to separate.
Keep the claims honest
What a brain scan cannot tell a filmmaker
Neurocinematics becomes weak when it is sold as mind reading or a machine for producing guaranteed hits. The science is more useful, and more interesting, when its limits remain visible.
Brain activity is not a secret review score
A more intense signal does not automatically mean a scene is better, more beautiful, more persuasive, or more meaningful. The interpretation depends on the task, region, timing, comparison, and other evidence.
Synchrony is not sameness
Two viewers can show similar sensory timing while understanding a character, moral conflict, joke, or political meaning very differently. Shared response and individual interpretation can coexist.
The scanner changes the cinema
Watching alone, lying still in a loud MRI scanner with a mirror and headphones is not the same as watching in a crowded theatre. Naturalistic stimuli make experiments richer, but the situation remains designed.
Average audiences can hide real audiences
Culture, language, age, expertise, neurodiversity, memory, identity, and personal history can all change a response. A group average should not erase the differences that film itself can make visible.
Reverse inference is a trap
Seeing activity in a region associated with fear does not prove that a participant felt fear. Most brain regions support several processes, so strong claims require converging behavioral and experimental evidence.
Neuroscience does not replace criticism
Measurement can reveal timing, pattern, and difference. It cannot decide what a film should mean, whose experience matters, whether a representation is ethical, or how a work belongs to history.
A research agenda for film people
Better questions begin where film form meets human difference.
- 01Where does the film ask different viewers to look at the same thing?
- 02Which event boundaries divide the story into memorable episodes?
- 03When does sound lead the image, and when does the image rewrite the sound?
- 04Which cuts disappear into continuity, and which become consciously felt?
- 05Where do viewers share a response, and where does the film create room for divergence?
- 06How do subtitles, language, disability, expertise, and culture alter the route through a scene?
These questions can inform cinema, games, immersive media, accessibility, education, interaction design, and AI-generated drama. They do not ask science to replace creative judgment. They ask creative judgment to become more curious about the minds and bodies it is designing for.
The larger idea
Film does not merely show cognition. It organizes cognition in time.
A director cannot control what every viewer thinks. But a film can coordinate when information appears, when attention shifts, when a possibility becomes likely, when a memory must be retrieved, and when an event feels complete. Neuroscience gives us methods for observing parts of that choreography.
The deeper opportunity is not to find a biological formula for cinema. It is to build a richer conversation between measurable response and irreducible experience. The brain data can show where something happened. Film language, culture, and criticism help explain what that happening means.
Sources and further reading
Follow the evidence
- 01NCKU Department of Psychology: development and applied cognitive neuroscience curriculum
- 02NCKU Mind Research and Imaging Center
- 03Hasson et al. (2004): Intersubject Synchronization of Cortical Activity During Natural Vision
- 04Hasson et al. (2008): Neurocinematics: The Neuroscience of Film
- 05Hasson et al. (2008): Enhanced Intersubject Correlations During Movie Viewing Correlate with Successful Episodic Encoding
- 06Herbec et al. (2015): The effect of editing on the neural representation of audiovisual stimuli
- 07Nastase et al. (2019): Measuring shared responses across subjects using intersubject correlation
- 08Vanderwal et al. (2019): Movies in the magnet: Naturalistic paradigms in developmental functional neuroimaging
- 09Keles et al. (2024): Multimodal single-neuron, intracranial EEG, and fMRI responses during movie watching
- 10Zou et al. (2025): The neural impact of editing on viewer narrative cognition in VR films