Real-world memory and the brain
How do we construct and retrieve memories of complex real-world episodes? In this research we use realistic stimuli (such as movies and narratives) and behaviors (such as spoken recall) that contain rich natural semantics and unfold continuously across multiple timescales. Employing between-brain temporal and pattern analysis methods, we ask how mnemonic and sensory systems operate together dynamically to create the present moment.
In the mind, the present moment is a convergence point of two information streams: one, a continuous flow of sensory input from the outside world; and two, a series of elements from our past experiences, i.e., memories. Memories may be triggered by sensory stimuli, they may themselves cue more memories, and they may change the way incoming stimuli are interpreted, all of which become part and parcel of our current experience.
Past information casts an influence across multiple timescales: events that occurred a moment ago, a minute ago, and a day ago may all impact the present. In order to understand how the mind and brain work, we need an account of how memories of past events, across multiple timescales, continuously influence and merge with ongoing perception and behavior.
Studying real memory requires using real stimuli. Scientists often trade realism for control; we use lists or configurations of random items, attempting to isolate selected variables. However, this approach can strip away the very richness and complexity that made memory such a compelling topic in the first place, and cause us to neglect phenomena that emerge only when stimuli are as dynamic and detailed as the real world.
My work aims to understand how we construct and retrieve memories of complex real-world episodes. I use realistic stimuli (such as movies and narratives) and behaviors (such as spoken recall) that contain rich natural semantics and unfold continuously across multiple timescales. Using novel between-brain temporal and pattern analysis methods, I ask how mnemonic and sensory systems operate together dynamically to create the present moment.
Lee H, Bellana B, Chen J (2020). What can narratives tell us about the neural bases of human memory? Current Opinion in Behavioral Sciences. [PDF]
Zuo X, Honey CJ, Barense MD, Crombie D, Norman KA, Hasson U, Chen J (2020). Temporal integration of narrative information in a hippocampal amnesic patient. NeuroImage. [PDF]
Sadeh T, Chen J, Y Goshen-Gottstein, Moscovitch M (2019). Overlap between hippocampal pre-encoding and encoding patterns supports episodic memory. Hippocampus. [PDF]
Aly M, Chen J, Turk-Browne N, Hasson U (2018). Learning naturalistic temporal structure in the posterior medial network. Journal of Cognitive Neuroscience. [PDF]
Baldassano C, Chen J, Zadbood A, Pillow JW, Hasson U, & Norman KA (2017). Discovering event structure in continuous narrative perception and memory. Neuron. [PDF]
Zadbood A, Chen J, Leong YC, Norman KA, & Hasson U (2017). How we transmit memories to other brains: constructing shared neural representations via communication. Cerebral Cortex. [PDF]
Chen J*, Leong YC*, Honey CJ, Yong CH, Norman KA, Hasson U (2017). Shared memories reveal shared structure in neural activity across individuals. Nature Neuroscience. (*co-authorship) [PDF]
Chen J, Honey CJ, Simony E, Arcaro MJ, Norman KA, Hasson U (2016). Accessing real-life episodic information from minutes versus hours earlier modulates hippocampal and high-order cortical dynamics. Cerebral Cortex. [PDF]
Shared experience, shared memory. Patterns in the brain which emerge during perception are later reactivated during spoken recall, are robustly similar across different individuals, and transform systematically between perception and memory.
A hierarchy of processing timescales. In order to interpret a continuous stream of input from the world, the brain must integrate information over multiple timescales. We propose that stimulus processing is distributed across a hierarchy of cortical regions, with processing timescales increasing along a gradient from low-level sensory areas (e.g., visual cortex) up to high-level association areas (e.g., default network).
[Hierarchical process memory: memory as an integral component of information processing. U Hasson, J Chen, CJ Honey; Trends in cognitive sciences 19 (6), 304-313.] [Processing timescales as an organizing principle for primate cortex (commentary). J Chen, U Hasson, CJ Honey; Neuron 88 (2), 244-246.] [How long is now? The multiple timescales of language processing (commentary). CJ Honey, J Chen, K Müsch, U Hasson; The Behavioral and brain sciences 39, e77.]
Under natural conditions, memories can persist in high-order cortex for minutes. It is well known that formation of new episodic memories depends on the hippocampus, but in real- life settings (e.g., conversation), hippocampal amnesics can utilize information from minutes earlier. What neural systems outside the hippocampus support this minutes-long retention? My work using functional MRI in the healthy brain suggests that default network cortical regions can intrinsically retain information for several minutes during continuous, semantically rich natural stimulation.
Analyses of functional neuroimaging data
Naturalistic stimuli and behavior
Collect brain data as people watch movies and listen to stories, and as they describe their memories out loud
Compare activity between the brains of different people, both in the temporal domain and the spatial domain
Multi-voxel pattern classification
Identify neural patterns that are specific to certain periods in the stimulus, such as a particular movie scene
Semantic model construction
Predict patterns of neural activity given combinations of stimulus features
Princeton Dataspace (preprocessed NIFTI)
OpenNeuro (Raw BIDS)
Datalad (preprocessed for MATLAB tutorial)
Publications & Preprints that use the Sherlock dataset
(from our group and others)
Heusser et al. (2018) bioRxiv
Manning (2019) PsyArXiv
Tan et al. (2019) arXiv
Thornton & Tamir (2019) PsyArXiv
Thornton & Tamir (2020) Social Cognitive and Affective Neuroscience
Liu et al. (2020) bioRxiv
Jolly et al. (2020) Neuroimage
Kim et al. (2020) Neuropsychologia
Brandman et al. (2020) bioRxiv
Song et al. (2020) bioRxiv
Brandman et al. (2021) Communications Biology
Postdoctoral Research Fellow
I received my BA in Psychology at Carleton College and my PhD in Psychology at the University of Pennsylvania. My doctoral dissertation investigated how task context modulates item-level neural representations of word meanings.
In my post-doctoral work, I plan to study how information is transformed in the brain across changes in time and representational format, for example from (1) initial encoding to subsequent recollection, and from (2) an observed sequence of audiovisual events to a retrieved and verbalized narrative. I will use a combination of neuroimaging and behavioral methods to address these questions.
Postdoctoral Research Fellow
I received my BA and MA in Psychology at Yonsei University, South Korea. I did my PhD in Cognitive Neuroscience at New York University, where I studied mnemonic content representations in human posterior parietal cortex using functional neuroimaging.
I am broadly interested in how high-level association areas in the brain support complex cognitive functions such as remembering naturalistic events. In my current project, I am studying the relationship between the structure of narratives and neural responses in the default mode network regions during encoding and recall of movies and stories.
Postdoctoral Research Fellow
I am broadly interested in the ideas of depth of processing, and what it means to deeply engage with incoming information.
My doctoral work at the University of Toronto explored this topic via episodic memory. There, I examined the influence of prior knowledge on our ability to recollect recent experiences, using measures of behaviour and fMRI.
During my post-doc, I will explore depth of processing using narratives. Narratives provide a natural example where we draw upon prior knowledge to contextualize and enrich the concrete details of a story. Using fMRI, I hope to gain insight into 1) the neural systems that can reliably predict moments when we are deeply processing a narrative, and 2) the functional contributions of these neural systems to deep processing.
I’m a PhD student who joined Chen lab in the Fall of 2019. In my PhD work, I hope that I could understand how pieces of low-level sensory information are integrated and transformed into episodic memory. In my ongoing work, I’m investigating whether scene-specific brain patterns in the default mode network, evoked during naturalistic movie-viewing, have a low-dimensional organization, i.e., can be described by a small number of ‘basis’ brain states.
Before joining Chen lab, I received BA in Economics at Chung-Ang University, and MA in Psychology at Yonsei University, South Korea. I love plants, yoga and traveling. I was born and grew up in Seoul where over nine millions of Koreans live, and where you can find beautiful old palaces and glassy modern buildings at the same time.
I’m a Ph.D. student (Fall 2020) in Chen lab. Broadly, I am interested in understanding how the human brain understands and encodes on-going events, how such encoded information may be modified over time as new relevant information and demands come in, and how our brain retrieves such memory. I am also interested in building computational models to stimulate relevant brain activity and behavioral results to aid theories and hypotheses.
Before joining the lab, I have received my BA at Capital Normal university in Beijing, MA in Integrative Neuroscience at the University of Chicago, and have worked as a Research Associate at Harvard Medical School. My past research incorporated various neuroimaging and electrophysiological techniques with behavioral and survey data to explore the relationship between brain and behavior.
I received my BS in neuroscience from Johns Hopkins in 2020, and have been the lab manager for the Chen and Honey labs since the summer of 2019.
I’m currently involved in a project investigating the information density of naturalistic stimuli and the relationship with memory. In particular, I've examined how having prior context and the context dependence of a movie scene influences how a viewer perceives the amount of information in the scene. I'm also investigating how the use of language through narration, dialogue, and text in movies is transformed during recall.
I received my BA in Psychology and English from Johns Hopkins in the spring of 2021, and I currently work full time as an undergraduate Admissions Officer for Johns Hopkins.
My primary research interests include memory and perception. Specifically, I am interested in combining my passion for literature with psychology by studying fictional narratives and how their structures can affect our memories of these stories.
Undergraduate Research Assistant
I am a sophomore double majoring in neuroscience and cognitive science. I am primarily interested in the neural basis of memory, how emotions can affect it, and how it is applicable in everyday life.
Currently, I am working on a project to see how the disjointed nature of Internet browsing affects a narrative's built-in recall.
Undergraduate Research Assistant
I'm a junior double majoring in psychology and neuroscience.
The project I'm assisting in the lab focuses on how story causality and choice may affect memory recall. My overarching research interests mainly lie in the neural encoding of memory and emotions, computational modeling of their activities, as well as how these cognitive functions may relate to mental health treatment.
Not currently recruiting.