• Real-world memory and the brain

    How do we construct and retrieve memories of complex real-world episodes? In this research we use realistic stimuli (such as movies and narratives) and behaviors (such as spoken recall) that contain rich natural semantics and unfold continuously across multiple timescales. Employing between-brain temporal and pattern analysis methods, we ask how mnemonic and sensory systems operate together dynamically to create the present moment.

    ...

     

  • Research

    In the mind, the present moment is a convergence point of two information streams: one, a continuous flow of sensory input from the outside world; and two, a series of elements from our past experiences, i.e., memories. Memories may be triggered by sensory stimuli, they may themselves cue more memories, and they may change the way incoming stimuli are interpreted, all of which become part and parcel of our current experience.

     

    Past information casts an influence across multiple timescales: events that occurred a moment ago, a minute ago, and a day ago may all impact the present. In order to understand how the mind and brain work, we need an account of how memories of past events, across multiple timescales, continuously influence and merge with ongoing perception and behavior.

     

    Studying real memory requires using real stimuli. Scientists often trade realism for control; we use lists or configurations of random items, attempting to isolate selected variables. However, this approach can strip away the very richness and complexity that made memory such a compelling topic in the first place, and cause us to neglect phenomena that emerge only when stimuli are as dynamic and detailed as the real world.

     

    My work aims to understand how we construct and retrieve memories of complex real-world episodes. I use realistic stimuli (such as movies and narratives) and behaviors (such as spoken recall) that contain rich natural semantics and unfold continuously across multiple timescales. Using novel between-brain temporal and pattern analysis methods, I ask how mnemonic and sensory systems operate together dynamically to create the present moment.

    ...

  • Snapshots

    Shared experience, shared memory. Patterns in the brain which emerge during perception are later reactivated during spoken recall, are robustly similar across different individuals, and transform systematically between perception and memory.

    A hierarchy of processing timescales. In order to interpret a continuous stream of input from the world, the brain must integrate information over multiple timescales. We propose that stimulus processing is distributed across a hierarchy of cortical regions, with processing timescales increasing along a gradient from low-level sensory areas (e.g., visual cortex) up to high-level association areas (e.g., default network).

    Under natural conditions, memories can persist in high-order cortex for minutes. It is well known that formation of new episodic memories depends on the hippocampus, but in real- life settings (e.g., conversation), hippocampal amnesics can utilize information from minutes earlier. What neural systems outside the hippocampus support this minutes-long retention? My work using functional MRI in the healthy brain suggests that default network cortical regions can intrinsically retain information for several minutes during continuous, semantically rich natural stimulation.

  • Methods

    Analyses of functional neuroimaging data

    Naturalistic stimuli and behavior

    Collect brain data as people watch movies and listen to stories, and as they describe their memories out loud

    Inter-subject correlation

    Compare activity between the brains of different people, both in the temporal domain and the spatial domain

    Multi-voxel pattern classification

    Identify neural patterns that are specific to certain periods in the stimulus, such as a particular movie scene

    Semantic model construction

    Predict patterns of neural activity given combinations of stimulus features

  • People

    Principal Investigator

    Assistant Professor

    Postdoctoral Research Fellow

    I received my BA in Psychology at Carleton College and my PhD in Psychology at the University of Pennsylvania. My doctoral dissertation investigated how task context modulates item-level neural representations of word meanings.

     

    In my post-doctoral work, I plan to study how information is transformed in the brain across changes in time and representational format, for example from (1) initial encoding to subsequent recollection, and from (2) an observed sequence of audiovisual events to a retrieved and verbalized narrative. I will use a combination of neuroimaging and behavioral methods to address these questions.

    Postdoctoral Research Fellow

    I received my BA and MA in Psychology at Yonsei University, South Korea. I did my PhD in Cognitive Neuroscience at New York University, where I studied mnemonic content representations in human posterior parietal cortex using functional neuroimaging.

     

    I am broadly interested in how high-level association areas in the brain support complex cognitive functions such as remembering naturalistic events. In my current project, I am studying the relationship between the structure of narratives and neural responses in the default mode network regions during encoding and recall of movies and stories.

    Postdoctoral Research Fellow

    I am broadly interested in the ideas of depth of processing, and what it means to deeply engage with incoming information.

     

    My doctoral work at the University of Toronto explored this topic via episodic memory. There, I examined the influence of prior knowledge on our ability to recollect recent experiences, using measures of behaviour and fMRI.

     

    During my post-doc, I will explore depth of processing using narratives. Narratives provide a natural example where we draw upon prior knowledge to contextualize and enrich the concrete details of a story. Using fMRI, I hope to gain insight into 1) the neural systems that can reliably predict moments when we are deeply processing a narrative, and 2) the functional contributions of these neural systems to deep processing.

    Graduate Student

    What does it mean for two people to be thinking about the same thing? Remembering the same event? How does communication allow us to come to shared understandings despite differences in experience? How are these similarities in concepts, memory, and understanding reflected in the brain? I hope to develop a better grasp of the nature and answers to questions like these by taking advantage of the power and flexibility of intersubject fMRI analyses using complex, naturalistic stimuli.

     

    I did my undergraduate work at Princeton University where I landed in Psychology from a background in Computer Science and Philosophy, which both seemed too solipsistic (for different reasons). Past research topics I've worked with include the computational underpinnings of human learning, neural network models of cognition, and the statistical properties of methods in fMRI analysis.

    Lab Manager

    I received my bachelor's degree in Cognitive Science at UCLA, where I worked with Dr. Jesse Rissman on fMRI connectivity analysis and Dr. Michelle Craske on positive and negative emotions in people with anxiety and depression.

     

    My current project involves using naturalistic stimuli fMRI data to understand hippocampal involvement in long-timescale abilities in healthy and hippocampal amenesic brains. At the same time, I'm interested in developing a functionally informed anatomical alignment method to map between healthy and amnesic brains.

     

    Please don't hesitate to reach out to us if you’d like to find out more about who we are and what we do :)

    Undergraduate Research Assistant

    I’m a junior Psychology major student with a second major in Cognitive Science. I am an international student from Wuxi, China, which is a city on the east coast of China, two and a half hours away from Shanghai. I am also a transfer student from UNC Chapel Hill, where I studied Psychology and Economics.

     

    I’m interested in several psychology topics, including sleep patterns, emotions, relationships, etc, but memory is definitely my favorite. I’m currently involved in a study that looks at whether people’s brain activity differ when they recall temporally specific events that can be targeted to a precise moment in the past, versus when they summarize information that happened over a longer period of time.

    Undergraduate Research Assistant

    I’m a junior Neuroscience major with a minor in Psychology.

     

    My research interest involved the study of brain activity with regards to memory and recall. More specifically, we’re currently working on a project that maps the memory of participants recalling a set of ten short stories. We’re looking to analyze the times of memory search, transition periods, and other elements of their responses before matching it up to their fMRI.

     

    A fun fact about me? I love grocery shopping and watching people cook.

    Yoonjin Nah

    Alumni

    Yoonjin is a former graduate student in the Psychological & Brain Sciences Department interested in the behavioral and neural mechanisms underlying encoding and retrieval processes of real-life experiences. His research focused on understanding how the human mental processes deal with real-life experiences by adopting naturalistic fMRI paradigms, how these systems operate to guide us to both successfully encode and retrieve a multitude of information which contains various contextual factors, and how individual differences have a direct/indirect impact on these processes. Previously, he had been involved in fMRI studies exploring neural mechanisms of human memory and decision-making processes in both young healthy group and various clinical groups under the supervision of Dr. Sanghoon Han at Yonsei University, Korea. Yoonjin is currently pursuing his dream of becoming a pilot, and will soon become the member with the highest-flying career in our lab :)

    Alumni

    David is a Junior undergraduate at Johns Hopkins University, majoring in Public Health Studies and minoring in Film and Media Studies. He previously worked in our lab as an undergraduate Research Assistant.