How do we know the difference between sarcasm and sincerity, or understand that in many social situations people don’t always say what they mean?
The subtleties of speech and hearing have eluded neuroscientists for more than a century, but now two Columbia professors have come up with a novel approach to try to understand the intricacies of verbal communication.
Nima Mesgarani, assistant professor of electrical engineering, and Sameer Sheth, assistant professor of neurological surgery, are tackling the age-old problem by observing the brain’s electrical patterns when an individual listens to complex conversation. They want to observe how multiple regions of the brain interact with and support one another to produce comprehension.
The novelty of their approach lies in the fact that they are placing electrodes deep inside the brain; previous research placed sensors on top of the cortical surface on top of the brain. The technique will allow them to identify low-level sensory processing, such as when the brain distinguishes a murmur from a yelp, as well as high-level cognitive processing, such as our ability to recognize and understand contextual information, long-term danger and ambiguity.
“Our study will enable the creation of more accurate models of the brain mechanisms involved in speech communication and will advance our understanding of how these processes become impaired,” said Mesgarani. “At the same time these models and algorithms could enable machines to more closely perform such human tasks as speech and language comprehension.”
The research could also lead to significant advances in health care, particularly the diagnosis and treatment of patients with speech-related illnesses such as dyslexia, language-learning delay, autism and cerebral palsy, as well as cognitive impairment resulting from brain trauma and injury.
The National Institute of Deafness and Other Communication Disorders (NIDCD) reports that approximately eight million U.S. residents have some form of language impairment, while an additional five percent of American children have speech disorders without a known cause.
The professors received a University grant of $160,000 over two years through a highly competitive program called Research Initiatives for Science & Engineering (RISE), run through the Office of the Executive Vice President for Research. In addition to Mesgarani and Sheth, the University granted seed funding to five other winning teams out of 60 applicants for its 2015 competition.
“RISE encourages PIs [principal investigators] to take risks and to push the boundaries of their conventional disciplines, always with the goal of making fundamental new discoveries,” explains Dr. G. Michael Purdy, executive vice president for research.
The University has long been known for being in the forefront of neuroscience research, on both its Morningside and Medical Center campuses. Columbia was among the first schools to set up a program focused on the neural sciences, dating from its 1921 affiliation with the Neurological Institute of New York. Today it is ranked among the top 10 neuroscience research universities in the world.
Mesgarani and Sheth’s brain research involves working with patients who are undergoing invasive surgery to place electrodes deep in their brains. Such interventions are done solely for medical reasons, such as identifying the source of seizures, and the scientists are able to use the opportunity for other neuro-scientific research. The research protocols are all approved by the Institutional Review Board (IRB) at Columbia University.
By programming computers to evaluate the data gathered by the brain’s scattered electrical impulses, Mesgarani and Sheth aim to shed light on such neural mysteries as how we can overhear someone mutter our names in a crowded room, or attend to a particular person at a cocktail party while suppressing the other voices.
“An accurate model that explains the transformation involved in speech communication will have an overarching impact in many areas,” said Mesgarani. “And a detailed model of the mechanisms involved in speech will advance our understanding of how these processes break down in people who suffer from various speech and communication disorders and may motivate new therapeutic measures.”
Perhaps Mesgarani and Sheth’s novel methods in the exploration of this uncharted territory of human brain electrophysiology may even help us learn someday how to craft that elusive perfect conversation.
— By Marley Bauce