Menu

LFP publications

Local field potential recordings made with TDT systems.

Nature Communications · Feb 2025

Magnetically reshapable 3D multi-electrode arrays of liquid metals for electrophysiological analysis of brain organoids (opens in a new tab)

Enji Kim, Eunseon Jeong, Yeon-Mi Hong, … Jang-Ung Park

Abstract

To comprehend the volumetric neural connectivity of a brain organoid, it is crucial to monitor the spatiotemporal electrophysiological signals within the organoid, known as intra-organoid signals. However, previous methods risked damaging the three-dimensional (3D) cytoarchitecture of organoids, either through sectioning or inserting rigid needle-like electrodes. Also, the limited numbers of electrodes in fixed positions with non-adjustable electrode shapes were insufficient for examining the complex neural activity throughout the organoid. Herein, we present a magnetically reshapable 3D multi-electrode array (MEA) using direct printing of liquid metals for electrophysiological analysis of brain organoids. The adaptable distribution and the softness of these printed electrodes facilitate the spatiotemporal recording of intra-organoid signals. Furthermore, the unique capability to reshape these soft electrodes within the organoid using magnetic fields allows a single electrode in the MEA to record from multiple points, effectively increasing the recording site density without the need for additional electrodes.

eneuro · Jan 2025

Cortical HFS-Induced Neo-Hebbian Local Plasticity Enhances Efferent Output Signal and Strengthens Afferent Input Connectivity (opens in a new tab)

Xiao Li, Xue Wang, Xiaohan Hu, … Jufang He

Abstract

High-frequency stimulation (HFS)-induced long–term potentiation (LTP) is generally regarded as a homosynaptic Hebbian-type LTP, where synaptic changes are thought to occur at the synapses that project from the stimulation site and terminate onto the neurons at the recording site. In this study, we first investigated HFS-induced LTP on urethane-anesthetized rats and found that cortical HFS enhances neural responses at the recording site through the strengthening of local connectivity with nearby neurons at the stimulation site rather than through synaptic strengthening at the recording site. This enhanced local connectivity at the stimulation site leads to increased output propagation, resulting in signal potentiation at the recording site. Additionally, we discovered that HFS can also nonspecifically strengthen distant afferent synapses at the HFS site, thereby expanding its impact beyond local neural connections. This form of plasticity exhibits a neo-Hebbian characteristic as it exclusively manifests in the presence of cholecystokinin release, induced by HFS. The cortical HFS-induced local LTP was further supported by a behavioral task, providing additional evidence. Our results unveil a previously overlooked mechanism underlying cortical plasticity: synaptic plasticity is more likely to occur around the soma site of strongly activated cortical neurons rather than solely at their projection terminals.

Scientific Reports · Dec 2024

An ultra low frequency spike timing dependent plasticity based approach for reducing alcohol drinking (opens in a new tab)

Anders J. Asp, Suelen Lucio Boschen, Su-Youne Chang, … J. Luis Lujan

Abstract

Alcohol use disorder (AUD) is a chronic relapsing brain disorder characterized by an impaired ability to stop or control alcohol consumption despite adverse social, occupational, or health consequences. AUD affects nearly one-third of adults at some point during their lives, with an associated cost of approximately $249 billion annually in the U.S. alone. The effects of alcohol consumption are expected to increase significantly during the COVID-19 pandemic, with alcohol sales increasing by approximately 54%, potentially exacerbating health concerns and risk-taking behaviors. Unfortunately, existing pharmacological and behavioral therapies for AUD are associated with poor success rates, with approximately 40% of individuals relapsing within three years of treatment.Pre-clinical studies have shown that chronic alcohol consumption leads to significant changes in synaptic function within the dorsal medial striatum (DMS), one of the brain regions associated with AUD and responsible for mediating goal-directed behavior. Specifically, chronic alcohol consumption has been associated with hyperactivity of dopamine receptor 1 (D1) medium spiny neurons (MSN) and hypoactivity of dopamine receptor 2 (D1) MSNs within the DMS. Optogenetic, chemogenetic, and transgenic approaches have demonstrated that reducing the D1/D2 MSN signaling imbalance decreases alcohol self-administration in rodent models of AUD.Here, we present an electrical stimulation approach that uses ultra-low (≤ 1 Hz) frequency (ULF) spike-timing-dependent plasticity (STDP) in mouse models of AUD to reduce DMS D1/D2 MSN signaling imbalances by stimulating D1-MSN afferents into the GPi and ACC glutamatergic projections to the DMS in a time-locked stimulation sequence. Our data suggest that GPi/ACC ULF-STDP selectively decreases DMS D1-MSN hyperactivity leading to reduced alcohol consumption without evoking undesired affective behaviors using electrical stimulation rather than approaches requiring genetic modification. This work represents a step towards fulfilling the unmet need for a reliable method of treating severe AUD through cell-type-specific control with clinically available neuromodulation tools.

Science Advances · Nov 2024

Multifunctional hydrogel electronics for closed-loop antiepileptic treatment (opens in a new tab)

Jin Qu, Kai Xie, Shu Chen, … Peng Shi

Abstract

Closed-loop strategies offer advanced therapeutic potential through intelligent disease management. Here, we develop a hydrogel-based, single-component, organic electronic device for closed-loop neurotherapy. Fabricated out of conductive hydrogels, the device consists of a flexible array of microneedle electrodes, each of which can be individually addressed to perform electrical recording and control chemical release with sophisticated spatiotemporal control, thus pioneering a smart antiseizure therapeutic system by combining electrical and pharmacological interventions. The recorded neural signal acts as the trigger for a voltage-driven drug release in detected pathological conditions predicted by real-time electrophysiology analysis. When implanted into epileptic animals, the device enables autonomous antiseizure management, where the dosing of antiepileptic drug is controlled in a time-sensitive, region-selective, and dose-adaptive manner, allowing the inhibition of seizure outbursts through the delivery of just-necessary drug dosages. The side effects are minimized with dosages three orders of magnitude lower than the usage in approaches simulating existing clinical treatments.

Advanced Science · Sep 2024

Sequential Deactivation Across the Hippocampus-Thalamus-mPFC Pathway During Loss of Consciousness (opens in a new tab)

Xiaoai Chen, Samuel R. Cramer, Dennis C.Y. Chan, … Nanyin Zhang

Abstract

How consciousness is lost in states such as sleep or anesthesia remains a mystery. To gain insight into this phenomenon, concurrent recordings of electrophysiology signals in the anterior cingulate cortex and whole‐brain functional magnetic resonance imaging (fMRI) are conducted in rats exposed to graded propofol, undergoing the transition from consciousness to unconsciousness. The results reveal that upon the loss of consciousness (LOC), there is a sharp increase in low‐frequency power of the electrophysiological signal. Additionally, fMRI signals exhibit a cascade of deactivation across a pathway including the hippocampus, thalamus, and medial prefrontal cortex (mPFC) surrounding the moment of LOC, followed by a broader increase in brain activity across the cortex during sustained unconsciousness. Furthermore, sliding window analysis demonstrates a temporary increase in synchrony of fMRI signals across the hippocampus‐thalamus‐mPFC pathway preceding LOC. These data suggest that LOC may be triggered by sequential activities in the hippocampus, thalamus, and mPFC, while wide‐spread activity increases in other cortical regions commonly observed during anesthesia‐induced unconsciousness may be a consequence, rather than a cause of LOC. Taken together, the study identifies a cascade of neural events unfolding as the brain transitions into unconsciousness, offering insight into the systems‐level neural mechanisms underpinning LOC.

Science Advances · Sep 2024

Oscillatory-Quality of sleep spindles links brain state with sleep regulation and function (opens in a new tab)

Cristina Blanco-Duque, Suraya A. Bond, Lukas B. Krone, … Vladyslav V. Vyazovskiy

Abstract

Here, we characterized the dynamics of sleep spindles, focusing on their damping, which we estimated using a metric called oscillatory-Quality (o-Quality), derived by fitting an autoregressive model to electrophysiological signals, recorded from the cortex in mice. The o-Quality of sleep spindles correlates weakly with their amplitude, shows marked laminar differences and regional topography across cortical regions, reflects the level of synchrony within and between cortical networks, is strongly modulated by sleep-wake history, reflects the degree of sensory disconnection, and correlates with the strength of coupling between spindles and slow waves. As most spindle events are highly localized and not detectable with conventional low-density recording approaches, o-Quality thus emerges as a valuable metric that allows us to infer the spread and dynamics of spindle activity across the brain and directly links their spatiotemporal dynamics with local and global regulation of brain states, sleep regulation, and function.

Cognitive, Affective, & Behavioral Neuroscience · Aug 2024

Cortico-striatal beta oscillations as a reward-related signal (opens in a new tab)

M. F. Koloski, S. Hulyalkar, S. A. Barnes, … D. S. Ramanathan

Abstract

The value associated with reward is sensitive to external factors, such as the time between the choice and reward delivery as classically manipulated in temporal discounting tasks. Subjective preference for two reward options is dependent on objective variables of reward magnitude and reward delay. Single neuron correlates of reward value have been observed in regions, including ventral striatum, orbital, and medial prefrontal cortex. Brain imaging studies show cortico-striatal-limbic network activity related to subjective preferences. To explore how oscillatory dynamics represent reward processing across brain regions, we measured local field potentials of rats performing a temporal discounting task. Our goal was to use a data-driven approach to identify an electrophysiological marker that correlates with reward preference. We found that reward-locked oscillations at beta frequencies signaled the magnitude of reward and decayed with longer temporal delays. Electrodes in orbitofrontal/medial prefrontal cortex, anterior insula, ventral striatum, and amygdala individually increased power and were functionally connected at beta frequencies during reward outcome. Beta power during reward outcome correlated with subjective value as defined by a computational model fit to the discounting behavior. These data suggest that cortico-striatal beta oscillations are a reward signal correlated, which may represent subjective value and hold potential to serve as a biomarker and potential therapeutic target.

Nature Communications · Aug 2024

Multisensory gamma stimulation mitigates the effects of demyelination induced by cuprizone in male mice (opens in a new tab)

Daniela Rodrigues-Amorim, P. Lorenzo Bozzelli, TaeHyun Kim, … Li-Huei Tsai

Abstract

Demyelination is a common pathological feature in a wide range of diseases, characterized by the loss of myelin sheath and myelin-supporting oligodendrocytes. These losses lead to impaired axonal function, increased vulnerability of axons to damage, and result in significant brain atrophy and neuro-axonal degeneration. Multiple pathomolecular processes contribute to neuroinflammation, oligodendrocyte cell death, and progressive neuronal dysfunction. In this study, we use the cuprizone mouse model of demyelination to investigate long-term non-invasive gamma entrainment using sensory stimulation as a potential therapeutic intervention for promoting myelination and reducing neuroinflammation in male mice. Here, we show that multisensory gamma stimulation mitigates demyelination, promotes oligodendrogenesis, preserves functional integrity and synaptic plasticity, attenuates oligodendrocyte ferroptosis-induced cell death, and reduces brain inflammation. Thus, the protective effects of multisensory gamma stimulation on myelin and anti-neuroinflammatory properties support its potential as a therapeutic approach for demyelinating disorders.

eLife · Aug 2024

Disparity in temporal and spatial relationships between resting-state electrophysiological and fMRI signals (opens in a new tab)

Wenyu Tu, Samuel R Cramer, Nanyin Zhang

Abstract

Resting-state brain networks (RSNs) have been widely applied in health and disease, but the interpretation of RSNs in terms of the underlying neural activity is unclear. To address this fundamental question, we conducted simultaneous recordings of whole-brain resting-state functional magnetic resonance imaging (rsfMRI) and electrophysiology signals in two separate brain regions of rats. Our data reveal that for both recording sites, spatial maps derived from band-specific local field potential (LFP) power can account for up to 90% of the spatial variability in RSNs derived from rsfMRI signals. Surprisingly, the time series of LFP band power can only explain to a maximum of 35% of the temporal variance of the local rsfMRI time course from the same site. In addition, regressing out time series of LFP power from rsfMRI signals has minimal impact on the spatial patterns of rsfMRI-based RSNs. This disparity in the spatial and temporal relationships between resting-state electrophysiology and rsfMRI signals suggests that electrophysiological activity alone does not fully explain the effects observed in the rsfMRI signal, implying the existence of an rsfMRI component contributed by ‘electrophysiology-invisible’ signals. These findings offer a novel perspective on our understanding of RSN interpretation.

PLOS Biology · Jul 2024

Aberrant activation of hippocampal astrocytes causes neuroinflammation and cognitive decline in mice (opens in a new tab)

Jae-Hong Kim, Nakamura Michiko, In-Sun Choi, … Kyoungho Suk

Abstract

Reactive astrocytes are associated with neuroinflammation and cognitive decline in diverse neuropathologies; however, the underlying mechanisms are unclear. We used optogenetic and chemogenetic tools to identify the crucial roles of the hippocampal CA1 astrocytes in cognitive decline. Our results showed that repeated optogenetic stimulation of the hippocampal CA1 astrocytes induced cognitive impairment in mice and decreased synaptic long-term potentiation (LTP), which was accompanied by the appearance of inflammatory astrocytes. Mechanistic studies conducted using knockout animal models and hippocampal neuronal cultures showed that lipocalin-2 (LCN2), derived from reactive astrocytes, mediated neuroinflammation and induced cognitive impairment by decreasing the LTP through the reduction of neuronal NMDA receptors. Sustained chemogenetic stimulation of hippocampal astrocytes provided similar results. Conversely, these phenomena were attenuated by a metabolic inhibitor of astrocytes. Fiber photometry using GCaMP revealed a high level of hippocampal astrocyte activation in the neuroinflammation model. Our findings suggest that reactive astrocytes in the hippocampus are sufficient and required to induce cognitive decline through LCN2 release and synaptic modulation. This abnormal glial–neuron interaction may contribute to the pathogenesis of cognitive disturbances in neuroinflammation-associated brain conditions.

iScience · Jun 2024

Behavioral and cortical arousal from sleep, muscimol-induced coma, and anesthesia by direct optogenetic stimulation of cortical neurons (opens in a new tab)

Rong Mao, Matias Lorenzo Cavelli, Graham Findlay, … Chiara Cirelli

Abstract

The cerebral cortex is widely considered part of the neural substrate of consciousness, but direct causal evidence is missing. Here, we tested in mice whether optogenetic activation of cortical neurons in posterior parietal cortex (PtA) or medial prefrontal cortex (mPFC) is sufficient for arousal from three behavioral states characterized by progressively deeper unresponsiveness: sleep, a coma-like state induced by muscimol injection in the midbrain, and deep sevoflurane-dexmedetomidine anesthesia. We find that cortical stimulation always awakens the mice from both NREM sleep and REM sleep, with PtA requiring weaker/shorter light pulses than mPFC. Moreover, in most cases light pulses produce both cortical activation (decrease in low frequencies) and behavioral arousal (recovery of the righting reflex) from brainstem coma, as well as cortical activation from anesthesia. These findings provide evidence that direct activation of cortical neurons is sufficient for behavioral and/or cortical arousal from sleep, brainstem coma, and anesthesia.

Science Advances · Apr 2024

Cortico-cerebellar coordination facilitates neuroprosthetic control (opens in a new tab)

Aamir Abbasi, Rohit Rangwani, Daniel W. Bowen, … Tanuj Gulati

Abstract

Temporally coordinated neural activity is central to nervous system function and purposeful behavior. Still, there is a paucity of evidence demonstrating how this coordinated activity within cortical and subcortical regions governs behavior. We investigated this between the primary motor (M1) and contralateral cerebellar cortex as rats learned a neuroprosthetic/brain-machine interface (BMI) task. In neuroprosthetic task, actuator movements are causally linked to M1 “direct” neurons that drive the decoder for successful task execution. However, it is unknown how task-related M1 activity interacts with the cerebellum. We observed a notable 3 to 6 hertz coherence that emerged between these regions’ local field potentials (LFPs) with learning that also modulated task-related spiking. We identified robust task-related indirect modulation in the cerebellum, which developed a preferential relationship with M1 task–related activity. Inhibiting cerebellar cortical and deep nuclei activity through optogenetics led to performance impairments in M1-driven neuroprosthetic control. Together, these results demonstrate that cerebellar influence is necessary for M1-driven neuroprosthetic control.

bioRxiv · Apr 2024

Hippocampal sharp wave ripples and coincident cortical ripples orchestrate human semantic networks (opens in a new tab)

Akash Mishra, Serdar Akkol, Elizabeth Espinal, … Stephan Bickel

Abstract

Episodic memory function is predicated upon the precise coordination between the hippocampus and widespread cortical regions. However, our understanding of the neural mechanisms involved in this process is incomplete. In this study, human subjects undergoing intracranial electroencephalography (iEEG) monitoring performed a list learning task. We show sharp-wave ripple (SWR)-locked reactivation of specific semantic processing regions during free recall. This cortical activation consists of both broadband high frequency (non-oscillatory) and cortical ripple (oscillatory) activity. SWRs and cortical ripples in the anterior temporal lobe, a major semantic hub, co-occur and increase in rate prior to recall. Coincident hippocampal-ATL ripples are associated with a greater increase in cortical reactivation, show specificity in location based on recall content, and are preceded by cortical theta oscillations. These findings may represent a reactivation of hippocampus and cortical semantic regions orchestrated by an interplay between hippocampal SWRs, cortical ripples, and theta oscillations.

iScience · Apr 2024

Restoring vision in adult amblyopia by enhancing plasticity through deletion of the transcriptional repressor REST (opens in a new tab)

Dmytro Shmal, Giulia Mantero, Thomas Floss, … José Fernando Maya-Vetencourt

Abstract

Visual cortical plasticity is high during early life, but gradually decreases with development. This is due to the Otx2-driven maturation of intracortical inhibition that parallels the condensation of extracellular matrix components into perineuronal nets mainly around parvalbumin-positive GABAergic neurons. Repressor Element 1 Silencing Transcription (REST) epigenetically controls the expression of a plethora of neuron-specific genes. We demonstrate that the conditional knockout of REST in the primary visual cortex of adult mice induces a shift of ocular dominance after short-term monocular deprivation and promotes the recovery of vision in long-term deprived animals after reverse suture. These phenomena paralleled a reduction of perineuronal net density and increased expression of REST target genes, but not of the homeoprotein Otx2 in the visual cortex contralateral to the deprived eye. This shows that REST regulates adult visual cortical plasticity and is a potential therapeutic target to restore vision in adult amblyopia by enhancing V1 plasticity.

eLife · Mar 2024

Heterosynaptic plasticity of the visuo-auditory projection requires cholecystokinin released from entorhinal cortex afferents (opens in a new tab)

Wenjian Sun, Haohao Wu, Yujie Peng, … Jufang He

Abstract

The entorhinal cortex is involved in establishing enduring visuo-auditory associative memory in the neocortex. Here we explored the mechanisms underlying this synaptic plasticity related to projections from the visual and entorhinal cortices to the auditory cortex in mice using optogenetics of dual pathways. High-frequency laser stimulation (HFS laser) of the visuo-auditory projection did not induce long-term potentiation. However, after pairing with sound stimulus, the visuo-auditory inputs were potentiated following either infusion of cholecystokinin (CCK) or HFS laser of the entorhino-auditory CCK-expressing projection. Combining retrograde tracing and RNAscope in situ hybridization, we show that Cck expression is higher in entorhinal cortex neurons projecting to the auditory cortex than in those originating from the visual cortex. In the presence of CCK, potentiation in the neocortex occurred when the presynaptic input arrived 200 ms before postsynaptic firing, even after just five trials of pairing. Behaviorally, inactivation of the CCK + projection from the entorhinal cortex to the auditory cortex blocked the formation of visuo-auditory associative memory. Our results indicate that neocortical visuo-auditory association is formed through heterosynaptic plasticity, which depends on release of CCK in the neocortex mostly from entorhinal afferents.

Nature · Feb 2024

Multisensory gamma stimulation promotes glymphatic clearance of amyloid (opens in a new tab)

Mitchell H. Murdock, Cheng-Yi Yang, Na Sun, … Li-Huei Tsai

Abstract

The glymphatic movement of fluid through the brain removes metabolic waste 1–4 . Noninvasive 40 Hz stimulation promotes 40 Hz neural activity in multiple brain regions and attenuates pathology in mouse models of Alzheimer’s disease 5–8 . Here we show that multisensory gamma stimulation promotes the influx of cerebrospinal fluid and the efflux of interstitial fluid in the cortex of the 5XFAD mouse model of Alzheimer’s disease. Influx of cerebrospinal fluid was associated with increased aquaporin-4 polarization along astrocytic endfeet and dilated meningeal lymphatic vessels. Inhibiting glymphatic clearance abolished the removal of amyloid by multisensory 40 Hz stimulation. Using chemogenetic manipulation and a genetically encoded sensor for neuropeptide signalling, we found that vasoactive intestinal peptide interneurons facilitate glymphatic clearance by regulating arterial pulsatility. Our findings establish novel mechanisms that recruit the glymphatic system to remove brain amyloid.

Nature Communications · Feb 2024

In-vivo integration of soft neural probes through high-resolution printing of liquid electronics on the cranium (opens in a new tab)

Young-Geun Park, Yong Won Kwon, Chin Su Koh, … Jang-Ung Park

Abstract

Current soft neural probes are still operated by bulky, rigid electronics mounted to a body, which deteriorate the integrity of the device to biological systems and restrict the free behavior of a subject. We report a soft, conformable neural interface system that can monitor the single-unit activities of neurons with long-term stability. The system implements soft neural probes in the brain, and their subsidiary electronics which are directly printed on the cranial surface. The high-resolution printing of liquid metals forms soft neural probes with a cellular-scale diameter and adaptable lengths. Also, the printing of liquid metal-based circuits and interconnections along the curvature of the cranium enables the conformal integration of electronics to the body, and the cranial circuit delivers neural signals to a smartphone wirelessly. In the in-vivo studies using mice, the system demonstrates long-term recording (33 weeks) of neural activities in arbitrary brain regions. In T-maze behavioral tests, the system shows the behavior-induced activation of neurons in multiple brain regions.

PLOS Computational Biology · Jan 2024

Somnotate: A probabilistic sleep stage classifier for studying vigilance state transitions (opens in a new tab)

Paul J. N. Brodersen, Hannah Alfonsa, Lukas B. Krone, … Colin J. Akerman

Abstract

Electrophysiological recordings from freely behaving animals are a widespread and powerful mode of investigation in sleep research. These recordings generate large amounts of data that require sleep stage annotation (polysomnography), in which the data is parcellated according to three vigilance states: awake, rapid eye movement (REM) sleep, and non-REM (NREM) sleep. Manual and current computational annotation methods ignore intermediate states because the classification features become ambiguous, even though intermediate states contain important information regarding vigilance state dynamics. To address this problem, we have developed "Somnotate"—a probabilistic classifier based on a combination of linear discriminant analysis (LDA) with a hidden Markov model (HMM). First we demonstrate that Somnotate sets new standards in polysomnography, exhibiting annotation accuracies that exceed human experts on mouse electrophysiological data, remarkable robustness to errors in the training data, compatibility with different recording configurations, and an ability to maintain high accuracy during experimental interventions. However, the key feature of Somnotate is that it quantifies and reports the certainty of its annotations. We leverage this feature to reveal that many intermediate vigilance states cluster around state transitions, whereas others correspond to failed attempts to transition. This enables us to show for the first time that the success rates of different types of transition are differentially affected by experimental manipulations and can explain previously observed sleep patterns. Somnotate is open-source and has the potential to both facilitate the study of sleep stage transitions and offer new insights into the mechanisms underlying sleep-wake dynamics.

Network Neuroscience · 2024

Gamma frequency connectivity in frontostriatal networks associated with social preference is reduced with traumatic brain injury (opens in a new tab)

Morteza Salimi, Tianzhi Tang, Milad Nazari, … Dhakshin S. Ramanathan

Abstract

Among the myriad of complications associated with traumatic brain injury (TBI), impairments in social behaviors and cognition have emerged as a significant area of concern. Animal models of social behavior are necessary to explore the underlying brain mechanisms contributing to chronic social impairments following brain injury. Here, we utilize large-scale brain recordings of local field potentials to identify neural signatures linked with social preference deficits following frontal brain injury. We used a controlled cortical impact model of TBI to create a severe bilateral injury centered on the prefrontal cortex. Behavior (social preference and locomotion) and brain activity (power and coherence) during a three-chamber social preference task were compared between sham and injured animals. Sham rats preferred to spend time with a social conspecific over an inanimate object. An analysis of local field oscillations showed that social preference was associated with a significant increase in coherence in gamma frequency band across widespread brain regions in these animals. Animals with a frontal TBI showed a significant reduction in this social preference, visiting an inanimate object more frequently and for more time. Reflecting these changes in social behavior, these animals also showed a significant reduction in gamma frequency (25–60 Hz) coherence associated with social preference.

Nature Neuroscience · Oct 2023

Dissecting neural computations in the human auditory pathway using deep neural networks for speech (opens in a new tab)

Yuanning Li, Gopala K. Anumanchipalli, Abdelrahman Mohamed, … Edward F. Chang

Abstract

The human auditory system extracts rich linguistic abstractions from speech signals. Traditional approaches to understanding this complex process have used linear feature-encoding models, with limited success. Artificial neural networks excel in speech recognition tasks and offer promising computational models of speech processing. We used speech representations in state-of-the-art deep neural network (DNN) models to investigate neural coding from the auditory nerve to the speech cortex. Representations in hierarchical layers of the DNN correlated well with the neural activity throughout the ascending auditory system. Unsupervised speech models performed at least as well as other purely supervised or fine-tuned models. Deeper DNN layers were better correlated with the neural activity in the higher-order auditory cortex, with computations aligned with phonemic and syllabic structures in speech. Accordingly, DNN models trained on either English or Mandarin predicted cortical responses in native speakers of each language. These results reveal convergence between DNN model representations and the biological auditory pathway, offering new approaches for modeling neural coding in the auditory cortex.