Integrating structure and statistics in language processing: an ecological neural dynamics and manifolds approach

Language is a vital aspect of human life. Whether it is processed predominantly by its structure (grammar) or statistics (word probabilities) is a contention long debated in Psychology and Linguistics, reignited by the rise of statistics-based models like ChatGPT. Crucially, the brain preserves linguistic structure (speech sounds, grammar, pragmatics) while adapting to the statistics of a changing environment (new words, slang, diverse accents). Unraveling our ability to encode both structure and statistics is key to neurocomputational theories of language. Contemporary neuroscientific models excel at harnessing statistical properties of text, but overlook both linguistic structure and face-to-face language use— the source of the ‘true’ statistics the brain relies on. DYNALANG pioneers a new path for the neuroscience of language by using neural dynamics to integrate structure with ecologically-valid statistics. In contrast to existing approaches, it hypothesizes that structure and statistics support each other. It uses MEG and dual-EEG across four typologically-distinct languages and behaviors(listening, speaking, multimodal conversation) to discern the boundary conditions for integrating structure and statistics, forming the basis of innovative models reflecting the brain's statistical experience. DYNALANG harnesses neural dynamics, the intricate patterns in interconnected neuronal activity, as the brain's intrinsic computational method for information integration. DYNALANG develops manifold language models—latent spaces where neural and behavioral dynamics combine with linguistic annotations, whose dimensions and geometry isolate neural computations. DYNALANG will give unique insight into how speech and gesture form language in the brain, envisioning linguistic representation as dynamic rather than static. This will fundamentally change computational and neurobiological theories of language, impacting future brain-computer interfaces, clinical prostheses, and AI.