It can be tempting to consider language and thought as inextricably linked. As such we might conclude that LLM's human-like capabilities for manipulating language indicate a corresponding level of thinking.
However, neuroscience research suggests that thought and language can be teased apart, perhaps the latter is more akin to an input-output interface, or an area of triage for problem-solving. Language is a medium into which we can translate and transport concepts.
Our guest this week is Anna Ivanova, Assistant Professor of Psychology at Georgia Institute of Technology. She's conducted experiments that demonstrate how subjects with severe aphasia (large-scale damage to the language area of their brains) remain able to reason socially. She's also studied how the brains of developers work when reading code. Again the language network is largely bypassed.
Anna's work and other research in cognitive science suggest that the modularity of brains is central to their ability to handle diverse tasks.
Brains are not monolithic neural nets like LLMs but contain networked specialized regions.