In the late 60s and early 70s I had a small student office in the Linguistic Department at the University of Texas at Austin. That’s where I wrote my dissertation on the development of nasal vowels.
Next to me was the office of Jeffrey Elman, a fellow graduate student. I knew that he was working on something called “neural networks”, known also as “connection machines,” a field that was not taught in the department or anywhere at UT as far as I am aware.
Skeptical of received linguistic theory, I felt that Jeffrey was on a path leading to the understanding of language while I was not. Having fried my brain by majoring in philosophy as an undergraduate, I avoided semantics and even syntax (with all of Chomsky’s transformation rules) and huddled down at the phonological level (I didn’t have enough math to go all the way down to the phonetic level).
Today I came across this amazing video on Youtube:
The video begins by describing the state of neural nets when I was still in graduate school and follows their development up to the present day. Each step is clearly explained and exciting to listen to even though I don’t have the mathematical language needed to read the papers that they’re based on.
Jeffrey’s contribution comes early on. I find his work amazing because his machine learned to identify word boundaries from streams of letters (e.g.,sentenceswithnospacesbetweenwords), something that we linguists could imagine no way to do.
I was even more excited when researchers introduced the idea of attention as a way to reduce the need for temporary memory. They might be using this word metaphorically but I suspect that the concept is surprisingly close to that of human attention. This is exciting because attention seems to be a fundamental component of what we mean by consciousness.
Equally thought-provoking is the speculation that “all perception is language”!
The video finishes up by saying that there are two schools of thought with respect to neural networks. Much as I respect Noam Chomsky as a linguist, I’m afraid I don’t agree with his opinion of neural networks. I’m on the side of those who say, “if it looks like thought, then it is thought”.
So why do I feel illiterate? Because I can’t read articles on neural networks, which are written in mathematical notation. After receiving my PhD I finally studied calculus and loved it as I prepared to study computer science. As a graduate student in UNC-CH’s computer science department, I wasn’t allowed to substitute a useless required math course used to subsidize professors who didn’t understand computers with two math courses that would have helped me understand neural networks.
(I’m also illiterate because I can’t read music.)