Stephen Downes

I think one of the biggest differences between LLMs and humans is memory. As this article notes, "large language models (LLMs) are stateless: they don't remember anything between conversations." So they need "a system outside the model that writes things down and hands them back later." As a result, "AI researchers borrowed four categories from cognitive science to describe what agents store (the CoALA framework is the usual reference)." But (in my opinion, at least) that's not how human memory works. In human memory, the models and weights are never frozen. The model is the memory. It takes so much energy and computing power today to train models than ongoing training isn't practical, but in a human, that function never turns off. There's no separate read-write system in the brain, only the one out there in the world, consisting of PostIt notes and textbooks and alarm clocks.