Memory and a New World

I have been using LLMs for quite a while now, especially ChatGPT, since December 2022. And something about using it caught my attention almost immediately: what looked like memory was actually the previous conversation being made available to it again.

Every time I sent a message, the model worked from the context it received. What I said before, what it replied, and what I was saying now. It didn’t independently remember receiving my previous message. There was machinery around it keeping that conversation available.

In those early days, I would sometimes be talking to it and suddenly the continuity would break. We were just discussing something, right? But now it behaved as though it had no idea what I meant.

Context windows were limited. Today, they are much larger, and we have better ways of carrying information forward, including compaction: reducing earlier conversations so there is room to continue. But the basic distinction remains. There is the model generating a response, and there is the machinery making the previous conversation available to it.

Now suppose we remove that machinery. No previous messages, no summary, nothing carried forward. Whatever I send is all it has of our exchange.

And that made me wonder: suppose humans didn’t have memory either?

Whatever we heard would be happening now. And then now again. Except we wouldn’t even know it was “again,” would we? That requires something of the previous moment to remain.

Just to be clear, I am not saying time doesn’t exist in reality. Or that your body would stop getting older because you couldn’t remember your age. I am talking about our experience of it.

Something has to carry the previous moment into this one. Call it memory, or some mechanism in the brain. Without any of that, what would “before” mean to me?

The brain doesn’t work exactly like an LLM, of course. But that was the connection that interested me. Memory lets me experience a past, and I bring that past into whatever I am experiencing now.

Interesting, isn’t it?

What Does It Mean to Be Young?

And this becomes more personal as I get older. Because obviously I am getting older, right? I find myself asking what the purpose of life is, and so on. But why didn’t I ask these questions as much when I was a teenager or in my early twenties?

I think I have an answer for myself.

There was so much novelty. I was experiencing things for the first time, and I was so engaged in those experiences that I wasn’t constantly asking what it all meant.

Then, gradually, the patterns became familiar. I had experienced much of what my surroundings offered, and I kept experiencing versions of it again and again.

Maybe life wasn’t boring. But it wasn’t as exciting.

Give me something new to figure out, something that actually interests me, and I become busy with it. The question of purpose becomes quieter. Perhaps part of why I keep asking it is that I am no longer sufficiently occupied by the novelty of living.

Which brings me to another question: what does it mean to be young?

There is the body, obviously. How it moves, how it recovers, what it lets you do. But there is also how old you feel.

Personally, I think I am still twenty-two. It’s just that I have so much information now that I cannot behave like a twenty-two-year-old.

But I want to.

I want that openness, that excitement. I want my body to support it too. In my mind, I don’t want to get old. Why would I want to lose the part that makes life exciting?

But I have memory. I cannot simply switch it off. I know how certain things tend to go. I anticipate the outcome. That makes me more practical, and we call it becoming mature.

Sometimes I think that’s bullshit. Sometimes it feels like I have learned to dismiss an experience before I have even had it.

For me, being young has a lot to do with meeting things for the first time.

And surely that is part of why we travel, right? We want to encounter something outside the patterns we already know. Another place, another way of living, another version of ourselves in unfamiliar surroundings.

Even a journey made for religious reasons can contain that discovery. You leave what is familiar, experience something different, and perhaps return understanding yourself differently.

But how much novelty can we keep finding?

Another mountain might be taller, but there are still rocks, trees, and a path to climb. A tree on another continent may be a different species, but it is still recognizably a tree. The world has differences, of course. Yet gradually we begin to notice the similarities.

Maybe the limitation is in what the world offers. Maybe it is in how we experience it.

Either way, if I want that first-time excitement again, I seem to have two options. Keep finding new experiences, or figure out a cheaper way.

Well, what is the cheaper way?

Forget an experience, and perhaps I can have it again as though it were new.

And then I think, oh, come on. Maybe nature is already doing something like this.

We are born, we live, we die. The material that makes us returns to the world and becomes part of other things. New life comes into being, and somewhere there is another first rainbow, another first lake, another first moment of amazement.

The world is familiar to one life and completely new to another.

Maybe nature experiences novelty through us. Through all these organisms. Maybe that is one way to think about death: everything gets dismantled, and the same world becomes new again to someone else.

I’m not saying nature has a plan. But sometimes I wonder whether it looks like that, no?

What If We Could Forget?

Which brings me back to something more practical.

Could we ever do some version of this within our own lives?

With AI, I have watched machines become increasingly capable of recognizing patterns and doing useful things with them. I don’t know whether that explains all of intelligence. Probably there are things we still don’t understand. But it makes me wonder how much more we might eventually understand about the brain.

I find myself thinking that some problems which look impossible today might turn out to be engineering problems. Reading what is happening in the brain, understanding it well enough to work with it. I don’t know how far away that is. Reading a signal is obviously not the same as understanding a memory.

But hear me out. Suppose, one day, I could safely remove a particular memory.

I remember learning to ride a bicycle. Suppose I could forget that experience and encounter it again. Could I feel that thrill again?

Forgetting the event wouldn’t necessarily erase the skill, so perhaps a bicycle is not the simplest example. But you understand what I am asking. Could something familiar become an actual discovery again?

And then there are memories I would want to lose for an entirely different reason.

Something happened years ago, but it still troubles me now. It gets in the way of how I think, how I behave, what I allow myself to do.

Maybe that memory served a purpose. Maybe it helped make me the person I am. But does that automatically mean it is still helping me?

I doubt it.

Sometimes it seems to do the opposite. It keeps me stuck. Why should I have to preserve every experience simply because it happened?

Of course there would be a cost. A memory is context. Remove it, and perhaps I lose some understanding along with the pain.

But maybe that particular context is a cost I can afford.

Stories like Silo, Homecoming, and Black Mirror come to mind when I think about this. The possibility is fascinating, and so is the question of who gets to decide what someone remembers.

I would want that choice to be mine.

If it were safe, I would want to experience some things again. And there are other things I would be happy not to keep reliving.

Maybe these desires aren’t so different. In one case, I want the past to stop making everything familiar. In the other, I want it to stop troubling me.

Both make me wonder how much of the world I experience is the world itself, and how much is what I keep bringing to it.

Maybe I don’t always need a new world.

Maybe I want to meet this one again.