Can LLMs really be intelligent?

I am often involved in conversations where we end up discussing if LLMs can really be intelligent. After all, they are fundamentally just learning statistical regularities in the data to predict the next word. Predicting the next word does not seem like a particularly intelligent activity, so if that is all they do, how could they possibly be intelligent?

The problem with this argument is that intelligence and predicting the next word are by no means mutually exclusive. This can be shown by a simple thought experiment. Imagine LLMs living inside a box—let’s make it black to fit our metaphor. We can send a question as text into this box. All we get back is one new word at a time, which we add to the end of our original text. As we repeat this process, our text grows longer and eventually forms what looks like a question followed by an answer.

What we’re asking here is whether the contents of the box could possibly be intelligent. Given that this word-by-word process is the only way we can interact with whatever’s inside the box, could intelligence exist there? We’re not claiming that what’s in the box is definitely intelligent—only that intelligence remains possible even with this limited interaction method.

There is a fairly simple way to answer this question. Let’s remove the LLM from the box and replace it with a human. We repeat the process as before, passing in strings of text to the box and getting words out that we can append to that string. However, this time it is a person inside reading our input string and coming up with a new word to pass out. Now, unless you disagree that humans can be intelligent, then we cannot say that this process itself excludes the possibility of intelligence.

So we have shown that the process of predicting the next word does not exclude the possibility of intelligence. The natural next question to ask is whether something that has only ever interacted with the world through predicting the next word could be intelligent. To this, I do not have as convincing an answer. Instead, I shall respond with a question of my own. Could something that has only interacted with the world by waving its arms and legs and making silly noises really be intelligent?

This argument only highlights the absurdity of the claim that LLMs cannot be intelligent. We have given support here for the view that there exists a set of weights where it is undeniable that the model is truly intelligent. However, the question remains, does this apply to our current models? Some theorists say no. They argue that LLMs only simulate intelligence. A pragmatist at heart I find this fairly difficult to engage with. I believe that a perfect simulation is functionally identical to the thing it simulates—in every practical sense, they are the same. If we cannot distinguish between simulated intelligence and “true” intelligence through any test or measurement, then the distinction becomes philosophical rather than practical, and perhaps not something we should worry about.

In closing, I think we need to be more open-minded about what constitutes intelligence. Our intuitions about intelligence are shaped by our experience as humans, but intelligence might manifest in ways we don’t immediately recognise. The mechanism of interaction—whether it’s predicting words or something else—doesn’t determine whether something is intelligent. What matters is the underlying capability. As LLMs continue to develop, perhaps we should focus less on arbitrary distinctions about what “counts” as intelligence and more on understanding these systems on their own terms. After all, if something can engage thoughtfully with complex ideas, solve difficult problems, and adapt to new information—whether it’s made of neurons or weights in a neural network—isn’t that what we mean by intelligence in the first place?