On the importance of having a sound epistemological framework and a theory of intelligence before talking about AI

This text is a philosophical reflection, not a research article, and it should be taken for what it is: a preliminary note to any reflection on "artificial intelligence", about the work that has to be done before talking about it. It restates an earlier article I had written in too polemical a tone, whose substance still seems right to me although its form was not. I have kept the distinctions and taken out the punches.

There is a more general reason for writing these pages, and it goes beyond AI. One cannot do science without having done a little history of philosophy, history of science and epistemology. Not a great deal, and not as a specialist; enough to know that the questions one asks have already been asked, that the words one uses have a history that loads them, and that the notions one takes as obvious (cause, law, explanation, observation, model, intelligence) were battlefields before they became tools. Whoever is unaware of this is not doing science without philosophy, as they believe. They are doing metaphysics without knowing it, that is to say badly, borrowing from the spirit of the times theses about what exists and about what counts as evidence, theses they have never examined and could not defend if they were shown them. The engineer who says "it's only computation" is unknowingly repeating the argument Leibniz raised against Hobbes, for whom reasoning was nothing other than calculating, and is unaware that the question has been disputed for three centuries. The researcher who says "the model understands" is unknowingly repeating a thesis about the relation between behaviour and mind that behaviourism defended and that Chomsky demolished in Skinner, and has read neither of them.

The history of science teaches the same lesson by example. The great advances have almost always required calling into question what contemporaries held to be too obvious to be discussed: simultaneity for Einstein, the separability of properties for quantum mechanics, the fixity of species for Darwin. In each case, progress did not consist in accumulating results within a framework, but in seeing that the framework itself contained a hypothesis, and in formulating it. This move is not scientific in the narrow sense; it is epistemological, and those who made it all had a philosophical culture they did not consider incidental. A field that boasts of not doing philosophy has not freed itself from it. It has only stopped seeing its own.

That is why the first thing to do when faced with AI is not to measure it, but to know with what words and what presuppositions one is about to measure it. The rest of this text is that preliminary work, done once so that one can then dispense with it.

The questions "does AI think?" and "is AI really intelligent?" seem to me misplaced, and that is precisely what makes them interesting. Misplaced because they are asked as questions of science, to which one would expect laboratories to answer one day with a result, whereas they are not such questions. A scientific question has a form: one knows what would count as an answer, and one can go and look for it. These do not have that form. No measurement, no benchmark, no architecture will answer them, because what is being asked is not a fact about the machine but a decision about what we call thinking, and that decision precedes any measurement. It belongs to the philosophy of mind, which is a respectable discipline without results that laboratories could produce, and not to the engineering sciences, which have results and have nothing to say on the matter.

Interesting, because they touch on what we are and they force us, for the first time, to say it precisely, failing which we can no longer distinguish what we do from what these systems do. And with no place in the sciences, because science does not settle what it has not defined, and it has better things to do than arbitrate between Descartes and Aristotle with learning curves.

I must say that as a practitioner, my pragmatism deliberately passes over all of this, and with a certain relief. To build, evaluate, teach or criticise these systems, I need to know neither whether they think nor whether they understand. I need to know what they do, under what conditions they do it badly, and to whom what they produce should be attributed. These three questions are empirical, they have answers, and they largely fill a research career. What follows is therefore not a plea for computer scientists to become philosophers. It is a plea for them to know, when they use the word "intelligence", which question they are answering, and to stop claiming to answer the other one.

I do not like the term "artificial intelligence"

Before anything else, a declaration of distrust towards vocabulary. I do not like the expression "artificial intelligence", and not out of affectation: every time I come across it in a serious text, I note that it is doing one of three jobs there, and none of the three is a job of knowledge.

Either the expression means nothing. This is the case when "intelligence" is not defined, which is almost always the case. The term appears in the proposal by McCarthy, Minsky, Rochester and Shannon of 31 August 1955 for the Dartmouth workshop, and McCarthy later explained that he had chosen it to give a name to a research programme distinct from Wiener's cybernetics. The proposal itself does not define intelligence; it puts forward the conjecture that every aspect of it can be described precisely enough for a machine to simulate it. This is not a thesis about the nature of what was going to be built, it is a bet on a method. Seventy years later, the field still has no definition of intelligence that commands consensus, and the reference textbook by Russell and Norvig, in its third edition, opened with a table of eight competing definitions before choosing the most convenient one. A technical term without a definition is not a technical term. It is a label, and no consequence can be drawn from a label.

Or the expression explains nothing. This is the case when "intelligence" designates, for convenience, the set of things done by the systems we call intelligent. One then says that a program plays chess thanks to its intelligence, just as Molière put people to sleep by dormitive virtue. The term names the effect and passes it off as a cause. What explains that a program plays chess is an evaluation function, a search algorithm, an opening database; what explains that a language model translates is an architecture, a corpus, a training objective. "Intelligence" adds nothing to these explanations and, worse, masks them, because it suggests that there is something more behind them, something that would be intelligence proper, and of which no one will ever say what it is.

Or the expression is a tautology. This is the case whenever it is defined by its use. "Artificial intelligence is what artificial intelligence researchers do." "A system is intelligent if it behaves intelligently." "AI is getting machines to do what requires intelligence when a human does it." This last formula, which is Minsky's in the introduction to Semantic Information Processing (1968), looks like a definition and is not one, since it refers to a human intelligence that it does not define any further. It also has a known defect, called Tesler's theorem, formulated by Larry Tesler in the 1970s and popularised by Hofstadter: "AI is whatever hasn't been done yet", because as soon as something is done by a machine, people stop saying that it required intelligence. Character recognition, route finding, image compression were artificial intelligence; they are now phone features. A term whose extension empties as it succeeds designates nothing stable.

This threefold defect is not trivial. It explains why the public conversation about these systems is so poor: one cannot reason correctly with a term that, depending on the moment, says nothing, names an effect as a cause, or goes round in circles. One can, on the other hand, speak precisely about what these systems are and do, provided one names them by what they are. A language model is a statistical model of text sequences, trained to predict the continuation of a context. A convolutional neural network is a learned filter. A recommender system is a matrix factorisation. These names sell less well. They have the advantage of saying something, and of then allowing the only question that matters, which is whether what these objects do deserves, in a sense to be specified, the name of intelligence.

I will therefore use the acronym "AI" in what follows, because it has become the common name of these systems and it would be pedantic to refuse it. But I use it as one uses "X-rays" or "black hole", as a proper name whose etymology should not be taken for a description. The whole article is an attempt to say what would have to be settled for the etymology to become a description again.

The debate goes round in circles because it starts in the wrong place

These two questions have no answer as long as one has not said what one means by thinking and by intelligent, nor how one would go about finding out. This is not a philosopher's precaution. It is the reason why ten years of public debate on artificial intelligence have produced a great deal of noise and almost no shared conclusion.

The pattern is always the same. A system does something that was held to be reserved for the human mind: playing chess, translating, writing a sonnet, passing a medical exam. Half the commentators conclude that the machine thinks. The other half moves the criterion: that was not what thinking was, it was something else, which remains out of reach. Then the machine does that other thing, and the criterion moves again. Neither camp is wrong about the facts. Both are wrong about the method, because they judge first and define afterwards.

The thesis of this article is simple. Before saying anything at all about the intelligence of a machine, two things must have been settled: a theory of intelligence, that is, what one takes to be constitutive of thinking, and an epistemological framework, that is, what one accepts as evidence that this thing is present or absent. Whoever has not done this work is not talking about AI. They are talking about their own reactions to AI, and dressing them up as judgements.

This work does not require agreement on substance. A dualist and a physicalist will never agree on what the mind is. But if each of them has set out their criteria before looking at the machine, they will at least be able to say precisely what they disagree about, and to note that the machine, in general, does not settle their disagreement. That observation is worth more than the opposite illusion, which is that technical progress will settle a question nobody has formulated.

"Intelligence" is not a univocal word

The first obstacle is lexical, and it is more serious than it seems. The word "intelligence" covers at least three things that the philosophical tradition has always taken care to distinguish, and that everyday vocabulary conflates.

There is first intelligence in the functional sense: the capacity of a system to achieve goals in an environment, to solve problems, to adapt. In this sense, a thermostat is a little intelligent, a dog more so, a chess program very much so in a narrow domain. This is the sense computer science adopted in 1955 when McCarthy coined the expression "artificial intelligence", and it is an honest sense as long as it is kept in its place. It says nothing about what goes on inside the system. It describes a behaviour.

There is next intelligence in the sense of discursive reasoning: chaining inferences, moving from one proposition to another according to rules, breaking a problem down and putting a solution back together. In contemporary vocabulary, this is the syntactic part of thought, the part that operates on the form of representations without needing their content, and it is the part that logic, and then computer science, have progressively mechanised. That a machine reasons in this sense has no longer been open to dispute since Frege and Turing. The only question is whether it does so in the proper sense or in a derived sense, and that question refers to the third sense.

There is finally intelligence in the sense of understanding proper: grasping what a thing is, forming a concept that holds for all cases and not only for those encountered, aiming at a meaning and not merely manipulating signs. This is the semantic part, and it is the strong sense, the one that brings in the question of intentionality, consciousness and truth. Searle fixed the current vocabulary by distinguishing original intentionality, that of a mind whose states are about something by themselves, from derived intentionality, that of a text or a program whose states are about something only because a mind interprets them; Dennett disputes that the distinction holds, but he uses the same words. Philosophers have named this part in various ways depending on the period, intellect, the understanding, semantic comprehension, but it is this part that Aristotle, Descartes, Kant, Searle or Chalmers are talking about when they ask what thinking is. And it is this part, in general, that people mean when they ask whether AI "really thinks".

Confusing these three senses produces symmetrical sophisms. The first consists in noting the functional intelligence of a system and concluding that it has intellect: the machine solves problems, therefore it understands. The second consists in denying a system's intellect and concluding that it is not intelligent in any sense: the machine does not understand, therefore it does nothing intelligent, it is only a parrot. Both arguments slide from one sense to the other without saying so.

Philosophy has long had a tool for this, analogy, which is found in Aristotle and which the medievals systematised. (Wittgenstein's "family resemblances" are something else: overlapping similarities with no reference term.) To say that a dog is intelligent, that a solution is intelligent, that a man is intelligent, is not to use the word three times in the same sense, nor in three unrelated senses. It is to use it according to a proportion, and relative to a single reference term, human intelligence, from which the other uses take their meaning: in each case there is something that plays, at its own level, the role that understanding plays in man. The three senses distinguished above are therefore not merely juxtaposed; they are ordered from understanding. This tool makes it possible to say without contradiction that a language model is intelligent in the functional sense, that it reasons at least by imitation, and that it remains open whether it understands anything at all in the strong sense. Without this tool, one is condemned to one of the two sophisms.

It must be added that the inflation of vocabulary is not innocent. "Artificial intelligence" sells better than "statistical sequence processing". "The model understands your question" sells better than "the model produces a distribution over the next tokens". Those who use the strong sense to describe facts belonging to the weak sense often have an interest in doing so, and those who react by denying the weak sense often have an interest in not seeing what the machine actually does. In both cases, the absence of lexical distinction serves something, and that something is not knowledge.

One always has a framework, even when one believes one has none

An epistemological framework is the set of what one accepts as evidence. For a question such as "does this system think?", one must at the very least have answered three things: which facts count, what weight to give them, and what would make one change one's mind.

The first illusion to dispel is that of the neutral gaze. Nobody observes a machine without a framework. Whoever says "I'm just looking at what it does" has already decided that behaviour is the only relevant datum, which is a philosophical position, behaviourism, and one that almost nobody defends any longer with full knowledge of the matter. Whoever says "we can't know what it feels" has already decided that feeling is the criterion and that it is inaccessible, which is another position, and one that also has its difficulties. Whoever says "it's only computation" has decided that computation excludes thought, without asking whether the brain does not compute as well. The framework is there in each case; it is simply implicit, and therefore removed from scrutiny.

An implicit framework has two defects. The first is that it cannot be criticised, since it is not stated. The second, more serious, is that it can change without its holder noticing. This is what happens when the criteria of intelligence retreat as machines advance: the framework has moved, but since it was written down nowhere, the person who holds it sincerely believes they have never changed their mind.

The work of making it explicit consists in answering the three questions in writing, before looking at the machine. Which facts count: behaviour alone, or also the architecture, the causal history of the system, the nature of its substrate? What weight: is behaviour indistinguishable from a human's sufficient, or is it only an indication? And above all, what would make one change one's mind: what observation, if it occurred, would oblige me to revise? This last question is the most important, and it is the one almost nobody asks. A framework that does not answer it is not a framework, it is a mood.

One must be honest about a difficulty. For the question of intellect in the strong sense, it is not certain that there exists any external observation that could settle it. If understanding is something other than behaving like someone who understands, then no behaviour proves understanding, and one falls back on Searle's Chinese room argument, which is more than forty years old and has never been convincingly refuted, but which has never convinced those who did not already share its conclusion either. An honest framework must say what it does with this difficulty: either it admits that the question of the strong sense is beyond empirical reach and confines itself to the functional sense; or it claims a source of knowledge other than external observation, introspection for example, and it must then say why this source is valid for us and not for the machine.

This point deserves emphasis because it is the source of most misunderstandings. When two people argue about the intelligence of a language model, they generally believe they are arguing about the model. In reality they are almost always arguing about what counts as evidence, and that disagreement was already there before the machine existed. AI did not create the problem; it made it impossible to ignore.

Five frameworks, five predictions

To show what an explicit framework changes, it is enough to take a few that are actually defended today and look at what each says about AI before having looked at it. I present them without deciding between them; the point is that they give different and predictable answers, and that this predictability is precisely what a framework provides.

Functionalism. To think is to occupy a certain functional role: receiving inputs, transforming them according to an organisation of a certain type, producing outputs, all independently of the substrate that realises this organisation. This framework was formulated by Putnam in 1967 in "The Nature of Mental States" and it is, in fact, the implicit framework of most of computer science. Its prediction about AI is clear-cut: if a system realises the right functional organisation, it thinks, whatever it is made of. The criterion is behaviour, taken in a broad sense that includes the internal structure of the processing. The difficulty of functionalism has been known since the beginning, and Putnam himself ended up abandoning the position in 1988 in Representation and Reality: it must say which organisation is "the right one", and it has never been able to do so without circularity. The right organisation is recognised by the fact that it produces thought, and thought is recognised by the fact that it comes from the right organisation. Block showed as early as 1978, with the Chinese nation argument, that any functional organisation can be realised by a device that nobody would say thinks. Faced with a language model, the functionalist is therefore in an uncomfortable position: their framework tells them the question is open, and they have no criterion for closing it.

Reductive physicalism. To think is a physical process, in this case a cerebral one. Thought is not an abstract organisation but a concrete activity of a certain type of matter organised in a certain way. Depending on its variants, this framework predicts two opposite things. One variant holds that only the biological substrate produces thought, in which case no machine will think, whatever its performance; this is Searle's biological naturalism, which calls itself physicalist while rejecting functionalism, and which has never been able to say what the brain has that silicon lacks, except that it produces thought. The other variant holds that the substrate matters little provided the same causal processes are realised, and it then joins functionalism by another route, with the same difficulties. Physicalism has one strength: it is about what is, not about what appears. And one structural weakness: it has no answer to the problem of why a physical process is accompanied by lived experience, what Chalmers named the hard problem in 1995, and it has no generally accepted answer either to the indeterminacy argument Kripke drew from Wittgenstein in 1982, according to which no physical fact about a system fixes which of several incompatible contents that system represents. Its best reply goes through naturalistic theories of content, those of Dretske, Fodor or Millikan, which try to fix content through causal relations or through the history of functions; none has won general assent, but the objection has not gone unanswered. Faced with AI, the physicalist is the most exposed to seeing their criteria shift, because their framework does not tell them in advance what, in the physical process, is essential to thought.

Dualism, in its classical and contemporary forms. To think, in the strong sense, is the act of something that is not reducible to a physical process: a substance distinct from the body in Descartes, a non-physical property in property dualists such as Chalmers, an immaterial faculty in Aristotle. Aristotle's argument starts from the object of thought: the intellect grasps the universal, what the triangle is and not this triangle, whereas a material faculty receives forms only in their particularity; the faculty that grasps the universal is therefore not material. A contemporary version is found in those who start from Kripke's indeterminacy argument (1982), of which Boghossian (1989) gave the standard discussion, and turn it around: if no physical state fixes a determinate content, and if some of our thoughts have determinate content (addition, modus ponens, thinking that two and two make four and nothing else), then those thoughts are not physical states. The variants differ on what is non-physical and why, but they converge on the prediction: no machine will have thought in the strong sense, whatever its performance, because performance belongs to functional intelligence and to the imitation of reasoning, not to understanding. Whatever sensible output a machine produces, it produces with derived intentionality: the reason for being of its outputs lies with those who built, trained and queried it, just as Dretske's thermometer indicates the temperature without knowing anything about it. The strength of this framework is that it was set out well before the first machine and that it does not need to move; its decisive premise, that the intellect grasps the universal and that thought has determinate content, is the one its opponents deny, and we will see below that this disagreement is not settled. Its real weakness lies elsewhere: it says what the machine is not, but nothing about its proper action, about what it does as an instrument that no other instrument does. That question it hands over to science, which alone can answer it.

Enactivism and theories of embodied cognition. To think is the activity of a living organism that maintains its existence in an environment, and meaning arises from this vital relation before any symbolic processing. Varela, Thompson and Rosch set out the line in 1991 in The Embodied Mind; Noë and Thompson extended it. Its prediction is that language models, which have no body, no environment and no interest of their own in their maintenance, produce text without meaning for them, whatever its quality. Unlike dualism, this framework leaves a door open: an artificial system that was autonomous, embodied and subject to real precariousness could, in principle, satisfy its criteria. Its weakness is that it does not say clearly what, in life, is necessary for meaning, and so it risks redefining life after the fact according to what machines do not yet do; the criterion "having an interest of one's own in one's maintenance" is itself a teleological notion that the framework borrows without grounding it.

Panpsychism and integrated information theories. Experience is not produced by certain organisations but present, to varying degrees, in every physical organisation, and what varies is its degree of integration. Goff and Strawson for the philosophical version, Tononi for the formal version with integrated information theory. The prediction is the most open of all: the question is not whether a machine has experience, but how much, and the answer depends on its architecture, not on its behaviour. A feed-forward language model, without a feedback loop, would according to Tononi have zero integration despite its performance. This framework has the advantage of giving a criterion independent of behaviour. Its weakness is that this criterion is contested down to its measurability, that an open letter from more than a hundred researchers called the theory pseudoscience in 2023, and that the underlying thesis, the ubiquity of experience, is held by many to be unintelligible: it explains how experience combines by asserting that it is everywhere, which displaces the problem without solving it.

What this comparison shows is not that one framework is the right one. It is that the five give, for the same object, different verdicts, and that each verdict follows from premises set before the observation. The functionalist will say "perhaps, and I have no means of knowing". The physicalist will say "it depends on what in the process is essential, and I don't know that yet". The dualist will say "no, by nature, and here is why". The enactivist will say "no, not this one, but another could". The panpsychist will say "a little, and here is how to measure it". None of the five is refuted by what language models do in 2026. Whoever has no framework believes the models of 2026 settle the matter; they are mistaken, and they will be mistaken again with the models of 2030.

How to recognise the absence of a framework

The absence of a framework does not announce itself through silence. It announces itself through recognisable symptoms, and it is useful to name them, because they are common among people who are otherwise competent.

The first symptom is the retroactive criterion. One observes what the machine does not yet do, and makes it the mark of true intelligence. In 1997, it was chess; once Deep Blue had got past it, it was no longer chess, it was go, which required intuition. In 2016, it was no longer go, it was language. In 2023, it was no longer language, it was reasoning, or creativity, or understanding of the physical world. Each criterion was sincere at the moment it was set. None had been set before the previous one fell. A criterion manufactured after the defeat of the previous one is not a criterion; it is a line of retreat.

The second symptom is oscillation. The same person, depending on the day and on the example shown to them, says that the model "is only a stochastic parrot" and that it "obviously understands what it is asked". The two statements are not compatible, and their alternation shows that the judgement follows the impression of the moment rather than a principle. Oscillation is the sign of a framework that does not exist, or that has not been written down.

The third symptom is the moving Turing test. In 1950 Turing proposed an explicit, behavioural and falsifiable criterion: if an interrogator cannot distinguish the machine from the human, we will say that it thinks. One can dispute this criterion, and many have, but it was a criterion. What has been observed since systems have more or less passed it is that almost nobody accepts the conclusion, and that almost nobody explicitly says they are abandoning the criterion either. It is modified in silence: an expert interrogator was needed, a longer duration, questions about lived experience. Each of these additions is defensible. None was announced before the test was passed.

The fourth symptom is the substrate argument invoked on demand. "It's only matrix multiplications" is an accurate description of a language model. "It's only ion exchanges across membranes" is an accurate description of a brain. Whoever takes the first statement as a refutation of thought and the second as a neutral description must explain the difference. If they cannot, they do not have an argument, they have a preference.

The fifth symptom, finally, is the reverse of the previous ones: generosity without a criterion. Understanding, intentions, emotions are attributed to the system because its text has their form. It is the same defect in the other direction. The text produced has the form of understanding because it is produced from texts that had that form; this says nothing, in one direction or the other, about what goes on in the system. Whoever concludes that there is understanding on this basis has no more of a framework than whoever concludes that it is absent on the basis of matrices.

These five symptoms have a common cause. In each case, the judgement is formed before the criterion, and the criterion is reconstructed to justify the judgement. It is the opposite of what a framework does, whose function is precisely to fix the criterion before one is tempted to judge.

What a good framework must provide

A framework does not need to be true to be useful to the discussion. It needs to fulfil four functions, and a framework that fulfils them is worth more, for thinking about AI, than a correct but unformulated intuition.

The first function is to provide criteria prior to the facts. Before looking at a system, the framework says what would have to be observed to conclude one way or the other. This presupposes a theory of intelligence precise enough to generate criteria, and this is where most implicit frameworks fail: "it thinks if it really understands" generates no criterion, because "really understanding" is not defined independently of the judgement one is trying to ground.

The second function is to distinguish the tool from the agent. This is the most useful distinction in practice and the one least often made. A tool produces effects whose reason for being lies with its user; an agent produces effects whose reason for being lies within it. The hammer does not drive nails for itself. In contemporary vocabulary, this is the distinction between derived teleology and intrinsic teleology: Millikan attempted in 1984 to naturalise the latter through the selective history of functions, Dennett has argued since 1987 that there is only derived teleology, including in us, and the debate is not closed. This distinction is independent of performance: a very sophisticated tool remains a tool, a very limited agent remains an agent. It is also independent of the substrate. It does, however, require a theory of what it is to have a reason for being in oneself, and this is where frameworks diverge. Frameworks that have a strong notion of organism, substance or autonomy (Aristotelian-inspired dualism, enactivism) have such a theory, and it is one of their real strengths. Here it is in one sentence: the instrument has an action of its own, but the effect it produces exceeds it, because someone else is using it; the agent has in itself the reason for its effect. The difference between a language model and a saw therefore lies not in the category of instrument, which they share, but in their proper action, and that action is there to be studied. Strict functionalism has it much less, because it defines everything by role, and the role of a tool and the role of an agent can coincide functionally. Its best answer comes from Millikan's teleosemantics: a device has a proper function when it exists because earlier devices of the same type produced that effect and were selected for it. The reason for being then lies neither with the user nor in a substance, but in a history of selection. Applied to a language model, the question becomes precise: is training, which retains the parameters that produce certain effects, a history of selection in this sense, or merely the execution of an objective set by others? Both answers can be defended, and the question can be investigated. Without this answer, the functionalist, faced with a language model, hesitates between tool and agent, and this hesitation is not a personal failing but a consequence of their framework.

The third function is to fix the conditions for revision. A good framework says what would make it change. Not "if the machine really understood", which says nothing, but a determinate observation: if a system, without having been trained on a given class of problems, solves one by explaining a general rule it has seen nowhere; if a system refuses a task for a reason of its own and maintains it against its user; if a system corrects its own model of the world on the basis of an inference and not of a datum. Each of these conditions is debatable. The point is that whoever has written them down in advance will be able to say, on the day one of them is met, whether or not they revise, and why. Whoever has not written them down will say, as always, that that was not it.

The fourth function is to locate the disagreement. Two people who each have an explicit framework can say precisely on which premise they diverge. A dualist and a functionalist diverge on whether the content of a thought is a functional relation between a system and its environment or something that no relation of that kind exhausts. It is an old divergence, older than AI, and it will not be settled by a language model. Knowing this avoids arguing about the model when one is really arguing about Descartes or Aristotle. Two people without a framework, on the other hand, argue about everything and locate nothing, because their disagreements concern premises that neither of them has formulated.

It will be noted that none of these four functions requires the framework to be that of the majority, or to be compatible with the state of science. An explicit framework, whatever it is, with its criteria and its conditions for revision, is more useful for thinking about AI than a vague position, materialist or not, that believes itself exempt from criteria. Science is not a framework; it is what one does inside a framework, and it does not exempt one from having one.

The price of irrefutability

There is an objection to all of the above, and it is a serious one. If a framework is good because it gives stable answers, the best framework is the one that is never caught out. But a framework that is never caught out is also a framework that learns nothing. Popper built his philosophy of science on this point, and it applies here without modification.

Let us take the dualist framework, since it is the most stable of the five. Its thesis about AI, "an artefact cannot have thought in the strong sense; whatever sensible output it produces, it produces as an instrument", is irrefutable by construction. Whatever the behaviour of a future machine, it will be filed under the explanation by the instrument. If a machine proves a theorem that nobody has ever proved, it is the intelligence of its designers and its sources acting in it at a distance. If it refuses an order, it is because its designers put that refusal in it. No observation can contradict the thesis. That is its dialectical strength, and it is its epistemic defect.

The defect is not logical. An irrefutable thesis can be true or false; its irrefutability says nothing either way. The defect is that the thesis, at this point, has stopped explaining and is content to classify. It says what the machine is not, and it says it well. It ranks the machine among instruments, and the category does not distinguish a language model from a saw. What distinguishes them is their proper action, and that is precisely what one would need to understand; the framework does not exempt one from that study, it hands it over to science, and whoever adopts it does themselves a poor service if they believe they have already done it.

One must nevertheless be fair to irrefutability, because the Popperian objection does not apply without caution. A thesis about nature does not have to be refutable by a behaviour, any more than "there are infinitely many primes" is refutable by a calculator that stops. Popper himself reserved his criterion for empirical theories and did not apply it to logic or to metaphysics, which he held to be meaningful without being refutable. The real question is therefore not whether dualism is refutable by AI, it is not and does not claim to be, but whether its premise holds. That premise is that some of our thoughts have determinate content. Its defenders argue that denying it has a price: whoever asserts that the content of their thoughts is indeterminate asserts something determinate, namely that, and if they concede that their assertion is itself determinate only "for practical purposes", they seem to give up saying anything true. Its opponents reply that the argument assumes what it sets out to prove: asserting does not require determinate content in the strong sense, only conditions of use shared by a community, which is what Kripke himself proposed as the sceptical solution, and truth can be understood in terms that ask for nothing more. The debate remains open. It is this debate, and not the behaviour of machines, that decides the strength of the dualist framework.

The same defect exists, in another form, in the physicalist who holds that "it's only computation". That thesis too absorbs everything: whatever behaviour the machine displays, it was computation. It cannot be surprised, and it learns nothing. The honest physicalist is the one who admits that their framework does not tell them in advance what, in computation, would make thought, and who is therefore in a position to be surprised. This position is uncomfortable. Its discomfort is information.

The lesson is not that stable frameworks should be abandoned for unstable ones. It is that a framework must account for its own irrefutability. The dualist who says "my thesis cannot be refuted by a behaviour, because it is about nature and not about behaviour, and here is what I lose by accepting this" is in a better position than one who believes that the irrefutability of their thesis is a proof of its truth. The first knows where their blind spot is; the second does not even know they have one. The same requirement applies to the panpsychist, whose measure of integration can be adjusted until it gives the desired result, and to the enactivist, whose notion of life can retreat as machines gain in autonomy.

There is a simple test. Ask the holder of a framework what a machine would have to do for them to revise. If they answer with a precise observation, they have a framework and are in a position to learn. If they answer "nothing, on principle", they have a metaphysics, which is respectable, but they must then stop claiming to talk about AI. They are talking about the nature of the mind, and AI is for them only an example, which will teach them nothing because they have nothing to learn from it. Both positions are tenable. What is not tenable is holding the second while giving oneself the airs of the first.

Application: what language models tell us, and what they do not

Let us apply these frameworks to the object that has occupied everyone since 2022. Language models settle none of the disagreements between frameworks. But they teach each of them something, and that something is not the same from one framework to another.

To the functionalist, they teach that the functional organisation of language is much simpler than was believed, or at least much more learnable from data. For fifty years, linguistics held that syntax required specific innate structures; systems that contain nothing of the kind produce flawless syntax. This does not prove that humans do not have these structures. It proves that they are not necessary for the behaviour, and the functionalist, who defines everything by behaviour, must conclude from this that the functional role of language can be realised by routes they had not foreseen. Their framework comes out of it broadened, not refuted.

To the physicalist, they teach that the question of the substrate is more pressing than they thought. As long as the only systems that spoke were brains, one could assume that thought and the biological substrate went together without having to say why. Now that a radically different substrate produces comparable linguistic behaviour, the physicalist must either admit that the substrate is not essential, and become a functionalist, or say what in the biological substrate is essential and missing from silicon, and they do not, today, have an answer that is not a retroactive criterion. It is the most uncomfortable position, and therefore the one with the most to look for.

To the dualist, they teach something subtler and more disturbing. Most dualisms distinguish discursive reasoning, which chains inferences, from simple understanding, which grasps what a thing is, and they hold that the former proceeds from the latter: one cannot reason about what one does not understand. Yet here are systems that reason, at least in appearance, without its being possible to establish that they understand. The dualist has two answers. The first is the instrument: this reasoning is not the machine's but that of the humans whose texts it recombines. It is coherent and irrefutable, with the cost described above. The second is more honest towards the facts: discursive reasoning is more mechanisable than was thought, and only understanding resists. This second answer costs the underlying thesis little and it obliges the dualist to recut their theory of mind, which is exactly what a living framework should do in the face of a new fact. Language models therefore put to the dualist not the question "do they think?", which the dualist has settled, but the question "what is reasoning without understanding?", which the dualist had never had to ask.

To the enactivist, they teach that language can be produced without a body, which is a surprise for a framework that held meaning to be rooted in embodied action. It remains for the enactivist to show that this bodiless language is meaningless, and they must do so by a criterion that is not merely the absence of a body. To the panpsychist, finally, they offer a test case: a system with very rich behaviour and, according to the panpsychist's own measures, zero integration. If the measure is good, behaviour is not an indication of experience, which is a strong conclusion. If it is bad, it is the framework that is in trouble.

These lessons have something in common. In each case, what the machine teaches the holder of the framework is not something about the machine. It is something about their own framework: a functional organisation simpler than expected, a substrate harder to defend than expected, a distinction within the mind more porous than expected. For the philosophy of mind, language models are less an object of study than an instrument for measuring our theories. They do not tell us whether they think. They tell us where our theories of thought were vague, because those theories had never had to pronounce on a case of this kind.

What the models do not tell us, on the other hand, is just as important to pin down. They do not tell us whether there is something it is like to be them. No behaviour can establish it, since behaviour is precisely what they have been trained to produce. They do not tell us whether understanding is something other than a capacity to produce appropriate text, because that is a question of definition and not of observation. And they do not tell us which of the frameworks is the right one, because all of them absorb the models without damage, each in its own way. Whoever expects the next generation of models to settle these questions will wait a long time. They are not the kind of question a fact settles.

Conclusion: choose, take responsibility, say under what conditions one would revise

There is no position without a framework. The only alternative is between an explicit framework, which can be criticised and can learn, and an implicit framework, which moves without one knowing it and serves only to justify judgements already formed. The latter is the norm in the public debate on AI, among enthusiasts as among sceptics, and that is why this debate does not move forward.

What I propose to anyone who wants to talk about AI seriously comes down to three requirements. First, say which theory of intelligence one holds: functional, physicalist, dualist, embodied, panpsychist or other, but one, and with its internal distinctions, starting with the one between the intelligence of behaviour, that of reasoning and that of grasping meaning. Next, say what counts as evidence, and in particular whether introspection is evidence, and why it would be valid for us and not for the machine. Finally, and this is the point least often honoured, say in advance what would lead to revision, or acknowledge that one holds a position that nothing will lead one to revise, and draw the consequences as to what one is actually talking about.

These requirements favour no camp. They are as costly for those who attribute thought to machines as for those who deny it to them. The former must say what they mean by thinking and why behaviour is enough to establish it. The latter must say what the machine lacks and how they know it. In both cases, the work is philosophical before it is technical, and it cannot be delegated to engineers or to benchmarks.

I have not said here which framework I hold, and that is deliberate. This text is about method, not about nature: settling the question of what the mind is would mean writing another text.

There is, on the other hand, a minimal framework that everyone who publishes in this field should accept, whatever their underlying position. When I do research, I try to check my philosophical opinions at the door, and to disregard those of my co-authors, not out of indifference but because a result that depends on the metaphysical convictions of the person who obtained it is not a result. If a measure of asymmetry between two formulations of the same question only makes sense for a functionalist, it measures nothing; if a robustness evaluation is only acceptable to a dualist, it evaluates nothing. Scientific work on these systems must be able to be read, checked and extended by people who share none of my premises about the mind, and I must be able to do the same with theirs. This minimal framework is methodological naturalism, and one should say what it is: not a hypothesis about what exists, but the very definition of the object of the experimental sciences, which deal only with what is observable and reproducible. Presented in this way, no one can refuse it, and it claims nothing about being. It goes together with a pragmatism, which serves as a rule for choosing questions in the laboratory: one asks those whose answer changes something in what one builds, predicts or corrects. This pragmatism does not judge the truth of other questions, only their place, which is elsewhere.

This framework does not require anyone to renounce their convictions. The dualist can hold methodological naturalism as a working convention, without conceding anything on substance; the physicalist will see in it a description of what they think anyway, except that a framework that needs this naturalism to be ontological as well in order to function is confusing its method with its metaphysics, and owes an explanation; the panpsychist and the enactivist will find in it a discipline that obliges them to make their criteria measurable. What the framework requires is that philosophical discourse and research results be kept separate in the text, and that one can remove the former without the latter collapsing. Most of what is published on the "intelligence" of language models does not pass this test: take away the author's philosophical premises, and all that is left are curves of which one no longer knows what they show. This is the sign that the author had no scientific framework, and let their philosophical framework do the work in its place.

This does not prevent saying what the exercise costs each person outside the laboratory, because it is the same cost in different forms. Whoever holds a stable framework, dualist for example, gains by not panicking and loses by not being able to be surprised; they must then acknowledge that the question "what is it?" remains entirely open for them and that it calls for precisely the naturalist toolkit described above, which will say nothing about the question of nature. Whoever holds an open framework, functionalist or physicalist, gains by being able to learn and loses by not knowing in advance what counts; they must then write down their criteria before the next generation of models shifts them without their knowing. In both cases, the work is the same: knowing what one's framework is about, and what it is not about.

This is perhaps the shortest formulation of what I wanted to say. A good framework is not the one that answers every question. It is the one that knows which questions are its own.

Sources