What AI changes in a programming course

A coding assistant now writes, in a few minutes, the project I set my students last year: the code, the README, the diagrams. I am not saying this to complain. I am saying it because part of my job is becoming obsolete, and it is better to look at that than to ban it.

Many teachers react in two stages: detection and prohibition first, integration later. That is what Lau and Guo's survey of programming instructors across nine countries shows (Lau and Guo, 2023). Both reflexes are understandable. The first one fails, because detectors are unreliable and a project done at home cannot be checked. The second stops halfway: "letting students use AI" is not a pedagogy.

Here is what I take from this reading, and the course I am going to rebuild from it.

What the assistant does to beginners

The most solid result comes from a controlled trial run with nearly a thousand Turkish high-school students in mathematics (Bastani et al., 2025). During the practice sessions, the students with access to a ChatGPT-like assistant did markedly better than the others. With that access removed for the exam, they did worse than those who had never had the tool. Only a restricted version, one that gave hints instead of answers, preserved the learning. These are high-school students in maths, not computing undergraduates, and I transpose with caution; the mechanism, though, is the one every programming teacher sees every year: a generated answer feels like understanding. I have already cited this trial in the text of the pre-term course in prépa, where I was addressing the students; what I take from it here has to do with course design.

Among novice programmers, Prather and his colleagues observed two populations diverging, in the lab and with eye tracking (Prather et al., 2024). Those who already have the basics use the assistant to go faster and know how to reject a wrong suggestion. The others sink: they accept code they do not understand and build false mental models for themselves. The study is qualitative and the sample small, but it confirms what an earlier experiment with Codex had already suggested: the best-prepared students are the ones who benefit most from the tool (Kazemitabaar et al., 2023).

I draw a working principle from this: you cannot check what you do not understand. Being able to steer an AI presupposes being able to read, diagnose and modify code without one.

Two things to teach, in that order

This principle has a consequence I had not gauged when I started. A programming course now has to teach two distinct things, and to teach both of them.

The first is the basics without AI: reading code, finding what is off in it, modifying it, justifying a choice. Nothing new in the list; what is new is that they have to be certified explicitly, in conditions where the assistant is absent, because nothing else certifies them any more.

The second is the use of AI to go beyond the basics. That skill is not the prompt, which goes out of date in six months. It is four more durable things: framing a problem before delegating it, starting from the interfaces; checking what has been produced, through review and through tests designed to catch the tool's mistakes; integrating twenty generated contributions into a codebase that stays coherent; and answering for every line, which means being able to explain it and to make it evolve. Bearman and her colleagues call the second one "evaluative judgement", the capacity to judge the quality of work, one's own and other people's, and they make it the central outcome of an education in the age of generative AI (Bearman et al., 2024).

The order matters. Points two and four are impossible without the first skill. A student who cannot read code does not steer the AI, they are carried along by it. Teaching the second without having certified the first means training people who accept what they do not understand; that is exactly the population Prather describes.

What a submission proves

A submitted project no longer says much about the person submitting it. The problem predates LLMs: a Finnish study showed that a sizeable share of students could not explain how the code they had themselves handed in worked (Lehtinen et al., 2021). AI changed the scale of it.

Several universities have built a framework out of this, the University of Sydney's being the most explicit: two assessment lanes. A secured lane, in person and without AI, which certifies what has been learnt. An open lane, where AI is allowed since it cannot be banned, and which serves for learning. The AIAS scale by Perkins and his colleagues formalises the same idea as levels of permitted use, to be set for each assessment according to what it measures (Perkins et al., 2024).

I sum this up with an image: the driving licence and the driving. The AI-free test issues the licence, the project with AI marks the driving, and a driving mark without a licence certifies nothing. The image has its limits, including one that matters: you do not resit your licence every semester, whereas a student is making progress while being assessed. So my written exam will be on paper, and will ask the student to read code, find the flaw in it, propose the change and justify it. Including AI-generated code, with planted flaws, since reviewing has become a central skill.

On that last point, a METR study deserves to be cited along with its limits. In 2025, sixteen experienced developers took 19% longer on their tasks with an AI assistant, while feeling that they had gone faster (Becker et al., 2025). The sample is tiny and the confidence interval wide; the 2026 replication, with 57 developers, finds an effect close to zero. What remains is the gap between perception and measurement, and that is what interests me as a teacher. A developer does not know, from the inside, whether AI is helping. So they have to be taught to check rather than to trust their impression.

Two formats that are dying

Two formats do not survive what precedes, and I would rather say so plainly.

The lecture first. Its function was to transmit information, and that has become the least scarce resource there is. An assistant explains polymorphism as well as a lecture hall does, on demand, at three in the morning, and it answers questions. What it does not give is feedback on what the student produces, and judgement exercised in front of them. The format spent most of the contact time on what is abundant, and almost none on what is scarce.

The guided lab next. A step-by-step handout, with the skeleton supplied and the pattern's name in the title, is solved in thirty seconds with an assistant. Even before, it produced code that works without the student having made a single decision. It no longer has any formative value, since nothing is learnt in it that cannot be delegated, nor any assessment value, since the result says nothing about its author.

What is dying is long spoken transmission and the application exercise with no decision in it. What is not dying is explanation, when it is short and comes at the moment the need has just been felt, and hands-on practice, when it starts from a situation and demands a choice.

My videos, and the investment for nothing

I had already moved my courses online, as videos. Many hours of filming. My first reaction on reading all this was to tell myself that the investment was lost: if the assistant explains better than the lecture hall, it also explains better than the filmed lecture hall.

That is partly true, and it should be said. A video that reproduces a lecture reproduces its flaws, without even the teacher's presence. The reference study on the subject, run over nearly seven million viewing sessions of online courses, shows that attention drops after six minutes whatever the length of the video, that lectures filmed in a room engage poorly, and that formats where a problem is solved on screen do better than narrated slides (Guo, Kim and Rubin, 2014). It measures engagement, not learning, and on MOOCs, not on degree programmes. But the message is unambiguous: the filmed tunnel is no better than the tunnel in the room.

Only partly, though. What loses its value is the idea that video replaces the course. What keeps its value is video as an on-demand resource: six minutes on one precise point, watched by a student because they got stuck on it, followed by a question they have to answer. In the course I am rebuilding, it becomes half of what happens outside the session: the support for the one who lost the thread during the explanation, the thing the advanced student skips, and the place to send someone when a quiz reveals a weak notion. My hours of filming are not lost; I am going to recut them, and nobody will watch them passively any more. Watching a video produces the same illusion of mastery as reading a generated answer. With no task behind it, it is worth nothing.

Time spent together

What AI makes obsolete is the transmission of information. Contact time now goes to feedback and judgement.

Research said as much before LLMs. The meta-analysis by Freeman and his colleagues, over 225 studies in science and engineering, finds a gain of about half a standard deviation on exams for active learning, and a failure rate falling from 34% to 22% (Freeman et al., 2014). In computing, peer instruction, where you vote, discuss with your neighbour, then vote again, roughly halves the failure rate across four courses followed for ten years (Porter, Bailey Lee and Simon, 2013).

One serious caveat. "Active" does not mean "work it out yourselves". For novices, explicit teaching and worked examples do better than discovery (Kirschner, Sweller and Clark, 2006). That paper is contested, and it reads as a strong position rather than a consensus, but its core holds: guidance helps those who do not yet know, and gets in the way of those who already do. Hence the format I am adopting: short inputs of five to seven minutes, live on code, followed by a task at three depths of the student's choosing. Everyone moves at the same hour, not at the same depth.

And a warning I will give my students in the very first session: in a randomised experiment at Harvard, students in active learning learnt more but felt they were learning less (Deslauriers et al., 2019). The comfort of the lecture hall is an illusion of mastery, like the generated answer.

Reading before writing

The blank page is the terrain where LLMs are at their best. Existing code, with its constraints, its bugs and a change request, forces you to understand before you act. It is also the daily reality of the job: we maintain far more code than we write.

Computing education research has pointed this way for a long time. Being able to trace and explain code predicts the ability to write it (Lopez et al., 2008; Xie et al., 2019), and approaches that have students predict, run, investigate and modify before they create give better results in a mixed-ability class (Sentance, Waite and Kallia, 2019). This work often deals with beginners younger than my students, and the earlier studies are correlational; they converge all the same.

So my exercises will start from a bug, a code review or a change request, never from "create a class".

What remains to be shown

I am not going to claim that everything I have just described is validated. Two of these choices have, as far as I know, no published study behind them: weighting the mark of an AI-assisted project by that of an individual AI-free test, and having each student modify their own repository, offline, under time pressure. These are hypotheses consistent with what precedes, not results. I am putting them in place, I am collecting the data, and I will say what it shows, including if it proves me wrong.

What is established fits in a few lines. AI widens the gap between those who have the basics and those who do not. So both have to be taught, the basics without the tool, then the tool to go beyond them, and the first has to be certified before the second is marked. The submission no longer proves anything; explaining and modifying still prove something. Contact time is worth more when it goes to feedback and judgement than to transmission. And the feeling of learning is a poor indicator, for the student as for the teacher.

I still have January's exam paper to write. It will fit on two pages of code I did not write, and the first question will be: what is wrong here?

References