Veritasium: What Everyone Gets Wrong About AI and Learning

Thank you. Thank you everyone for coming tonight. It is my absolute pleasure to be here to talk about how will AI change education. I must admit, maybe first off, that I wanted to sort of shoehorn AI into this talk a little bit, because what I'm really passionate about is education. So my question is really how is AI going to play a role in that? But first we have to understand the bigger picture of how education actually works, or doesn't work in some cases. You may have seen this clip. This is an AI tutor from ten months ago, so let's take a look. "Can you first identify which sides of the triangle are the opposite, adjacent, and hypotenuse relative to angle alpha?" "All right. So, I'm pretty sure this is the angle alpha right here, right?" "That's correct. Now, looking at the triangle, which side do you think is the hypotenuse?" "Um... I'm not totally sure. I think it might be this one, but I really am not sure. This side, AC?" "You're close. Actually, side AC is called the adjacent side to the angle alpha. The hypotenuse is the longest side of a right triangle and is directly opposite the right angle. Can you find which one is the hypotenuse?" "Oh, okay, I see. So, I think the hypotenuse is this really long side from A to B. Would that be correct?" "Exactly. Well done." Okay, so it's pretty impressive. And again, this was ten months ago, so AI tutors have only gotten better, I'm sure, in that time. I think if you had told me five years ago that such a technology would exist, I might not have believed you. AI, as we all know, has gone incredibly quickly. So the question is: is this the future? Is this how everyone's going to be learning in the future? Well, I want to reflect on two things. The first is that you can always complain about education. You can do it now, people were doing it 10 years, 100 years ago. You can always complain about the state of education, and I can kind of see why. I've got some examples of the failures of the education system for you right here. "What's water made of?" "Water?" "Yeah. What makes water?" "Um, water." "Okay, what elements does it take to make water?" "H2O." "So what does that mean?" "That it's water." Yes, H2O, it is water. Okay. This one seems like it's maybe going in a good direction, but then it takes a turn. See if you can spot where things go wrong. "So what's happening to the level of carbon dioxide in the atmosphere right now?" "Um, well, I guess, you know, we've got this whole climate change and, you know, and I guess people believe that obviously carbon dioxide is a—you know, impacts on climate change, you know? So, you know, carbon dioxide, everything emits carbon dioxide, you know, trees, human beings, um, I guess like animals..." "So the trees are contributing to the..." "Oh, definitely. Definitely. So, yeah. So, well, trees are, I guess, you know, one of the original primary sources of, emitters of CO2." It's the trees, guys! I don't know if you knew. Just cut them all down, we'll be okay. CO2-emitting trees. Yeah, it's not good. You can always complain about education. You can always say, you know, it's no good. So we can ask a question: why aren't people learning? You might point to certain things: maybe the education system is stale, maybe it was made to create factory workers, maybe it's because we didn't have AI tutors yet. I'm going to come back to this question, but I think it is an important one to think about: what is really going on? What are the outcomes we're getting from the school system and why? The other thing I wanted to point to, the second thing, is this word 'revolutionize' when it comes to education. Here we have a book by Sal Khan—you saw him in the first video—'Brave New Words: How AI Will Revolutionize Education'. But he's not the only one saying this: 'Four Ways AI Could Revolutionize Education Technology' or 'How AI is Revolutionizing the World of Education'. So clearly the expectation of a revolution is here. But the truth is, people expecting a revolution in education have been around for at least 100 years. We have Thomas Edison, this is in 1922: 'The motion picture is destined to revolutionize our educational system and that in a few years, it will supplant largely, if not entirely, the use of textbooks.' You remember how we got rid of textbooks 100 years ago? He said something like education from a textbook is 2% efficient, whereas education from a motion picture is 98% efficient. I have no idea where he came up with those numbers, I don't think anyone does. But you can imagine that in his time, the motion picture would have seemed like magic, in the same way that first clip I showed you of an AI agent tutoring a kid and doing it somewhat competently, that also seems like magic. So you can see why this argument for revolution is kind of convincing. In the 1930s, you had people saying radio would revolutionize education. Why? Because you could get rid of all the teachers, you could just have babysitters in the classroom, and then you'd have the expert radio in, and they could radio into a thousand classrooms. So immediately you have this economy of scale. And people see the same thing today: Elon Musk says AI education will be like having Einstein as a teacher for every child. The same idea: we're going to scale up the best and the brightest, we're going to have this most intelligent entity teaching everyone instead of what we have today. In the 1950s, it was TV. Academics actually conducted studies where they would have a lecture going on in one room like this one, then they'd have it videoed and closed-circuit TV in the lecture room next door, and they would test the students to see who learned more or less. The answer: no significant difference. No big surprise, because they're getting basically the same experience, assuming the TV system works reasonably well. Then in the 1980s, people at MIT thought that they had the revolution in education, because the thing we had been missing was interactivity. And now you could actually interact with the computer, you could go back and forth with it, you could program. So there was this idea that if we taught kids how to program—say, how to program a turtle—then they would get better reasoning skills overall. What happened? The kids got proficient at programming the turtle, but those skills did not transfer to other sorts of reasoning. We didn't get better thinkers, and that was the concept. That was why computers and interactive computers and this whole turtle programming was meant to revolutionize education once more. In the 1990s, you get this: 'The use of videodiscs in classroom instruction is increasing every year and promises to revolutionize what will happen in the classroom of tomorrow.' Many of you here may not know what a videodisc is. It was a huge CD-like thing, like a DVD. I used to use these when I was in high school, so they were there. Did they revolutionize education? No. There's one more I'll talk about: MOOCs, Massive Open Online Courses, about 13 years ago. This was all the rage: 'The Big Idea That Can Revolutionize Higher Education: MOOC', 'Will MOOCs Revolutionize Higher Education?', 'Revolution Hits the Universities', 'The Phenomenon of MOOCs: Revolutionizing Education in the Digital Age', 'The Online Education Revolution Drifts Off Course', 'The MOOC Revolution May Not Be as Disruptive as Some Had Imagined', 'Why MOOCs Won't Revolutionize Higher Ed', 'What if MOOCs Revolutionize Education After All?' A story in ten headlines. Anyway, I hope you take my point: people are all too excited, all too ready to put the word 'revolutionize' next to education. This is old school, new school—it's the same school. What is going on? We shouldn't be using this word 'revolutionize'. You keep using that word, I do not think it means what you think it means. You know it's a good talk if there's a Princess Bride reference. So why didn't these revolutions materialize? You could say the inertia of educational institutions—they have a lot of inertia, they're not willing to change, we're all just set in our ways. You could say it was just technological hype. Or you could say we didn't have the capabilities of AI yet, this time is different. Or maybe it's something else? So I'm going to explore this concept a little bit. I think one hint comes from this clip, a question that I asked to some people in Los Angeles, so let's take a look. "You go into a toy store, and there's a toy bat and a toy ball. Together they cost $1.10. And the bat costs a dollar more than the ball. How much does the ball cost?" "10 cents." "10 cents." "We're all wrong, aren't we?" "What's the answer?" Think about what is the answer. What is the answer? 5 cents, yes, very good. Now, I don't think these people are silly, I don't think they're stupid. I think they could work it out. If the ball was 10 cents and the bat was a dollar more than the ball, it would be $1.10, and together they would be $1.20, right? So the correct answer, 5 cents. So why did they all say 10 cents? It's because that number came to their head as they listened to the question. They don't know where that number came from, but it was in their head, and it sounded right, so they blurted it out, all at the same time. Really quite amazing, right? Something I love about that clip. So why do they do that? Well, the insights are in this book, which is one of my favorite books of all time, 'Thinking, Fast and Slow' by Daniel Kahneman. In the book, he's talking about our two systems of thought, that we've got two kinds of processes going on in our brains at one time: the fast processes, System 1, and slow processes, System 2. At the start of one of his chapters, he imagines: what if there was a Hollywood movie where these systems are turned into characters? So I decided to take that inspiration and run with it. I created these two characters inside my head. System 1 is on the left, and System 2 is on the right. Now, System 2 is kind of the person who you think you are. That's the voice in your head. It's a slow, effortful system. Thinking takes some effort. If you ask him to multiply 13 times 17, he's going to say, 'I don't want to do it.' But you could force him to do it, and he can go through a series of steps. That's the thing about System 2: it's slow, it's methodical, it can catch mistakes, it can work through processes, it can think about thinking. Now, System 1, on the other hand, is rapid fire. This guy is doing things fast, in the background. You have no idea that he's doing it. He's collecting all the information from your senses, he's pulling out the relevant information, and he is getting rid of everything else. You're just getting bombarded with stimuli, so he's pulling out those key pieces of information that you're going to work with, and he does all of this without you, the System 2 thinking, conscious thing, really being aware that he's doing it. And you notice also that he was in a library setting, so he's sort of associated with all of your long-term memory. He's got all of that, and that's what allows him to work so quickly and so effectively. When those people said 10 cents to me, he served up the answer 10 cents, and System 2 was just like, 'Yeah, 10 cents. I'm not going to think about it, I'm not going to check that answer.' He could check the answer, but he doesn't want to because he's lazy, and fair enough. The goal for us should not be always to use System 2, but it should be for us to know when we need System 2 versus when System 1 will be fine. Hopefully you want to optimize: allow System 1 to handle everything that they can handle and only hand off to System 2 things that you need to. Now in my travels, I came across these two characters. Something that I thought about later was that they almost embody System 1 and System 2 together. So the question that I asked them was: how long does it take for the Earth to go around the Sun? Okay, so: "How long does it take for the Earth to go around the sun? What do you reckon, cuz?" "Isn't it 24 hours?" "Obviously a day, yes." Okay. I asked the question to System 2. He was like, 'I don't want to—what do you think?' System 1 over here, 'What do you think?' He's like, 'Obviously a day. Sounds good. Sun comes up, sun goes down, yeah.' And then what did he do? He just spat it out. He didn't stop to think about it, he was like, 'Obviously, yeah, it's a day.' What was amazing about this conversation was as this conversation moved on, and I had a whole set of questions that I asked them, I think at this point I was asking them: did dinosaurs live at the same time as humans, or something silly like that. That's what I was doing, and then I noticed a pause. And I was looking at his face, and you could see that he was thinking. You could see that System 2 had snapped into action. So I just didn't interrupt, okay? Watch what happens. "Hey, the Earth doesn't take one day to get around the Sun. It takes like a year!" You see the shock on his face? The whole time he was just running System 1, and then at the very end he was like, 'Whoa, hang on. Wait a minute.' All right. The thing about System 2 is that it is very limited. This paper came out in '55, it's a classic: 'The Magical Number Seven, Plus or Minus Two: Some Limits on Our Capacity for Processing Information' by George A. Miller. The idea here was really that you can only handle seven new things in your working memory that System 2 can deal with at one time, plus or minus two. It turns out, over the years, that estimate of seven has been revised down to about four. And there's a test you can do to assess this and figure out what is my span, how many pieces of information can I just work with at one time. So the way this test goes is I show you some numbers that you can look at briefly, and then I take them away, and what you try to do is say each number on the beat, but you add one to it. Increment each digit by one after the numbers have disappeared. I'll do it for you so you guys don't feel weird about this. I just wanted to demonstrate how we can show that this span is quite limited. Here we go. There's some numbers: 5, 4, 9, 1. Six, five, zero, two. Right? Okay, that wasn't too hard. You can make the task harder by instead of adding one, you add three, and instead of having four numbers you can go to five or to six to really push yourself and see what happens. What's amazing about this task, what's interesting about it, is that while you're doing it, while you're really activating System 2, there's physiological responses that take place. Your heart beats a bit faster, your skin gets a little sweatier, and interestingly, your pupils dilate. When you're really thinking hard, when you're using that System 2 thinking, your pupils are actually expanding just a little bit. So I got a super macro lens and some friends, and I showed them numbers and then got them to do the add-one task. Try to see if you can spot the pupil growing of this eye while he's doing the task. Here we go: "Four, three, nine, seven, two. Five, four, zero, eight, three." Right there in the middle, when he's in the middle of the numbers and he's still got to remember the remaining numbers, it just expands, and then at the end it sort of starts collapsing back: 'Okay, now I can relax a little bit again.' It's interesting to see just how effortful this is. This measure is known as cognitive load, or another way of thinking of how much mental effort you are investing in something. We can break down how the cognitive load is being expended into three categories. There's intrinsic cognitive load, which is just the amount of cognitive load you need for a particular task. If I'm giving you six numbers, it's going to require a large intrinsic cognitive load. Or sometimes when we're doing something in physics, even something simple like F = ma, for a new student each of those things is new and sophisticated, a complex concept, so just teaching that has a large intrinsic cognitive load. There's also things that are totally extraneous: someone chewing next to you, your seat's uncomfortable, distractions taking away from your ability to focus. Extraneous cognitive load is not great. And then finally we have germane cognitive load, which is: can you actually use your attention, use System 2, to do some thinking about thinking, observing what you're going through, noticing patterns that might be helpful for you to use later? Germane load is great if you can get it. Now, this is another really classic study. One question you might have is: if we have such a limited System 2, how do we accomplish anything? If you go back several decades, people were wondering: what makes a chess master a chess master? Do they have really high IQ? Do they have great spatial reasoning? Do they have a really large working memory span? What makes them unique? These researchers got together and did a study where they showed some different level chess players—from beginner, intermediate, to master—a chessboard like this that was stopped in the middle of a game. They showed them this chessboard for five seconds. They had it in front of them, and then they covered up the chessboard, and they had another blank chessboard beside it. They said to these chess players, 'Now put the pieces on the board exactly as they were in that chessboard you just observed.' When you ask novices to do this after five seconds, they will get about four pieces correct after looking at the board just once. The grandmaster could do 16 pieces after one glance, after just five seconds of looking at it. Why were the grandmasters so good at remembering where all those pieces were on the board? That's a lot. It's because of a phenomenon called chunking, where you can see things that look like disparate bits of information, separate pieces of information, but you can actually see them as one thing. In the case of the example I gave you earlier, if you reverse the order of these numbers, you get 1945. Which is no longer four numbers, it's one thing: the year that the Second World War ended, right? And so that makes it much easier to remember. And this is true for basically everything we come into contact with. We're not normally dealing with it in its most basic chunk. Words: you don't have to parse out every letter; phrases have meaning, you think of them as one thing; or equations, like F = ma or the Schrödinger equation. For a lot of people in the room, that's just a single entity in working memory. The point is that the more we get experience with things, the more we practice, the more we interact, the more we use System 2 to work through problems, the more we develop this long-term memory. And that long-term memory allows us to chunk the things in our world, and that allows us to deal with much more complex situations. So that's exactly what's happening with a chess master. I have heard a lot of people say this: 'That class really taught me how to think,' or 'that teacher taught me how to think.' And I think that's more of a question than it might seem at first. At first it might seem reasonable: 'After that, I really learned how to think, I learned how to reason.' But if you go back to this chessboard study, the next thing they did was they rearranged the pieces—same number of pieces as before, but now they positioned them as though they would never occur in a regular chess match. They just put the pieces all over the place at random. You would never ever see this in a regular chess match. And they tested again the novices and the masters. And now, the masters did no better than the novices. The thing that made them so good at remembering where the pieces should go was that they had seen a lot of chessboards, they'd seen a lot of configurations just like that. And if the pattern wasn't there, then they were no better off than the novices. My argument here is that there is no general thinking skill. There is no general problem-solving skill. What there is are these complex webs of long-term memory. This is what we build up over our lifetimes. So a physicist is not necessarily going to be a good chess player, and a chess player is not going to be a good physicist. This applies to all domains. The idea is an expert in one field is not an expert in another, and it's just because of this: that long-term memory that your System 1 is working with is really specialized to reflect the experiences you've had, to allow you to chunk new situations when they come to you. It's so powerful when your System 1 is this really complex web that you don't even feel like you're thinking. In the same way that those people said 10 cents to me, if you ask Magnus Carlsen, 'What is he doing when he plays chess?' "Most of the time, I know what to do. I don't have to figure it out." Right? He's saying, 'I don't need System 2. System 1 is so well developed that it can solve any problem at a glance.' For him, chess is a game of recognition. He recognizes the board in the same way that we recognize faces, or physicists recognize a physics problem. That's the way we get to be really good at things. So really what we want to be doing in education is using System 2's resources very carefully and repeatedly, such that we store information in long-term memory to allow our System 1 to do things that effectively feel automatic. So what are the implications of this for education? The first one is that we should eliminate extraneous cognitive load, which is fairly obvious: comfortable seat, see the board, sound from the microphone pristine, no confusing accent. Subtitles sometimes help. Point number two: limit intrinsic cognitive load. We don't want to overfill what you can handle. My guess for you is that the reason why a lot of physics lectures fail is because the professor doesn't limit intrinsic cognitive load enough. They go through too much novel material in the one lesson, and it's just too much for anyone to handle in terms of their working memory. When you're teaching, start where your students are, keep the work bite-sized, because as soon as you start introducing four, five, six novel concepts in one lesson, you're going to lose people. Another way to limit intrinsic cognitive load, when it comes to music, is get people to play songs that they already know. When they already know what the rhythm should be, they can focus on the technique. Another way is to slow things down. Anyone who's practicing music—like this harpist, who's a friend of mine—describes her practice like this: "You can practice everything exactly as it is and exactly as it's written, but at just such a speed that you have to think about and know exactly where you are and what your fingers are doing and what it feels like." She's really talking about using System 2 there very slowly, deliberately, effortfully thinking through every little thing that she's doing. And when she does that enough times, enough repetitions, you get effortless, beautiful mastery. Every time I see superhuman performance, the thing that I'm thinking about is how they have used System 2 very slowly and effortfully to build up that structure in long-term memory that allows them to do things that look so amazing and effortless. It's also at this point that I bring up this idea that discovery learning can be dangerous. When I was going through school, the dominant paradigm in education was something called constructivism. The basic tenet of constructivism is that students are active constructors of their own knowledge, something I do not disagree with. System 2 is about being active and effortful in how we engage with material, how we practice. The problem is, some people didn't know how to implement teaching for this constructivist paradigm. Their thinking was: if the students need to be active, then telling won't work. But that's not true! I'm telling you stuff right now, but I bet you're still being active and effortful in your brains, which is the place where you need to be active and effortful. It led to some classrooms where people really pulled the scaffolding away from students too early. We want to teach problem solving? 'Here's a problem, solve it. You don't know how to do that? Figure it out. Construct!' If you go to a new city and you don't use a GPS, it's really hard to figure out where you have to go. But if you do it, and you use your System 2 to figure out and follow every street, at the end of that you'll probably be able to retrace your steps. If you use the GPS, it'll be much easier for you, but can you find your way back? No, you're going to need the GPS. I think there has to be some balance in education between the GPS guidance and internal guidance. Ideally, I would love to see a gradual phasing out of the support to allow the person to figure out the directions on their own. This is the worked example effect: first giving someone a problem and showing how you solve it, then a problem partially done, and then one for you to do from start to finish. I'm going to give you this scaffolding, because I know that your working memory is very limited. Point three: repeat effortful practice until mastery. Why is mastery so important? Because when you show a skill with mastery, that means it's now a System 1 domain. Times tables: mastered, in System 1, you don't have to think about it. When you come to do other problems that require that information, you're not going to overload System 2, because all that stuff is automatic. That's the key: building up more and more capabilities that are automatic. If you never get to the level of mastery, moving on is going to always cause you problems. Point four: increase germane cognitive load. How do you get people to engage in really active thinking? One thing that was tried with the bat-and-ball problem from the Cognitive Reflection Test: typically about 90% of people get at least one question wrong. What some researchers tried was, instead of printing out this test in clear font on nice white paper, they printed it out in a terrible, very hard-to-read font, photocopied it a few times so it gets really difficult to read. And what happens? The error rate drops to 35%! Making the test harder to read made students more likely to answer it correctly. Why? Because they didn't have that easy, 'Oh, I can read it and an answer immediately comes to mind.' Instead, they had, 'Oh, this is challenging to read,' which kicked their System 2 into action, and it found the correct answer. I'm not suggesting this is a good strategy—making things more confusing—but it can be effective. You see advertisers do this: 'The power of Un'—unexplained, televisun, un-cost the earth, un-pay more, un-spend more. It confused people, made them engage System 2 to figure out what it meant, and then revealed it was an insurance ad. So what role is AI going to play? The positive role that I see for AI is that it can provide timely feedback. That's essential. When you're learning any skill, you're going to go through repeated rounds of practice, and you need immediate feedback in order to learn. Harp, tennis—you know right away. But the thing that I'm really worried about is how AI has this opportunity to reduce effortful practice. I have four kids who are 8, 6, 4, and 0. And I worry about them. Will they write an essay? Will they write a hundred essays? If there is a generative AI that can write for them, what forces them to practice crafting those sentences? And if they don't craft those sentences, what happens to their brains? You get good at your command of language, at speaking in front of people, at expressing your thoughts in writing by doing it again and again and again, and you should suck at the beginning! And you shouldn't let that stop you. If they never do that, what gets into System 1? What is that vast network of connected knowledge that they can draw on? I fear that they won't have it. The world is full of heavy objects, and yet most people are not ripped. The world is full of fields; not many people running on them. There's plenty of ways to get exercise, but obesity is a huge problem. I think about teachers a little bit like personal trainers. The gyms are there, but unless there's someone who you're going to, who holds you accountable, someone to say, 'Another rep, keep going, it's burning, keep going,' someone to hold you accountable and energize you, that's when you see results. This doesn't just happen in a vacuum. Tech hype comes from a place of believing that the problem of education is not being able to get the information to the student. That's not the problem! The information is all there in books. You're not going to learn very much unless you have a great teacher, unless you have a group of like-minded peers to go through that with you, unless you have a reason to do it. Education is a social activity. People care about other people. The goal should always be to get more teachers and make them better. That's why none of these technologies are ever going to revolutionize education. Thank you. Host: Thank you, Derek. That was a fantastic talk. Now we have some time for questions. Audience Member 1: What do you think about using AI to answer a question when you're working on a problem, as opposed to looking up the answers yourself in books or whatever? Derek: As long as the AI is sound, I don't think there's a difference really. You may as well just talk to the AI. It's strictly a question of efficiency, and at that point, you can be more efficient, that's fine. Host: We have a question from online: Do you have any suggestions for how to get these insights about education enacted into policy? Should that come from teachers, government, or the public? Derek: Policy is very challenging. In Australia recently, they've implemented what they call an explicit instruction policy, which is kind of what I'm talking about—that telling is not wrong. Education is a difficult field to research because of so many variables and researcher bias. But research pointing toward building long-term memory, effortful practice, pushing people beyond comfort zones—hopefully policy follows evidence. Audience Member 2: People who chase art already do it without a promise of a reward. Do you think our fear should be that people start chasing art without understanding origin, or will people still create when AI can do it for them? Derek: That's great and encouraging that people make art for the sake of art. For my kids, I'm worried about whether they will learn how to write. Look at what happened to handwriting—we type, so handwriting got worse. What if we become not great at writing at all? Experts need that vast structure to have great insights. Young scientists often make breakthroughs because they build fresh networks without the dead ends of older paradigms. If a machine does the work for you, we lose those insights. Audience Member 3: With AI, we are using System 1 more often and keeping System 2 on rest. Curiosity and learning go hand in hand. If you had AI at the age of 20, do you think Veritasium would exist? Derek: That is a crazy counterfactual. I don't know! Hopefully we're still encouraged to follow the things that intrigue us. In some ways, AI provides speed. The real risky use case is when people use it to do the work for them. That's the one we've got to get rid of. Audience Member 4: How would you solve the problem of scaling? Personal connection works for 10 people, but scaling to 100 is harder. How would you approach that? Derek: How do you scale a personal trainer? How do you scale a plumber or an electrician? You don't. You just have lots of them. And I think that's the answer. The goal should be to get more teachers and make them better. Audience Member 5: Within this context, where do you think the place of AI would be in learning line-by-line equations? Derek: AI can offer lots of scaffolding—giving hints, filling gaps. What a great tool if used effectively. The place where it won't be great is where it allows you to do the work without doing the work. Audience Member 6: With chatbots making it so easy to get assignments done without work, do you think it would be better to outright remove it from early stages of education? Derek: Absolutely, there's going to have to be some of that. Like graphing calculators: we had calculator and non-calculator sections on exams. With writing, there will be writing with AI aids, and where it has to be no AI, having in-class writing sessions. Audience Member 7: How do you think we can effectively get people excited about 'breaking their brains' on their own when they don't understand something? Derek: That is the existential question! Most people don't know the answers to simple questions like what water is made of. We're not evolved as a species to seek out theoretical physics. Only a tiny minority do. We have to have compassion; people are into what they're into. Connecting socially is what humans evolved for. Audience Member 8: Is there a sweet spot where you get content overload when you make longer 30-minute videos? Derek: YouTube algorithmically pushed us toward longer videos. In the early days, videos were 2-3 minutes; now they're 30. If we make a good 30-minute video, it gets shown more. Sometimes when we put in integrals and derivatives at minute 20, we may lose people, but it's there for those who want to go deep. Viewers can pause, rewatch, and digest. Audience Member 9 (Child): What's your favorite video you've ever made in your life? Derek: Every video is like one of my children. The one that's done the best for us is on Gödel's Incompleteness Theorem. I also love the Shade Balls video, and the one on the most common cognitive bias (37 minutes long). Audience Member 10: Students use ChatGPT on assignments and then fail midterms. Should we change evaluation methods to be more exam-heavy? Derek: Maybe assignments need to be done under exam-like conditions. The practice has to happen in the lesson. In my 400-student lectures at UTS, I gave students cards (A/B, C/D) and asked conceptual questions every few minutes to keep System 2 engaged and prevent tuning out. Audience Member 11 (Child): If we didn't completely remove AI from education, what tools would we add and what tools would we remove? Derek: What an amazing question! Keep the drill-and-practice tools that give immediate feedback and encourage reps. Remove the tools where AI does the work for you, like writing the essay or doing the thinking. Audience Member 12: You say 'revolutionize' isn't the right word. If not revolutionize, what word would you use? Derek: A tool. Technology becomes a tool in the hands of educators. The internet transformed communication, but in the classroom, pedagogy and core learning remain largely unchanged. It's about human connection and effortful practice.