Recently the New York Times published an article titled Is A.I. the Future of Test Prep? The article discusses a recent startup that promises to interrupt the test prep space. The startup, by harnessing similar algorithms to those found in ChatGPT, intends to finally create an effective test prep platform that will eventually expand into other spaces. Currently, their technology has impressive capabilities: it can predict SAT scores quickly and, along the way, identify weak areas which it can then target with practice questions. The technology appears to be better suited for the SAT Math.
The idea is not new. There are already platforms which attempt to perform a similar function —identify weakness and assign practice material— but these platforms only identify broad deficit areas. The hope is that the new test prep AI will be able to replicate an ability only skilled tutors possess: the ability to specific knowledge or skills gaps. For example, does a student understand that a rate (e.g., miles per hour, dollars per foot, etc) can be understood as a slope? That’s a question that current technologies have difficulty answering. Without granular insights, these current platforms simply generate a variety of questions in the hopes that one or two of its sample questions will target this specific gap.
The hope is that test prep AIs can bring much needed precision to this process. The process is conceptually straightforward. The would first be to train the AI to understand the steps required to solve questions. Then when a student answers a question incorrectly, the AI can assign slightly different questions that require variations of these steps. The AI could then use a process known as subtraction to deduce the exact misconception or knowledge gap that is holding back the student. In contrast to a clumsy algorithm which might flag “Linear Equations,” an intelligent AI would report a deficit in “Understanding the Connection Between Slopes and Rates.” This is actionable diagnostic information that can help tutors teach more efficiently. Right now, so much of our work involves identifying these knowledge gaps rather than teaching them.
But what about the clickbaity subtext of the article, that AIs will one day be as or more effective than tutors? It’s unclear from the article how the AI plans to actually teach content. The startup seems to be developing a “coaching” product, not a teaching product. What makes one-on-one tutoring so effective is that tutors can, through their interactions, trigger something called mesolimbic activation. We’re all familiar with this phenomenon. Think back to that one lecture in college that you will never forget, —the one that captured your imagination. The neurological process that caused that lecture to stick with you is called mesolimbic activation. By stirring their students’ interest and imagination, great tutors are able to trigger mesolimbic activation, even if the topic is not particularly interesting. We at Ivy Tutor call this “the gift.” At some point tutors may get replaced by AI, but we think that will require either life-like robots or, more plausibly, a virtual reality environment.
