// HACKER NEWS — CYBERSECURITY
How I changed teaching after AI managed to do all my homework assignments
Around 2021, well before ChatGPT launched, Vincent Hellendoorn suggested I try GPT-3 on the reading quizzes in my course. It produced convincing answers passing our rubric without actually seeing the assigned paper. At the time, I changed nothing. Five years later, AI agents could do all my assignments and I have redesigned most assessments in that course, even though what I want students to learn has barely changed. The strategy is always the same: No longer test understanding with anything that is done at home and instead focus on interactions with a TA, on exam, and on a video demo. Some of these changes violate evidence-based best pedagogy practices, and I made them anyway.
For the last couple of years, I have mostly taught the course Machine Learning in Production, an upper-level course on building production-ready software around ML models with a heavy focus on MLOps, usually with 100 to 170 students. These days one common question when talking to other educators is how we have changed teaching in the age of generative AI and coding agents, so let me outline what we did.
We have shifted covered topics with changing AI innovations and tools, but I barely touched the overall learning goals. I am fortunate that this is not an intro course and that the learning goals are not about writing code or using specific tools; they are about engineering tradeoffs, anticipating and mitigating risks, and teamwork. I think these are skills still worth acquiring, even if some can be simulated and offloaded to a model. (Revising an intro course or a traditional software engineering course likely would shift learning goals much more.)
Also possibly important: We give students permission to use AI in all settings, in any form, without attribution, except for written and oral exams. We even encourage the use of AI tools in many places. I do not think policing AI is feasible even if we wanted, and more importantly I do think that students need to learn responsible use of these technologies anyway.
Let’s start with this point upfront, since it is more important than what we actually do: Unfortunately, AI is actively undermining several evidence-based teaching practices (e.g., see How Learning Works and The ABCs of How We Learn). For example, the evidence favors frequent low-stakes assessments with feedback (e.g., homework, quizzes) over few high-stakes ones (e.g., exams) – but AI is undermining practice in low-stakes settings and pushing us more toward exams.
Similarly, I always provided a safety net where students can make mistakes and resubmit a limited number of assignments to regain lost points (a core recommendation of specifications grading and grading for equity to focus on learning outcomes, not the process), but we felt that this process was abused with AI: first submit a generated assignment solution without thinking and only look at the issues raised in grading for a resubmission (the typical story of externalizing the cost of AI use). In response, we have since taxed resubmissions with a 10% penalty.
Also in-class interactions allow engaging with materials in an early low-stakes setting, but with AI I have seen many student groups offload the discussion questions to a model. Pen-and-paper submissions could fix this, but aside from a higher grading workload, it would also raise stress for students, take away from the low-stakes environment, and delay feedback.
In general, this is a balancing act and I tend to err on the side of keeping low-stakes repeated interactions even though it can be abused. Yes, some students will get through the class without much deep learning, but it provides a better environment for those students who want to learn. I don’t want to get back to the model I’ve experienced during my own studies in Germany with mostly optional homework and a single exam at the end of the semester that was responsible for 100% of the grade in the course. This was nice for students who were self-motivated and good at lea