I am teaching some math courses, and I see how LLMs disrupted all standard approaches, and I don't know what to do.

I rely on written exams that are open book, but forbid any use of computers and smartphones in class.

The university insists that homework can't be optional, but it lost its meaning. I've tried to explain to my students that it's in their interest to think about home assignments on their own, but the majority will obviously use gen AI, and it's such a waste of time to check and grade LLM output.

I just had an experience where all students were given "sample problems" to try at home and prepare for the written test. Many of them just dumped the document into an LLM, asked it to produce some "exam guide", and showed up with that thing printed out, asking me during the test to explain what LLM output meant.

It seems like the university also has many people pushing for AI use for everything, but I teach basic stuff where the goal is to make students think on their own and digest some fundamental ideas, LLMs can produce perfect solutions, but relying on them is pointless.

I'm taking a year-long sabbatical to study a language at university right now and they're changing their assignment processes to curtail AI use. We have two versions of written assignments - the first to be written by hand in class with just a Swedish-Swedish word book available, and the second version to be typed at home implementing the professor's corrections from v1.

At least in this course students seem to be taking their strict 'no AI' policy quite seriously. Then again this is also the kind of course you do when you want to actually learn the material. It provides a qualification for further higher education in Swedish, so if we don't legitimately learn the stuff there's just no point as we won't manage in future courses.

There is indeed no point in grading home assignments anymore, but why not randomly ask a student at the start of each class to present their solution on the board?

And do a final exam on oen and paper and nothing else, and an exam that relies more on thinking than rote memorization of formulas (or print the relevant parts of the course on the exam document itself).

> Many of them just dumped the document into an LLM, asked it to produce some "exam guide", and showed up with that thing printed out, asking me during the test to explain what LLM output meant.

That student presumably performed poorly on the test. Your approach is working as intended here, isn't it?

Also, is the AI-generated exam guide all that different to a human-written guide they could have downloaded a decade ago?

That sounds pretty frustrating. I wonder if it wouldn't be a good approach to, say:

1. Make the exam the only thing that determines the grade (or technically, proves that the student learned enough);

2. Give out test assignments students can do, but don't have to;

3. Offer students for the professor to grade their work if they want to get some feedback on how they're doing.

Would that lead to less time wasted by tutors grading AI generated answers? From where I stand, students should be allowed to prepare for exams any way they like, with or without their professor's help. If they managed to learn, they pass.

Put a mixture of questions in that are impossible to answer but don't tell them, and as usual give them marks for explaining their working. Tell them you've devised the questions so some are "AI resistant" but don't tell them how. They'll never know if the AI has tripped up and hallucinated or given them the right answer. Only those that really understand what's going on will set out some working that shows they were on the right route and then got stuck where they were supposed to.

Just don't make it open note then? Or you provide the only notes or cheat sheet available to them at the start of each test. That's what some math tests I've had had done before and I believe the SAT also does the same thing with formulas in the back of the test to reference.

> such a waste of time to check and grade LLM output

Small nitpick: you teach math, why don't you use an automated grader?

I agree homework should be optional and your college is wrong. Even without AI, for some students it's easy to cheat and maybe for some students it's a waste of time. Maybe you can assign it a very low score, like 5% of the final grade, and/or award the grade based on attempting rather than getting the right answer.

> I've tried to explain to my students that it's in their interest

Perhaps some people do not belong at university? It is just a waste of everyone time, to insist 50% of population studies until 25 years old.

I've always wanted to find out if this would work:

If I knew most of my students were using LLMs I would encourage the rest of them to use them too. I would then change the marking criteria such that if they get a single point wrong that's 20% off the entire grade. 5 mistakes gets you a big fat zero.

LLMs are great but they make mistakes. At this point your students would have had to spend so long checking, re-checking and triple checking the LLM's output that they will have accidentally learn what they need to. They will likely need to cross-reference multiple LLMs and at least have read their output which is likely an upgrade on today.

Or some variation of the above, I'd be interested to hear your thoughts.