This is all so crazy to me.

I went to school long before LLMs were even a Google Engineer's brianfart for the transformer paper and the way I took exams was already AI proof.

Everything hand written in pen in a proctored gymnasium. No open books. No computers or smart phones, especially ones connected to the internet. Just a department sanctioned calculator for math classes.

I wrote assembly and C++ code by hand, and it was expected to compile. No, I never got a chance to try to compile it myself before submitting it for grading. I had three hours to do the exam. Full stop. If there was a whiff of cheating, you were expelled. Do not pass go. Do not collect $200.

Cohorts for programs with a thousand initial students had less than 10 graduates. This was the norm.

You were expected to learn the gd material. The university thanks you for your donation.

I feel like i'm taking crazy pills when I read things about trying to "adapt" to AI. We already had the solution.

I basically agree with the thrust of what you're saying, but also:

> I wrote assembly and C++ code by hand, and it was expected to compile. No, I never got a chance to try to compile it myself before submitting it for grading.

Do you, like, really think this is the best way to assess someone's ability? Can't we find a place between the two extremes?

Personally, I'd go with a school-provided computer with a development environment and access to documentation. No LLMs, except maybe (but probably not) for very high-level courses.

> Cohorts for programs with a thousand initial students had less than 10 graduates. This was the norm.

And why is this a flex exactly? Almost sounds like fraud. Get sold on how you'll be taught well and become successful. Pay. Then be sent through an experience that filters so severely, only 1% of people pass. Receive 100% of the blame when you inevitably fail. Repeat for the other 990 students. The "university thanks you for your donation" slogan doesn't sound too hot all of a sudden.

It's like some malicious compliance take on both teaching and studying. Which shouldn't even be surprising, considering the circumstances of the professors e.g. where I studied, as well as the students'.

Mind you, I was (for some classes) tested the same way. People still cheated, and grading stringency varied. People still also forgot everything shortly after wrapping up their finals on the given subjects and moved on. People also memorized questions and compiled a solutions book, and then handed them down to next year's class. Because this method does jack against that on its own. You still need to keep crafting novel questions, vary them more than just by swapping key values, etc.

I've had colleagues argue (prior to LLMs) that oral exams are superior to paper exams, for diagnosing understanding. I don't know how to validate that statement, but if the assumption is true than there is merit to finding a way to scale them. Not saying this is it, but I wouldn't say that it's fair to just dismiss oral exams entirely.

TFA's case involved examinations about the student's submitted project work. It's not the same thing. Even for a more traditional examination with no such context attached one might still want to rely on AI for grading. (Yeah, I know, that comes across as "the students are not allowed to use AI for cheating, but the profs are!".)

Also, IMO oral examinations are quite powerful for detecting who is prepared and who isn't. On the down side they also help the extroverts and the confident, and you have to be careful about preventing a bias towards those.

Admitting 1000 students to get 10 graduates means there are morons in admissions doing zero vetting to make sure the students are qualified.

I currently go to school for engineering, and it is the same way.