Remix.run Logo
datahack 4 hours ago

This is how higher education was done for literal centuries before moving to the written word. So it’s not exactly innovative to return to oral arguments.

But what it does threaten to do is effectively abandon all the efficiencies of the written word.

When higher education evolved into a mass system in the 1800s and into the 1900s, pure oral examination was looked at as super inefficient. A written paper could be handed in and graded without lining everyone up in front of a panel.

The oral system scales poorly. One examiner can only hear so many students in a day. There are damn good reasons Medieval universities were small and modern ones are not.

Requiring every major paper to be defended live reintroduces the bottleneck that written exams were designed to escape.

Accessibility takes a hit too. Students with speech differences, anxiety disorders, hearing impairments, or simply less fluency in the language of instruction face an extra barrier that a written submission does not impose. The written form, for all its flaws, leveled some of that ground. Shall we abandon the differently abled because machines disrupt the academic system and essentially make cheating effective? This does not seem very logical, or sensitive.

So while it’s nice that academics are rediscovering the old ways, I don’t really see this as a scalable answer to raising education levels for the masses, but more of a knee jerk way to try to bypass the obvious technological disruption by returning to non-scalable, inaccessible ways of doing things that our ancestors abandoned for a reason.

It has its place for graduate students and smaller volume education systems. It’s fine for keeping the ivory tower clean. But it’s not the answer to expanding education to everyone and taking advantage of the technological disruption to advance education and learning throughout greater humanity, which is what AI definitely possesses as a possibility.

bonoboTP 4 hours ago | parent | next [-]

Oral exams are still common in Germany and many European countries in case of smaller courses taken by 20-30 students in a semester, and sometimes even in larger courses, and the task of examining is then split among multiple professors and lecturers.

In Hungary it's a pipeline. You go in a room, you pick a piece of paper out of a hat, which gives you some task or topic to prepare for. Then you sit down at a desk (the room is filled with about 15 students working already), then you solve the problem or sketch out your notes to present the topic you picked for like 20 minutes, then you go to the professor and you have to, in low voice, explain your solution, the professor asks back and pokes at it to make sure you understand. The professor then gives a grade immediately and you go home.

In Germany, it's usually no random pick of a topic, you just go through the entire semester's material over 30-45 minutes and the professor stops to spend more or less time on each topic to explore your knowledge. You can explain things by making sketches on paper. A PhD student sits there quietly taking notes of what was asked and what the student answers approximately. You leave the room, the prof and the PhD student discuss the performance, decide the grade and you come back, and the professor explains the grade and what you did well and not well.

scarlehoff 3 hours ago | parent [-]

I'm not familiar with the German universiry system, but I think a 1 hour, 2 professors (or one professor and one PhD student) would be impossible to scale. A class of about 100 students could take an entire month! (or better, 2-months of professor-hours!)

fabian2k 3 hours ago | parent | next [-]

But that is how certain exams are done right now. Though it's usually 30 minutes including overhead. It's not something you can do in classes with multiple hundred students, but it's certainly done at scale.

I was the PhD student in the room in dozens of these exams, and I toke some of them myself. But that was for classes with a bit less than 40 to maybe 60-80 students. The load for the larger ones was shared among professors, if I remember correctly. But it was certainly the case that a single professor spent 1-2 days doing nothing else but these exams.

And it was 1 professor and 1 PhD students, and those don't count. They essentially grow on trees and are always available, if you need some.

hx8 3 hours ago | parent | prev | next [-]

> would be impossible to scale

I think education is something better scaled horizontally than vertically. More schools, with more professors instead increasing the student per professor throughput. The classes where I learned the absolute most, and had the best experience, were closer in size to 25 than 100.

B-Con 2 hours ago | parent [-]

The US has proved it can do horizontal scaling, as the majority of adults gets a college degree.

The horizontal capacity is ineffectively used by having many degree mills and low-value degrees, but if we re-tooled the knowledge sets we could raise the bar.

hx8 2 hours ago | parent [-]

I actually think of degree mills as being vertically scaled. Online courses, a lot of course work graded by computers, large classrooms, etc.

As for the quality, I think small classrooms with good teachers probably produces a better outcome than large classrooms with great teachers.

dfdydx 2 hours ago | parent | prev | next [-]

One big difference I observed between the US and $HOMECOUNTRY is that US colleges require a lot of courses with little credit, so a normal college year is N courses x K credits per courses, N large, K small. In $HOMECOUNTRY it is typically N small, K large - think instead of having Calc IV, Trig whatever, Algebra 2 and Introduction to Proofs, you have a single "Mathematics" course that last a whole year, not just a semester.

Since each course needs its eval, assessment is a function of N, not N*K or K. Much easier to scale if there is ~5 evals per student per year than ~20.

bonoboTP 3 hours ago | parent | prev | next [-]

At low student counts, like 20, (think specialist niche elective master's courses, not the huge bachelor's courses) it's perceived by profs as more efficient in fact than the ordeal of organizing a written exam.

With an oral exam, the prof blocks off 3 days or so, with ~8 students per day and it's done. It's not a huge additional effort compared to all the effort of giving the course over the full semester anyway.

An added, though minor benefit is that the professor might decide to offer a student assistant job to someone who performs well, because it's much easier to judge who is actually good in an oral exam vs in written. There's a reason that job interviews aren't just tests filled out on paper but involve personal interactions.

Almondsetat 3 hours ago | parent | prev [-]

What is the point of this reply? GP already specified oral exams are mostly reserved to courses with smaller numbers

fabian2k 3 hours ago | parent | prev | next [-]

Oral exams can also compensate a lot for different types of exam anxiety. Blanking on individual points doesn't matter as much, the examiner will help you along or ask along a different path. Doesn't help for someone that completely shuts down, but it's much easier to determine if a student was just stuck on some minor point and actually knows the topic in an oral exam.

They are likely prone to more biases than written exams, and the variability among examiners is likely high. My experience is both in taking these exams myself, and also sitting in on them as the note taker many times. Though that was only with one specific professor, and almost always on the same course. But that was in my experience a very fair exam, even for those students that were very noticeably nervous.

ninalanyon 35 minutes ago | parent | prev | next [-]

Even in fairly recent times viva voce was common. I had to defend my final year report on a long piece of practical work as part of the final for my BSc in Applied Physics in 1975.

You don't need to do everything orally. A return to handwritten final exams done without electronic aids would help too. They can be open note as mine were in the mid 70s or even open book.

As for scalability, I really don't see the problem; if we are producing so many qualified people then we surely can use some of them to examine the next generation. It sounds like the argument that often appears on the web where someone, usually an American, claims that a solution that works in, say the UK, can't work in their country simply because their population is larger.

Also, we really should consider if shoe horning every young person into an academic institution is useful either to society or to the person concerned. It certainly has not been an unmixed success in the UK.

zero_shift 3 hours ago | parent | prev | next [-]

I think you have the right observation but the wrong conclusion.

The appeal of digital systems, at a social level, is that they can transmit information accurately for almost no marginal cost. They come at the risk of fraud.

With AI the levels of risk have increased. Anyone can now be a fraudster, and it gets harder and harder to prove anything online. Either of us two, in this conversation, could be a well tuned LLM.

So society, collectively and at an individual level, is forced to reassess reliance on purely digital information. Which means that the advances in the technology, actually end up reducing the utility of digital systems more generally.

You can argue that this makes education costly and less accessible. It does. It absolutely does. There is also no alternative except to see higher education get wiped out, which would have devastating economic and social effects.

So we are going back to blue books and vivas.

steve-atx-7600 2 hours ago | parent [-]

"So we are going back to blue books..." exactly. And, it is also efficient. You just need a proctor to ID the test taker and make sure they aren't using AI to cheat in order to verify that the student learned something which seems like a good idea.

parsimo2010 2 hours ago | parent | prev | next [-]

This is basically my thought- my school is considering all sorts of things to combat AI cheating, and most of them scale poorly. You can implement them and increase your reputation, but your tuition will have to increase to pay for a lower student/teacher ratio.

So we’re probably going to see a split- mass production schools that still act as job training for Industrial Revolution era jobs will keep doing what they are doing. Schools that want to keep their academic prestige will implement these inefficient safeguards and raise prices accordingly. Employers will distinguish between these two types of diplomas when there are enough options out there that are combatting cheating- much like in-person schools are generally valued higher than online educations, or how certain schools (MIT, Stanford) are valued higher then others. This might lead us to the transformation of higher education many have been anticipating, because the current system cannot remain as-is. We all thought technology was the thing causing the upheaval but AI has really accelerated things.

Neil44 2 hours ago | parent | prev | next [-]

I graduated in 2000 and remember doing the oral defence. It wasn't a huge deal. You already have a personal relationship with your tutors and they are familiar with you and your communication style. They are not the Gestapo.

JumpCrisscross an hour ago | parent | prev | next [-]

> A written paper could be handed in and graded without lining everyone up in front of a panel

It can also be graded blindly. That allows for removing–or at least mitigating–biases in evaluation.

altairprime 4 hours ago | parent | prev | next [-]

It’s a good answer for keeping online education honest. It doesn’t need to be applied to all in-person schools, where teachers distribute spot checks across the course relatively evenly, but it makes perfect sense in conjunction with online testing. You only have to challenge one essay per student per term to catch cheaters with a near-perfect success rate. Assuming a 12 week course with 36 students, that’s qty.3-4 of teacher:student one on one meetings per week for the term; I think that’s doable. If you’ve oversold your teachers and have more than 36 students per, guess you’ll have to increase your labor expenses :D

SoftTalker 3 hours ago | parent | prev | next [-]

It shouldn't take long for educators suss out whether students know the subject they are talking about. You don't have to question them exhaustively to determine that they likely did or didn't do the work they are claiming. TFA is about high school projects, not PhD defenses.

jimbokun 4 hours ago | parent | prev | next [-]

Complete bullshit.

Universities in the US charge fortunes and pay instructors a pittance while plowing ever more money into “administrators”. Fire half the administrators and hire more instructors to both proctor exams and give more personalized instruction.

kccqzy 2 hours ago | parent | prev | next [-]

I don't think the world really works like that. Hearing an elevator pitch has a very different impact from reading a short-form business proposal. And of course the efficiency of the written word is not absolute; in lively discussions and debates, there is no time to put thoughts down into the written word; they had to be uttered. I am personally of the opinion that a successful university graduate should not only be able to write down their ideas in words but also be able to speak to defend their ideas.

hellohello2 2 hours ago | parent | prev | next [-]

"The oral system scales poorly. One examiner can only hear so many students in a day."

I'm not joking: you could have an AI quiz the students and test their understanding. That way the tests go deeper into testing the understanding by following through on the answers, and are standardized and fair for everyone. I don't think we are there yet at all but I actually think its not insane for the future. I expect downvotes though/I don't think our generation would agree with this.

What's old is new :)

moffkalast 4 hours ago | parent | prev | next [-]

> Students with speech differences, anxiety disorders, hearing impairments, or simply less fluency in the language of instruction face an extra barrier that a written submission does not impose. The written form, for all its flaws, leveled some of that ground. Shall we abandon the differently abled because machines disrupt the academic system and essentially make cheating effective?

Universities are entirely elective, why would they need to cater specifically to disabilities? Driving schools don't make exceptions for the blind. There are skills that people expect you to have with a certain kind of diploma, and it reflects poorly on the school if they don't have a standard to hold people to. Not being able to present your work in a basic way is one of those mandatory skills that imo makes you unqualified to graduate if you don't grasp it.

I'm not saying you can't make an exception for a person or two who is clearly as good or better than the rest otherwise, but people will always try to misuse as many exceptions as they can, en masse, as an excuse to just avoid doing required work.

zero_shift 3 hours ago | parent [-]

I feel this was already a severe problem pre-AI. From what I hear from my partner (in university administration), nearly half the students demand accomodations for disabilities. Many of those conditions are very difficult to falsify

It doesn't feel good, but at some point we have to ask, can we support every single case, at the expense of the whole? I'd like to be convinced that's not a necessary tradeoff

cpburns2009 an hour ago | parent | next [-]

What I don't get is why we disadvantage everyone who's honest. Just give everyone extra time to level the playing field.

moffkalast 3 hours ago | parent | prev [-]

I don't think it's that bad yet, but it's certainly on its way there. We're like at this intersection of lots of conditions that previously weren't documented becoming actual diagnoses, that pulling along a lot of peer-pressure kind of trendiness to have some kind of fault that makes you more unique, and the practicality of a system that was designed to give massive benefits to rare exceptions, for which now so many easily qualify on paper.

The list of conditions that qualify for what will most likely have to be redone completely from scratch.

ares623 4 hours ago | parent | prev [-]

But even with the written exam, wouldn't the bottleneck still be there where the professors still need to read through all the papers? I guess its definitely more efficient and asynchronous, but I can't imagine by much assuming the same and equal rigor is applied.

bonoboTP 4 hours ago | parent | next [-]

Grading written exams is delegated to teaching assistants / PhD students.

ninalanyon 31 minutes ago | parent [-]

Not everywhere.

jimbokun 4 hours ago | parent | prev [-]

You can be certain many instructors are having AI do a lot of their grading.