| ▲ | lordnacho 3 days ago | |
I went to see the last Mallard Song. Just to say I went, of course. It looked like a bunch of weirdos in a courtyard to me, but it was a literally once-in-a-century event, and I was living less than a minute away, so why not? I don't think I've ever heard of a scheduled ritual that has a longer period. You're guaranteed to never have anyone present at more than one of these, so surely many aspects of the ritual will wander quite far from the original? As for LLMs on the All Souls test, it's predictable that it mostly whiffs. After all it takes in a diet of Reddit+Wikipedia+etc, none of which is the kind of writing they are looking for. Reddit is a lot of crappy comments. If you have no grounding in reality (being a thing that lives in a datacentre), how are you going to curate it? Some subs are really quite good, but most are really quite bad. It's not easy to get guidance, of the kind you would get if you sat with a professor for three or four hours a week for a few years, which is what the humanities students actually do. Wikipedia is a great reference work, but it tends to not have any of the kinds of connections you're supposed to make in these essays. It has a lot of factual stuff, so questions about Persia will look ok, like in the article. But questions that glue together ideas across areas? Nah. Even if that's in the dataset somewhere, how is the LLM supposed to know that the sort of refined writing of a cross-subject academic is the highest level of the humanities? It doesn't, so it spits out what the average Redditor might glue together from a bit of googling. | ||
| ▲ | dash2 3 days ago | parent [-] | |
OK, interesting hypothesis. So, I wondered how it would do with "Why should cultural historians care about ice cores?" which indeed requires gluing together ideas across areas. I asked ChatGPT 5 on Thinking mode: https://chatgpt.com/share/689e5361-fad8-8010-b203-f4f80d1457... It does a pretty good job summarizing an abstruse, but known, subfield of frontier research. (So, perhaps not doing its own "gluing" of areas....) It clearly lacks "depth", in the sense of deep thinking about the why and how of this. (Many cultural historians might have reasons for deep scepticism of invasion by a bunch of quantitative data nerds, I suspect, and might be able to articulate why quite well.) It's bullet points, not an essay. I tried asking it for a 1000 word essay specifically and got: https://chatgpt.com/share/689e5545-0688-8010-8bdf-632d3c3466... which seems only superficially different - an essay in form, but secretly a bunch of bullet points. For a comparison, here's a Guardian article that came up when I googled for "cultural historians ice cores": https://www.theguardian.com/science/2024/feb/20/solar-storms... It seems to do a good job at explaining why they should, though not in a deep essayistic style. | ||