I don't think riddles are necessarily 'solvable' in that there's only one right answer; the very fact that they're open to interpretation, but when you get the 'right' answer it (hopefully) makes sense. So if an AI/LLM can answer such a nebulous thing correctly- that's more of the angle I was going at.

Regarding the wizards example, I'm a bit confused; I was thinking that the best way to judge answers for problem solving/creativity was for correctness. I'll think more on whether the 'thought process' counts in and of itself.

The answer to my riddle is 'ball'.

▲ shagie 17 hours ago | parent | next [-]

Perfect correctness is what you'd expect from a computer. I could write a program that solved it - and that would be an indication of my creativity as a human solving something that I haven't encountered before. Incidentally, that's also how it approached solving the block problem (by writing a program).

If you ask me the goat, wolf, cabbage problem I'd be able to recite (as an xkcd fan https://xkcd.com/1134/ and https://xkcd.com/2348/ and the exploration of what else it could do). However, if someone hasn't seen the problem before it could be a useful tool at seeing how they approach solving it.

The question of how does it tackle a new problem is one of creativity and exploration of thought in a new (untrained) domain.

A possible claim of "well, it's been trained on the meta-problem of how to solve problems that weren't in its training set" would get a side eye.

For the "ball" being the answer... consider the second response to https://chatgpt.com/share/6920b9e2-764c-8011-a14a-012e97573f... (make sure you click on the "Thought for 1m 5s" to get the internal process)

▲ johnisgood 19 hours ago | parent | prev [-]

How did you get "ball" from your riddle? I read it and I have no idea! :(

▲ shagie 16 hours ago | parent [-]

In my sibling comment, I linked the chat session where I prompted ChatGPT for possible answers and reasoning.

https://chatgpt.com/share/6920b9e2-764c-8011-a14a-012e97573f...

    Given the following riddle, identify the object to which it refers.
    #
    Chaos erupts around
    The shape moot
    The goal is key
    #
    Identify 10 different possible answers and identify the reasoning behind each guess and why it may or may not be correct.

The second item in the possible answers:

    Soccer ball
    Why it fits:
        “Chaos erupts around”: Players cluster and scramble around the ball; wherever it goes, chaos follows.
        “The shape moot”: Modern footballs vary in panel design and surface texture, but they must all be broadly spherical; to the game itself, variations in cosmetic shape are mostly irrelevant.
        “The goal is key”: Everyone’s objective is to get the ball into the goal.
    Why it might not be correct:
        The third line emphasizes the goal, which points more strongly to the scoring structure or concept of scoring rather than the ball.