They are generally things I don't particularly want to share, as I view them as ideas close to my heart. The general pattern was what felt like redirecting from the harder problems and an unwillingness, if we can call it that, to go there. In some sense though, it's hard to understand how these more advanced models get around that, because these models are tuned to the most probable answer. When you come at them with a novel idea, they always seem to water it down in the end.

I have also found this — once you get out into some really novel workspaces they will revert back to replication of adjacencies that already exist. Fable 5 and 5.1 in particular will talk around the issue in a way that no model prior seemed to.

I think the issue is fairly fundamental, perhaps because of tuning, perhaps even beyond what they are tuned or trained on, because the bulk of what they can go find and ground into seems to drag them heavily toward implicit biases that split around categories I can’t quite discern. The category boundaries sometimes look like academic fields, but sometimes they seem like more fundamental attractors.