Top K and repetition penalty and the other main knob I forget the name of are partially to blame here, I think. I can test it tomorrow with this exact prompt and fiddling those values.

That pattern, not the words, is in there a lot. That riddle was posted everywhere online, in email chains, etc. I think if you let it choose from more than the top 40 and let it "stutter" with repetitions it might realize the riddle is a non-sequitur (is that the right term?)

And the third knob is not temperature, although I'd try turning that up first just to check. Yes, up.