Dunning-Kruger vibes from this blog post.

Look into how LLMs work and it's pretty clear why these character counting scenarios often fail if not invoking Thinking/Python Scripting.

Hey, just woke up. Good morning, From my understanding, the reason that this happens is that as LLM's work from token to token which breaks the word being the reason why this type of anamoly occurs. The only thing I know is that I know nothing to be honest and I wish to learn more, and this was just me sharing something that I found interesting :-D

The thing which I find interesting though is that even if you don't involve Thinking, the r's in strawberry test always succeeds

https://chatgpt.com/s/t_69f85fb01d8881918016f2ceb3d1f314 & https://chatgpt.com/c/69f85e1d-25e8-8320-ba10-2dbe5857fc74

https://chatgpt.com/share/69f85fd6-4534-8322-a3f9-a06f3e26ec...

Now one can say that this patch was added as the r in strawberry got added into training test but I remember some comments at that time where it suddenly changed the answers as it started getting more humiliating for OpenAI as everyone started to pick this knowledge up and this might be the same reason why it says that it has three e's so many times because it tries to always say the three r's BUT that is just my opinion (which I don't wish to present as fact)

I think that the issue I have is that the people working in AI also present it sometimes as the holy grail when it has some clear flaws and they gloss over it.

If you found this result unsurprising, good for you!, but I feel like even as someone who spent a lot of time in the internet with LLM's, I didn't expect it because I thought that it was a completely solved problem in all LLM's, and I just shared this with everyone.

Have a nice day :-D