I have my own challenge: I think LLMs can do everything that a human can do and typically way better if the context required for the problem can fit in 10,000 tokens.

For now this challenge is text only.

Can we think of anything that LLMs can’t do?

> For now this challenge is text only.

That is like saying, "My program is better than any human, but binary inputs only!"..

Even restricted to text, the LLMs are not better than a human who is expert in a domain. Try talking to it regarding any topic. Even in topics that I am not an expert it, the responses from LLMs quickly becomes bland and uninteresting and devoid of additional information..

This is true even in tech things that after a certain point, I stop talking to it and search for a blog/post/so answer written by a human, which if found, would immediately break the plateau of progress that I was facing with the LLM.

This is a “no true scotsman” challenge. People are going to say llms can’t do certain things and you are going to say they can.

Not very interesting.

LLMs without tooling can not do most tasks. It’s the tools and skills that enable them to do things.

So you may as well ask “is there anything a python script can’t do”. It’s just not a meaningful question.

Sure. “Tell me a joke”

They can't beat even a mediocre chess player at chess.

* code

* write interesting prose

* generate realistic images