Remix clone Hacker News

from a foundational research perspective, the pokemon benchmark is one of the most important ones.

these models are trained on a static task, text generation, which is to say the state they are operating in does not change as they operate. but now that they are out we are implicitly demanding they do dynamic tasks like coding, navigation, operating in a market, or playing games. this are tasks where your state changes as you operate

an example would be that as these models predict the next word, the ground truth of any further words doesnt change. if it misinterprets the word bank in the sentence "i went to the bank" as a river bank rather than a financial bank, the later ground truth wont change, if it was talking about the visit to the financial bank before, it will still be talking about that regardless of the model's misinterpretation. But if a model takes a wrong turn on the road, or makes a weird buy in the stock market, the environment will react and change and suddenly, what it should have done as the n+1th move before isnt the right move anymore, it needs to figure out a route of the freeway first, or deal with the FOMO bullrush it caused by mistakenly buying alot of stock

we need to push against these limits to set the stage for the next evolution of AI, RL based models that are trained in dynamic reactive environments in the first place

▲

hansmayer a day ago | parent [-]

Honestly I have no idea what is this supposed to mean, and the high verbosity of whatever it is trying to prove is not helping it. To repeat: We already tried making computers play games. Ever heard of Deep Blue, and ever heard of it again since the early 2000s?

▲

lechatonnoir a day ago | parent | next [-]

Here's a summary for you:

llm trained to do few step thing. pokemon test whether llm can do many step thing. many step thing very important.

▲

hansmayer 20 hours ago | parent [-]

Are you showing off how the the extensive LLM usage impaired your writing and speaking capabilities?

	▲	drdeca 2 hours ago \| parent [-]
		You complained about the high verbosity.

▲

Rudybega a day ago | parent | prev [-]

The state space for actions in Pokemon is hilariously, unbelievably larger than the state space for chess. Older chess algorithms mostly used Brute Force (things like minimax) and the number of actions needed to determine a reward (winning or losing) was way lower (chess ends in many, many, many fewer moves than Pokemon).

Successfully navigating through Pokemon to accomplish a goal (beating the game) requires a completely different approach, one that much more accurately mirrors the way you navigate and goal set in real world environments. That's why it's an important and interesting test of AI performance.

▲

hansmayer 20 hours ago | parent [-]

Thats all wishful thinking, with no direct relation to the actual use cases. Are you going to use it to play games for you? Here is a much more reliable test: Would you blindly copy and paste the code the GenAI spits out at you? Or blindly trust the recommendations it makes about your terraform code ? Unless you are a complete beginner, you would not, because it sometimes generates downright the opposite of what you asked it to do. It is because the tool is guessing the outputs and not really knowing what it all means. It just "knows" what character sequences are most likely (probability-wise) to follow the previous sequence. Thats all there is to it. There is no big magic, no oracle having knowledge you dont etc. So unless you tell me you are ready to blindly use whatever the GenAI playing pokemon tells you to do, I am sorry, but you are just fooling yourself. And in the case you are ready to blindly follow it - I sure hope you are ready for a life of an Eloi?

▲

Rudybega 6 hours ago | parent [-]

All of that is totally unrelated to the point I'm trying to make.

Pokemon is interesting because it's a test of whether these models can solve long time horizon tasks.

That's it.

	▲	hansmayer 5 hours ago \| parent [-]
		Ok, well now that you phrase it clearly like that, it makes much more sense, so it's a test of being able to keep a relatively long context-length. Another incremental improvement I suppose.