Remix.run Logo
dandaka 3 hours ago

I think this is a well known concept, which we can't deliver yet. LLM/transformer give us reasoning engine as a byproduct of its design, but it is quite ineffective. If we can distill reasoning, if reasoning can be achieved without general knowledge, it will be a very effective machine.

Some amount of knowledge is required for reasoning. Maybe such model can dynamically knowledge domains to have taxonomy. For example, model can't effective reason about development task, if it has no knowledge about development best practices. But population of New York or recipies can definitely be loaded run time with tools.

XCSme 15 minutes ago | parent | next [-]

Yup, you still need knowledge. Even if you have access to all the data and tools, you still need to know what to search for, what tools to use and to understand what the user is asking.

Our computers can already do everything, have access to all the tools and information, yet they still need a human/intelligence to use it and apply to specific problems.

Even defining the problem requires knowledge.

As for the tools, if the model has access to 1000 tools, how would it know which one to use if it doesn't have any knowledge itself?

What if I ask for "table tennis spin" it had a "magnus effect calculator", how would it know to make the connection between the two?

sigmoid10 2 hours ago | parent | prev [-]

>Some amount of knowledge is required for reasoning.

This is the root of problem. If you think about STEM universities, they don't really teach you things you need in the real world. They teach you what you need to know in order to go out there and accumulate the necessary information which can then be used to solve problems. Giving a person access to the internet or a super powerful calculator (like Mathematica) won't mean that they can do anything useful. They need tons of experience to use these tools in an effective way. That experience is basically all that implicit adjacent knowledge that we pick up along the way getting our degrees. And LLMs pick that up during pre-training. Drop this part and the outcome will be worthless.