Remix.run Logo
▲ Topfi 4 hours ago

Recommend checking out Logical Intelligence, chaired by among others Yann LeCun, for more context on EBMs [0]. Has great potential in areas that LLMs traditionally do not perform that well in and, because they want to show the advantages of EBMs, also tends to be a great source of examples LLMs struggle with, like primary school level maths and counting (while ironically providing Millennium level Lean proofs). Does of course open the not unreasonable discussion of whether it matters if LLMs can accurately count when writing code to do the calculation is reliable and fast...

[0] https://logicalintelligence.com/blog/energy-based-model-sudo...

▲abeppu 15 minutes ago | parent | next [-]

> We disabled code execution for all models in the demo, to prevent the LLMs from writing a brute-force backtracking solver in Python and running it, which would tell us something about their coding ability but nothing about their reasoning ability.

This seems sketchy. Reasoning how to solve the problem with a program is still reasoning, and if it yields programs which are more efficient (bc even a slow interpreter is faster/cheaper than an LLM) and potentially reusable, then so much the better!

▲throwaway0q5347 8 minutes ago | parent [-]

I think the goal in this case is to isolate the LLM from that avenue in order to determine whether or not the approach is improving the LLM intrinsic ability, not because it is a better solution than having the LLM create programs.

▲testtestmetest 3 hours ago | parent | prev | next [-]

Logical Intelligence is most definately a scam, that should hardly be taken seriously. Its one of the handful of companies that is able to automatically fundraise a fk ton of momey just by putting LeCun's name on it (a great way to make a bunch of money off VCs last year).

Its funny everytime ebms get brought up people have to name drop this company. They literally do nothing.

▲throwaway0q5347 2 minutes ago | parent [-]

> Logical Intelligence is most definately a scam, that should hardly be taken seriously

citation needed?

▲nilsherzig 4 hours ago | parent | prev | next [-]

> See the live benchmark

broken login wall. truly agi.

▲Topfi 4 hours ago | parent [-]

Yeah, that's embarrassing. It was a neat showcase when it was online. Was only able to find this video that shows the demo [0].

[0] https://www.linkedin.com/posts/brianbuntz_sudoku-is-apparent...

▲otabdeveloper4 3 hours ago | parent | prev [-]

> while ironically providing Millennium level Lean proofs

They're doing that by generating a metric shitload of schizo syllogism junk that is then sifted through Lean to find a valid nugget. (Or not.)

The implication is that "Millennium level" is mostly stuff that's too tedius for meatsack humans to sift through.