Is this not the exact gap that happened for OpenAI's accidental hack of huggingface? They tried to sandbox network access but the Antifactory or whatever package has holes that the collective of agents abused

I agree, that seems like a configuration/policy/operational approach, which is how we handle this problem now for humans using RBAC and authn/authz, just applied to AI. People do a crude version of this today with sandboxing (where the AI's sphere of influence is strictly limited by its environment, barring misconfiguration of the sandbox or breaking out of the sandbox, of course) and with workflows (where AIs are integrated into deterministic workflows, and then deterministic, non-agentic code decides how to handle AI outputs). But integrating this into more agentic architectures with finer control just seems like a best practice, said that way. It's not a tradeoff, there's no drawback, just do it. In other words, yes, a no-brainer.

That’s not what this is about. This is an algorithm for giving a model more freedom while nudging it in the right direction. It’s not about what tools are available.

doesnt work. this is a no-brainer because it's a bad solution.

The whole point of intelligence is that it's generalizable. If you constrain it to some controlled things, then it ceases to be useful. its incompatible. the whole incentive with ai is to let it do whtv it wants.

Yes, it appears the submitter rewrote the title (strike 1) without even reading TFA (strike 2). Not great.

Actual title: "New MIT Algorithm Meets Every Hard Constraint in Simulated Tests"

If fairness this article is poorly written and doesn’t explain what they actually did at all.

Indeed. Which is a shame because it's still pretty cool work.

> but if we only check the correctness at the end, then isn't it possible that it already collided with an obstacle?

You would put the check before it actually does the thing. It's at the "end" of the process of figuring out what it wants to do, not the end of fulfilling the request or prompt

I think you are thinking of the wrong 'end'. It isn't talking about the end of the entire movement path, we are talking about the end of the LLMs decision making process, and the output (whether that is the full path the arm should take, or just a subset of the path) is checked against the requirements.

Basically, anything that is leaving the LLM is checked, rather than the internal LLM reasoning process.

Outcome reward vs process reward models. The second is obviously better.. like getting partial credit on a physics test for wrong answers but correct method. Research is gradually hybridizing them but historically we avoided doing it the right way because of practical difficulties (labels required, more expensive and difficult) and more ideological ones (believers in magical machine intuition think it sounds too classical / logic based to be useful, pin their hopes on unproven faith in grokking at scale).

I’m pretty sure this is just a poorly written article.

HardFlow seems to be a strategy for nudging the model in the right direction while giving it more freedom. Only applying the constraints at the end is a mischaracterization from what I can tell.

It’s talking about physical safety. Not ethical safety. The concern here is robots physically colliding with things.

essentially this https://xkcd.com/2044/

Reminds me of the huggingface hack