Remix.run Logo
▲ flowerlad 2 hours ago

You have stated your position but not explained it. How does it help you if the weights are open?

▲danielmarkbruce 2 hours ago | parent | next [-]

Risk.

▲wholinator2 2 hours ago | parent [-]

Can i have more explanation please? I'm not in risk management but i would like to understand the situation better. Just risk that they change it under you? Or that they're training on your data, or what kinds of risks are we most worried about?

▲danielmarkbruce 2 hours ago | parent | next [-]

They can make your model disappear. If you have a system and a set of prompts and evals and everything is working well, it's no fun for that model to just disappear. Some will reply "do a better job building your system so it's not model dependent" and they are sort of right, but that isn't always economically feasible and/or just isn't the reality of how a lot of systems get built.

▲NBJack an hour ago | parent | prev [-]

Models hosted as a service can be subtly tweaked without any announcement, or suddenly be 'upgraded' to a new one with a ticking clock on how long the old one sticks around. In both cases, the flows people establish around them are disrupted or broken.

I've lost track of the anguished cries of those unhappy when either of these has happened.

▲lern_too_spel an hour ago | parent [-]

That's a problem with decision models, for example, but coding models have no such risk. I switch between multiple coding models from multiple providers each day.

▲arein3 2 hours ago | parent | prev [-]

You must be new here

History shows that if you give a corporation power to extract money from you, most of them will do that without any regard to the user.

That power is the model weights. If there is a market around open weights, they will exist and corps will not be able to squeeze so hard.

Unfortunately not many people understand this simple and intuitive reality, and thats why we suffer

▲peterbell_nyc 2 hours ago | parent [-]

This assumes material switching costs. I use frontier max plans for now but have fail over models for each class of model action with tuned context so I can keep the performance when I lose the max plan subsidy at a lower price that staying with the frontier labs for certain classes of work.

I see minimal downside in using closed weight models as long as they're behind a router, you have good prod sourced evals and you have fail over to other models configured and tested (running either dark or active:active) with real prod workloads.

▲munk-a 2 hours ago | parent [-]

If you want relatively repeatable and predictable responses (even within the reality of being non-deterministic) then making sure you'll have continued access to a model to replay responses for debugging and tuning can be important.