Remix.run Logo
▲ wholinator2 2 hours ago

Can i have more explanation please? I'm not in risk management but i would like to understand the situation better. Just risk that they change it under you? Or that they're training on your data, or what kinds of risks are we most worried about?

▲danielmarkbruce 2 hours ago | parent | next [-]

They can make your model disappear. If you have a system and a set of prompts and evals and everything is working well, it's no fun for that model to just disappear. Some will reply "do a better job building your system so it's not model dependent" and they are sort of right, but that isn't always economically feasible and/or just isn't the reality of how a lot of systems get built.

▲NBJack an hour ago | parent | prev [-]

Models hosted as a service can be subtly tweaked without any announcement, or suddenly be 'upgraded' to a new one with a ticking clock on how long the old one sticks around. In both cases, the flows people establish around them are disrupted or broken.

I've lost track of the anguished cries of those unhappy when either of these has happened.

▲lern_too_spel an hour ago | parent [-]

That's a problem with decision models, for example, but coding models have no such risk. I switch between multiple coding models from multiple providers each day.