It looks like these frontier-model companies don't really monitor their systems. Like OpenAI not realizing that it is their own AI which is attacking HuggingFace.

▲

thewebguyd 3 days ago | parent | next [-]

> Like OpenAI not realizing that it is their own AI which is attacking HuggingFace

Or, they knew and let it continue because they are not a good company.

"Never attribute to malice.." blah blah, I have a hard time believing the very smart people at OpenAI would just let their off leash model run hands off with no monitoring and not immediately pull the plug when it jumped its containment.

▲

causal 3 days ago | parent | prev | next [-]

Yeah if anything it makes Anthropic look incompetent

▲

moralestapia 3 days ago | parent | prev | next [-]

How does that connect with @throwa356262's argument?

▲

kami23 3 days ago | parent [-]

That they should be able to find distillation 'attacks' if they had enough observability.

▲

moralestapia 3 days ago | parent [-]

That's not @throwa356262's argument.

@throwa356262 argument is that it is infeasible to distill and release a new frontier model in two weeks.

▲

kami23 3 days ago | parent [-]

Ah I interpreted it as 'of course they can't stop distillation if they couldn't stop a model from escaping its sandbox'

I can see how there's a big leap there, but I agree somewhat. If they are aware these are happening and can detect it as it is happening why are they not stopping them? What do you do there? It'll be cat and mouse for a while. Thinking of reasons they wouldn't try and stop it is just a lot of speculation in my brain.

It's probably a way harder problem than I think it is, but they are aware of them now, so I assume they are going to get more aggressive about it.

Let's say then that they can't detect them near real time or even a bit after, maybe they do have a big observabilty gap that no one has solved adequately.

The speed which they add features I've needed for governance is pretty close to the speed I 'manually' write those for my company. To me personally we are all just going fast and breaking everything and not having enough time to set up safe environments. I'm sure it's in the backlog.

	▲	moralestapia 3 days ago \| parent [-]
		Hmm ... so the gist of the issue is this. Training and releasing a model like Kimi K3 takes months-to-a-year (and that's if you're really good at it). 'months-to-a-year' ago there was no Fable, so there was no way for them to distill them.

▲

pas 2 days ago | parent | prev | next [-]

how would they detect?

▲

Grimblewald 3 days ago | parent | prev [-]

alternativly the HF is a gpt2/strawberry/mythos style marketing stunt.

Does no one remember the extreme fearmongering around gpt2 which barely produced coherent text?