A reasonable next move, if the US was interested in acting like a democracy, would be legislation forcing this decision:

- prove that you had the rights for all of your training data

- open source the model

Give the labs a 3 month grace period in which to comply, so competition can persist even with dubiously sourced data, but the people can't be locked away from derivatives of their contributions for any significant amount of time.

> It is very tiring to say “I don’t necessarily disagree with you about AI ‘art’, but in my field—which you do not understand, and in which the underlying build process is often not the creative output—AI presents very real productivity gains” for the umpteenth time.

That’s not a justification. If a company were poisoning the water to your home as a byproduct, would you be satisfied if they told you “we don’t necessarily disagree with you about polluting the water, but in our field—which you do not understand, and in which the underlying build process is often not the water pollution—what we’re doing presents very real productivity gains”?

> I am skeptical of there being sufficient data to build “ethical” training datasets

Then you don’t build any. What fucked up world we live in where people think it’s OK to be unethical because they want something and can’t think of any other way to do it. What monumentally selfish rotten babies.

> has already irreversibly made its way into every modern dataset

The "gray goo" scenario finally happens... for AI. That's actually the good ending for humanity. I love it! Poetic and believable. Data doesn't "heal" like nature. :D

I think you can train on math /science using synthetic generation to a significant extent. Training for coding requires more of the "scraping github / stackoverflow" angle but IMO that's a shallower hill to climb than scraping copyrighted art and literature.

There are actual models trained on ethical datasets but they are obviously not very high powered. If companies with the resources of an anthropic or openai were doing it (ha) it would be more feasible

>“I don’t necessarily disagree with you about AI ‘art’, but in my field—which you do not understand, and in which the underlying build process is often not the creative output—AI presents very real productivity gains”

Being able to prove such gains in better products would be a start. And an emphasis on how it assists existing engineers/mathmaticians/researchers, not that any accomplishment made with AI assistance is "AI solves problem".

I don't know whatever happened to "words are cheap". I guess it literally made money to say words, so that adage is false for the time being.

>I am skeptical of there being sufficient data to build “ethical” training datasets

Well if all those scam job ads paying 100/hr to create AI training content was not a scam and instead the approach from the start, there may have been a chance to bridge that gap ethically. The industry chose to break things and is trying to act mad that people are mad at all the broken stuff.

These results are entirely a consequences of the actions chosen. And I don't believe there was ever an honest consideration of there being ethical training datasets. They just thought they could brute force society with fearmongering and bribes. The BOTD was already low in the beginning but completely gone now.

Yeah and it only took stealing all the intellectual output of everyone ever made.

But sure, there's um, an ethical way of doing that?

Artists take pride in the intentionality of every brushstroke, and for them any given work is much more likely to reach a point where it's considered "finished".

While coders may care about the craft (and I do), it's not as if the value of my code is in the exact variable names I chose.

So you recognize that discovery cannot be cut off from other discovery, that it's never just one or the other, and your solution is to say shut down all discovery?

you can distinguish between the LLMs you have zero tolerance for and a system like Google Translate? You have a sharp line you can draw for when something becomes "an LLM"?

While that is true, theoretically a regulation could be enacted that output tokens must focus on STEM research and other practical tasks and the LLM must refuse tasks outside of those areas, just as Claude disallowed cybersecurity tasks. Obviously this would never happen, but the theft of the training data wouldn’t matter as much if the usecases were less sinister.

openai and anthropic trained on actually stolen data since it was pirated datasets.

google OTOH already had a lot of this dataset in their possession (e.g. Google Books etc), still questionably licensed for how they used it, but not quite as bad. They did apparently break through NYT paywalls and stuff like that though, still theft.

I can't tell if you're serious or not.