Remix.run Logo
Move tests to closed source repo(github.com)
47 points by nilsbunger a day ago | 28 comments

See also https://simonwillison.net/2026/Feb/25/closed-tests/

alt187 2 hours ago | parent | next [-]

Whether this was a joke or a backtracking, or this dared waste your oh so precious time- You're missing the forest for the trees. There's extreme covert and even overt hostility between how people stand on AI's gluttonous usage of the commons.

We're about to waltz into a deep period of tension between developers, and people who, empowered by multimillion dollars corporations, bravely violate developers' copyrights in the hopes of replacing their jobs, while bullying these same developers who dare express their discontent.

This is not gonna end well.

aichen_dev 42 minutes ago | parent | prev | next [-]

The SQLite comparison someone mentioned is spot-on. SQLite has kept TH3 (their 100% branch coverage test suite) proprietary for years as a monetization strategy, and nobody bats an eye.

Whether tldraw's issue was a joke or not, it highlights a real tension: open source maintainers are watching AI companies train on their code, tests, and documentation - the very artifacts that make software reliable - and then use it to generate competing implementations. Tests are arguably more valuable than the code itself because they encode the specification and edge cases.

I suspect we'll see more projects adopt a split model: open source the runtime, keep the validation suite proprietary. It's a natural response when your test suite becomes a training signal for competitors.

sluongng 41 minutes ago | parent [-]

https://sluongng.substack.com/i/186718212/test-is-king I wrote about this less than a month ago. Things are moving pretty fast in this direction.

hellcow 4 hours ago | parent | prev | next [-]

This is interesting because it’s also one of SQLite’s monetizations. SQLite is in the public domain, but you need a commercial license to access their TH3 test harness with 100% branch coverage used to validate SQLite on different platforms.

latchkey 5 hours ago | parent | prev | next [-]

Read the thread, it was a joke.

"Sorry folks, this issue was more of a joke (am I allowed to do that?) but I'll keep the issue open since there's some discussion here."

simonw 4 hours ago | parent [-]

That's from this comment here: https://github.com/tldraw/tldraw/issues/8082#issuecomment-39...

Well that's embarrassing! I reported it as if it wasn't a joke. I thought the joke issue was this one about translating everything to Chinese: https://github.com/tldraw/tldraw/issues/8092

unfunco 3 hours ago | parent | next [-]

If it was a joke (the test suite issue), then it was a really shit joke. It reads more like backtracking, I don't think _you_ should feel any embarrassment.

gempir an hour ago | parent [-]

The gag started on Twitter after Cloudflare vibe coded a nextjs replacement clone.

If you know that context and the tweet I feel this is more obvious that it is a joke.

Just because you didn't get the joke, does not make it a really shit joke. The funniest jokes rely on context.

latchkey 4 hours ago | parent | prev [-]

happens to the best of us. these days, we need to double and triple check everything before we react.

ramoz 4 hours ago | parent [-]

AI does have positive contributions to society after all.

Dwedit 4 hours ago | parent | prev | next [-]

Maybe we just Jai Tan to provide some fresh test data.

plesiv 2 hours ago | parent | prev | next [-]

The "this wasted my time" comments are missing the point...

In addition to his great sense of humor, Steve is usually ahead of the curve in terms of trends. There's a lesson in this. LLMs have become incredible constraint solvers ("SAT-solvers for code"). Well-thought-out tests, types, specs, and docs are all incredibly valuable constraints. This has big implications - for example what happens to licenses when you can cheaply rewrite the codebase and therefore unencumber it.

Iolaum 15 minutes ago | parent [-]

Is it really re-writing - legally - if you are starting from the codebase itself? Not a lawyer, am wondering however if the Google vs Oracle Java lawsuit has some implications for this.

pona-a a day ago | parent | prev | next [-]

I'm thinking of migrating to ExcaliDraw or Xournal++ next time I need a whiteboard.

The performative closing of public contributions citing the slop scare felt disingenuous from the start. You couldn't be bothered to implement _any_ mitigations that leave the community engaged with the project?

Writing a contributor karma bot, moving to a non-social or obscure git forge (most slop contributors are resume farming and GitHub is the only forge the HR cares about), newbie-unfriendly non-public workflows like git send-mail, or references from Discord... This isn't an AGI on the other side of the screen, planning the perfect strategy to infiltrate your project; it's a sub-script-kiddie trying to fill a portfolio with quick "contributions" doing the more annoying version of "fixing typos" in docs.

anitil a day ago | parent | prev | next [-]

This is concerning, it feels a bit tragedy-of-the-commons I suppose where having public tests are a valuable public good, thought I can't quite get the analogy straight in my head.

crabmusket an hour ago | parent | next [-]

> feels a bit tragedy-of-the-commons ... I can't quite get the analogy straight in my head

I have a personal theory that "tragedy of the commons" has a very specific meaning, and beyond this meaning it just adds confusion. This isn't your fault - it's an overused phrase.

I'd try to examine the root of your discomfort. Why does it make you feel bad? Avoid thinking about "big ideas" like the commons or the public good.

cwillu 4 hours ago | parent | prev [-]

It was a joke.

koolala 4 hours ago | parent [-]

The joke is that its not open source?

benatkin 5 hours ago | parent | prev | next [-]

The headline should be changed, because it is moving from one closed source repo to another closed source repo, and on HN misleading headlines tend to be corrected even if they're deliberate on the part of the authors.

simonw correctly describes it as "not technically open source" - though OSI doesn't have the trademark, the term open source, capitalized or not, refers to the what the Open Source Definition codifies. There are other terms such as shared source, for this sort of stuff.

cwillu 4 hours ago | parent [-]

The headline should be changed because it was a joke: “Sorry folks, this issue was more of a joke (am I allowed to do that?) but I'll keep the issue open since there's some discussion here.”

verdverm a day ago | parent | prev | next [-]

I wonder if TLDraw realizes that Ai can probably run the software and generate an even better test suite. Days to replicate +1?

threatofrain 5 hours ago | parent | next [-]

Some test suites are gold, and not in the range of days to replicate.

javier123454321 4 hours ago | parent | prev | next [-]

So why the pushback?

monster_truck 4 hours ago | parent | prev [-]

Hours

worthless-trash 3 hours ago | parent [-]

Doubt.

ddtaylor 3 hours ago | parent | prev [-]

What a strange joke that wasted the time of so many.

alt187 2 hours ago | parent [-]

Entitled mentality.

poly2it an hour ago | parent [-]

Open source is a comittment. It is entitled of companies to grow developer user bases by promising that they will continue to provide their product to consumers and foster an open community, then pull the rug once openness no longer benefits them. The decision to go open source should less often be guided by financial reasons. It is foremost a social system of distributed labour and dependence.