Remix.run Logo
▲ nico 3 hours ago

This is actually a genius idea. Imagine being able to compress a 1T parameter model to 10M params by creating a lossy model-scaffolding, like DNA

Something that more or less guarantees you’ll get about the same capabilities as the original model, but with a different final expression of certain characteristics

I wonder if this could be a good distribution model too, selling “dried” models (almost like raisins - or the beings from the 3 body problem), and then you can buy them to fully train (raise?) with your own data

▲spwa4 2 hours ago | parent | next [-]

Aside from violating everything we know about neural networks (technically this would mean there is a way to reduce the rank of even something like ChatGPT to 1. There isn't. No way) it would also be VERY unlike what DNA does. DNA has lots of ways to express things that aren't easily disturbed by mistakes. Those mistakes are then an actual positive, in the very long term, because they help natural selection along.

A random seed that works would be the exact opposite of that.

Why wouldn't DNA just encode "good feelings" when IT gets what it wants? Right temperature -> ahhh. Food -> ahhhhhh. Sex -> aaaaaahhhhhhhhhh. And then just let reinforcement learning take care of all the details in between. No need for storage space for fire making or for "Hey girl, how are you doing?" ... and therefore cheaper.

▲dingaling911 an hour ago | parent | prev [-]

I'm somewhat of a genius (self-described.)

I like your idea of "base" models for sale.