| ▲ | Weight-Space Programming builds neural networks like a 1990s researcher(notesbylex.com) | |||||||
| 65 points by lexandstuff 2 days ago | 5 comments | ||||||||
| ▲ | noduerme 2 hours ago | parent | next [-] | |||||||
I love the idea, somehow, that these systems had to be built with domain knowledge integrated into the choices of layers and the order of algorithms, using rather brilliant discoveries for optimizing lines and edges, or whatever was relevant to the problem. That's an order of specificity that's lacking from our modern reliance on "one shot" solutions. Letting a one shot solver grind its brains against creating a custom solution to a specific set of transformations is kind if a closing of the circle when it comes to performance and resources for a certain task. I'm looking forward to the day we can have an optical device that runs only on light input, that looks like a piece of quartz, that you can place over a handwritten number (0-9) and on the top it displays the number in a clean font. Where the neural network to handle this can be translated into physical transformations once it's discovered, no inference processing power required. | ||||||||
| ||||||||
| ▲ | agnosticmantis 3 hours ago | parent | prev | next [-] | |||||||
Thanks for putting this together. Very nicely demystifies the weight-space programming phenomenon which I otherwise found quite magical and mysterious. It's not that the models magically "intuit" meaning from a blob of weights, but they methodically iterate on setting the weights with reasoning similar to a human engineer's reasoning. | ||||||||
| ▲ | robrenaud 5 hours ago | parent | prev [-] | |||||||
Would PPO learn quickly from the hand engineered solution? Is PPO getting a terrible init? It's interesting to see how tolerant of tedium the models are. | ||||||||
| ||||||||