| ▲ | Nevermark 5 hours ago | |
> I’m sorry but isn’t this just over fitting based on conversational adaption. But worse since it’s self conversation? Resulting in the exact same problem 4o1 had: sycophancy? But again worse. To me this comes across as word salad. The edits are done based on reasoning, to accomplish a "target" improvement, that is itself reasoned and designed, taking into account a reasoned understanding of the existing functionality of the weights. This is the furthest thing possible from some gradient optimization "over fitting" weights to tune a pedantic measure/target. That could be relegated to one-time greenfield-model bootstrapping. | ||
| ▲ | tsunamifury an hour ago | parent [-] | |
Maybe use an LLM to figure it out. It reasons via conversation with itself. How is this word salad. Are you unaware of the fundamentals of this field? Over fitting is then reasoning on its already set states and not introducing any novel data. Leading to jsut cyclically overfitting of an existing reward condition. Also the paper stated it didn’t work 4 out of 5 times. So I dunno? This is a big nothing. | ||