Remix.run Logo
daedrdev a day ago

generic biases can also be fixed

orbital-decay a day ago | parent [-]

*Some generic biases. Some others like recency bias, serial-position effect, "pink elephant" effect, negation accuracy seem to be pretty fundamental and are unlikely to be fixed without architectural changes, or at all. Things exploiting in-context learning and native context formatting are also hard to suppress during the training without making the model worse.