| ▲ | sigbottle 2 hours ago | |
"Encoding" implies some GOFAI symbolic formal rule machinery. I'd argue that transformers are a pretty good indication that intelligence isn't "encodable" in the way we think it means. Usually, most "model" vocabulary means that we can explain and constrain the "data" from the "rules". Except the mere "data" is trillions of interacting weights. That may be encoding in a physical sense, but that still doesn't explain the intuition in any legible way to humans. Cynically, we've been able to encode everything already by just saying everything's a transition in a huge lookup table. Not very informative though. | ||