| ▲ | hacker_homie 3 hours ago | |
because it's a separate context window, it makes the model bigger, that space is not accessible to the "user". And the "language understanding" basically had to be done twice because it's a separate input to the transformer so you can't just toss a pile of text in there and say "figure it out". so we are currently in the era of one giant context window. | ||
| ▲ | codebje 3 hours ago | parent [-] | |
Also it's not solving the problem at hand, which is that we need a separate "user" and "data" context. | ||