| ▲ | MWil an hour ago | |
have you considered implementing the addition of a leading canary sentinel that fires at the earliest/cheapest possible point instead of only on lag of some actual load-bearing constraint violation? | ||
| ▲ | zambelli an hour ago | parent [-] | |
Do you mean catching errors as tokens stream back versus waiting for the full message? If so, then no I hadn't looked into that. This was mostly geared towards local models so token cost isn't really a big deal, though latency might be. And if you didn't mean that then please elaborate :) | ||