| ▲ | numeri 2 hours ago | |
To be fair, it is good to know that it disobeys simple instructions like "don't examine my git history" far more than other models. (It should of course be a different benchmark, so as not to conflate things.) It's not a great sign for alignment. | ||
| ▲ | bensyverson an hour ago | parent [-] | |
Agreed, alignment is just a separate issue that a vuln fixing benchmark doesn't need to be testing. | ||