The non-disclosure of the training set may raise legitimate skepticism about the training set containing tasks similar to those in the test set. That would make the benchmark result moot. There are benchmarks that by design can't be manipulated by incorporating similar tasks in the training set, for example those that dynamically generate tests and only reveal the global result. Testing on one of those would give a strong measure of the strength of this legal AI framework.

I think using adapters for different legal systems could improve results by 2% or 3% at a similar cost (perhaps under 1% of additional parameters) and allow for independent improvements. If the GLM team hasn't already considered this and adopts my suggestion, tips would be welcome.