Remix.run Logo
▲ calebkaiser 44 minutes ago

There's a lot of research along this particular line, if you're interested!

This is a dataset basically constructed by taking a bunch of DeepSeek R1 trajectories and filtering out the incredibly long rambling ones: https://huggingface.co/datasets/open-thoughts/OpenThoughts-1...

There's also a bunch of RL research around strategies to coax the student model towards more concise trajectories.

In general, a lot of people in industry are finding cost savings these days by tuning a smaller model over longer agent trajectories (not exactly the same as distilling the reasoning traces, since those aren't exposed by many frontier labs anymore) collected from their production agents. Typically works best when you have a pretty workflow-y agent whose tasks all share a similar shape.