Remix.run Logo
gabosarmiento an hour ago

What's the benchmark against graphify?

shrishdwi 35 minutes ago | parent [-]

In our tests, graft found the right code about twice as often as graphify (MRR 0.73 vs 0.38, recall@10 54% vs 20%). The difference is simple: graft searches inside the actual code, while graphify only looks at names and file paths.

You feel it while working too. graft hands the agent the exact file and line it needs for each question, so Claude keeps using graft.

Same is true for any other cli tools, claude never actually uses them as it's trained to use grep. but for graft as we set the directive to use graft at the start of the session and before the turn, claude just knows graft exists and also get the relevant context without it going and calling tools it was not trained on.

gabosarmiento 8 minutes ago | parent [-]

Thanks, that makes sense. The “Claude actually uses it” part is especially interesting. I actually find it better this approach don’t add another skill Claude might ignore, but make better retrieval the default.