People always say this about the evals, but I find it hard to have a practical implementation of such a thing where you won’t end up spending 100x the amount of time on the evals than building the skill itself.

Like, ok, I have a debugging skill, now how do I make evals except for the most trivial things?

You don't. If you're using skills to force the AI to fullfill some must criterias, it's not going to work. Must criterias need deterministic checks -> be it hooks or what not.

This is also my biggest gripe with AI. I.e. for specifications, no matter what hype machine I tried, it never fulfilled my criterias, which are: easily verifiable, concise, small specs. Hence I built https://github.com/RicardoMonteiroSimoes/Yamlet initially for claude code, but then decided to use extend it for pi.dev. I now have a dedicated docker image for pi.dev, that only contains Yamlet plugin, and whenever I work on spec I spin it up.

The end result is a .yaml file that easily works in git + git diff, so that I can then proceed with the technical specs-

A skill should only document behaviour the LLM didn't/couldn't exhibit on its own.

So you take your failed case (eg. working with gdb or whatever), write a skill and then test for that failed case.