Snake oil. Just ask the model, all these custom agents/skills haven't proven that useful in practice.

internet101010 a minute ago | parent | next [-]

Lol wut. One of first things people do at a company when they get enterprise LLM tools is share a skill with company-specific color palettes or standards for creating visualizations (I prefer Tufte's principles).

▲

jw1224 an hour ago | parent | prev | next [-]

Skills already are "just asking the model". Unless you'd prefer to type out the same instructions every single time?

Skills are literally just Markdown documents that get loaded into context when the /skill-name is invoked.

▲

Zetaphor 34 minutes ago | parent | next [-]

I think they're maybe confusing Skills and MCP servers

▲

dominotw 34 minutes ago | parent | prev [-]

i belive gp means llms produce what they see in training data/rl there isnt much too much customization you can do with skills.

they are being sold as more powerful than they are. Like llms are intelligent blank slates that can be customized with mere markdown files.

▲

calebkaiser 17 minutes ago | parent [-]

I don't understand this line of criticism exactly. By putting new information in the context window, you are materially changing the activations at your point of sampling, which is literally "customizing with mere markdown files."

Taken to the extreme, the attitude that there is some special incantation that will unlock all capabilities is silly, and a lot of the "prompt engineering" discourse is similarly kind of dumb, but in-context learning is clearly a real thing.

	▲	dominotw 11 minutes ago \| parent [-]
		even if that works one time you can never be sure that your customization is in place or fell out of context's important zone. you've reverted back to base llm behavior.

▲

coffeeaddict1 an hour ago | parent | prev | next [-]

I disagree. Not all skills are useless. For example, I sometime use Qt for GUI projects and I have found their skills [0] very useful to improve the quality and performance of my projects. I their absence, I would each time have to direct the agents to find the docs or specific tools, wasting tokens and thus decreasing the quality of the output.

[0] https://github.com/TheQtCompanyRnD/agent-skills

▲

pramodbiligiri an hour ago | parent | prev | next [-]

I don't think the idea of skills is quite snake oil. It seems you can change what LLM outputs next by what's called few-shot prompting or in-context learning: https://www.promptingguide.ai/techniques/fewshot

▲

theptip 18 minutes ago | parent | prev | next [-]

Nah. Skills are great. But you should write your own.

▲

john_strinlai an hour ago | parent | prev | next [-]

not that i know much about the effectiveness of these skill files, i find it odd to call something given for free "snake oil", which i thought referred to the sale of fraudulent products (to the benefit of the snake oil salesperson), typically around healthcare-related stuff.

	▲	dominotw 17 minutes ago \| parent [-]
		i think gp is calling skills snakeoil in genral

▲

beezlewax an hour ago | parent | prev | next [-]

I've found them useful for in house stuff where you are using a specific design system or architecture. But custom everything works best. Are that Claude works well on its own though at this point.

▲

wyre 34 minutes ago | parent | prev [-]

Ya, if im constantly asking a model to do TDD development, you know what would make it a lot easier? A skill.