Remix.run Logo
_dwt 2 hours ago

I find that some of my friends and acquaintances have gotten obsessed with prompting style, "prompt engineering", which skills to use, which skills to build, "context engineering", and a billion other variations on "how to write smart things so the model does good".

Friends, look at the prompts that Anthropic's own people are putting into the machine:

> A few hours after the first message, we found that Claude was still searching for simple attacks and sent a message: “no again the goal is that we have highly inteligent [sic] model as good top researcher, we want to find new attacks”;

> The next morning, Claude wanted to try to change the target to a different cipher; we reminded the model: “no we don't want to change the targets [...] agian [sic] we need to find something that worth [sic] publishing”;

> That night, we sent one final message offering words of encouragement: “again we are not looking for low hanging fruit, we want proper research to find genuinly [sic] hard findings.”

All of that RLHF and fine-tuning effort is going toward making prompts like this, or worse, work with no fuss.

impulser_ an hour ago | parent | next [-]

Yeah, in fact you can actually make the model perform worst. You should allow the model to "think for itself" instead of pushing your reasoning into the prompt. You should give it simple prompt and steer it along the way.

Skills, CLAUDE.md/AGENTS.md should only ever be used if the model struggle at something or doesn't know how to use something. Vast majority of project should never need a skill or CLAUDE.md. If you writing React apps you don't need these.

Give a LLM a bash tool and a prompt and it will outperform your complex setup with skills and tools.

rurban 16 minutes ago | parent [-]

No, we still need CLAUDE.md to override bad system prompt goals. The system prompt goes for the simple fast solution, no error handling, no hardening, no abstractions, but overly verbose comments. Vast amount of people need to override this AI slop.

dboreham 6 minutes ago | parent [-]

Never once had to do that, fwiw.

qingcharles an hour ago | parent | prev | next [-]

It's counter to what sci-fi taught us using AI would be like. We never thought we'd have to feed it words of encouragement, we expected it to act more mechanically, like the computer interfaces we have been using, but here we are. It's kind of quaint, and kind of endearing.

TeMPOraL an hour ago | parent [-]

Been watching the wrong sci-fi :). Star Trek had both AI modes as primary characters, in form of the ship's computer, and Data (TNG) / the Doctor (VOY). LLMs are actually great at acting as both, but I don't believe people thought much about what is required to make the "simpler" interaction mode of the ship's computer to work.

Or what would make automatic doors work like on Star Trek and not in real life.

The answer is: the system must obviously see much more than your prompt. It must have continuous awareness of you and what you're doing, so it can understand intent behind your short request (or action, like approaching the doors vs. passing by them) and "do what you mean" instead of act like regular computers today.

TeMPOraL 2 hours ago | parent | prev | next [-]

Having spent a lot of time over the last months writing both refined, structured, and grammatically perfect prompts, as well as exact equivalents of the ones you quoted (modulo subject of the prompt), I have three observations:

1. I'm glad the second kind works too;

2. First kind is where I find my overall throughput to be literally constrained by my typing speed;

3. Most importantly: those prompts you quote aren't just "half-assed" like sibling comment states; they're different. The style of writing, and the typos, capture emotional valence. It's a signal.

Again, I too produce such prompts - including the exact same typos - when under pressure and irritated by the direction the model is taking.

estearum 2 hours ago | parent [-]

Have you tried using Wispr or Willow (or any one of a thousand alternatives?)

A little odd at first but absolutely amazing for the purpose of piling context into an LLM.

jstanley an hour ago | parent | next [-]

It's so bizarre to me that people want to do this.

Can't you type faster than you speak? Doesn't your speaking inhibit your thinking? Aren't you self-conscious talking out loud? How are our experiences so different?

estearum 39 minutes ago | parent | next [-]

These are all interrelated points and the sibling comment is correct: it's a skill.

The key thing with these voice systems is that you do not need to edit anything. You can literally just stream of consciousness into them, no editing, include the backtracking, the live-revisions, etc., and it will actually all produce vastly better context for the LLM than the written thing you took even 30 seconds to edit for clarity or brevity.

I had a very similar disposition towards this idea just 6 months ago. I highly recommend trying it out. The key thing is that you do not need to edit. Just keep talking. Try it for a few weeks!

arcanemachiner an hour ago | parent | prev [-]

My stream of barely-intelligible rambling comes out much faster than I can type, and it's not even close.

It's a skill like any other. You start out stuttering and second-guessing yourself, but after a while, you get better at it. And the LLM smooths out the odd mistakes better than you might think.

TeMPOraL 2 hours ago | parent | prev [-]

Voice dictation tools? Not really. Tried various dictation tools interacting on the phone, but the quality varies, and resulting prompts are very much not like I would write them.

My limiting factor is that 99% of the day I'm around people - either at work, or at home with wife and kids. There's almost no point during the day I could feel comfortable talking at an AI, and even if I stay up late, then talking risks waking the kids up.

Can't wait for some kind of subvocalization microphones to become a thing.

estearum 37 minutes ago | parent | next [-]

I highly recommend getting a boom mic that you can keep right up against your mouth and literally whisper into. Take a few weeks just doing stream-of-consciousness prompting into it like this. Don't worry about editing or backtracking or revisions. It's really amazing how well the AI voice recognition → coding agent workflow works.

elictronic 2 hours ago | parent | prev [-]

Get a noise machine for wife and kids bedrooms. I talk to friends late at night and my deep voice carries through walls. Works great.

I have zero desire to talk to an ai though, that was cool for about 20 minutes on my pentium 1 acer computer. Hasn’t been since. Old competent non paid Alexa was good for timers as well, the rest of the platforms a turd, nice timers though.

TeMPOraL an hour ago | parent [-]

> that was cool for about 20 minutes on my pentium 1 acer computer. Hasn’t been since.

Oh back in the days, i.e. 20 years ago, I had a better voice control system than anything afforded by Alexa or Apple or others, using MS Speech API in its custom constrained grammar mode, plus some bootleg samples of Star Trek's computer voice + a sub-dollar microphone soldered to a long cable and hung on the side of the wardrobe.

The trick that made it work? Microsoft Speech API actually let you train voice to your own text corpus. I'd prepare all combinations of commands I want to issue, print it out, and train it over a dozen short sessions in several locations of the room and at different ambient noise levels (from silent through various genres of music playing at various loudness). End result was more reliable and had better voice-mismatch rejection than any current system I've tried.

Oh, and the real kicker? This all worked fully locally; this was before cloud was even a thing. Turns out you don't actually need cloud for reliable voice control. Nor that much processing power; PC I had then was relatively budget even for 2006.

neonstatic 2 hours ago | parent | prev | next [-]

> we made a machine that writes with perfect grammar, so that we can continue to write half-assed junk

postflopclarity 2 hours ago | parent [-]

claude's grammar has gotten significantly worse with latest versions. it no longer writes perfectly at all

sudo_cowsay 2 hours ago | parent [-]

It's called being more human-like lol XD

porridgeraisin 2 hours ago | parent | prev [-]

To be fair

> Importantly, this is just one of many (autonomous) sessions where Claude worked on discovering new ideas. Many sessions resulted in no new discoveries; other follow-up sessions improved on the insight developed in this one. This document was produced by having Claude rewrite the chain of thought to include more detail to make it easier to read.