Remix.run Logo
rglover 4 days ago

What I've found works best:

1. Assume that any model will start to lose focus beyond 50K-100K tokens (even with a huge context window).

2. Be gluttonous with chats. At the first sign of confusion or mistakes, tell it to generate a new prompt and move to a new chat.

3. Write detailed prompts with clear expectations (from how the code should be written to the specific implementation that's required). Combine these with context like docs to get a fairly consistent hit rate.

4. Use tools like Cline that let you switch between an "Act" and "Plan" mode. This saves a ton of tokens but also avoids the LLM getting stuck on a loop when it's debugging.

I recently wrote this short blog post related to this: https://ryanglover.net/blog/treat-the-ai-like-it-s-yourself

The above approach helped me to implement a full-blown database wrapper around LMDB for Node.js in ~2 weeks of slow back-and-forth (link to code in post for those who are curious).