| ▲ | benjiro29 an hour ago | |
!! Be careful when testing the model. A lot of people are testing it, and reporting disappointed results / benchmaxxxing claim. But do not realize that thinking has a issue with the default configuration. Important - make sure that THINKING is enabled. By default it wasn't although I was passing the flag --default-chat-template-kwargs '{"enable_thinking": true}' in vllm recipe. The generation_config.json file that is included has by default max_new_tokens as 32k which seems to be cutting off thinking altogether so increase it. At first I was very disappointed with the output I was seeing, but once thinking is enabled, the code quality seems to be MUCH better. More real world testing to be done. https://www.reddit.com/r/LocalLLaMA/comments/1v2pg99/laguna_... | ||
| ▲ | voxgen an hour ago | parent | next [-] | |
Even the official provider on OpenRouter seems to have this issue. Hope it's an easy fix for them. | ||
| ▲ | nshotton 32 minutes ago | parent | prev [-] | |
Thanks for posting this, it made a huge difference tweaking the recipe. | ||