|
| ▲ | jcul 2 hours ago | parent | next [-] |
| It's crazy.
Are they doing any precomputing as you type, I wonder if you paste a block of text is it the same speed. |
| |
| ▲ | nerdsniper 2 hours ago | parent | next [-] | | I pasted and instantly hit enter on this prompt: "I generated a filter set using REW v5.31.3 using real-world sweep tone measurements from the room I'm listening in . How can I use it as my MacOS output equalizer so that my spotify music is adjusted for this room and speakers" and it gave a very reasonable answer in non-perceptible time. | |
| ▲ | throwuxiytayq 2 hours ago | parent | prev [-] | | No, it really does take ~0.03s to generate the answer. Try your browser's developer tools and watch the requests. |
|
|
| ▲ | 8cvor6j844qw_d6 2 hours ago | parent | prev | next [-] |
| I'd like to imagine the things that can be done with this speed and the current frontier models. |
| |
| ▲ | throwup238 an hour ago | parent | next [-] | | Fully interactive games where you can talk to every NPC by text or voice and have an LLM drive the story (with your own meta prompts to guide it, if you so wish). Maybe even have them generate assets on the fly too. I’m still trying to figure out coding agents. I can’t even begin to imagine the things it would enable. Even the most mundane ideas like LLMs-in-HiFreq-trading have huge implications. | |
| ▲ | ElijahLynn 2 hours ago | parent | prev | next [-] | | Truth, it feels like we're in the dial-up age of LLMs right now. And this Jimmy AI is fiber. | |
| ▲ | Dig1t 2 hours ago | parent | prev [-] | | Seriously, if Fable or even Opus was this fast that would be a real game changer. | | |
| ▲ | reducesuffering 34 minutes ago | parent [-] | | RSI will be models better than Fable running faster than this, you won't even need a human in the loop to figure out what to do. The high level goal will be accomplished better than the human in an instant |
|
|
|
| ▲ | kooi 29 minutes ago | parent | prev | next [-] |
| "Stochastic gradient descent algorithm in Haskel" "LMS algorithm in bash" Just barfed it up lol. Amazing. |
|
| ▲ | baal80spam 34 minutes ago | parent | prev | next [-] |
| Just wow, I made a similar request. Result: Generated in 0.042s • 14,201 tok/s This is crazy. |
|
| ▲ | HDThoreaun 2 hours ago | parent | prev [-] |
| For what its worth the frontier lab models can surely be a lot faster if they wanted them to be but theyre supply constrained so theyre doing stuff like multi tenancy. Since you cant self host them no one outside the labs really knows speed as a solo tenant |
| |
| ▲ | ericd 28 minutes ago | parent [-] | | You can kind of get a sense by running these things at home - I'm currently running Laguna. One interesting thing is that per stream doesn't actually slow down that much with multiple concurrents, because the bottleneck remains the memory bandwidth until pretty significant request depths, and then eventually you hit the GPU's limits. It's one of the big forces that pushes for centralization in this stuff, the fixed costs to run one are huge, the marginal costs of additional tenants, relatively small. |
|
