| ▲ | everfree 6 hours ago |
| And if you give the LLM a simple calculator MCP, then the calculator's outperformance disappears. |
|
| ▲ | bayindirh 6 hours ago | parent | next [-] |
| Nope. It'll be more efficient and faster. |
| |
| ▲ | everfree 6 hours ago | parent [-] | | That's just semantics around the word "performance". We're messaging each other right now inside a heavily sandboxed virtual environment that interprets XML-like markup and runs a text-based JITted scripting language from the 90s. It doesn't really matter that it's extremely inefficient, since it's still effectively instant. | | |
| ▲ | bayindirh 6 hours ago | parent [-] | | Actually not. Watt per task is a metric bothers more than LLM operators and users. We care about it a lot, and working a lot to measure and improve it, and we're not doing it in context of AI or LLMs. Effectively instant means nothing, since everything is fast for small n. I love to give this example from a codebase I have worked on during my Ph.D. The problem which took half an hour under Matlab was done in ~40 seconds with mine, however, a thousand-step version of the same problem would take 11 hours. Saving a single second from that 40 second loop would shave my work by ~20 minutes, and this will compound for bigger problems still. Just because we see the results instantly, doesn't mean it's fast or efficient by any means. It means we might forego optimizing it for sufficiently small n, if we don't care about that part of the code. That's all. | | |
| ▲ | everfree 6 hours ago | parent [-] | | Generally when I want to use a calculator, I use an app on my phone or computer. The watts per task is abysmal compared to a desk calculator. The app is less efficient only in an abstract, impractical way. | | |
| ▲ | bayindirh 5 hours ago | parent [-] | | > The watts per task is abysmal compared to a desk calculator. It depends on the calculator and task at hand. Consider graphing, integration or any programmable needs, and they might converge. ALU of a processor is pretty efficient at what it does. > The app is less efficient only in an abstract, impractical way. Again, depends, as I said above. You tend to brush efficiency away because you only mentally measure time to completion, only, which is understandable. |
|
|
|
|
|
| ▲ | guelo 6 hours ago | parent | prev | next [-] |
| Imagine performing literally trillions of matrix multiplications in order to figure out the answer to 1 + 2. |
| |
| ▲ | everfree 6 hours ago | parent | next [-] | | Your computer/phone just performed millions of them in order to display the contents of my comment to you. | |
| ▲ | cubefox 5 hours ago | parent | prev [-] | | Pretty much what our brain does. |
|
|
| ▲ | wrigby 6 hours ago | parent | prev [-] |
| Sure, but what about performance per joule? |
| |
| ▲ | everfree 6 hours ago | parent [-] | | If performance per joule is important in this loosely-defined hypothetical scenario, then it's important. If it's not, then it's not. |
|
