| |
| ▲ | bayindirh 5 hours ago | parent [-] | | Actually not. Watt per task is a metric bothers more than LLM operators and users. We care about it a lot, and working a lot to measure and improve it, and we're not doing it in context of AI or LLMs. Effectively instant means nothing, since everything is fast for small n. I love to give this example from a codebase I have worked on during my Ph.D. The problem which took half an hour under Matlab was done in ~40 seconds with mine, however, a thousand-step version of the same problem would take 11 hours. Saving a single second from that 40 second loop would shave my work by ~20 minutes, and this will compound for bigger problems still. Just because we see the results instantly, doesn't mean it's fast or efficient by any means. It means we might forego optimizing it for sufficiently small n, if we don't care about that part of the code. That's all. | | |
| ▲ | everfree 5 hours ago | parent [-] | | Generally when I want to use a calculator, I use an app on my phone or computer. The watts per task is abysmal compared to a desk calculator. The app is less efficient only in an abstract, impractical way. | | |
| ▲ | bayindirh 5 hours ago | parent [-] | | > The watts per task is abysmal compared to a desk calculator. It depends on the calculator and task at hand. Consider graphing, integration or any programmable needs, and they might converge. ALU of a processor is pretty efficient at what it does. > The app is less efficient only in an abstract, impractical way. Again, depends, as I said above. You tend to brush efficiency away because you only mentally measure time to completion, only, which is understandable. |
|
|
|
