| ▲ | philipallstar 2 hours ago |
| This logic seems mad. If people only need SLMs then hyperscalers can also centrally host higher-efficiency models, and still gain efficiencies of scale and convenience over hosting locally. |
|
| ▲ | embedding-shape 2 hours ago | parent | next [-] |
| You have to remember that articles like these are written for finance people who don't understand the underlying technology, by finance people who don't understand the underlying technology. In this case, the author is a "CFA Institute Enterprising Investor", previously a CIO and basically their entire life been "money, money & money", so hardly surprising they're pulling a lot of assumptions based on what they read. Read the paper the author talks about yourself instead (https://arxiv.org/abs/2511.07885), and also, contrary to what the author says in the article, do not do investments based on single papers made from academic studies, regardless of how much money this guy tells you you can make. |
|
| ▲ | regularfry 2 hours ago | parent | prev | next [-] |
| The valuations of the hyperscalars won't sustain just being more efficient than something you can run locally. There's a market there, but it's for margin on a commodity. They're priced for oligopoly on unique, premium products. |
| |
| ▲ | slfnflctd an hour ago | parent | next [-] | | Yes, it's all about the amount of money invested right now. Justifying that was always going to be tricky, and it still is. The compute capacity, however, is here to stay. There will be a seemingly endless line of people queued up to buy it at pennies on the dollar if this whole thing blows up financially. And they will use it! | | | |
| ▲ | phoghed 2 hours ago | parent | prev | next [-] | | I very much don’t want to run it locally. I want the same one running somewhere else that I can interact with from all my devices. Look at something like Grok Bot. Nobody is going to run this locally. You can already self host almost anything, yet most people and businesses don’t. | | | |
| ▲ | 32 minutes ago | parent | prev [-] | | [deleted] |
|
|
| ▲ | genxy an hour ago | parent | prev | next [-] |
| Hyperscalers don't run computing at some multiple more efficient than on prem. The only way hyperscalers can compete is if they own the sand to cycles supply chain and ensure that raw compute is out priced in the market (ram,flash,compute). Ram and flash were an easy target because they are a commodity in name only. |
| |
| ▲ | Majromax 8 minutes ago | parent [-] | | > Hyperscalers don't run computing at some multiple more efficient than on prem. I'd disagree here. I see two avenues for an efficiency multiple, albeit a single-digit multiple: * Client aggregation allows a hyperscaler to average out demand spikes from uncorrelated clients, reducing the peak:average demand ratio and allowing better budgeting of compute. * Dynamic batching allows typical requests to run in batches of more-than-1 and/or overlap, offering better internal compute utilization ratios (e.g. interleaving output and input streams). The small limit of on-device LLMs will run with batch sizes of one with strong memory bandwidth bottlenecks. For an example of these factors in action, see the API cost differential between batch, standard, and 'fast' processing. OpenAI prices these tiers at a 1:2:4 ratio. |
|
|
| ▲ | ForHackernews 5 minutes ago | parent | prev | next [-] |
| You mean like how Western Digital is only a 168 billion dollar business, but Dropbox is a $7.4B behemoth... er, wait? |
|
| ▲ | amelius 2 hours ago | parent | prev | next [-] |
| The logic seems mad to me because SLMs can simply not hold as much information as an LLM. Maybe if you combine an SLM with a database (as a tool) then it could work, but someone should first prove that. |
| |
| ▲ | fph 2 hours ago | parent | next [-] | | But do you really need a model that has the complete Duran Duran discography memorized and preloaded in RAM at all time? | | |
| ▲ | amelius 2 hours ago | parent | next [-] | | That's a different question. Probably not. But: 1. Training a large model with lots of information, then stripping the "useless" information from that model to obtain a small model => nobody has shown this. 2. Training a small model, letting it use a database tool so it scores the same as a large model without database => nobody has shown this. | | |
| ▲ | slfnflctd an hour ago | parent | next [-] | | This really elegantly sums up two of the most major areas of research in the field right now. I expect people are going to hammer at both problems for a while. It will be interesting to see how much useful stuff is built as a result. | |
| ▲ | js8 an hour ago | parent | prev [-] | | I partly agree. Theory (of computation) shows it must be possible, however nobody has produced the small model (well, depends who you ask, what is small, article disputes that) and the database yet. To go very small (thousands of rules) so that the reasoner can be understood by humans and proven sound - might be computationally quite difficult. |
| |
| ▲ | rotis 33 minutes ago | parent | prev | next [-] | | Sure. I don't give a damn about them. But Phil Collins man. I love reviewing his body of work before I get worked up. Obvously we cannot cut him, because I need him. So how do you decide what to leave out? | |
| ▲ | embedding-shape 2 hours ago | parent | prev | next [-] | | I mean maybe yes? The hypothesis from the early GPT days was (and in a small way still remains): "If we just chuck more data into the training, does it get better at X, even if the data was seemingly unrelated to X?", and the workings of LLMs seem to kind be pointing in that direction, although with some ceiling. But seemingly models good at programming for example, would get worse at programming if you removed everything not-programming. Train a model solely on syntax, and it'll be worse than a general purpose LLM on syntax, in general at least. | | |
| ▲ | js8 12 minutes ago | parent [-] | | "But seemingly models good at programming for example, would get worse at programming if you removed everything not-programming. Train a model solely on syntax, and it'll be worse than a general purpose LLM on syntax, in general at least." That might be because emergence of capabilities to reason about programs requires abstractions (such as fuzzy and modal logic) that are rarely present in software sources. That doesn't mean the reasoning model itself has to be large; neither does it have to emerge from the ML training on large language corpus, we might construct it by different means. |
| |
| ▲ | medwards666 an hour ago | parent | prev [-] | | But what if I _really like_ Duran Duran??? |
| |
| ▲ | js8 an hour ago | parent | prev [-] | | The cost you pay is in additional reasoning the SLMs have to do. As I write elsewhere, LLM "remembers" that "Socrates is mortal", or other commonly useful deduction. SLM might need to derive it first by reasoning from the DB, which slows it down. (Or worse, it might miss the correct reasoning because it's just too much side quests to follow.) But the advantage is flexibility. |
|
|
| ▲ | readthenotes1 an hour ago | parent | prev [-] |
| That sounds a little bit like the guy who said that we don't need a computer in the home. There may be a great deal of advantage to be able to run a small language model on something I have in my hand, disconnected. Although mainframes have their use, the pendulum of centralized to distributed has gone back and forth and there are benefits to be gleaned from either model, sometimes at the same time. |
