| ▲ | wtetzner 2 days ago |
| Even if he was wrong about those statements, exploring other types of models isn't necessarily a bad thing. Especially if we can find ones that perform well with less demanding hardware requirements. |
|
| ▲ | dragonwriter a day ago | parent | next [-] |
| Its not a bad thing, but exploratory research as opposed to reaearch that finetunes along a path already paved by exploratory research is something that doesn’t usually happen much in for-profit product-oriented firms when there is such a path available; it happens in academia and corporate labs pick it up once it reaches the point where the basics show commercial promise. CS is a little bit of an exception because the capital cost of basic research isn’t, proportionally, as high, and because the promise-to-product path is often short enough that having done the basic work in house can be a meaningful advantage in getting to market first (though, see Google on much of modern AI for a counterexample). But even in computing, when there is a promising way forward that the basic reaearch has been done on, its hard to justify side quests. |
| |
| ▲ | bigfatkitten a day ago | parent | next [-] | | Those exploratory "side quests" in industry are how we got the transistor, the liquid crystal display, Shannon's information theory, the laser and practical photovoltaic cells, among many other foundational technologies upon which we rely today. | | |
| ▲ | publlus_enigma a day ago | parent | next [-] | | Blue LEDs too, which have transformed lighting and displays. | | |
| ▲ | Der_Einzige 12 hours ago | parent [-] | | Blue LEDs have been a massive net negative for humanity. Blue light hazard is very real (why blue lasers kill eyesight instantly and green and red often don’t in accidental exposure). All display technology past CRTs has been a mistake. The vectrex in the 1980s had better refresh rates, color fidelity, beautiful phosphor glow, etc - and we abandoned it for a horrible pixel based future we are now in. We should return to vectors and lasers. |
| |
| ▲ | catlifeonmars a day ago | parent | prev | next [-] | | Laser printers were a side project that almost got the inventor fired. | | |
| ▲ | bitwize a day ago | parent [-] | | And then they made back 200x the operating expenses of the lab (Xerox PARC) they came from over its entire lifetime. |
| |
| ▲ | WalterBright a day ago | parent | prev [-] | | Jet engines, too. |
| |
| ▲ | RetroTechie a day ago | parent | prev | next [-] | | > and corporate labs pick it up once it reaches the point where the basics show commercial promise. How does that work when the research team moved elsewhere? Re-hire talent? That takes time - the 1 thing you don't have in a fast-moving field like AI. Post sums it up nicely: "Do the math—how much is it worth to halve training and inference cost given that the industry has investments approaching a trillion dollars?" How much is these researchers' salaries, compared to what MSFT spends on compute? Chances are it's peanuts in comparison. | | |
| ▲ | lovich a day ago | parent [-] | | Yes but you forget that spending on compute is spending on machines that the company owns while spending on researchers is spending on other apes that trigger the social hierarchies of executives brains and who have the dreaded key word of “rights” Companies will try to ruthlessly rid themselves of labor as long as those members of labor have agency and rights. If someone disagrees they can convince me by explaining why a company wouldn’t choose to spend X on a machine they wholly own vs X on an employee they don’t. There’s some room for argument that employees can do more than machines can but in the current AI zeitgeist arguing that we have machines with equivalent to human intelligence I don’t see how you wouldn’t see this coming if you believed the zeitgeist. | | |
| ▲ | RetroTechie 18 hours ago | parent [-] | | If compute:people were roughly equal spend, I'd agree with that thinking. But we all know AI's 100s of billions $$ spend is on datacenters+GPUs, not people. In a way this suggests that MS doesn't aim to be a leader in AI <the core tech>. Only in its applications + related services (read: profit to be had there). Nothing wrong with that btw. But understandable that fundamental researchers then pack up & leave. | | |
| ▲ | lovich 4 hours ago | parent [-] | | > But we all know AI's 100s of billions $$ spend is on datacenters+GPUs, not people. That’s bolstering my argument. Companies are willing to spend a shit ton on machines. They are not only unwilling to spend on labor but are actively seeking ways to remove labor spend, even though this new technology supposedly makes every unit of labor significantly more productive and thus profitable. This is a cultural choice not an economic one. If I see a company laying off people while admitting they’ve maximized a niche and automation lets them increase profitably, or I see them go on a hiring spree because labor is now netting them 10x-100x-1000x-whatever productivity increase they claim, then I will believe them about the value add of AI. Currently I see companies using AI’s productivity increase as a reason to cut labor and achieve roughly the same productivity. They aren’t competing, they just want to cut out workers. |
|
|
| |
| ▲ | pyrale a day ago | parent | prev | next [-] | | > CS is a little bit of an exception because the capital cost of basic research isn’t, proportionally, as high, and because the promise-to-product path is often short enough that having done the basic work in house can be a meaningful advantage in getting to market first (though, see Google on much of modern AI for a counterexample) ...And because they have monopoly money. Basically, research labs are usually vanity spending for corporations with too much money. Xerox parc was funded by a monopoly. Bell labs was funded by a monopoly. Google funds its moonshots with its monopoly. | | |
| ▲ | robotresearcher a day ago | parent [-] | | Bell and Xerox became powerful because they invented game-changing technologies. They were fundamentally research-based organizations from the start. |
| |
| ▲ | vlovich123 a day ago | parent | prev | next [-] | | Remind me again where the transformer comes from? | | | |
| ▲ | eli_gottlieb a day ago | parent | prev [-] | | > it happens in academia and corporate labs pick it up once it reaches the point where the basics show commercial promise. Ah, well, good thing we've almost entirely defunded academic science. |
|
|
| ▲ | dfdydx a day ago | parent | prev | next [-] |
| It’s a simple exploration/exploitation trade off. Always explore, never exploit is probably not the optimal strategy every time. |
| |
| ▲ | bmitc a day ago | parent [-] | | It seems to me that everyone is in "always exploit" mode. Seems weird when it's become apparent that LLMs aren't really special sauce outside of just having the compute. |
|
|
| ▲ | gscott a day ago | parent | prev | next [-] |
| The right idea usually comes from a wrong one. |
|
| ▲ | frays a day ago | parent | prev [-] |
| The issue isn't the exploration itself, it's the opportunity cost. When a figure of Yann LeCun's influence pushes a specific direction, it pulls key talent and resources away from mainstream LLM work. That kind of organizational distraction has a massive cost, and would've stopped Meta from delivering a polished product like Muse sooner. |
| |
| ▲ | thatsabadlook a day ago | parent | next [-] | | The mainstream is wrong and in a lot of ways hazardous. He had every right to say he wasn't willing to sign is name on the wrong technology when he new better. When you hire smart people you listen to them. Elsewise you get what you deserve. | | |
| ▲ | frozenseven a day ago | parent [-] | | >when he new better He was against LLMs, reinforcement learning, etc. Claiming that those are a dead-end. He was wrong. His own "JEPA" ideas have basically gone nowhere. Meta was right in moving on without him. | | |
| ▲ | ForHackernews a day ago | parent [-] | | I'm not sure he was wrong. No one has solved the fundamental problem with LLMs, which is that they cannot be trusted. | | |
| ▲ | jack_pp a day ago | parent | next [-] | | Most humans can not be trusted either, that's why you have processes in place to test and review work | | |
| ▲ | johnnyanmac a day ago | parent [-] | | Sure. And we made efforts over millenia to ensure that. Let me know when LLM's get those. Otherwise I trust it as much as a human under anarchic rule. Probably less because I know exactly who controls it. |
| |
| ▲ | frozenseven a day ago | parent | prev [-] | | Ok, so we're talking about applying connectionist systems/models to the messiness of the real world. And "trust" isn't an on-off switch. The goal is to build systems that are smarter and more general. It's a process. I don't think even Yann LeCun has made any strong claims beyond that. He just thinks that his way of doing deep learning is better. |
|
|
| |
| ▲ | esseph a day ago | parent | prev | next [-] | | It's not a distraction if the research pans out and then they become the market leader for the next several years or more. | | |
| ▲ | siva7 a day ago | parent | next [-] | | The job of the CEO is to assess the research risk correctly. Zuck made the right bet here to replace him. | | |
| ▲ | hvb2 a day ago | parent | next [-] | | The guy that named his company after his own diversion? It's his call for sure, but rather funny as an argument. | | | |
| ▲ | esseph 16 hours ago | parent | prev [-] | | Oh yeah? How's the metaverse doing? |
| |
| ▲ | shadowpho a day ago | parent | prev | next [-] | | Ok yeah but it’s not a certain bet. Resources are limited and you gotta allocate them efficiently. 10% chance to be market leader doesn’t sound as good as 60% second place to them for example | | |
| ▲ | esseph a day ago | parent [-] | | This is a prime example of why large companies have a problem innovating. Once you get close to the top, the risks don't look worth it. |
| |
| ▲ | geertj a day ago | parent | prev | next [-] | | > It's not a distraction if the research pans out It is because even if it would pan out, it would mean temporarily losing AI leadership, reducing stock price, losing the ability to hire talent, and by virtue of that losing the opportunity permanently. | |
| ▲ | a day ago | parent | prev [-] | | [deleted] |
| |
| ▲ | gamblor956 a day ago | parent | prev [-] | | LeCun wasn't wrong. He simply had higher standards for results and accountability that LLMs are, by design, incapable of reaching. Zuckerberg wanted to placate his shareholders with a shiny new thing now. would've stopped Meta from delivering a polished product like Muse sooner. Muse is not polished, unless usability and programming standards have fallen so far in the LLM vibe coding era that a pre-alpha state is now considered polished. | | |
| ▲ | teruakohatu a day ago | parent | next [-] | | You will find brilliant scientists in every discipline who are not the least bit interested in a particular branch of research because they/their lab didn’t invent it. That’s fine for public research but not helpful private research. | |
| ▲ | solyin a day ago | parent | prev | next [-] | | Compared to the user buying a Mac mini and following a tutorial off Twitter to install some random shit they found online? Muse is imperfect, as is all software, but calling it pre-alpha is a bit much. | |
| ▲ | a day ago | parent | prev [-] | | [deleted] |
|
|