| ▲ | EgoIncarnate 10 days ago |
| The article seems to be be based on the current limitations of LLMs. I don't think YC and other VCs are betting on what LLMs can do today, I think they are betting on what they might be able to do in the future. As we've seen in the recent past, it's difficult to predict what the possibilities are for LLMS and what limitations will hold. Currently it seems pure scaling won't be enough, but I don't think we've reached the limits with synthetic data and reasoning. |
|
| ▲ | DeathArrow 10 days ago | parent | next [-] |
| >The article seems to be be based on the current limitations of LLMs. I don't think YC and other VCs are betting on what LLMs can do today, I think they are betting on what they might be able to do in the future. Do we know what LLMs will be able to do in the future? And even if we know, the startups have to work with what they have now, until that future comes. The article states that there's not much to work with. |
| |
| ▲ | brookst 10 days ago | parent [-] | | Show me a successful startup that was predicated on the tech they’re working with not advancing? | | |
| ▲ | rsynnott 10 days ago | parent | next [-] | | Most successful startups were able to make the thing that they wanted to make, as a startup, with existing tech. It might have a limited market that was expected to become less limited (a web app in 1996, say), but it was possible to make the thing. This idea of “we’re a startup; we can’t actually make anything useful now, but once the tech we use becomes magic any day now we might be able to make something!” is basically a new phenomenon. | |
| ▲ | jeltz 10 days ago | parent | prev | next [-] | | Most? I can list tens of them easily. For example what advancements were required for Slack to be successful? Or Spotify (they got more successful due to smartphones and cheaper bandwidth but the business was solid before that)? Or Shopify? | | |
| ▲ | brookst 10 days ago | parent [-] | | Slack bet on ubiquitous, continuous internet access. Spotify bet on bandwidth costs falling to effectively zero. Shopify bet on D2C rising because improved search engines, increased internet shopping (itself a result of several tech trends plus demographic changes). For a counterexample I think I’d look to non-tech companies. OrangeTheory maybe? |
| |
| ▲ | teamonkey 9 days ago | parent | prev | next [-] | | The notion of a startup gaining funding to develop a fantasy into reality is relatively new. It used to be that startups would be created to do something different with existing tech or to commercialise a newly-discovered - but real - innovation. | |
| ▲ | talldayo 10 days ago | parent | prev [-] | | Every single software service that has ever provided an Android or iOS application, for starters. |
|
|
|
| ▲ | kokanee 10 days ago | parent | prev | next [-] |
| Tomorrow, LLMs will be able to perform slightly below-average versions of whatever humans are capable of doing tomorrow. Because they work by predicting what a human would produce based on training data. |
| |
| ▲ | herval 10 days ago | parent | next [-] | | This severely discounts the fact that you’re comparing a model that _knows the average about everything_ to a single human’s capabilit. Also they can do it instantly, instead of having to coordinate many humans over long periods of time. You can’t straight up compare one LLM to one human | | |
| ▲ | namaria 10 days ago | parent [-] | | "Knows the average relationship amongst all words in the training data" ftfy | | |
| ▲ | herval 9 days ago | parent [-] | | it seems that's sufficient to do a lot of things better than the average human - including coding, writing, creating poetry, summarizing and explaining things... | | |
| ▲ | namaria 9 days ago | parent [-] | | A human specialized in any of those things vastly outperforms the average human let alone an LLM. | | |
|
|
| |
| ▲ | steveBK123 10 days ago | parent | prev [-] | | It's worth considering 1) all the domains there is no training data Many professions are far less digital than software, protect IP more, and are much more akin to an apprenticeship system. 2) the adaptability of humans in learning vs any AI Think about how many years we have been trying to train cars to drive, but humans do it with a 50 hours training course. 3) humans ability to innovate vs AIs ability to replicate A lot of creative work is adaptation, but humans do far more than that in synthesizing different ideas to create completely new works. Could an LLM produce the 37th Marvel movie? Yes probably. Could an LLM create.. Inception? Probably not. |
|
|
| ▲ | layer8 10 days ago | parent | prev | next [-] |
| You could replace “LLM” in your comment with lots of other technologies. Why bet on LLMs in particular to escape their limitations in the near term? |
| |
| ▲ | samatman 10 days ago | parent [-] | | Because YCombinator is all about r-selecting startup ideas, and making it back on a few of them generating totally outsized upside. I think that LLMs are plateauing, but I'm less confident that this necessarily means the capabilities we're using LLMs for right now will also plateau. That is to say it's distinctly possible that all the talent and money sloshing around right now will line up a new breakthrough architecture in time to keep capabilities marching forward at a good pace. But if I had $100 million, and could bet $200 thousand that someone can make me billions on machine learning chip design or whatever, I'd probably entertain that bet. It's a numbers game. | | |
| ▲ | namaria 10 days ago | parent [-] | | > But if I had $100 million, and could bet $200 thousand that someone can make me billions on machine learning chip design or whatever, I'd probably entertain that bet. It's a numbers game. Problem with this reasoning is twofold: start-ups will overfit to getting your money instead of creating real advances; competition amongst them will drive up the investment costs. Pretty much what has been happening. |
|
|
|
| ▲ | KaiserPro 10 days ago | parent | prev [-] |
| > I think they are betting on what they might be able to do in the future. Yeah, blind hope and a bit of smoke and lighting. > but I don't think we've reached the limits with synthetic data Synthetic data, at least for visual stuff can, in some cases provide the majority of training data. For $work, we can have say 100k video sequences to train a model, they can then be fine tuned on say 2k real videos. That gets it to be slightly under the same quality as if it was train on pure real video. So I'm not that hopeful that synthetic data will provide a breakthrough. I think the current architecture of LLMs are the limitation. They are fundamentally a sequence machine and are not capable of short, or medium term learning. context windows kinda makes up for that, but it doesn't alter the starting state of the model. |
