That's like saying the only way to do real engineering is with Google-scale Borg deployments. You can do quite a lot on very little hardware, r/StableDiffusion is a prime example.

▲

oersted 2 hours ago | parent | next [-]

You can do plenty of "real engineering" under normal conditions. But specifically when it comes to LLM engineering, no there's really not much you can do, they are called "large" for a reason. You can play around at small scale, but those lessons you learn will not be very relevant to the real problems in the market.

Sure you can gradually climb the ladder by demonstrating your skills bit by bit and getting access to more resources. It has very good prospects if you do manage to push through. But it's a hard and risky path, and you will not be able to get any interesting results for the longest time.

For a young middle-class student, it just doesn't make much sense. You can do much more impressive and impactful things with your time without getting into that black hole.

I know how to build an LLM, I know plenty of fellow young engineers that do too. It's really not that complex. But they can't do much with it without capital or access.

Good engineering has never been a bottleneck in this field, it's been all about having access to capital and taking smart but dangerous risks burning it on compute, without much idea of how long you need to keep burning for. There's still no end in sight, some are still managing to convince investors and keep burning, and we are seeing progress, but the business case is still unclear. If you want to get in that game, go ahead, but it's not something I would advice the average young engineer.

▲

stymaar 44 minutes ago | parent | next [-]

> . But specifically when it comes to LLM engineering, no there's really not much you can do, they are called "large"

The “large” qualifier dates back to pre-transformer language models, where even training a multi-million model was hard due to how poorly it scaled. GPT-2 was a large language model, despite being only 124 millions parameters.

Due to how much high quality data is readily available, anyone can now train a sub-billion (L?)LM on commodity hardware.

And I'm personally convinced that pretty much any enterprise use-case of an LLM (except coding) is better served by a fine-tuned small (<2B) model that is trained specifically on the task, rather than a generalist frontier model, so learning the engineering around fine-tuning is a key skill that companies will realize they need sooner than later.

	▲	mike_hearn 17 minutes ago \| parent [-]
		Why? Oersted is correct, for any size class you can find an LLM that is free and well trained at this point. They are highly adaptable even without fine tuning, in-context learning is still superior to fine tuning in most cases also. And real world fine tuning is mostly about data gathering and cleaning. The actual adapter training is automated and put behind simple APIs. I saw my first language model in action in 2014, I was writing blog posts about them back in 2016. In recent years, like many of us, I spent some time learning ML frameworks to see if it'd be a fun career pivot. Eventually I concluded a few things: 1. It doesn't seem especially creative. All the frontier labs have nearly identical model personalities, capabilities and even app designs. We're not seeing them differentiate from each other, implying that the design space might not be that large. In which case the opinions and unique approaches of specific engineers aren't that important, they are interchangeable at the right level of skill, and what to do next is usually obvious to everyone. There are counterpoints, like the rumours that Noam Shazeer came back to Google and immediately fixed some critical bug in the Gemini pipeline, and I totally believe the span of skill in ML is huge. And in the past it seems that creativity and ideation mattered a lot more. If you read about how the transformer paper was written, that's a great example of what really interesting work looks like. But nowadays that skill seems to surface in optimization and debugging more than something that will really help you stand out long term. 2. It doesn't seem like an especially big job market. A lot of ML jobs were wiped out in recent years by the rise of LLMs. Lots of NLP specialists etc were suddenly replaceable with a cheap API. The jobs that remain have compacted into a small number of companies. It's a small community which greatly increases career risk, especially as so many are unprofitable and/or have strong ideological requirements. 3. Using the models on the other hand is a very large design space, and has a lot of scope for creativity. I see use cases for AI everywhere, but most companies seem to stop at putting a chatbot on their website or asking Copilot to rewrite an email before they send it. A lot of companies have hollowed out their IT departments over the past twenty years. It feels like a new golden age of consulting work could be upon us. I really don't think I'd tell a 17 year old to learn how to train LLMs. Learn how they work and how to use them, sure, absolutely.

▲

reacharavindh an hour ago | parent | prev | next [-]

> I know how to build an LLM, I know plenty of fellow young engineers that do too. It's really not that complex.

I’m 40, and I don’t.I took that abstraction for granted and “left it to the big labs”. However I want to build my own LLM for learning purposes.

On needing big expensive hardware.. necessity is the mother of great innovation. Perhaps 18year olds trying to build their own LLMs in constrained resources environments will result in ground breaking ideas of achieving better intelligence than the one we currently have….

The world needs pragmatic folks who work at a higher abstraction and make LLMs useful, AND also folks who think why not “this other way”? And build newer ways to do fundamental things.

Given the usefulness of current LLMs, I would certainly encourage anybody to try and build their own LLMs, and see what they come up with…

Heck if they build a rack full of old laptops and run something with it that could be done “better” with modern servers, I’d still appreciate the learning running things on those little machines bring.

	▲	rrr_oh_man 26 minutes ago \| parent [-]
		> On needing big expensive hardware.. necessity is the mother of great innovation. Perhaps 18year olds trying to build their own LLMs in constrained resources environments will result in ground breaking ideas of achieving better intelligence than the one we currently have…. 10000%.

▲

danpalmer 2 hours ago | parent | prev [-]

Agreed. It's hard to learn unless you have access to quite high end hardware, and even paying by the hour is expensive. There's a low ceiling on what you can learn without doing training runs.

You can however learn everything you need to know to get on the career ladder as a software engineer on a regular home PC.

	▲	jbs789 an hour ago \| parent [-]
		While the topic here is narrow, the concept is broader. Do you take the first step or rule it out because you don’t yet see the complete picture. As a teenager I never hesitated to try things out. As a young adult I wanted the whole picture. Now I’m back to playing / trying things out. I kinda wish I’d not given it up. PG being a bit older and reminiscing - I bet he’s in that bucket too, whereas someone trying to establish themselves professionally probably (aka me early 20s) wants to see the path.

▲

DanielHB an hour ago | parent | prev | next [-]

But can you _sell_ that? If you can't you can't get a job doing it.

▲

kevmo314 26 minutes ago | parent [-]

I suspect most who worked at Google did not work on Google infrastructure before getting there.

	▲	mike_hearn 8 minutes ago \| parent [-]
		No, but the skills Google needed (back then) were just normal programming and sysadmin skills scaled up. I had eight interviews that covered Linux sysadmin, programming, debugging, networking, maths and more. If Google hadn't wanted me there'd have been plenty of other companies who needed those skills. If you look at model training jobs a lot of the work at this point is creating RL gyms (normal programming work), but most people still think the work is all neural architecture research. Doing the former is fine but won't teach you much about how to build LLMs, whatever that means now. Doing the latter is a very hard market to get into: not many jobs and requirements are often like, "you must have published at one of the following conferenecs". Prior experience is assumed. Most of them seem to treat Google as ML university and source of new recruits. It's understandable given the cost of training runs.

▲

teaearlgraycold 2 hours ago | parent | prev [-]

Interesting/capable diffusion models are much smaller than similarly interesting language models. But yes you could always scale things down to learn the fundamentals.

	▲	kevmo314 27 minutes ago \| parent [-]
		There are plenty of similarly tiny language models in the realm of tts too. Gatekeeping what’s interesting misses the forest for the trees.