Why would you tell people that the correct order is to build foundational knowledge before exploring a subject? For some (many?) people, a 'proper' understanding develops _after_ the exploration.

▲

BoredomIsFun 2 hours ago | parent [-]

> Why would you tell people that the correct order

Because I can?. JK. Because that was my experience, of someone who is 2.5 older than 17?

> For some (many?) people, a 'proper' understanding develops _after_ the exploration.

I am afraid you have a too confrontational attitude here, but I'll answer anyway: because I do not believe you can simply "explore" such complex topics like building an LLMs. You'd simply be unable to build LLM drom scratch, unless you'd call cargo-cult chaining magic numpy incantations you've taken from Karpathy's tutorials "exploring".

If I were in "exploratory" state of mins, I'd rather go from entirely different side - I'd try playing with LoRA-ing existing small LLMs, such as venerable 2 y.o. Mistral Nemo, to get "feeling" for what training is and how hyperameters influence the process.

▲

molf an hour ago | parent [-]

> unless you'd call cargo-cult chaining magic numpy incantations you've taken from Karpathy's tutorials "exploring"

That's what I would call exploring.

I too started exploring programming as a teen by cargo culting. Fooling around and getting results is what made it fun. Understanding came later.

▲

BoredomIsFun 29 minutes ago | parent [-]

> That's what I would call exploring.

Then it is not "building llm from scratch" in my book. Just mindlees following instructions. Could be educational yes, but only trivially useful, if you have no bloody idea what you are doing.

> Fooling around and getting results is what made it fun. Understanding came later.

It is not how LLMs are "built from scratch".

	▲	molf 10 minutes ago \| parent [-]
		No it's not useful, no you don't know what you're doing, and no it is not 'proper'. But you're learning. Maybe just accept that not everyone starts from fundamental theory, and there are lots of people who start learning by fooling around.