Making a vintage LLM from scratch

cyberge99 an hour ago | parent | next [-]

There are certain things you can only truly learn by doing. I remember doing Linux From Scratch over a weekend and the depth of linux that I still understand to this day.

Thanks for the writeup. A more granular followup would be cool too.

	▲	breezybottom 12 minutes ago \| parent [-]
		Except in this case he vibe-coded it

▲

croqaz 19 hours ago | parent | prev | next [-]

I am creating my tiny Llama 340M base model from scratch. If you're curious about the steps, challenges and cost, read on. I am still working on the instruct model.

▲

rxm 2 hours ago | parent | prev [-]

Nice project. I’m curious to see how it writes after instruct.