Remix.run Logo
CS336: Language Modeling from Scratch(cs336.stanford.edu)
131 points by kristianpaul 3 hours ago | 10 comments
skerit an hour ago | parent | next [-]

> GPU compute for self-study

Those suggestions they make for a B200 start at $4.99 an hour.

Is that really required, for starting out? I've been tinkering with my own from-scratch LLM, but in the early phases I don't need anything more than a 4090 on Vast.ai

root-parent 21 minutes ago | parent [-]

You dont even need a GPU to train your own LLM.

meken an hour ago | parent | prev | next [-]

I have fond memories of cs224d [1] taught by richardsocher. It’s a bit dated now as it was created in the pre-transformer era, but it was very cool introduction to applying deep learning to nlp at the time.

[1] https://cs224d.stanford.edu

egl2020 25 minutes ago | parent [-]

Similar thoughts here. That was when I realized the potential of the Internet: I didn't have to be a grad student at a tier 1 research university to learn about the frontier.

airstrike 30 minutes ago | parent | prev | next [-]

I wonder if people prefer to learn this on their own or if building a community around open learning is something that others are interested in

storus 2 hours ago | parent | prev | next [-]

Thanks for releasing this again! What are this year's changes to prior offerings?

tmule an hour ago | parent | prev [-]

Are video lectures available online?

Bilal_io an hour ago | parent | next [-]

Youtube playlist link from the page https://www.youtube.com/watch?v=JuoVZkPBiKk&list=PLoROMvodv4...

aerohit an hour ago | parent | prev | next [-]

https://www.youtube.com/watch?v=JuoVZkPBiKk&list=PLoROMvodv4...

mindcrime an hour ago | parent | prev [-]

https://www.youtube.com/playlist?list=PLoROMvodv4rMqXOcazWaT...