I guess it wouldn't sell shit if it used a language suitable for this type of work.

I mean, what happened to "use the right tool for the job"? There is Rust, C++, Julia, D, and certainly many more. Are they hard or what? Are they harder than mastering the math and algorithms that are relevant to an LLM? The code is actually pretty simple, certainly simpler than many "boring" apps.

▲

belter 8 months ago | parent | next [-]

Your comment shows such a fundamental misunderstanding of how modern AI/LLM works that is hard to be kind and thoughtful....

Python is simply the orchestration layer. The heavy lifting is done by low-level libraries used in the repo, written in C++, CUDA, and Rust (e.g., PyTorch’s C++ backend, Flash Attention’s CUDA kernels, FAISS’s SIMD optimizations, or Hugging Face’s Rust-powered tokenizers).

Python’s role is to glue these high-performance components together with minimal overhead, while providing accessibility. Claiming it’s "unsuitable" ignores the entire stack beneath the syntax.

A critique that is like blaming the steering wheel for a car’s engine performance.

	▲	d_tr 8 months ago \| parent [-]
		Again, I am extremely well aware of all of this. You assumed too much.

▲

simonw 8 months ago | parent | prev | next [-]

Arguing that Rust, C++, Julia or D are a better "right tool for the job" than Python for a book that teaches people about LLMs is a bit of an eyebrow-raiser.

▲

d_tr 8 months ago | parent [-]

How so? Since when is Python a good language for numerical computation? What if the reader wants to try something that cannot be achieved by plumbing canned C++? They are out of luck I guess.

Good job teaching the sloppy semantics of a scripting language for numerics I guess.

▲

simonw 8 months ago | parent [-]

"Since when is Python a good language for numerical computation?"

30 years. Numeric came out in 1995, then evolved into NumPy in 2005. https://en.m.wikipedia.org/wiki/NumPy

Almost every AI researcher and AI lab does most of their research work in Python.

▲

d_tr 8 months ago | parent [-]

I know all of these facts. Doesn't mean it is how it is for the right reasons, and even if it is, it does not imply that it is a good way to teach.

▲

simonw 8 months ago | parent | next [-]

Taking constant side-quests into Rust memory management during a class on LLMs doesn't sound like a productive way to teach to me.

▲

sokoloff 8 months ago | parent | prev [-]

It is possible that the vast majority of AI researchers are flat-out incorrect and need to be shown a better direction by you.

It is also possible that your own fitness-for-purpose coefficients are tuned differently than the majority of the field and they've made a sensible choice for their situation.

I'd wager on the latter.

	▲	d_tr 8 months ago \| parent [-]
		You can all raise your eyebrows and appeal to authority all you want. Doesn't change the fact that as soon as I want to write a tight (or not so tight) loop to do something that is not covered by all these (undoubtedly great) libraries running underneath, I am very probably out of luck and have to use another language. Doesn't sound like a very creative teaching environment to me. Not to mention that, depending on what you count as "AI research", the vast majority of it is pure garbage (come on, we all know this).

▲

antononcube 8 months ago | parent | prev [-]

I assume you mean book's code shown in the Jypyter notebooks in the repository. (Which I think is both simple and messy.)