> Even the "why does matrix multiplication look that way" is incredibly deep but practically impossible to motivate from other considerations. You just start with "well that's the way it is" and grind away

In my experience it need not be like that at all.

One can start by defining and demonstrating linear transformations. Perhaps from graphics -- translation, rotation, reflection etc. Show the students that these follow the definition of a linear transformation. That rotating a sum is same as summing the rotated(s).

[One may also mention that all differentiable functions (from vector to vector) are locally linear.]

Then you define adding two linear transformations using vector addition. Next you can define scaling a linear transformation. The point being that the combination can be expressed as linear transformations themself. No need to represent the vectors as R^d, geometric arrows and parallelogram rule would suffice.

Finally, one demonstrates composition of linear transformations and the fact that the result itself is a linear transformation.

The beautiful reveal is that this addition and composition of linear transformations behave almost the same as addition and multiplication of real numbers.

The addition asociates and commutes. The multiplication associates but doesn't necessarily commute. Most strikingly, the operations distributes. It's almost like algebra of real numbers !

Now, when you impose a coordinate system or choose a basis, the students can discover that matrix multiplication rule for themselves over a couple of days of playing with it -- Look, rather than maintaining this long list of linear transformations, I can store it as a single linear transformation in the chosen basis.

I didn't think any part of linear algebra was boring. I was hooked from the moment I saw Ax=b => x = b/A. Gaussian elimination is a blast, like an actually-productive Sudoku puzzle, and once you have it down you can blaze through the first 2/3rds of an undergrad linear algebra course. I don't consciously try to gain automaticity with math subjects, but matrix-column multiplication I got pretty quickly and now I just have it.

I learned from Strang, for what it's worth, which is basically LU, spaces, QR, then spectral.

I am really bad at math, for what it's worth; this is just the one advanced math subject that intuitively clicked for me.

Linear algebra was such a tedious thing to learn I skipped over it to abstract algebra and doubled back once I had some kind of minimally interesting framework to work with it against. Normally I think this is a foolish way to do things, but sometimes things are just so dull you have to take the hard route to power through at all.

For anyone who’s interested in graphics programming and/or is a visual learner/thinker, there’s an incredibly motivating and rewarding way to learn the basics of linear algebra. (And affine algebra, which tends to be handwaved away, unfortunately. I’m writing a MSc thesis about this and related topics.)

The older I get the more convinced I am that "math is not hard; teaching math is hard".

> Even the "why does matrix multiplication look that way" is incredibly deep but practically impossible to motivate from other considerations.

Short, simple answer to that question by Michael Penn: https://www.youtube.com/watch?v=cc1ivDlZ71U

Another interesting treatment by Math the World: https://www.youtube.com/watch?v=1_2WXH4ar5Q&t=4s

There's no impenetrable mystery here. Probably just bad teaching you experienced.

> Even the "why does matrix multiplication look that way" is incredibly deep but practically impossible to motivate from other considerations.

It's only difficult if you are wedded to a description of matrices and vectors as seas of numbers that you grind your way through without trying to instill a fuller understanding of what those numbers actually mean. The definition makes a lot more sense when you see a matrix as a description of how to convert one sense of basis vectors to another set of basis vectors, and for that, you first need to understand how vectors are described in terms of basis vectors.

I found the university level presentation of "Vector spaces -> Linear functions -> Matrices are isomorphic to linear functions" much more motivating than the rote mechanics I was taught in highschool, but it's hard to see if I would've had that appreciation without being taught the shitty way first.

What I find amazing is, given how important linear algebra is to actual practical applications, high school math still goes so deep on calculus at the expense of really covering even basic vectors and matrices.

Where vectors do come up it’s usually only Cartesian vectors for mechanics, and only basic addition, scalar multiplication and component decomposition are talked about - even dot products are likely ignored.

> This "little book" seems to take a fairly standard approach, defining all the boring stuff and leading to Gaussian elimination. The other approach I've seen is to try to lead into it by talking about multi-linear functions and then deriving the notion of bases and matrices at the end. Or trying to start from an application like rotation or Markov chains.

Which books or “non-standard” resources would you recommend then, that do a better job?

Why do you say it's practically impossible to motivate matrix multiplication? The motivation is that this represents composition of linear functions, exactly as you follow up by mentioning.

It's a disservice to anyone to tell them "Well, that's the way it is" instead of telling them from the start "Look, these represent linear functions. And look, this is how they compose".

There was a brief time when I understood how linear algebra was used to render 3d models but that's all gone now.

I wonder if these days that would be a better starting point.

I had the opposite experience when learning linear algebra as I was also doing 3D computer graphics at the time. It was super interesting and fun. I guess you just have to find an application for it.

I went through Khan academy on linear algebra a long time ago because I wanted to learn how to write rendering logic, for me I was implementing things as I learned them with immediate feedback for a lot of the material. Was probably the single most useful thing I learned then.

Actually, the calculation of matrix multiplication is extremely easy to understand if you show it done with a row and column vector first. Once your students understand dot products, they will recognize matrix multiplication as the mere pairwise calculation of dot products.

Probability and statistics falls into that category too. It’s one of the more boring undergraduate math courses, but is mind-boggling useful in the real world.

(Basic probability / combinatorics is actually pretty cool, but both tend to be glossed over.)

I hated precalc, but I loved proof-based linear algebra, my first university math course (linalg + multivariate calc).

The simplicity(/beauty) of matrix multiplication still irks me though, in the sense of "wow, seriously? when you work it out, it really looks that simple?"

I don't know, maybe it's because I read the book on the side in highschool when I was supposed to be doing something else but I really loved Linear Algebra. Once I understood what it was I used matrix operations for everything. Vector spaces are such a powerful abstraction.

This looks very nice. Thanks for posting.

Nice pairing the text with 3Blue1Brown's lectures on linear algebra!

https://www.youtube.com/playlist?list=PLZHQObOWTQDPD3MizzM2x...

I remember how betrayed I felt when I got to calculus and realized all of those equations I learned in high school for physics were just introductory calculus. Like why did we have to learn all that the hard way?

I switched to the epub at that point. Still, credit I think to github that the page renders as well as it does.

I like Serge Lang's books for clarity of explanations. He has an Introduction to Linear Algebra which concisely covers the basics (265 pages in the main text), and grounds the matrix computations in the geometric interpretation.

Be aware that Lang has another book, called just "Linear Algebra", which is more theoretical.

I found the book "Linear Algebra" and accompanying lecture recordings by Jim Hefferon very approachable and solid.

It's free, including exercises and an (also free) solutions book.

I had the same experience when I first learned linear algebra. I don't have any book recommendations, but I did want to say that for some topics, it is better to learn it by applying it than by using a book. Linear algebra was like that for me, but oddly enough I was able to learn tensor calculus from a book later (after doing a lot of problems).

Thanks to everyone recommending books too!

You might want to checkout the book Practical Linear Algebra: A Geometry Toolbox by Dianne Hansford and Gerald Farin (its 1st edition was simply named The Geometry Toolbox: For Graphics and Modeling) to get an intuitive and visual introduction to Linear Algebra.

Pair it with Edgar Goodaire's Linear Algebra: Pure & Applied and you can transition nicely from intuitive geometric to pure mathematical approach. The author's writing style is quite accessible.

Add in Introduction to Applied Linear Algebra: Vectors, Matrices, and Least Squares by Stephen Boyd et al. and you are golden. Free book available at https://web.stanford.edu/~boyd/vmls/

I've found the "No bullshit Guide to Linear Algebra" pretty good. Could be worth checking it out. It's the one resource that has things click more for me.

A good use of linear algebra that I'm working with at the moment is the use of splines as a basis for real (vector) functions. After obtaining the matrix/vector representations you can solve for the spline coefficients (and then plot them).

Linear transforms (such as rotations and displacements) in GPU graphics.

Fourier series in signal processing.

JPEG compression.

Obtaining the best fit element in a vector space of curves given data or other constraints.

Understanding autodiff in JAX.

The mathematical definition of a tensor helps develop intuition for manipulating arrays/tensors in array libraries.

Transition matrices of a Markov chain.

PageRank.

There were some hints upstream: https://news.ycombinator.com/item?id=45107638

Machine learning, LLMs, RSA, etc.

It's generally useful for multivariate statistics, 3D flies (insects), in 3D space, clustering about a narrow slanting plane of light from a window slit are points that can be projected onto "the plane of best fit" - nominally the slanting plane of light.

That right there is a geometric picture of fitting a line, a plane, a lower order manifold, to a higher order data set, the errors (distance from plane), etc. and something of what Singular Value Decomposition is about (used for image enhancement, sharpening fuzzy data, etc).

The real test of applications is what kind of work do you see yourself doing? - A quick back read suggests your currently a CS student, so all unfocused potential for now (perhaps).

A little surprising to me this doesn’t come up the most! Excellent text, and look, you can get the 4th edition (2024) for free as a pdf at http://axler.net.

Update: using Sigil to look inside the .epub, I saw it was produced by `pandoc` and the math is rendered as MathML.

Yep, same here. It is a good refresher.

[Author here] We hear you! Here’s a similar book for Calculus: https://github.com/the-little-book-of/calculus

In this book, I cover Functions, Derivatives, Integrals, Multivariable Calculus, and Infinite Processes. In addition, I’ve included appendices with sketch proofs and applications to Physics, Probability and Statistics, and Computer Science.

Serge Lang's "Short Calculus" is clear and concise, efficiently covering the basics in ~170 pages instead of hundreds of pages like some books. It then has a few more chapters going into a bit more depth but on topics that are also essential if you want to take things further

This was one of the reasons for me wanting to study the J programming language. I have a notion AI/ML programming might be better done using array languages.

Some books for studying Mathematics using J are listed here - https://code.jsoftware.com/wiki/Books

That's basically what you'll get if you pick up a book on 3D game programming. However, progress will come to a halt when you get to things like determinants and eigenvalues that don't show up in core 3D graphics pipelines. You'll have to find other ways to motivate a C version of that part of the curriculum... but I agree, that's a well-worthwhile thing to ask for.

You will never find a level of "beginner friendly" that suits everyone.

I agree that this is not an ideal start - at least without any further clarification - for beginners but I think it works well for people that already known mathematical notation but not many specifics of linear algebra.

Also, I don't want to be the preacher bringing this into every argument but this is one of the genuinely good uses for AI that I have found. Bringing the beginning of a beginner friendly work down to my level. I can have it explain this if I'm unsure about the specific syntax and it will convey the relevant idea (which is explained in a bit of unnecessary complexity / generality, yes) in simple terms.

I agree. I wouldn’t consider someone who has taken (and remembers) a course in set theory a beginner without some added qualifier.

One of my pet peeves is using mathematical symbols beyond basic arithmetic without introducing them once by name. Trying to figure out what a symbol is and what branch of math it comes from is extremely frustrating.