Honestly, if you’re wanting to produce something as good as this, Three.js or other such things just aren’t particularly helpful. It is easier to just ignore all the libraries and do it all from scratch. Popular libraries are good at producing finished products in a particular shape. When you’re wanting to demonstrate the implementation steps and allow intricate fiddling and have everything polished like you want it, they’re generally somewhere between painful and hopeless.

You could still keep Three.js for bits like vector calculations, but it just doesn’t feel worth it, it’s easy enough to implement yourself—or copy and paste from some such library and modify as needed—and will be much lighter. And you build up the bits and pieces you need over time.