I have been filling my brain.

A long time ago I played around with neural net stuff and had some fun making tiny little things. To give an idea of time frame, this was before people were using ReLU.

Going back to it after the recent advances was incredible seeing how much has happened. So many times I'd see something and wondered how I could have missed it the first time only to realise it hadn't been invented yet when I did things last time.

It feels like there is a much higher focus on statistical mathematics now in a way that it permeates everything. That in itself requires a whole lot of new learning to get to grips with, but I also feel like there might be some value in looking at a lot of these things from a different perspective. I think I tend to look at things from a more geometric point of view.

In that vein I have been looking at some transformers using unit n-sphere embeddings with V values as geodesics, just to see what happens.

As I learn new things, I keep finding fun new ideas to muck around with, I'm just an amateur, so I'm not really restricted by areas I look at. Today I'm wondering about whether Wasserstein distance could be quickly approximated by a learnable method (especially if the inputs had access to parts of the ml components that generated the things being Wasserstein compared).

I'm almost certainly treading ground well explored by others, but my way of learning seems to be to rapidly jump between many different things picking up a small understanding of each as I go until I just seem to know things that I didn't before. Focusing on a topic and pushing in that direction never seemed to work for me so much. This is probably why I am an amateur :-)

> I'm almost certainly treading ground well explored by others

This is a good thing. One requires smoothed pathways to run!