the sentence is ambiguous because "Python" can mean python + a certain library and even a different Python implementation

but I find it illuminating to compare what a certain hardware can do in principle (what is possible) vs what I can "reach" as programmer within a certain system/setup

in this case NVIDIA A100 vs "Python" that does not reach a A100 (without the help of CUDA and PyTorch)

another analogy:

I find it useful to be able to compare what the fastest known way is to move a container from A to B using a certain vehicle (e.g. truck) and how that compares to how fast a person that can not drive that truck can do it + variants of it (on foot, using a cargo bike, using a boat via waterway, …)

I'm also interested in how much energy is needed, how much the hw costs and so on

Often there are many ways to do things, comparing is a great starting point for learning more

related to the truck analogy: an advantage of the way slower Python approach is: it does not need a GPU

that said: Python can get to more FLOPs by changing the representation: https://docs.python.org/3/library/array.html