IMHO, as an analogy, matter is not 'a bowling-ball on a mattress', but more like a scrunched-up section of table-cloth. Tiny knots or whirlpools of space-time/quantum fields, different particles are different topologies of knot, albeit the nature of space-time is unclear and it may well be a projection.

An ant traveling at constant speed on a "scrunched up section of a table cloth" will still take the same amount of time following the same path to get from A to B. Any material analogy requires some kind of stretching or compression.