I don't get this - I can do LORA on my mac... ok I can't do LORA on a 1tn param model, but if I was in the tn parameter model game I would get some kit that I could use to do that...

What's their moat / secret sauce?

Like, buy and set up the physical hardware? I cba with that. Plus the hardware you want for LoRA (the type but especially the quantity) is different than what you want for inference, so either you'd under-spec it and wait forever for fine-tuning runs, or over-spec it and have low utilization most of the time. And even then who knows if it would be good enough to LoRA next year's best open source model. AWS gets great margins for renting out commodity hardware as a service because it built the right abstractions and can serve them efficiently at scale, I think the arguments here are basically the same.