Jeez that phone was available new for £10. Do such phones exist these days?

Maybe not quite that low, and not from established brands, but search aliexpress for lte phone and there's several for $30-$50. I didn't look at the lte bands they support, which is important if you're going to use one or these (but I have trust issues that would need to be solved first)