I assume the cost was just very low? If it was 50-100k, maybe they figured they'd just try and see.

Oh yes, according to [1], training GPT-2 1.5B cost $50k in 2019 (reproduced in 2024 for $672!).

	▲	stavros 4 days ago \| parent [-]
		That makes sense, and it was definitely impressive for $50k.