I have a feeling Intel got rid of Octane a couple years too soon.

Yeah, (Optane) 3D XPoint from Micron would have a decent density speed trade-off. I don't know whether it makes the margin they can get from RAM though. Or whether CiM for masked ROM with a 3mo wait for parameter update (and much lower power) is a better trade-off for inference.