How long before they will raise the amount of context it can hold?

Or, it there a ceiling that we can't go passed?

The context length scales quadratically in terms of compute, so barring algorithmic improvements, there's definitely a limit.