Difficulty only adjusts every 2016 blocks. If the system gets out of whack enough it could slow down to a crawl for an extended period of time.

In practice it’s not much of an issue because bitcoin is not use for commerce but it’s a store of value and it some of the trades are not even on chain.