This is exactly it - you can cache all the wrong things easily, cache only the code you wanted changed, or cache nothing but one small critical file nobody knows about.

No wonder many just turn caching entirely off at some point and never turn it back on.