If you switch models, you have no cache. There is no way to change this as they depend on sending the tokens through the weights and saving the computation results. Different model, different tokens, different weights, different calculation.

Actually yes because context is kept on kV cache