Can't you decode the embeddings to tokens for debugging?

You can but this is lossy (as it drops context; it’s a dimensionality reduction from 512 or 1024 to a few bytes) and non-reconvertible.