I think it only keeps history for user edited files, agent edited files don't seem to end up in it for me (Claude code) but maybe it works with other agents with the proper plugins I'm not sure.

+1 OP here, this is the problem I'm solving for. Agents use tools and may be in multiple places editing; therefore, you need to watch the file system.