Depends on your setup but we've had success with building custom harnesses and processing metrics via OpenTelemetry + Prometheus. This is running on a massive K8s cluster spinning up pods per N agents.

If you are using a proprietary harness like CC or Codex then look into automatically ingesting their memory logs and internal datastores to track progress and then instrument the host to send back metrics