So the benefit is in caching the resources to avoid web queries, and massaging them to make them amenable to analysis? For intellectual work I imagine it would be useful if it could access gated content, like commercial reports?

Right, it's the whole search pipeline problem defined. Check this out: https://github.com/deepbluedynamics/lume