Most of your local requests will fail (with Python/Curl/whatever) if you are not using a remote-controlled Firefox/Chrome. And even then you will be bitten brutally by captchas, rendering them basically unusable for mass requests. So it's better to outsource your web search to an external service with a stable API.

The existing search machines that offer free APIs for local harnesses are super bad with low quality.

This is worth the invest as LLMs can be guided closer to existing information/code to adapt it.