Remix.run Logo
mmmaantu 2 days ago

Building pyreqwest, a high-performance Python HTTP client backed by Rust’s reqwest. It has gotten quite feature rich: async and sync APIs, similar ergonomic interface of reqwest, full type hints, and built-in testing/mocking. It has no unsafe code, and no Python-side dependencies. (Started after getting too annoyed with all the issues httpx has.)

https://github.com/MarkusSintonen/pyreqwest

sirfz 2 days ago | parent | next [-]

I've recently updated an internal tool which basically acts as a configuration and dependency/context manager for performing hundreds of api calls. I added an httpx backend (to test vs the current urllib3 backend) and also introduced an async API (httpx as well). However, from your benchmarks it seems like I should've went with aiohttp for faster async? I will work on integrating pyreqwest as well

mmmaantu 2 days ago | parent [-]

Yes httpx is badly broken. Eg its connection pooling implementation is not great at all. There are various issues in httpx/httpcore. There are also old open PRs trying to fix issues but maintainer(s) are just not intrested.

sirfz 2 days ago | parent [-]

Good to know, will be interesting when we run our tests. Thanks for the info and for your work on pyreqwest, looks very promising

iib 2 days ago | parent | prev [-]

That sounds awesome. But I have two curiosities: What are the problems of httpx? And was pycurl not enough for what you wanted to do?

mmmaantu 2 days ago | parent | next [-]

Httpx/httpcore are abandoned and their connection pooling etc implementations are completely broken https://github.com/encode/httpx/issues/3215 Many PRs trying to fix varying issues https://github.com/encode/httpcore/pulls But maintainer(s) are not intrested in fixing any of the issues.

mmmaantu 2 days ago | parent | prev [-]

pycurl doesnt support async, right?