▲ | itamarst 21 hours ago | ||||||||||||||||||||||||||||
I talk about this more explicitly in the PyCon talk (https://pythonspeed.com/pycon2025/slides/ - video soon) though that's not specifically about Pydantic, but basically: 1. Inefficient parser implementation. It's just... very easy to allocate way too much memory if you don't think about large-scale documents, and very difficult to measure. Common problem with many (but not all) JSON parsers. 2. CPython in-memory representation is large compared to compiled languages. So e.g. 4-digit integer is 5-6 bytes in JSON, 8 in Rust if you do i64, 25ish in CPython. An empty dictionary is 64 bytes. | |||||||||||||||||||||||||||||
▲ | cozzyd 19 hours ago | parent [-] | ||||||||||||||||||||||||||||
Funny to see awkward array in this context! (And... do people really store giant datasets in json?!?). | |||||||||||||||||||||||||||||
|