Remix.run Logo
lordnacho 3 days ago

Clickhouse is excellent btw. I took it for a spin, loading a few TB of orderbook changes into it as entire snapshots. The double compression (type-aware and generic) does wonders. It's amazing how you get both the benefit of small size and quick querying, with minimal tweaks. I don't think I changed any system level defaults, yet I can aggregate through the entire few billion snapshots in a few minutes.

fibers 3 days ago | parent [-]

By snapshots do you mean the entire orderbook in a specific point in time or the entire history that gets instiantiated?

lordnacho 3 days ago | parent [-]

At each point in time, the entire orderbook at that time.

So you could replay the entire history of the book just by stepping through the rows.