| ▲ | boltzmann64 5 hours ago | |
Learn SQL and interface with Duck. You will be 100x faster than Pandas/Polaris duo at fraction of memory. Also SQL is supported literally everywhere with a much more capable than Pandas API. Duck outputs to a Pandas Dataframe, but just treat that like a dictionary. Do all your processing, filtering and aggregation in Duck. Also, try fx.wtf as a replacement for jq. it comes with a in-built tui viewer that supports vi-keybindings. Ecmascript is built into fx.wtf so you can query the JSON with JS notation (where JSON was born). You can use any JS functions including map/reduce/filter or perform any kind of transformation instead of learning jq dsl that you will forget tomorrow. | ||
| ▲ | entropicdrifter 2 hours ago | parent [-] | |
Polars also supports SQL now, per the announcement we're all commenting on. | ||