| ▲ | __mharrison__ 2 hours ago | |||||||
Use pandas if you want advanced analytics, visualization, or ml. Use SQL if you need to move data around. | ||||||||
| ▲ | bijowo1676 2 hours ago | parent [-] | |||||||
How would you use sql to move data from three excel files (some of them may have arbitrary number of worksheets), pack of json files, two csv files, one mysql, one postgres db, some parquets, some of them in S3) ??? And in the end save processed data in aws s3 in another format like iceberg or whatever In pandas a lot of these are one-liners that are impossible in sql (depends on what sql engine you have?) | ||||||||
| ||||||||