SQL.

It is a joke, but an SQL engine can be massively parallel. You just don't know it, it just gives you what you want. And in many ways the operations resembles what you do for example in CUDA.

CUDA backend for DuckDB or Trino would be one of my go-to projects if i was laid off.

▲

drivebyhooting 12 hours ago | parent | next [-]

My issue with SQL is lack of composability and difficulty of debugging intermediate results.

▲

mamcx 5 hours ago | parent | next [-]

Yes, SQL is poor.

What could be good is relational + array model. I have some ideas on https://tablam.org, and building not just the language but the optimizer in tandem I think will be very nice.

▲

asadm 12 hours ago | parent | prev [-]

is it a language problem though? it's just lack of tooling.

▲

theLiminator 11 hours ago | parent [-]

The dataframe paradigm (a good example being polars) is another good alternative that's more composable (imo).

	▲	fifilura 7 hours ago \| parent [-]
		It is true. I still hate it. I think because it always offers 10 different ways to do the same thing. So it is just too much to remember.

▲

taeric 9 hours ago | parent | prev | next [-]

More generally, the key here is that the more magic you want in the execution of your code, the more declarative you want the code to be. And SQL is pretty much the poster child declarative language out there.

Term rewriting languages probably work better at this than I would expect? It is kind of sad how little experience with that sort of thing that I have built up. And I think I'm above a large percentage of developers out there.

▲

dvrp 11 hours ago | parent | prev [-]

If you want to work in data engineering for massive datasets (many petabytes) pls hit me up!