Remix.run Logo
VirusNewbie 4 days ago

>one could also bash other "proper" engineers that can do the CI/CD but don't know shit about how to be efficient with petabyte-scale processing.

But that would be SWEs no?

I was a 'data engineer' (until they changed the terrible title) at a startup and I ended up having to fight with Spark and Apache Beam at times, eventually contributing back to improve throughput for our use cases.

That's not the same thing as a Business Analyst who can run a little pyspark query.