After using Argo Workflows, I don't think I will ever return to Airflow. Kubernetes is not an easy system to manage, but managing an Airflow setup is somehow worse. The story around disaster recovery and scheduler redundancy was an absolute nightmare for me.

▲

datadrivenangel 5 days ago | parent [-]

Argo workflows is much more painful for data processing than Airflow in my experience.

	▲	flakes 5 days ago \| parent [-]
		It’s a tradeoff. Ease of modeling the pipelines vs ease of managing the infrastructure. Im not really a fan of either syntax for defining DAGs, but they're the best options out there imo.