Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Using the KubernetesPodOperator for everything adds a huge amount of overhead. You still need Airflow worker nodes, but they're just babysitting the K8S pods doing the real work.

I know it's 2020 and memory is cheap or whatever, but Airflow is shockingly wasteful of system resources.



Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: