Apache Airflow
Also written as Airflow
The de facto standard tool for scheduling and orchestrating data pipelines as code — defining tasks and their dependencies as a DAG (directed acyclic graph) so complex, multi-step data jobs run reliably and in the right order.
Think of it like
Like a kitchen's prep schedule pinned to the wall, showing exactly which dishes must be started before others and in what order — not just a list of tasks, but their dependencies.
Junior or senior?
Junior sounds like
Has only written a DAG that works on the happy path.
Senior sounds like
Has debugged one that failed partway through and can describe recovering it without reprocessing everything.
Ask them
“Tell me about an Airflow DAG that failed partway through in production. How did you recover it without reprocessing everything?”