Apache Airflow: A platform for workflow orchestration
Author, schedule, and monitor complex workflows on an array of workers easily.
The latest from the blog
All about Apache Airflow
Dynamic
Airflow pipelines are configuration as code (Python), allowing for dynamic pipeline generation. This allows for writing code that instantiates pipelines dynamically.
Extensible
Easily define your own operators, executors and extend the library so that it fits the level of abstraction that suits your environment.
Elegant
Airflow pipelines are lean and explicit. Parameterizing your scripts is built into the core of Airflow using the powerful Jinja templating engine.
Scalable
Airflow has a modular architecture and uses a message queue to orchestrate an arbitrary number of workers. Airflow is ready to scale to infinity.
Testimonials
A framework that delivers the flexibility and advanced functionality our customers need.
–Framework user
Before airflow, I was using interns to schedule all of my workflows.
–Manager
Airflow has rich functionality that addresses very well the problems of workflow orchestration.
–Workflow orchestrator
Airflow is an open source community and contributions are greatly appreciated!
If you’d like to contribute, please see the Contribute section.