The Airflow plugin is used to orchestrate our Meltano pipelines. Follow the orchestrate docs to get dbt installed in your project.
In the production environment we are running Airflow on Kubernetes so we use a custom KubernetesPodOperator to fan out all of our tasks across the cluster but locally we want to use the BashOperator as if were running in our terminal.
We also use a DAG generator because want to limit as much DAG writing as possible to avoid duplicate code, maintenance burden, and potential for mistakes.
DAGs are defined in the dag_definition.yml file (eventually will be pulled into Meltano as part of meltano.yml probably) which is read by the generator to build DAG definitions on the fly.
The DAG generator is referenced by both the locally configured userdev environment and the prod environment which is built and deployed using the deploy module.
To start up locally, using 3 terminals (or add -D for background), then go to http://0.0.0.0:8080 and login.
# Webserver
meltano --environment=userdev invoke airflow webserver
# Create an admin user for UI login
meltano --environment=userdev invoke airflow users create --username melty --firstname melty --lastname meltano --role Admin --password melty --email melty@meltano.com
# Scheduler
meltano --environment=userdev invoke airflow scheduler