Data Engineer – Apache Airflow (New Grad) – Astronomer
Astronomer is the company behind Astro — the fully managed Apache Airflow platform trusted by 700+ companies including Glassdoor, J.B. Hunt, and Condé Nast to orchestrate millions of data pipelines daily. Apache Airflow is the world's #1 workflow orchestration platform with 10+ million downloads per month, and Astronomer is its primary commercial backer and contributor. Astronomer's Astro platform handles the infrastructure, security, and scalability concerns so data teams can focus purely on writing great DAGs. We are hiring New Grad Data Engineers to join Astronomer's customer success and platform engineering teams building and operating Apache Airflow pipelines at scale.
Responsibilities
- Design and build Apache Airflow DAGs (Directed Acyclic Graphs) orchestrating complex multi-step data pipelines across cloud data platforms
- Implement ELT pipelines using Astronomer's Astro platform — integrating Airflow with dbt, Spark, and cloud data warehouses (Snowflake, BigQuery, Redshift)
- Develop custom Airflow operators and hooks in Python enabling pipeline integration with proprietary data sources and third-party APIs
- Build monitoring and alerting for Airflow pipeline failures — implementing SLA miss alerts, retry logic, and Slack notification operators
- Optimize Airflow DAG performance — tuning task parallelism, executor configuration, and XCom data passing for high-volume pipeline workloads
- Help Astronomer's enterprise customers migrate on-premise Airflow deployments to Astro cloud managed platform
Requirements
- Bachelor's degree in Computer Science, Software Engineering, or Data Engineering
- Strong Python programming skills for Airflow DAG development and operator customization
- Understanding of workflow orchestration, task dependencies, and data pipeline design patterns
- Familiarity with Apache Airflow, Luigi, Prefect, or similar pipeline orchestration tools
- SQL proficiency and experience with cloud data warehouses (Snowflake, BigQuery, or Redshift)
Benefits
- Competitive salary with Astronomer pre-IPO equity
- Astro managed Airflow platform access for personal pipeline development
- Medical, dental, and vision benefits
- 401(k) with Astronomer matching
- 100% remote with distributed-first culture and annual team retreats