Senior Data Engineer: AI-Driven Big Data Pipelines
Posted Updated
Description
Autodesk is seeking a Senior Data Engineer for our Compliance team to design, build, and optimize large-scale data pipelines and analytics. You will work with Python, SQL, Spark, Airflow, Hive, Snowflake, and AWS to enable scalable, compliant data processing and AI-assisted engineering workflows.
The role focuses on reliability, performance, and cost efficiency across batch and near real-time processing, with opportunities to impact product usage analytics and governance reporting.
Design and implement large-scale batch and near real-time data pipelines using Spark/PySpark. Build and optimize ETL/ELT workflows across distributed platforms. Integrate AI-assisted development practices into engineering workflows. Ensure data quality, observability, and reliable pipelines with monitoring and alerts. Collaborate with product managers, data scientists, and stakeholders to translate requirements into scalable solutions. Develop data models and curated datasets for compliance, analytics, and reporting. 4–7 years in data engineering or related roles. 3+ years with Spark, PySpark, Spark SQL. 3+ years Python and SQL with clean, modular code. Strong distributed computing fundamentals and Spark tuning. Hands-on Airflow experience. ETL/ELT design and large-scale processing knowledge. Experience with AI tools in development workflows. AWS data services and Snowflake/Hive/Presto familiarity. Git, CI/CD, and automation tooling.Skills
Sign in to read and join the discussion.