Data Engineer
Posted Updated
SUMMARY
Responsibilities
Design, build, and optimize ETL/ELT pipelines for batch and streaming data
Develop and maintain data warehouses and data lakes (e.g., Snowflake, BigQuery, Redshift)
Collaborate with data scientists, analysts, and product teams to understand data needs
Ensure data quality, reliability, and observability across pipelines
Build and manage orchestration workflows (e.g., Airflow, Prefect, Dagster)
Implement best practices for data modeling, partitioning, and performance tuning
Monitor and troubleshoot data infrastructure issues in production
Requirements
5+ years of experience in data engineering or a related role
Proficiency in SQL and Python (or Scala/Java)
Experience with cloud platforms (AWS, GCP, or Azure)
Hands-on with big data tools: Spark, Kafka, dbt, or similar
Familiarity with data warehousing concepts and dimensional modeling
Strong understanding of CI/CD and version control (Git)
Nice to Have
Experience with real-time/streaming pipelines (Kafka, Flink, Kinesis)
Exposure to ML pipelines or MLOps tooling
Knowledge of data governance and data catalog tools (e.g., Atlan, Alation)
Design, build, and optimize ETL/ELT pipelines for batch and streaming data
Develop and maintain data warehouses and data lakes (e.g., Snowflake, BigQuery, Redshift)
Collaborate with data scientists, analysts, and product teams to understand data needs
Ensure data quality, reliability, and observability across pipelines
Build and manage orchestration workflows (e.g., Airflow, Prefect, Dagster)
Implement best practices for data modeling, partitioning, and performance tuning
Monitor and troubleshoot data infrastructure issues in production
Requirements
5+ years of experience in data engineering or a related role
Proficiency in SQL and Python (or Scala/Java)
Experience with cloud platforms (AWS, GCP, or Azure)
Hands-on with big data tools: Spark, Kafka, dbt, or similar
Familiarity with data warehousing concepts and dimensional modeling
Strong understanding of CI/CD and version control (Git)
Nice to Have
Experience with real-time/streaming pipelines (Kafka, Flink, Kinesis)
Exposure to ML pipelines or MLOps tooling
Knowledge of data governance and data catalog tools (e.g., Atlan, Alation)
Skills
- Airflow
- AWS
- Azure
- Big Data
- BigQuery
- CI/CD
- Cloud
- Dagster
- Data Engineering
- Data Governance
- Data Lake
- Data Modeling
- Data Quality
- Data Warehousing
- dbt
- Dimensional Modeling
- ELT
- ETL
- Flink
- GCP
- Git
- Java
- Kafka
- Kinesis
- Machine Learning
- MLOps
- Observability
- Prefect
- Python
- Redshift
- Scala
- Snowflake
- Spark
- SQL
- Version Control
