Senior Data Engineer
Summary
Build and optimize scalable ETL/ELT pipelines, data warehouses, and streaming pipelines using Python, Spark, and cloud platforms like Snowflake or BigQuery.
Design, develop, and optimize scalable ETL/ELT pipelines to ingest, transform, and process large volumes of structured and unstructured data.
Build and maintain enterprise-grade data warehouses, data lakes, and lakehouse architectures.
Develop reliable batch and real-time streaming data pipelines using modern data engineering technologies.
Implement data quality validation, monitoring, governance, lineage, and observability processes.
Optimize SQL queries, pipeline performance, storage costs, and processing efficiency.
Work closely with Data Scientists, BI Developers, Machine Learning Engineers, Product Owners, and Software Engineers to deliver data-driven solutions.
Integrate data from APIs, enterprise systems, ERP platforms, SaaS applications, and third-party sources.
Develop reusable data models and transformation frameworks following best engineering practices.
Participate in cloud migration and modernization initiatives.
Support AI and Machine Learning teams by building scalable feature engineering and model-serving data pipelines.
Implement CI/CD, Infrastructure as Code, and automated deployment strategies for data platforms.
Mentor junior engineers and contribute to architecture discussions.
Required Skills & Experience
- Bachelor's degree in Computer Science, Information Technology, Engineering, Mathematics, or related discipline.
- 5+ years of professional Data Engineering experience.
- Strong SQL skills with experience optimizing complex queries.
- Strong Python programming experience.
- Hands-on experience with ETL/ELT development.
- Experience working with large-scale data processing frameworks such as Apache Spark.
- Experience using orchestration tools such as Apache Airflow.
- Strong understanding of dimensional modeling, star schema, snowflake schema, normalization, and denormalization.
- Experience with cloud data platforms on AWS, Azure, or Google Cloud Platform.
- Experience with one or more modern data warehouse technologies:
- Snowflake
- BigQuery
- Redshift
- Synapse
- Databricks
- Experience building REST API integrations.
- Experience using Git and CI/CD pipelines.
- Understanding of data governance, security, and access control.