Senior Data Engineer
Posted Updated
Summary
Senior Data Engineer building and maintaining data pipelines on GCP and Azure Databricks — using SQL, Python and PySpark for ingestion, transformation, data-quality validation and production troubleshooting, with client-management responsibilities.
Job Description:
- 5-8 yrs exp in GCP and Azure (Hands on experience in GCP and Azure Databricks)
- SQL: Strong hands-on expertise in complex queries, joins, aggregations, window functions, data reconciliation, and query optimization.
- Python: Strong programming skills with solid understanding of core concepts, transformations, and data-processing logic.
- PySpark/Spark: Hands-on experience with DataFrames, joins, transformations, aggregations, window functions, and distributed processing.
- ETL/Data Integration: Experience developing and supporting data ingestion, transformation, validation, and error-handling frameworks.
- Data Quality: Experience with reconciliation, completeness and consistency checks, deduplication, and data-integrity validation.
- Troubleshooting: Ability to independently debug SQL/Python/data issues, perform RCA, and resolve production problems.
- System Design: Good understanding of scalable data-engineering architecture and system-design principles.
- Strong analytical, problem-solving, communication, and stakeholder-management skills.
- Good Communication Skills.
- Experience in effective client management