Data Engineering II-SUPPORT SERVICES-Data & Analytics - In House Engineering
Summary
Senior Data Engineer builds and optimizes cloud-based data pipelines and warehouses using Python, PySpark, Airflow, and Databricks to support analytics for a large financial institution.
Job Description: Senior Data Engineer
Position Summary
We are seeking a highly skilled Senior Data Engineer with experience in designing, developing, and optimizing large-scale data platforms and ETL pipelines across cloud environments. The ideal candidate will have strong expertise in Python, SQL, PySpark, Airflow, Databricks, AWS, GCP, and modern data warehousing technologies. The role involves building scalable data solutions, driving cloud migrations, improving data quality, and enabling analytics for business stakeholders.
Required Skills & Qualifications
Technical Skills
- Strong programming skills in Python, SQL, and PySpark.
- Hands-on experience with Apache Airflow for workflow orchestration.
- Experience with Databricks and distributed data processing frameworks.
- Expertise in cloud platforms:
- AWS (Glue, Lambda, S3, Step Functions, CloudWatch, Redshift)
- GCP (BigQuery, Cloud Storage, Cloud Functions)
- Experience with modern data warehousing technologies:
- BigQuery
- Snowflake
- Amazon Redshift
- Knowledge of DBT, Delta Lake, and Apache Iceberg.
- Experience with Git and software development best practices.
Professional Experience
- 5+ years of experience in Data Engineering and ETL development.
- Experience working across Financial Services, Healthcare, Retail/QSR, or similar domains.
- Proven track record in cloud migration, data modernization, and performance optimization initiatives.
- Experience handling large-scale datasets ranging from hundreds of GBs to multi-terabyte environments.
Education
- Bachelor’s degree in computer science, Electronics & Computer Engineering, Information Technology, or a related discipline.