AWS Databricks Data Engineer
Summary
Data Engineer building scalable ETL pipelines using Databricks on AWS/Azure, with Python/PySpark and SQL, plus API integrations, SFTP ingestion, and monitoring workflows.
JOB DETAILS:
Position - DATA ENGINEER
Relevant Experience - 4 to 7 years
Location Option – Pune / Gurgaon / Bangalore ( Manyata Tech Park, Nagavara)
Skills: Databricks, Azure / AWS, SQL, Python, PySpark, GitHub Actions, Jira, Agile development
SQL – Must Have
1. Window functions (ROW_NUMBER, RANK, LEAD, LAG).
2. joins and query optimization.
JD:
Specialized in building scalable pipelines using GitHub, optimizing ETL workflows using native Databricks capabilities and Azure/ Amazon web services
- Build and maintain API integrations
- Develop SFTP pipelines and automated file ingestion
- Create error handling, notifications, and monitoring workflows
- Implement low-complexity ingestions (e.g., file drops, partner data shares)
- Maintain scripts and GitHub repositories
- Test pipelines and Document flows