Data Engineer (Databricks, PySpark)
Summary
Designs and builds scalable data pipelines and products using Databricks, PySpark, and Azure to process vehicle signals, diagnostics, and customer feedback for mobility/logistics systems.
Senior: 8+Yrs and Exp 4+Yrs Exp
Positions: 2 Nos
Role Summary
We are seeking an experienced Mid & Senior Data Engineer to design, build, and own scalable data products and pipelines using vehicle signals, diagnostics, and customer feedback data. The role requires strong expertise in Databricks, PySpark, Azure, and modern data platforms, along with the ability to provide technical leadership and drive end-to-end delivery of production-grade data solutions.
Key Responsibilities
- Design, develop, and maintain scalable data products and pipelines.
- Build solutions for large-scale vehicle, diagnostic, and time-series data.
- Ensure data quality, reliability, scalability, and privacy compliance.
- Develop CI/CD pipelines, automation, monitoring, and deployment processes.
- Own production support, troubleshooting, and continuous improvements.
- Lead technical design and architecture discussions.
- Mentor engineers and promote engineering best practices.
- Collaborate with architects, product teams, and stakeholders to deliver business value.
Required Skills
- Strong experience in Python, SQL, Git/Gerrit, CI/CD, Jenkins, and GitHub Actions.
- Expert-level experience with Databricks and PySpark.
- Experience with Azure, Snowflake, Airflow, Docker, and cloud-native data platforms.
- Strong background in data engineering, system design, and software architecture.
- Experience with API integrations, large-scale datasets, and data visualization tools (Power BI).
- Ability to independently drive technical solutions and mentor team members.
- M.Sc. in Computer Science, Data Engineering, Data Science, Software Engineering, or equivalent.
Preferred Skills
- Experience with vehicle signals, diagnostics, and time-series data.
- Knowledge of UDS diagnostic communication protocols.