Big Data Engineer
Posted Updated
Summary
Senior big data engineer (9-14 yrs) designing and building scalable data pipelines and transformation components on Apache Spark and Azure Databricks, supporting an enterprise data platform and troubleshooting large-volume processing workloads. Python is a plus. Position is in Bengaluru, posted via staffing firm Careernet.
Key Skills: Data Engineer, Azure Databricks, Spark, Python
Roles and Responsibilities:
- Design and develop scalable data engineering solutions using Apache Spark and Azure Databricks.
- Build robust data processing workflows for large-volume and complex datasets.
- Develop reusable data transformation components and support enterprise data platform initiatives.
- Troubleshoot data processing issues and improve reliability, throughput, and maintainability of Spark workloads.
- Collaborate with engineering and data teams to deliver production-ready data solutions.
Skills Required:
- 9-14 years of relevant experience in data engineering and big data platform development.
- Strong hands-on experience with Apache Spark for distributed data processing.
- Extensive experience with Azure Databricks and enterprise data engineering platforms.
- Demonstrated experience working as a Data Engineer on large-scale data solutions.
- Ability to design scalable data processing solutions and implement reliable data pipelines.
Good to Have:
- Professional experience developing data engineering solutions with Python.
Education: Bachelor's or master's degree in Computer Science, Information Technology, Engineering, or a related discipline.