Data Engineer
Posted Updated
Description
Key Responsibilities
We are seeking an experienced Data Engineer with a strong background in Databricks, Python, Spark/PySpark and SQL to design, develop, and optimize large-scale data processing applications. The ideal candidate will build scalable, high-performance data engineering solutions and ensure seamless data flow across cloud and on-premise platforms.
Key Responsibilities:
- Design, develop, and maintain scalable data processing applications using Databricks, Python, and PySpark/Spark.
- Write and optimize complex SQL queries for data extraction, transformation, and analysis.
- Collaborate with data engineers, data scientists, and other stakeholders to understand business requirements and deliver high-quality solutions.
- Ensure data integrity, performance, and reliability across all data processing pipelines.
- Perform data analysis and implement data validation to ensure high data quality.
- Implement and manage CI/CD pipelines for automated testing, integration, and deployment.
- Contribute to continuous improvement of data engineering processes and tools.
Required Skills & Qualifications:
- Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field.
- Proven experience as a Databricks with strong expertise in Python, SQL and Spark/PySpark.
- Strong proficiency in SQL, including working with relational databases and writing optimized queries.
- Solid programming experience in Python, including data processing and automation.
Skills
Sign in to read and join the discussion.