Data Engineer
Posted Updated 40
views
Summary
Data Engineer in Pune who designs, builds, and maintains data pipelines for analytics and BI initiatives, working with data scientists and analysts. Core stack is SQL, PySpark, and Azure Data Lake Storage (ADLS Gen2), with a focus on data quality, performance tuning, and troubleshooting.
We are seeking a skilled Data Engineer to join our dynamic team in Maharashtra. The ideal candidate will have a strong background in data transformation services and will be responsible for designing, implementing, and maintaining data pipelines that support our data analytics and business intelligence initiatives. As a Data Engineer, you will work closely with data scientists and analysts to ensure that data is accessible, reliable, and ready for analysis.
Key Responsibilities:
- Develop and maintain robust data pipelines using SQL and PySpark to facilitate data transformation and integration from various sources.
- Collaborate with cross-functional teams to understand data requirements and deliver solutions that meet business needs.
- Optimize data storage and retrieval processes in Azure Data Lake Storage (ADLS Gen2) to ensure efficient data access and performance.
- Monitor and troubleshoot data pipeline performance, implementing improvements as necessary.
- Ensure data quality and integrity through rigorous testing and validation processes.
- Stay updated with the latest industry trends and technologies in data engineering and analytics.
Required Skills:
- Proficiency in data engineering concepts and practices.
- Strong experience with SQL for data manipulation and querying.
- Hands-on experience with PySpark for big data processing.
- Familiarity with Azure Data Lake Storage (ADLS Gen2) and cloud-based data solutions.
- Excellent problem-solving skills and attention to detail.
- Ability to work collaboratively in a team environment and communicate effectively with stakeholders.
If you are passionate about data and have the skills to drive data transformation initiatives, we encourage you to apply for this exciting opportunity.