Data engineer

Role Overview

We are seeking a skilled and motivated Data Engineer to design, develop, and maintain scalable data pipelines and data solutions. The ideal candidate will work closely with business stakeholders, data analysts, data scientists, and application teams to enable efficient data ingestion, transformation, storage, and reporting.

Key Responsibilities

• Design, build, and maintain scalable data pipelines and ETL/ELT processes.
• Develop and optimize data integration solutions from various structured and unstructured data sources.
• Build and manage data warehouses, data lakes, and data marts.
• Ensure data quality, integrity, security, and governance across platforms.
• Create and maintain data models that support business reporting and analytics.
• Optimize data processing performance and troubleshoot issues in production environments.
• Collaborate with cross-functional teams to understand data requirements and deliver solutions.
• Implement monitoring, logging, and alerting mechanisms for data pipelines.
• Support cloud-based data platforms and migration initiatives.
• Maintain documentation for data architecture, workflows, and processes.

Required Skills

• Strong proficiency in SQL and database concepts.
• Experience with ETL/ELT tools and data integration frameworks.
• Hands-on experience in Python, PySpark, Scala, or Java.
• Experience with Apache Spark, Hadoop, Kafka, or similar big data technologies.
• Knowledge of Azure Data Factory (ADF), Azure Databricks, Azure Synapse, or equivalent cloud data platforms.
• Experience with relational databases such as SQL Server, Oracle, PostgreSQL, or MySQL.
• Understanding of Data Warehouse concepts and dimensional modeling.
• Familiarity with CI/CD pipelines and version control tools like Git.
• Strong analytical and problem-solving skills.

Preferred Skills

• Experience with cloud platforms such as Azure, AWS, or GCP.
• Knowledge of data governance, data quality, and metadata management.
• Exposure to real-time data streaming technologies.
• Understanding of DevOps and Infrastructure as Code (IaC).
• Experience with AI/ML data pipelines is an added advantage.

Educational Qualification

• Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, or a related field.

Key Competencies

• Strong communication and stakeholder management skills.
• Ability to work independently and within cross-functional teams.
• Excellent problem-solving and debugging skills.
• Strong ownership and accountability.

Nice to Have:

• Microsoft Azure Certifications (DP-203, Azure Data Engineer Associate)
• Databricks Certifications
• Experience in Retail, E-commerce, or Supply Chain domains