D 365- Data Engineer

We are seeking a highly skilled Senior Data Engineer with 8–12 years of experience in designing, developing, and managing scalable data platforms and enterprise data solutions. The ideal candidate will have strong expertise in data engineering, cloud technologies, ETL/ELT development, data warehousing, and big data ecosystems.

The candidate will play a key role in building modern data platforms, enabling advanced analytics, business intelligence, AI/ML initiatives, and data-driven decision-making across the organization.

Key Responsibilities:

• Design, develop, and maintain scalable data pipelines and data integration frameworks.
• Build and optimize ETL/ELT processes for structured and unstructured data sources.
• Develop and manage enterprise data warehouses, data lakes, and lakehouse architectures.
• Implement data ingestion, transformation, and orchestration solutions using cloud-native services.
• Collaborate with business analysts, data scientists, architects, and stakeholders to understand data requirements.
• Ensure data quality, governance, security, and compliance standards are met.
• Optimize database and query performance for large-scale datasets.
• Design and implement real-time and batch data processing solutions.
• Support reporting, analytics, AI/ML, and advanced data engineering initiatives.
• Participate in architecture reviews and provide technical leadership to the data engineering team.
• Develop CI/CD pipelines and automate deployment processes for data platforms.
• Mentor junior engineers and establish engineering best practices.

Qualification:

Required Technical Skills

Data Engineering

• Data Modeling (Dimensional & Relational)
• ETL/ELT Development
• Data Warehousing Concepts
• Data Lake & Lakehouse Architecture
• Data Migration & Data Integration
• Real-Time Data Processing

Cloud Platforms (Any One or More)

• Microsoft Azure
  • Azure Data Factory (ADF)
  • Azure Synapse Analytics
  • Azure Data Lake Storage (ADLS)
  • Azure Databricks
  • Microsoft Fabric
• AWS
  • Glue
  • Redshift
  • EMR
  • S3
• Google Cloud Platform (GCP)
  • BigQuery
  • Dataflow
  • Cloud Storage

Big Data Technologies

• Apache Spark
• Databricks
• Hadoop Ecosystem
• Kafka
• Delta Lake

Databases

• SQL Server
• Oracle
• PostgreSQL
• MySQL
• Snowflake
• Azure SQL Database

Programming Skills

• Python
• SQL
• PySpark
• Scala (Preferred)
• Shell Scripting

DevOps & Version Control

• Azure DevOps
• Git
• CI/CD Pipelines
• Jenkins

Required Experience

• 8–12 years of overall IT experience.
• Minimum 5+ years of experience in Data Engineering.
• Strong hands-on expertise in SQL and Python.
• Experience with cloud-based data platforms (Azure preferred).
• Expertise in building data pipelines, ETL/ELT frameworks, and data warehouses.
• Experience working with large-scale structured and unstructured datasets.
• Knowledge of data governance, security, and compliance frameworks.
• Experience in Agile/Scrum delivery models.

Preferred Qualifications

• Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, or related field.
• Microsoft Certified: Azure Data Engineer Associate (DP-203).
• Microsoft Fabric Certification.
• Databricks Certified Data Engineer.
• AWS or GCP Data Engineering Certifications.