Data Engineer
Posted Updated
Description
Latinum is seeking an experienced Data Engineer for a client in Bengaluru. You will design, develop, and maintain scalable ETL/ELT pipelines using Azure Data Factory and Azure Synapse Analytics, with strong SQL and Power BI skills.
The ideal candidate will build end-to-end data solutions with CI/CD using Azure DevOps, and collaborate with stakeholders to translate requirements into robust data models. Familiarity with AI-assisted coding tools and scalable architectures is highly valued.
Design, develop, and maintain end-to-end ETL/ELT pipelines using Azure Data Factory and Azure Synapse Analytics. Manage pipeline orchestration, scheduling, dependency management, error handling, monitoring, and performance optimisation. Integrate data from multiple enterprise sources, including Oracle, Salesforce, and other business applications. Develop and manage ADF pipelines using Git-based version control and Azure DevOps. Implement CI/CD practices, including repository management, branching strategies, pull requests, and code reviews. Onboard new source systems by defining extraction strategies, incremental and daily loading mechanisms, and connectivity standards. Build and maintain Power BI semantic models, datasets, reports, and dashboards. Develop and support data transformations using Databricks wherever required. Create and maintain technical documentation, including data mapping documents, pipeline specifications, and data model documentation. Collaborate with business analysts and stakeholders to understand requirements, identify gaps, and translate business needs into technical solutions. Participate in testing, deployment, troubleshooting, and post-deployment validation. Leverage AI coding assistants such as ChatGPT and Claude for development support, documentation, and initial code reviews. Review and validate AI-assisted or agent-generated code and pull requests, ensuring adherence to engineering standards and production quality. 8+ years of hands-on Data Engineering experience. Strong, hands-on expertise in Azure Data Factory and Azure Synapse Analytics. Excellent SQL skills and experience working with complex data transformations. Strong understanding of ADF pipeline architecture, orchestration, scheduling, monitoring, error handling, and performance optimisation. Practical experience with Azure DevOps and Git-based CI/CD, including repositories, branching, pull requests, and code management. Hands-on experience with Power BI, particularly semantic modelling, datasets, and DAX. Experience integrating Salesforce data through REST/Bulk APIs, SOQL, or connector-based approaches. Experience integrating data across multiple enterprise source systems, including Oracle and other business applications. Strong understanding of ETL/ELT patterns, data warehousing concepts, and enterprise data architecture. Ability to independently develop scalable, maintainable data pipelines and solutions from scratch. Experience using AI-assisted coding tools as part of a professional software development workflow. Strong technical documentation, analytical, troubleshooting, and problem-solving skills.Skills
Sign in to read and join the discussion.