Senior Software Engineer
Posted Updated
Summary
Senior data engineer at IT services firm Virtusa in Dubai, building and optimizing large-scale data pipelines with PySpark and Informatica BDM ETL/ELT workflows for banking clients. Day-to-day work covers data integration, quality, governance, warehouse/lake support, and banking regulatory compliance.
Key Responsibilities
- Design, develop, and maintain scalable data pipelines using PySpark for processing large volumes of structured and unstructured data.
- Develop and manage ETL/ELT workflows using Informatica BDM.
- Work closely with data architects, analysts, and business stakeholders to understand data requirements and translate them into technical solutions.
- Optimize data processing jobs to ensure high performance and reliability.
- Implement data quality, data governance, and data validation processes.
- Integrate data from multiple sources including databases, APIs, and enterprise systems.
- Support data warehouse and data lake initiatives within the organization.
- Ensure compliance with banking regulatory and security standards.
- Troubleshoot and resolve data pipeline issues and performance bottlenecks.
- Participate in code reviews, testing, and deployment processes to maintain high development standards.
Required Skills & Qualifications
- 6+ years of experience in Data Engineering or related roles.
- Strong hands‑on experience with PySpark for large‑scale data processing.
- Solid experience with Informatica BDM (Big Data Management).
- Experience working with Big Data technologies and distributed data processing frameworks.
- Strong knowledge of SQL and data modeling concepts.
- Experience with data integration, ETL development, and workflow orchestration.
- Familiarity with cloud platforms or big data ecosystems (Hadoop/Spark environments) is a plus.
- Good understanding of data governance, security, and compliance requirements in banking.
- Strong analytical, problem‑solving, and communication skills.
Domain Requirement
Mandatory experience in the Banking or Financial Services domain, preferably working with enterprise data platforms, regulatory reporting systems, or large-scale financial datasets.
Preferred Qualifications
- Experience with data lake architectures and data warehouse platforms.
- Exposure to Agile development methodologies.
- Knowledge of Python‑based data engineering frameworks and automation.