Data Engineer (Data Quality)

Job Summary:

We are looking for a highly motivated Data Quality Engineer to join our data engineering team. The ideal candidate will be responsible for building and maintaining robust data quality frameworks to ensure the accuracy, consistency, and reliability of data across platforms. You will collaborate with cross-functional teams to embed quality checks into data pipelines and support data-driven decision-making.

Key Responsibilities:

• Design and implement automated data quality checks and validation frameworks.
• Perform data profiling, anomaly detection, and root cause analysis using Python and SQL.
• Integrate data quality processes into ETL/ELT pipelines using PySpark and Databricks.
• Monitor and resolve data quality issues across Azure/AWS-based data platforms.
• Develop CI/CD pipelines for data quality automation using Git and DevOps tools.
• Work with JDBC connectors to validate data movement and integrity across systems.
• Document data quality rules, processes, and findings for technical and business stakeholders.

Required Technical Skills:

• Programming Languages: Python, SQL
• Big Data Tools: PySpark, Databricks
• Cloud Platforms: Azure/AWS (e.g., S3, Lambda)
• Data Connectivity: JDBC
• Version Control: Git
• DevOps: CI/CD tools (e.g. GitHub Actions), Airflow
• Data quality metrics visualization (e.g. PowerBI)

Soft Skills:

• Agile Experience: Proven experience working in Agile development environments.
• Analytical Thinking: Strong analytical and problem-solving abilities.
• Critical Thinking: Ability to evaluate data issues logically and propose effective, scalable solutions.
• Communication: Excellent communication skills for cross-team collaboration and stakeholder engagement.

Preferred Qualifications:

• Experience with data governance and metadata management tools.
• Familiarity with data cataloging solutions.
• Understanding of data privacy and compliance standards.