Data Engineer

We are looking for skilled Data Engineers to design, develop, implement, and maintain scalable data engineering solutions. The ideal candidate should have experience in data integration, ETL/ELT development, data pipelines, data warehousing, big data technologies, and cloud-based data platforms.

Candidates with experience in one or more relevant data engineering technologies are encouraged to apply.

Key Responsibilities

• Design, develop, and maintain scalable data pipelines and data processing solutions.
• Build and optimize ETL/ELT workflows for data ingestion, transformation, integration, and loading.
• Extract, process, clean, validate, and transform structured, semi-structured, and unstructured data.
• Develop and optimize SQL queries, stored procedures, data models, and database structures.
• Work with data warehouses, data lakes, lakehouses, and enterprise data platforms.
• Implement batch processing and real-time data streaming solutions where required.
• Ensure data quality, integrity, consistency, reliability, and performance.
• Work with cloud-based data engineering services and platforms.
• Monitor, troubleshoot, and resolve data pipeline and production issues.
• Implement data security, governance, access controls, and data management best practices.
• Collaborate with data architects, analysts, data scientists, application developers, and business stakeholders.
• Participate in code reviews, technical documentation, testing, deployment, and continuous improvement.

Technical Skills

Programming & Query Languages: Python, SQL, PySpark, Scala, Java, R, Shell Scripting.

Big Data Technologies: Apache Spark, Hadoop, Hive, Kafka, Flink, HDFS.

Data Warehousing & Lakehouse: Snowflake, Databricks, Delta Lake, Microsoft Fabric, Apache Iceberg, Amazon Redshift, BigQuery, Teradata.

Data Modelling: Dimensional Modelling, Star Schema, Snowflake Schema, Data Modelling, Data Marts, Data Warehousing.

Workflow Orchestration: Apache Airflow, Azure Data Factory, AWS Step Functions, Dagster, Prefect.

Data Governance & Quality: Data Validation, Data Profiling, Data Lineage, Metadata Management, Master Data Management, Data Cataloguing, Data Security.

Requirements

• Bachelor's degree in Computer Science, Information Technology, Engineering, or a related discipline.
• Relevant experience in Data Engineering, ETL Development, Data Integration, Data Warehousing, or Big Data.
• Strong understanding of SQL, data structures, data modelling, and data processing concepts.
• Hands-on experience with one or more programming languages and data engineering tools.
• Experience developing, maintaining, or optimizing data pipelines and data integration workflows.
• Familiarity with relational databases, data warehouses, or cloud data platforms.
• Good analytical, troubleshooting, and problem-solving skills.
• Ability to work independently and collaboratively in a team environment.
• Good communication skills and the ability to work with technical and business stakeholders.

Preferred Qualifications

• Experience with AWS, Azure, or GCP data services.
• Exposure to real-time data processing, streaming technologies, or distributed computing.
• Knowledge of data governance, security, and compliance practices.
• Experience with Agile/Scrum methodologies, CI/CD, and automated deployments.
• Relevant cloud, data engineering, or platform certifications are an advantage.