Data Engineer (Lahore) – Sora Systems – Full‑Time Role 2024
Posted
Summary
Designs, builds, and maintains batch and streaming data pipelines and AWS-based lakehouse architecture using Python, SQL, Spark, and Apache Iceberg. Works hands-on with S3, AWS Glue, and Athena, and handles ETL/ELT, performance tuning, data governance, and CI/CD.
Sora Systems in Lahore seeks a Data Engineer with 4‑7 years experience to create and maintain AWS‑based data lakehouse pipelines, using Python, Spark and Apache Iceberg.
Sora Systems, a technology firm based in Lahore, is hiring a full‑time Data Engineer to design and sustain scalable data pipelines and AWS‑powered lakehouse solutions.
Responsibilities
- Design, build and maintain both batch and streaming data pipelines.
- Construct and optimise lakehouse architectures on AWS using Apache Iceberg.
- Develop ETL/ELT processes for structured and unstructured datasets.
- Work with Amazon S3, AWS Glue, Glue Data Catalog and Amazon Athena.
- Implement data partitioning, schema evolution, query optimisation and cost‑effective cloud resources.
- Monitor pipeline health, resolve failures and enhance performance.
- Ensure data quality, security, governance and appropriate access controls.
- Set up CI/CD pipelines and keep technical documentation up to date.
- Collaborate with engineering, AI and data‑science teams.
Who Should Apply
- 4–7 years of professional experience in data engineering.
- Proficient in Python and SQL.
- Hands‑on experience with AWS services (S3, Glue, Athena) and Apache Iceberg.
- Familiarity with PySpark or Apache Spark and data lakehouse concepts.
- Understanding of cloud security, IAM and monitoring tools.
- Experience with Git, CI/CD tools such as Jenkins and version control.
- Bachelor’s or Master’s degree in Computer Science or related field is preferred.