Data Engineer
Posted
Summary
Senior/lead data engineer in Singapore who designs, builds, and scales data platforms and ETL/ELT pipelines using Python, SQL, Informatica, Airflow, and cloud services (AWS/GCP/Azure, S3, BigQuery/Redshift/Snowflake), while mentoring a small team and supporting ML products; 6+ years' experience and security clearance required.
Job:
- Own the design, development, and scaling of our data platform and pipelines.
- Lead a small team of data engineers, work closely with Data Science, Analytics, and Product, and ensure our data is reliable, scalable, and accessible for decision-making and ML products.
- Be hands‑on with architecture and coding, while also mentoring engineers and setting engineering standards.
- Design and implement scalable, reliable, and cost‑effective data pipelines and data platforms.
- Build and maintain ETL/ELT pipelines, data models, and APIs using Python, SQL, and modern cloud tools.
- Own data ingestion, storage, transformation, orchestration, and monitoring in cloud environments.
- Guide engineers, conduct code reviews, and drive best practices in data engineering.
- Partner with Data Science, Analytics, Product, and Business teams to translate requirements into technical solutions.
- Drive end‑to‑end delivery of data projects from requirements gathering to production deployment and support.
- Implement testing, monitoring, and data quality frameworks.
- Enforce data security, privacy, and compliance standards.
- Optimize pipeline performance, cost, and reliability.
- Troubleshoot production issues.
- Masters or Degree in Computer Science, Engineering, Mathematics, or related field.
- Equivalent practical experience will be considered.
- Has at least 6+ years in data engineering, with 2+ years in a lead/senior role.
- Strong in Python and SQL. Experience with PySpark is a plus.
- Hands‑on experience with AWS, GCP, or Azure. Services like S3/ADLS, BigQuery/Redshift/Snowflake, Airflow, Kafka, DBT.
- ETL/ELT Tools: Informatica PowerCenter / Informatica Intelligent Cloud Services (IICS).
- Experience with data integration, mappings, transformations, and workflow orchestration in Informatica.
- Strong experience with dimensional modeling, data warehousing, and lakehouse architectures.
- Experience with Airflow, Prefect, or similar workflow tools.
- Engineering practices: CI/CD, Docker, Git, Infrastructure as Code, unit testing.
- Good to Have: Kafka, Spark Streaming, Flink; Databases: PostgreSQL, MongoDB, Redis.
- Working with feature stores and model deployment pipelines.
- Experience hiring, mentoring, and doing technical interviews.
- Can explain complex technical concepts to non‑technical stakeholders.
- Proactive problem solver who takes end‑to‑end ownership.
- Works well across Data Science, Analytics, and Engineering teams.
- The role will need to secure a clearance, due to the sensitivity of the project.
EA Licence No: 20S0237
Skills
- Airflow
- Analytics
- API
- AWS
- Azure
- BigQuery
- CI/CD
- Cloud
- Data Engineering
- Data Ingestion
- Data Pipelines
- Data Quality
- Data Science
- Data Warehousing
- dbt
- Dimensional Modeling
- Docker
- ELT
- ETL
- Flink
- GCP
- Git
- Infrastructure as Code
- Kafka
- Lakehouse
- Machine Learning
- Model Deployment
- MongoDB
- PostgreSQL
- Prefect
- PySpark
- Python
- Redis
- Redshift
- Requirements Gathering
- S3
- Snowflake
- Spark
- SQL
- Unit Testing
- Workflow Orchestration