Junior Data Engineer
Summary
Data engineer in Crédit Agricole CIB's Digital Excellence Centre (Singapore) designing, building, and maintaining Python-based data pipelines (batch and real-time) to support ML and analytics, including backend APIs, data validation, and model deployment using Airflow, Docker/Kubernetes, SQL/NoSQL, and cloud platforms.
Position
In a challenging and multicultural environment, we are looking for a Data Engineer to join our Digital Excellence Centre (DEC) department of Crédit Agricole CIB. The department handles the development of transversal and international projects.
The ideal candidate is a Python programmer with hands-on experience building data pipelines and integrating data workflows to support ML and analytics use cases. You'll work closely with data engineers, ML engineers, and analysts to design, build, and maintain scalable data pipelines that are robust and production ready.
Main responsibilities
§ Design and develop data ingestion, transformation, and delivery pipelines using Python and modern ETL tools.
§ Build backend services, APIs or UIs (if ReactJS experience is present) to support ML applications
§ Maintain and optimize batch and real-time data pipelines for performance and scalability
§ Support the packaging and deployment of machine learning models into production environments.
§ Write clean, modular, testable code and participate in code reviews.
§ Collaborate with data scientists, engineers and DevOps teams to scalable and reliable data pipeline.
§ Support data exploration, feature engineering, and occasional model building where needed.
§ Automate data validation, quality checks, and pipeline monitoring.
§ Work with cloud platforms and container technologies (Docker, Kubernetes)
§ Follow best practices for versioning, logging, and CI/CD.
Qualifications and Profile
Mandatory:
§ Have Graduate or Master’s degree in the relevant field of AI / ML / Data.
§ 2+ years of experience in software development, ML engineering, or data pipeline engineering.
§ Strong programming skills in Python (pandas, scikit-learn, FastAPI or Flask)
§ Familiarity with SQL and working with relational databases or cloud data warehouses (e.g., BigQuery, Snowflake, Redshift).
§ Knowledge on NoSQL databases (any experience in Graph database is desirable)
§ Hands-on with data pipeline tools like Apache Airflow, Luigi, or similar tools.
§ Experience with version control (Git), testing, and agile development practices.
§ Experience with CI/CD pipelines and containerization (Docker, Kubernetes).
§ Exposure to cloud environments (AWS, GCP, or Azure) and containerization (Docker).
§ Strong debugging and problem-solving skills.
Preferred skills:
§ Experience in ReactJS or building frontend components for ML dashboards or monitoring tools.
§ Exposure to streaming data (e.g., Kafka, Spark Streaming).
§ Familiarity with MLOps tools like MLflow, TFX, or Seldon.
§ Basic understanding of machine learning concepts and model deployment lifecycle
Other Professional Skills and Mind-set
§ Ability and willingness to learn and adopt new technologies
§ Strong organizational and communication skills
§ Strong analytical and problem solving skills
§ Awareness of various software development procedures
§ Ability to follow defined procedures
§ Understanding and respect of cultural diversity
Skills
- Agile
- AI
- Airflow
- Analytics
- API
- AWS
- Azure
- BigQuery
- CI/CD
- Cloud
- Containerization
- Data Ingestion
- Data Pipelines
- DevOps
- Docker
- ETL
- FastAPI
- Feature Engineering
- Flask
- GCP
- Git
- Kafka
- Kubernetes
- Machine Learning
- MLflow
- MLOps
- Model Deployment
- NoSQL
- pandas
- Python
- React
- Redshift
- scikit-learn
- Snowflake
- Spark
- SQL
- Version Control