Data Engineer
Summary
The Data Engineer will design, build, and maintain scalable data pipelines and systems using Python, SQL, and cloud platforms like AWS or Azure. The role focuses on ETL/ELT processes, database optimization, and API integration to support data-driven initiatives.
About the Company
Khonology is a digital services company focused on software development, Application Support, data analytics and engineering.
We are looking for a skilled Data Engineer to join our team. The ideal candidate will have a strong background in Python, AWS/Azure Cloud technologies, data pipeline management, ETL and ELT principle. You will be responsible for designing, building, and maintaining scalable and robust data systems to support our data-driven initiatives and productionizing data related systems.
Core Responsibilities and Competencies
- Programming: Write efficient and maintainable code in Python and SQL.
- Data Pipeline Management: Design, develop, and maintain data pipelines to ensure smooth data flow from various sources to the data warehouse.
- Production Systems: Apply best practices in production systems management to ensure reliability and scalability.
- ETL and ELT Processes: Implement and optimize ETL (Extract, Transform, Load) and ELT (Extract, Load, Transform) processes to ensure data is correctly ingested, transformed, and loaded.
- Data API Integration: Develop and maintain data APIs to facilitate seamless data access and integration with other systems.
- Database Management: Manage and optimize relational databases to ensure high performance and reliability.
- Data Modeling: Design and implement data models to support data product and machine learning needs.
- Query Optimization: Ensure that queries are always optimized for performance to enhance system efficiency.
Our current tech stack
The ideal candidate should be familiar with our current tech stack:
- Cloud Platforms: AWS, Microsoft Azure
- Main Programming language: Python, SQL (Domain-Specific Language)
- Code Repositories: BitBucket, GitHub, CodeCommit
- Deployment frameworks: Bamboo, Octopus Deploy, Codebuild and CodeDeploy
- Experience working with AI, Machine Learning, Generative AI, or data platforms that enable AI-driven products will be highly advantageous.