Data Engineer
Summary
Build and maintain scalable data pipelines and ETL/ELT processes using Python, SQL, and cloud platforms (AWS/Azure) to support analytics and AI-driven products.
About the Company:
Khonology is a digital services company focused on software development, Application Support, data analytics and engineering.
We are looking for a skilled Data Engineer to join our team. The ideal candidate will have a strong background in Python, AWS/Azure Cloud technologies, data pipeline management, ETL and ELT principles. You will be responsible for designing, building, and maintaining scalable and robust data systems to support our data-driven initiatives and productionizing data related systems.
Core Responsibilities and Competencies:
- Programming: Write efficient and maintainable code in Python and SQL.
- Data Pipeline Management: Design, develop, and maintain data pipelines to ensure smooth data flow from various sources to the data warehouse.
- Production Systems: Apply best practices in production systems management to ensure reliability and scalability.
- ETL and ELT Processes: Implement and optimize ETL (Extract, Transform, Load) and ELT (Extract, Load, Transform) processes to ensure data is correctly ingested, transformed, and loaded.
- Data API Integration: Develop and maintain data APIs to facilitate seamless data access and integration with other systems.
- Database Management: Manage and optimize relational databases to ensure high performance and reliability.
- Data Modeling: Design and implement data models to support data product and machine learning needs.
- Query Optimization: Ensure that queries are always optimized for performance to enhance system efficiency.
Our current tech stack:
The ideal candidate should be familiar with our current tech stack:
- Cloud Platforms: AWS, Microsoft Azure
- Main Programming language: Python, SQL (Domain-Specific Language)
- Code Repositories: BitBucket, GitHub, CodeCommit
- Deployment frameworks: Bamboo, Octopus Deploy, Codebuild and CodeDeploy
- Experience working with AI, Machine Learning, Generative AI, or data platforms that enable AI-driven products will be highly advantageous.