Data Engineer
Summary
Design, build, and maintain scalable data pipelines, ETL processes, and data warehouses that support analytics and business intelligence. Works with large datasets using SQL, Python, warehousing tools (Snowflake, Redshift, BigQuery), big data tech (Spark, Hadoop), and cloud platforms (AWS, Azure, GCP) on-site in Kimberley.
Our client is looking for a skilled Data Engineer to join their expanding data team in Kimberley. This role is essential for designing, building, and maintaining efficient and reliable data pipelines and infrastructure that support the organization's data analytics and business intelligence needs. You will work with large datasets, implement data warehousing solutions, and ensure the quality and accessibility of data for analysis. The ideal candidate has a strong background in data architecture, ETL processes, and database management.
Key Responsibilities
- Design, construct, install, test, and maintain highly scalable data management systems and pipelines.
- Develop and optimize ETL (Extract, Transform, Load) processes to consolidate data from various sources into data warehouses or data lakes.
- Ensure data quality, consistency, and availability across all data systems.
- Collaborate with Data Scientists and Analysts to understand their data requirements and provide them with reliable data sets.
- Monitor data pipeline performance and troubleshoot any issues to ensure timely data delivery.
- Implement data security and governance best practices within the data infrastructure.
Requirements
- Bachelor's degree in Computer Science, Information Technology, or a related quantitative field, or equivalent work experience.
- Minimum of 3 years of professional experience in data engineering or a related role.
- Proven experience in designing and building data pipelines using SQL and programming languages like Python.
- Hands-on experience with data warehousing solutions (e.g., Snowflake, Redshift, BigQuery) and ETL tools.
- Familiarity with big data technologies (e.g., Spark, Hadoop) and cloud platforms (AWS, Azure, GCP).
- Strong understanding of database design principles and data modeling techniques.
Benefits
- Competitive annual salary with performance-based incentives.
- Comprehensive health and retirement benefits package.
- Opportunities for professional growth and development in the data engineering field.
- A collaborative team environment located in Kimberley, focused on leveraging data for business success.
- Exposure to modern data technologies and architectures.