Senior Data Engineer
Posted Updated
- We are seeking a highly skilled and innovative Senior Data Engineer to join our team
- In this role, you will be responsible for designing, building, and managing our data infrastructure and analytics solutions to enable data-driven decision-making across the organization
- You will work on complex datasets and collaborate with cross-functional teams to unlock insights and deliver actionable intelligence
- Design, develop, and optimize big data pipelines, architectures, and data integration workflows
- Build scalable and efficient data storage solutions, such as data lakes, warehouses, or cloud-based systems
- Collaborate with data scientists, analysts, and stakeholders to understand data requirements and deliver solutions
- Develop and maintain ETL processes for ingesting, transforming, and loading data from diverse sources
- Conduct advanced data analysis, modelling, and visualization to identify patterns and insights
- Ensure data quality, reliability, and security across systems
- Work with distributed computing frameworks (e.g., Spark, Hive, Hadoop) to process and analyse large datasets
- Stay up to date with the latest trends and technologies in big data, analytics, and cloud platforms
- Hands-on experience with Databricks, SQL, and data modelling
- Excellent problem-solving and interpersonal skills
- Solid grasp of ETL workflows within databases / warehouses such as Oracle, PostgreSQL, SQL Server etc
- Proficiency with big data tools (e.g., Apache Spark, Apache Kafka, Presto, etc.) and cloud services (e.g., Azure, AWS, GCP)
- Bachelor’s / master’s degree or equivalent experience in computer science engineering, or a related field
- 10+ years of meaningful experience in big data and analytics
- Strong programming skills in languages such as Python, Java, or Scala
- Familiarity with data visualization tools and business intelligence (e.g., Power BI)
- Experience with machine learning frameworks and tools such as Azure ML Studio, AWS Sagemaker etc
- Knowledge of real-time data processing and stream analytics such as Apache Kafka, Azure Event Hubs or similar
- Hands on experience with real-time analytical databases such as Clickhouse or Apache Druid/Pinot or similar