Expert Data Engineer
Posted Updated 4
views
Description
Job Description
About the Role: The Data Engineer will work on enterprise-wide data provisioning, spanning multiple data governance domains and data assets. Responsibilities include ensuring secure data sharing, adhering to protection and compliance requirements, supporting enterprise Data & Analytics initiatives (including high-priority use cases), and enabling data provisioning for operational processes.
Role Responsibilities- Building and maintaining large-scale Big Data pipelines on cloud-based data platforms
- Ensuring secure and compliant data sharing aligned with information classification standards
- Supporting enterprise Data & Analytics initiatives and high-priority use cases
- Continuously improving and automating data engineering processes
- Evaluating emerging tools and technologies to drive innovation
- Mentoring and upskilling team members
- Maintaining high-quality technical documentation
- Terraform
- Docker
- Linux / Unix
- Cloud Formation
- Code Build / Code Pipeline
- Cloud Watch
- SNS
- S3
- Kinesis Streams (Kinesis, Firehose)
- Lambda
- Dynamo DB
- Step Functions
- Parameter Store
- Secrets Manager
- Python 3.x
- SQL (Oracle / Postgre SQL)
- Py Spark
- Boto3
- ETL development
- Big Data platforms
- Power Shell / Bash
- Glue
- Athena
- Technical data modelling & schema design (hands-on, not drag-and-drop)
- Kafka
- AWS EMR
- Redshift
- Business Intelligence (BI) experience
- Strong data governance and security understanding
- Advanced data modelling expertise, especially in Oracle SQL
- Strong analytical skills for large, complex datasets
- Experience with testing, data validation, and transformation accuracy
- Excellent documentation, written, and verbal communication skills
- Ability to work independently, multitask, and collaborate within teams
- Experience building data pipelines using AWS Glue, Data Pipeline, or similar
- Familiarity with AWS S3, RDS, and Dynamo DB
- Solid understanding of software design patterns
- Experience preparing technical specifications, designing, coding, testing, and debugging solutions
- Strong organisational abilities
- Knowledge of Parquet, AVRO, JSON, XML, CSV
- Experience with Data Quality tools such as Great Expectations
- Experience working with REST APIs
- Basic networking knowledge and troubleshooting skills
- Understanding of Agile methodologies
- Experience with documentation tools such as Confluence and JIRA
- Relevant IT, Business, or Engineering Degree
- Experience developing technical documentation and artefacts
- Experience with enterprise collaboration tools
- AWS Cloud Practitioner
- AWS Sys Ops Associate
- AWS Developer Associate
- AWS Architect Associate
- AWS Architect Professional
- Hashi Corp Terraform Associate
Skills
- Agile
- Analytics
- API
- Athena
- Avro
- AWS
- Aws Glue
- Bash
- Big Data
- Business Intelligence
- Cloud
- Confluence
- Data Engineering
- Data Governance
- Data Modeling
- Data Pipelines
- Data Quality
- Design Patterns
- Docker
- ETL
- Jira
- JSON
- Kafka
- Kinesis
- Lambda
- Linux
- Networking
- Oracle
- Parquet
- Python
- RDS
- Redshift
- REST
- S3
- SNS
- Spark
- SQL
- Terraform
- Unix
- XML
Sign in to read and join the discussion.