Data Engineer - Hong Kong Conglomerate
Summary
Data Engineer on a long-term enterprise data transformation programme for a Hong Kong conglomerate, working in an Agile Reporting & Downstream squad. Day to day: designing and building scalable AWS data pipelines using S3, Glue, Spark, PySpark and Python to consolidate data domains for Power BI reporting and analytics.
Salary: $45,000 – $65,000 per month
About the role
TEKsystems is looking for experienced Data Engineer to join a dedicated Reporting & Downstream squad supporting a major enterprise data transformation programme. This is a long-term capability build focused on developing and enhancing an AWS-based data platform, building scalable data pipelines and bringing together multiple data domains to support reporting, analytics and business performance insights.
Key responsibilities
Design, build and maintain scalable data pipelines
Ingest and transform data from multiple source systems into the enterprise data platform
Develop solutions using AWS S3, AWS Glue, Spark and Python
Work with PySpark and Spark SQL for large-scale data processing and transformation
Support the consolidation of different data domains into a unified data platform
Prepare data for downstream Power BI reporting and analytics
Enable analysis across business performance, P&L and different business segments
Work closely with Data QA, Product Owner, Business Analyst, Scrum Master and Technical Lead
Follow strong engineering practices around automation, testing, governance and documentation
Deliver within a fully Agile squad environment
About you
Strong experience with AWS, Amazon S3 AWS Glue, Apache Spark
Strong experience with Python / PySpark
Strong experience with Spark SQL
Experience with data lakes
Experience with ETL / ELT
Experience with data ingestion and transformation
Experience with data pipeline development