Associate Data Engineer
Summary
Build and maintain cloud-based data pipelines on AWS and Databricks, ensuring high uptime and performance while optimizing costs and implementing robust monitoring and incident response.
Job Type : 1-year renewable contract
Associate Data Engineer Job Requirement: • Degree in Computer Science or Computer Engineering • Minimum 6 years working experience in system operations compliance and management areas • Project hands-on experience specifically with AWS platform (primary requirement) • project experience in cloud operations or cloud architecture • Must be cloud certified (AWS) Core Technical Skills: • proficiency in Databricks platform, including workspace management, cluster configuration, and job orchestration • Strong expertise in Apache Spark within Databricks environment, including Spark SQL, DataFrames, and RDDs • Good in-depth understanding of data warehouse concepts, data profiling, data verification and advanced analytics techniques • Strong knowledge of monitoring, incident management, and cloud cost control Technology Stack Experience: • Databricks • AWS cloud services and architecture • IDMC (Informatica Data Management Cloud) • Tableau for data visualization • Oracle Database management • ML Ops practices within Databricks environment • STATA for statistical analysis is advantage • Amazon SageMaker integration with Databricks • DataRobot platform integration Desirable Requirements: • AWS certification (Associate or Professional level) - highly preferred • Exposure to hospital information/clinical systems is an added advantage • Understanding of DevOps practices and CI/CD pipelines for Databricks-based data engineering projects Responsibilities: Operational Responsibilities: • Monitor and maintain production data pipelines to ensure 99.9% uptime and optimal performance • Implement comprehensive logging, alerting, and monitoring systems using Application monitoring tools • Perform regular health checks performance, job execution times, and resource utilization to identify and resolve bottlenecks proactively • Manage incident response procedures for pipeline failures, including root cause analysis, resolution, and post-incident reviews • Establish and maintain disaster recovery procedures and backup strategies for critical data assets within the Databricks environment • Conduct regular performance tuning of Spark jobs and Databricks cluster configurations to optimize cost and execution efficiency • Maintain comprehensive documentation for operational procedures, runbooks, and troubleshooting guides • Coordinate scheduled maintenance windows and system upgrades with minimal business impact • Manage user access controls, workspace configurations, and security policies within Application environments.
Kindly revert with below details in addition to updated resume to (HIDDEN TEXT) or whatsapp
+ 65 9473 3867 for faster response.
Full Name as per the Passport or IC : Current Salary Exp Salary (in SGD) Notice period (availability) Reason for Change Residential Status in SG
Thanks & Best Regards Rekha N Senior Consultant Whatsapp : +65 9473 3867 (HIDDEN TEXT) https://www.helius-tech.com/careers/ www.helius-tech.com Singapore -Malaysia- India - Thailand – Japan www.technoproholdings.com/en/ Agility | Flexibility | Transparency
Associate Data Engineer Job Requirement: • Degree in Computer Science or Computer Engineering • Minimum 6 years working experience in system operations compliance and management areas • Project hands-on experience specifically with AWS platform (primary requirement) • project experience in cloud operations or cloud architecture • Must be cloud certified (AWS) Core Technical Skills: • proficiency in Databricks platform, including workspace management, cluster configuration, and job orchestration • Strong expertise in Apache Spark within Databricks environment, including Spark SQL, DataFrames, and RDDs • Good in-depth understanding of data warehouse concepts, data profiling, data verification and advanced analytics techniques • Strong knowledge of monitoring, incident management, and cloud cost control Technology Stack Experience: • Databricks • AWS cloud services and architecture • IDMC (Informatica Data Management Cloud) • Tableau for data visualization • Oracle Database management • ML Ops practices within Databricks environment • STATA for statistical analysis is advantage • Amazon SageMaker integration with Databricks • DataRobot platform integration Desirable Requirements: • AWS certification (Associate or Professional level) - highly preferred • Exposure to hospital information/clinical systems is an added advantage • Understanding of DevOps practices and CI/CD pipelines for Databricks-based data engineering projects Responsibilities: Operational Responsibilities: • Monitor and maintain production data pipelines to ensure 99.9% uptime and optimal performance • Implement comprehensive logging, alerting, and monitoring systems using Application monitoring tools • Perform regular health checks performance, job execution times, and resource utilization to identify and resolve bottlenecks proactively • Manage incident response procedures for pipeline failures, including root cause analysis, resolution, and post-incident reviews • Establish and maintain disaster recovery procedures and backup strategies for critical data assets within the Databricks environment • Conduct regular performance tuning of Spark jobs and Databricks cluster configurations to optimize cost and execution efficiency • Maintain comprehensive documentation for operational procedures, runbooks, and troubleshooting guides • Coordinate scheduled maintenance windows and system upgrades with minimal business impact • Manage user access controls, workspace configurations, and security policies within Application environments.
Kindly revert with below details in addition to updated resume to (HIDDEN TEXT) or whatsapp
+ 65 9473 3867 for faster response.
Full Name as per the Passport or IC : Current Salary Exp Salary (in SGD) Notice period (availability) Reason for Change Residential Status in SG
Thanks & Best Regards Rekha N Senior Consultant Whatsapp : +65 9473 3867 (HIDDEN TEXT) https://www.helius-tech.com/careers/ www.helius-tech.com Singapore -Malaysia- India - Thailand – Japan www.technoproholdings.com/en/ Agility | Flexibility | Transparency