Senior Executive (Data Engineer), AIO Management Decision Support Office
Summary
Build and maintain ETL pipelines, data warehouses, and cloud storage for healthcare analytics using Databricks, SQL, and Python.
- Data Pipeline Development & Management
- Design, develop, and maintain robust ETL/ELT pipelines to process large volumes of structured data
- Implement batch data processing solutions using Databricks
- Monitor data pipeline performance and troubleshoot issues to ensure reliable data flow
- Optimize existing pipelines for improved performance and cost efficiency
- Data Architecture & Infrastructure
- Collaborate with senior engineers to design and implement scalable data architectures
- Build and maintain data warehouses, data lakes, and cloud-based storage solutions
- Ensure data quality, consistency, and reliability across all systems and datamarts
- Implement data governance best practices and security protocols defined by MOH HIM policies and DGPO
- Technical Implementation
- Develop and maintain notebooks for data access and integration
- Write clean, efficient, and well-documented code following best practices
- Implement automated testing and deployment processes for data pipelines
- Collaboration & Communication
- Partner with data scientists, analysts, clinical and business stakeholders to understand data requirements
- Provide technical guidance on data availability, limitations, and best practices to stakeholders
- Document data transformation processes, onboarding processes and automation processes
- Participate in code reviews and contribute to team knowledge sharing
Requirements - Bachelor's degree in Computer Science, Information Systems, Analytics, or a relevant field of study
- 1-3 years of experience in data engineering or a related role
- Proficient in SQL and at least one programming language (Python, Java, Scala)
- Decent understanding of data warehousing concepts and ETL processes
- Familiarity with cloud platforms such as AWS and Databricks
- Familiarity with Git commands and Unix/Linux system experience
- Experience with data visualization tools (Tableau, Power BI)
- Strong analytical and problem-solving skills
- Familiar with database management, architecture design, system integration, security and data governance
- Good interpersonal skills, a detail-oriented and flexible person who can work across different areas within the team as well as align cross functionally outside the team
- Ability to deliver clear, concise reports and presentations/dashboards and effectively articulate observations and recommendations