Mid-Senior Data Engineer (Databricks | Healthcare)
Description
About the role
We are looking for an experienced Mid-Senior Data Engineer with hands-on experience in Databricks and a background in the healthcare industry. The ideal candidate will be responsible for designing, developing, implementing, and maintaining scalable data pipelines and data engineering solutions to support the organization's data analytics and business intelligence initiatives.
Key responsibilities
Design, develop, and maintain scalable data pipelines using Databricks.
Build and optimize ETL/ELT workflows for processing large volumes of structured and unstructured data.
Work with Apache Spark, PySpark, and SQL to transform, process, and manage complex datasets.
Develop and maintain data lakes, lakehouses, and data warehouse solutions.
Integrate data from multiple sources, particularly healthcare systems and applications.
Ensure data accuracy, consistency, quality, and reliability across data pipelines.
Implement data security, privacy, and governance practices when handling sensitive healthcare information.
Optimize data processing performance, troubleshoot pipeline issues, and improve system efficiency.
Collaborate with data analysts, data scientists, software engineers, and other stakeholders to deliver data solutions.
Document data architecture, workflows, technical processes, and best practices.
About you
Bachelor's degree in Computer Science, Information Technology, Data Engineering, or a related field.
Mid-Senior level experience in Data Engineering or a similar technical role.
Hands-on experience using Databricks is required.
Strong proficiency in Python, PySpark, and SQL.
Experience developing and managing ETL/ELT data pipelines.
Solid understanding of data warehousing, data lakes, and lakehouse architecture.
Experience working with cloud platforms such as Microsoft Azure, AWS, or Google Cloud.
Background in the healthcare industry, preferably involving healthcare data, systems, or applications.
Understanding of healthcare data privacy, security, and compliance requirements.
Strong analytical, problem-solving, and troubleshooting skills.