Senior Data Engineer - Azure Databricks

Role : Data Engineer | Azure | Databricks | Healthcare Tech

Join a globally distributed engineering team building cutting-edge data solutions that power real impact in healthcare! If you love solving complex data problems and thrive in a fast-paced, collaborative environment, this one's for you.

What You'll Do:

- Build a self-service data platform for reporting & analytics.

- Design robust data pipelines using Databricks, SQL, Python, Spark & Delta Lake on Azure (Blob Storage, Data Factory, Event Hubs).

- Champion ETL/ELT best practices data quality, partitioning, scalability & reusability.

- Collaborate with cross-functional teams to integrate data platforms with Tableau, Power BI, APIs, and custom apps.

- Drive CI/CD & automation using Git, Terraform, Azure DevOps/GitHub Actions/Jenkins, and monitoring tools like Azure Monitor/Datadog.

- Mentor peers and help shape a strong engineering culture.

Must-Have Skills:

- 5+ years of hands-on experience in Apache Spark & Databricks, building scalable pipelines and distributed data systems in cloud environments.

- Deep expertise in the Databricks ecosystem Delta Lake, Delta Live Tables (DLT), Unity Catalog, Workflow orchestration (Jobs).

- Strong programming skills in PySpark/Spark (Python or Scala preferred) for large-scale data engineering.

- Proven experience optimizing Spark jobs partitioning, caching, AQE, joins, skew handling.

- Strong Data Structures & Algorithms (DSA) fundamentals.

Good to Have:

- Integration experience with Azure Data Factory, Event Hubs/streaming pipelines, Airflow.

- Working knowledge of Azure cloud platforms Blob Storage, NoSQL DBs.

- Experience with relational databases (MS SQL, PostgreSQL, MySQL).

- Exposure to data governance & security Unity Catalog, RBAC, data lineage.

What You Bring:

- BS/MS/BE/B Tech Computer Science or related field (or equivalent experience).

Why Join Us?

Work on tech that directly transforms healthcare, with a team that values innovation, collaboration, and growth.

Interested?

Share your profile at snerangasamy@logixhealth for immediate processing!