W2 ONLY -Data Engineer ( Databricks, Scala , PYSPARK)

Experienced Senior Data Engineer to work on a focused migration project — converting existing Scala-based Spark pipelines to PySpark and transitioning job workloads from classic Databricks clusters to Databricks Serverless compute. The ideal candidate has hands-on experience with both Scala and PySpark, deep knowledge of Azure Databricks, and a strong understanding of Serverless compute constraints and best practices. Key Responsibilities Review and analyze existing Scala Spark jobs — identify …