Lead Data Engineer (m/f/d) for the BMP Migration Tool
Posted Updated
Description
softwareXperts GmbH is seeking a Lead Data Engineer (m/f/d) for the BMP Migration Tool. This role offers full remote work with 2–3 days per year on-site in Germany.
You will provide hands-on technical leadership, drive data profiling, define mappings, and build pipelines using Databricks, PySpark, Spark SQL, and Delta Lake. The ideal candidate has strong experience in data migrations, data quality monitoring, CI/CD, and excellent English communication in a hybrid setup.
Provide hands-on technical leadership for the BMP data migration. Lead profiling and assessment of source data. Define and implement data mappings and transformation rules. Develop and maintain data pipelines using Databricks, PySpark, Spark SQL and Delta Lake. Establish and oversee data quality checks and monitoring. Lead source-to-target reconciliation and resolve data discrepancies. Guide testing, CI/CD deployment and code quality practices. Prepare runbooks and support cutover activities Extensive hands-on experience delivering data migrations and building production-grade data pipelines. Deep Databricks expertise including Unity Catalog and Delta Lake. Strong PySpark and Spark SQL skills with data profiling and reconciliation. Experience with Databricks Data Quality Monitoring and Lakebridge. Proven technical leadership and ability to coordinate delivery.Skills
Sign in to read and join the discussion.