Senior Data Engineer: Azure Databricks & PySpark Expert
Posted Updated
Description
Applaudo is seeking a Senior Data Engineer to design, build, and maintain scalable data pipelines using Azure Databricks and Python. You will transform data from multiple sources into reliable datasets and ensure data quality and consistency across the data lifecycle.
You will collaborate with a US-based team, own pipelines end-to-end, and document all processes while advocating for best engineering practices and clear communication.
Design, build, and maintain production-grade data pipelines using Azure Databricks and Python. Develop scalable data transformations using PySpark or Spark SQL. Ingest data from APIs, relational databases, cloud storage, and SFTP. Integrate data from enterprise platforms into a centralized Data Hub. Maintain data warehouse structures and dimensional models. Write and optimize SQL transformations for accuracy and performance. Implement data quality validations and data consistency checks. Document data pipelines and technical processes. Collaborate with stakeholders to clarify requirements and manage risks. Proactively communicate progress and blockers. Proficiency in Azure Databricks, Python, and Apache Spark. Advanced SQL and data warehousing knowledge. Experience integrating data from multiple enterprise systems. Remote-possible? not specifiedSkills
Sign in to read and join the discussion.