Databricks Architect
Summary
Lead the architecture and implementation of enterprise-scale AWS Databricks platforms, designing Lakehouse and data warehouse solutions, optimizing data pipelines, and driving AI/BI modernization with technologies like Databricks, AWS S3/Redshift/Lake Formation, Spark/PySpark, Delta Lake, and Unity Catalog.
- 12+ years of experience in Data & Analytics.
- 5+ years of hands-on Databricks Architecture experience.
- Must have delivered at least one end-to-end Databricks implementation.
- Strong expertise in Databricks Data Engineering, ETL and data pipeline development.
- Strong AWS Cloud experience; AWS is the primary client environment.
- Hands-on experience with AWS S3, Redshift, Lake Formation and related data services.
- Strong Apache Spark / PySpark expertise.
- Experience designing Data Lakes, Data Warehouses and Lakehouse solutions.
- Strong understanding of Data Modeling, Data Architecture, Metadata, Governance and Data Quality.
- Experience with Databricks Unity Catalog and Delta Lake.
- Exposure to Mosaic AI, Databricks Genie and Agent Bricks.
- Experience with AI/GenAI, Data Modernization and Advanced Analytics is preferred.
- Strong consulting, stakeholder management and communication skills.
- Excellent problem-solving and delivery mindset.
- AWS Solution Architect Certification is preferred.
Responsibilities
- Lead architecture and implementation of enterprise-scale AWS Databricks platforms.
- Design scalable and secure Lakehouse, Data Lake and Data Warehouse architectures.
- Build and optimize Databricks data transformation and pipeline solutions.
- Integrate AWS S3, Lake Formation, DevOps pipelines and Databricks workflows.
- Drive data and AI platform modernization initiatives.
- Implement Databricks Genie, AI/BI and Agent Bricks capabilities.
- Define architecture strategy, governance, security and technical standards.
- Lead architecture reviews and mentor engineering teams.
- Partner with business and technical stakeholders to deliver enterprise solutions.