Senior Data Engineer / Data Architect - Databricks

Job Title: Senior Data Engineer / Data Architect - Databricks
Work Arrangement: Onsite
Location: Charlotte, NC
Employment Type: Contract
Engagement Type: W2/1099
Final Interview:
Pay rate- 40-50/hr

Screening Note: Databricks and Insurance domain experience are mandatory. candidates with 12+ years of experience only.

Position Summary

Dhaka Technologies Limited Company is hiring on behalf of our client for a Senior Data Engineer / Data Architect with deep Databricks and insurance domain expertise. This is an onsite contract engagement in Charlotte, NC, and requires a minimum of 12 years of professional experience in data engineering and data architecture.

The selected consultant will design, develop, and implement scalable enterprise data solutions on Databricks, working closely with business stakeholders, data architects, engineers, analysts, and technology teams to build modern data platforms and analytics capabilities.

Key Responsibilities

• Design and develop scalable data architecture and data engineering solutions using Databricks.
  
• Build and maintain robust ETL and ELT data pipelines using Databricks, Apache Spark, Python, and SQL.
  
• Design data lakes, lakehouse architectures, data warehouses, and enterprise data platforms.
  
• Develop high performance batch and streaming data pipelines.
  
• Implement data ingestion from multiple internal and external sources.
  
• Develop data transformation, cleansing, validation, and integration processes.
  
• Design scalable and reusable data models for analytics and reporting.
  
• Work with Delta Lake, Delta Live Tables, Unity Catalog, and Databricks Workflows where applicable.
  
• Optimize Spark jobs, SQL queries, pipelines, and data processing workloads.
  
• Implement data quality, data governance, security, lineage, and access control processes.
  
• Collaborate with data scientists, BI teams, business analysts, product owners, and application teams.
  
• Translate business requirements into technical data architecture and engineering solutions.
  
• Participate in architecture reviews and establish data engineering best practices.
  
• Troubleshoot production data issues and provide root cause analysis.
  
• Mentor junior and mid level data engineers.
  
• Support cloud migration and modernization initiatives.
  

Insurance Domain Experience

Insurance domain experience is mandatory. A strong working knowledge of Property and Casualty (P&C) insurance data is highly preferred, and experience in Life, Health, or other insurance lines will also be considered.

The consultant should have hands-on experience with insurance data domains such as policy, policyholder, customer, claims, premium, billing, underwriting, rating, coverage, loss, agent and broker, payments, risk, product, quote, and policy administration. Experience integrating data from policy administration, claims, billing, underwriting, and other core insurance applications is a strong plus.

Required Technical Skills

• Databricks
  
• Apache Spark and PySpark
  
• Python
  
• SQL
  
• Data engineering and data architecture
  
• ETL and ELT design and development
  
• Data lake and lakehouse architecture
  
• Delta Lake
  
• Data modeling
  
• REST APIs and data integration
  
• Git and CI/CD
  

Cloud Experience

Strong experience with at least one major cloud platform is required: Microsoft Azure, Amazon Web Services, or Google Cloud Platform. Azure Databricks experience is highly preferred. Experience with Azure Data Factory, Azure Data Lake Storage, AWS S3, AWS Glue, Snowflake, Kafka, and Airflow is a plus.

Databricks Experience

• Databricks Workspace
  
• Apache Spark and PySpark
  
• Delta Lake and Delta Live Tables
  
• Unity Catalog
  
• Databricks Workflows
  
• Databricks SQL
  
• Cluster configuration, optimization, and performance tuning
  
• Data governance and data security within Databricks
  
• CI/CD for Databricks
  

Qualifications

• Bachelor's degree in Computer Science, Information Technology, Engineering, or a related field.
  
• Minimum of 12 years of professional experience in data engineering, data architecture, or related roles.
  
• Strong hands-on experience with Databricks and Apache Spark.
  
• Strong Python and SQL development experience.
  
• Proven experience designing enterprise scale data platforms.
  
• Strong understanding of data modeling and data integration.
  
• Experience working in Agile and Scrum environments.
  
• Strong communication and stakeholder management skills.
  
• Insurance industry experience is mandatory.
  

Preferred Qualifications

• Databricks certification.
  
• Cloud platform certification.
  
• Experience with enterprise insurance platforms.
  
• Experience with Property and Casualty insurance data.
  
• Experience with data governance and master data management.
  
• Experience with real time and streaming data.
  
• Experience with cloud migration and legacy modernization.
  
• Experience leading data architecture initiatives.
  

How to Apply

Qualified candidates are invited to submit an updated resume to talent@dhakatech.us with the position title in the subject line.