Databricks Data Engineer - Contract-to-Hire

We are seeking a Databricks Data Engineer to join our growing data engineering team in Pune, India. This role will play a key part in a large-scale modernization initiative to migrate a complex, enterprise-grade Microsoft SQL Server data warehouse ecosystem to the Databricks Lakehouse Platform. The ideal candidate has strong hands-on experience across Databricks data engineering capabilities, with exposure to AI/ML features being a plus, while maintaining a core focus on scalable, reliable data pipelines and analytics workloads.

Key Responsibilities

• Design, build, and optimize scalable data pipelines using Databricks (Apache Spark, Delta Lake, Unity Catalog).
• Participate in the migration of a ~20TB compressed on-prem Microsoft SQL Server data warehouse to Databricks.
• Convert and modernize hundreds of SQL Server tables, thousands of SSIS jobs, and downstream SSRS/SSAS workloads.
• Re-engineer SSIS ETL processes into Databricks notebooks, workflows, and orchestration frameworks.
• Support migration or redesign of cube-based analytics (SSAS) into Databricks SQL, Delta tables, and modern semantic models.
• Implement data quality, validation, reconciliation, and audit controls during migration.
• Optimize performance and cost through efficient Spark usage, partitioning, and query tuning.
• Collaborate with analytics, BI, and AI/ML teams to enable downstream reporting and advanced analytics.
• Apply data governance, security, and access-control standards using Unity Catalog.
• Contribute to reusable frameworks, documentation, and platform best practices.

• Bachelor’s degree in Computer Science, Engineering, or a related field.
• 4–7 years of overall data engineering experience.
• Hands-on experience with Databricks on AWS (preferred); Azure experience acceptable.
• Strong proficiency in Spark (PySpark and/or Scala) and SQL.
• Proven experience migrating on-prem SQL Server data warehouses to cloud-based data platforms.
• Experience converting SSIS-based ETL pipelines into Spark-based data engineering solutions.
• Solid understanding of data warehousing concepts, dimensional modeling, and analytical workloads.
• Experience with Delta Lake, incremental processing patterns, and data versioning.
• Familiarity with Databricks Workflows, Jobs, and production-grade deployments.
• Practical experience with performance tuning and large-volume data processing.

Preferred Skills

• Experience modernizing SSAS cube-based reporting solutions.
• Exposure to Databricks SQL Warehouses and BI integrations (Power BI preferred).
• Working knowledge of cloud-native data engineering practices aligned with Databricks best practices.
• Familiarity with MLflow, feature engineering, or AI-enablement within Databricks.
• Experience working in environments that follow Databricks-recommended data engineering patterns.
• Databricks certification is a plus.

Customized Energy Solutions India offers competitive compensation commensurate with experience, along with performance-based incentives aligned to individual and company results. Our benefits are designed to support both professional and personal well-being and include comprehensive medical insurance coverage for associates and eligible dependents, generous paid time off, and structured opportunities for professional development through training programs and skill enhancement initiatives. We are committed to creating an environment where associates are supported, challenged, and positioned for long-term career growth.

Customized Energy Solutions provides equal employment opportunities to all applicants without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability, or status as a protected veteran.