Skip to content
Search jobs inside your AI agent. Claude, ChatGPT, Claude Code, Codex, Gemini, Cursor and Grok
Connect

careers.hcltech.com

NewBe an early applicant

Technical Architect (DPO)

Posted Updated

Description

Job Summary

Job Description : Data Engineer

Key Responsibilities

Must-Have Skills

Databricks: Expert Databricks Engineer, Delta Lake, DLT / Lakeflow, Streaming Pipelines, Workflow Orchestration, PySpark

Advanced skills: Structured Streaming, Window Functions, Partitioning, Broadcast joins, Adaptive Query Execution, Performance Tuning

SQL Expert level: Plain Text, Advanced Joins, CTEs, Window Functions, MERGE, Subqueries, Recursive CTEs, Query Optimization, Execution Plans

AWS: S3, Glue, IAM, Lambda, CloudWatch, Kinesis

Data Quality: DQX, Great Expectations, Delta Expectations, Reconciliation Frameworks

AI Data Engineering: Chunking methods, Document ingestion, Embedding pipelines, Vector Search implementation, Metadata enrichment

CICD: Git, Azure DevOps / GitHub Actions, Terraform, Databricks Asset Bundles

Required Skills & Qualifications:

  • Bachelor’s or Master’s degree in computer science, Information Systems, or a related field.
  • 7-10 years of experience in data governance, data management, or a related field.
  • Proven experience in data engineering and data governance roles.
  • Proficiency in SQL, Python, and ETL tools. Experience with cloud data platforms. Familiarity with data governance tools.
  • Strong understanding of Databricks, data modeling, data warehousing, and data architecture.
  • Experience with data quality management, metadata management, and data lineage practices.
  • Excellent analytical, problem-solving, and decision-making skills.
  • Strong communication and interpersonal skills, with the ability to work effectively with cross-functional teams.
  • Ability to manage multiple projects and priorities in a fast-paced environment.

Skill Requirements

Must-Have Skills

Databricks: Expert Databricks Engineer, Delta Lake, DLT / Lakeflow, Streaming Pipelines, Workflow Orchestration, PySpark

Advanced skills: Structured Streaming, Window Functions, Partitioning, Broadcast joins, Adaptive Query Execution, Performance Tuning

SQL Expert level: Plain Text, Advanced Joins, CTEs, Window Functions, MERGE, Subqueries, Recursive CTEs, Query Optimization, Execution Plans

AWS: S3, Glue, IAM, Lambda, CloudWatch, Kinesis

Data Quality: DQX, Great Expectations, Delta Expectations, Reconciliation Frameworks

AI Data Engineering: Chunking methods, Document ingestion, Embedding pipelines, Vector Search implementation, Metadata enrichment

CICD: Git, Azure DevOps / GitHub Actions, Terraform, Databricks Asset Bundles

Required Skills & Qualifications:

  • Bachelor’s or Master’s degree in computer science, Information Systems, or a related field.
  • 7-10 years of experience in data governance, data management, or a related field.
  • Proven experience in data engineering and data governance roles.
  • Proficiency in SQL, Python, and ETL tools. Experience with cloud data platforms. Familiarity with data governance tools.
  • Strong understanding of Databricks, data modeling, data warehousing, and data architecture.
  • Experience with data quality management, metadata management, and data lineage practices.
  • Excellent analytical, problem-solving, and decision-making skills.
  • Strong communication and interpersonal skills, with the ability to work effectively with cross-functional teams.
  • Ability to manage multiple projects and priorities in a fast-paced environment.

Other Requirements

Skills

Apply