Skip to content
Search jobs inside your AI agent. Claude, ChatGPT, Claude Code, Codex, Gemini, Cursor and Grok
Connect

Ampcus Inc

NewBe an early applicant

Data Engineer Lead - W2

Posted Updated

Description

Data Engineer Lead - W2

As a Data Engineer Lead, you will play a pivotal role in designing, developing, and managing large-scale data pipelines and data lake patterns. You will lead a team of data engineers, driving the adoption of enterprise-authorized AI-assisted engineering practices to improve code quality, delivery speed, and operational outcomes. You will be responsible for building robust ETL/ELT pipelines, optimizing distributed processing workloads, and ensuring data quality, governance, and compliance.

Responsibilities:

  • Execute creative software solutions, design, development, and technical troubleshooting with the ability to think beyond routine or conventional approaches to build solutions or breakdown technical problems.
  • Drive team adoption of enterprise-authorized AI-assisted engineering practices within the work environment to improve code quality, delivery speed, and operational outcomes.
  • Lead architecture and hands-on delivery of large-scale data pipelines (ETL/ELT) covering ingestion, transformation, validation, reconciliation, and publishing across curated layers.
  • Design and operationalize data lake patterns, including partitioning strategies, data quality controls, lineage, governance, and reusable datasets/data products.
  • Build and optimize distributed processing workloads using Apache Spark and modern storage/file formats (e.g., Parquet, Avro).
  • Drive performance tuning across compute and storage (e.g., Spark tuning: shuffle, joins, caching, skew handling; and warehouse tuning where applicable).
  • Implement engineering best practices: code quality, automated testing, CI/CD, observability (metrics/logs/traces), security-by-design, and operational readiness.
  • Build and maintain Java-based services and components that support data workflows (e.g., ingestion services, orchestration helpers, APIs, data access layers).
  • Develop REST APIs and integration components to enable downstream consumption and platform interoperability.
  • Apply modern application engineering practices (clean architecture, SOLID principles, test automation, and secure coding practices).
  • Apply knowledge of tools within the Software Development Life Cycle toolchain, including enterprise-authorized AI-assisted development and automation capabilities, to improve the value realized by automation.

Requirements:

<>Required Skills:
  • Formal training or certification on software engineering concepts and 5+ years applied experience.
  • Demonstrated experience leading effective use of approved AI-assisted software development tools (e.g., for coding, code review, test acceleration, troubleshooting) with the ability to set team expectations for validating AI outputs for correctness, performance, and security.
  • Strong understanding of responsible AI use in engineering workflows, including data sensitivity considerations, secure handling of inputs/outputs, and adherence to resiliency and security expectations; experience coaching engineers on safe, compliant adoption within delivery practices.
  • 10+ years of software engineering experience, with strong depth in data engineering/big data platforms.
  • Strong hands-on development experience in Java plus proficiency in at least one data-focused language such as Python.
  • Proven experience designing and building robust ETL/ELT pipelines and data integration frameworks.
  • Strong experience with Apache Spark and distributed processing concepts (fault tolerance, partitioning, performance tuning).
  • Strong understanding of data storage serialization and formats such as Parquet and Avro.
  • Solid knowledge of data lake/lakehouse concepts and patterns (e.g., Medallion architecture), including data quality, metadata, and governance considerations.
  • Experience working with cloud data warehouses such as Snowflake (loading/unloading patterns, performance basics) or equivalent.
  • Ability to lead technical discussions, communicate clearly to varied stakeholders, and drive delivery in cross-functional environments.
<>Preferred Skills:
  • Experience with orchestration and workflow scheduling tools (e.g., Airflow, Dagster, Control-M, etc.).
  • Exposure to streaming/event-driven processing (e.g., Kafka, Spark Structured Streaming) and incremental processing patterns (CDC, upserts).
  • Experience with table formats / ACID layers (e.g., Delta Lake, Apache Iceberg, Hudi).
  • Strong production operations mindset (on-call practices, SLAs/SLOs, observability, incident response).
  • Experience in regulated environments (risk, audit, compliance, privacy, PII handling).
  • Experience with cost optimization / FinOps practices for data platforms.
  • Experience with working with Agentic AI / Gen AI solutions and capabilities will be an added advantage.

Skills

What Lead Data Engineering jobs ask for — and how much of it you have →
Apply

See also

Data Engineering jobs by country — openings, pay and top skills →