Data Engineer

Data Engineer

Data Product Design & Delivery

• Design and deliver end-to-end data products for assigned business domains across the bronze, silver, and gold layers, including the dimensional and semantic models that serve reporting and AI.

• Build robust ingestion from Amana's enterprise systems — Oracle Fusion (Finance, HCM, P2P, CRM), Hyperion, CMiC, project and operations systems, SharePoint, and Power Platform — using change data capture, incremental, and API-based patterns.

• Lead workstreams within the Synapse Analytics to Databricks migration: re-engineering pipelines, running parallel loads, reconciling results, and preparing for cutover.

• Contribute reusable frameworks, templates, and patterns, and propose improvements to the standards owned by the Data Engineering Lead (Data Architect).

Data Quality, Governance & Security

• Implement data quality frameworks, monitoring, and reconciliation across your domains, and drive issues to resolution at source with data owners.

• Implement access control, lineage, classification, and data-handling rules in Unity Catalog and Microsoft Purview, including PDPL controls for personal and HR data.

• Partner with the Data Governance Office to turn domain policies into working controls and documented, certified datasets.

Performance, Reliability & Cost

• Tune Spark and SQL workloads for performance and cost within the platform's cluster and workload policies.

• Own the operational health of your domains: monitoring, alerting, incident response, and root-cause fixes.

• Apply CI/CD, automated testing, and DataOps practices using Git and Azure DevOps.

Analytics & AI Enablement

• Build AI-ready data products — feature tables and document and vector pipelines — for machine learning, RAG, and agent use cases, in partnership with the Data Scientist and AI Engineer.

• Deliver Power BI semantic models and certified datasets with the Data Analyst, supporting self-service adoption through the Amana Digital Academy.

• Apply AI-assisted engineering — code generation, testing, documentation, and lineage discovery — with appropriate human review, and help the team adopt it safely.

Collaboration & Mentoring

• Translate requirements from finance, project, and operations stakeholders into technical designs with clear delivery outcomes.

• Mentor Data Engineers through pairing, design reviews, and code review.

• Explain design choices, trade-offs, and risks clearly to technical and business audiences.

• 5+ years of hands-on data engineering experience, including at least 2 years with Databricks, Apache Spark, and Delta Lake in production.

• Designed and delivered end-to-end data products — from ingestion to modelled, consumable data — for one or more business domains.

• Experience integrating complex enterprise sources such as Oracle Fusion, SAP, Microsoft Dynamics, or CMiC, including incremental and API-based extraction.

• Data quality and access controls implemented in practice using Unity Catalog, Microsoft Purview, or equivalent tooling.

• Strong SQL, Python, and PySpark, with Git-based CI/CD delivery (Azure DevOps or equivalent).

• Excellent written and verbal communication skills across technical and business audiences.
  
  Required (held, or achieved after joining within the period shown):

• Databricks Certified Data Engineer Associate or Microsoft Certified: Azure Databricks Data Engineer Associate (DP-750) — within 6 months

• Databricks Certified Data Engineer Professional — within 12 months

• Preferred (one or more):

• Databricks Certified Generative AI Engineer Associate

• Microsoft Certified: Fabric Data Engineer Associate (DP-700)

• Microsoft Certified: Fabric Analytics Engineer Associate (DP-600)

• Microsoft Certified: Azure Data Engineer Associate (DP-203 — retired 2025; accepted if held)