Senior Data Engineer
Posted Updated
Description
- At Emburse, you’ll not just imagine the future – you’ll build it. As a leader in expense intelligence, we are creating a future where technology drives business value and inspires extraordinary results. Our AI-powered platform helps organizations modernize financial operations, increase visibility, and optimize spend across the enterprise
- Architects, deploys, and maintains production-grade Python microservices, serverless components (AWS Lambda), batch workers, web services, and shared core libraries across the analytics ecosystem
- Engineers RESTful interfaces and application integrations to expose dataset capabilities to platform applications, encompassing secure object storage exchange, query orchestration, automated report delivery, and embedded analytics infrastructure
- Constructs backend infrastructure powering AI analytics solutions, incorporating LLM and natural language query translation engines like Snowflake Cortex, controlling dialog state, guaranteeing multi-tenant data isolation before execution, and optimizing throughput and resource utilization
- Authors clean, resilient codebase backed by comprehensive test suites while directing GitHub Actions continuous integration and continuous deployment pipelines for automated compilation, verification, security inspection, and production releases
- Safeguards system configuration and secrets using key-pair authentication, parameter management, and service credentials while proactively updating software dependencies based on vulnerability scanning metrics
- Constructs and manages ingestion and transformation workflows utilizing Airflow, AWS Glue, Spark, and Snowflake dynamic tables to feed multi-tier lakehouse environments from diverse sources
- Engineers automated data cleansing and enrichment routines integrating machine learning models and record reconciliation methods, while safely executing massive historical data re-processing
- Architects robust multi-tenant security frameworks in Snowflake employing row-level policies, secure functions, and isolated access controls across global cloud infrastructure regions
- Implements metadata-driven automation to dynamically generate client-specific semantic layers from templates, incorporating state tracking, automated recovery, and version rollbacks
- Executes data governance protocols and privacy compliance workflows, including systematic tenant data purging across all analytical storage tiers
- Develops internal platform analytics modules to track, aggregate, and report infrastructure consumption and operational expenditures across cloud vendor ecosystems
- Establishes proactive telemetry and alerting, drives resolution for system incidents, conducts thorough root-cause investigations, and deploys long-term technical remediation
- Follows SDLC processes, including adopting agile-based processes, peer code-reviews, and technical preparations required for scheduled releases and demos
- Partners with product, analytics and data science team to drive requirements that take into account all parties’ needs
- Establishes monitoring, responds to alerts and resolves issues within data pipelines
- Uses and helps shape AI-assisted development practices and tooling for the team
- Executes intricate, multi-faceted capabilities spanning microservices, automated workflows, and analytical storage infrastructure with high autonomy while providing technical leadership across mid-scale initiatives
- Assumes direct accountability for the reliability, security architecture, and long-term viability of deployed systems while systematically uncovering technical remediation opportunities
- Pinpoints architectural risks across peer technical deliverables and formulates pragmatic, alternative implementation strategies
- Streamlines operational processes, resolves critical system defects, performs advanced codebase refactoring, and exhibits expert troubleshooting capabilities across application code, cloud infrastructure, and analytical queries
- Authors comprehensive technical documentation, operational runbooks, and detailed data dictionaries to support platform capabilities
- Drives system architecture discussions and fulfills expanded peer code review obligations across the engineering organization
- Spearheads onboarding initiatives and provides technical coaching and mentorship to junior staff members
Benefits
- Flexible spending accounts
- Generous paid time off for every employee and flexible work schedules
- Paid leave for new parents
- Volunteering opportunities
- Health savings accounts (HSA)
- Medical, dental, vision, disability and life insurance plans
- Generous match to pre- or post-tax retirement savings accounts
- Financial planning services
- Program in partnership with eCornell to grow their knowledge and their careers
- Delicious supply of snacks and beverages
- Quarterly outings and Friday get-togethers to build a strong community at work
- Expert SQL query optimization, data modeling techniques, and extensive background delivering cloud warehouse architectures using Snowflake
- Demonstrated background of at least six years architecting enterprise software solutions and big data platforms in operational environments
- Hands-on expertise utilizing Amazon Web Services ecosystems, including AWS Lambda, S3 storage, Glue jobs, IAM policies, Key Management/Secrets Manager, and EventBridge integration
- Proven mastery deploying and orchestrating data processing flows in live production through Apache Airflow
- Advanced proficiency in Python engineering, spanning distributed microservices, internal modules, RESTful interface design, unit testing frameworks, and GitHub Actions release pipelines rather than ad-hoc scripting
- Track record of effective cross-functional partnership in product-driven organizations alongside engineering peers and product managers
- Required: Bachelor’s degree in Computer Science or related field, or equivalent years’ experience
- Deep comprehension of multi-tenant security architecture—incorporating row-level access policies, fine-grained access governance, strict tenant isolation, and compliant handling of restricted financial assets across international cloud regions
- Spark (AWS Glue or Databricks)
- Snowflake advanced features: row access policies, secure views, dynamic tables, stored procedures, Cortex AI / Cortex Analyst, semantic views
- Infrastructure as code and GitHub Actions
- Financial, expense or travel industry experience
- Experience with ML-assisted data matching or entity resolution
- Experience building LLM-powered or text-to-SQL features, or other GenAI applications in production
- Fivetran or an equivalent ELT tool; Looker or an equivalent BI platform, including embedded analytics
Skills
- Agile
- AI
- Airflow
- Analytics
- Authentication
- Automation
- AWS
- Aws Glue
- Big Data
- CI/CD
- Cloud
- Data Governance
- Data Modeling
- Data Pipelines
- Data Science
- Databricks
- ELT
- Eventbridge
- Fivetran
- Generative AI
- GitHub
- GitHub Actions
- IAM
- Infrastructure as Code
- Lakehouse
- Lambda
- LLM
- Looker
- Machine Learning
- Microservices
- Python
- REST
- S3
- SDLC
- Serverless
- Snowflake
- Spark
- SQL
- Unit Testing
- Vulnerability Scanning
Sign in to read and join the discussion.