Lead Software Engineer - Data Platform
Posted Updated
Description
We have an opportunity to impact your career and provide an adventure where you can push the limits of what's possible.
As a Lead Software Engineer - Data Platform at JPMorgan Chase, within the Commercial & Investment Banking – Data Analytics – Payments Technology team, you are an integral part of an agile team that works to enhance, build, and deliver trusted market-leading technology products in a secure, stable, and scalable way. As a core technical contributor, you are responsible for conducting critical technology solutions across multiple technical areas within various business functions in support of the firm’s business objectives.
Job responsibilities
- Executes creative software solutions, design, development, and technical troubleshooting with ability to think beyond routine or conventional approaches to build solutions or breakdown technical problems
- Design, build and manage enterprise applications at scale to ensure robust and fault tolerant data processing and ingestion using Spark, Kafka, and Flink
- Drives team adoption of enterprise-authorized AI-assisted engineering practices within the work environment to improve code quality, delivery speed, and operational outcomes (e.g., AI-assisted code review/refactoring, test strategy acceleration, incident/root-cause analysis support), while establishing consistent validation standards (secure coding, peer review, automated testing) and promoting reuse of effective patterns across the team
- Applies knowledge of tools within the Software Development Life Cycle toolchain, including enterprise-authorized AI-assisted development and automation capabilities, to improve the value realized by automation.
- Design and build event-driven microservices workflows, using Kafka for inter-service communication and reliable asynchronous processing (at-least-once delivery, retries, and DLQ patterns).
- Partners with product managers and business stakeholders to translate requirements into production-grade solutions.
- Ensure production readiness through observability (metrics, logs, tracing), alerting, incident response runbooks, and proactive problem management.
- Develops secure high-quality production code, and reviews and debugs code written by others
- Identifies opportunities to eliminate or automate remediation of recurring issues to improve overall operational stability of software applications and systems
- Leads evaluation sessions with external vendors, startups, and internal teams to drive outcomes-oriented probing of architectural designs, technical credentials, and applicability for use within existing systems and information architecture
- Leads communities of practice across Software Engineering to drive awareness and use of new and leading-edge technologies.
Required qualifications, capabilities, and skills
- Formal training or certification on software engineering concepts and 5+ years of applied experience
- Hands-on practical experience delivering system design, application development, testing, and operational stability
- Demonstrated professional experience focused on software engineering or data platform development
- Demonstrated experience leading effective use of approved AI-assisted software development tools (e.g., for coding, code review, test acceleration, troubleshooting) with the ability to set team expectations for validating AI outputs for correctness, performance, and security.
- Strong understanding of responsible AI use in engineering workflows, including data sensitivity considerations, secure handling of inputs/outputs, and adherence to resiliency and security expectations; experience coaching engineers on safe, compliant adoption within delivery practices
- Advanced and hands on experience with development using Java, including performance tuning and optimizations.
- Hands-on experience with distributed data processing frameworks such as Apache Spark and Flink
- Solid understanding of data modeling techniques (star schema, snowflake) and query optimization
- Proficiency with cloud data services (AWS S3, Glue, Redshift, Athena, EMR, Lake Formation, or equivalent)
- Experience with AWS cloud platforms, including deploying and operating services in a cloud environment
- Advanced understanding of agile methodologies such as CI/CD, Application Resiliency, and Security
Preferred qualifications, capabilities, and skills
- Experience in designing and operating data pipelines on Databricks using orchestration tools such as Apache Airflow is a plus
- Experience with data modeling strategies (fact and dimensional, wide tables) to support analytics, reporting, and downstream consumption is a plus
- Demonstrated proficiency in software applications and technical processes within a technical discipline (e.g., cloud, artificial intelligence, machine learning, mobile, etc.)
Skills
- Agile
- AI
- Airflow
- Analytics
- Athena
- Automation
- AWS
- CI/CD
- Cloud
- Data Analytics
- Data Modeling
- Data Pipelines
- Databricks
- Event Driven Architecture
- Flink
- Information Architecture
- Java
- Kafka
- Machine Learning
- Microservices
- Observability
- Redshift
- Root Cause Analysis
- S3
- Secure Coding
- Snowflake
- Spark
- Test Automation