Software Engineer III - GenAI Foundational Services
Posted Updated
We have an opportunity to impact your career and provide an adventure where you can push the limits of what’s possible.
As a Software Engineer III at JPMorganChase within the Chief Data and Analytics Office, you are an integral part of an agile team that works to enhance, build, and deliver trusted market-leading technology products in a secure, stable, and scalable way. You are responsible for contributing to critical technology solutions across multiple technical areas within various business functions in support of the firm’s business objectives—with a focus on Go-based backend services, gRPC/Protobuf, and bidirectional streaming to support LLM-enabled experiences.
Job responsibilities
- Execute creative software solutions, design, development, and technical troubleshooting for LLM-enabled applications and platforms.
- Design, build, and maintain backend services in Go, delivering secure, high-quality production code with strong testing discipline.
- Build gRPC services defined with Protobuf, including API design, schema evolution, and compatibility/versioning practices.
- Implement bidirectional streaming gRPC endpoints for LLM workloads (e.g., interactive sessions, token/partial-response streaming, incremental results, cancellations, backpressure/flow control).
- Integrate LLM services with supporting systems (authn/z, rate limits/quotas, caching, queues/streams, persistence) and partner with platform/SRE teams as needed.
- Review and debug SDK and service integrations; participate in code reviews and contribute to shared engineering standards.
- Instrument services with strong observability (metrics/logging/tracing), and contribute to runbooks and operational readiness (alerts, dashboards, on-call support as applicable).
- Identify recurring issues and contribute fixes and automation to improve reliability, latency, and operational stability.
- Add to the team culture of diversity, opportunity, inclusion, and respect.
Required qualifications, capabilities, and skills
- Formal training or certification on software engineering concepts and 3+ years applied experience
- Proficiency in Go for backend/service development, including concurrency patterns (goroutines/channels), context usage, and performance troubleshooting.
- Working knowledge of Python for supporting services, internal tooling, automation, evaluation harnesses, or data/ETL tasks.
- Hands-on experience with gRPC and Protobuf in production, including API design and backward/forward compatibility considerations.
- Experience building and operating microservices/APIs with a production mindset (monitoring, debugging, incident support, and continuous improvement).
- Proficiency in automation and continuous delivery methods; strong unit/integration testing discipline.
- Proficient in secure coding practices and core SDLC activities (design, build, test, deploy, operate).
Preferred qualifications, capabilities, and skills
- Experience building LLM streaming experiences end-to-end (token/partial-response streaming, tool/function calling orchestration, session state management, safety/guardrail hooks).
- Familiarity with OpenTelemetry and tracing/metrics patterns for streaming RPCs (correlation IDs, span links, per-message metrics).
- Experience with Kubernetes/Docker and operating services in containerized environments.
- Messaging/streaming systems (Kafka/RabbitMQ/SQS), caching (Redis), and patterns for async orchestration alongside streaming.
- Security best practices for RPC systems: mTLS, authn/z, secrets management, secure-by-default API design, abuse prevention (rate limiting, payload limits).
- Understanding of cost management and spend alerting for AI/LLM workloads.