At JPMorganChase, we are building the infrastructure that powers the next generation of enterprise AI — and we need talented engineers who are passionate about LLM inference to help us do it. This is your opportunity to work at the intersection of cutting-edge machine learning and large-scale production systems, directly contributing to how one of the world's largest financial institutions deploys and optimizes AI at scale. As a Lead Software Engineer at JPMorganChase within the AI/ML Data Platform team, you will be a key technical contributor on LLM inference performance — supporting optimization strategy, benchmarking, and efficiency at scale. You will collaborate closely with senior engineers and engineering leadership to help shape how our platform evolves, ensuring every model we serve is fast, cost-efficient, and production-ready. This is a high-impact individual contributor role where your technical contributions will have direct, measurable influence on the firm's AI capabilities.Job ResponsibilitiesExecute systematic benchmarking and performance characterization across production LLM workloads, establishing reproducible baselines, identifying regressions, and quantifying the impact of configuration changes before they reach productionDesign and run quantization experiments — FP8, INT8/INT4 (GPTQ/AWQ), and next-generation precision formats — measuring accuracy delta, throughput improvement, memory reduction, and cost-per-token impactSupport speculative decoding strategy across the model portfolio, including draft model, n-gram, and multi-token prediction approaches, contributing to acceptance rate measurement and per-workload configuration recommendationsBuild and maintain GPU efficiency metrics covering utilization, memory headroom, cost per 1K tokens, and waste identification — providing engineering teams with a data-driven view of platform efficiencyBenchmark the platform against external providers and published industry numbers, identifying gaps and contributing to improvement initiativesParticipate in inference engine upgrade evaluations, including new scheduler architectures, async tensor parallelism, disaggregated prefill/decode, and advanced speculative decoding, supporting systematic validation before production promotionContribute to GPU chaos engineering efforts, including induced failure scenarios, hardware diagnostic monitoring, and detection and recovery measurementLeverage enterprise-authorized AI coding assist tools within the work environment to improve code quality, delivery speed, and productivity (e.g., code generation/refactoring, unit test creation, documentation), while validating outputs through peer review, automated testing, and secure coding standardsApply knowledge of tools within the Software Development Life Cycle toolchain, including enterprise-authorized AI-assisted development and automation capabilities, to improve the value realized by automationRequired qualifications, capabilities, and skillsFormal training or certification on software engineering concepts and advanced applied experience – preferably Go / Python Hands-on experience with LLM inference systems — vLLM, TensorRT-LLM, SGLang, LLM-D, or equivalent production serving enginesStrong understanding of GPU memory architecture, including KV cache sizing and dynamics, memory-bandwidth versus compute bottlenecks, and the practical implications of quantization at inference timeExperience with quantization techniques and their real-world tradeoffs at scaleFamiliarity with speculative decoding and the variables that drive acceptance rates in production workloadsRigorous benchmarking skills using GuideLLM, custom harnesses, or equivalent tooling, with the ability to support every performance claim with dataExperience operating in cloud GPU infrastructure at scale (AWS, Kubernetes-based managed inference services)Ability to communicate technical trade-offs clearly to engineering peers and senior stakeholdersHands-on experience using enterprise-authorized AI-assisted software development tools within the work environment (e.g., for coding, testing, troubleshooting, or documentation) with demonstrated ability to critically evaluate and validate AI-generated outputsUnderstanding of responsible AI use in engineering workflows, including data sensitivity considerations, secure handling of inputs/outputs, and adherence to resiliency and security expectationsPreferred qualifications, capabilities, and skillsExperience with disaggregated prefill/decode serving architecturesFamiliarity with GPU hardware diagnostics tools such as DCGM, NVML, or XID event trackingExperience with ML observability and production monitoring for inference workloadsAwareness of the LLM inference competitive landscape with a track record of applying industry benchmarks to drive platform improvementsJ.P. Morgan is a global leader in financial services, providing strategic advice and products to the world’s most prominent corporations, governments, wealthy individuals and institutional investors. Our first-class business in a first-class way approach to serving clients drives everything we do. We strive to build trusted, long-term partnerships to help our clients achieve their business objectives.
We recognize that our people are our strength and the diverse talents they bring to our global workforce are directly linked to our success. We are an equal opportunity employer and place a high value on diversity and inclusion at our company. We do not discriminate on the basis of any protected attribute, including race, religion, color, national origin, gender, sexual orientation, gender identity, gender expression, age, marital or veteran status, pregnancy or disability, or any other basis protected under applicable law. We also make reasonable accommodations for applicants’ and employees’ religious practices and beliefs, as well as mental health or physical disability needs. Visit our FAQs for more information about requesting an accommodation.
Our professionals in our Corporate Functions cover a diverse range of areas from finance and risk to human resources and marketing. Our corporate teams are an essential part of our company, ensuring that we’re setting our businesses, clients, customers and employees up for success.Full timePosting Date: 2026-08-31