LiftmyCVLiftmyCV

Member of Technical Staff, Inference Systems

Inco AIHybridPalo Alto, US
AI job summary

Inco is seeking a founding member to enhance the performance of inference systems for AI models. The role demands expertise in optimizing serving stacks, improving latency, and managing costs associated with token delivery across GPUs. Ideal candidates will possess a strong technical background in GPU systems, modern AI workflows, and coding capabilities in Python and systems languages. As part of a flat team, you'll have ownership of significant performance metrics and contribute to innovative AI infrastructure development. In-person presence is preferred in Palo Alto or Brooklyn.

Job description

About the role

We are looking for founding members to build the systems that serve our frontier models at scale: the kernels, runtimes, batching, KV cache, load balancing, and scheduling that determine how quickly and cost-effectively each token is delivered across thousands of GPUs. As agentic workloads drive inference demand sharply higher, solving these problems will define the next generation of AI infrastructure. Your mandate is to unlock the next order-of-magnitude gain in serving performance. You will work across the serving stack to improve latency, throughput, and cost per token, and when it is the highest-leverage path, go deeper into runtimes, kernels, quantization, or speculative decoding. We’re looking for builders who are energized by owning these outcomes end to end, rather than staying in one layer. We built TensorRT-LLM and other tools that have shaped the field. As a founding engineer, you will help set the next standard. Our team is flat, the problems are consequential, and you will carry work from first principles and prototype to production serving.

About you

  • You chase performance wherever it lives, from kernels to serving systems to the model itself, and you think in terms of the whole system rather than a single layer
  • You would rather own a metric end to end (latency, throughput, cost per token) than a single component
  • You are drawn to the risk and upside of an early-stage startup
  • You embrace ambiguity and have a strong bias for action
  • You operate with a tech-lead mentality regardless of title or years of experience
  • You treat AI as a force multiplier: fluent with coding agents and modern AI/ML workflows, you fold them into how you ship, and you are willing to push the boundary of what’s possible with agents

Requirements

  • Experience optimizing latency- and throughput-critical inference across the full stack, from kernels and systems to the model layer
  • Familiarity with modern GPU systems and deployment architectures, including HGX B300 and GB300 NVL72
  • Familiarity with serving infrastructure, gateways, and inference stacks such as Kubernetes, NVIDIA Dynamo, Kong, SMG, PyTorch, vLLM, SGLang, and TensorRT-LLM
  • Working knowledge of modern LLM and multimodal architectures, including the quantization and speculative-decoding tradeoffs behind them
  • Production proficiency in Python and at least one systems language: Rust, C++, or Go

Bonus

  • Hands-on experience authoring kernels with CUDA, Triton, or PTX
  • Experience profiling, tracing, and debugging across model, runtime, kernel, and network layers using tools such as Nsight Systems (nsys)
  • Experience with batching, KV-cache management, and scheduling strategies
  • Knowledge of quantization formats and algorithms, including NVFP4, FP8, PTQ, QAT, and QAD
  • Familiarity with system-level inference optimization for NVL72-scale deployments, including wide expert parallelism, disaggregated prefill and decode, and related high-bandwidth interconnect techniques
  • A track record of running inference workloads across 1,000+ GPUs
  • A background in HPC or other high-performance systems
  • Contributions to open-source ML or systems infrastructure

Compensation & Benefits

  • Highly competitive salary and equity
  • Comprehensive health coverage for you and your dependents
  • Strong 401(k) match
  • Meals and snacks on us + discretionary productivity budget

Strong preference for in-person in Palo Alto or Brooklyn.

About Inco

Inco AI (inco.ai) builds the agent-native inference platform for tomorrow's token economics. Through hardware-model co-design, it turns any GPU fleet into efficient token production, trains inference-aware models on each customer's real traffic, and deploys portably across heterogeneous compute. We've raised $20M+ and are already building for customers: bare-metal neoclouds and alternative hardware providers looking to differentiate their compute as inference infrastructure, and agentic software companies for whom inference cost is the line between viable and broken margins. We're the team behind industry-defined techniques such as AWQ, DFlash, and TensorRT-LLM, with combined decades in AI efficiency leadership and prior successful exits. Our researchers and engineers work where frontier research meets production.

We operate on five principles:

  • Operate at the frontier. The field moves every day, and so do we. We reinvent ourselves constantly and push the frontier forward with our research. We keep process light and decisions fast, so a good idea reaches production in days, not quarters. Legacy doesn’t slow us down.
  • Mission over individuals. We are obsessed with our customers and humble in pursuit of the goal. We share credit, name mistakes early, and change our minds when the evidence says so. The question is always what moves the mission forward, never who was right.
  • Amplify with agents. Code is free now; architecture and process aren’t. We hand agents everything they can own, from code to operations, and spend our own time on design, judgment, and the problems only people can solve.
  • Protect the maker’s time. Focus is our scarcest resource. We keep meetings few and protect long, uninterrupted stretches of work. We automate overhead, cover meals, and handle logistics, so our hours go to research and product.
  • Be a team player. We win as one team. We share context generously, step in wherever we’re needed, and hold each other to a high bar, because exceptional work comes from exceptional peers.

Inco is already seeing strong early demand. The company is already serving more than 1T tokens a day, has inbound interest from hardware and accelerator companies, and has two enterprise engagements underway, including one focused on building data, fine-tuning, and inference-serving flywheel for an industrial vision use case.

Job details

Published
November 13, 2026
Last checked
November 17, 2026
Experience
Staff
Job source
ashby
Workplace
Hybrid · Palo Alto, US
First seen
November 13, 2026

Benefits

Highly competitive salary and equity, comprehensive health coverage, strong 401(k) match, meals and snacks provided.

Skills mentioned in this job

latencythroughputPythonRustC++GoKubernetesNVIDIA DynamoPyTorchTensorRT-LLM

LiftmyCV exclusive data

Inco AI hiring intelligence

Inco AI currently has 4 active jobs tracked by LiftmyCV. 4 new jobs in the past 7 days.

Verified November 17, 2026

Active openings

4

Currently tracked by LiftmyCV

New in 7 days

4

4 new jobs in the past 30 days

Published salaries

25%

Median disclosed salary: $275,000

Average opening age

5 days

Across 4 active openings

Workplace breakdown

Remote0%
Hybrid100%
On-site0%

Hiring countries

United States of America

Top hiring cities

Brooklyn
Palo Alto

Most in-demand roles

  • • Member of Technical Staff, Developer Infrastructure & Agent Automation
  • • Member of Technical Staff, Inference Research
  • • Member of Technical Staff, Inference Systems
  • • Member of Technical Staff, Kernel Performance & Automation

Frequently requested skills

Python
PyTorch
TensorRT-LLM
Bash
Buildkite
C++
CI/CD
CUDA

Median disclosed salary: $275,000 · 25% of jobs publish a salary

As of November 18, 2026

Hiring insights calculated from active jobs tracked by LiftmyCV.

AI match explained

See how well you match this job

Log in or sign up and Eva will compare your profile with this role, explain your strengths and show what to improve.

Sign up to see your matchAlready have an account? Log in

More at Inco AI

Similar jobs from this company

Across the market

Similar jobs from other companies

Related job searches

Third-party content disclaimer

Full job descriptions are sourced directly from third-party applicant tracking systems (ATS), job boards, recruitment platforms, and employer career pages.

LiftmyCV does not claim ownership of this content or guarantee its accuracy or current availability. All original content is the intellectual property of the respective employers or other rights holders.

LiftmyCV hiring intelligence, availability checks, and related-job recommendations are generated from LiftmyCV data and public sources.

If you prefer, you may apply directly via the original listing for transparency.