LiftmyCVLiftmyCV
MA

Runtime Engineer

MatXRemoteMountain View, US$120,000–$475,000 per year
AI job summary

MatX is seeking a Runtime Engineer to build custom silicon for large-language-model inference and training, emphasizing HW/SW co-design. The role involves developing host-side interfaces, managing executable formats, designing custom-kernel ABIs, and enhancing LLM inference serving stacks. Ideal candidates have strong systems programming skills and experience with GPU computing and Python integration. The compensation is competitive, ranging from $120,000 to $475,000 depending on career level, plus equity and various employee benefits.

Job description

What MatX is Building

MatX is building custom silicon for large-language-model inference and training, with HW/SW co-design across ISA, RTL, simulator, compiler, and kernels so each layer benefits from the others. The runtime owns the host-side stack and the contracts that bind those teams together.

What You'll Do Here

  • Build the host-side interface library — device memory management, DMA, streams and events, sync primitives — that every compiler-emitted program runs on top of
  • Own and extend the executable format: the compiler→runtime contract, its versioning, the weight and quantization layouts that let compiler and runtime evolve independently
  • Design the custom-kernel ABI — calling convention, sync semantics, lifecycle — and the host-side marshaling layer (DLPack, the buffer protocol, numpy) that gets Python tensors to the device
  • Build Python bindings via PyO3, with a C-ABI shim as the alternative integration path for downstream consumers
  • Build the LLM inference serving stack — paged KV cache, continuous batching, request scheduling, token streaming — and the cluster orchestration primitives underneath it
  • Bring up interconnect topology from the host and own the failure-detection and clean-teardown path for stop-restructure-resume recovery across racks
  • Design what the chip exposes to host-side profilers and debuggers — perf counters, traces, and the Python surfaces ML engineers actually use — and hit measurable performance targets on runtime overhead and serving throughput

Who You Are

  • Strong experience in a systems programming language — Rust, C, C++, or Go — including memory management, allocator design, and FFI/ABI work
  • Have built Python interop layers in production (PyO3, ctypes, pybind11, or equivalent C-ABI bridging)
  • Have designed and maintained API or ABI contracts between teams — versioning, evolution, breaking-change discipline — not just consumed someone else's
  • Hands-on with at least one accelerator programming model (CUDA, ROCm, oneAPI Level Zero, TPU, or comparable) — enough to reason about device memory, async execution, and kernel launch
  • ML-systems literate — comfortable with the training and inference loop, what collectives do, what a tensor layout is. Research depth not required.

Bonus Points If You Have

  • LLM inference internals — vLLM, TensorRT-LLM, or SGLang (paged attention, scheduler design)
  • Rust at depth, including proc macros, unsafe with soundness reasoning, and complex lifetime/trait work
  • Custom allocator design (slab, paged, arena) or other low-level memory work
  • ML framework integration experience (PyTorch custom backends, JAX/XLA, ONNX runtime)
  • Profiler or tracing infrastructure work (perfetto, Nsight, or a custom stack)
  • Driver-adjacent or kernel-bypass work, or prior new-silicon bring-up

Compensation

The US base salary for this full-time position is determined based on a variety of factors including role, experience, location, job related skills, and relevant education and training. Career length is only a guideline for compensation.

  • Early Career - $120,000 - $250,000 + equity
  • Mid Career - $175,000 - $362,500 + equity
  • Senior Career - $250,000 - $475,000 + equity

What We Offer

  • A Stake in our success - Generous equity, with option cash/equity swap at offer, and option to employee early exercise.
  • Health & Wellness - Company subsidized Health, Dental, Vision, and Life insurance; Pre-tax Health Savings Accounts with generous company contribution (even if you don’t)
  • Time To Recharge - 4 weeks paid time off (accrued), 12 company holidays, and 3 weeks remote/flexible work per year
  • Support to Parents - Up to 12 weeks of paid parental leave, regardless of your path to parenthood
  • Learning & Development - $1,500 yearly towards your professional development e.g. conferences, courses, and other learning opportunities
  • Team Connection - Team Lunches, quarterly off-sites, and regular town halls
  • Financial Wellbeing - 401K and/or Roth IRA, with 5% company contribution, even if you don’t!
  • Flexible Spending Accounts - Pre-tax spend accounts for medical, dental/vision, dependent care, parking, and transit expenses
  • Commute On Us - For those commuting up to 1 hour, put your rideshare cost on our company card and reclaim the drive-time to get work done!
  • MatX E[x]tras - $50 per month to use on the perks you care about most
  • Remote Perks - We work remotely Monday & Friday, supported by home-tech setup, and remote wifi expense reimbursement

As part of our dedication to the diversity of our team and our focus on creating an inviting and inclusive work experience, MatX is committed to a policy of Equal Employment Opportunity and will not discriminate against an applicant or employee on the basis of race, color, religion, creed, national origin or ancestry, sex, gender, gender identity, gender expression, sexual orientation, age, physical or mental disability, medical condition, marital/domestic partner status, military and veteran status, genetic information or any other legally recognized protected basis under federal, state or local laws, regulations or ordinances. This position requires access to information that is subject to U.S. export controls. This offer of employment is contingent upon the applicants capacity to perform job functions in compliance with U.S. export control laws without obtaining a license from U.S. export control authorities. MatX does not accept unsolicited resumes from individual recruiters or third-party recruiting agencies in response to job postings. No fee will be paid to third parties who submit unsolicited candidates directly to our hiring managers or People team and any resumes submitted are deemed to be the property of MatX.

Job details

Published
May 27, 2026
Last checked
August 16, 2026
Job source
greenhouse
Workplace
Remote · Mountain View, US
First seen
May 27, 2026
AI match explained

See how well you match this job

Log in or sign up and Eva will compare your profile with this role, explain your strengths and show what to improve.

Sign up to see your matchAlready have an account? Log in

More at MatX

Similar jobs from this company

None listed.

Across the market

Similar jobs from other companies

Third-party content disclaimer

Full job descriptions are sourced directly from third-party applicant tracking systems (ATS), job boards, recruitment platforms, and employer career pages.

LiftmyCV does not claim ownership of this content or guarantee its accuracy or current availability. All original content is the intellectual property of the respective employers or other rights holders.

LiftmyCV hiring intelligence, availability checks, and related-job recommendations are generated from LiftmyCV data and public sources.

If you prefer, you may apply directly via the original listing for transparency.