LiftmyCVLiftmyCV

AI Inference / Systems Software Architect

NetlistOn-siteIrvine, US$275,000 per year
AI job summary

Netlist, a leader in high-performance memory subsystems, seeks a highly skilled AI Inference / Systems Software Architect. This role requires deep expertise in AI workloads, memory management, and systems software. You will collaborate with top engineers to optimize AI performance and innovate in memory architectures. The ideal candidate will have over 10 years of experience in AI infrastructure and system optimization, knowledgeable in programming languages like C/C++ and Python. Join us to advance the future of AI data centers and memory systems.

Job description

AI Inference / Systems Software Architect Reports To: VP, Engineering

Company Overview:

Netlist, Inc. (OTCQB: NLST) is a leading provider of high-performance modular memory subsystems and next-generation storage-class memory solutions. We specialize in cutting-edge technologies that enable innovation in AI, cloud computing, and enterprise infrastructure. Our product portfolio includes DDR4/DDR5 DIMMs, NAND-based SSDs, and advanced interconnect technologies such as CXL and PCIe. Join our world-class engineering team and be part of shaping the future of memory and storage architecture.

Position Summary:

This is a highly technical role for an individual who deeply understands AI Data Center and Edge AI Inference workloads, model execution, systems software, memory management, and hardware/software co-design. The successful candidate will work closely with the Chief AI Systems Architect, AI Product Strategist, AI Hardware / Memory Systems Architect, and R&D engineering teams to understand how AI workloads execute on current infrastructure, identify software bottlenecks, and architect software that enables new hardware and memory technologies to deliver measurable customer value. We are particularly interested in investigating opportunities involving AI workload, software stack, and inference runtime usage of the hardware to deliver the required performance and efficiency. Key Responsibilities:

AI Inference Architecture

  • Develop a deep understanding of modern AI inference workloads and their system-level behavior
  • Analyze the execution characteristics of Large Language Models (LLMs), Generative AI, Agentic AI, Long-context inference, Mixture-of-Experts (MoE), Retrieval-Augmented Generation (RAG), Multimodal AI and Edge inference
  • Analyze the differences between prefill and decode workloads
  • Characterize the impact of batch size, sequence length, concurrency, model size, attention mechanisms, and memory access patterns
  • Identify whether workload bottlenecks are caused by compute, memory bandwidth, memory capacity, latency, interconnect, software overhead, or scheduling

AI Runtime and Systems Software

  • Architect software infrastructure that allows AI workloads to efficiently utilize heterogeneous memory and compute resources

Areas may include AI inference runtimes, model execution engines, scheduling, resource management, memory management, KV-cache management, data movement, prefetching, caching, model partitioning, load balancing, multi-device execution, distributed inference, accelerator orchestration and telemetry and performance monitoring

  • Evaluate and contribute to open-source and commercial AI software stacks such as PyTorch, CUDA, ROCm, Triton, vLLM and other emerging inference runtimes and compiler frameworks

AI Memory Management

  • Work closely with the AI Hardware / Memory Systems Architect on next-generation memory architectures to make the entire memory hierarchy appear as an efficient, application-aware resource rather than a collection of independent storage tiers
  • Explore how software can intelligently manage HBM, DDR, CXL memory, memory pools, disaggregated memory, NVMe, persistent storage, local versus remote memory, model weights, context state and KV cache
  • Develop policies and mechanisms for data placement, tiering, migration, caching, prefetching, eviction, compression, decompression, memory sharing, QoS, isolation and resource allocation

Hardware / Software Co-Design

  • Work closely with the hardware architects to ensure that architectural features and hardware capabilities like CXL memory, memory pooling, smart memory controllers, near-memory compute, PIM, inference engines, AI accelerators, compression engines and specialized cache architectures can be effectively exposed and utilized by the software stack
  • Collaborate on AI runtime architecture, memory management, KV-cache management, scheduling, data placement, prefetching, compression, workload orchestration, APIs and software interfaces and hardware telemetry and monitoring
  • Ensure that hardware innovations translate into measurable application-level benefits rather than isolated component-level benchmarks
  • Develop and lead performance optimization methodologies based on real AI workloads. Measure and optimize tokens/sec, time-to-first-token (TTFT), time-per-output-token (TPOT), latency, throughput, memory bandwidth utilization, memory capacity utilization, accelerator utilization, power efficiency, cost per inference, cost per million tokens and system level TCO

Inference Performance Optimization

  • Identify opportunities to improve performance through kernel optimization, quantization, sparsity, compression, operator fusion, memory placement, scheduling, batching, speculative decoding, KV-cache optimization, prefill/decode optimization and communication reduction

Software Architecture for CXL and Disaggregated Memory

  • Help define the software architecture necessary to make CXL-based memory expansion, pooling, and disaggregation practical for AI workloads
  • Investigate host memory versus CXL memory placement, shared versus dedicated memory, memory allocation policies, NUMA-aware scheduling, remote-memory latency, bandwidth allocation, memory QoS, memory migration, application-level
  • Develop software mechanisms that allow applications to benefit from additional capacity without unacceptable latency or performance penalties
  • Develop software architecture for AI context memory systems. Potential capabilities include KV-cache tiering, KV-cache compression, KV-cache quantization, prefix sharing, context reuse, cache eviction, context migration, session-aware caching, distributed context management, GPU-to-remote-memory transfers and context prefetching

versus system-level control and multi-host memory orchestration AI Context / KV-Cache Software

Architecture Modeling and Prototyping

  • Develop software prototypes to validate new hardware architectures
  • Build workload models and emulation environments before silicon availability
  • Create benchmark suites representative of customer workloads
  • Evaluate architecture concepts using existing CPUs, GPUs, FPGAs, or emulation platforms
  • Quantify software overhead and expected system-level benefits
  • Establish performance baselines against existing GPU, CPU, CXL, and inference solutions

Required Qualifications:

  • Bachelor's or Master's degree in Computer Science, Computer Engineering, Electrical Engineering, or a related technical field; PhD is a plus
  • 10+ years of experience in systems software, AI infrastructure, computer architecture, inference systems, compilers, runtimes, or related areas
  • Deep understanding of modern AI/ML inference workloads
  • Strong systems programming skills, preferably in C/C++, Python, or comparable languages
  • Strong understanding of computer architecture, memory systems, operating systems, and data movement
  • Experience architecting or optimizing large-scale AI inference systems
  • Demonstrated experience translating application/workload requirements into systems architecture
  • Strong understanding of performance modeling and benchmarking
  • Excellent communication skills and ability to work across hardware and software organizations

Preferred Qualifications:

Experience with several of the following:

  • LLM inference
  • Transformer architecture
  • KV cache
  • vLLM
  • PyTorch
  • CUDA
  • Triton
  • TensorRT
  • ROCm
  • XLA
  • MLIR / LLVM
  • FlashAttention
  • MoE
  • Quantization
  • Speculative decoding
  • Distributed inference
  • GPU systems
  • AI accelerators
  • HBM
  • DDR
  • CXL
  • NUMA
  • Memory pooling / disaggregation
  • PIM / near-memory computing
  • AI networking
  • AI storage
  • Edge AI
  • Hardware/software co-design

Salary Range: $275,000

Job details

Published
November 13, 2026
Last checked
November 17, 2026
Job source
breezy
Workplace
On-site · Irvine, US
First seen
November 13, 2026

Requirements

Bachelor's or Master's degree in Computer Science or related fields; 10+ years of experience in systems software; deep understanding of AI/ML inference workloads.

Skills mentioned in this job

AI workloadsmemory managementC/C++Python

LiftmyCV exclusive data

Netlist hiring intelligence

Netlist currently has 13 active jobs tracked by LiftmyCV. 10 new jobs in the past 7 days.

Verified November 17, 2026

Active openings

13

Currently tracked by LiftmyCV

New in 7 days

10

13 new jobs in the past 30 days

Published salaries

85%

Median disclosed salary: $210,000

Average opening age

6 days

Across 13 active openings

Workplace breakdown

Remote0%
Hybrid0%
On-site100%

Hiring countries

United States of America

Top hiring cities

Irvine

Most in-demand roles

  • • AI Hardware / Memory Systems Architect
  • • AI Inference / Systems Software Architect
  • • AI Product Strategist Leader
  • • Artificial Intelligence Architect (AI)
  • • Director of Financial Planning
  • • Director, FP&A

Frequently requested skills

Accounting
CXL
DDR
Excel
GAAP
HBM
Python
Account Management

Median disclosed salary: $210,000 · 85% of jobs publish a salary

As of November 18, 2026

Hiring insights calculated from active jobs tracked by LiftmyCV.

AI match explained

See how well you match this job

Log in or sign up and Eva will compare your profile with this role, explain your strengths and show what to improve.

Sign up to see your matchAlready have an account? Log in

More at Netlist

Similar jobs from this company

Across the market

Similar jobs from other companies

Related job searches

Third-party content disclaimer

Full job descriptions are sourced directly from third-party applicant tracking systems (ATS), job boards, recruitment platforms, and employer career pages.

LiftmyCV does not claim ownership of this content or guarantee its accuracy or current availability. All original content is the intellectual property of the respective employers or other rights holders.

LiftmyCV hiring intelligence, availability checks, and related-job recommendations are generated from LiftmyCV data and public sources.

If you prefer, you may apply directly via the original listing for transparency.