
AI Inference / Systems Software Architect
Netlist, a leader in high-performance memory subsystems, seeks a highly skilled AI Inference / Systems Software Architect. This role requires deep expertise in AI workloads, memory management, and systems software. You will collaborate with top engineers to optimize AI performance and innovate in memory architectures. The ideal candidate will have over 10 years of experience in AI infrastructure and system optimization, knowledgeable in programming languages like C/C++ and Python. Join us to advance the future of AI data centers and memory systems.
Job description
AI Inference / Systems Software Architect Reports To: VP, Engineering
Company Overview:
Netlist, Inc. (OTCQB: NLST) is a leading provider of high-performance modular memory subsystems and next-generation storage-class memory solutions. We specialize in cutting-edge technologies that enable innovation in AI, cloud computing, and enterprise infrastructure. Our product portfolio includes DDR4/DDR5 DIMMs, NAND-based SSDs, and advanced interconnect technologies such as CXL and PCIe. Join our world-class engineering team and be part of shaping the future of memory and storage architecture.
Position Summary:
This is a highly technical role for an individual who deeply understands AI Data Center and Edge AI Inference workloads, model execution, systems software, memory management, and hardware/software co-design. The successful candidate will work closely with the Chief AI Systems Architect, AI Product Strategist, AI Hardware / Memory Systems Architect, and R&D engineering teams to understand how AI workloads execute on current infrastructure, identify software bottlenecks, and architect software that enables new hardware and memory technologies to deliver measurable customer value. We are particularly interested in investigating opportunities involving AI workload, software stack, and inference runtime usage of the hardware to deliver the required performance and efficiency. Key Responsibilities:
AI Inference Architecture
- Develop a deep understanding of modern AI inference workloads and their system-level behavior
- Analyze the execution characteristics of Large Language Models (LLMs), Generative AI, Agentic AI, Long-context inference, Mixture-of-Experts (MoE), Retrieval-Augmented Generation (RAG), Multimodal AI and Edge inference
- Analyze the differences between prefill and decode workloads
- Characterize the impact of batch size, sequence length, concurrency, model size, attention mechanisms, and memory access patterns
- Identify whether workload bottlenecks are caused by compute, memory bandwidth, memory capacity, latency, interconnect, software overhead, or scheduling
AI Runtime and Systems Software
- Architect software infrastructure that allows AI workloads to efficiently utilize heterogeneous memory and compute resources
Areas may include AI inference runtimes, model execution engines, scheduling, resource management, memory management, KV-cache management, data movement, prefetching, caching, model partitioning, load balancing, multi-device execution, distributed inference, accelerator orchestration and telemetry and performance monitoring
- Evaluate and contribute to open-source and commercial AI software stacks such as PyTorch, CUDA, ROCm, Triton, vLLM and other emerging inference runtimes and compiler frameworks
AI Memory Management
- Work closely with the AI Hardware / Memory Systems Architect on next-generation memory architectures to make the entire memory hierarchy appear as an efficient, application-aware resource rather than a collection of independent storage tiers
- Explore how software can intelligently manage HBM, DDR, CXL memory, memory pools, disaggregated memory, NVMe, persistent storage, local versus remote memory, model weights, context state and KV cache
- Develop policies and mechanisms for data placement, tiering, migration, caching, prefetching, eviction, compression, decompression, memory sharing, QoS, isolation and resource allocation
Hardware / Software Co-Design
- Work closely with the hardware architects to ensure that architectural features and hardware capabilities like CXL memory, memory pooling, smart memory controllers, near-memory compute, PIM, inference engines, AI accelerators, compression engines and specialized cache architectures can be effectively exposed and utilized by the software stack
- Collaborate on AI runtime architecture, memory management, KV-cache management, scheduling, data placement, prefetching, compression, workload orchestration, APIs and software interfaces and hardware telemetry and monitoring
- Ensure that hardware innovations translate into measurable application-level benefits rather than isolated component-level benchmarks
- Develop and lead performance optimization methodologies based on real AI workloads. Measure and optimize tokens/sec, time-to-first-token (TTFT), time-per-output-token (TPOT), latency, throughput, memory bandwidth utilization, memory capacity utilization, accelerator utilization, power efficiency, cost per inference, cost per million tokens and system level TCO
Inference Performance Optimization
- Identify opportunities to improve performance through kernel optimization, quantization, sparsity, compression, operator fusion, memory placement, scheduling, batching, speculative decoding, KV-cache optimization, prefill/decode optimization and communication reduction
Software Architecture for CXL and Disaggregated Memory
- Help define the software architecture necessary to make CXL-based memory expansion, pooling, and disaggregation practical for AI workloads
- Investigate host memory versus CXL memory placement, shared versus dedicated memory, memory allocation policies, NUMA-aware scheduling, remote-memory latency, bandwidth allocation, memory QoS, memory migration, application-level
- Develop software mechanisms that allow applications to benefit from additional capacity without unacceptable latency or performance penalties
- Develop software architecture for AI context memory systems. Potential capabilities include KV-cache tiering, KV-cache compression, KV-cache quantization, prefix sharing, context reuse, cache eviction, context migration, session-aware caching, distributed context management, GPU-to-remote-memory transfers and context prefetching
versus system-level control and multi-host memory orchestration AI Context / KV-Cache Software
Architecture Modeling and Prototyping
- Develop software prototypes to validate new hardware architectures
- Build workload models and emulation environments before silicon availability
- Create benchmark suites representative of customer workloads
- Evaluate architecture concepts using existing CPUs, GPUs, FPGAs, or emulation platforms
- Quantify software overhead and expected system-level benefits
- Establish performance baselines against existing GPU, CPU, CXL, and inference solutions
Required Qualifications:
- Bachelor's or Master's degree in Computer Science, Computer Engineering, Electrical Engineering, or a related technical field; PhD is a plus
- 10+ years of experience in systems software, AI infrastructure, computer architecture, inference systems, compilers, runtimes, or related areas
- Deep understanding of modern AI/ML inference workloads
- Strong systems programming skills, preferably in C/C++, Python, or comparable languages
- Strong understanding of computer architecture, memory systems, operating systems, and data movement
- Experience architecting or optimizing large-scale AI inference systems
- Demonstrated experience translating application/workload requirements into systems architecture
- Strong understanding of performance modeling and benchmarking
- Excellent communication skills and ability to work across hardware and software organizations
Preferred Qualifications:
Experience with several of the following:
- LLM inference
- Transformer architecture
- KV cache
- vLLM
- PyTorch
- CUDA
- Triton
- TensorRT
- ROCm
- XLA
- MLIR / LLVM
- FlashAttention
- MoE
- Quantization
- Speculative decoding
- Distributed inference
- GPU systems
- AI accelerators
- HBM
- DDR
- CXL
- NUMA
- Memory pooling / disaggregation
- PIM / near-memory computing
- AI networking
- AI storage
- Edge AI
- Hardware/software co-design
Salary Range: $275,000
Job details
- Published
- November 13, 2026
- Last checked
- November 17, 2026
- Job source
- breezy
- Workplace
- On-site · Irvine, US
- First seen
- November 13, 2026
Requirements
Bachelor's or Master's degree in Computer Science or related fields; 10+ years of experience in systems software; deep understanding of AI/ML inference workloads.
Skills mentioned in this job
LiftmyCV exclusive data
Netlist hiring intelligence
Netlist currently has 13 active jobs tracked by LiftmyCV. 10 new jobs in the past 7 days.
Verified November 17, 2026
Active openings
13
Currently tracked by LiftmyCV
New in 7 days
10
13 new jobs in the past 30 days
Published salaries
85%
Median disclosed salary: $210,000
Average opening age
6 days
Across 13 active openings
Workplace breakdown
Hiring countries
Top hiring cities
Most in-demand roles
- • AI Hardware / Memory Systems Architect
- • AI Inference / Systems Software Architect
- • AI Product Strategist Leader
- • Artificial Intelligence Architect (AI)
- • Director of Financial Planning
- • Director, FP&A
Frequently requested skills
Median disclosed salary: $210,000 · 85% of jobs publish a salary
As of November 18, 2026
Hiring insights calculated from active jobs tracked by LiftmyCV.
See how well you match this job
Log in or sign up and Eva will compare your profile with this role, explain your strengths and show what to improve.
Sign up to see your matchAlready have an account? Log inMore at Netlist
Similar jobs from this company
Across the market
Similar jobs from other companies
- Software Engineer, Inference SystemsRiver AI Inc.Palo Alto, USOn-site
- Software System ArchitectGeneral Motors LLCMilford, Michigan, United States of AmericaOn-site
- Software Systems ArchitectThe Aerospace CorporationEl Segundo, CAOn-site
- System Software ArchitectNVIDIA Graphics BengaluruBengaluru, INOn-site
- Software Architect - SystemsUSA Wind RiverUnited States Remote WorksiteRemote
- Software and System ArchitectNVIDIA USASanta Clara, USOn-site
Related job searches
Third-party content disclaimer
Full job descriptions are sourced directly from third-party applicant tracking systems (ATS), job boards, recruitment platforms, and employer career pages.
LiftmyCV does not claim ownership of this content or guarantee its accuracy or current availability. All original content is the intellectual property of the respective employers or other rights holders.
LiftmyCV hiring intelligence, availability checks, and related-job recommendations are generated from LiftmyCV data and public sources.
If you prefer, you may apply directly via the original listing for transparency.