Software Engineer, Runtime
Ollama seeks a Software Engineer to enhance its local runtime for open models, which supports a wide array of systems and hardware. You will join a talented team focused on efficiency and collaboration, working with Go and C/C++. Responsibilities include optimizing model performance, integrating architectures, and collaborating with hardware vendors. Ideal candidates possess strong systems knowledge, familiarity with GPU computing, and a commitment to delivering top-tier software.
Job description
Ollama is the most popular way for developers to access open models. What started as an open-source, local-first runtime is now the largest developer network in the open-model ecosystem: 8.9 million monthly active developers and over 67,000+ community-built integrations. We're backed by Y Combinator, Benchmark, 8VC, and Theory Ventures. Our team is small and talent dense. We're flat, low-ego, and fast-moving. We like people who are truth-seeking, passionate, design-driven, and who enjoy shipping code.
About the role
You'll work on the heart of Ollama — the local runtime that runs open models on developers' own machines. It loads models, manages memory, drives GPU acceleration across NVIDIA, AMD, Intel, Qualcomm, and Apple Silicon (including our MLX integration), and makes all of it feel instant. You'll work in Go and C/C++ and touch the model formats and inference engines underneath, shipping to macOS, Linux, and Windows across an enormous range of hardware.
What you'll do
- Make open models run fast and reliably on consumer and enterprise hardware — from a MacBook Pro to server-grade NVIDIA GPUs.
- Own pieces of the runtime: model loading & scheduling memory management, quantization, GPU hardware backends.
- Integrate new model architectures and quantization formats so the latest open models work on day one.
- Improve cold-start, time-to-first-token, and throughput.
- Partner with model labs and hardware vendors on early access and deep integrations.
- Ship in the open: Ollama is open source, and you'll work with the community.
Example projects
- Add support for a new model family end-to-end — format parsing, weights loading, and the defaults that make it useful out of the box.
- Cut cold-start for a popular model in half by streaming weights and lazy-loading layers.
- Land a new quantization format so a 70B model runs on a single consumer GPU.
- Wire up a new GPU backend and find a 2x throughput win with kernel selection and memory tuning.
- Improve the "Auto" experience — picking the right model and settings for a machine's hardware without the user thinking about it.
You may be a fit if
- You have strong systems fundamentals and are comfortable in Go, C, or C++.
- You've worked close to the metal — GPU compute, inference, game engines, databases, OS, or networking.
- You care about performance and have profiled and optimized real workloads.
- You're comfortable shipping to millions of users and handling the long tail of hardware and OS combinations.
- Bonus: experience with model quantization, GPU programming (CUDA/Metal/SYCL), Apple MLX.
Job details
- Published
- July 9, 2026
- Last checked
- November 17, 2026
- Job source
- ashby
- First seen
- July 11, 2026
See how well you match this job
Log in or sign up and Eva will compare your profile with this role, explain your strengths and show what to improve.
Sign up to see your matchAlready have an account? Log inMore at Ollama
Similar jobs from this company
None listed.
Across the market
Similar jobs from other companies
Third-party content disclaimer
Full job descriptions are sourced directly from third-party applicant tracking systems (ATS), job boards, recruitment platforms, and employer career pages.
LiftmyCV does not claim ownership of this content or guarantee its accuracy or current availability. All original content is the intellectual property of the respective employers or other rights holders.
LiftmyCV hiring intelligence, availability checks, and related-job recommendations are generated from LiftmyCV data and public sources.
If you prefer, you may apply directly via the original listing for transparency.