LiftmyCVLiftmyCV
OL

Software Engineer, Cloud

OllamaPalo Alto, US
AI job summary

Ollama, a leading platform for open model access, seeks a Software Engineer for its cloud team. You'll work on a scalable inference platform essential for developers, focusing on high-throughput, low-latency distributed systems. Join a small, focused, and agile team that's passionate about design-driven solutions and impactful code deployment.

Job description

Ollama is the most popular way for developers to access open models. What started as an open-source, local-first runtime is now the largest developer network in the open-model ecosystem: 8.9 million monthly active developers and over 67,000+ community-built integrations. We're backed by Y Combinator, Benchmark, 8VC, and Theory Ventures. Our team is small and talent dense. We're flat, low-ego, and fast-moving. We like people who are truth-seeking, passionate, design-driven, and who enjoy shipping code.

About the role

You'll build Ollama’s cloud, a scalable inference platform that lets developers run large, capable open models in their workflow. You'll work on high-throughput, low-latency distributed systems — inference serving, GPU fleet management, routing, metering, and the platform that Pro, Max, Team, and Enterprise customers rely on to process trillions of tokens.

What you'll do

  • Build and scale the inference platform that serves every request from ollama.com.
  • Design the routing and capacity layer that places workloads across GPUs and regions for cost, latency, and availability.
  • Own multi-tenant infrastructure: isolation, quotas, usage metering, billing, and Pro/Max/team/enterprise tiering.
  • Build the reliability, observability, and cost controls for our team and customers

You may be a fit if

  • You have deep experience with high-throughput, low-latency distributed systems — inference serving, traffic routing, real-time data pipelines, or large-scale APIs.
  • You're comfortable with cost/performance tradeoffs at scale and have owned a production service end-to-end.
  • You've worked with Kubernetes, GPU scheduling, or inference infrastructure.
  • You think in terms of reliability, SLOs, and honest capacity planning.
  • Bonus: experience building an inference platform, GPU fleet management, or billing/metering for an AI service.

Job details

Published
July 9, 2026
Last checked
November 17, 2026
Job source
ashby
First seen
July 11, 2026
AI match explained

See how well you match this job

Log in or sign up and Eva will compare your profile with this role, explain your strengths and show what to improve.

Sign up to see your matchAlready have an account? Log in

More at Ollama

Similar jobs from this company

None listed.

Across the market

Similar jobs from other companies

Third-party content disclaimer

Full job descriptions are sourced directly from third-party applicant tracking systems (ATS), job boards, recruitment platforms, and employer career pages.

LiftmyCV does not claim ownership of this content or guarantee its accuracy or current availability. All original content is the intellectual property of the respective employers or other rights holders.

LiftmyCV hiring intelligence, availability checks, and related-job recommendations are generated from LiftmyCV data and public sources.

If you prefer, you may apply directly via the original listing for transparency.