
Staff Site Reliability Engineer
Job description
About the Role
We're looking for a Staff Site Reliability Engineer who is equally at home writing production software and running the infrastructure it lives on — and who wants to take ownership of one of the hardest, highest-leverage problems on our platform: intelligently managing a large fleet of GPU-backed models. We run nearly 30 models across heterogeneous hardware, and keeping that fleet fast, reliable, and cost-effective is a serious engineering challenge. You'll build the GPU management and scheduling platform that sits at the center of it — collecting utilization and load metrics, interpreting what they actually mean, and using them to make real-time decisions about admission control and scaling. The goal: route and schedule inference calls so we use our capacity efficiently without exceeding it, and scale model replicas up and down automatically as demand shifts. This is a senior role for someone with a decade in the field who can move fluidly between systems engineering and software development, and who is excited to own a complex, evolving system end to end.
What You'll Do
- Design and build our GPU management and scheduling platform — the system that decides when, where, and how inference calls run across a fleet of ~30 models on heterogeneous hardware
- Build the metrics pipeline that collects GPU load and utilization data, and the logic that turns those signals into decisions
- Implement admission control to protect capacity — deciding when to accept, queue, or shed inference requests so we operate within fleet limits
- Build autoscaling that adjusts the number of model replicas in response to real-time demand and utilization
- Develop cloud orchestration systems and operators in Python and Go to manage the model fleet
- Architect and operate scalable, fault-tolerant, secure production systems on AWS, GCP, or Azure
- Design and build infrastructure automation and deployment pipelines (Terraform, CI/CD) as first-class software
- Stand up and maintain monitoring, logging, and alerting that keep the platform reliable and performant
- Develop and enforce security and compliance policies appropriate to a healthcare AI platform
- Partner with engineers and research scientists to diagnose and resolve complex infrastructure, deployment, and operational issues
- Mentor engineers and raise the technical bar across the team
We value in-person teamwork and believe the best ideas happen together. Our team is expected to be in the office five days a week in Gurugram. This role will also require engaging with a US based team requiring some flexibility for overlapping hours with California.
What You Bring
Must-Have
- 10+ years of professional experience across site reliability / DevOps engineering and software engineering
- Computer Science Degree Required from a top CS program.
- Strong software engineering fundamentals — you build orchestration and scheduling systems in Python and/or Go, not just configure off-the-shelf tools
- Experience designing systems that make decisions from operational metrics — collecting signals, interpreting them, and driving control loops such as autoscaling, load shedding, or admission control
- Deep experience with infrastructure automation and CI/CD (Terraform, GitLab CI/CD, or similar)
- Hands-on production experience with at least one major cloud platform (AWS, GCP, or Azure)
- Strong knowledge of containerization and orchestration (Docker, Kubernetes)
- Experience with monitoring and logging stacks (ELK, Grafana, Datadog, or similar)
- Familiarity with secrets management and security tooling (HashiCorp Vault, AWS KMS, Azure Key Vault)
- Excellent problem-solving skills and the ability to work both independently and collaboratively
- Strong communication and interpersonal skills
Nice-to-Have
- Experience managing GPU fleets or scheduling workloads across heterogeneous accelerators
- Familiarity with ML inference serving and model deployment (e.g. Triton, KServe, Ray Serve, or similar)
- Experience with Kubernetes autoscaling internals (HPA/VPA, custom metrics, custom controllers)
- Experience implementing HIPAA and SOC 2 compliance
- Experience operating in an HPC environment
- Bachelor's or Master's in Computer Science, Computer Engineering, or a related field
Join our team at Hippocratic AI and help shape the future of clinically safe, production-grade AI systems.
Why Join Hippocratic AI
Reinvent healthcare with AI that puts safety first. We’re building the world’s first healthcare‑only, safety‑focused LLM — a breakthrough platform designed to transform patient outcomes at a global scale. This is category creation. Work with the people shaping the future. Hippocratic AI was co‑founded by CEO Munjal Shah and a team of physicians, hospital leaders, AI pioneers, and researchers from institutions like El Camino Health, Johns Hopkins, Washington University in St. Louis, Stanford, Google, Meta, Microsoft, and NVIDIA. Backed by the world’s leading healthcare and AI investors. We recently raised a $126M Series C at a $3.5B valuation, led by Avenir Growth, bringing total funding to $404M with participation from CapitalG, General Catalyst, a16z, Kleiner Perkins, Premji Invest, UHS, Cincinnati Children’s, WellSpan Health, John Doerr, Rick Klausner, and others. Build alongside the best in healthcare and AI. Join experts who’ve spent their careers improving care, advancing science, and building world‑changing technologies — ensuring our platform is powerful, trusted, and truly transformative.
Equal Opportunity
Hippocratic AI is an equal opportunity employer. We do not discriminate on the basis of race, color, religion, national origin, sex, age, disability, sexual orientation, gender identity or expression, genetic information, military or veteran status, or any other characteristic protected by applicable law. We are committed to building a team that reflects the patients we serve. We actively encourage applications from candidates of all backgrounds. If you require accommodations during the hiring process, please contact. Please be aware of recruitment scams impersonating Hippocratic AI. All recruiting communication will come from @hippocraticai.com email addresses. We will never request payment or sensitive personal information during the hiring process.
Job details
- Published
- November 4, 2026
- Last checked
- November 17, 2026
- Experience
- Staff
- Job source
- ashby
- First seen
- November 4, 2026
LiftmyCV exclusive data
Hippocratic AI hiring intelligence
Hippocratic AI currently has 15 active jobs tracked by LiftmyCV. 2 new jobs in the past 7 days.
Verified November 17, 2026
Active openings
15
Currently tracked by LiftmyCV
New in 7 days
2
11 new jobs in the past 30 days
Published salaries
—
Not published by this employer yet
Average opening age
21 days
Across 15 active openings
Workplace breakdown
Hiring countries
Top hiring cities
Most in-demand roles
- • Agent Deployment Architect
- • Audio Transcription Specialist
- • Chief Commercial Officer, Biotech
- • Deployment Strategist (Payors)
- • Director of IT and Workplace
- • Director, Quality Assurance SaMD
Frequently requested skills
As of November 18, 2026
Hiring insights calculated from active jobs tracked by LiftmyCV.
See how well you match this job
Log in or sign up and Eva will compare your profile with this role, explain your strengths and show what to improve.
Sign up to see your matchAlready have an account? Log inMore at Hippocratic AI
Similar jobs from this company
- Technical Program Manager - Frontier AI Data LabelingHippocratic AIMenlo Park, US
- Global Risk ComplianceHippocratic AIMenlo Park, US
- Senior Applied Scientist, Reinforcement LearningHippocratic AIMenlo Park, US
- Senior Manager, Strategic FinanceHippocratic AIMenlo Park, US
- Senior Recruiter (Contractor)Hippocratic AIUSRemote
- Director, Quality Assurance SaMDHippocratic AIMenlo Park, US
Across the market
Similar jobs from other companies
- Staff Site Reliability EngineerStryker Global Technology Center Pvt Ltd.Gurugram, INOn-site
- Staff Site Reliability EngineerReplitGBRemote
- Staff Site Reliability EngineerLegora ABNew York City, USOn-site
- Staff Site Reliability EngineerCircle Internet Management Services LLCSan Francisco, USRemote$195,000–$257,500 per year
- Staff Site Reliability EngineerTapingo LtdTel Aviv, ILOn-site
- Staff Site Reliability EngineerGrindrChicagoHybrid
Third-party content disclaimer
Full job descriptions are sourced directly from third-party applicant tracking systems (ATS), job boards, recruitment platforms, and employer career pages.
LiftmyCV does not claim ownership of this content or guarantee its accuracy or current availability. All original content is the intellectual property of the respective employers or other rights holders.
LiftmyCV hiring intelligence, availability checks, and related-job recommendations are generated from LiftmyCV data and public sources.
If you prefer, you may apply directly via the original listing for transparency.