
Member of Technical Staff - Pre-Training Evals Infra
Job description
Our Mission
Reflection is a research lab making intelligence open and accessible for everyone to use, customize, and build on. We build open models that let anyone control their intelligence and help shape the future of AI. Our mission: make intelligence open and accessible to all.
About the Role
You will build the evaluation infrastructure that researchers use to understand model capabilities, guide training decisions, and assess readiness for release. The role spans evaluation authoring, distributed execution, and results analysis, with a particular focus on pre-training. You will work directly with researchers to make evaluations faster, more reliable, and easier to use, ensuring that the results accurately reflect the models being evaluated.
- Design and evolve evaluation abstractions. Build clear APIs and data models for evaluation tasks, datasets, configurations, model outputs, and scores, making it straightforward to add new evaluations and extend existing workflows.
- Build scalable, reliable execution infrastructure. Develop distributed systems for evaluating model checkpoints throughout training and ahead of release, with robust orchestration, observability, and recovery from failures.
- Optimize evaluation and inference performance. Profile inference servers and distributed execution to improve throughput, GPU utilization, and time to results.
- Establish evaluation correctness and reproducibility. Build validation and regression tests, version datasets and configurations, and preserve the provenance needed to reproduce results and investigate discrepancies.
- Build researcher-facing tools. Create intuitive interfaces and dashboards for inspecting model outputs, comparing checkpoints and experiments, and debugging unexpected behavior.
- Operate and monitor evaluations as training progresses. Track evaluation workloads and results, triage failures, and develop alerts and diagnostics that help distinguish changes in model quality from inference or evaluation issues.
- Turn research needs into reusable systems. Partner directly with researchers to identify bottlenecks, prioritize high-impact improvements, and turn bespoke workflows into self-service tools that work across teams.
About You
- Strong proficiency in Python, C++, or Rust, with the ability to reason deeply about concurrency, memory behavior, and performance.
- Experience designing, building, and operating reliable, maintainable distributed systems, with sound judgment about interfaces, failure modes, and architectural tradeoffs.
- Working knowledge of model inference and evaluation, including how implementation and configuration choices can affect measured model quality.
- Ability to investigate complex problems across system boundaries, follow evidence, and identify root causes rather than just address symptoms.
- Strong product judgment and experience translating the needs of researchers or other technical users into clear APIs and usable tools.
- Comfort owning ambiguous problems end to end, prioritizing competing research needs, and balancing rapid delivery with long-term reliability and maintainability.
- Attention to detail and the persistence to investigate subtle discrepancies in model outputs, evaluation results, and system behavior.
What We Offer:
We believe that to make intelligence open and accessible to all, you need to start at the foundation. Joining Reflection means building from the ground up as part of a talent-dense team. You will help define our future as a company, and help define the future of open foundational models. We want you to do the most impactful work of your career with the confidence that you and the people you care about most are supported.
- Top-tier compensation: Salary and equity structured to recognize and retain our talent globally.
- Stock options: Everyone who joins and contributes to Reflection's success gets to share in the upside through stock options.
- Health & wellness: Comprehensive medical, dental, vision, and life, with an annual wellness allowance.
- Meals: Lunch and dinner are provided in the office daily.
- Life & family: 22 weeks paid parental leave for all new birthing and non-birthing parents, including adoptive and surrogate journeys.
- Vacation days: Unlimited paid time off in the U.S. and 30 days in the U.K.
- Sponsorship support: We sponsor visas to help exceptional talent join our team and support long-term immigration pathways where applicable.
- Team building: We have regular off-sites, happy hours, and team celebrations.
Export Control Notice: This position may require access to technology or source code subject to the U.S. Export Administration Regulations. Any offer of employment for this role may be conditioned on the Company's ability to provide the candidate with access to such technology or source code in compliance with applicable U.S. export control laws, which may require the Company to seek government authorization. Candidate and Employee Privacy Notice: We collect and process personal data about applicants for the purposes described in our Candidate and Employee Privacy Notice
Job details
- Published
- October 13, 2026
- Last checked
- November 18, 2026
- Experience
- Staff
- Job source
- ashby
- First seen
- October 13, 2026
LiftmyCV exclusive data
Reflection hiring intelligence
Reflection currently has 12 active jobs tracked by LiftmyCV. 1 new jobs in the past 7 days.
Verified November 18, 2026
Active openings
12
Currently tracked by LiftmyCV
New in 7 days
1
10 new jobs in the past 30 days
Published salaries
—
Not published by this employer yet
Average opening age
18 days
Across 12 active openings
Workplace breakdown
Hiring countries
Top hiring cities
Most in-demand roles
- • AGC, Head of Litigation
- • Brand Designer
- • Contracts Manager
- • Human Data Strategist
- • Member of Deployment Staff - Public Sector
- • Member of Technical Staff - Pre-Training Data
Frequently requested skills
As of November 17, 2026
Hiring insights calculated from active jobs tracked by LiftmyCV.
See how well you match this job
Log in or sign up and Eva will compare your profile with this role, explain your strengths and show what to improve.
Sign up to see your matchAlready have an account? Log inMore at Reflection
Similar jobs from this company
- Research Program Manager, Research Infra and Training FrameworksReflectionSan Francisco, US
- Research Program Manager, Pre-trainingReflectionSan Francisco, US
- Member of Deployment Staff - Public SectorReflectionNew York, US
- Member of Technical Staff - Pre-Training DataReflectionSan Francisco, US
- Member of Technical Staff - Pre-Training Evals Infra (Inference)ReflectionGlobal Office
Across the market
Similar jobs from other companies
- Member of Technical Staff - EvalsP-1 AIUSRemote$170,000–$200,000 per year
- Member of Technical Staff, EvalsProfoundNew York, US$215,000–$300,000 per year
- Member of Technical Staff, EvalsMagicSan Francisco, USRemote
- Member of Technical Staff, EvalsHandshakeSan Francisco, USOn-site
- Member of Technical Staff, Training Infra EngineercohereParisRemote
- Member of Technical Staff - Pre-Training InfraReflectionSan Francisco, USRemote
Third-party content disclaimer
Full job descriptions are sourced directly from third-party applicant tracking systems (ATS), job boards, recruitment platforms, and employer career pages.
LiftmyCV does not claim ownership of this content or guarantee its accuracy or current availability. All original content is the intellectual property of the respective employers or other rights holders.
LiftmyCV hiring intelligence, availability checks, and related-job recommendations are generated from LiftmyCV data and public sources.
If you prefer, you may apply directly via the original listing for transparency.