LiftmyCVLiftmyCV
II

Senior Model Evaluation Analyst

Ignite ITHybridSuitland, US

Job description

Ignite IT Hub is seeking a senior Model Evaluation Analyst to assess the accuracy and reliability of the U.S. Census Bureau's LLM Autocoder. You will maintain evaluation datasets, run comparisons, investigate coding errors, and produce evidence for Government review. Working with the contract’s Data Scientist and Government subject matter experts (SMEs), you will evaluate enhancements for the American Community Survey (ACS) and expansion to other surveys and fields. Responsibilities

  • Prepare and maintain versioned evaluation datasets, reference labels, and test cases with documented provenance and representative coverage of coding categories.
  • Execute evaluation methods established with the Senior Data Scientist and calculate accuracy, joint coding rates, coverage, and results against agreed error thresholds.
  • Compare candidate models and coding methods with approved baselines; identify material category-level regressions that aggregate results may conceal.
  • Analyze difficult, ambiguous, and underrepresented cases; assemble examples for SME adjudication and track decisions and label corrections.
  • Develop repeatable Python or SQL analyses and evaluation checks, preserving the data, code, prompt, model, and configuration versions behind each result.
  • Maintain test evidence, review-comment disposition, and traceability from requirements and acceptance criteria to evaluated results and deliverables.
  • Prepare clear performance reports, tables, and briefings that explain findings and limitations; support pilot assessments, release reviews, and ongoing monitoring.

Job details

Published
September 15, 2026
Job source
jworkable
Workplace
Hybrid · Suitland, US
First seen
September 15, 2026
AI match explained

See how well you match this job

Log in or sign up and Eva will compare your profile with this role, explain your strengths and show what to improve.

Sign up to see your matchAlready have an account? Log in

More at Ignite IT

Similar jobs from this company

None listed.

Across the market

Similar jobs from other companies

Third-party content disclaimer

Full job descriptions are sourced directly from third-party applicant tracking systems (ATS), job boards, recruitment platforms, and employer career pages.

LiftmyCV does not claim ownership of this content or guarantee its accuracy or current availability. All original content is the intellectual property of the respective employers or other rights holders.

LiftmyCV hiring intelligence, availability checks, and related-job recommendations are generated from LiftmyCV data and public sources.

If you prefer, you may apply directly via the original listing for transparency.