LiftmyCVLiftmyCV
CS

Manager, AI Benchmarking and Evaluation Research

CrowdStrike S.R.L.RemoteRO
AI job summary

CrowdStrike, a leader in cybersecurity, is seeking a Manager for AI Benchmarking and Evaluation Research. The ideal candidate will guide a team in designing evaluations for AI models used in cybersecurity, ensuring rigorous standards for effectiveness. Responsibilities include leading a team of researchers, defining evaluation metrics for AI systems, and collaborating cross-functionally to enhance security operations. Ideal candidates will possess SOC experience, leadership skills, and an understanding of AI and cybersecurity. Competitive benefits include salary, stock options, and healthcare.

Job description

About the Role:

The CrowdStrike Data Science Team is looking for an experienced and driven leader to build and guide a team dedicated to designing and building evaluations for AI models that perform cybersecurity tasks. Our mission is to establish rigorous, reproducible standards for measuring how well AI and agentic systems support real-world security operations. As the Evaluation Lead, you will bring deep, hands-on SOC expertise to the table, defining what "good" looks like for AI models operating in security workflows. You will build the datasets and methodologies that ground our models in the realities of frontline defense, and lead a team that turns findings into actionable insights for engineering and product stakeholders.

What You'll Do:

  • Lead, mentor and grow a team of researchers and engineers focused on evaluating AI models applied to cybersecurity tasks
  • Define the strategy, roadmap and success metrics for evaluating AI/LLM and agentic systems across security use cases (e.g. incident response, threat hunting, alert triage, investigation)
  • Design and standardize evaluation methodologies, benchmark datasets and reproducible testing pipelines rooted in real SOC workflows
  • Assess models for accuracy, robustness, reliability and operational effectiveness in security-analyst scenarios
  • Establish quantitative and qualitative metrics that reflect how AI models and agentic systems perform against genuine threat-detection and response challenges
  • Collaborate cross-functionally with engineering, product and threat-research teams to translate evaluation findings into scalable improvements
  • Communicate results, trade-offs and recommendations clearly to both technical and executive audiences

What You'll Need:

  • Hands-on SOC experience with a strong understanding of day-to-day security operations
  • At least 3 years in a management or team-leadership position, with a proven track record of mentoring and growing technical teams (formal manager experience is a plus)
  • Strong knowledge of incident response and threat hunting, including detection, investigation and remediation workflows
  • Broad knowledge of the cybersecurity landscape, including attack vectors, defense mechanisms, and the analyst workflows that AI aims to augment
  • Practical experience working with AI, including familiarity with AI/LLM capabilities and knowledge of agentic systems and how they operate
  • Ability to define, design and standardize evaluation methodologies and reproducible testing pipelines
  • Exceptional communication skills, including the ability to present complex technical findings clearly to both technical and non-technical audiences
  • Demonstrated track record of delivering results, supported by shareable projects or measurable outcomes
  • A genuine passion for working at the forefront of AI and cybersecurity, with a proactive mindset toward continuous learning and innovation in a rapidly evolving field

Bonus Points:

  • Relevant security certifications (e.g., GCIA, GCIH, GCFA, OSCP, or equivalent)
  • Experience building or working with LLM/agentic evaluation frameworks and benchmarking pipelines
  • Programming experience in Python and familiarity with data-science tooling
  • Familiarity with SOC platforms, SIEM/SOAR tools, and detection engineering
  • Exposure to adversarial testing, red-teaming, or robustness evaluation of AI systems
  • Understanding of MITRE ATT&CK and how it maps to detection and response workflows

What We Offer:

  • Competitive salary
  • Stock options
  • Private Healthcare insurance
  • Life insurance
  • Training budget
  • Working with the latest technologies
  • Flexible time off

Benefits of Working at CrowdStrike:

  • Market leader in compensation and equity awards
  • Comprehensive physical and mental wellness programs
  • Competitive vacation and holidays for recharge
  • Paid parental and adoption leaves
  • Professional development opportunities for all employees regardless of level or role
  • Employee Networks, geographic neighborhood groups, and volunteer opportunities to build connections
  • Vibrant office culture with world class amenities
  • Great Place to Work Certified™ across the globe

CrowdStrike is proud to be an equal opportunity employer. We are committed to fostering a culture of belonging where everyone is valued for who they are and empowered to succeed. We support veterans and individuals with disabilities through our affirmative action program. CrowdStrike is committed to providing equal employment opportunity for all employees and applicants for employment.

Job details

Published
September 8, 2026
Job source
myworkdayjobs
Workplace
Remote · RO
First seen
September 8, 2026
AI match explained

See how well you match this job

Log in or sign up and Eva will compare your profile with this role, explain your strengths and show what to improve.

Sign up to see your matchAlready have an account? Log in

More at CrowdStrike S.R.L.

Similar jobs from this company

None listed.

Across the market

Similar jobs from other companies

Third-party content disclaimer

Full job descriptions are sourced directly from third-party applicant tracking systems (ATS), job boards, recruitment platforms, and employer career pages.

LiftmyCV does not claim ownership of this content or guarantee its accuracy or current availability. All original content is the intellectual property of the respective employers or other rights holders.

LiftmyCV hiring intelligence, availability checks, and related-job recommendations are generated from LiftmyCV data and public sources.

If you prefer, you may apply directly via the original listing for transparency.