Note: The job is a remote job and is open to candidates in USA. Fetch is building AI and automation systems that enhance operational efficiency. The Evaluation Lead will oversee automation and evaluation programs within AI Operations, translating business goals into scalable systems and ensuring high-quality performance.
Responsibilities
- Lead complex, high-impact automation and evaluation initiatives across workflows or teams
- Architect end-to-end workflows integrating datasets, evaluations, automations, and HITL processes
- Define best practices for datasets, evaluation methodologies, and quality measurement
- Influence priorities and partner with Engineering, Product, and AI teams to deliver solutions
- Lead redesign of workflows, tooling, and processes to improve performance and durability
- Define success metrics and communicate performance and recommendations to leadership
- Raise the bar through your work and actively mentor others by sharing approaches, guiding problem-solving, and enabling the team to build stronger automation and evaluation capabilities
- Stay current on emerging tools and approaches; pilot and translate them into actionable improvements for the team
Skills
- 5+ years of experience and the proven ability to lead complex automation or evaluation initiatives across systems or teams
- Experience designing, building, and scaling evaluation frameworks, datasets, and quality systems at scale for LLMs or AI products
- Strong understanding of AI/LLM workflows, automation platforms, and system design
- Experience in influencing cross-functional stakeholders and aligning priorities
- Demonstrated ability to define metrics and drive measurable business impact
- Familiarity with data pipelines, APIs, or production systems
- Experience mentoring or leading technical contributors
- Background in AI Operations, applied AI, or process optimization
Benefits
- Competitive compensation packages including base, equity, and benefits
Company Overview
Company H1B Sponsorship