Senior Software Engineer, AI Evals
Sentry · Engineering
- Location
- San Francisco, California, United States · Remote
- Metro
- San Francisco Bay Area
- Workplace
- Hybrid
- Level
- Senior
- Type
- Full-time
- Pay
- $155K–$400K/yr (published by employer)
- Posted
- 9 months ago
- Source
- Ashby
Read the full posting & apply ↗
What the posting covers
About Sentry
Software runs the world and the pace is faster than ever. Sentry helps developers fix errors and performance issues before users notice, so teams can spend less time firefighting and more time building. Trusted by 200,000+ organizations, Sentry is today’s…
About the role
As a Senior Software Engineer on Sentry’s AI/ML team, you’ll be responsible for building the evaluation infrastructure that measures the accuracy, reliability, and real-world performance of our AI systems. This role is critical to ensuring that our debugging…
In this role you will
Design and build robust evaluation frameworks to measure accuracy, reliability, regressions, and edge cases in AI systems Create and curate high-quality datasets, golden test cases, and benchmarks grounded in real production data Build automated test…
You’ll love this job if you
Care deeply about correctness, rigor, and measurement in AI systems Enjoy turning fuzzy product goals and model behavior into concrete tests and metrics Like building foundational infrastructure that unlocks faster iteration and higher confidence for the…
Qualifications
Minimum 5+ years of professional experience with a Bachelor’s degree in computer science, machine learning, or a related field Experience building testing, evaluation, or data infrastructure for complex systems (AI/ML experience strongly preferred) Comfort…
Excerpts only. The employer’s page has the complete posting.