AI Infrastructure Engineer, Serving Platform
Scale AI · Horizontals EPD
- Location
- London, UK
- Metro
- London
- Workplace
- Unspecified
- Level
- Mid-level
- Type
- Unspecified
- Posted
- 3 months ago
- Source
- Greenhouse
Read the full posting & apply ↗
PythonGoRustC++AWSGCPKubernetesDockerTerraformLLMs
What the posting covers
About the company
As a Software Engineer on the ML Infrastructure team, you will design and build platforms for scalable, reliable, and efficient serving of LLMs. Our platform powers cutting-edge research and production systems, supporting both internal and external use cases…
You will
Build and maintain fault-tolerant, high-performance systems for serving LLMs and other models at scale. Build an internal platform to empower LLM capability discovery. Collaborate with researchers and engineers to integrate and optimize models for production…
Ideally you'd have
4+ years of experience building large-scale, high-performance backend systems. Strong programming skills in one or more languages (e.g., Python, Go, Rust, C++). Experience with LLM serving and routing fundamentals (e.g. rate limiting, token streaming, load…
Nice to haves
Experience with modern LLM serving frameworks such as vLLM, SGLang, TensorRT-LLM, or text-generation-inference. PLEASE NOTE: Our policy requires a 90-day waiting period before reconsidering candidates for the same role. This allows us to ensure a fair and…
About Us
At Scale, our mission is to develop reliable AI systems for the world's most important decisions. Our products provide the high-quality data and full-stack technologies that power the world's leading models, and help enterprises and governments build, deploy,…
Excerpts only. The employer’s page has the complete posting.