Software Engineer, Inference / Compute Infrastructure Engineering
Together AI· San Francisco, USA
Posted November 9, 2026
- Experience
- Not specified
- Pay
- $160K – $280K per year
- Education
- Not specified
- Location
- San Francisco, USA
- Work mode
- Hybrid
- Job type
- Full-time
About this role
Together AI is an AI cloud that runs inference, fine-tuning and training for AI developers. This engineer builds the Kubernetes-based control plane that sets up and runs its GPU fleet, with self-service APIs and automatic repair. Open at junior, senior and staff levels.
What you will do
- Build the state machine for the full life of a physical host
- Build self-service APIs so teams can create inference clusters with one call
- Automate finding, draining and repairing bad nodes
- Keep the provisioning pipeline reliable: retries, rollback, drift checks
- Work with the inference and ML platform team on cluster needs
- Treat infrastructure as software with tests, review and CI/CD
What the employer expects
- Strong software skills in Go, Python, Rust or similar
- Experience with durable workflow tools such as Temporal or Cadence
- Experience with control planes or Kubernetes controllers
- Experience with event-driven systems such as Kafka, NATS or SQS
- Product mindset for internal platforms and APIs
- Willing to work 4 days a week in the San Francisco office
Benefits
- Startup equity
- Health insurance
Skills
- Go
- Python
- Rust
- Kubernetes
- Temporal
- Kafka
- GPU infrastructure
- Event-driven systems
Apply on the employer's site
The application opens on Together AI careers page (Greenhouse). Dev Portal Notes does not collect your application or charge a fee.
Know someone who fits this role?
Applications are handled on the employer's own site.
Prepare for this role
Close the skill gap
Golang Backend Masterclass: REST APIs & Microservices
Learn Go from fundamentals through real projects: concurrency, REST, GraphQL, gRPC, databases, testing, Docker and AWS.
Get the courseThe Python Mega Course
Python taught by building real applications, from scripting through web apps and data work.
Get the courseSome links on this page may generate a small commission at no extra cost to you.