- Experience
- 2 – 5 years
- Pay
- ₹20 L – ₹50 L per year
- Education
- Not specified
- Location
- Bengaluru, India
- Work mode
- On-site
- Job type
- Full-time
About this role
100ms builds live video infrastructure and is developing AI agents for healthcare. The AI Engineer designs, evaluates, and scales LLM-powered agents, focusing on datasets, evaluations, model performance, and systems that improve agent quality and reliability in real workflows.
What you will do
- Build, curate, and maintain agentic workflows and datasets for training and evaluation
- Run systematic LLM evaluations and track regressions
- Define and implement LLM performance metrics
- Experiment with prompting, fine-tuning, retrieval strategies, and model configurations
- Develop internal tooling for evaluation, annotation, and iteration
- Build automated pipelines for regression checks and model monitoring
- Create mechanisms to turn user interactions into model improvements
- Debug complex system behaviours spanning prompts, tools, APIs, and model responses
- Tune latency and conversational naturalness for voice AI systems
- Write clean code in Python or JS for model pipelines and integrations
What the employer expects
- 2–5 years experience in software engineering with strong fundamentals
- 1–2 years experience on AI-backed products (LLM engineering, evaluations, prompting, or agent development)
- Hands-on experience with LLMs, prompt engineering, evaluation frameworks, or dataset creation
- Strong understanding of how LLMs work including prompting, fine-tuning, evaluation, and guardrails
- Solid software engineering skills and comfort with APIs and cloud environments
- Experience building internal tools or automation pipelines
- Ability to think from the customer backward
Skills
- LLMs
- prompt engineering
- evaluation frameworks
- dataset creation
- Python
- JavaScript
- APIs
- cloud environments
Apply on the employer's site
The application opens on Lever. Dev Portal Notes does not collect your application or charge a fee.
Know someone who fits this role?
Applications are handled on the employer's own site.
Prepare for this role
Close the skill gap
Udemy
Become an AI Engineer: LLMs, RAG and Agents
Learn to build with large language models: prompting, fine-tuning, RAG and agents, in working code.
Get the courseUdemy
The Python Mega Course
Python taught by building real applications, from scripting through web apps and data work.
Get the courseSome links on this page may generate a small commission at no extra cost to you.