## Job Description

We are looking for an Agentic Development & Evals Ops Engineer to design and develop AI agents and support evaluation operations for AI systems. The role involves building agent-based workflows, monitoring AI performance, and creating evaluation frameworks to ensure the reliability and accuracy of AI models.

**Key Responsibilities:**

- Design and develop **AI agents and agentic workflows** using modern AI/LLM frameworks.
- Build and maintain **evaluation pipelines (Evals Ops)** to test AI agent performance, accuracy, and reliability.
- Implement automated **evaluation frameworks for LLM outputs and agent behaviours**.
- Collaborate with engineering and product teams to improve **AI model quality and performance**.
- Develop **monitoring systems for AI agents** to track metrics such as response quality, latency, and accuracy.
- Create and maintain **documentation, evaluation reports, and testing datasets**.
- Work on **continuous improvement loops** by integrating feedback into agent training and system updates.
- Ensure **scalability, reliability, and governance** of AI systems.

**Required Skills**

- Experience with **Agentic Development frameworks** (LangChain, AutoGen, CrewAI, etc.).
- Knowledge of **LLMs and AI systems** such as OpenAI, Anthropic, or similar platforms.
- Experience with **evaluation frameworks for AI/LLMs (Evals Ops)**.
- Strong programming skills in **Python**.
- Experience with **prompt engineering and AI workflow automation**.
- Knowledge of **data analysis and model performance evaluation**.
- Familiarity with **CI/CD pipelines and experimentation workflows**.

**Preferred Skills**

- Experience with **AI observability tools**.
- Knowledge of **vector databases (Pinecone, Weaviate, FAISS)**.
- Experience working with **RAG (Retrieval Augmented Generation) systems**.
- Understanding of **AI safety, evaluation metrics, and benchmarking**.

**Qualifications**

- Bachelor’s or Master’s degree in **Computer Science, AI, Data Science, or related field**.

2+ years of experience in **AI/ML development or related roles**.

[Apply Now](https://docs.google.com/document/create)