Senior Performance Analysis Manager

Netomi is the leading agentic AI platform for enterprise customer experience. We work with the largest global brands like Delta Airlines, MetLife, MGM, United, and others to enable agentic automation at scale across the entire customer journey. Backed by WndrCo, Y Combinator, and Index Ventures, we help enterprises drive efficiency, lower costs, and deliver higher quality customer experiences.

Want to be part of the AI revolution and transform how the world's largest global brands do business Join us!

We are looking for a Manager – AI Performance & Analytics to lead a team responsible for analyzing conversational AI performance and driving continuous improvements to Netomi's AI platform.This role sits at the intersection of data analytics, conversational AI, and LLM evaluation. The team analyzes customer conversations at scale, identifies quality gaps and automation opportunities, and partners with AI and product teams to improve model performance.

You will help define how we measure and improve the performance of AI systems that power customer support automation across millions of interactions.

AI Performance Analytics

Analyze large volumes of chatbot and AI conversation data to identify performance gaps and improvement opportunities.

Define and track key AI performance metrics such as resolution rate, containment rate, response quality, and customer satisfaction.

Build dashboards and reporting frameworks using SQL, Python, and BI tools such as Tableau.

LLM Evaluation & Quality Insights

Design frameworks to evaluate the response quality of LLM responses across dimensions such as:

tone and customer experience

safety and compliance

Build scalable processes for LLM-based automated evaluation and human review workflows.

Generate insights that inform model training data, prompts, and product features.

Lead and mentor a team of data analysts focused on AI performance and conversational analytics.

Experience

• 7–10 years of experience in data analytics, data science, or AI analytics.
• Experience managing or mentoring analytics teams.
• Experience analyzing conversational data, customer support data, or AI-driven products is a strong plus.

Strong SQL and data analysis skills.

Experience with BI tools such as Tableau or similar platforms.

Experience using Python for data analysis or experimentation.

Familiarity with machine learning or large language models (LLMs).

Preferred Experience

Experience evaluating AI or machine learning model performance.

Experience working with conversational AI platforms or chatbots.

Experience with experimentation frameworks and A/B testing.