AI Evals for MCP in AIOps: how to evaluate and improve your AI agents

SECTION 1

The challenge of evaluating AI agents in production

AIOps (Artificial Intelligence for IT Operations) systems increasingly rely on autonomous AI agents to monitor, diagnose and resolve IT incidents. With the emergence of the Model Context Protocol (MCP), these agents can now access rich context and execute complex actions.

This growing autonomy raises critical governance and reliability questions. How do you make sure these agents work correctly? How do you measure their performance, detect regressions and guarantee they won't make critical mistakes in production?

This is where AI Evaluations (Evals) come in: rigorous frameworks for testing, measuring and continuously improving AI agents. At Adservio, we've developed a complete evaluation methodology for AI agents in AIOps contexts, particularly those using MCP. This article shares our approach and the best practices drawn from our consulting engagements.

SECTION 2

What is the Model Context Protocol (MCP)?

Model Context Protocol (MCP) is an open standard that lets AI agents access rich context and interact with external systems in a standardized way. Developed by Anthropic, MCP solves a fundamental problem: how do you give LLMs access to the data and tools they need to be useful in production?

MCP rests on four key components: Context Servers, which expose contextual data from multiple sources (databases, APIs, filesystems) in a unified, queryable format; a standardized protocol, which removes the need for adapters specific to each data source; a secure tool-execution mechanism, which lets agents act beyond simply reading data; and a built-in security layer offering granular access controls, authentication and full action auditing.

In a real operational context, an AIOps agent using MCP can orchestrate an incident's entire lifecycle: querying logs via an Elasticsearch Context Server to identify error patterns, correlating anomalies with Prometheus metrics, consulting the CMDB to understand dependencies, then executing remediation actions via MCP Tools (service restart, resource scaling, deployment rollback).

The problem: with that much power, access to critical systems plus the ability to act, how do you make sure the agent doesn't make catastrophic mistakes?

@cite:le-protocole-model-context-au-dela-de-la-tendance

SECTION 3

Why classic evaluations fall short

Traditional software testing approaches are inadequate for AI agents, for three structural reasons.

Unit tests assume determinism: yet an LLM can generate different outputs for the same input, invalidating the "function(X) must return Y" paradigm. Integration tests assume enumerable scenarios: yet AI agents exhibit emergent behaviors in a near-infinite state space, which calls for coverage based on statistical distributions rather than isolated cases. Finally, classic production monitoring measures uptime and latency, not decision quality: an agent can show 99.9% availability while making bad decisions 30% of the time.

To ensure the operational viability of AI agents in an enterprise context, Adservio has identified five fundamental requirements: evaluate the semantic and factual quality of responses (not just their availability), test on statistical distributions of inputs, measure robustness and graceful degradation in the face of unexpected situations, detect hallucinations and behavioral drift, and guarantee continuous alignment with business objectives and regulatory constraints.

SECTION 4

Evaluation framework for AIOps AI agents

At Adservio, we use a 5-dimension evaluation framework that draws on industry best practices while incorporating the specifics of the AIOps context.

### Dimension 1: functional correctness

Does the agent correctly accomplish the requested task? The evaluation rests on golden datasets: reference corpora annotated by domain experts, pairing real inputs with expected outputs. For example, a "High CPU usage on prod-web-01" alert with its context (95% CPU, 45% memory, a recent deployment of api-service version 2.3.1) is paired with an expected high-confidence diagnosis pointing to that deployment as the likely cause, along with recommended actions: check the service's logs, review the version's changes, consider a rollback.

Automated scoring then compares the agent's output against the golden output using three metrics: exact match (too strict for LLMs, reserved for structured formats), semantic similarity (embeddings cosine similarity) and component match (presence of the business-critical elements: correct diagnosis, relevant actions). Typical success thresholds: diagnosis similarity above 0.85, action recall above 0.70 and an overall score above 0.75, evaluated against the golden dataset on every model or prompt change.

### Dimension 2: safety and guardrails

Does the agent avoid dangerous actions? Three test categories structure this dimension. Forbidden actions first: the agent must never execute certain commands (rm -rf /, DROP DATABASE, disabling firewalls), even when the scenario seems to justify them, facing logs that saturate the disk, it must recommend rotation or archiving, never brute-force deletion. Appropriate escalation next: for critical or ambiguous situations, the agent must delegate to a human rather than act alone. Policy compliance finally: no restarts during peak hours, no changes without an ITSM ticket, consistent with clients' governance frameworks (ITIL, COBIT).

The associated metrics target a 0% forbidden-action rate, an unnecessary-escalation rate under 5% and a 0% missing-escalation rate. In addition, a dedicated red team actively tries to derail the agent (prompt injection, contextual manipulation, ambiguous cases) to uncover unanticipated failure modes, at Adservio, a quarterly exercise with team rotation.

### Dimension 3: context utilization

Does the agent make effective use of the available MCP contexts? Three aspects are evaluated. Retrieval precision: fetch only relevant data, facing a latency spike on the /users API, query the API's logs and metrics and the database metrics, not the frontend logs or the mail server, with expected precision above 0.8 and recall above 0.9. Context integration: intelligently combine multiple sources, connect a CPU spike at 14:32, a deployment started at 14:30 and the CMDB entry identifying api-service v2.1, to form the causal hypothesis and go fetch the version's release notes. Query optimization: minimize the number of queries while maintaining diagnostic quality, using precise queries rather than broad, costly scans.

The metrics: context precision and recall, average number of queries per resolved incident, and a human score on the quality of the multi-context synthesis.

### Dimension 4: latency and performance

Does the agent respond fast enough to be useful in production, where every second counts toward MTTR? End-to-end latency is measured as a full distribution (P50, P95, P99) to capture tail behavior, and breaks down into four components: LLM inference, context retrieval, tool execution and system overhead. A typical diagnosis profiles as roughly 1.2 seconds of context retrieval, 3.5 seconds of LLM inference and 0.8 seconds of tool execution when an action is required.

Common optimizations follow a measurable hierarchy of impact: caching frequently accessed contexts (40-60% fewer MCP calls), parallel retrieval of multiple contexts (30-50% lower total latency), streaming responses for perceived responsiveness, and fast models for triage while reserving reasoning models for deep analysis. Add to this cost efficiency: the cost per resolved incident (LLM calls, MCP queries, infrastructure) must be minimized without sacrificing service quality.

### Dimension 5: robustness and reliability

Does the agent stay performant when facing the unexpected? Four test families cover this dimension. Adversarial inputs: an empty alert, an alert tens of thousands of characters long, a vague alert ("Help!!! EVERYTHING IS DOWN") or a contradictory one, the agent must never crash, but ask for clarification or decline gracefully. Missing context: if the logs context server goes down, the agent must explicitly flag the absence of that source, continue its diagnosis with the available data and lower its confidence level, typically below 0.7. Distribution shift: measure the performance degradation on an out-of-distribution dataset representative of emerging scenarios, targeting degradation under 20%. Temporal stability finally: a weekly evaluation against the fixed golden dataset, with an automatic alert as soon as the score drops below 90% of the reference.

The target metrics: more than 95% of adversarial inputs handled correctly, a graceful degradation score above 0.7 and out-of-distribution performance retention above 0.8.

SECTION 5

Continuous evaluation pipeline

Evaluation isn't a one-off event but a continuous process built into the development and deployment cycle: any code, prompt or model change automatically triggers the evaluation suite (functional, safety and performance tests), whose results are aggregated into an overall score, per-dimension scores and regression detection, which decide whether to promote to staging or block the deployment with a team alert.

The cadence follows a granularity progression aligned with development cycles: smoke tests at pre-commit (under one minute, critical tests only), the full suite at pre-merge (10 to 15 minutes), extended evaluation and adversarial testing nightly (1 to 2 hours), human evaluation and red team exercises weekly, and continuous monitoring plus interaction sampling in production.

Gating criteria define the thresholds that must not be crossed to authorize a deployment: an overall functional score of at least 0.75 with zero regressions; a strictly zero forbidden-action rate and a missing-escalation rate under 5% on the safety side; a P95 latency under 120 seconds and a cost per incident under $0.50 on the performance side; an adversarial success rate above 95% and an out-of-distribution retention above 80% on the robustness side. If any of these gates fails, the deployment is automatically blocked.

SECTION 6

Tools and frameworks for AI Evals

The AI evaluation tooling ecosystem has matured considerably. Four families stand out: OpenAI Evals, the extensible reference library for building and running multi-model LLM evaluations with custom scorers; LangSmith (LangChain), a testing and monitoring platform with end-to-end tracing, versioned datasets and prompt A/B testing; PromptFoo, a data-driven prompt-testing framework with a CLI and native CI/CD integration; and internal frameworks, indispensable for AIOps-native metrics deeply integrated with existing infrastructure (MCP, observability stack).

The typical stack at Adservio combines these building blocks in a hybrid strategy: evaluation orchestration is driven by Airflow, which runs OpenAI Evals alongside custom internal evaluations, stores results in PostgreSQL and surfaces them in Grafana dashboards for continuous monitoring.

@cite:comment-evaluer-un-systeme-llm

SECTION 7

Case study: evaluating an incident-diagnosis agent

A client operated an AI agent for automated diagnosis of critical production incidents, using MCP to access logs (Elasticsearch), metrics (Prometheus), the CMDB and runbooks (Confluence). Business goal: diagnose more than 70% of incidents without human intervention to cut MTTR by 35%.

The golden dataset took three months of collection and annotation by the SRE teams: 500 real, anonymized incidents from the past 12 months, each annotated with its root cause double-validated by a senior SRE and the remediation actions that worked; a stratified split of 400 cases for prompt tuning and few-shot and 100 for the final evaluation; a distribution of 60% application incidents, 25% infrastructure and 15% network.

The baseline evaluation across the 5 dimensions yielded: 68% exact match and 82% semantic accuracy on functional correctness; 2 forbidden actions attempted out of 50 adversarial scenarios (a 4% failure rate), failing the safety gate; an average of 8.3 MCP queries per diagnosis (target: under 5) with context precision of 0.91 and recall of 0.87; a P95 latency of 145 seconds (target: under 120), failing a second gate, at a cost of $0.32 per diagnosis; and out-of-distribution retention of 71%. Two gates failed: deployment blocked.

Three iterations over six weeks corrected course. The first addressed safety: explicit guardrails in the system prompt, a validation layer before tool execution and 30 new adversarial cases in the red team dataset, forbidden-action rate brought down to 0%. The second optimized latency: parallel retrieval of three MCP contexts, a faster latest-generation model for triage while reserving a full reasoning model for complex cases, caching of frequent runbooks (65% hit rate),P95 cut from 145 to 98 seconds, a 32% reduction. The third improved accuracy: few-shot examples per major incident type, light fine-tuning on the 400 training cases and a guided reasoning structure in the prompt, semantic accuracy raised from 82% to 88%.

With every gate passed, the deployment was approved. Three months later: 73% of incidents diagnosed automatically (against a 70% target), MTTR cut by 35% (from an average of 45 to 29 minutes), SRE satisfaction of 4.3/5, zero security incidents across 1,247 incidents handled, and an estimated €180k in annual operational savings.

SECTION 8

Best practices

Lessons drawn from our client deployments converge on six guiding principles.

Start with a quality golden dataset: it determines the performance ceiling you can reach. Plan on 3 to 6 months for a mature dataset that is diverse (70% common cases, 20% complex cases, 10% adversarial edge cases) and regularly updated as new incident types appear.

Automate as much as possible: evaluations built into CI/CD for immediate feedback, automatic regression alerts with configurable thresholds, real-time dashboards, automate the routine, escalate the exceptional.

Combine automated metrics with human reviews: automation for volume and speed, humans for contextual judgment and ambiguous cases, with smart sampling focused on low-confidence predictions, typically 95% automated evaluation and 5% human review on a stratified sample.

Version everything: datasets (Git LFS, DVC), prompts (Git with a review process), models (MLflow, Weights & Biases) and evaluation results, to guarantee reproducibility and root-cause analysis of regressions.

Iterate quickly: evals under 15 minutes for short feedback loops, systematic A/B testing of prompts and models, and a virtuous data flywheel, production, enriched dataset, improvement, production, with a complete cycle under one week.

Monitor production: real-time metrics (latency, cost, error rate) with proactive alerting, sampling of 1-5% of traffic for periodic human review, automatic drift detection and a structured feedback loop where SREs rate the diagnoses. Production is the ultimate evaluation environment.

SECTION 9

Conclusion

Rigorous evaluation of AI agents isn't optional, it's a strategic imperative for any serious production deployment, particularly in critical domains like AIOps where automated decisions directly impact operational continuity.

The 5-dimension framework (functional correctness, safety, context utilization, performance, robustness) provides complete, structured coverage of what needs to be evaluated, and integrating it into a continuous evaluation pipeline ensures changes don't degrade quality while enabling high iteration velocity.

With MCP letting agents access rich context and execute critical operational actions, the quality and security stakes are even higher. At Adservio, we help our clients set up these frameworks, adapted to their specific contexts and regulatory constraints: the initial investment (3 to 6 months for a full setup) is amply repaid by more reliable, safer, higher-performing agents that generate quantifiable business value.

The era of AI agents in production is here. Rigorous evaluation isn't a brake on innovation, it's the enabler that makes it possible to deploy AI responsibly and at scale. Let's make sure we evaluate these agents to match their responsibilities.

Note: The statements and opinions expressed in this article are those of the author and do not necessarily reflect the positions of Adservio.

FAQ

Frequently asked questions