How to Evaluate an LLM System

SECTION 1

Introduction

Evaluating applications built on large language models (LLMs) is inherently complex due to the unique nature of these systems. Unlike traditional software applications, where outputs are deterministic and predictable, LLMs generate outputs that can vary on every run, even with the same input. This variability stems from the probabilistic nature of these models, which means there's no single correct output for a given input. As a result, testing LLM-based applications requires specialized evaluation techniques, known today as "evals",to ensure they meet performance and reliability standards.

SECTION 2

Why Are Evals So Critical?

There are several reasons why AI evals matter so much. Broadly speaking, they're valuable in four key ways:

1. They establish performance standards.

Evaluation helps establish performance standards for LLM systems, guiding the development process by providing directional results for design choices and hyperparameters. By setting benchmarks, developers can measure the effectiveness of different approaches and make informed decisions to improve model performance.

2. They can help ensure consistent and reliable results.

Consistency and reliability are essential for the practical deployment of LLM systems. Regular evaluations help identify and mitigate issues that could lead to unpredictable or erroneous outputs. Ensuring the system produces stable, reliable results builds trust and confidence among users and stakeholders.

3. They provide insights to guide improvement.

Continuous evaluation provides valuable insights into an LLM system's performance. It highlights where the system excels and where it needs improvement, creating opportunities for targeted enhancements. By understanding the model's strengths and weaknesses, developers can refine and optimize the system for better performance.

4. They enable regression testing.

Whenever changes are made to an LLM system, whether to prompts, design choices, or underlying algorithms, regression testing becomes essential. Evaluation ensures these changes don't degrade output quality. It verifies that new updates maintain or improve the system's performance, preventing unintended consequences and preserving the application's integrity.

Evaluating LLM systems can broadly be divided into two categories: pre-deployment evaluations and production evaluations. Each category serves distinct purposes and is critical at different stages of the development and deployment lifecycle.

SECTION 3

Pre-Deployment Evaluations

Pre-deployment evaluations focus on assessing LLM systems during the development phase. This phase is critical for shaping the system's performance and reliability before it goes live. Here's why pre-deployment evaluations are essential:

Measuring performance and setting benchmarks:

During the development phase, evaluating your LLM system provides a clear measure of its performance. Using a variety of metrics and evaluation techniques, developers can assess the system's capabilities. Setting these benchmarks helps compare different model versions and understand the impact of various architectural and design choices. By identifying strengths and weaknesses early, developers can make informed decisions to improve efficiency, accuracy, and overall performance.

Ensuring regression-free updates:

As the system undergoes continuous development, changes to code, model parameters, or data can unintentionally introduce regressions, unintended reductions in performance or accuracy. Regular pre-deployment evaluations help ensure that every change improves, or at least maintains, performance standards.

SECTION 4

How to Run a Pre-Deployment Evaluation

To run a pre-deployment evaluation, here are the steps you need to follow:

### Create a Ground-Truth Dataset

Create a ground-truth dataset for evaluation, The first step, and perhaps the most critical, in evaluating LLM systems is creating a robust ground-truth dataset. This dataset comprises a set of question-answer pairs generated by expert users. They essentially serve as a benchmark against which to evaluate the LLM's performance.

Ground-truth data is essential because it provides a reference point against which the model's outputs can be compared. It should be representative of the kinds of questions end users are likely to ask in production and include a diverse range of possible questions to cover different scenarios and contexts.

Creating ground-truth data requires the expertise of expert users who have a deep understanding of the business domain and user behavior. These experts can accurately predict the kinds of questions users will ask and provide the best answers. This level of understanding and contextual knowledge is something LLMs, despite their advanced capabilities, may not possess.

### Can LLMs Create Ground Truth?

Can LLMs generate ground truth? While LLMs can help generate ground-truth data, they shouldn't be solely responsible for this task. Here's why:

They don't understand user behavior:

LLMs don't understand user behavior or the specific context of your business domain. They can generate plausible questions and answers, but these may not accurately reflect the kinds of queries your users will actually ask or the answers that would be most useful to them.

They require human oversight:

Human experts are needed to review and refine the questions and answers generated by LLMs. They ensure the dataset is realistic, contextually accurate, and valuable for end users.

Ensuring quality and relevance is vital:

The quality of the ground-truth dataset is paramount. Human oversight ensures the questions and answers aren't just relevant but also adhere to company standards and user expectations.

Here's a good example of a ground-truth dataset for a RAG application. In addition to the query and the answer, this dataset provides the various passages relevant to the query from the knowledge base.

### Identify and Calculate the Relevant Metrics

Identify the relevant metrics for your LLM system, Selecting the right evaluation metric is crucial for assessing LLM system performance. The choice of metric depends on the LLM system's specific use case, since different applications may need to measure different aspects of model performance.

Here are some examples of evaluation metrics and their definitions:

Answer Relevance

Definition: This metric measures how relevant the provided answer is to the question asked. It assesses whether the answer directly addresses the query and provides useful, relevant information. Importance: Ensuring the model's answers are relevant helps maintain user satisfaction and trust in the system. Irrelevant answers can confuse or frustrate users, diminishing the application's value.

2. Coherence

Definition: Coherence assesses the logical flow and clarity of the generated text. It checks whether the answer is internally consistent and makes sense as a whole. Importance: Coherent answers are easier for users to understand and follow. This metric is vital for applications where clarity and comprehensibility are essential, such as customer support or educational tools.

3. Contextual Relevance

Definition: This metric measures how well the model's output aligns with the broader context provided. It assesses whether the answer properly accounts for the surrounding text or conversation. Importance: Contextual relevance ensures the model's answers are appropriate and meaningful within the given context. This is critical for maintaining continuity and relevance in conversations or content.

4. Responsibility Metrics

Definition: Responsibility metrics assess the ethical and appropriate nature of the model's output. This includes checking for bias, harmful content, and adherence to ethical standards. Importance: Ensuring responsible AI use is crucial for preventing the spread of misinformation, harmful stereotypes, and unethical content. These metrics help build trust and ensure the LLM system adheres to societal and ethical norms.

@cite:quatre-techniques-de-recuperation-pour-ameliorer-la-rag

The RAG triad consists of the following metrics:

Retrieval Metrics

Faithfulness

The factual accuracy of the generated answer

Context Precision

The signal-to-noise ratio of the retrieved context

Answer Relevance

How relevant the generated answer is to the question

Context Recall

Can it retrieve all the relevant information needed to answer the question?

### Task-Specific Metrics

While the metric examples above can be used across various use cases and tasks, you'll need metrics that are more tailored to the particular task you're performing. Task-specific metrics assess model performance on particular tasks, tailored to the specific requirements of the application. Examples include metrics for summarization, translation, and sentiment analysis.

Why task-specific metrics are needed

Most evaluation metrics are generic and provide a broad assessment of LLM system performance. However, to understand system performance on specific tasks, custom metrics tailored to the task at hand are needed. Task-specific metrics offer detailed insight into the model's effectiveness at implementing particular features, ensuring the LLM system meets each application's unique requirements.

For example, for abstractive summarization tasks, Kryscinski et al. (2019) propose the following:

How grammatical and readable the summary is

How well the summary flows and connects ideas

Whether the summary is factually consistent with the source

Calculate the score for each of your defined metrics against ground truth, The next step in evaluating your LLM system is calculating scores for each defined metric against your ground truth. For each question in your ground-truth dataset, use the answer generated by the LLM system to calculate the respective metric. If one or more metrics yield unsatisfactory results, make the necessary adjustments to your LLM system to improve those metrics. Libraries like DeepEval and Relari-ai use NLP libraries to compare your LLM answers against ground truth and calculate these metrics. Metrics are calculated by leveraging LLMs, other NLP models, or traditional code functions.

It's important to make metric-driven decisions about your LLM system's design based on the metrics observed. For example, low recall for questions expecting short factual answers might require you to reduce your chunk sizes. Low precision even with high K values could benefit from re-ranking your retrieved chunks. Similarly, the various components of your LLM system's workflow, prompts, inference parameters, chunking strategy, retrieval mechanisms, embedding choice, etc.,should be optimized based on the metrics.

There's an emerging trend of using a powerful LLM (for example, GPT-5.6) as a reference-free metric (without ground truth) to evaluate other LLMs' generations. This is sometimes referred to as "LLM-as-judge." The G-Eval framework is a good example of this. The paper argues that, when used through this framework, a frontier LLM has a strong correlation with human evaluators. Similar arguments have been made in the Vicuna and QLoRA papers.

However, the reliability and granularity achieved by using ground truth for evals is far better than what's achieved using an LLM evaluator, as detailed in this post. What's more, some metrics like context recall can't be measured without ground truth.

In conclusion, while LLMs can provide valuable directional insights and help streamline the evaluation process, they can't fully replace evaluations obtained from ground-truth data. Ground-truth evaluations, while labor-intensive, offer unmatched accuracy and reliability. Combining both approaches can be beneficial.

### Integrate Evals Into Your Deployment Process

To ensure your LLM system consistently meets the required performance criteria, it's essential to integrate evaluations into your deployment pipelines. This integration not only validates model performance before deployment but also maintains quality and reliability throughout the development lifecycle.

Tests run automatically on every commit and before a deployment release to ensure code changes don't introduce errors or degrade performance. This article covers writing unit test cases for LLMs in fairly detailed fashion.

In addition to running the automated tests you've written, tools like Giskard can help run scans in your deployment pipelines to test your LLM across several dimensions like harmfulness, hallucinations, and sensitive information. Below are some examples of automated tests implemented using Giskard to check for hallucinations and harmfulness. Giskard runs these tests as part of your deployment process.

One key point to note about automated testing for LLM systems is that we'll also need to write tests for the data preprocessing and ingestion steps.

SECTION 5

Post-Deployment Evaluations and Data Flywheels

To ensure your LLM system continues performing optimally after deployment, it's crucial to implement robust observability layers. These layers provide the necessary traces of input-output interactions, helping you understand where the system might be failing or underperforming.

Monitoring interactions lets you capture real-time data on how the LLM system handles different questions and scenarios. Set up continuous monitoring to catch anomalies and performance issues. While automated tests provide continuous monitoring, they aren't sufficient on their own. Human evaluations are essential for capturing the nuance and context automated systems might miss. Schedule periodic evaluation sessions with domain experts to assess the LLM's performance and provide feedback for improvement. Finally, build a feedback mechanism where users can report issues or provide feedback directly from the interface.

### Data Flywheels and Continuous LLM Improvement

The concept of data flywheels is fundamental to the continuous improvement of LLM systems. A data flywheel is a self-reinforcing loop that leverages data collected from operational environments to drive ongoing performance improvements. For LLM systems, this means using real-time observations and feedback from production environments to refine your workflow, ensuring it becomes more accurate, relevant, and efficient over time. As discussed earlier, insights from metrics can be used to make changes to the various components of your workflow, from chunking strategy to embeddings to prompts and retrieval methods.

It's important to note that the metrics you land on aren't static, they may need to be adjusted and adapted over time. This happens as you learn more about end-user behavior and identify new usage patterns and failure scenarios in production.

By understanding how users interact with the system and where misunderstandings or inefficiencies arise, you can adjust your LLM system to be clearer, more specific, and better aligned with user intent. Similarly, workflows can be streamlined to reduce friction and improve the overall user experience.

The beauty of the data flywheel approach is its cyclical nature. As you implement improvements based on your observations, those changes generate new data, providing fresh insights for further refinement. This creates a continuously improving system that becomes more efficient and effective with each iteration.

@cite:testing-des-systemes-ia-construire-la-confiance-dans-le-non

SECTION 6

Evals First: The Key to Building Reliable LLM Applications

To build reliable, high-performing LLM applications, shifting evaluations earlier in the development workflow isn't just beneficial, it's essential. By embedding an evals-driven approach from the start, teams can proactively identify gaps, improve their implementations, and ensure alignment with user expectations from day one.

The choice of evaluation methods should be guided by the nature of the application and the desired user experience. That means it's crucial to think from the outset about what the right metrics and benchmarks are. Don't treat evals as an afterthought, make them a foundational part of your development process for building robust, user-centered AI applications.

Disclaimer: The statements and opinions expressed in this article are those of the author(s) and do not necessarily reflect the positions of Adservio.

FAQ

Frequently asked questions