How Can AI Simplify and Speed Up User Acceptance Testing?

SECTION 1

Introduction

User acceptance testing (UAT) has long been the final gatekeeper before a product goes live. It's where the rubber meets the road, where product owners and business stakeholders put on their user hats and test a new feature or application. But for many, this critical process feels less like a smooth handoff and more like an obstacle.

AI, however, has the potential to help us overcome those obstacles and make user acceptance testing an easier, less painful step in the product development process.

SECTION 2

Today's UAT Pain Points

Before diving into the future, let's acknowledge the common frustrations we all know. Today's UAT and feature-validation process has significant limitations.

### A Functional Mindset, Not a Technical One

A functional mindset, not a technical one, UAT is typically carried out by business people who are experts on the feature but not on code execution. They understand what the feature should do, but they lack the technical know-how to dig deep when something goes wrong. Because they also aren't well-versed in the technical systems, it becomes hard for them to test new scenarios.

This information asymmetry creates frustration on both sides. Business testers feel powerless when they hit problems they can't diagnose. Technical teams feel interrupted by support requests that could be avoided with better tooling.

At Adservio, we regularly see this pattern: a product owner discovers unexpected behavior, can't determine whether it's a bug or intended behavior, and has to interrupt the engineering team for clarification. That back-and-forth eats up valuable time and delays validation.

### A Dependency on the QA Team

A dependency on the QA team, Business teams often need a lot of help from the quality assurance (QA) team just to run their test cases. QA teams typically create the scenarios and hand them off to UAT people, and running those scenarios requires a lot of hand-holding.

This creates a bottleneck and a dependency that slows down the entire process. If a product owner wants to test a specific scenario, they typically have to wait for a QA engineer to be available to help them navigate the test environment and run the necessary steps.

This "support ticket" model for testing doesn't scale. As products grow more complex and release cycles more frequent, the volume of UAT support requests can overwhelm QA teams, creating delays and frustration.

### The Race Against the Clock

Within the small window of time allotted for UAT, business teams have to make sure every scenario is covered. That pressure can lead to rushed checks, missed edge cases, and a less polished final product. The sheer volume of test cases often makes it impossible to cover everything manually within a limited timeframe.

At Adservio, we've seen situations where business teams, under deadline pressure, had to drastically prioritize tests, focusing only on the "happy paths" and skipping error scenarios or edge cases. The result? Bugs discovered in production that could have been caught during UAT if time and resources had allowed.

SECTION 3

The AI-Powered Solution: LLMs and the Model Context Protocol (MCP)

What if we could tackle these problems and fundamentally change how UAT is done? The answer lies in harnessing the power of large language models (LLMs) and a Model Context Protocol (MCP) server.

### Rethinking Test Architecture

While feature testing is typically limited to manual testing, using AI in this context requires a different approach. Specifically, QA teams need to write their test cases in a modular, functional way, so they can be called and run independently.

Instead of writing long, scripted test plans, QA will need to write clean, clear, reusable functions. This mindset shift is crucial: moving from "documenting tests" to "architecting test primitives."

This approach isn't new in itself. The concept of reusable test functions has existed in test automation for a long time. What changes with AI is the ability for non-technical people to compose these primitives via natural language, without needing to understand programming syntax or system architecture.

SECTION 4

How This Works in Practice

Let's walk through an example to see how this works in practice, consider feature testing for an online shopping application. The requirement here would be to test the APIs that let users log in, add items to their cart, and submit an order.

### Define the Test Primitives

Each of these functions represents specific actions or scenarios:

loginWithCredentials(username, password) or submitOrder(items, shippingAddress)

QA will need to map these various test cases or write new functions to test different features. For example:

loginAsNewUser(): Logs in a newly created user ; loginAsReturningUser(userId): Logs in an existing user ; addItemToCart(itemId, quantity): Adds an item to the cart ; removeItemFromCart(itemId): Removes an item from the cart ; applyDiscountCode(code): Applies a promo code ; navigateToCheckoutPage(): Navigates to the checkout page ; submitOrder(paymentMethod): Finalizes the order

### Expose the Functions via an MCP Server

Once created, these functions can be exposed through a central MCP server. This server acts as a bridge, making the functions callable by the LLM. A product owner or business user can then simply write a scenario in natural language, such as:

"As a new user, log in and verify that I can add three items to my cart and proceed to the checkout page."

LLMs, or GenAI models, are good at understanding a user's question and breaking it down into subsequent tasks or sub-questions. Most frontier GenAI models are also good at classifying user intent. For greater accuracy, models could also be fine-tuned or augmented with a RAG system.

Translation and execution, The LLM then translates this request into a sequence of function calls to the MCP server:

loginAsNewUser() ; addItemToCart('item1') ; addItemToCart('item2') ; addItemToCart('item3') ; navigateToCheckoutPage()

The MCP server then executes these functions and reports the results.

Suddenly, UAT personas can run complex, end-to-end tests without needing constant help from the QA team. They're empowered to be self-sufficient testers, focusing on the scenarios that matter most to them as users.

SECTION 5

A New Mindset for Quality Assurance

With this approach, the QA team's role evolves from manual testers to architects of an automated test framework. For this to work, the way QA teams write their functions has to change.

Design principles for test functions, Every function needs to be designed to be called by an LLM. That means:

Clarity is king: Functions need to be named clearly and descriptively. For example, verifySuccessfulPayment() is better than checkPay(). The function name should unambiguously communicate what it does. An LLM, like a human, relies on these semantic cues to figure out which function to use for which intent.

Relevant information: Every function should have a docstring or clear, concise description explaining what it does, its inputs, and its expected outputs. That's the context the LLM needs to make smart decisions.

An example of good documentation: for a function addItemToCart(itemId, quantity) that adds an item to the logged-in user's cart, the docstring specifies the role of each parameter (itemId, the unique identifier of the product to add; quantity, the number of units, which must be greater than zero), the structure of the returned result (a dictionary with a success boolean, the new cart total cartTotal, and an optional errorMessage), as well as the possible errors (a ValueError if quantity is less than or equal to zero, an AuthenticationError if no user is logged in).

This documentation isn't just good practice for humans, it's essential for the LLM to understand when and how to use the function.

Clear input/output definitions: The function's parameters and return values must be explicitly defined. That structured data is what lets the LLM and the MCP server take the right actions and produce a meaningful, understandable result.

QA's new role, By adopting these practices, QA teams can build a robust, maintainable, human-readable library of test actions. These are modular functions that let testers (UAT) run test scenarios in the language of the business domain.

This turns QA into more than just testing, they become enablers of a faster, more effective UAT process. QA becomes a role of architecture and enablement rather than repetitive execution.

At Adservio, we've found that this evolution of the QA role is generally well received by teams. QA professionals are often frustrated by the repetitive, manual aspects of their work. The opportunity to evolve into a more strategic, technical role, architecting frameworks rather than executing scripts, is appealing to many.

SECTION 6

Beyond Efficiency: New Capabilities

This approach doesn't just make existing tests faster. It unlocks new capabilities that were previously impractical.

Augmented exploratory testing, With the technical barrier lowered, product owners can run exploratory tests far more freely. Instead of being limited to scenarios pre-scripted by QA, they can improvise, follow their intuition, and test spontaneous hypotheses.

"What happens if a user applies two promo codes at once?" "What if someone changes their shipping address after starting payment?" These questions can now be explored immediately, without waiting for a QA engineer to write and run the scenario.

On-demand regression testing, Before a major release, a product owner can ask: "Test every checkout scenario we validated last month to confirm they still work."

The LLM can interpret this request, identify the relevant test functions, run them, and produce a summary report. What could have taken hours of manual work can now be done in minutes.

Persona-driven testing, Product owners can specify user personas, and the LLM can adapt test behavior accordingly.

"Test the checkout flow from the perspective of a senior user who isn't comfortable with technology, simulate hesitation, typical input errors, and check that help messages are clear."

This ability to build behavioral nuance into tests creates more realistic coverage of real user scenarios.

@cite:cas-de-tests-generes-par-ia-a-partir-de-user-stories

SECTION 7

Challenges and Considerations

While this approach is promising, it isn't without challenges.

Upfront investment, Building a complete library of well-documented test functions requires significant upfront investment. QA teams will need to refactor existing tests and develop new primitives.

At Adservio, we recommend an incremental approach: start with the most frequently tested critical flows, demonstrate value, then progressively extend coverage. This approach lets you prove the concept and build buy-in before making a major investment.

Maintenance and evolution, As the product evolves, the test-function library needs to evolve with it. APIs change, flows get refactored, new features get added. Keeping test functions aligned with the current system requires discipline and process.

We recommend treating test functions as production code: version control, code reviews, automated tests, up-to-date documentation. The same engineering standards that apply to product code should apply to test primitives.

LLM limitations, Today's LLMs, while impressive, have limits. They can misinterpret ambiguous requests, compose sequences of functions suboptimally, or hallucinate capabilities that don't exist.

Testers need to stay vigilant and confirm that AI-run tests really match their intentions. At least initially, a "human-in-the-loop" process, where the LLM proposes a test plan for approval before execution, is the prudent approach.

@cite:shift-left-testing-benefices

SECTION 8

Conclusion: From Bottleneck to Quality Driver

By combining the power of LLMs with a modular server architecture, we can turn UAT from a frustrating bottleneck into an efficient, collaborative phase. Product owners can now test features independently and quickly, cutting the time to validation and delivering higher-quality releases.

The UAT and validation process shifts from a fragmented, manual effort to an automated, user-driven experience. This isn't just an improvement, it's a fundamental shift in how we define and achieve quality in the age of AI.

At Adservio, we believe the future of testing isn't "QA vs. AI" but "QA as AI enablers." The best QA engineers won't be replaced by AI, they'll become the architects of the frameworks that let AI amplify the testing capacity of the whole organization.

This vision calls for a mindset shift, but the potential payoff is huge: more thorough tests, faster cycles, better quality, and more fulfilled teams who spend less time on repetitive tasks and more time on complex problems that require human judgment and creativity.

Note: The statements and opinions expressed in this article are those of the author and do not necessarily reflect the positions of Adservio.

FAQ

Frequently asked questions