How to Build Trust With AI-Powered Coding Assistants

SECTION 1

Introduction

Does your team use AI-powered coding assistants? Have you been pushed to increase usage rates? Have you noticed that, in many cases, team members aren't using AI-powered coding assistants to write code? Beyond conventional knowledge-sharing and use-case sharing, are there other effective ways to drive usage?

Ever since my team turned on GitHub Copilot three months ago, these questions have been on my mind, this blog post is my attempt to find some answers.

SECTION 2

Why Increase Usage of AI-Powered Coding Assistants?

Before discussing the "how," we first need to clarify the "why." It all rests on a fundamental assumption: AI-powered coding assistants can improve development efficiency, and the more they're used, the bigger that efficiency gain. Let's temporarily accept this assumption as valid and verify it in practice.

So can we mandate that team members use AI coding assistants in all their work? Since the assumption isn't proven, such a move carries significant risk, what if it doesn't actually improve efficiency? The key to handling this kind of situation is reducing potential losses, and the best approach is granting developers autonomy: letting them decide when and in which scenarios to use the assistants.

When developers have that freedom of choice, does usage grow naturally? Based on our team's three months of experience with GitHub Copilot, developers tend to use it frequently in the following scenarios:

Understanding code. ; Analyzing solutions. ; Function-level code refactoring. ; Function-level algorithm implementation. ; Generating unit tests.

They rarely use it in these scenarios:

Line-level code changes. Changes that could otherwise be made with IDE shortcuts. Bug fixes. Implementing complex tasks (especially ones spanning multiple files or modules).

From the perspective of inline code suggestions, usage is already near 100%, since these features are enabled by default. But line-level suggestions offer limited help: they don't reduce cognitive load, and therefore can't meaningfully improve development efficiency.

To go further, the key lies in increasing the usage rate for complex scenarios. How do we achieve that when developers have autonomy? That's what the following sections address.

@cite:a-quelle-vitesse-les-assistants-de-codage

SECTION 3

A Usage-Rate Model for AI-Powered Coding Assistants

An AI coding assistant can be understood as an automation tool capable of completing tasks assigned by developers. The field of human-machine interaction has many studies on such tools; the model in Trust, self-confidence and operators' adaptation to automation is particularly well-suited to explaining the usage rate, defined here as the percentage of time developers use the assistant relative to their total development time.

Four factors affect this rate: trust (developers' level of trust in the assistant, scorable 1 to 10 based on subjective perception), self-confidence (developers' confidence in their own abilities), bias b (the negative tendency toward the assistant, mainly shaped by past experience and environment, the higher b is, the lower the willingness to use it), and the shape parameter s (the strength of attachment to existing work habits, larger s values mean greater resistance to change).

It's worth noting that an AI coding assistant isn't a single-function automation tool like a keyboard shortcut, but a general tool capable of handling any programming task. For the same assistant, the model's parameters therefore vary by task type: a developer may trust it to generate code blocks without trusting it to fix bugs; feel comfortable implementing a feature within an existing structure without feeling comfortable refactoring an architecture; or experiment freely on personal projects without wanting to take the same risks on business projects.

### Eliminating Bias (b)

Every developer's bias varies. Some are very optimistic, willing to try all kinds of tasks, with failures barely affecting their attitude. Others are very cautious, trying only after proof of effectiveness on the specific task type, and giving up quickly after a failure.

To eliminate bias, we first need to strengthen motivation: factor the ability to use these tools into skill assessments, encourage experimentation in daily work, share success stories, summarize best practices, and solve the team's pain points.

Next, we need to recognize the costs of usage: writing prompts, reviewing code, fixing errors, and trying different approaches take time and effort, with no guaranteed payoff, sometimes repeated attempts reveal the assistant can't complete the task at all. If these costs aren't acknowledged, developers bear them alone and grow increasingly hesitant.

Finally, we need to improve information sharing within the team: unlike IDEs, assistants have no fixed usage pattern, and each person may discover approaches with very different success rates. Continuously collecting successful or failed usage patterns and widely promoting effective practices is the best way to reduce bias.

### Providing More Information to Calibrate Subjective Perception

Developers' trust and self-confidence are subjective perceptions. We need to provide enough information to objectively surface the assistants' development efficiency as well as the developers' own.

For example, we can tag user stories and commits produced with an assistant's help, then separately analyze their metrics to assess the tool's actual efficiency contribution.

That leaves the main lever: improving the trust level. Often reserved for interpersonal relationships, the notion of trust has been extended, in human-machine interaction, to automation tools. That's the focus of the next section.

SECTION 4

A Trust Model for AI-Powered Coding Assistants

### What Is Trust?

I use the definition from Trust in Automation: Designing for Appropriate Reliance: trust is the attitude that an agent (such as an AI coding assistant) will help achieve an individual's goals in a situation characterized by uncertainty and vulnerability.

Various models describe the components of trust: competence, integrity, consistency, loyalty, and openness for some; credibility, reliability, and intimacy for others; ability, benevolence, and integrity for yet others. Here I adopt the model from the same paper, which breaks it down into three pillars: Trust = Performance + Process + Purpose.

### Performance, Process, Purpose

Performance covers the assistant's current and historical performance, reliability, predictability, capability, in concrete tasks: is it familiar with the developer's domain and task context, can it complete tasks within the expected timeframe, with the expected quality, and reproducibly across multiple uses?

Process describes how the assistant completes tasks: does it ask good questions, provide a detailed plan before implementation, do its results match the announced plan, does it follow the developer's best practices and instructions, respect their feedback, allow interruption at any time, and offer a well-designed permission-control mechanism with easy environment restoration?

Purpose measures the consistency between the tool's design intent and the developer's goals: does it produce hallucinations, does it guarantee data security and compliance, does it engage in malicious or deceptive operations, does it genuinely respect the developer's goals?

SECTION 5

A Trust-Improvement Model for AI-Powered Coding Assistants

Before discussing the trust-improvement model, we need to classify the assistant's potential erroneous behaviors using the SRK behavior model (Skill-Rule-Knowledge), so we can analyze trust-affecting behaviors in a targeted way.

### Classifying Behavioral Errors

Based on the SRK model, assistant errors fall into three categories. Slips (skill): the behavior is correct but poorly executed, flagging the need to install a dependency but generating the wrong install command, wanting to read command-line output but failing to. Rule-based errors: the behavior is incorrect because of unsuitable rules, writing code before tests instead of following TDD, piling logic into existing modules instead of creating new ones. Knowledge-based errors: the rules or knowledge needed to decide correctly are missing, generating outdated code for lack of the language's latest features, misunderstanding the project's terminology.

### The Improvement Model

To improve trust, first establish an enabling team to drive the whole process. Next, visualize the performance of both assistants and developers to precisely calibrate trust and self-confidence levels. Finally, organize knowledge-sharing initiatives, sessions, training, workshops, playbooks, at every level (company, department, team) so everyone quickly grasps the company's AI strategies, successful or failed use cases, and how the assistants work under the hood.

On that basis, adopt a trial-and-error approach: keep experimenting, collect failure cases, identify the behaviors that erode trust, and fix them. Each identified behavior is analyzed Ishikawa-diagram-style to trace the root cause: a high hallucination rate signals a purpose problem, not following TDD signals a process problem, the inability to read command-line output signals a performance problem.

Purpose problems belong to the business, since they relate to how the tool was acquired: a high hallucination rate may come from a contract that doesn't activate advanced models, a data leak from inadequate protection mechanisms. Solving them, updating the contract or switching tools, has the strongest impact on trust. Process problems belong to the team, since practices differ from team to team: not following TDD, generating code without explanations or without an upfront plan is fixed by filling in the missing skills, rules, or knowledge, with a moderate impact. Performance problems belong to the developer, since everyone faces different tasks: misunderstanding the project's tech stack, command-line plugin integration issues, generating oversized classes, the unit impact is smaller, but cumulative.

SECTION 6

Applying the Trust-Improvement Model in GitHub Copilot

### Copilot Instructions Template

Based on the trust-improvement model, we optimize GitHub Copilot's behavior along three dimensions, knowledge, rules, and skills, by designing a structured Copilot instructions template.

Knowledge addresses knowledge-based errors: when we discover one, we add the missing knowledge to this section (system architecture, coding guidelines, tech stacks).

Rules address rule-based errors: we add the rules applicable to the scenario (define problems before solving them, create plans before implementation, use TDD during implementation).

Skills address slips: we add the missing skills (problem definition, solution planning, TDD skills).

### Team Collaboration

This template is mainly maintained by the team's AI lead, who keeps GitHub Copilot's development process aligned with best practices: problem definition, planning, TDD, thinking out loud, running tests and commands.

The template is versioned in the code repository, so every member adopts best practices when developing with Copilot. That doesn't guarantee every task succeeds, it only increases the probability of success. For failed tasks, developers analyze the reasons and complete the template with the missing knowledge, rules, or skills, gradually turning it into the repository's own Copilot instructions.

AI leads track each repository's instructions, identify widely reused elements, and promote them into the shared template. Through this method, the team's trust in GitHub Copilot grows, and usage rates grow with it.

@cite:l-ia-n-est-pas-seulement-un-partenaire-de-codage

SECTION 7

Conclusion

An AI-powered coding assistant is like a new hire. Despite potentially strong capabilities, we still need to train them to align with our values, follow our best practices, and learn the required skills. Only once we fully trust them can we assign them important tasks.

Disclaimer: The statements and opinions expressed in this article are those of the author(s) and do not necessarily reflect the positions of Adservio.

FAQ

Frequently asked questions