When a chatbot invents a price, you are the one who pays

That was false. The airline's actual policy excluded any retroactive claim. Air Canada refused to refund him.

The case went to British Columbia's civil resolution tribunal. The airline argued, among other things, that the chatbot should be treated as a separate entity responsible for its own statements. The tribunal dismissed that argument and held that Air Canada was responsible for all the information on its website, whether it came from a static page or a conversational assistant. The decision, issued on 14 February 2024, ordered the airline to refund the fare difference, around 650 Canadian dollars, rising to a little over 800 with interest and fees.

The amount is trivial. The principle is not: what your assistant tells a customer, you are the one telling them.

Why artificial intelligence invents

It is neither a bug nor an accident. It is a consequence of how these systems are trained and evaluated.

Researchers from OpenAI and Georgia Tech set it out in a paper published in September 2025, titled Why Language Models Hallucinate. Their thesis fits in one sentence: language models invent because training and evaluation procedures reward guessing over admitting uncertainty.

The image they use is telling: like a student facing a hard question, a model would rather attempt an answer than leave the page blank, because a wrong answer and no answer score the same, while a lucky right answer earns points.

Apply that to your shop. A buyer asks the price of an item your assistant does not know. The model has no natural mechanism for saying “I do not know”. It produces a plausible price. And a plausible price is precisely a price nobody spots.

The three ways this costs you money

The invented price. A customer read 180 dirhams, you deliver at 240. Either you honour the quoted price and sell at a loss, or you refuse and lose the sale, the customer, and probably a public review. In a country where people pay on delivery, that scene plays out at the door, with a courier waiting.

The promised lead time. “You will be delivered tomorrow” when you deliver to Agadir in four days. Refusal on delivery is all but certain, and you pay for the trip out and back.

The claimed feature. The assistant says the bag holds ten kilos, when your product page says nothing of the sort. The customer buys it for that. They send it back for that.

In all three cases the assistant served nobody: it turned a hesitant buyer into an unhappy customer, which costs more than a missed sale.

What Moroccan law says

Two texts deserve your attention.

Law 31-08 on consumer protection places a general information duty on the seller: the consumer must know the price, the terms of sale and the delivery arrangements. It also requires the price to be shown inclusive of all taxes. These obligations draw no distinction based on the channel through which the information is communicated. A price announced by your assistant is a price you announced.

In addition, the National Commission for the Control of Personal Data Protection has been working since 2025 on framing AI-based processing, and asked in August 2026 that any AI-generated content be clearly labelled as such, relying on the Penal Code provisions on artificial montages and content. The practical arrangements for that labelling, its format and its placement, remain to be specified, and the context of that statement was electoral. But the direction is clear, and it matches what is taking shape elsewhere: in California, a law has for several years required telling someone they are talking to a program; the European Union is moving the same way.

The practical conclusion is simple: tell your customers they are talking to an assistant. Not only is that where the law is heading, but experience shows it bothers nobody. What the buyer remembers is having got a correct answer at 10 pm.

The Klarna lesson, which is not the one people think

In February 2024, the Swedish payments company Klarna announced that its conversational assistant was doing the work of seven hundred agents, had handled 2.3 million conversations in a month, and should improve its result by around forty million dollars. The announcement travelled the world.

In May 2025, its chief executive Sebastian Siemiatkowski publicly reversed course. He explained that cost had weighed too heavily in the trade-offs, that quality had suffered, and that the future lay in real investment in the quality of human support. Klarna started hiring again.

Many read that “AI had failed at Klarna”. That is not what happened, and the company kept publishing productivity figures tied to its assistant. What failed was a scope: handing the machine situations that needed a human. The lesson is not to give up, it is to draw the line.

The six questions to ask any assistant supplier

If you are considering equipping your company, these questions are enough to sort the field.

1. 1Where does the information the assistant gives come from? The only good answer is: from my data, and from nothing else.
2. 2What happens when it does not know? The only good answer is: it says so and hands over. An assistant that can never say “I do not know” will one day say anything.
3. 3Are the figures shown verified before they appear? A price, a stock level, a total must never be written by the model: they must be read back from the database and inserted as they are.
4. 4Can I see what it answered? Every conversation, readable, correctable.
5. 5Can I correct it without going through you? If every adjustment needs a developer, you have not bought a product.
6. 6Does it introduce itself as an assistant? And if so, how, where, in which language.

What to take away

An assistant that talks to your customers commits your name, your price and your liability. That is not a reason to do without one: it is a reason to demand that it know how to keep quiet.

The technology for that exists. You constrain the answer to the company's data, you check every number before displaying it, and you explicitly allow the machine to answer that it does not know. Those are design choices, not options.