← All posts

November 7, 2026 · 2 min read

AI shopping agents got it right 57% of the time. How often does yours?

A new Visa benchmark tested 22 AI shopping agents and found an overall success rate of about 57%, with the best results going to agents that asked about hidden requirements. Here is how a small business can test its own AI agent before trusting it with real work.

If you gave your AI agent the same task three times, would it get it right all three times? Have you ever checked?

Visa just ran a test that made me ask myself this question. It gave 22 AI shopping agents the job of buying products like a TV, a laptop, and a hair appliance for a customer, and ran each agent through 15 shopping tasks.

Here is what stood out to me:

  • The overall success rate was about 57%.
  • 54% of the failed runs still ended in a purchase.
  • Agents that uncovered the most hidden requirements passed 70.2% of the time. Those that uncovered the fewest passed 44.4%.
  • Across three attempts at the same scenario, every attempt passed only 34.2% of the time.

The customer in each test did not share every need up front, just like real people. The agents that did well were the ones that asked good questions before acting. Even Visa said, "We're not there yet from a trust or technology perspective."

Why this matters for a small business

You may not use a shopping agent, but the lesson fits any AI agent that books meetings, answers client questions, or drafts proposals. An agent that acts before it understands can look busy and still get the job wrong. And one good result does not mean the next one will be good too.

This is where a person in the loop matters. Automate the process, not the relationship. Let the agent do the steps, and keep a human checking that the steps match what the client really needs.

What I would do this week

  • Pick one task your AI agent handles and run it three times with the same request. Compare the results.
  • Add a simple rule to its instructions: ask clarifying questions before taking any action.
  • Keep a person approving anything that spends money or reaches a client until the results are steady.

My takeaway: A good AI agent is not the one that acts fastest. It is the one that asks first, gets it right every time, and leaves the final word with a person.

Sources

  1. AI agents went shopping for TVs and laptops—and messed up nearly 50% of the time, study shows · Fortune (November 5, 2026)
  2. Visa benchmark tests how well AI agents handle shopping tasks · The Paypers (November 6, 2026), reporting on Visa's own benchmark research