The Challenges and Progress of AI Agents in Shopping

The Challenges and Progress of AI Agents in Shopping

As personal AI agents like Muse or Claude become integral to daily life, they are increasingly tasked with shopping. However, these agents are facing challenges in executing shopping tasks effectively.

A groundbreaking study by Visa tested 22 different AI agents to purchase three common items: a TV, a laptop, and a hair appliance. While some agents outperformed others, the overall success rate was only about 57%. Details of this study are available on a new website by Visa, showcasing the results from 15 separate shopping expeditions conducted by each agent.

Advanced AI models performed the best. Google’s Gemini’s Aragon 4 version and Meta’s Making Wonder version achieved the highest success rates, successfully completing purchases 82% of the time. A Claude agent came third with a 76% success rate, and an OpenAI agent ranked sixth at 64% success.

A visual diagram by Visa displays the performance of all 22 models, with the vertical axis reflecting success rate and the horizontal axis showing the cost to run each inquiry. Notably, the five most expensive models had an average success rate of 67% compared to the cheapest five at 50%.

The difficulties AI agents face often stem from the complexity of purchase motivations. Consumers frequently lack clear information on what they desire. Visa’s experiment reflected this by requiring TV shoppers, for instance, to identify specific attributes like compatibility with gaming consoles, a 30-day return policy, and Dolby Vision support.

Agents needed to extract missing information from shoppers to achieve success, which involved buying a TV meeting the shopper’s criteria. Many agents either bought incorrect products or failed to suggest suitable options.

“We’re not there yet from a trust or technology perspective. But we’ll soon get into a world where consumers can trust and delegate,”

commented Rubail Birwadker, Visa’s Global Head of Growth, to Fortune. He remarked that agent-driven shopping, while in its early stages, is advancing quickly. Visa’s experiment focused on product discovery as a core component of agentic commerce, which retailers and AI firms are keen to optimize.

For AI agents to enter mainstream shopping, smooth and secure payments are essential. Visa and Mastercard are already addressing these challenges by educating merchants and developing tools for agentic commerce. Additionally, unanswered legal questions persist about liability for rogue purchases or fraud by AI agents. Birwadker believes these issues will be resolved over time, akin to the evolution of ecommerce, though agent-based shopping will differ greatly from current online shopping.

Visa’s announcement of the agent shopping experiment’s results includes launching a website for consumers to interact with the tested agents.

Leave a Reply

Your email address will not be published. Required fields are marked *