Study Finds AI Chatbots Make Twice as Many Errors in Extended Conversations.

How AI Chatbots Impact Communication Quality

According to TSN.ua: New research from Microsoft Research and Salesforce reveals a significant flaw in popular AI chatbots: their error rate more than doubles during long conversations. This decline in performance is attributed to two key issues: 'hallucinations,' where the AI generates false information, and 'response inflation,' which degrades the overall quality of user interaction. These findings are crucial as chatbots become primary tools for customer service and information retrieval.

An analysis of over 200,000 chatbot conversations showed that while models successfully answer single queries about 90% of the time, their performance plummets to just 65% accuracy in extended dialogues. This demonstrates a clear correlation: the longer an interaction continues, the more likely the AI is to produce an incorrect or misleading response.

The Problem of Inflated Responses and Its Effects

The study also identified that in multi-turn dialogues, AI responses become 20% to 300% longer. This 'response inflation' can overwhelm users, making information harder to digest and increasing the potential for confusion and misunderstanding.

• Among popular chatbots, ChatGPT dominates the global market, holding over 80% of the user base.
• Its main competitors, such as Perplexity and Google Gemini, collectively account for just 15% of users.

Consequently, the research underscores an urgent need to refine chatbot algorithms to ensure greater accuracy and usability in sustained communications. This improvement represents a vital step in the evolution of AI technologies that are increasingly woven into users' daily lives.

Given the immense popularity of chatbots, led by ChatGPT, enhancing their long-conversation reliability could profoundly impact user experience and communication efficiency across diverse sectors, including business and customer support.