Where and Why is My Bot Failing? A Visual Analytics Approach for Investigating Failures in Chatbot Conversation Flows

Abstract

The ongoing coronavirus pandemic has accelerated the adoption of AI-powered task-oriented chatbots by businesses and healthcare organizations. Despite advancements in chatbot platforms, implementing a successful and effective bot is still challenging and requires a lot of manual work. There is a strong need for tools to help conversation analysts quickly identify problem areas and, consequently, introduce changes to chatbot design. We present a visual analytics approach and tool for conversation analysts to identify and assess common patterns of failure in conversation flows. We focus on two key capabilities: path flow analysis and root cause analysis. Interim evaluation results from applying our tool in real-world customer production projects are presented.