Automatic call quality monitoring using cost-sensitive classification

Abstract

In this paper, we propose advanced text analytics and costsensitive classification-based approaches for call quality monitoring and show that automatic quality monitoring with ASR transcripts can be achieved with a high accuracy. Our system analyzes ASR transcripts and determines if a call is a good call or a bad call. The set of features were identified through analysis of a large number of human monitoring results, which aim to estimate agent's attitude and customer's sentiment during the call. To enhance the accuracy of feature extraction, we apply various techniques to improve the quality of transcribed calls, such as sentence boundary detection and disfluency removal. We further note that quality monitoring has skewed class distribution and unequal classification error costs, and thus apply cost sensitive classification algorithms. Validation on 386 customer calls confirms the benefits of our approach. A SVMbased method produces a classification accuracy of 83.16% and 67.66% in F 1Score for identifying bad calls, which is promising. This system can therefore be used to conduct initial monitoring of all the calls in a contact center and to select calls that require human monitoring. Copyright © 2011 ISCA.