Escalation prediction using feature engineering: addressing support ticket escalations within IBM’s ecosystem
Date
2017-08-28
Authors
Montgomery, Lloyd Robert Frank
Journal Title
Journal ISSN
Volume Title
Publisher
Abstract
Large software organizations handle many customer support issues every day in the form of bug reports, feature requests, and general misunderstandings as submitted by customers. Strategies to gather, analyze, and negotiate requirements are comple- mented by efforts to manage customer input after products have been deployed. For the latter, support tickets are key in allowing customers to submit their issues, bug re- ports, and feature requests. Whenever insufficient attention is given to support issues, there is a chance customers will escalate their issues, and escalation to management is time-consuming and expensive, especially for large organizations managing hundreds of customers and thousands of support tickets. This thesis provides a step towards simplifying the job for support analysts and managers, particularly in predicting the risk of escalating support tickets. In a field study at our large industrial partner, IBM, a design science methodology was employed to characterize the support process and data available to IBM analysts in managing escalations. Through iterative cycles of design and evaluation, support analysts’ expert knowledge about their customers was translated into features of a support ticket model to be implemented into a Ma- chine Learning model to predict support ticket escalations. The Machine Learning model was trained and evaluated on over 2.5 million support tickets and 10,000 escalations, obtaining a recall of 79.9% and an 80.8% reduction in the workload for support analysts looking to identify support tickets at risk of escalation. Further on- site evaluations were conducted through a tool developed to implement the Machine Learning techniques in industry, deployed during weekly support-ticket-management meetings. The features developed in the Support Ticket Model are designed to serve as a starting place for organizations interested in implementing the model to predict support ticket escalations, and for future researchers to build on to advance research in Escalation Prediction.
Description
Keywords
Customer Relationship Management, Empirical Study, Design-Science Methodology, Machine Learning, Escalation Prediction, Support Ticket, IBM