AI Trust in Business Processes: The Need for Process-Aware Explanations

Business processes underpin a large number of enterprise operations including processing loan applications, managing invoices, and insurance claims. The business process management (BPM) industry is expected to grow at approximately 16 Billion dollar by 2023. There is a large opportunity for infusing AI to reduce cost or provide better customer experience with a $15.7 trillion “potential contribution to the global economy by 2030”. To this end, the BPM literature is rich in machine learning solutions including unsupervised learning to gain insights on clusters of process traces, classification models to predict the outcomes, duration, or paths of partial process traces, extracting business process from documents, and models to recommend how to optimize a business process or navigate decision points. More recently, deep learning models including those from the NLP domain have been applied to process predictions.Unfortunately, very little of these innovations have been applied and adopted by enterprise companies. We assert that a large reason for the lack of adoption of AI models in BPM is that business users are risk-averse and do not implicitly trust AI models. There has, unfortunately, been little attention paid to explaining model predictions to business users with process context. We challenge the BPM community to build on the AI interpretability literature, and the AI Trust community to understand what it means to take advantage of business process artifacts in order to provide business level explanations.