Metareasoning in Modular Software Systems: On-the-Fly Configuration Using Reinforcement Learning with Rich Contextual Representations

Authors

  • Aditya Modi University of Michigan
  • Debadeepta Dey Microsoft Research
  • Alekh Agarwal Microsoft Research
  • Adith Swaminathan Microsoft Research
  • Besmira Nushi Microsoft Research
  • Sean Andrist Microsoft Research
  • Eric Horvitz Microsoft Research

DOI:

https://doi.org/10.1609/aaai.v34i04.5965

Abstract

Assemblies of modular subsystems are being pressed into service to perform sensing, reasoning, and decision making in high-stakes, time-critical tasks in areas such as transportation, healthcare, and industrial automation. We address the opportunity to maximize the utility of an overall computing system by employing reinforcement learning to guide the configuration of the set of interacting modules that comprise the system. The challenge of doing system-wide optimization is a combinatorial problem. Local attempts to boost the performance of a specific module by modifying its configuration often leads to losses in overall utility of the system's performance as the distribution of inputs to downstream modules changes drastically. We present metareasoning techniques which consider a rich representation of the input, monitor the state of the entire pipeline, and adjust the configuration of modules on-the-fly so as to maximize the utility of a system's operation. We show significant improvement in both real-world and synthetic pipelines across a variety of reinforcement learning techniques.

Downloads

Published

2020-04-03

How to Cite

Modi, A., Dey, D., Agarwal, A., Swaminathan, A., Nushi, B., Andrist, S., & Horvitz, E. (2020). Metareasoning in Modular Software Systems: On-the-Fly Configuration Using Reinforcement Learning with Rich Contextual Representations. Proceedings of the AAAI Conference on Artificial Intelligence, 34(04), 5207-5215. https://doi.org/10.1609/aaai.v34i04.5965

Issue

Section

AAAI Technical Track: Machine Learning