Generating Cross-References for Multimedia Explanation

Book One

Volume

Proceedings of the AAAI Conference on Artificial Intelligence, 10

Explanation and Tutoring

When explanations include multiple media, such as text and illustrations, a reference to an object can be made through a combination of media. We call part of a presentation that references material elsewhere a cross-reference. We are concerned here with how textual expressions can refer to parts of accompanying illustrations. The illustration to which a cross-reference refers should also satisfy the specific goal of identifying an object for the user. Thus, producing an effective cross-reference not only involves text generation, but may also entail modifying or replacing an existing illustration and in some cases, generating an illustration where previously none was needed. In this paper, we describe the different types of cross-references that COMET (Coordinated Multimedia Explanation Testbed) generates and show the roles that both its text and graphics generators play in this process.