Intelligent Multimedia Interfaces
Multimedia communication is ubiquitous in daily life. When we converse with one another, we utilize a wide array of media to interact including spoken language, gestures, and drawings. We exploit multiple human sensory systems or modes of communication including vision, audition, and taction. Some media and modes of communication are more efficient or effective than others for certain tasks, users, or contexts (e.g., the use of speech to control devices in hand and eyes-busy contexts, the use of maps to convey terrain and cartographic information). Whereas humans have a natural facility for managing and exploiting multiple input and output media, computers do not. Consequently, providing machines with the ability to interpret multimedia input and generate multimedia output would be a valuable facility for a number of key applications such as information retrieval and analysis, training, and decision support. This book focuses specifically on those intelligent interfaces that exploit multiple media and modes to facilitate human-computer communication. This edited collection will be of interest to researchers and practitioners in computer science, artificial intelligence, human computer interaction, cognitive science, and graphic design.
This collection includes contributions from the United States, Canada, Germany, and Italy and is organized organized into three sections: Automated Presentation Design; Intelligent Multimedia Interfaces; and Architectural and Theoretical Issues. The chapters in the first section focus on methods for the automatic design of multimedia presentations. Multimedia design involves a number of complex issues addressed by these papers including temporal coordination of multiple media, the relationship of textual and graphical generation, automatic design of graphics, and modality selection (e.g., realizing language as text or speech.) The chapters in the second section report on several investigations into systems that integrate multimedia input and generate coordinated multimedia output. These prototypes point the way to possible future systems that will enhance human-computer interaction. A final section considers knowledge sources and processes required for processing multiple media. These include the need to represent and reason about models of tasks and information, media, the user, and the discourse context. While research in this entire area is still in its formative stages, the individual contributors and I hope that this initial collection will help foster the scientific interchange and motivate necessary research to solve many of the remaining fundamental problems.