Interpreting Schematic Sketches Using Physical Reasoning

Sketch Understanding

Volume

Sketch Understanding

Contents

We describe a program that uses both geometric and physical reasoning to interpret schematic sketches of physical devices. The program takes freehand sketches of physical devices as input. It begins by recognizing the symbols used in the sketch. It then uses geometric and physics-based reasoning techniques to identify the meaning of each symbol within the context of the sketch. The physical reasoning relies on qualitative behavior models, simple qualitative models describing energy flow through physical components. Through the physical reasoning, the program is able to disambiguate the meanings of the individual symbols and produce a natural language (text) description of how the sketched device would operate.