Generating User Interfaces for Pen-based Computers

Contents

We provide an overview of the Penguins project. The aim of this project is to develop tools that facilitate the development of software for pen-based graphics editors. The project is based on the intelligent pen and paper metaphor for human-computer interaction. In this metaphor, the user communicates with the computer using an application specific visual language composed of handwritten text and diagrams. As the diagram is drawn with a pen, the underlying graphic editor parses the diagram, performing error correction and collecting geometric constraints which capture the relationships between diagram components. During manipulation these constraints are maintained by the editor in order to preserve the semantics of the diagram. The Penguins system contains tools which, given a grammatical specification of a visual language, automatically construct a core user interface for that visual language which embodies the intelligent pen and paper metaphor. This core user interface consists of a tokenizer, constraint solver, layout controller and constraint-based graphics editor together with an incremental parser for the specified visual language. The specification language is based on constraint multiset grammars.