Evidence is presented showing that bottom-up grouping of image features is usually prerequisite to the recognition and in interpretation of images. We describe three functions of these groupings: 1) segmentation, 2) three-dimensional in-terpretation, and 3) stable descriptions for accessing object models. Several principles are hypothesized for determin-ing which image relations should be formed: relations are significant to the extent that they are unlikely to have arisen by accident from the surrounding distribution of features, relations can only be formed where there are few alterna-tives within the same proximity, and relations must be based on properties which are invariant over a range of imaging conditions. Using these principles we develop an algorithm for curve segmentation which detects significant structure at multiple resolutions, including the linking of segments on the basis of curvilinearity. The algorithm is able to detect structures vhich no single-resolution algorithm could detect. Its performance is demonstrated on synthetic and natural image data.