We segment the trajectory of a moving object into piecewise smooth motion intervals separated by motion boundaries. Motion boundaries are classified into various types, including starts, stops, pauses, and discontinuous changes of motion due to force impulses. We localize and classify motion boundaries by fitting a mixture of two polynomials near the boundary. Given a classification of motion boundaries, we use naive physical rules to infer a set of changing contact relationships which explain the observed motion. We show segmentation and classification results for several image sequences of a basketball undergoing gravitational and nongravitational motion.