This paper describes an approach for creating detailed full coverage labellings of human activity. Our goal is to create global maps of physical positions labelled with a distribution over the most likely place name and most likely activity. We ground our ontology of labels as: the term that a person would want to display to someone before they initiate a communication. Rather than compiling a canonical list of possible labels, we piggyback the label data collection in a situated communicative exchange. Using ideas inspired by image segmentation and extended to support our goals we propose machine learning techniques for smoothing distributions across gaps in existing data.