This paper introduces a technique for planning in hierarchical belief spaces and demonstrates the idea in an autonomous assembly task. The objective is to effectively propagate belief across multiple levels of abstraction to make control decisions that expose useful state information, manage uncertainty and risk, and actively satisfy task specifications. This approach is demonstrated by performing multiple instances of a simple class of assembly tasks using the uBot-6 mobile manipulator with a two-level hierarchy that manages uncertainty over objects and assembly geometry. The result is a method for composing a sequence of manual interactions that causes changes to the environment and exposes new information in order to support belief that the task is satisfied. This approach has the added virtue that it provides a natural way to accomplish tasks while simultaneously suppressing errors and mitigating risk. We compare performance in an example assembly task against a baseline hybrid approach that combines uncertainty management with a symbolic planner and show statistically significant improvement in task outcomes. In additional demonstrations that challenge the system we highlight useful artifacts of the approach---risk management and autonomous recovery from unexpected events.