Autonomous mobile robots need to detect potential failures reliably and react appropriately. Due to uncertainties about the robots and their environment, it is extremely difficult to design reliable systems from the start. Instead, we advocate the structured control methodology, in which one starts with plans that work in nominal situations, and then incrementally adds reactive behaviors to handle previously unanticipated situations. We have developed the Task Control Architecture to facilitate this methodology by enabling monitors and exception handlers to be added to existing hierarchical plans. This paper details the application of this methodology to a walking rover and an indoor office robot. In both cases, robot systems were produced that can autonomously traverse long distances in obstacle-filled environments.