A truly collaborative human-robot interaction framework should allow the participating agents to assume and relinquish initiative depending upon their own capabilities and their understanding of the environment. The goal of this work is to define and develop a mixed-initiative human-robot collaborative architecture in which affect-based sensing plays a critical role in initiative switching. Affect-based sensing implies that the robot detects the human’s emotional state in order to determine which actions to pursue. We have conducted an extensive literature review of Mixed-Initiative Interaction that has provided a basis for our architectural development. In particular, we are applying Riley’s (Riley 1989) general model of mixed-initiative interaction to our architecture development. We have developed a preliminary architecture and are now collecting affect-based participant data that will be used to test the system. The purpose of this paper is to provide an overview of Riley’s model, its application in our development, present our preliminary architecture, and present the affect-based interaction constraints that affect the architecture development.