Computer-based emotion recognition is an emerging field with envisioned applications ranging from customer satisfaction evaluation to human-machine interaction. In this paper we present a general framework for continuous emotion inference based on Bayesian biometric data fusion and the circumplex model of affect. We apply this framework to the field of assistive robotics focused on elderly and impaired people who require a wheelchair for mobility purposes. The objective is to provide an emotion-based safety layer that complements the classical collision avoidance approaches typically included in these systems. In many real-case applications the calculation of the emotional valence is not feasible, and therefore we also present here a promising novel context-based alternative currently under development.