Real-time model-based deduction has recently emerged as a vital component in Al’s tool box for developing highly autonomous reactive systems. Yet one of the current hurdles towards developing model-based reactive systems is the number of methods simultaneously employed, and their corresponding melange of programming and modeling languages. This paper offers an important step towards unification of reactive and model-based programming, providing the capability to monitor mixed hardware/software systems. We introduce RMPL, a rich modeling language that combines probabilistic, constraint-based modeling with reactive programming constructs, while offering a simple semantics in terms of hidden state Markov processes. We introduce probabilistic, hierarchical constraint automata, which allow Markov processes to be expressed in a compact representation that preserves the modularity of RMPL programs. Finally, a model-based executive, called RBurton is described that exploits this compact encoding to perform efficent simulation, belief state update and control sequence generation.