Previously, we have introduced an anticipatory robot that could generate a cognitive map while simultaneously localizing itself relative to it. Inspired by recent hippocampal research, there, we demonstrated that the robot could exhibit anticipatory behavior if episodic memories that encode both spatial/nonspatial stimuli and behavioral actions are adequately utilized. In this paper, we will propose various ways to improve the previous method. For example, to address the computational complexity problem observed in the previous method, we will incorporate a new internal state variable inspired by the somatic marker hypothesis (a biological premise that speculates the role of emotional responses in our brain in terms of memory and decision making). Furthermore, we conjecture that this framework for anticipation can be even extended farther for a robot to deal with a novel situation (i.e., improvisation). We will discuss the concept of our improvisational robot in terms of anticipatory failures and their possible solutions.