Evacuations have a significant impact on saving human lives during hurricanes. However, as a complex dynamic process, it is typically difficult to know individual evacuation decisions in real-time. Since a large amount of information is continuously posted through social media platforms, we can use them to understand individual evacuation behavior. In this paper, we collect tweets during Hurricane Irma in 2017 and train a text classifier in an active learning way to distinguish tweets expressing positive evacuation decisions from both negative and irrelevant ones. Additionally, we perform a demographic analysis and content clustering to investigate the potential causes and correlates of evacuation decisions. The results can be used to help inform planning strategies of emergency response agencies.