Participants in Fantasy Sports make a critical decision: selecting productive players for their fantasy team. The wellestablished Wisdom of Crowd effect can predict productive, rewarding players; popular, frequently selected players are potentially good choices. Previous performance data permits the identification of a subset of participants who collectively predict productive players. However, performance data may not always be available. Here we study the assembly of a small subset of the crowd a priori using another important crowd property: semantic diversity. We infer diversity from participants’ Twitter posts (tweets) that users voluntarily, and naturally provide as part of their reasoning. We propose the SmartCrowd framework to select a small, smart crowd using participants’ Twitter posts. SmartCrowd includes three steps: 1) characterize participants using their social media posts with summary word vectors, 2) cluster participants based on these vectors, and 3) sample participants from these clusters, maximizing multiple diversity measures to form final diverse crowds. We evaluated our approach to diversity characterization for the Fantasy Premier League (FPL) captain prediction problem, in which participants predict a successful weekly captain among a set of soccer players. Empirical evaluation shows that SmartCrowd generates diverse crowds outperforming random crowds, 93% of individual participants, and crowds consisting of the top 10%, 20% experts identified from previous performance data. We provide converging evidence that social media based diversity supports the sampling of smarter crowds that collectively predict productive players. These results have implications for other domains, such as economics and geopolitical forecasting, that benefit from aggregated judgments.