AAAI

Association for the Advancement of Artificial Intelligence

Home > Proceedings / Proceedings of the International AAAI Conference on Web and Social Media, 15 > Vol. 15 (2021): Fifteenth International AAAI Conference on Web and Social Media

Media Cloud: Massive Open Source Collection of Global News on the Open Web

Authors

Hal Roberts,Rahul Bhargava,Linas Valiukas,Dennis Jen,Momin M. Malik,Cindy Sherman Bishop,Emily B. Ndulue,Aashka Dave,Justin Clark,Bruce Etling,Robert Faris,Anushka Shah,Jasmin Rubinovitz,Alexis Hope,Catherine D'Ignazio,Fernando Bermejo,Yochai Benkler,Ethan Zuckerman

Berkman Klein Center for Internet & Society, Harvard University Media Cloud,College of Arts, Media and Design, Northeastern University Media Cloud,Media Cloud,Media Cloud,Berkman Klein Center for Internet & Society, Harvard University,Media Cloud,Media Cloud,School of Information & Library Science, University of North Carolina at Chapel Hill,Berkman Klein Center for Internet & Society, Harvard University,Berkman Klein Center for Internet & Society, Harvard University Media Cloud,Shorenstein Center on Media, Politics and Public Policy, Harvard University Berkman Klein Center for Internet & Society, Harvard University Media Cloud,Civic Studios Media Lab, Massachusetts Institute of Technology,Media Lab, Massachusetts Institute of Technology,Media Lab, Massachusetts Institute of Technology,Department of Urban Studies and Planning, Massachusetts Institute of Technology,Media Cloud Berkman Klein Center for Internet & Society, Harvard University,Harvard Law School, Harvard University Berkman Klein Center for Internet & Society, Harvard University Media Cloud,College of Information & Computer Sciences, University of Massachusetts Amherst Berkman Klein Center for Internet & Society, Harvard University Media Cloud


Proceedings:

Vol. 15 (2021): Fifteenth International AAAI Conference on Web and Social Media

Volume

Issue:

Vol. 15 (2021): Fifteenth International AAAI Conference on Web and Social Media

Track:

Dataset Papers

Downloads:

Download PDF

Abstract:

We present the first full description of Media Cloud, an open source platform based on crawling hyperlink structure in operation for over 10 years, that for many uses will be the best way to collect data for studying the media ecosystem on the open web. We document the key choices behind what data Media Cloud collects and stores, how it processes and organizes these data, and its open API access as well as user-facing tools. We also highlight the strengths and limitations of the Media Cloud collection strategy compared to relevant alternatives. We give an overview two sample datasets generated using Media Cloud and discuss how researchers can use the platform to create their own datasets.

DOI:

10.1609/icwsm.v15i1.18127


ICWSM

Vol. 15 (2021): Fifteenth International AAAI Conference on Web and Social Media