TY - JOUR AU - Gundersen, Odd Erik AU - Gil, Yolanda AU - Aha, David W. PY - 2018/09/28 Y2 - 2024/03/29 TI - On Reproducible AI: Towards Reproducible Research, Open Science, and Digital Scholarship in AI Publications JF - AI Magazine JA - AIMag VL - 39 IS - 3 SE - Articles DO - 10.1609/aimag.v39i3.2816 UR - https://ojs.aaai.org/aimagazine/index.php/aimagazine/article/view/2816 SP - 56-68 AB - Background: Science is experiencing a reproducibility crisis. Artificial intelligence research is not an exception. Objective: To give practical and pragmatic recommendations for how to document AI research so that the results are reproducible. Method: Our analysis of the literature shows that AI publications fall short of providing enough documentation to facilitate reproducibility. Our suggested best practices are based on a framework for reproducibility and recommendations given for other disciplines. Results: We have made an author checklist based on our investigation and provided examples for how every item in the checklist can be documented. Conclusion: We encourage reviewers to use the suggested best practices and author checklist when reviewing submissions for AAAI publications and future AAAI conferences. ER -