One of the pathways by which the scientific community confirms the validity of a new scientific discovery is by repeating the research that produced it. When a scientific effort fails to independently confirm the computations or results of a previous study, some fear that it may be a symptom of a lack of rigor in science, while others argue that such an observed inconsistency can be an important precursor to new discovery. Concerns about reproducibility and replicability have been expressed in both scientific and popular media. As these concerns came to light, Congress requested that the National Academies of Sciences, Engineering, and Medicine conduct a study to assess the extent of issues related to reproducibility and replicability and to offer recommendations for improving rigor and transparency in scientific research. Reproducibility and Replicability in Science defines reproducibility and replicability and examines the factors that may lead to non-reproducibility and non-replicability in research. Unlike the typical expectation of reproducibility between two computations, expectations about replicability are more nuanced, and in some cases a lack of replicability can aid the process of scientific discovery. This report provides recommendations to researchers, academic institutions, journals, and funders on steps they can take to improve reproducibility and replicability in science.

One of the pathways by which the scientific community confirms the validity of a new scientific discovery is by repeating the research that produced it. When a scientific effort fails to independently confirm the computations or results of a previous study, some fear that it may be a symptom of a lack of rigor in science, while others argue that such an observed inconsistency can be an important precursor to new discovery. Concerns about reproducibility and replicability have been expressed in both scientific and popular media. As these concerns came to light, Congress requested that the National Academies of Sciences, Engineering, and Medicine conduct a study to assess the extent of issues related to reproducibility and replicability and to offer recommendations for improving rigor and transparency in scientific research. Reproducibility and Replicability in Science defines reproducibility and replicability and examines the factors that may lead to non-reproducibility and non-replicability in research. Unlike the typical expectation of reproducibility between two computations, expectations about replicability are more nuanced, and in some cases a lack of replicability can aid the process of scientific discovery. This report provides recommendations to researchers, academic institutions, journals, and funders on steps they can take to improve reproducibility and replicability in science.

Recent events have vividly underscored the societal importance of science, yet the majority of the public are unaware that a large proportion of published scientific results are simply wrong. The Problem with Science is an exploration of the manifestations and causes of this scientific crisis, accompanied by a description of the very promising corrective initiatives largely developed over the past decade to stem the spate of irreproducible results that have come to characterize many of our sciences. More importantly, Dr. R. Barker Bausell has designed it to provide guidance to practicing and aspiring scientists regarding how (a) to change the way in which science has come to be both conducted and reported in order to avoid producing false positive, irreproducible results in their own work and (b) to change those institutional practices (primarily but not exclusively involving the traditional journal publishing process and the academic reward system) that have unwittingly contributed to the present crisis. There is a need for change in the scientific culture itself. A culture which prioritizes conducting research correctly in order to get things right rather than simply getting it published.

Questions about the reproducibility of scientific research have been raised in numerous settings and have gained visibility through several high-profile journal and popular press articles. Quantitative issues contributing to reproducibility challenges have been considered (including improper data measurement and analysis, inadequate statistical expertise, and incomplete data, among others), but there is no clear consensus on how best to approach or to minimize these problems. A lack of reproducibility of scientific results has created some distrust in scientific findings among the general public, scientists, funding agencies, and industries. While studies fail for a variety of reasons, many factors contribute to the lack of perfect reproducibility, including insufficient training in experimental design, misaligned incentives for publication and the implications for university tenure, intentional manipulation, poor data management and analysis, and inadequate instances of statistical inference. The workshop summarized in this report was designed not to address the social and experimental challenges but instead to focus on the latter issues of improper data management and analysis, inadequate statistical expertise, incomplete data, and difficulties applying sound statistic inference to the available data. Many efforts have emerged over recent years to draw attention to and improve reproducibility of scientific work. This report uniquely focuses on the statistical perspective of three issues: the extent of reproducibility, the causes of reproducibility failures, and the potential remedies for these failures.

This open access book presents a ground-breaking approach to developing micro-foundations for demography and migration studies. It offers a unique and novel methodology for creating empirically grounded agent-based models of international migration – one of the most uncertain population processes and a top-priority policy area. The book discusses in detail the process of building a simulation model of migration, based on a population of intelligent, cognitive agents, their networks and institutions, all interacting with one another. The proposed model-based approach integrates behavioural and social theory with formal modelling, by embedding the interdisciplinary modelling process within a wider inductive framework based on the Bayesian statistical reasoning. Principles of uncertainty quantification are used to devise innovative computer-based simulations, and to learn about modelling the simulated individuals and the way they make decisions. The identified knowledge gaps are subsequently filled with information from dedicated laboratory experiments on cognitive aspects of human decision-making under uncertainty. In this way, the models are built iteratively, from the bottom up, filling an important epistemological gap in migration studies, and social sciences more broadly.

Showcasing methodological rigour and state-of-the-art methods as hallmarks of modern international business (IB) research, this book offers a collection of the most relevant and highly cited research methods articles from the Journal of International Business Studies (JIBS). Each piece is accompanied by a new Commentary written by experts in the field; some also include Further Reflections by the original authors. Encompassing both qualitative and quantitative approaches, this comprehensive volume explores research design, testing and reporting, as well as specific methodological issues such as endogeneity, common method variance, and theorising from case studies. With recommendations for best practices relating to interaction effects, hypothesis testing, and replicability, this book is a unique and up-to-date reference source on the latest research methods and practices in international business. The book will also be essential reading for those studying any sub-discipline of IB research, including international economics, entrepreneurship, finance, management and marketing.

In computational science, reproducibility requires that researchers make code and data available to others so that the data can be analyzed in a similar manner as in the original publication. Code must be available to be distributed, data must be accessible in a readable format, and a platform must be available for widely distributing the data and code. In addition, both data and code need to be licensed permissively enough so that others can reproduce the work without a substantial legal burden. Implementing Reproducible Research covers many of the elements necessary for conducting and distributing reproducible research. It explains how to accurately reproduce a scientific result. Divided into three parts, the book discusses the tools, practices, and dissemination platforms for ensuring reproducibility in computational science. It describes: Computational tools, such as Sweave, knitr, VisTrails, Sumatra, CDE, and the Declaratron system Open source practices, good programming practices, trends in open science, and the role of cloud computing in reproducible research Software and methodological platforms, including open source software packages, RunMyCode platform, and open access journals Each part presents contributions from leaders who have developed software and other products that have advanced the field. Supplementary material is available at www.ImplementingRR.org.