On September 28, 2023, the U.S. National Science Foundation and the Institute for Progress (IFP), with its Metascience Working Group, announced an agreement to design and execute experiments to explore how the agency funds and supports research and innovation. These experiments are aimed to solve the problems of low reviewing efficiency and conservative preference funding. Based on this, the turnaround time on final decision will be shortened, the influence of the project will be improved, and the new pathways for researchers will be developed. IFP will also help develop tests of different mechanisms of interest to NSF staff, as well as potential analyses of NSF data to measure the outputs of NSF funding.

The NSF’s rigorous merit review process has long been regarded as the gold standard for vetting and funding research. However, since its inception in the 1950s, emergent circumstances — such as the significant growth in overall population of principal investigators (PIs) — have introduced a slew of challenges and inefficiencies to the traditional peer-review grantmaking process. The "CHIPS and Science Act of 2022" charged NSF with considering alternatives to, or experimentation within, NSF's traditional processes for receiving, reviewing, and funding research and innovation. Under this agreement, IFP will design and implement one or more economic and social experiments, statistical tests of a hypothesis.

The tax on research productivity as PIs submit about 2.3 proposals for every award they receive and spend an average of 116 hours grant-writing per NSF proposal (i.e., “grantsmanship”), corresponding to a staggering loss of nearly 45% of researcher time; the orientation of grantsmanship towards incremental research with the highest likelihood of surviving highly-competitive, consensus-driven, and points-based review (versus riskier, novel, or investigator-driven research); rating bias against interdisciplinary research or previously unfunded researchers as well as reviewer fatigue. The result of such inefficiencies is unsettling: as fewer applicants are funded as a percentage of the increasing pool, some economic analysis suggests that the value of the science that researchers forgo for grantsmanship may exceed the value of the science that the funding program supports. At the same time, because of the complicated reviewing processes and long delays in funding, researchers may miss out on research opportunities.

Back on August 28, the economist of IFP had published an article on the Washington Post to give recommends for these issues. First, Congress should give the NSF the authority to quickly reject low-quality proposals. Congress requires the NSF to fully review all applications that are submitted. Because many of these are low-quality, this inflates workloads and presumably lengthens the time it takes to consider more promising proposals. Second, the NSF should give researchers greater ability to change course in their work when needed by targeting a larger share of their funds toward individual scientists rather than specific projects.

 "The NSF merit review process is the gold standard for evaluating the most promising research," said NSF Director Sethuraman Panchanathan. "We know there are always ways to improve, and these 'science of science' experiments will allow us to test new mechanisms to accelerate decision making. The results will enhance how NSF makes investments by reducing turnaround time on final decisions, allowing greater flexibility in pivoting projects toward maximal impacts, and developing new pathways for researchers."

Over the course of the collaboration, IFP will consult with NSF on the current funding mechanisms and review processes used by the agency to decide which proposals to award and will propose tests for different ways to fund high-risk and high-reward proposals. IFP will then help develop tests of different mechanisms of interest to NSF staff, as well as potential analyses of NSF data to measure the outputs of NSF funding.