March 31, 2011 § Leave a comment
Posted on behalf of Julia Lane, National Science Foundation
Federal investments in science can play a vital role in job creation, innovation, and economic growth. Yet, although this role has been documented in a number of studies, there is an underdeveloped evidence basis to document the results of scientific investments (1). An increasing interest in the role of science and technology in the economy, along with the recognition that agencies lack the infrastructure to appropriately assess the results of their investments, is evidenced by recent memoranda released by the White House Office of Management and Budget (OMB) and Office of Science and Technology Policy (OSTP). These memoranda call on agencies to develop tools to better manage their research and assess the impact of their investments (2, 3). In order to achieve this, research agencies need better ways of linking inputs with outputs and outcomes.The STAR METRICS (Science and Technology for America’s Reinvestment: Measuring the Effects of Research on Innovation, Competitiveness and Science) program is a collaboration between science agencies and research institutions. The program has taken the first steps to build a system that can describe the effects of scientific research on an ongoing basis.
Creating a system that can describe the effects of scientific research on an ongoing basis is difficult. There is currently no data infrastructure that identifies the universe of individuals funded by federal science agencies. Science funding infrastructure has been based on identifying and managing high quality science, not on understanding its impact. The main sources of data on federal funding of research and development in the U.S., the Survey of Federal Funds for Research and Development (the federal funds survey) and the Survey of Federal Science and Engineering Support to Universities, Colleges, and Nonprofit Institutions, were designed to describe the types and levels of science investments, not their effects (4). Indeed, the Roadmap published by National Science and Technology Council (NSTC) Science of Science Policy Interagency Group in 2008, found that “current science and technology investment decisions are based on analyses that lack a strong theoretical and empirical basis” (5).
STAR METRICS Approach
In May 2010, a memorandum of understanding was signed between the National Science Foundation (NSF), the National Institutes of Health (NIH), and OSTP to partner in the creation of the STAR METRICS program. New partners being added include the Environmental Protection Agency (EPA) and the Department of Energy (DOE). The goal of the program is to create a database that will combine scientific investment data, initially from the NSF and the NIH, with data acquired from voluntarily participating research institutions. The approach is based on the STAR pilot project that was initiated in July 2009.
The STAR METRICS program consists of two phases. Phase I of the project documents the effect of science investments on employment. It uses existing administrative data from participating research institutions to account for the number of scientists, students and research staff on research organization payrolls that are supported by federal funds for scientific research. Data on subawards and contracts are also captured. In Phase II, STAR METRICS aims to build on this information to measure the effect of science funding on broader scientific, social, economic, and workforce outputs and outcomes.
- C. Macilwain, Nature 465, 682 (Jun 10, 2010).
- M-09-27 Memorandum for the Heads of Executive Departments and Agencies, Science and Technology Priorities for the FY2011 Budget. August 4, 2009.
- M-10-30 Memorandum for the Heads of Executive Departments and Agencies, Science and Technology Priorities for the FY2012 Budget. July 21, 2010.
- “Data on Federal Research and Development Investments: A Pathway to Modernization” (National Resources Council, Washington, D.C., 2010).
- “The Science of Science Policy: A Federal Research Roadmap” (National Science and Technology Council, Washington DC, 2008).