A Research-Based Understanding and Approach to Broader Impacts

At some point in time, if we ever want to move past, “hear-say”, when it comes to broader impacts, we have to start treating broader impacts just like any other field and profession.

This means to hold ourselves accountable to a professional standard and stop with the anecdotal evidence and look at all of the data/facts and conduct rigorous research.

Once we begin to do those things we can then start to develop appropriate answers to questions that have been asked concerning broader impacts. I guess those are my two cents.

If you are interested in reading more about research and evidence-based approach to broader impacts click the link below.

Moving Towards A Research-Based Understanding and Approach to Broader Impacts

Seventeen (17) Things to Consider When Starting a Broader Impacts Organization

When considering or starting a broader impacts organization on your campus it is important to understand that context matters. The research-based scholarly definition of broader impacts is “a process with people/stakeholders to achieve a societal benefit in a finite time that is measured. This can be through one’s teaching, research, service, and occupation. There can be broader impacts of almost anything. If done appropriately broader impacts can lead to sustainable positive impacts” [1] [2] [3] [4]. Understanding this as it relates to broader impacts is also important – whether you are reshaping an old organization to align with a broader impacts concept; starting an office or organization focused on the National Science Foundation (NSF) Broader Impacts Criterion; focused on other agency or foundation broader impacts; or establishing a similar organization internationally based off an international broader impacts-like term or phrase.

Based on lessons learned, I have also provided seventeen (17) objectives, goals, or tasks below that should help you succeed in your broader impacts office or organization endeavors. It may also help you navigate potential unseen pitfalls along the way. Use this as a “Start a Broader Impacts Office or Organization Checklist”, as it is suggested that each item (i-xvii) be completed and assessed every year.

i. set up a viable, flexible, sustainable, and evolving infrastructure to meet university and faculty needs;

ii. determine organizational model, establish vision, mission, and value propositions;

iii. determine the number of primary, secondary, and tertiary audiences;

iv. establish a presence throughout your University and State as a broader impacts organization;

v. help faculty with all aspects of NSF broader impacts;

vi. help the university and state to understand broader impacts in general;

vii. establish the trajectory for successfully moving beyond the POC phase;

viii. determine and show the Return On Investment (ROI);

ix. establish evaluative and assessment protocols and strategies to determine sustainability;

x. develop a research-based approach to practice as it pertains to achieving societal benefit;

xi. develop relationships and partnerships with administrative leaders;

xii. develop an understanding and appreciation of faculty needs at your university, college, or in your specific unit (start doing this immediately!!!);

xiii. establish a university communication and response plan;

xiv. understand university structure and research office;

xv. assess if your organization is having a positive impact on the university, state, and national culture – international if applicable;

xvi. determine impact path of the organization;

xvii. develop a yearly report for upper administration summarizing i-xvi with appropriate examples and data, how it specifically links to their strategic plan or strategic action plan and university initiatives, the specific goals from last year and how they were accomplished, the specific goals for the next year, unexpected opportunities and challenges, and long-term goals in five and ten years.



[1]. Adetunji, O. and Renoe, S. (2017). Assessing Broader Impacts. MRS Advances, 1-6. Doi: 10.1557/ adv.2017.136.

[2]. Adetunji, O. and Thompson, M. (2016). The Broader Impacts Conceptual Framework (BICF) 2014 Lexicon Modification for the Brown University Engaged Scholarship and Broader Impacts Joint Committee Year End Report of 2015-2016. Brown University.

[3]. The Broader Impacts in Research (BIR) Organization. (2014). BI Definitions Guide: An abbreviated collection of explanations that begins to provide a common language when discussing, practicing, understanding, and better articulating the dimensions of broader impacts (BI). [Brochure]. [Norman, Oklahoma]. Thompson, M.

[4]. The Broader Impacts in Research (BIR) Organization. (2014). Broader Impacts Conceptual Framework (BICF) Lexicon. The University of Oklahoma. Thompson, M.

Understanding National Science Foundation (NSF) Broader Impacts Part 3

Here is Part 3 of the basic core competencies needed concerning NSF broader impacts. In this video you will find out the: research-based scholarly definition of broader impacts; difference between broader impacts and impact; meaning of broader impacts identity and; five basic structures of broader impacts through the Research and Scholarship of Broader Impacts (SoBI). Part 4 will be coming soon!

Historical Account of the Major Events Associated with the NSF Broader Impacts Criterion: From 1996 – 2017

Before 1996 NSF used four criteria to determine if a proposal would be awarded. The four criteria were: (I) Researcher Performance Competence; (II) Intrinsic Merit of Research; (III) Utility or Relevance of the Research and; (IV) Effect of the Research on the Infrastructure of Science and Engineering. These criteria were first adopted in 1981 and used until 1996.

The re-assessment and subsequent replacement of these four criteria resulted from a combination of events that happened at the national level and with NSF. Congress passed the Government Performance and Results Act (GPRA) in 1993. This now meant that NSF’s goals and strategies needed to be linked to the outcomes of the investments it had made throughout the country.

By the mid-1990s NSF had expanded its portfolio to include several things like broad education initiatives. NSF had also adopted a new strategic plan in 1995 (NSF 95-24). NSF 95-24 embraced some new long-term core strategies and goals. One of the aims of these long-term strategies and goals was about promoting knowledge in service or “benefit” to society. This strategic plan was called “NSF in a Changing World”. Meaning that NSF specifically needed to exemplify, participate, provide proof, acknowledge, and measure the societal benefit of its agency’s mission. “They needed to have a broader impact”! All of which was consistent with the trend happening internationally.

Official re-assessment of these four criteria were performed by a combined National Science Board (NSB) and NSF Task Force. The potential implementation of this term as a criterion was first provided in the National Science Board (NSB) and NSF Staff Task Force on Merit Review Discussion Report. A discussion about NSF’s Intellectual Merit criterion was also provided in this summary, NSB and NSF 1996 Task Force Report.

After the task force’s findings, NSB and NSF were convinced that the effective utilization and implementation of the broader impacts criterion would better allow NSF to connect its investments to societal value and benefit. NSF thought that it would allow them to better demonstrate and more clearly align their goals to their strategic plan. In addition, the use of this term, as a criterion, would allow NSF to combine and replace at least two out of the four criteria used from 1981 to 1996. A result that would also allow NSF to streamline its review process.

In 1997 NSF officially introduced broader impacts to the nation via notice 121. The two 1981-1996 criteria, (labelled III and IV above), were replaced by the Broader Impacts Criterion. NSF stated that broader impacts would be used as one of their criteria for determining if a proposal would be awarded. This new term was defined as encompassing the potential to benefit society and contribute to achievement of specific, desired societal outcomes. The term that replaced the other two 1981-1996 criteria, (labelled I and II above), was called Intellectual Merit.

In 1998 and 1999 Congress told NSF to work with the National Academy of Public Administration (NAPA) to determine the effects of their new merit review criteria. The 2001 NAPA report discussed the academic and research community’s views and challenges of the broader impacts criterion from 1997-2000. As J. Britt Holbrook wrote in his National Science Foundation: Second Merit Review Criterion editorial, “NAPA stated that NSF needed to address a host of philosophical issues raised by the broader impacts criterion”.

After the NAPA report, NSF began to try to address these philosophical issues. NSF continues to make efforts to address many of these challenges. This has been in part due to the apparent confusion about broader impacts and how it was inconsistently applied as a criterion. This sentiment has continued to be expressed by many across the nation.

Several events that followed the NAPA report began to shape NSF’s current version of the Broader Impacts Criterion. During the mid-2000’s many started to question the efficacy of and issues surrounding NSF broader impacts. This led to the American Creating Opportunities to Meaningfully Promote Excellence in Technology, Education, and Science Act of 2007. This is commonly known as the American COMPETES Act.

In the years following there were also other major attempts to address NSF’s Broader Impacts Criterion. Major events that have shaped or been influential in the response to the criterion between 2010-2017 are provided below. Note: the specific nuances outlining how each of these events historically developed will not be discussed.

First, Congress approved the American Creating Opportunities to Meaningfully Promote Excellence in Technology, Education, and Science (COMPETES) Reauthorization Act in 2010. This legislation mandated broader impacts and encouraged Institutes of Higher Education (IHEs) to assist investigators in achieving the Broader Impacts criterion. It also required investigators to provide evidence of institutional broader impacts resources.

In the year that followed, the National Science Board (NSB) provided statements supporting the Broader Impacts Criterion. The NSB stated that NSF broader impacts would continue. NSB also stated that IHEs should provide support for investigators because there was still confusion about the criterion.

Between 2011 and 2012, NSF made some procedural changes which provided nine over-arching long-term outcomes as it related to broader impacts. This is discussed further in Jeffery Mervis’s write-up in Science Mag News online. Some refer to this as Broader Impacts 2.0.

In 2013, the National Alliance for Broader Impacts (NABI) was formed. The National Alliance for Broader Impacts (NABI) received funding through a NSF Research Coordination Network (RCN) grant. NABI is comprised of about 200 institutions, companies, colleges, and organizations across the nation.

The goal of NABI is to create a community of practice that fosters the development of sustainable and scalable institutional capacity and engagement in the broader impacts of STEM research activity. NABI holds a national summit every year. In 2018 it will be hosted by Brown University.

In early 2015, NABI convened a working group to develop a guiding document for the NSF Broader Impacts Criterion. The purpose of this document was to assist program managers, proposal reviewers, and review panels in their evaluation of broader impacts components of NSF proposals. The guiding document was the first nation-wide/community attempt to begin the process of normalizing the way review panels evaluated and rated proposed NSF broader impacts plans. This document was released at the end of 2015 and has been nationally vetted and used across the nation.

In 2016, the American Innovation and Competitiveness Act (AICA) was passed by the 114th Congress. The AICA was considered the successor to the America COMPETES Acts of 2007 and 2010. A brief discussion and the implications of AICA was provided by Mitch Ambrose in the American Institute of Physics’ (AIP)  Science Policy News.

Also in 2016, a workshop about NSF broader impacts was organized by Sheldon Jacobson (from University of Illinois), Jerome Hajjar (from Northeastern University), Dawn Tilbury (from University of Michigan), and Andrew Johnson (from Texas A&M University). This workshop was called, “Setting a Broader Impacts Innovation Roadmap”. It took place in Arlington, Virginia and was supported by NSF (CMMI-1629955). At this workshop, researchers and administrators, mostly from the Mechanical, Industrial, and Civil Engineering disciplines, deliberated about how investments made by NSF could be enhanced by broader impacts contributions.

There have been other workshops, meetings, and seminars on broader impacts. However, this appears to be the first time publicly any group of specific disciplines, specifically several engineering disciplines came together to provide answers to questions about the NSF Broader Impacts Criterion and begin to articulate aspects of the concept. This will be discussed more in a future paper.

Lastly, in 2017, approximately a month after their yearly summit, NABI held a two-day meeting to discuss the Future Directions of Broader Impacts. This meeting brought together NSF officers, policy professionals, university administrators, broader impact and extension professionals, and representatives from two- and four-year institutions. This included Historically Black College and Universities (HBCUs), Minority Serving Institutions (MSIs), and Hispanic Serving Institutions (HSIs). During this time, there were discussions on developing needs assessments, strategies for creating consensus, and how to move broader impacts forward professionally and logistically. One topic that began to surface in this meeting centered on the Research and Scholarship of Broader Impacts (SoBI), see footnote six.

Origins of Broader Impacts in the West: Sovereign Nations Direct and Indirect Influence on NSF’s Broader Impacts Criterion

When it comes to broader impacts, most people have focused on how to do the National Science Foundation (NSF) broader impacts criterion. Many have not spent much time trying to understand the “entire” broader impacts concept. The inability to explain the “entire” broader impacts concept has slowed down the advancement of those concerned with broader impacts. This is especially true for those submitting proposals to NSF and those trying to address the philosophical issues associated with the NSF broader impacts criterion. This is because understanding the “entire” concept helps one to successfully accomplish the criterion. Understanding all the nuances of the concept also adequately informs organizations how to facilitate the criterion in a community.

It is common knowledge that the term, “broader impacts”, was coined in 1996 by NSF to be implemented as a scoring criterion for grant proposals. What appears not to be commonly known is that the origins of the “entire” broader impacts concept have practical and philosophical roots in values and thought established Before the Common Era (BCE). From a humanistic perspective, origins of this concept can also be found in other cultures throughout the world.

In the context of historical Western ideology and modern United States, some of these ideas and values were articulated by individuals such as Aristotle and Jean-Jacques Rousseau. The European ideas of this concept changed and evolved over time. However, these ideas evolved and developed most rapidly when Europeans encountered, learned, and mixed with peoples in the Americas.

More specifically, the rapid development and evolution of these ideas happened because these Ancient American civilizations, which spanned the Paleo-Indian, Archaic, and Formative Periods, had already achieved broad integration and utilization of the concept before Europeans arrived in the Americas. In addition, the Ancient Americans utilized this concept within a value system that was vastly different from Europeans. This Ancient American “societally competent” value system allowed for the facile and quick transmission of this concept to others, e.g., Africans, Asians, and Europeans. Over time, as Europeans moved-in and colonization took place, some of these values and value systems were indirectly used, adopted, or stolen. Some of these values and value systems are now practiced in modern United States.

In general, the combination of Ancient American and European practice and thought, along with many other cultures, have helped to shape the values that are held today in modern United States. In particular, specific Ancient American values and value systems combined with certain Aristotelian ideals as well as ideas from others have shaped the concept that currently underpins the broader impacts criterion1. A topic that has not been well-articulated in the broader impacts conversation.

Furthermore, the specific Sovereign Nation ontological, epistemological, cosmological, and axiological underpinnings of and influence on the broader impacts concept and its use as a NSF criterion have also not been discussed. This in some ways is very surprising. Broad integration and utilization of this concept, in the context of Ancient American civilizations, has been well documented through many of the written and oral histories of the Peoples of the Sovereign Nations2.

For example, the “medicine man”, contrary to what has been portrayed ideologically in Western media, was not a babbling mystic. Depending on the Sovereign Nation and again contrary to popular Western belief, the “medicine man” could be a woman. These medicine people could also be educators, researchers conducting research, artists, providers, humanitarians, engineers, historians, philosophers, administrators, social scientists, chemists, or biologists. These persons were positioned to benefit society in real tangible ways. For the Peoples of the Choctaw Nation, one way this would have been accomplished is through varying social practices of service by serving others and developing relationships.

This act of creating connections was termed “I’yyi kowa” or (broken foot). As a concept and practice, it meant that this service needed to ensure beneficial stability or “social balance” and advancement of their society. In other words, people in these professions needed to achieve societal benefits and desired societal outcomes. Note: I’yyi kowa can be spelled differently between the varying Mississippi and Oklahoma Choctaw communities but the concept it represents is the same.

More importantly, this idea of societal benefit was a position taken by many Indigenous communities. The idea was called by different names. It was a required value and a cultural expectation. All together, these practical, philosophical, and historical ideas foundational to the broader impacts concept are some of the keys to understanding and clarifying the practical, theoretical, and philosophical issues surrounding the NSF broader impacts criterion3.



1 There are other cultures and civilizations that influenced the broader impacts concept that have not been provided in this discussion.

2 The use of the phrase “Sovereign Nations” is representative of Native Americans, Indigenous tribes, and their officially recognized independent governments/countries.

3 A comprehensive explanation of the practical, philosophical, and historical roots of the broader impacts concept will be provided at a later time.