Equitable Outcomes

Equitable Outcomes are just and right; fair; reasonable, real, viable, valuable, and sustainable benefits for participants and other beneficiaries as a result of the activities and outputs of an event performed in an accountable way. These should be benefits that advance, add, or contribute to the positive and not to the negative.

It is a concept and practice inherent to the field of Research and “Scholarship of Broader Impacts (SoBI)”. Scholarship of Broader Impacts (SoBI) was originally coined by coined by Oludurotimi Adetunji at Brown University.

The Equitable Outcomes concept is part of the Nature of Broader Impacts and is characterized and exemplified in the Broader Impacts Conceptual Framework (BICF).

Understanding National Science Foundation (NSF) Broader Impacts Part 3

Here is Part 3 of the basic core competencies needed concerning NSF broader impacts. In this video you will find out the: research-based scholarly definition of broader impacts; difference between broader impacts and impact; meaning of broader impacts identity and; five basic structures of broader impacts through the Research and Scholarship of Broader Impacts (SoBI). Part 4 will be coming soon!

The Research-Based Scholarly Definition of Broader Impacts

The research-based scholarly definition of broader impacts isa process with people/stakeholders to achieve a societal benefit in a finite time that is measured. This can be through one’s teaching, research, service, and occupation. There can be broader impacts of almost anything. If done appropriately broader impacts can lead to sustainable positive impacts” [1] [2] [3] [4].

 

[1]. Adetunji, O. and Renoe, S. (2017). Assessing Broader Impacts. MRS Advances, 1-6. Doi: 10.1557/ adv.2017.136.

[2]. Adetunji, O. and Thompson, M. (2016). The Broader Impacts Conceptual Framework (BICF) 2014 Lexicon Modification for the Brown University Engaged Scholarship and Broader Impacts Joint Committee Year End Report of 2015-2016. Brown University.

[3]. The Broader Impacts in Research (BIR) Organization. (2014). BI Definitions Guide: An abbreviated collection of explanations that begins to provide a common language when discussing, practicing, understanding, and better articulating the dimensions of broader impacts (BI). [Brochure]. [Norman, Oklahoma]. Thompson, M.

[4]. The Broader Impacts in Research (BIR) Organization. (2014). Broader Impacts Conceptual Framework (BICF) Lexicon. The University of Oklahoma. Thompson, M.

Historical Account of the Major Events Associated with the NSF Broader Impacts Criterion: From 1996 – 2017

Before 1996 NSF used four criteria to determine if a proposal would be awarded. The four criteria were: (I) Researcher Performance Competence; (II) Intrinsic Merit of Research; (III) Utility or Relevance of the Research and; (IV) Effect of the Research on the Infrastructure of Science and Engineering. These criteria were first adopted in 1981 and used until 1996.

The re-assessment and subsequent replacement of these four criteria resulted from a combination of events that happened at the national level and with NSF. Congress passed the Government Performance and Results Act (GPRA) in 1993. This now meant that NSF’s goals and strategies needed to be linked to the outcomes of the investments it had made throughout the country.

By the mid-1990s NSF had expanded its portfolio to include several things like broad education initiatives. NSF had also adopted a new strategic plan in 1995 (NSF 95-24). NSF 95-24 embraced some new long-term core strategies and goals. One of the aims of these long-term strategies and goals was about promoting knowledge in service or “benefit” to society. This strategic plan was called “NSF in a Changing World”. Meaning that NSF specifically needed to exemplify, participate, provide proof, acknowledge, and measure the societal benefit of its agency’s mission. “They needed to have a broader impact”! All of which was consistent with the trend happening internationally.

Official re-assessment of these four criteria were performed by a combined National Science Board (NSB) and NSF Task Force. The potential implementation of this term as a criterion was first provided in the National Science Board (NSB) and NSF Staff Task Force on Merit Review Discussion Report. A discussion about NSF’s Intellectual Merit criterion was also provided in this summary, NSB and NSF 1996 Task Force Report.

After the task force’s findings, NSB and NSF were convinced that the effective utilization and implementation of the broader impacts criterion would better allow NSF to connect its investments to societal value and benefit. NSF thought that it would allow them to better demonstrate and more clearly align their goals to their strategic plan. In addition, the use of this term, as a criterion, would allow NSF to combine and replace at least two out of the four criteria used from 1981 to 1996. A result that would also allow NSF to streamline its review process.

In 1997 NSF officially introduced broader impacts to the nation via notice 121. The two 1981-1996 criteria, (labelled III and IV above), were replaced by the Broader Impacts Criterion. NSF stated that broader impacts would be used as one of their criteria for determining if a proposal would be awarded. This new term was defined as encompassing the potential to benefit society and contribute to achievement of specific, desired societal outcomes. The term that replaced the other two 1981-1996 criteria, (labelled I and II above), was called Intellectual Merit.

In 1998 and 1999 Congress told NSF to work with the National Academy of Public Administration (NAPA) to determine the effects of their new merit review criteria. The 2001 NAPA report discussed the academic and research community’s views and challenges of the broader impacts criterion from 1997-2000. As J. Britt Holbrook wrote in his National Science Foundation: Second Merit Review Criterion editorial, “NAPA stated that NSF needed to address a host of philosophical issues raised by the broader impacts criterion”.

After the NAPA report, NSF began to try to address these philosophical issues. NSF continues to make efforts to address many of these challenges. This has been in part due to the apparent confusion about broader impacts and how it was inconsistently applied as a criterion. This sentiment has continued to be expressed by many across the nation.

Several events that followed the NAPA report began to shape NSF’s current version of the Broader Impacts Criterion. During the mid-2000’s many started to question the efficacy of and issues surrounding NSF broader impacts. This led to the American Creating Opportunities to Meaningfully Promote Excellence in Technology, Education, and Science Act of 2007. This is commonly known as the American COMPETES Act.

In the years following there were also other major attempts to address NSF’s Broader Impacts Criterion. Major events that have shaped or been influential in the response to the criterion between 2010-2017 are provided below. Note: the specific nuances outlining how each of these events historically developed will not be discussed.

First, Congress approved the American Creating Opportunities to Meaningfully Promote Excellence in Technology, Education, and Science (COMPETES) Reauthorization Act in 2010. This legislation mandated broader impacts and encouraged Institutes of Higher Education (IHEs) to assist investigators in achieving the Broader Impacts criterion. It also required investigators to provide evidence of institutional broader impacts resources.

In the year that followed, the National Science Board (NSB) provided statements supporting the Broader Impacts Criterion. The NSB stated that NSF broader impacts would continue. NSB also stated that IHEs should provide support for investigators because there was still confusion about the criterion.

Between 2011 and 2012, NSF made some procedural changes which provided nine over-arching long-term outcomes as it related to broader impacts. This is discussed further in Jeffery Mervis’s write-up in Science Mag News online. Some refer to this as Broader Impacts 2.0.

In 2013, the National Alliance for Broader Impacts (NABI) was formed. The National Alliance for Broader Impacts (NABI) received funding through a NSF Research Coordination Network (RCN) grant. NABI is comprised of about 200 institutions, companies, colleges, and organizations across the nation.

The goal of NABI is to create a community of practice that fosters the development of sustainable and scalable institutional capacity and engagement in the broader impacts of STEM research activity. NABI holds a national summit every year. In 2018 it will be hosted by Brown University.

In early 2015, NABI convened a working group to develop a guiding document for the NSF Broader Impacts Criterion. The purpose of this document was to assist program managers, proposal reviewers, and review panels in their evaluation of broader impacts components of NSF proposals. The guiding document was the first nation-wide/community attempt to begin the process of normalizing the way review panels evaluated and rated proposed NSF broader impacts plans. This document was released at the end of 2015 and has been nationally vetted and used across the nation.

In 2016, the American Innovation and Competitiveness Act (AICA) was passed by the 114th Congress. The AICA was considered the successor to the America COMPETES Acts of 2007 and 2010. A brief discussion and the implications of AICA was provided by Mitch Ambrose in the American Institute of Physics’ (AIP)  Science Policy News.

Also in 2016, a workshop about NSF broader impacts was organized by Sheldon Jacobson (from University of Illinois), Jerome Hajjar (from Northeastern University), Dawn Tilbury (from University of Michigan), and Andrew Johnson (from Texas A&M University). This workshop was called, “Setting a Broader Impacts Innovation Roadmap”. It took place in Arlington, Virginia and was supported by NSF (CMMI-1629955). At this workshop, researchers and administrators, mostly from the Mechanical, Industrial, and Civil Engineering disciplines, deliberated about how investments made by NSF could be enhanced by broader impacts contributions.

There have been other workshops, meetings, and seminars on broader impacts. However, this appears to be the first time publicly any group of specific disciplines, specifically several engineering disciplines came together to provide answers to questions about the NSF Broader Impacts Criterion and begin to articulate aspects of the concept. This will be discussed more in a future paper.

Lastly, in 2017, approximately a month after their yearly summit, NABI held a two-day meeting to discuss the Future Directions of Broader Impacts. This meeting brought together NSF officers, policy professionals, university administrators, broader impact and extension professionals, and representatives from two- and four-year institutions. This included Historically Black College and Universities (HBCUs), Minority Serving Institutions (MSIs), and Hispanic Serving Institutions (HSIs). During this time, there were discussions on developing needs assessments, strategies for creating consensus, and how to move broader impacts forward professionally and logistically. One topic that began to surface in this meeting centered on the Research and Scholarship of Broader Impacts (SoBI), see footnote six.

Debunking the Twenty (20) Major Misconceptions About Broader Impacts, One at a Time:

Misconception 1: Broader Impacts only pertains to the National Science Foundation (NSF) Broader Impacts Criterion.

The global conceptual definition of broader impacts is a process with stakeholders/people to achieve a societal benefit in a finite time that is measured. This can be through one’s, teaching, research, service, occupation, and etc. There can be broader impacts of almost anything. If done appropriately broader impacts can lead to sustainable positive impacts. Having a global conceptual understanding of broader impacts helps answer questions such as – Who decides what will be done? Who benefits from those actions? Society through groups, communities, organizations, institutions, families, and individuals ultimately decides the value of actions as well as who benefits from those actions. This includes faculty. Having a global definition also allows us to begin to differentiate between broader impacts and impacts.

So what about NSF? The NSF broader impacts definition for the criterion is stated as encompassing the potential to benefit society and contribute to achievement of specific, desired societal outcomes. This NSF definition of broader impacts aligns well with and can be explained through the global broader impacts conceptual description. An example of this alignment can be seen in outcome number four in NSF’s 1-9 desired societal outcomes. Outcome number four is defined as improved well-being of individuals in society. NSF desired societal outcome number four delineates and anticipates a positive societal outcome that has yet to be specified. I call it a just in case and everything else clause. Outcome number four implies that there may be a moment where specific actions will be taken that allow an improvement to an individual’s well-being. This improvement would be achieved based on a process with stakeholders/people deciding what will be done and who benefits. These benefits would not fit within the other eight NSF desired societal outcome categories. This supports the idea that there can be broader impacts of almost anything and further shows alignment with the broader impacts global concept.

Furthermore NSF, a federal agency, has specific foci as it relates to broader impacts. This is in regards to mostly STEM and social science fundamental research. This could be accomplished through: the research itself; activities that are directly related to specific projects; and activities that are supported by, but are complementary to the project. A more detailed description of how this specifically relates to the broader impacts global concept will be discussed at another time.

Outside of NSF, there are other agencies and foundations that also have a “benefits to society” requirement but these requirements are applicable to their specific broader impacts foci. The difference between NSF and local, state, regional, national and international agencies and foundations is that they do not specifically use the term broader impacts. They call “it”, meaning broader impacts something else. There are many of these broader impacts-like terms, phrases, and ideas used around the world. A few examples of these are terms like Valorization (in Netherlands), Knowledge Mobilization (in Canada), Equity in Development (in India), Capacity Building (in Africa), National Economic and Social Development and Social Influence, (in China from NSFC Article 14-15), Ultimate Outcomes (in United States Department of Education), and concepts underpinning Responsible Research and Innovation and the Research Excellence Framework (the European Union).

Broader impacts is multi-dimensional, multi-level, and multi-encompassing. Broader Impacts can also be accomplished in many different ways. So regardless of whether “it”: is fully integrated as a requirement; indirectly implied in a vision or mission; a subset of an organizations funding foci; called a different name; or explicitly called broader impacts, “it” can still be understood and explained through a global conceptual definition. Therefore all specific foci and aspects of broader impacts rest on a global verifiable concept and not solely according to an agency or foundation requirement.

 

[References below are provided to give a way to be made aware of some of the applicable resources. This is not a comprehensive reference list and the references provided do not necessarily meet the latest format MLA, APA, and etc., rules.]

Adetunji, O. (2016). Engaged Scholarship and Broader Impacts Joint Committee. Year End Report 2015-2016. Brown University.

Adetunji, O. and Renoe, S. (2017). Assessing Broader Impacts. MRS Advances, 1-6. Doi: 10.1557/ adv.2017.136

Adetunji, O. and Thompson, M. (2016). The Broader Impacts Conceptual Framework (BICF) 2014 Lexicon Modification for the Brown University Engaged Scholarship and Broader Impacts Joint Committee Year End Report of 2015-2016. Brown University.

Davis, M. and Laas, K. (2014), “Broader Impacts” or “Responsible Research and Innovation”? A Comparison of Two Criteria for Funding Research in Science and Engineering. Science and Engineering Ethics, Volume 20, pp 963-983.

National Alliance for Broader Impacts (NABI). (2015). Broader Impacts Guiding Principles and Questions for National Science Foundation Proposals. [Tri-fold]. [Columbia, MO]: Adetunji, O., Coakley, C., Dawe, J., Dugan, C., Fields, J., Kobilka, S., Koroly, M., Menninger, H., Renoe, S., Ristvey, J., Scowcroft, G., Spohr, K., Sundararajan, S., Thompson, M., and Youngblood, T.

National Science Foundation (NSF). (2015). Perspectives on Broader Impacts. Online at, https://www.nsf.gov/od/oia/publications/Broader_Impacts.pdf (accessed 31, May 2017).

National Science Foundation (NSF). (2017). NSF 17-1 Proposal and Award and Procedures Guide (PAPPG), OMB Control Number 3145-0058.

The Broader Impacts in Research (BIR) Organization. (2014). BI Definitions Guide: An abbreviated collection of explanations that begins to provide a common language when discussing, practicing, understanding, and better articulating the dimensions of broader impacts (BI). [Brochure]. [Norman, Oklahoma]. Thompson, M.

The Broader Impacts in Research (BIR) Organization. (2014). Broader Impacts Conceptual Framework (BICF) Lexicon. The University of Oklahoma. Thompson, M.