23 August 2019
Ziyad Marar, SAGE Publishing
From climate change to cyber security, inequality to violent crime – the social sciences are critical in enabling us to understand and respond to the most pressing issues facing humankind today. Despite this, funding for and use of evidence-based social research is precarious, and an absence of demonstrated impact only serves to increase this precarity. Marking, measuring, and celebrating the impact of social science – beyond counting citations – is essential to incentivising this critical work and ensuring that it continues for generations to come.
The challenge to understand the full societal impact of academic work is not a new one; the scholarly community has been wrestling with this particular problem for decades. But it is only in recent years that we have seen an uptake in requirements from funders to prove the impact of scholarly work, as well as intensified scrutiny about the value of social and behavioural science for society. However, current dominant measures fail to capture the influence social research has on policy, practice, and the public. Furthermore, the dominant metrics of measuring journal and article quality are particularly ill-suited to quantifying impact for social science disciplines.
Against this backdrop, how then can the scholarly community – including researchers, universities, societies, publishers, and librarians – effectively demonstrate and measure the true impact of the social sciences?
SAGE Publishing has been an active voice in the social science impact debate for a number of years, as it has evolved from a controversial idea to an established part of most national research systems. Five years ago, we published The Impact of the Social Sciences with the LSE (London School of Economics) – the result of a three-year project focused on developing an understanding of the societal value and impact of social science knowledge and research in the UK. A year later, we published The Metric Tide, an independent assessment of the role of metrics in assessing research.
Since then, the conversation has progressed and we are now seeing increased emphasis placed on the impact of academic work in the UK. The next Research Excellence Framework (REF 2021), for example, will put more weight on the real world impact of research. Moreover, entirely new impact measures, such as the Knowledge Exchange Framework (KEF), have also been introduced. In the US, the outlook is similar: The National Science Foundation now requires “broader impacts” to be explicit in the research it funds and other funders increasingly require clear and direct demonstrations of impact from their grantees. Even the Pentagon, through its Defense Advanced Research Projects Agency (DARPA), has commissioned social scientists to create an artificial intelligence system to quantitatively measure the reliability of social science research. Known as SCORE, the system’s aim is to “increase the effective use of [social and behavioural science] literature and research to address important human domain challenges”.
But despite these advances, a consensus is yet to emerge within the social sciences, or beyond, as to how social science impact can best be demonstrated. This void has allowed other disciplines to set the agenda, ensuring that metrics like citation counts – a measure that particularly ill-serves the social sciences, where citations rarely accrue as quickly as in other disciplines – remain the dominant measures for academic work. (Indeed, when 2018 Journal Impact Factors were announced earlier this year, we decided to focus on the five-year Impact Factors of our social science journals, instead of the more typical two-year measure, to encourage a longer-term view). A new survey of faculty at four US universities by the Association of College & Research Libraries finds that social science researchers are shifting their conceptions of demonstrating impact toward “ways more aligned with the Sciences and Health Sciences” (and away from those favoured in the arts and humanities).
Taking all of this into account, at SAGE, we are reigniting the debate to stimulate fresh thinking on the impact of social science. Earlier this year, we assembled a working group at Google’s main campus to share ideas for helping scholars navigate the slippery concept of impact and the shortcomings of established metrics. As part of this effort we have produced a report, The Latest Thinking About Metrics for Research Impact in the Social Sciences, which highlights the findings of this group. The report maps out stakeholder categories, defines key terms and questions, puts forward four models for assessing impact, and presents a list of 45 resources and data sources that could help in creating a new impact model. It also establishes imperatives and recommended actions to improve the measurement of impact including:
- recognition from the community that new impact metrics are useful, necessary, and beneficial to society
- establishing a robust regime of measurement that transcends but does not supplant literature-based systems
- coming to a shared understanding that although social science impact is measurable like STEM, its impact measurements are unlikely to mirror STEM’s
- creating a global vocabulary, taxonomy, global metadata, and a global set of benchmarks for talking about measurement
Our hope is that the report becomes a helpful tool in opening deeper conversations and crowdsourcing new ideas when it comes to improving social science impact metrics. In this spirit, we have carried the conversation forward through panels at conferences such as the Association for Psychological Science’s (APS) Annual Convention and the Canada’s Annual Congress of Federation for the Humanities and Social Sciences. With the goal of engaging all members of the scholarly community, we will also be convening similar conversations at a number of upcoming library conferences.
Of the APS panel, Judy Ruttenberg, Director of the Association of Research Libraries, wrote, “These issues hit very close to home for research libraries … Resources like ‘The Latest Thinking About Metrics for Research Impact in the Social Sciences,’ … provide excellent food for thought for libraries undertaking this work, and confronting its challenges along with the research and learning community.”
Beyond this, we are actively disseminating research by leading scholars in formats accessible to policy makers and the general public. For example, we recently launched a new book series, ‘What do we know and what should we do about…’, with the first three titles tackling the topics of Immigration, Inequality and The Future of Work. These short-form texts give a quick overview of the current research around each subject, as well as suggested ideas based on the authors’ extensive knowledge of the field.
We are also working with partners to acknowledge impactful social science research. This includes support for the Economic and Social Research Council’s annual Celebrating Impact Prize, and the Golden Goose Award.
Ultimately, whilst the work of social scientists is undoubtedly impactful and increasingly important, the current metrics in place are inherently limited and there remains much work to do in this area. So, in an effort to keep the conversation alive for the longer-term, we have developed a new impact section of the SAGE-sponsored community site Social Science Space. This space itself is also being used to gather ideas, amplify diverse opinions, and engage in debate about impact with global actors engaged on the topic.
We welcome suggestions, ideas, and conversation round this topic: email info@socialsciencespace.com, or Twitter @ZiyadMarar.
These views are the author's own and do not necessarily reflect the views of UKSG.