Academics need to make sure their evidence matters

By Kathleen O’Grady and Noralou Roos

Stepping beyond the ‘hallowed paywalls’ of academic publication

A version of this commentary appeared in Policy Options 

Academics need to make sure their evidence mattersAn average paper in a peer-reviewed academic journal is read by no more than 10 people, according to Singapore-based academic, Asit Biswas, and Oxford-researcher, Julian Kirchherr, in their controversial commentary, “Prof, no one is reading you,” which went viral last year.  They cite some remarkable statistics – as many as 1.5 million peer-reviewed articles are published annually with as many as 82 percent never cited once, not even by other academics.

In other words, most academic writing rarely influences thinking beyond the privileged circles in which it is constructed – and the vast majority is far from influencing public policy and debate on critical issues.

Other academics question such dire statistics and use different approaches to shore up the numbers, such as changing what counts as a citation (including self-citations and non-academic citations), and using a longer window to check for citations.  But their attempts to improve the portrait of academic influence are not much better.

A cheeky article on the topic in the Smithsonian, entitled, “Academics write papers arguing over how many people read (and cite) their papers” notes that a 2007 study claimed half of academic papers are read only by the author and the journal editors – a stab to the heart if there ever was one.  They also cite (not without irony) a 1990 study which claims a whopping 90 percent of academic papers are never cited.

Dahlia Remler on the London School of Economic blog casts doubt on the damning figures in her article, “Are 90 percent of academic papers really never cited?” and notes that citation rates actually vary widely by field.  Still she acknowledges that as many as one third of social sciences articles go uncited, 82 percent for the humanities and 27 percent for the natural sciences.  What started out as a skeptical rejoinder ends with a simple plea: “Academic publication needs fixing.”

Lest the sciences think they come out looking good here by comparison, evidence from other quarters is not so favourable.  Twenty years ago, the journal Science found a mere 45 percent of articles published in the top 4,500 science journals were cited within five years.  A more recent study found a decline in that figure – only 40.6 percent of articles in the top science and social science journals were cited within five years.

In other words, the problem is not a new one and appears to be worsening.  Why?  There’s simply too much to read.

We are awash in a sea of unread journal articles

There has been a dramatic growth in the number of journals, and by extension, journal articles, published every year.  The STM Report from 2015 notes there are more than 28,000 active scholarly peer-reviewed English-language journals which publish around 2.5 million articles annually.  The report notes the number of journals is growing at a rate of at least 3.5 percent per year; this reflects a growth in researchers of about three percent per year.

Growth in the number of academic journals is possibly also linked to their surprisingly high profitability – as well as the counter movement to create open-access journals.  To put these growth numbers another way: the rate of journal article output more than doubles every 20 years, notes the Chronicle of Higher Education.   And those are the optimistic numbers.  Another study pegs journal article growth at an average 6.3 percent per year.

The result?  A trend toward fewer citations per paper.  It appears we are awash in studies no one ever reads.

It’s not bad news for everyone as it turns out – just for most.  As the number of journal articles increases, the number of citations on average per article, decreases.  The STM report, however, notes that the distribution of citations is highly uneven, with 80 percent of citations coming from fewer than 20 percent of articles.  As one researcher summarized the trend: “fewer journals and articles [are] cited, and more of the citations [are] to fewer journals and articles.”

In other words, the deluge of uncited papers is punctuated by a handful that rise to the surface like the tip of an iceberg.  But lest you get the impression the established journals are the ones benefitting here, the trend is just the reverse.  The dominating influence of elite journals may be on the way out.  Turns out highly cited papers published in highly cited journals are on the decline, while the number of highly cited papers coming from new and less established journals rises steadily.

There are any number of mitigating factors for why we have so many unread journal articles – the publish or perish mantra that seems to kick in earlier with each generation of scholar; the parsing of a study into many component parts to maximize journal article output; the inflation of referenced articles and co-authors to boost impact factors; an academic system that rewards researchers for output and not necessarily influence – to name a few.  No change is imminent, unless, as one commentary notes, “the system of rewards is changed.”

But not least of all, part of the blame is surely the result of a system that looks down upon or at least disregards academic engagement with media, policy makers and the wider world it claims to address.  The truth is, few beyond the academy know or read academic journals.

Incentivizing Academics to Engage With the Media

It would be a small – but critical – step for academics to tell audiences why their research matters.  Presumably much of the research in journal articles would and should matter to those beyond academic circles – particularly those who are in the business of creating policy.

Biswas and Kirchherr in their “Prof, no one is reading you,” propose just such an approach.  The answer to researchers being trapped in the echo chamber of academic journals, they suggest, is to step beyond them – and engage the mainstream media: “If academics want to have an impact on policymakers and practitioners, they must consider popular media, which has been ignored by them.”  This is not a refutation of the journal publication, or the important evidence it imparts, but an extension of the publication process.

So why is this not happening already – at least not frequently?

An article in SciLogs put it this way: “The biggest hurdle is that academia has yet to find an incentive for them to take time away from the lab to engage the public.  Universities still operate under an ancient system that values only scholarly output.”

Academic physician, Daniel Cabrera calls for a change in the way traditional journal citation “impact factors’ are used for academic promotion.  He suggests instead we establish a system that also rewards academics who engage and share their knowledge with the public via traditional, new and social media too.  Journals and citation counts are no longer enough.

Cabrera points to the many publicly available and robust metrics now available from media and social media and concludes: “As Clinician Educators, we should champion the movement from a 50-year old journal-based index that imperfectly serves as a surrogate of influence to modern analytics with the ability to monitor, measure and share the real-time influence of scholars in their institutional spheres but also in their public spheres.”  The time has come, he suggests, for academics to engage beyond journal articles and communicate their evidence and ideas to a broader interested public.

By no means are these calls for academic engagement with the media – and beyond – isolated.  It’s become somewhat of a clarion call.

Last year, the Guardian published a commentary entitled, “Academics: leave your ivory towers and pitch your work to the media.”  As an academic herself, but also a journalist, the author, Kristal Brent Zook says she’s wondered why her academic colleagues don’t engage or write for the media more often: “Why don’t we hear more from the doctors behind the data?” So she asked around.  The answers she got from a survey of academics surprised her.

It turns out the main reason is fear.  According to Zook, academics don’t engage with the media because of fear of the unknown – both the media, and how it works, and of engaging with the general public.  There’s also an unspoken wariness, at worst, hostility, between academics and journalists who have differing cultures, time lines
and agendas.

However, according to Sense About Science USA, the journalist door is (almost) always open.  They recently published new media guides for scientists and surveyed more than 200 journalists in the process.  They found 92 percent of journalists are always open to scientists calling them if they have information; 94 percent of journalists said want to hear from sources if they feel they were misquoted or misrepresented; and 94 percent always or most of the time read the academic article(s) in question before contacting the scientist for interview about their research.

What Happens When Academics Step Outside the Echo Chamber of Academic Journals?

In her commentary, Zook notes the positive outcomes that arise when academics do engage the media.  One academic said that a single of her posts went viral, ended up on numerous syllabuses, and opened up opportunities to write academic articles, book chapters and even a book, and indirectly, a grant. “It was a catalyst,” she said.  Another highlighted how writing for the media made her a better writer.  She realized how riddled her writing was with jargon and that these big words were often a ‘crutch’ – “I have to cut through the bullshit and just say what I really mean.”

In a recent British Medical Journal blog, David Payne echoes the call for broader public engagement: “Being an academic is all about people knowing about your research…Don’t leave a paper to its own devices.” Publication is not enough, in other words – it’s the beginning of a process of engagement, not the end of it.

Duncan Green, similarly, in the London School of Economics Impact blog says that much of academic life is “spent within the hallowed paywalls of academic journals” but that academics could and should engage more with new and social media to attract an audience to their research and give their research meaningful impact.  He calls engagement with a wider audience “an antidote to futility.”

Like Payne and Zook, he cites specific instances where engagement clearly resulted in significantly increased journal article readership, among other real world and academic outcomes.  So it’s not about abandoning traditional academic publishing, but expanding its scope and reach – and making their evidence matter.  What academic wouldn’t want that?

A Wall Street Journal article headlined, “Why the Dean of Harvard Medical School Tweets” similarly outlines how successful engagement strategies – in this case, using Twitter – can make academic research meaningful beyond the academy, and reach other educators, policy makers, economists and politicos with important outcomes.  Harnessing traditional and new media to engage with wider audiences helps make the research live on, in other contexts, and affect change.

How the Public Reaps the Rewards of Academic Engagement

So engaging the public through traditional and new media is good for academic up-take in a wide range of ways, but it turns out it’s also good for the general public too.  According to Deepti Pradhan writing for the Op-Ed project, only two percent of citizens in the U.S. are actively and formally learning about science – the rest learn about science “through the general media.”  Quality evidence in media outlets can help shape public perceptions and public debate on important policy issues.  The lack of it can sometimes have devastating consequences.

Declining vaccination rates is a good example.  In a study last year, 60 autism scientists were polled about the importance of communicating their work to the public.  Fifty-nine felt their research would be of interest, yet less than half felt it was ‘very important’ for them to do this – in other words, they felt it wasn’t their job.  Half felt they didn’t have opportunities to communicate with the public, and half felt they didn’t have the time to do so.

But the study authors emphasize the outcome of just such a lack of engagement.  They note that the field of autism – including amongst the parents and caregivers of kids with autism – is riddled with misinformation.  This has real world consequences, not least of which is the all too recurring myths around the harms of vaccination (that they cause autism – substantial evidence says they don’t) which directly affects rates of vaccination in the population.

The evidence is there, and it’s good, but it’s not getting to the public.  The lack of communication between scientific researchers and the public “threatens the relationship with the community they’re trying to help,” claims an editorial from the Simons Foundation Autism Research Initiative.

Other calls for academic engagement with the world beyond journal publication include Ben Goldacre in the Times Higher Education who says we need more academic engagement in the public policy process specifically: “we need more of this interaction, not less.”  He writes articles in the mainstream media regularly and meets with policy wonks and politicos in an effort to get research out of journals, informing policy-making and legislation-building in the process.  According to Goldacre,“Policy staff are crying out to be lobbied” by knowledgeable academics.  They want and need the evidence to do their work and academics want their work to matter.  It’s a perfect marriage of sorts.  As one policy maker said to him, “If none of the [academics] are ever calling me, then what do they do all day that they all think is more useful?”  A damning indictment.

The refrain of having academics dive into the policy debates and discussions that have real life consequences come from many quarters and from around the world.  It is motivated – on the whole – not from a critique of the scholarship but from a desire to see quality evidence affect change, for our public investment in higher learning to bear fruit in public policy and beyond.  But there are real barriers that can’t be ignored.  The challenge for many academics is that even if they want to engage in such public discussions – to go where the audiences already are, in traditional, new and social media – they don’t often know how to go about it and few have the time or the resources to do it properly.

How 700 Words Can Make a Difference

It was in just such a context that EvidenceNetwork.ca was born in Canada.  Founders, academics Noralou Roos and Sharon Manson Singer were frustrated that research they knew well in the field of Canadian health policy seemed to rarely make it into the mainstream media.  There were a few notable exceptions – produced by the handful of remaining well-trained health journalists scattered across the country.  On the whole, they found the media discussion dominated by hyperbolic language from left- and right-wing think tanks and political parties.  The extremes got air time, but most of the nuance and depth of any research was lost.

Roos and Manson Singer wanted to inject more evidence into the discussions and to see the rich world of academic research given equal air time.  So they created EvidenceNetwork.ca – with a sizeable grant from the Canadian Institutes of Health Research and Research Manitoba, to create a bridge between the world of academia and the world of journalism.

In consultation with our academic experts, we discovered academics often worry they’ll lose control of the message and the way their research is represented in the media in traditional interview formats.  At the same time, we learned from our media advisors, how academics often failed to heed the tone, style of language and breadth of discussion permitted in submissions to media publications.

From this tension was born the idea to help our academic partners craft op-eds – the opinion-based commentaries that appear in most media outlets.  Op-eds frequently influence politicos, policy makers and other decision-makers, and are a popularly read media genre where studies and evidence come to the fore.  We knew that many Universities already offer media training to academics – one-day or one-hour ‘how-to’ courses on how to write and pitch op-eds – so we wanted to offer something more robust.

We adopted a slightly different model with the idea that academics can’t be all things to all people.  They can’t be expected to know inside-out the media environment or have the time or interest to develop relationships directly with journalists and editors.  So we decided to do that for them.

What started out as an experiment in 2011 has blossomed into a full media service for academics and a clearinghouse of high quality, original health policy articles ready for publication for media outlets.

EvidenceNetwork.ca acts as a mediator and editorial service for academics who are asked to first write a rough draft of their op-ed.  We guide them on how to proceed with the first draft – the dos and don’ts of the process.  After that, we take it from there.  Our editorial team works on tightening up the op-ed with the needs of specific media outlets in mind.  Rough drafts are often trimmed to specific word counts, the argument is tightened and jargon-laced language is excised in place of plain or conversational language suiting traditional op-ed style.

The author is part of this process throughout.  The op-ed routinely goes through three or more rounds of edits and – if the material is controversial or political in any way – it goes through an informal peer review process with other experts vetting the piece for balance and accuracy.  All points of evidence are hyperlinked to their sources.  Once we are certain the piece is ready for the media, we secure author approval, and then EvidenceNetwork.ca shops the op-ed to the major media outlets on behalf of the author.

The result is that the author maintains control of the message and the language of the article, while at the same time benefitting from the expertise of EvidenceNetwork.ca staff who liaise regularly with media editors to learn what styles and formats they prefer and require, and to establish trust and rapport.

We have robust data to prove it works.  In the last year alone, we edited and placed over 100 op-eds.  Almost four dozen were published in the top five media outlets alone (Globe and Mail, National Post, Toronto Star, La Presse and Le Devoir) in a single year, and 191 more were published in the bigger city media outlets and 665 in the smaller regional media in 2015.  Year over year we’ve improved these metrics.  In total, we’ve published 494 op-eds garnering more than 2000 media publications in fewer than five years, publishing in the biggest media outlets to the smallest niche and rural papers across the country.

2016 Number of OpEds by Where Published.png

 

Like the commentators who plead for more academic engagement with the media and who tout the real world benefits of the exercise, our little Canadian experiment has proven that a measly 700 words (the average size of an op-ed) matters.  Our op-ed writers, as a result of their op-eds, have been cited by Ministers, invited to Parliamentary hearings, committees and briefings – at the federal and provincial levels.  Our op-eds have kicked off other media investigations, editorials and interviews for our writers – in print, online, radio and TV – and several of our op-ed writers have won academic engagement awards from their more forward thinking universities for their efforts.  How much of the research documented in the op-eds has been picked up and used in political, policy and other circles is harder to measure – but we know at the very least the research is getting out there and not just sitting in academic journals.

Op-eds don’t require much time investment, they are high profile and often shared widely on social media.  They become part of the public discussion and dialogue around important issues of the day.  700 words, it turns out, can make a difference.

The Beginnings of a Global Movement

We hope that the bridge we’ve helped build between mainstream media and academic research sets a precedent for further exchanges and participation between these two distinct groups. We also hope that our op-ed model will be copied and replicated by others elsewhere.

EvidenceNetwork.ca is by no means the first.  There have been other successful models for helping academics to showcase their research to mainstream audiences.  There is the Prague- and New York-based Project Syndicate, which provides professionally edited and copy-ready commentaries authored by academics to global publications on a subscriber model (the media outlets subscribe to their paid service for content on a sliding scale).  They boast that 476 media outlets in 154 countries carry their content.

The United States, the United Kingdom and Australia each have their own, The Conversation, a quality content provider that uses professional journalists to collaborate with academics and ‘translate’ academic research into evidence-based news articles and commentaries.  They put a Creative Commons stamp on all their content so it is available free of charge and they partner with a variety of media outlets to push their content out globally.  Their funding comes from a wide range of University partners and Foundations.

Canada also has the Science Media Centre of Canada which does not provide content directly to media, but connects journalists with academic experts and organizes webinars and media backgrounders on complex research. They also share forthcoming academic articles with journalists in an effort to increase coverage of science issues.  They are funded from the public, private, and not-for-profit sectors, but mandate that no more than 10 percent of funding comes from any single source so they can maintain their independence.

Both Canada and the United States also have media projects to get more women’s voices into the media.  Canada’s Informed Opinion offers op-ed writing workshop in University and NGO settings across the country and offers strategic advice to get more women to both write for and be interviewed by the media.  Their fun tag line and comic graphic, “What if I really am the best person?” sums up the hesitation too many academic women feel before they wade into media waters and stake their claim as the expert they are.  Founder, Shari Graydon has also done original research on the underrepresentation of women as sources in Canadian media stories.  Unfortunately, despite much awareness about this issue in recent years, the statistics haven’t budged in a couple of decades with women being quoted on average in 21 percent of news stories compared to men.  Op-eds offer a unique way for women to control their voice and message and still push their expertise out to wider audiences.

Similarly, New York’s The Op-Ed Project aims to get more women’s voices into the commentary pages of American papers.  They seek out “under-represented experts” in a variety of fields, pair them with media mentors, and guide them through the media placement process.  They also conduct media workshops and partner with Universities, think tanks and NGOs to get more women’s voices into media stories.

It’s about time we’ve had this global movement now that pushes academic evidence out into the world in an accessible format so that it does not sit idle behind journal paywalls but makes a difference in the world in which we live.

We all benefit when research is read widely and discussed soundly.  It’s how we can make sure evidence matters.

 

This essay is adapted from the forthcoming eBook, Why We Need More Canadian Health Policy in the Media edited by Noralou Roos, Kathleen O’Grady et al., which will be available on Kindle, Apple, Google and other formats.

 

Kathleen O’Grady is the Managing Editor of EvidenceNetwork.ca and a Research Associate at the Simone de Beauvoir Institute, Concordia University.  She writes regularly for a range of media outlets including the Chicago Tribune, Huffington Post and the Montreal Gazette. 

Noralou Roos is the Founder and Director of EvidenceNetwork.ca and Professor, Max Rady College of Medicine at the University of Manitoba.  She was the Founding Director of the Manitoba Centre for Health Policy.

July 2016

This entry was posted in Commentaries, Health is More than Healthcare, Making News, Making News, Health more than Healthcare, Our Commentaries and tagged , , , , , , , , .

Comments are closed.

« Back to Commentaries // Lisez la version française;

License to Republish: Our commentaries, Infographics and videos are provided under the terms of a CreativeCommons Attribution No-Derivatives license. This license allows for free redistribution, commercial and non-commercial, as long as it is passed along unchanged and in whole, with credit to the author and EvidenceNetwork.ca
EvidenceNetwork.ca supports the use of evidence when reporting on health and health policy in the mainstream media. Specific points of view represented here are the author’s and not those of EvidenceNetwork.ca. Let us know how we’re doing: evidencenetwork@gmail.com