What is the impact of news and how do you measure it?
These are good questions that go to the heart of newsroom sustainability. Embedded within the answers is a better understanding of newsrooms’ added value and a value proposition for sustaining newsrooms – online and hyperlocal news sites, whether non-profit or for profits. For NPOs, foundations will want to know what impact their grants provide. For commercial sites, potential advertisers will want to know what they get in return for their advertising dollars. Click through rates measure activity but they do not demonstrate impact.
Many answers for describing impact of hyperlocal news
These questions were recently addressed in the BlockbyBlock Facebook Resource group. Michele McLellan, BlockbyBlock founder, said there are many ways to look at the issue of measuring impact.
Many answers. For starters, they may be looking for impact at a public policy level or in exposing wrongdoing. They may be looking at level of citizen engagement with the site and its content, which can be measured in a number of ways. They may be looking for how the site extends the content deeper into the community, say by hosting in-person forums, etc.
McLellan added that she views web stats — pageviews, uniques of reach — as measures of reach and engagement, not impact. [ View a Storify with additional comments from the BlockbyBlock community.]
What prompted the recent flurry of conversation was in part an announcement by The New York Times that the paper is looking for good short-term help to decipher the right metrics to measure the impact of news. Aron Pilhofer, who runs the social journalism newsroom team at the Times, and who is the co-founder of DocumentCloud.org and Hacks and Hackers, posted a short “Help Wanted” on his blog July 25. The Times will be bringing on a Knight-Mozilla fellow to dig in and unearth some real measures of news and — we can only hope — break through to a definition of impact. Pilhofer wrote:
If you’re an analytics nerd, a news junkie and think it would be neat to spend some time working on a problem like this using The New York Times newsroom as your laboratory, we’d like to hear from you. The deadline is August 11th.
What we do not have are ways of measuring how a piece of journalism changes the way people think or act. We don’t have a metric for impact.”
This was also a topic of conversation among writers who participate in the Carnival of Journalism [The thread is not publicly viewable].
RJI’s Mayer and J-Lab engagement research cited
Denise Cheng, a digital journalist who has worked for the Journalism Accelerator and The Rapidian in Grand Rapids notes the overlap between the NYT research and the work done by Joy Mayer during her Reynolds Journalism Institute fellowship. Cheng said:
[Mayer] interviewed many journalists and technologists to triangulate a definition of engagement, and in spring 2011, she and Reuben Stern convened around 30 practitioners from all over the country to survey different engagement tactics, how they were beginning to measure those or how they’d flesh out those metrics. It led to a white paper. It’s such a great spring board that I hope Aron and Greg tap her for her insights.
Jan Schaffer, Executive Director of J-Lab, said of the New York Times project:
Aron’s interests ring true with us at J-Lab and echo some of the frustrations we surfaced in our “Engaging Audiences” report and suggest there might be a correlation between impact and genuine audience engagement.
Schaffer said that J-Lab research found at least four types of engagement:
- Engagement as outreach, driving users to consume content.
- Engagement as reaction, inviting users to comment, share, like and chat.
- Engagement as stakeholder participation, getting users to contribute stories, time, funding.
- Engagement as civic participation, activating audience members to address community issues.
Schaffer topped it off saying.
You know your journalism has had impact when people start participating in more than just your website, Facebook page, blog or twitter feed.