Checkdesk: An Open Verification Toolkit for Journalists in the Digital Age

How can I verify this? That’s the constant refrain of journalists covering fast-moving stories online. Faced with a barrage of videos, tweets and photos from myriad sources, how do you know what’s true? Misinformation spreads like wildfire. Think Sandy, Boston, Syria. Sometimes even trusted news organizations spread it. How can you avoid the trap? Checkdesk can help you. Drawing on Craig Silverman’s Verification Handbook, Checkdesk offers a checklist of functions for checking a video, photo, or tweet. See an amazing new photo on a big story and Checkdesk will take you through the steps you need to check it’s true: reverse image search, reveal EXIF data, view satellite image of location. Checkdesk turbocharges you into the editor’s chair.

Photo of Tom Trewinnard
24 45

Written by

Let’s show you an example of how Checkdesk can help you get your job done.

You’re a journalist covering the conflict in Syria. You spot a photo trending on social media. What do you do? You have to establish how credible the image is. Here’s where Checkdesk comes in. It gives you a predefined list of operations from the Verification Handbook to help you verify the photo, available on the same page with a single click. For example:
- Reverse image search using Tineye
- Reveal the image EXIF data using
- View satellite imagery of the photo’s geolocation so you can identify landmarks
- Cross reference photo location, time and date with weather information using Wolfram Alpha

With these steps conducted, you feel more confident to report the photo accurately. You can share out a link or embed the photo on a blog. And with Checkdesk, you can also share the verification steps you took: our embed feature contains the results of the verification operations alongside the source content just like the health warning that accompanies an AP video script.  

What makes Checkdesk really powerful is that the checklists are customizable and shareable. We have checklists for videos and checklists for tweets, each with different strategies for verification. As new tools and platforms emerge, you can create and share out your own checklists - or adopt them from others. Why not collaborate to find the perfect formula?

Checkdesk then is so much more than a static feature set, it is an open framework of ‘integrations’ with 3rd party services. Think of the long arc as like an app store for verification. Developers with a journalistic mind can find their own ingenious ways to exploit third party tools and build new integrations that you can add to your verification checklists. For example, there is huge potential for collaboration with other tools such as Cryptocat (for end-to-end encrypted communication between journalists and media sources), Rbutr (to cross-reference any other work done to highlight media as misinformation), Alghayma (to archive citizen media and protect it from deletion) and Storyful’s Multisearch (for cross-social search). Further integrations could enable translation of key content or consistency checking of urls over time.

What we propose is not a silver bullet for verifying digital media, but a more structured, easy-to-use and expandable toolkit that makes it easier for you, the journalist, to use the best techniques already out there.
Who are the users or target customers of your project, and what have you learned from them so far? Please give specific examples.
Our users are journalists and editors working for newswires, major publications, and grassroots news organizations that use social and citizen media as a source for reporting. In our work thus far, we have noted that newsrooms rarely have a standardized procedure for verifying the citizen media they use in reporting, and even more rarely do they have a means of communicating their verification work to their readers and viewers.

Our project aims to add value to journalists by providing robust workflow tools based on best practices outlined in the Verification Handbook and checklist efficacy research conducted by Peter Pronovost. We aim to add value to media consumers by adding key information to the citizen and social media they consume such as the extent to which media has been verified. The need for this is highlighted in the BBC Trust report on BBC coverage of the Arab Spring and in an upcoming Tow Center report on User Generated Content in Broadcast Journalism.

One example of when Checkdesk may have spared a major news organization’s blushes and avoided confusion for readers is the BBC’s misuse of a photo originally taken in Iraq to illustrate a story about a purported massacre in Ghouta, Syria. Checkdesk could have walked BBC journalists through a checklist of verification techniques such as reverse image search that would have shown the image’s original source and ensured it wasn’t misused to show another highly emotive and contentious event.
What assumptions are you making in what you propose, and how will you test them?
Our proposal makes the following assumptions:
  1. That social and citizen media already are important sources for news publishers covering breaking news stories, and their importance will continue to grow with the widespread use of camera-enabled mobile phones and wireless networks.
  2. That as the above happens, there will also be an increased risk of unintentional misinformation and intentional disinformation spreading rapidly via social and traditional media (see Gupta et al.)
  3. That embedded checklists with 3rd party integrations will help balance the needs of newsrooms to provide rapid responses to ongoing news developments and media sent in by citizen journalists and anyone using social media with the important needs of newsrooms to maintain accuracy. These checklists will help embed the fact checking process within journalists’ regular process and facilitate rapid verification steps such as revealing EXIF data, checking the weather, and others.
How will you get your project in front of the necessary people or organizations?
Since 2011 Meedan have been building a network of partners across MENA and the English-speaking world, in both newsrooms and academia.  This includes the largest newspaper in Egypt (Al Masry Al Youm), the biggest news publisher in Palestine (Ma’an) and a leading newspaper in Lebanon (Annahar).  These publications have used an early version of Checkdesk in their reporting, as have 3 other MENA-based news organizations and one journalism school. Our early project work has also been presented at 4M conferences in Cairo and Marseille, at Stanford University’s Right to Information Conference, at the 4th Arab Bloggers meeting in Amman, and at the IPI News Innovation Conference in London. Additionally, we have discussed and demonstrated early versions of Checkdesk with the BBC, Al Jazeera, the Guardian, Internews and Austrian public broadcaster ORF.

In contributing to and translating the Verification Handbook we have established a strong relationship with Poynters and the European Center for Journalism. All of the above have validated the need for better tools and training resources for digital journalists working with citizen and social sources, and we are confident that — with Knight support to develop the toolkit — several of the aforementioned would use and advocate for the use of Checkdesk. One opportunity that is of particular interest is working with journalism schools. As our experience working with Birmingham City University has demonstrated, the next generation of journalists will need strong digital media literacy to succeed in the workplace. Checkdesk can provide a safe and instructive context for training verification techniques.
What are the obstacles to implementing your idea, and how will you address them?
Journalists and editors in today’s newsrooms are strapped for time and money, and in the present culture of instant reporting, accuracy has taken a back seat to speed. The major obstacle we foresee is striking a balance between newsrooms’ increased financial pressure to push out news quickly with the very raison d’etre of journalism, namely, to “ be honest, fair and courageous in gathering, reporting and interpreting information.” In addition to the Verification Handbook, we take lessons from Dr. Atul Gawande’s Checklist Manifesto, which has argued for the use of checklists in (amongst other fields) the medical profession, which faces a similar demand for balance between speed of services and necessity for accuracy and consistency.

By embedding checklists for important verification actions, we aim to improve accuracy while also helping journalists meet deadlines in a timely fashion.  Importantly, our goal is to make these actions easy to carry out. The aim of the checklists is to encourage integrations that facilitate rapid execution of verification steps.  For instance, when determining the veracity of a photo posted on a certain date, journalists can use our checklist to quickly check the weather for that day at that location. To the greatest extent possible, we aim to reduce the number of manual actions a journalist must take to verify a report.

Another important obstacle will be technical implementation. We anticipate two important technical challenges: security and third party integrations. With existing implementations of Checkdesk, we have faced problems w DDOS attacks and spam bots.  Historically, we have addressed those with known security filters, such as Captcha, HoneyPot and Cloudflare.  We also anticipate exploring other security programs like Google's Project Shield, but we will need to address each partner’s needs on a case by case basis.  We will also need to explore third party integrations, which will vary depending on the different types of source content. Some integrations will allow for API level integrations, others may require more creative technical solutions, and others may require walking journalists through more manual steps.
How much do you think your project will cost, and what are the major expenses?
We anticipate the project will cost approximately $350,000 to develop and implement. The major expenses we anticipate around the software include design, development, quality assurance testing, usability testing, product management and security.  In the area of partnerships, we expect costs related to outreach to potential journalistic outlets, and we hope to fund the work of a third-party academic institution in evaluating the impact and efficacy of the software implementation.
How will you develop a set of tools that are useful in short time cycle situations, such as breaking news?
Checkdesk is designed to address one of the crucial challenges of breaking news in the digital age: the balance between speed and accuracy. For a major breaking news story happening almost anywhere in the world, it is now common to see content emerging almost instantaneously online, posted by onlookers and citizen journalists. Consider Hurricane Sandy, the ongoing conflict in Syria, and the Boston Marathon bombing. In each of these cases, journalists are faced with a barrage of content, some of which may be useful but much of which may be false - indeed, Gupta et al analyzed that 29% of the most viral content spread on Twitter during the Boston crisis was rumor and fake content.

Checkdesk’s checklist provides ready-to-go steps that a journalist can take to verify some of that content and powerful integrations to perform and record those steps quickly. The software provides tools that can quickly help analyze social content and give that content a clearly-marked and evolving verification status. Readers and journalists alike would see content from social media that is clearly marked as “Verification in progress” accompanied by the various steps that newsrooms have taken to verify the content, and which updates dynamically to show the latest information available.

In ONE sentence, tell us about your project to strengthen the Internet for free expression and innovation.

Verifying breaking news information and debunking rumors in emergencies and conflicts saves lives, and with Checkdesk’s Verification Toolkit we aim to provide both veteran journalists and budding citizen journalists with the tools they need to effectively verify user generated content in critical situations.

Who will benefit from what you propose? What have you observed that makes you think that?

Checkdesk benefits:
1) Digital and citizen journalists >> By providing enhanced, open source tools Checkdesk will better equip journalists and citizen journalists to verify digital media quickly and efficiently. This is a major challenge for many newsrooms and digital journalists, as is evidenced by the chaotic reporting witnessed during times of crisis or breaking news stories, and the all too common social media hoaxes that lead to confusion, damaged reputations, or heightened tensions in emergencies or conflicts; and
2) Media audiences and emergency responders >> The public, and emergency responders, benefit from improved accuracy and reliability in media coverage, which is especially valuable in breaking news and emergency situations (Hurricane Sandy, Boston marathon bombing, Japan earthquake).

What progress have you made so far?

We have equipped six leading news publishers and grassroots media collectives in five countries across the Middle East with a proof-of-concept for Checkdesk, funded by Sida, the Arab Partnerships Fund, and the International Press Institute. Built in Drupal, the platform as it stands enables you to gather and embed rich social media content into a virtual ‘desk’, cross-reference it for consistency, and annotate that content with a verification stamp. For the next iteration, we have a product team in place furnished with a compelling product roadmap that will make the app more open access and more mobile, to support a broader range of journalists.

What would be a successful outcome for your idea or project?

Success means:
1) Journalists at media companies around the world using Checkdesk to verify links and debunk misinformation online for the benefit of readers; and
2) Developers building on Checkdesk to add their own tools and checklists.

Who is on your team, and what are their relevant experiences or skills?

Meedan’s team consists of developers, designers, journalists, researchers and project managers who are all passionate about supporting collaboration between newsroom and technology innovators. Since 2011 we have worked with many of the Middle East’s leading newsrooms to implement the Checkdesk project, which in 2012 won the International Press Institute’s News Innovation Challenge. In 2014 we contributed a case study to the Verification Handbook, and are presently translating the book into Arabic as part of our ongoing support for providing high quality training resources for journalists and citizen journalists in the Middle East. Beyond the Meedan team, we also have a dedicated network of partner organizations that are motivated and committed to defining and implementing rigorous and ethical online journalism.


San Francisco, CA, USA // Cairo, Egypt


Join the conversation:

Photo of Tom

Amazing insights here into approaches to UGC verification taken in some newsrooms:

"On the issue of verification though, Claire said that she was “shocked” when she was doing her interviews about how lax the approach was. She was still hearing from grizzled veterans that “gut instinct” was important when verifying information, and they wouldn’t be using even the simplest techniques like reverse image search. Fergus said the tools to verify UGC shouldn’t slow down a story if they are properly integrated into your workflows. As he pointed out: “The quickest verification on a breaking news story happens when someone still has the device in their hand.”

Claire did report one anonymous interview subject saying that people “groan” in the office when the topic of verification comes up. It seems so odd – you wouldn’t dream of not verifying other types of journalistic information. One of the problems she identified is that most news managers these days won’t have been on a breaking news desk for five years, as they’ve moved ranks up the organisation. Which means they’ve never experienced running a breaking news desk in the social media era."

View all comments