The Knight News Challenge accelerates media innovation by funding breakthrough ideas in news and information.

641 of 646

Opening up research proposals

Research proposals are among the most closely guarded secrets in academia. If instead they were public, they would stimulate global collaboration, help funders find and connect with under-valued research, enable journalists to follow a broader arc of the research process — and much more, limited only by the imagination of Internet users and developers. To stimulate change, we want to build and populate a platform for sharing, tweaking, aggregating and archiving actionable research proposals. We will work with pioneers who have already shared research proposals publicly, and with funders sharing our interest in better proposals. We will conduct research to understand the potential for opening research proposals and the impact of doing so.

Photo of Daniel Mietchen

Written by

In a nutshell
Many ideas are lost in the current closed system, and so are opportunities to collaborate and refine those few that are actually being worked on. We propose to elaborate mechanisms that would allow a transition from the current secretive model to one in which sharing research ideas is the default and seen as an invitation for collaboration, for accelerating and improving research rather than as a breach of private property.

Specifically, we plan to
(1) build a platform for sharing, tweaking, aggregating, archiving and querying actionable research proposals,
(2) demonstrate the value of public proposals by way of pilot projects that make use of proposals available through that platform to experiment with new approaches to

(2a) research funding mechanisms,
(2b) building research teams and organizing collaboration,
(2c) knowledge discovery,
(2d) science journalism and public engagement with research.

(3) survey researchers, research funders, educators, science communicators and other stakeholders - including the public - for their attitudes towards open research proposals, in order to inform decision-making within the project,
(4) engage research funders in

(4a) releasing proposals into the open,
(4b) issuing specific funding calls that require open proposals,
(4c) funding proposals posted on the site,
(4d) funding research into the impact of open proposals on research funding,

(5) engage with research institutions about adapting confidentiality  regulations to open proposals,
(6) engage with research assessment exercises about the value of open proposals.

In the following, we will briefly discuss these points and then put them in a wider perspecive through some concluding remarks.

The text of this proposal is available under the Creative Commons Attribution 4.0 License. The embedded media are licensed differently.

(1) A platform for actionable research proposals
Back in 1959, psychologist Myron Brender wrote
"I propose the creation [..] of a newsletter or journal to be devoted exclusively to the publication of unexecuted research proposals."

Since then, the ways of publishing newsletters or journals have evolved considerably, to the point that one could say "there is but one journal: the scientific literature". However, there is still no mechanism to share such unexecuted research. We want to prototype that.

The platform we envisage would allow authors, institutions or funders to upload research proposals and associated materials like reviews or presentations. It would also allow to draft proposals as well as to fork them, and it would have a harvester component that crawls the Web for research proposals, indexes them and - copyright permitting - uploads them to the platform and text mines them.

All research proposals - including the ones that are only indexed - can be commented upon. These comments - along with research proposals and external reviews (which could be author-solicited) - contribute to an on-site reputation system inspired by the ones in use at Stack Exchange or Publons.

Visitors of the site can read every public proposal and will have an option to see "related proposals" and "related papers", possibly along with "related people", "related institutions" or "related funders", all generated on the basis of the text mining. The recommendation engine will be designed as a plugin for easy deployment on external sites (e.g. publishers or repositories).

Every public proposal will receive a DOI and automatically display bibliographic metadata in popular formats, so as to facilitate citation and proper attribution.

The default license for proposals, associated materials and comments will be the Creative Commons Attribution 4.0 License (CC BY 4.0), indicated in both a human- and machine-readable fashion. Public Domain dedications are encouraged.

The platform would have an API that would provide all of the public site content plus some structured representation of the text mining results.

The OI Engine behind this very News Challenge could be a good starting point (if it were open), with some tweaks to support for the features listed above. An open option that also covers many of the suggested features is Invenio, the engine behind the CERN Document Server and its more public-facing sister platform Zenodo. Another open option that is currently under development is the engine for running the Open Journal of Astrophysics on top of arXiv.

(2) Pilot projects
The current reward systems around scholarly research are geared heavily against sharing research results (especially early on), so we plan to experiment with incentives that favor sharing. Specifically, we plan a number of pilot projects in order to stimulate innovation around specific aspects of research proposals that may benefit from more openness.

(2a) There are multiple ways in which traditional research funders could facilitate the uptake of open research proposals. For instance, they could make existing ones public, support research into open proposals, issue calls for open proposals, or provide additional benefits to projects, researchers or institutions making their proposals open. We will work with funders interested in pursuing any of these avenues of supporting open proposals, and add functionality to our site to facilitate that.

We will also seek to engage with philanthropists, foundations and thematic organisations like environmental advocacy groups to experiment with different approaches of funding. Together with each organisation, we would discuss the rules of funding grants or science prizes in view of stimulating openness. Also, we think that Corporate Social Responsibility programs can be a good match to open research proposals. Companies often look for actions that would contribute to some social good. A proposal that would gather non-material support from an environmental advocacy group could be an attractive target to a company that seeks to complement it's CSR program with societal actions. If this happens openly, many of the issues around traditional sponsorship would be considerably alleviated.

(2b) Research teams are often assembled in ways that do not necessarily optimize for the team's performance with respect to the planned research tasks. We want to strengthen that letter aspect and think that for researchers seeking support for implementing their ideas, the act of publishing research proposals would function as a call for funders and collaborators, thereby inverting and complementing classical "calls for proposals" by traditional research funders. An example that goes in this direction is an online community of European researchers who are quite openly (albeit not by sharing actionable proposals) seeking collaborators for research projects within the Horizon 2020 Framework of the European Commission.

(2c) The possibility to browse or mine proposals or to browse the results of mining would add a new and more up-to-date layer to knowledge discovery. As mentioned in point (1), we plan to work on a recommendation engine for both our platform and easy deployment elsewhere.

Another way to facilitate discoverability is to increase the citability of proposals once they are public. One way to do that would be to simply assign persistent unique identifiers to proposals, perhaps with provisions for versioning. Here, we plan to build on existing initiatives - Zenodo, for instance, has built-in support for minting DOIs, and the Force 11 Data Citation Principles as well as the code citation mechanism elaborated by Mozilla, GitHub and Figshare provide a good basis for thinking about citing research proposals.

(2d) Public research proposals would open the door for science journalism to go new ways: instead of headlines of the "researchers found out" kind once a research project has long finished, they could cover research projects from early on and highlight the process behind it, as some science bloggers already do. We see no principle reason why approaches like embedded journalism would have to remain limited to military contexts, or prohibited to report negative results. 

(3) Survey on opening up grant proposals
In order to inform the design of our platform as well as our strategies in communicating with target user communities, we will survey relevant stakeholders - including researchers, research funders (including their reviewers), research administrators, educators, science communicators, students, journalists and the public - about their views on the topic. We will pay special attention to the experiences of researchers who have already shared research proposals.

Key components of the survey will address incentive structures (past, present and future), technical and collaborative aspects as well as copyright issues around the opening of research proposals and their reviews. Expected outcomes will be a better understanding of the real and perceived risks and opportunities of making grant proposals open.

(4) Engaging with research funders
(4a) The largest aggregators of research proposals and reviews are research funders, so we plan to engage with them about releasing their corpuses, or parts thereof. Here, agreements like the one between PeerJ and Publon on sharing manuscript reviews may serve as a model.

(4b) We will also discuss with them how open proposals could be integrated with their existing portfolios or with upcoming calls for proposal, e.g. by requiring that proposals be open (like this News Challenge or the Jisc Elevator), or by giving bonuses for projects based on open proposals. The Ideas Lab concept at the National Science Foundation might be a good starting point for that.

(4c) The purpose of research proposals is to match research ideas with resources for implementing them. If funders were to use our platform as a way to navigate the "market of ideas", this would certainly incentivize authors and institutions towards making their proposals openly available. Such close contact with funders would also seem like a good basis for establishing a sustainable long-term business model for our platform. In this context, we consider funders in a very broad sense, which includes traditional research funders along with crowdfunding sites, charities, Corporate Social Responsibility projects, perhaps even some small percentage of research budgets, and others.

We plan to encourage them to explore options for co-funding (for which again this News Challenge is a good example), which would have the added benefit of minimizing double-dipping by researchers who get grants from more than one source for very similar projects.

(4d) We expect open research proposals to improve the way in which research is being done, thereby ultimately raising its efficiency. There is very little data on that, though, and we plan to pave the way to gather some data on the matter and to engage with funders about how they might support research into open research proposals.

For instance, past experience with open collaborative initiatives like the Polymath projects (covered in Michael Nielsen's talk embedded above) or the annotation of the EHEC genome suggests that problems attract open collaboration if they are too complex to be solved by individuals or existing research groups but nonetheless likely tractable if the right set of skills, knowledge and tools comes together. How that differs across research areas has yet to be studied.

(5) Institutional policies
In many research institutions, researchers are contractually limited in their freedom to make their research public without explicit approval. These limits also apply to making research proposals publicly available. We expect that a large number of research proposals may be affected by such policies. We will thus engage with research institutions across disciplinary, sectorial and territorial boundaries to quantify the nature of the problem, and to elaborate solutions towards making more proposals available to the public. Our focus here will be on institutions that are largely funded by the public, but other institutions shall be welcome to join the process.

(6) Assessing research proposals
The most comprehensive analysis to date of current practice in assessing research proposals, a 2007 Cochrane review, stated: "Experimental studies assessing the effects of grant giving peer review on importance, relevance, usefulness, soundness of methods, soundness of ethics, completeness and accuracy of funded research are urgently needed. Practices aimed to control and evaluate the potentially negative effects of peer review should be implemented meanwhile."

We want to help bridge this gap by exploring mechanisms for discovering and sharing research ideas using the Web in ways that stimulate collaboration.

Given that HEFCE are now only considering open-access publications in their research assessments, it does not seem unthinkable that the day might come when open research proposals would become the default for scholarly research.

Concluding remarks

The main challenge of implementing open proposals is not technical but cultural: researchers currently have no incentive to share research proposals, and research funders have no habit of making their funding decisions public, nor who has applied for what.

Here, it is important to keep in mind that we do not aim to switch the entire system in a binary fashion from the current closed model to an open one. Rather, we plan to identify opportunities for starting the transition, hence our emphasis on pilot projects.

The lack of transparency in traditional research funding sometimes has unintended side effects for funders as well, such as occasional strong overlaps between projects funded by different agencies, on which an observer commented "A central database for all grant proposals would be an excellent first step." - an idea that has already been implemented for cancer research. That is an important move towards our goal of making research proposals publicly available.

Another interesting aspect of this discussion is the use of Freedom of Information Act requests by journalists investigating this story who wanted to assess the similarity between some proposals that an automated tool had flagged as potentially having such strong overlap. Wouldn't it be simpler for everyone involved if they (or a script operated by them) could have just downloaded the proposals from a platform like the one we plan to build? Conversely, there may be a role for further FOI requests on the way towards opening up research proposals.

Currently, a common scenario is that a proposal was evaluated as excellent but rejected nonetheless, due to limited funds. What would be lost if these proposals and their assessments were made public and others could chip in to move the project forward?

Casino fund

In principle, research proposals could be abandoned entirely, and replaced by a system that combines some baseline research grants with a shift of focus from projects to people and with a more elaborate - and more transparent - system of science prizes for different degrees of achievements.

One suggestion relevant in this regard has been offered by Sydney Brenner in 2003 in the context of stimulating potentially transformative research of the kind that would be perceived as too risky by most traditional research funders:
"I propose that everybody who gives money for research should take 1% of it and put it into a fund which I call the casino fund, and write it off."

He is a Nobel laureate and has made this suggestion multiple times over the last decade, with no action taken to actually build such a fund. The chances to engage research funders that way are thus rather slim, but some of them are likely to be interested in improving the efficiency of their funding mechanisms. If so, they may be willing to contribute to a fund dedicated to research about ways to improve research funding, especially in terms of handling potentially transformative research.

We think that openness can contribute significantly to such improvements. For instance, some of the casino funding could be crowdsourced by allowing researchers to spend a small portion of their  grants on co-funding open research proposals evaluated as "most interesting but unfundable" by traditional means, or the public could be invited to contribute through crowdfunding platforms.

It is also worth considering that the overall environment for research communication has evolved since 2003, with researchers and the public much more exposed to sharing (including crowdfunding) and with open access to research results and data and even open science more generally being on high-level political agendas (see Neelie Kroes video embedded above).

What about being scooped?

Of course, there is the fear of getting scooped, i.e. that someone else might steal an idea worth implementing, and if they have the resources, they may get there first and reap off all the benefits.

Many have experienced this in the current system, but effective scooping is actually relatively simple now and much harder if your ideas are out there in the open: if everyone knows you were the first to propose (and actually pursue) that idea, anyone who tries to sell it as their own will risk loosing reputation, so they may actually prefer to work with rather than against you. More on that in the Harvard video embedded above.

Where to find researchers willing to collaborate openly, from early on?
Assuming (as we do) that they exist, there is no easy answer to that, and identifying conditions under which researchers would be inclined towards openly collaborative approaches is a key research component of our proposal, as described in section (3). Besides gathering data, the survey would double as an outreach tool to raise awareness of open approaches and to spur creative thinking around them, which could be fed back to the project.

As mentioned in section (4d), past experience with open collaborative initiatives suggests that problems attract open collaboration if they are too complex to be solved by individuals or existing research groups but nonetheless likely tractable if the right set of skills, knowledge and tools comes together.

With that in mind, it is worth considering that current research funding is systemically biased against proposals that do not fit into established disciplinary boundaries. In that context, computer scientist Ehud Shapiro recently wrote:
"Genuine interdisciplinary research is nothing like a competitive race. It is much more like a solitary exploratory hike through an uncharted landscape. [...] There are no peers to compete with". But there may be collaborators if they ask nicely.

Now consider how the initiator of the first Polymath project, Fields medalist Tim Gowers, described the open and collaborative nature of the project: "this process is to normal research as driving is to pushing a car."

We expect that at least some of Shapiro's solitary hikers would like to borrow Gowers' car to drive through the uncharted territory in front of them. In order to recruit open collaborators, they will have to provide them with information as to why the ride would be worth it, how to identify and reach targets and so on, which is precisely the classical function of research proposals. Sharing research before formal publication is not unheard of either, and in some fields, it has even become the norm (consider, for instance, the Bermuda Principles, or arXiv).

While our aim is to open up research proposals of any kind, we will pay special attention to ensuring that our proposed system fits the needs of research proposals that bridge across disciplines.

We are conscious that biases against other minorities exist within the research community, and we will strive to cater to their needs as well (that includes supporting languages other than English). In doing so, we will build on related experiences with open procedures in other research contexts. For instance, some scholarly journals engaged in public peer review address these issues by allowing reviewers to choose whether they want to remain anonymous, and we can imagine a similar option for authors of research proposals, although probably not for an extended period.

Apart from facilitating the publication and dissemination of new research proposals, we will work on incorporating proposals from the past - both funded and unfunded ones - into our system. As users, we do not only envisage human readers in search of grant proposals (e.g. to write their own, or to find out more about a particular approach or research group) but also machines, so we will keep an eye on providing programmatic access to the data, which may be of interest to science historians, research administrators or developers building discovery tools.

Given that funding bodies and institutions are now starting to make public detailed accounts of the money they spent on author-paid access to research publications, it seems not too unreasonable to expect that some of them may be willing to share more of their records soon, perhaps with some embargo periods that can be phased out over time.

A small number of grant proposals - both funded and unfunded ones - have already been made public at the discretion of their authors in areas like biology or mathematics. A few of them have also shared their thoughts about this in blog posts, or contributed to this proposal through comments. We plan to engage with these early adopters to help us guide in identifying and building incentive structures around that.

In ONE sentence, tell us about your project to strengthen the Internet for free expression and innovation.

We want to provide a framework in which researchers *want* to share their research ideas and proposals early on, so that they can be worked on collaboratively to the benefit of society.

Who will benefit from what you propose? What have you observed that makes you think that?

Most research proposals at most research funding agencies around the globe get rejected these days.

Researchers who share their ideas may find new collaborators and new funding opportunities, and those who collaborate with them find out about these ideas early on and can share in their implementation.

Research funders would benefit from a more efficient use of their resources, and the public could be engaged throughout the research project, rather than just at the end.

Finally, having grant proposals publicly available would be a great educational resource, not just in terms of writing proposals, but also in terms of science journalism and with regard to the history of ideas.

What progress have you made so far?

We helped draft or review a good number of research proposals in a collaborative fashion, sometimes in the open.

We are used to working in the open more generally, have experience with classical research funding as well as crowdfunding approaches and are involved in a number of activities at the frontiers of science communication, e.g. publishing journal articles to Wikipedia, publishing data, public peer review and reusing multimedia from scholarly publications in Wikimedia contexts, the latter of which actually resulted from a proposal drafted in the open.

What would be a successful outcome for your idea or project?

A public collection of actionable topical research proposals would be a success at basically any size, but a goal of getting a few dozens in the first year would seem realistic, with the first 1000 reached on a five-year frame. If a third or more of these have reviews published along, that would be a success too, and so would be open licensing of a substantial portion of this corpus. All of this requires new incentive structures, which are thus the primary target, with the platform as the primary tool.

Who is on your team, and what are their relevant experiences or skills?

At the moment, we are a team of four:
Daniel Mietchen, Museum für Naturkunde – Leibniz Institute for Research on Evolution and Biodiversity, Germany. His research interest span from 3D imaging of fossils and embryos to vocal learning in elephants, cold hardiness in insect larvae, and semantic integration of the biodiversity literature. As a volunteer, he has been involved in several open-source software projects on the interface between Wikimedia projects and scholarly research, most notably the Open Access Media Importer [1] that won one of the inaugural Accelerating Science Awards [2], as well as the Wikipedia Cite-o-Meter [3] and currently a project to signal the licensing of scholarly references cited on Wikipedia [4].
Paweł Szczęsny, Department of Bioinformatics, Institute of Biochemistry and Biophysics, Polish Academy of Sciences, Poland. Besides, Paweł is building a technical and social infrastructure for science through his NGO called The Systems Institute. His main project in this area right now is a crowdfunding and collaboration platform for open citizen science projects (to be available soon in Polish [5]).

Mike Linksvayer is an expert on commons-based policy and innovation. He advises many free software and open data projects, including serving on the boards of OpenHatch [6] and Software Freedom Conservancy. Previously [7], he served as CTO at Creative Commons [8] and co-founded Bitzi, an early (2001) mass collaboration/open data service [9].

Paul Gardner, Biomolecular Interactions Centre, School of Biological Sciences, Canterbury University, Christchurch, New Zealand. Paul ran the Rfam database [10] of non-coding RNA families for four years. During this tenure, he initiated the use of Wikipedia entries for biological database curation. The experiment was so successful that it has been imitated by several other biological databases (e.g. Pfam and miRBase). Paul continues to contribute to the database and plays a role as Editor in Chief for the journal RNA Biology [11] and Software Editor for PLOS Computational Biology [12]. He is a bioinformatics lecturer and one of the inaugural recipients of a Rutherford Discovery Fellowship, New Zealand's highest honour for early career academics.

Daniel is a biophysicist, Paweł is a biologist, Mike has a background in economics and software development, and Paul is a Bioinformatician. We all have been involved with open source, open access and open science for several years.

Note that the team has been assembled on the basis of this proposal being public.



Daniel is based in Jena, Germany, Paweł in Warsaw, Poland, Mike in Oakland, California, and Paul in Christchurch, New Zealand. All four of us are experienced in collaborating with researchers across disciplinary, national and sectorial boundaries.
9.1k 43 16


Join the conversation:

Photo of Markus

Your idea is good. Let me give some comments from the perspective of a possible future user of the platform, i.e., researcher applying for funding.

In my experience getting funding is a rather noisy process. That is, success of an application might not be completely random, but it does have a very very large random component. Therefore, should my carefully crafted application not be successful in Call for Applications A, a sensible thing for me to do (in the current system) would not be to make it public, but to submit a tuned version to Call for Applications B. Upon each submission I will have, say, a 10% success rate and thus after enough repetitions I will get some funding from somewhere.

The point I wish to make is that researchers might not be very keen on making their unsuccesfull applications public unless you are able to convince them that this approach is a more certain way to fund their work than repeated submission with, say, 10% likelihood. The Casion Fund, for example, seems to have a lot worse odds than a repeated submission.

The successful applications, however, are a completely different story. Here the funding agencies should even demand that the applications are made public. When it comes to the fashionable multi-disciplinary and multi-national applications, the demand should even be that the application be turned to a public wiki, in which the participants collaborate in solving the problem(s) they claimed to solve in the application. This would be a great improvement in transparency, for example, on the current EU grants (and the SFB grants here in Germany). An openly viewable wiki would really make it clear that the claimed collaborations within the consortium are taking place in the actual day to day research.

Coming back to the unsuccesful applications. If you wish to make these public, maybe the best approach would be the one in your point 4b. That is, making the applications public _before_ the decisions are made. This would allow for openly elaborating the applications before final submission, finding interested collaborations, and so on. All this could happen on your platform. And, as everything was public from the beginning on, after the decisions come I would not need to choose if I would like to make my unsuccessful application public. It would be out there already.

Photo of Daniel

I totally agree with all of that:
Yes, in most cases, resubmission is possible but time and effort is always lost. In some cases, resubmission is not possible (e.g. due to some age limits or temporary funding line), which may be another area for having a closer look.

Yes, funders should have at least some funding lines in which proposals are only considered when public.

Yes, "making the applications public _before_ the decisions are made" is the key element of the proposal.

View all comments