Opening up research proposals
In a nutshell
Many ideas are lost in the current closed system, and so are opportunities to collaborate and refine those few that are actually being worked on. We propose to elaborate mechanisms that would allow a transition from the current secretive model to one in which sharing research ideas is the default and seen as an invitation for collaboration, for accelerating and improving research rather than as a breach of private property.
Specifically, we plan to
(1) build a platform for sharing, tweaking, aggregating, archiving and querying actionable research proposals,
(2) demonstrate the value of public proposals by way of pilot projects that make use of proposals available through that platform to experiment with new approaches to
(2a) research funding mechanisms,
(2b) building research teams and organizing collaboration,
(2c) knowledge discovery,
(2d) science journalism and public engagement with research.
(3) survey researchers, research funders, educators, science communicators and other stakeholders - including the public - for their attitudes towards open research proposals, in order to inform decision-making within the project,
(4) engage research funders in
(4a) releasing proposals into the open,
(4b) issuing specific funding calls that require open proposals,
(4c) funding proposals posted on the site,
(4d) funding research into the impact of open proposals on research funding,
(5) engage with research institutions about adapting confidentiality regulations to open proposals,
(6) engage with research assessment exercises about the value of open proposals.
In the following, we will briefly discuss these points and then put them in a wider perspecive through some concluding remarks.
The text of this proposal is available under the Creative Commons Attribution 4.0 License. The embedded media are licensed differently.
(1) A platform for actionable research proposals
Back in 1959, psychologist Myron Brender wrote
"I propose the creation [..] of a newsletter or journal to be devoted exclusively to the publication of unexecuted research proposals."
Since then, the ways of publishing newsletters or journals have evolved considerably, to the point that one could say "there is but one journal: the scientific literature". However, there is still no mechanism to share such unexecuted research. We want to prototype that.
The platform we envisage would allow authors, institutions or funders to upload research proposals and associated materials like reviews or presentations. It would also allow to draft proposals as well as to fork them, and it would have a harvester component that crawls the Web for research proposals, indexes them and - copyright permitting - uploads them to the platform and text mines them.
All research proposals - including the ones that are only indexed - can be commented upon. These comments - along with research proposals and external reviews (which could be author-solicited) - contribute to an on-site reputation system inspired by the ones in use at Stack Exchange or Publons.
Visitors of the site can read every public proposal and will have an option to see "related proposals" and "related papers", possibly along with "related people", "related institutions" or "related funders", all generated on the basis of the text mining. The recommendation engine will be designed as a plugin for easy deployment on external sites (e.g. publishers or repositories).
Every public proposal will receive a DOI and automatically display bibliographic metadata in popular formats, so as to facilitate citation and proper attribution.
The default license for proposals, associated materials and comments will be the Creative Commons Attribution 4.0 License (CC BY 4.0), indicated in both a human- and machine-readable fashion. Public Domain dedications are encouraged.
The platform would have an API that would provide all of the public site content plus some structured representation of the text mining results.
The OI Engine behind this very News Challenge could be a good starting point (if it were open), with some tweaks to support for the features listed above. An open option that also covers many of the suggested features is Invenio, the engine behind the CERN Document Server and its more public-facing sister platform Zenodo. Another open option that is currently under development is the engine for running the Open Journal of Astrophysics on top of arXiv.
(2) Pilot projects
The current reward systems around scholarly research are geared heavily against sharing research results (especially early on), so we plan to experiment with incentives that favor sharing. Specifically, we plan a number of pilot projects in order to stimulate innovation around specific aspects of research proposals that may benefit from more openness.
(2a) There are multiple ways in which traditional research funders could facilitate the uptake of open research proposals. For instance, they could make existing ones public, support research into open proposals, issue calls for open proposals, or provide additional benefits to projects, researchers or institutions making their proposals open. We will work with funders interested in pursuing any of these avenues of supporting open proposals, and add functionality to our site to facilitate that.
We will also seek to engage with philanthropists, foundations and thematic organisations like environmental advocacy groups to experiment with different approaches of funding. Together with each organisation, we would discuss the rules of funding grants or science prizes in view of stimulating openness. Also, we think that Corporate Social Responsibility programs can be a good match to open research proposals. Companies often look for actions that would contribute to some social good. A proposal that would gather non-material support from an environmental advocacy group could be an attractive target to a company that seeks to complement it's CSR program with societal actions. If this happens openly, many of the issues around traditional sponsorship would be considerably alleviated.
(2b) Research teams are often assembled in ways that do not necessarily optimize for the team's performance with respect to the planned research tasks. We want to strengthen that letter aspect and think that for researchers seeking support for implementing their ideas, the act of publishing research proposals would function as a call for funders and collaborators, thereby inverting and complementing classical "calls for proposals" by traditional research funders. An example that goes in this direction is an online community of European researchers who are quite openly (albeit not by sharing actionable proposals) seeking collaborators for research projects within the Horizon 2020 Framework of the European Commission.
(2c) The possibility to browse or mine proposals or to browse the results of mining would add a new and more up-to-date layer to knowledge discovery. As mentioned in point (1), we plan to work on a recommendation engine for both our platform and easy deployment elsewhere.
Another way to facilitate discoverability is to increase the citability of proposals once they are public. One way to do that would be to simply assign persistent unique identifiers to proposals, perhaps with provisions for versioning. Here, we plan to build on existing initiatives - Zenodo, for instance, has built-in support for minting DOIs, and the Force 11 Data Citation Principles as well as the code citation mechanism elaborated by Mozilla, GitHub and Figshare provide a good basis for thinking about citing research proposals.
(2d) Public research proposals would open the door for science journalism to go new ways: instead of headlines of the "researchers found out" kind once a research project has long finished, they could cover research projects from early on and highlight the process behind it, as some science bloggers already do. We see no principle reason why approaches like embedded journalism would have to remain limited to military contexts, or prohibited to report negative results.
(3) Survey on opening up grant proposals
In order to inform the design of our platform as well as our strategies in communicating with target user communities, we will survey relevant stakeholders - including researchers, research funders (including their reviewers), research administrators, educators, science communicators, students, journalists and the public - about their views on the topic. We will pay special attention to the experiences of researchers who have already shared research proposals.
Key components of the survey will address incentive structures (past, present and future), technical and collaborative aspects as well as copyright issues around the opening of research proposals and their reviews. Expected outcomes will be a better understanding of the real and perceived risks and opportunities of making grant proposals open.
(4) Engaging with research funders
(4a) The largest aggregators of research proposals and reviews are research funders, so we plan to engage with them about releasing their corpuses, or parts thereof. Here, agreements like the one between PeerJ and Publon on sharing manuscript reviews may serve as a model.
(4b) We will also discuss with them how open proposals could be integrated with their existing portfolios or with upcoming calls for proposal, e.g. by requiring that proposals be open (like this News Challenge or the Jisc Elevator), or by giving bonuses for projects based on open proposals. The Ideas Lab concept at the National Science Foundation might be a good starting point for that.
(4c) The purpose of research proposals is to match research ideas with resources for implementing them. If funders were to use our platform as a way to navigate the "market of ideas", this would certainly incentivize authors and institutions towards making their proposals openly available. Such close contact with funders would also seem like a good basis for establishing a sustainable long-term business model for our platform. In this context, we consider funders in a very broad sense, which includes traditional research funders along with crowdfunding sites, charities, Corporate Social Responsibility projects, perhaps even some small percentage of research budgets, and others.
We plan to encourage them to explore options for co-funding (for which again this News Challenge is a good example), which would have the added benefit of minimizing double-dipping by researchers who get grants from more than one source for very similar projects.
(4d) We expect open research proposals to improve the way in which research is being done, thereby ultimately raising its efficiency. There is very little data on that, though, and we plan to pave the way to gather some data on the matter and to engage with funders about how they might support research into open research proposals.
For instance, past experience with open collaborative initiatives like the Polymath projects (covered in Michael Nielsen's talk embedded above) or the annotation of the EHEC genome suggests that problems attract open collaboration if they are too complex to be solved by individuals or existing research groups but nonetheless likely tractable if the right set of skills, knowledge and tools comes together. How that differs across research areas has yet to be studied.
(5) Institutional policies
In many research institutions, researchers are contractually limited in their freedom to make their research public without explicit approval. These limits also apply to making research proposals publicly available. We expect that a large number of research proposals may be affected by such policies. We will thus engage with research institutions across disciplinary, sectorial and territorial boundaries to quantify the nature of the problem, and to elaborate solutions towards making more proposals available to the public. Our focus here will be on institutions that are largely funded by the public, but other institutions shall be welcome to join the process.
(6) Assessing research proposals
The most comprehensive analysis to date of current practice in assessing research proposals, a 2007 Cochrane review, stated: "Experimental studies assessing the effects of grant giving peer review on importance, relevance, usefulness, soundness of methods, soundness of ethics, completeness and accuracy of funded research are urgently needed. Practices aimed to control and evaluate the potentially negative effects of peer review should be implemented meanwhile."
We want to help bridge this gap by exploring mechanisms for discovering and sharing research ideas using the Web in ways that stimulate collaboration.
Given that HEFCE are now only considering open-access publications in their research assessments, it does not seem unthinkable that the day might come when open research proposals would become the default for scholarly research.
Here, it is important to keep in mind that we do not aim to switch the entire system in a binary fashion from the current closed model to an open one. Rather, we plan to identify opportunities for starting the transition, hence our emphasis on pilot projects.
The lack of transparency in traditional research funding sometimes has unintended side effects for funders as well, such as occasional strong overlaps between projects funded by different agencies, on which an observer commented "A central database for all grant proposals would be an excellent first step." - an idea that has already been implemented for cancer research. That is an important move towards our goal of making research proposals publicly available.
Another interesting aspect of this discussion is the use of Freedom of Information Act requests by journalists investigating this story who wanted to assess the similarity between some proposals that an automated tool had flagged as potentially having such strong overlap. Wouldn't it be simpler for everyone involved if they (or a script operated by them) could have just downloaded the proposals from a platform like the one we plan to build? Conversely, there may be a role for further FOI requests on the way towards opening up research proposals.
Currently, a common scenario is that a proposal was evaluated as excellent but rejected nonetheless, due to limited funds. What would be lost if these proposals and their assessments were made public and others could chip in to move the project forward?
Casino fundIn principle, research proposals could be abandoned entirely, and replaced by a system that combines some baseline research grants with a shift of focus from projects to people and with a more elaborate - and more transparent - system of science prizes for different degrees of achievements.
One suggestion relevant in this regard has been offered by Sydney Brenner in 2003 in the context of stimulating potentially transformative research of the kind that would be perceived as too risky by most traditional research funders:
"I propose that everybody who gives money for research should take 1% of it and put it into a fund which I call the casino fund, and write it off."
He is a Nobel laureate and has made this suggestion multiple times over the last decade, with no action taken to actually build such a fund. The chances to engage research funders that way are thus rather slim, but some of them are likely to be interested in improving the efficiency of their funding mechanisms. If so, they may be willing to contribute to a fund dedicated to research about ways to improve research funding, especially in terms of handling potentially transformative research.
We think that openness can contribute significantly to such improvements. For instance, some of the casino funding could be crowdsourced by allowing researchers to spend a small portion of their grants on co-funding open research proposals evaluated as "most interesting but unfundable" by traditional means, or the public could be invited to contribute through crowdfunding platforms.
It is also worth considering that the overall environment for research communication has evolved since 2003, with researchers and the public much more exposed to sharing (including crowdfunding) and with open access to research results and data and even open science more generally being on high-level political agendas (see Neelie Kroes video embedded above).
What about being scooped?Of course, there is the fear of getting scooped, i.e. that someone else might steal an idea worth implementing, and if they have the resources, they may get there first and reap off all the benefits.
Many have experienced this in the current system, but effective scooping is actually relatively simple now and much harder if your ideas are out there in the open: if everyone knows you were the first to propose (and actually pursue) that idea, anyone who tries to sell it as their own will risk loosing reputation, so they may actually prefer to work with rather than against you. More on that in the Harvard video embedded above.
Where to find researchers willing to collaborate openly, from early on?
Assuming (as we do) that they exist, there is no easy answer to that, and identifying conditions under which researchers would be inclined towards openly collaborative approaches is a key research component of our proposal, as described in section (3). Besides gathering data, the survey would double as an outreach tool to raise awareness of open approaches and to spur creative thinking around them, which could be fed back to the project.
As mentioned in section (4d), past experience with open collaborative initiatives suggests that problems attract open collaboration if they are too complex to be solved by individuals or existing research groups but nonetheless likely tractable if the right set of skills, knowledge and tools comes together.
With that in mind, it is worth considering that current research funding is systemically biased against proposals that do not fit into established disciplinary boundaries. In that context, computer scientist Ehud Shapiro recently wrote:
"Genuine interdisciplinary research is nothing like a competitive race. It is much more like a solitary exploratory hike through an uncharted landscape. [...] There are no peers to compete with". But there may be collaborators if they ask nicely.
Now consider how the initiator of the first Polymath project, Fields medalist Tim Gowers, described the open and collaborative nature of the project: "this process is to normal research as driving is to pushing a car."
We expect that at least some of Shapiro's solitary hikers would like to borrow Gowers' car to drive through the uncharted territory in front of them. In order to recruit open collaborators, they will have to provide them with information as to why the ride would be worth it, how to identify and reach targets and so on, which is precisely the classical function of research proposals. Sharing research before formal publication is not unheard of either, and in some fields, it has even become the norm (consider, for instance, the Bermuda Principles, or arXiv).
While our aim is to open up research proposals of any kind, we will pay special attention to ensuring that our proposed system fits the needs of research proposals that bridge across disciplines.
We are conscious that biases against other minorities exist within the research community, and we will strive to cater to their needs as well (that includes supporting languages other than English). In doing so, we will build on related experiences with open procedures in other research contexts. For instance, some scholarly journals engaged in public peer review address these issues by allowing reviewers to choose whether they want to remain anonymous, and we can imagine a similar option for authors of research proposals, although probably not for an extended period.
Apart from facilitating the publication and dissemination of new research proposals, we will work on incorporating proposals from the past - both funded and unfunded ones - into our system. As users, we do not only envisage human readers in search of grant proposals (e.g. to write their own, or to find out more about a particular approach or research group) but also machines, so we will keep an eye on providing programmatic access to the data, which may be of interest to science historians, research administrators or developers building discovery tools.
Given that funding bodies and institutions are now starting to make public detailed accounts of the money they spent on author-paid access to research publications, it seems not too unreasonable to expect that some of them may be willing to share more of their records soon, perhaps with some embargo periods that can be phased out over time.
A small number of grant proposals - both funded and unfunded ones - have already been made public at the discretion of their authors in areas like biology or mathematics. A few of them have also shared their thoughts about this in blog posts, or contributed to this proposal through comments. We plan to engage with these early adopters to help us guide in identifying and building incentive structures around that.