The Challenge

3785 followers

How can we strengthen the Internet for free expression and innovation? read the brief

Entry

Opening up research proposals

Research proposals are among the most closely guarded secrets in academia. If instead they were public, they would stimulate global collaboration, help funders find and connect with under-valued research, enable journalists to follow a broader arc of the research process — and much more, limited only by the imagination of Internet users and developers. To stimulate change, we want to build and populate a platform for sharing, tweaking, aggregating and archiving actionable research proposals. We will work with pioneers who have already shared research proposals publicly, and with funders sharing our interest in better proposals. We will conduct research to understand the potential for opening research proposals and the impact of doing so.


In a nutshell
Many ideas are lost in the current closed system, and so are opportunities to collaborate and refine those few that are actually being worked on. We propose to elaborate mechanisms that would allow a transition from the current secretive model to one in which sharing research ideas is the default and seen as an invitation for collaboration, for accelerating and improving research rather than as a breach of private property.

Specifically, we plan to
(1) build a platform for sharing, tweaking, aggregating, archiving and querying actionable research proposals,
(2) demonstrate the value of public proposals by way of pilot projects that make use of proposals available through that platform to experiment with new approaches to

(2a) research funding mechanisms,
(2b) building research teams and organizing collaboration,
(2c) knowledge discovery,
(2d) science journalism and public engagement with research.

(3) survey researchers, research funders, educators, science communicators and other stakeholders - including the public - for their attitudes towards open research proposals, in order to inform decision-making within the project,
(4) engage research funders in

(4a) releasing proposals into the open,
(4b) issuing specific funding calls that require open proposals,
(4c) funding proposals posted on the site,
(4d) funding research into the impact of open proposals on research funding,

(5) engage with research institutions about adapting confidentiality  regulations to open proposals,
(6) engage with research assessment exercises about the value of open proposals.

In the following, we will briefly discuss these points and then put them in a wider perspecive through some concluding remarks.

The text of this proposal is available under the Creative Commons Attribution 4.0 License. The embedded media are licensed differently.
 


(1) A platform for actionable research proposals
Back in 1959, psychologist Myron Brender wrote
"I propose the creation [..] of a newsletter or journal to be devoted exclusively to the publication of unexecuted research proposals."

Since then, the ways of publishing newsletters or journals have evolved considerably, to the point that one could say "there is but one journal: the scientific literature". However, there is still no mechanism to share such unexecuted research. We want to prototype that.

The platform we envisage would allow authors, institutions or funders to upload research proposals and associated materials like reviews or presentations. It would also allow to draft proposals as well as to fork them, and it would have a harvester component that crawls the Web for research proposals, indexes them and - copyright permitting - uploads them to the platform and text mines them.

All research proposals - including the ones that are only indexed - can be commented upon. These comments - along with research proposals and external reviews (which could be author-solicited) - contribute to an on-site reputation system inspired by the ones in use at Stack Exchange or Publons.

Visitors of the site can read every public proposal and will have an option to see "related proposals" and "related papers", possibly along with "related people", "related institutions" or "related funders", all generated on the basis of the text mining. The recommendation engine will be designed as a plugin for easy deployment on external sites (e.g. publishers or repositories).

Every public proposal will receive a DOI and automatically display bibliographic metadata in popular formats, so as to facilitate citation and proper attribution.

The default license for proposals, associated materials and comments will be the Creative Commons Attribution 4.0 License (CC BY 4.0), indicated in both a human- and machine-readable fashion. Public Domain dedications are encouraged.

The platform would have an API that would provide all of the public site content plus some structured representation of the text mining results.

The OI Engine behind this very News Challenge could be a good starting point (if it were open), with some tweaks to support for the features listed above. An open option that also covers many of the suggested features is Invenio, the engine behind the CERN Document Server and its more public-facing sister platform Zenodo. Another open option that is currently under development is the engine for running the Open Journal of Astrophysics on top of arXiv.

(2) Pilot projects
The current reward systems around scholarly research are geared heavily against sharing research results (especially early on), so we plan to experiment with incentives that favor sharing. Specifically, we plan a number of pilot projects in order to stimulate innovation around specific aspects of research proposals that may benefit from more openness.

(2a) There are multiple ways in which traditional research funders could facilitate the uptake of open research proposals. For instance, they could make existing ones public, support research into open proposals, issue calls for open proposals, or provide additional benefits to projects, researchers or institutions making their proposals open. We will work with funders interested in pursuing any of these avenues of supporting open proposals, and add functionality to our site to facilitate that.

We will also seek to engage with philanthropists, foundations and thematic organisations like environmental advocacy groups to experiment with different approaches of funding. Together with each organisation, we would discuss the rules of funding grants or science prizes in view of stimulating openness. Also, we think that Corporate Social Responsibility programs can be a good match to open research proposals. Companies often look for actions that would contribute to some social good. A proposal that would gather non-material support from an environmental advocacy group could be an attractive target to a company that seeks to complement it's CSR program with societal actions. If this happens openly, many of the issues around traditional sponsorship would be considerably alleviated.

(2b) Research teams are often assembled in ways that do not necessarily optimize for the team's performance with respect to the planned research tasks. We want to strengthen that letter aspect and think that for researchers seeking support for implementing their ideas, the act of publishing research proposals would function as a call for funders and collaborators, thereby inverting and complementing classical "calls for proposals" by traditional research funders. An example that goes in this direction is an online community of European researchers who are quite openly (albeit not by sharing actionable proposals) seeking collaborators for research projects within the Horizon 2020 Framework of the European Commission.

(2c) The possibility to browse or mine proposals or to browse the results of mining would add a new and more up-to-date layer to knowledge discovery. As mentioned in point (1), we plan to work on a recommendation engine for both our platform and easy deployment elsewhere.

Another way to facilitate discoverability is to increase the citability of proposals once they are public. One way to do that would be to simply assign persistent unique identifiers to proposals, perhaps with provisions for versioning. Here, we plan to build on existing initiatives - Zenodo, for instance, has built-in support for minting DOIs, and the Force 11 Data Citation Principles as well as the code citation mechanism elaborated by Mozilla, GitHub and Figshare provide a good basis for thinking about citing research proposals.

(2d) Public research proposals would open the door for science journalism to go new ways: instead of headlines of the "researchers found out" kind once a research project has long finished, they could cover research projects from early on and highlight the process behind it, as some science bloggers already do. We see no principle reason why approaches like embedded journalism would have to remain limited to military contexts, or prohibited to report negative results. 

(3) Survey on opening up grant proposals
In order to inform the design of our platform as well as our strategies in communicating with target user communities, we will survey relevant stakeholders - including researchers, research funders (including their reviewers), research administrators, educators, science communicators, students, journalists and the public - about their views on the topic. We will pay special attention to the experiences of researchers who have already shared research proposals.

Key components of the survey will address incentive structures (past, present and future), technical and collaborative aspects as well as copyright issues around the opening of research proposals and their reviews. Expected outcomes will be a better understanding of the real and perceived risks and opportunities of making grant proposals open.

(4) Engaging with research funders
(4a) The largest aggregators of research proposals and reviews are research funders, so we plan to engage with them about releasing their corpuses, or parts thereof. Here, agreements like the one between PeerJ and Publon on sharing manuscript reviews may serve as a model.

(4b) We will also discuss with them how open proposals could be integrated with their existing portfolios or with upcoming calls for proposal, e.g. by requiring that proposals be open (like this News Challenge or the Jisc Elevator), or by giving bonuses for projects based on open proposals. The Ideas Lab concept at the National Science Foundation might be a good starting point for that.

(4c) The purpose of research proposals is to match research ideas with resources for implementing them. If funders were to use our platform as a way to navigate the "market of ideas", this would certainly incentivize authors and institutions towards making their proposals openly available. Such close contact with funders would also seem like a good basis for establishing a sustainable long-term business model for our platform. In this context, we consider funders in a very broad sense, which includes traditional research funders along with crowdfunding sites, charities, Corporate Social Responsibility projects, perhaps even some small percentage of research budgets, and others.

We plan to encourage them to explore options for co-funding (for which again this News Challenge is a good example), which would have the added benefit of minimizing double-dipping by researchers who get grants from more than one source for very similar projects.

(4d) We expect open research proposals to improve the way in which research is being done, thereby ultimately raising its efficiency. There is very little data on that, though, and we plan to pave the way to gather some data on the matter and to engage with funders about how they might support research into open research proposals.

For instance, past experience with open collaborative initiatives like the Polymath projects (covered in Michael Nielsen's talk embedded above) or the annotation of the EHEC genome suggests that problems attract open collaboration if they are too complex to be solved by individuals or existing research groups but nonetheless likely tractable if the right set of skills, knowledge and tools comes together. How that differs across research areas has yet to be studied.


(5) Institutional policies
In many research institutions, researchers are contractually limited in their freedom to make their research public without explicit approval. These limits also apply to making research proposals publicly available. We expect that a large number of research proposals may be affected by such policies. We will thus engage with research institutions across disciplinary, sectorial and territorial boundaries to quantify the nature of the problem, and to elaborate solutions towards making more proposals available to the public. Our focus here will be on institutions that are largely funded by the public, but other institutions shall be welcome to join the process.
 

(6) Assessing research proposals
The most comprehensive analysis to date of current practice in assessing research proposals, a 2007 Cochrane review, stated: "Experimental studies assessing the effects of grant giving peer review on importance, relevance, usefulness, soundness of methods, soundness of ethics, completeness and accuracy of funded research are urgently needed. Practices aimed to control and evaluate the potentially negative effects of peer review should be implemented meanwhile."

We want to help bridge this gap by exploring mechanisms for discovering and sharing research ideas using the Web in ways that stimulate collaboration.

Given that HEFCE are now only considering open-access publications in their research assessments, it does not seem unthinkable that the day might come when open research proposals would become the default for scholarly research.


Concluding remarks

The main challenge of implementing open proposals is not technical but cultural: researchers currently have no incentive to share research proposals, and research funders have no habit of making their funding decisions public, nor who has applied for what.

Here, it is important to keep in mind that we do not aim to switch the entire system in a binary fashion from the current closed model to an open one. Rather, we plan to identify opportunities for starting the transition, hence our emphasis on pilot projects.

The lack of transparency in traditional research funding sometimes has unintended side effects for funders as well, such as occasional strong overlaps between projects funded by different agencies, on which an observer commented "A central database for all grant proposals would be an excellent first step." - an idea that has already been implemented for cancer research. That is an important move towards our goal of making research proposals publicly available.

Another interesting aspect of this discussion is the use of Freedom of Information Act requests by journalists investigating this story who wanted to assess the similarity between some proposals that an automated tool had flagged as potentially having such strong overlap. Wouldn't it be simpler for everyone involved if they (or a script operated by them) could have just downloaded the proposals from a platform like the one we plan to build? Conversely, there may be a role for further FOI requests on the way towards opening up research proposals.

Currently, a common scenario is that a proposal was evaluated as excellent but rejected nonetheless, due to limited funds. What would be lost if these proposals and their assessments were made public and others could chip in to move the project forward?
 

Casino fund

In principle, research proposals could be abandoned entirely, and replaced by a system that combines some baseline research grants with a shift of focus from projects to people and with a more elaborate - and more transparent - system of science prizes for different degrees of achievements.

One suggestion relevant in this regard has been offered by Sydney Brenner in 2003 in the context of stimulating potentially transformative research of the kind that would be perceived as too risky by most traditional research funders:
"I propose that everybody who gives money for research should take 1% of it and put it into a fund which I call the casino fund, and write it off."

He is a Nobel laureate and has made this suggestion multiple times over the last decade, with no action taken to actually build such a fund. The chances to engage research funders that way are thus rather slim, but some of them are likely to be interested in improving the efficiency of their funding mechanisms. If so, they may be willing to contribute to a fund dedicated to research about ways to improve research funding, especially in terms of handling potentially transformative research.

We think that openness can contribute significantly to such improvements. For instance, some of the casino funding could be crowdsourced by allowing researchers to spend a small portion of their  grants on co-funding open research proposals evaluated as "most interesting but unfundable" by traditional means, or the public could be invited to contribute through crowdfunding platforms.

It is also worth considering that the overall environment for research communication has evolved since 2003, with researchers and the public much more exposed to sharing (including crowdfunding) and with open access to research results and data and even open science more generally being on high-level political agendas (see Neelie Kroes video embedded above).
 

What about being scooped?

Of course, there is the fear of getting scooped, i.e. that someone else might steal an idea worth implementing, and if they have the resources, they may get there first and reap off all the benefits.

Many have experienced this in the current system, but effective scooping is actually relatively simple now and much harder if your ideas are out there in the open: if everyone knows you were the first to propose (and actually pursue) that idea, anyone who tries to sell it as their own will risk loosing reputation, so they may actually prefer to work with rather than against you. More on that in the Harvard video embedded above.

Where to find researchers willing to collaborate openly, from early on?
Assuming (as we do) that they exist, there is no easy answer to that, and identifying conditions under which researchers would be inclined towards openly collaborative approaches is a key research component of our proposal, as described in section (3). Besides gathering data, the survey would double as an outreach tool to raise awareness of open approaches and to spur creative thinking around them, which could be fed back to the project.

As mentioned in section (4d), past experience with open collaborative initiatives suggests that problems attract open collaboration if they are too complex to be solved by individuals or existing research groups but nonetheless likely tractable if the right set of skills, knowledge and tools comes together.

With that in mind, it is worth considering that current research funding is systemically biased against proposals that do not fit into established disciplinary boundaries. In that context, computer scientist Ehud Shapiro recently wrote:
"Genuine interdisciplinary research is nothing like a competitive race. It is much more like a solitary exploratory hike through an uncharted landscape. [...] There are no peers to compete with". But there may be collaborators if they ask nicely.

Now consider how the initiator of the first Polymath project, Fields medalist Tim Gowers, described the open and collaborative nature of the project: "this process is to normal research as driving is to pushing a car."

We expect that at least some of Shapiro's solitary hikers would like to borrow Gowers' car to drive through the uncharted territory in front of them. In order to recruit open collaborators, they will have to provide them with information as to why the ride would be worth it, how to identify and reach targets and so on, which is precisely the classical function of research proposals. Sharing research before formal publication is not unheard of either, and in some fields, it has even become the norm (consider, for instance, the Bermuda Principles, or arXiv).

While our aim is to open up research proposals of any kind, we will pay special attention to ensuring that our proposed system fits the needs of research proposals that bridge across disciplines.

We are conscious that biases against other minorities exist within the research community, and we will strive to cater to their needs as well (that includes supporting languages other than English). In doing so, we will build on related experiences with open procedures in other research contexts. For instance, some scholarly journals engaged in public peer review address these issues by allowing reviewers to choose whether they want to remain anonymous, and we can imagine a similar option for authors of research proposals, although probably not for an extended period.

Apart from facilitating the publication and dissemination of new research proposals, we will work on incorporating proposals from the past - both funded and unfunded ones - into our system. As users, we do not only envisage human readers in search of grant proposals (e.g. to write their own, or to find out more about a particular approach or research group) but also machines, so we will keep an eye on providing programmatic access to the data, which may be of interest to science historians, research administrators or developers building discovery tools.

Given that funding bodies and institutions are now starting to make public detailed accounts of the money they spent on author-paid access to research publications, it seems not too unreasonable to expect that some of them may be willing to share more of their records soon, perhaps with some embargo periods that can be phased out over time.

A small number of grant proposals - both funded and unfunded ones - have already been made public at the discretion of their authors in areas like biology or mathematics. A few of them have also shared their thoughts about this in blog posts, or contributed to this proposal through comments. We plan to engage with these early adopters to help us guide in identifying and building incentive structures around that.
In ONE sentence, tell us about your project to strengthen the Internet for free expression and innovation.
We want to provide a framework in which researchers *want* to share their research ideas and proposals early on, so that they can be worked on collaboratively to the benefit of society.
Who will benefit from what you propose? What have you observed that makes you think that?
Most research proposals at most research funding agencies around the globe get rejected these days. Researchers who share their ideas may find new collaborators and new funding opportunities, and those who collaborate with them find out about these ideas early on and can share in their implementation. Research funders would benefit from a more efficient use of their resources, and the public could be engaged throughout the research project, rather than just at the end. Finally, having grant proposals publicly available would be a great educational resource, not just in terms of writing proposals, but also in terms of science journalism and with regard to the history of ideas.
What progress have you made so far?
We helped draft or review a good number of research proposals in a collaborative fashion, sometimes in the open. We are used to working in the open more generally, have experience with classical research funding as well as crowdfunding approaches and are involved in a number of activities at the frontiers of science communication, e.g. publishing journal articles to Wikipedia, publishing data, public peer review and reusing multimedia from scholarly publications in Wikimedia contexts, the latter of which actually resulted from a proposal drafted in the open.
What would be a successful outcome for your idea or project?
A public collection of actionable topical research proposals would be a success at basically any size, but a goal of getting a few dozens in the first year would seem realistic, with the first 1000 reached on a five-year frame. If a third or more of these have reviews published along, that would be a success too, and so would be open licensing of a substantial portion of this corpus. All of this requires new incentive structures, which are thus the primary target, with the platform as the primary tool.
Who is on your team, and what are their relevant experiences or skills?
At the moment, we are a team of four: Daniel Mietchen, Museum für Naturkunde – Leibniz Institute for Research on Evolution and Biodiversity, Germany. His research interest span from 3D imaging of fossils and embryos to vocal learning in elephants, cold hardiness in insect larvae, and semantic integration of the biodiversity literature. As a volunteer, he has been involved in several open-source software projects on the interface between Wikimedia projects and scholarly research, most notably the Open Access Media Importer [1] that won one of the inaugural Accelerating Science Awards [2], as well as the Wikipedia Cite-o-Meter [3] and currently a project to signal the licensing of scholarly references cited on Wikipedia [4]. Paweł Szczęsny, Department of Bioinformatics, Institute of Biochemistry and Biophysics, Polish Academy of Sciences, Poland. Besides, Paweł is building a technical and social infrastructure for science through his NGO called The Systems Institute. His main project in this area right now is a crowdfunding and collaboration platform for open citizen science projects (to be available soon in Polish [5]). Mike Linksvayer is an expert on commons-based policy and innovation. He advises many free software and open data projects, including serving on the boards of OpenHatch [6] and Software Freedom Conservancy. Previously [7], he served as CTO at Creative Commons [8] and co-founded Bitzi, an early (2001) mass collaboration/open data service [9]. Paul Gardner, Biomolecular Interactions Centre, School of Biological Sciences, Canterbury University, Christchurch, New Zealand. Paul ran the Rfam database [10] of non-coding RNA families for four years. During this tenure, he initiated the use of Wikipedia entries for biological database curation. The experiment was so successful that it has been imitated by several other biological databases (e.g. Pfam and miRBase). Paul continues to contribute to the database and plays a role as Editor in Chief for the journal RNA Biology [11] and Software Editor for PLOS Computational Biology [12]. He is a bioinformatics lecturer and one of the inaugural recipients of a Rutherford Discovery Fellowship, New Zealand's highest honour for early career academics. Daniel is a biophysicist, Paweł is a biologist, Mike has a background in economics and software development, and Paul is a Bioinformatician. We all have been involved with open source, open access and open science for several years. Note that the team has been assembled on the basis of this proposal being public. References [1] https://commons.wikimedia.org/wiki/User:Open_Access_Media_Importer_Bot [2] http://blogs.plos.org/plos/2013/10/announcing-the-recipients-for-the-accelerating-science-award-program/ [3] http://tools.wmflabs.org/cite-o-meter/ [4] https://en.wikipedia.org/wiki/Wikipedia:WikiProject_Open_Access/Signalling_OA-ness [5] http://www.naukaobywatelska.pl [6] https://openhatch.org/ [7] http://sfconservancy.org/ [8] https://creativecommons.org/ [9] http://web.archive.org/web/20131218024208/http://bitzi.com/ [10] http://rfam.sanger.ac.uk/ [11] https://www.landesbioscience.com/journals/rnabiology/editorialboard/ [12] http://www.ploscompbiol.org/static/edboard
Location
Daniel is based in Jena, Germany, Paweł in Warsaw, Poland, Mike in Oakland, California, and Paul in Christchurch, New Zealand. All four of us are experienced in collaborating with researchers across disciplinary, national and sectorial boundaries.

Comments

Join the conversation and post a comment.

Markus Miettinen

April 21, 2014, 10:36AM
Your idea is good. Let me give some comments from the perspective of a possible future user of the platform, i.e., researcher applying for funding.

In my experience getting funding is a rather noisy process. That is, success of an application might not be completely random, but it does have a very very large random component. Therefore, should my carefully crafted application not be successful in Call for Applications A, a sensible thing for me to do (in the current system) would not be to make it public, but to submit a tuned version to Call for Applications B. Upon each submission I will have, say, a 10% success rate and thus after enough repetitions I will get some funding from somewhere.

The point I wish to make is that researchers might not be very keen on making their unsuccesfull applications public unless you are able to convince them that this approach is a more certain way to fund their work than repeated submission with, say, 10% likelihood. The Casion Fund, for example, seems to have a lot worse odds than a repeated submission.

The successful applications, however, are a completely different story. Here the funding agencies should even demand that the applications are made public. When it comes to the fashionable multi-disciplinary and multi-national applications, the demand should even be that the application be turned to a public wiki, in which the participants collaborate in solving the problem(s) they claimed to solve in the application. This would be a great improvement in transparency, for example, on the current EU grants (and the SFB grants here in Germany). An openly viewable wiki would really make it clear that the claimed collaborations within the consortium are taking place in the actual day to day research.

Coming back to the unsuccesful applications. If you wish to make these public, maybe the best approach would be the one in your point 4b. That is, making the applications public _before_ the decisions are made. This would allow for openly elaborating the applications before final submission, finding interested collaborations, and so on. All this could happen on your platform. And, as everything was public from the beginning on, after the decisions come I would not need to choose if I would like to make my unsuccessful application public. It would be out there already.

Daniel Mietchen

April 25, 2014, 18:52PM
I totally agree with all of that:
Yes, in most cases, resubmission is possible but time and effort is always lost. In some cases, resubmission is not possible (e.g. due to some age limits or temporary funding line), which may be another area for having a closer look.

Yes, funders should have at least some funding lines in which proposals are only considered when public.

Yes, "making the applications public _before_ the decisions are made" is the key element of the proposal.

Kenneth Hawkins

April 11, 2014, 15:26PM
Great integration tof software viruses non existent safe for business

Jan Jensen

April 10, 2014, 09:41AM
As I read the post there are several different aims here: 1) getting the idea out in he open, 2) getting people to collaborate on it openly and 3) raising funds for it.

1) Getting the idea out there
By far the most effective way would be for funding institutions to simply post all proposals that they *receive*, preferably together with the reviews. If that is done under CC-BY others could aggregate the proposals at sites aimed at fostering collaboration and raising funding.

Another way is to systematically request funded proposals under various national freedom of information acts, post them, and go from there.

Another way is to create an "Ideas category" in PLoS ONE where a proposal could be be published (after some modification) and *peer reviewed*. The incentive here is an extra publication that on the CV looks like a *regular* article. This is now possible with negative results, instead of having a publication in, say, The Journal of Negative results appear in your CV.

I have not published my recent (mostly unsuccessful) proposals because they are all collaborative and I feel I would have to get permission from everyone involved just to post it. Too much work, no incentive at present.

2) Getting people to collaborate on it openly
That is a much, much tougher job as it fundamentally changes how people work on a day to day basis. I suggest you contact Mat Todd (Sydney, @mattoddchem) about his experience with open lab books.

3) Raising funds
No ideas

Daniel Mietchen

April 10, 2014, 21:50PM
Great suggestions - thanks!

Fully agree that funders should simply post proposals and reviews under CC BY, but this is not in reach, so the big part of the project is actually about (0) preparing the ground for your steps 1-3 to unfold.

Hadn't thought about the freedom of information act approach at all in this context - worth further thought, and be it just as a demo.

Initially, I think the carrot of making the funding and review process more effective and the research proposals more interesting may be more promising to get pilot projects initiated with some funders, but sticks like FOIA certainly have their merits too.

The "Ideas category" in a journal line of thought is on our minds and purposefully mentioned in the introduction, but we are not stressing the journal concept much in the proposal, as it is something many people have very well-defined conceptions about, which leaves little room for the fundamental changes we are aiming to facilitate.

Yes, reluctance of collaborators to open things up is a problem I know well. We plan to tinker with incentives for that, though.

I am aware of Mat Todd's work on open science and in good contact with him, including over this proposal.

Will try to work some of these thoughts in and clarify as needed.

Roderic Page

April 10, 2014, 03:32AM
I guess my question would be "why do this?". Openness is good (and I've shared grant proposals and challenge entries, e.g.
http://dx.doi.org/10.6084/m9.figshare.92091 ) but it's not clear to me what the compelling reasons are for have a single place for sharing proposals (if that is part of the goal). In other words, what would I get from this that I wouldn't get if I put them on my blog, or published them in, say, figshare?

One benefit might be comments from the community, but that assumes there will be a community that assembles around open proposals. Why would such a community form? In my experience communities tend to form around specific topics, or are based on specific expertise (e.g., http://stackoverflow.com), I'm not sure either applies here. It also assumes that all research is collaborative, which is one view of science, but not a complete one (see, e.g., http://dx.doi.org/10.1371/journal.pcbi.0010006 ).

Another benefit might be funding, but I don't think the Kickstarter model works here. Most "serious" science requires substantial funding. Even if the science isn't expensive, people are, so even a fairly standard research project (say a PI and a postdoc) quickly reaches a budget of a few hundred thousand pounds/dollars/euros. That's not the sort of money you are likely to get from voluntary contributions (unlike, say, small amounts of money raised on http://www.rockethub.com/projects/partner/scifund ).

If you're serious about the money side, then this approach would have to be attractive to those people/institutions with money, such as charities, foundations, and grant agencies. Interestingly, some grant agencies at least are getting frustrated by the kinds of proposals that they receive (typically low risk) and are looking at easy to encourage more innovative and risky proposals (I was involved in a NSF "ideas lab" on the tree of life that was designed to do this). So, I wonder whether something like what you are proposing could be done in such a way that the folks with money would see it as a place to go to see what might be on the horizon, or be falling between the cracks, and actually be prepared to fund something. A bit like a database of patents where you'd search to find what ideas it might be worth bringing to market. But this would require that agencies have a clear idea of the sorts of things they want to fund, and it may be frustrated by national boundaries (most national funding agencies fund their own researchers only).

I don't wish to be negative, I guess I'm trying to separate the general argument ("wouldn't it be great if we made proposals openly available?") from the specific argument that we need to do more than simply have individuals put their proposals online so Google will find them.

Daniel Mietchen

April 10, 2014, 21:28PM
Thanks for these thoughts, Rod.

"a single place for sharing proposals" is not "part of the goal", but a single mechanism for discovering proposals is, and it should have provisions for sharing too. That is facilitated by having everything in one place, but there are other means for that (e.g. browser plugins). An argument for something more centralized would be long-term preservation and - you guessed it - markup (or even indexing), if you prefer, so as to facilitate discovery.

Just imagine you'd be browsing Google Scholar or PMC or BHL and some recommendation engine would serve you research proposals related to the article you (or one of your scripts) are looking at.

Yes, communities form around topics and methods, and the focus here would be on integrating research proposals into the relevant communities' workflows, rather than on getting them to hop over to yet another platform.

I am well aware of SciFund and related initiatives and agree with your assessment of the funding aspects there, though I think the outreach components are relevant in the context of our proposal.

Yes, being attractive to funders is a key component of our vision, despite the pitfalls due to national, thematic and other boundaries.

We will go through the draft again to make those points clearer.

Richard Moore

April 09, 2014, 04:54AM
I think there are two somewhat separable issues here. The first is about getting funding for research, and the second is about accelerating the advancement of science. Daniel's proposal deals with both, and that is admirable. I'd like to focus for a moment on the second.

I submit that there is a fantastic open-source science journal in existence, called the Internet, and its table of contents is known as Google search. As a generalist with a serious interest in many branches of science, and as a retired person with no need of funding for my livelihood, I've found this vehicle quite adequate for coming up to speed on any field of interest. (Of course you need to have a basic understanding of math, physics, chemistry, and biology to begin with.) I've also found that in every field you soon find solid independent researchers who have moved on well beyond the understanding of mainstream researchers in that field, and who are of course ignored by the mainstream.

As regards accelerating the advancement of science, I'd say the most productive thing that could happen would be a cultural shift among scientists: to take their noses out of their specialist journals once in a while and check out what's going on in the wider world of independent research – not only in their own field, but in related fields. This of course requires reading in a different way: instead of looking first at the list of references, one needs to focus on the arguments themselves, one needs to be tolerant of non-specialist language and modes of expression, and one needs to be able to separate the wheat from the chaff – often solid research results are followed by inane conclusions and speculations.

Cross-pollonization among mainstream and independent researchers is needed because the independent researchers typically lack funding. Their work is often limited to theoretical reconsideration of fundamental assumptions, and they don't have the funding to put experiments on space probes, and the like. If those with funding and a bit of leeway in how they spend it were to expand their horizons, as I've suggested, science would advance much ore quickly.

Daniel Mietchen

April 09, 2014, 05:29AM
Thanks for the thoughtful comment, Richard. I agree with most of it, especially the last sentence and the part on search replacing tables of content. Another Richard had put it this way a while ago: "there is but one journal: the scientific literature" (cf. http://www.plosmedicine.org/annotation/listThread.action?root=16331 ), and my colleague Paweł has put this into perspective on another April 9: http://www.pawelszczesny.org/2010/04/09/one-scientific-journal-one-scientific-community/ .

The points I do not fully agree with are (1) that our proposal is just about funding, (2) that the one journal referred to above is the best way to stay abreast of current developments.

Ad 1: our proposal is about the spread of ideas, about feedback and collaboration, much of which could well take place (at least initially) with little or no funding (in https://blogs.ch.cam.ac.uk/pmr/2014/04/09/we-should-have-collaboration-not-competition-in-research-citizens-please-join-in/ , Peter Murray-Rust lists the Blue Obelisk as an example, Wikipedia would be another one).

The current research system is structured such that funding dominates these other aspects, and that is one of the things we wish to change.

Ad 2: the problem with that journal is that it mainly contains stuff that is already outdated when it comes out, and that there is little room to actually collaborate on the ideas expressed in there. We are focusing on sharing ideas early on, so that you (and everyone else) can read them in your favourite journal, and get together to act on them. This may well involve getting funding, but is not limited to that.

Richard Moore

April 09, 2014, 07:34AM
Please accept my profuse apology if I mischaracterized the proposal I was commenting on. I was focusing on the points I wanted to make. I very much support your work! Thanks for the links above which make very sensible statements. It is a pleasure to be able to participate here. :)

Daniel Mietchen

April 09, 2014, 16:37PM
Thanks again - please feel invited to join our hangout this Friday, where we will discuss many aspects of opening up research proposals, and further comments or questions are most welcome. Certainly no need for profuse apologies here.

Ivan Ferrero

April 01, 2014, 18:13PM
BTW while I await for your reply to my offers, have you read this yet? :-)

http://ec.europa.eu/research/participants/portal/desktop/en/funding/index.html

Daniel Mietchen

April 01, 2014, 18:40PM
I read it and like it but am at a conference right now (see https://en.wikipedia.org/wiki/User:Daniel_Mietchen/Talks/JATS-Con_2014/Inconsistent_XML_as_a_Barrier_to_Reuse_of_Open_Access_Content ). Will respond in more detail tonight.

Ted Strauss

March 28, 2014, 15:08PM
Very important effort.
So much knowledge and great ideas are locked away in research proposals that are never realized.

Daniel Mietchen

April 01, 2014, 21:30PM
Fully agree, Ted - do you see a way to help move this forward?

Ted Strauss

April 02, 2014, 01:26AM
well, two concrete ideas come to mind.
Idea 1 is to take an open publishing system that is already being used for open access and open review journals, install an instance of it, write up some submission guidelines, and put out an open call for research proposals. some template customization would need to be done to model what are the key properties to track in a research proposal.
Idea 2 is a bit more ambitious, and it follows from your point above that "the act of publishing research proposals could be reframed in terms of 'calls for funders' that would complement classical 'calls for proposals'". This approach sounds a bit like _ kickstarter for research proposals_ which i'm pretty sure exists... I would favor "calls for collaborators" where a proposer could describe his/her research and the dream team of collaborators who would work on it. The system could support teams, projects, review status, reputations, whatever helps people produce knowledge together.

Ivan Ferrero

April 02, 2014, 10:06AM
I agree, and it's part of my ideas.

I'm a bit concerned about the first idea, for it will be the same as many blogs.
There are Open Journals too (at least I saw them in the Neurosciences field): do we really need another one? :-)

The second idea is more ambitious, but more innovative and so has more opportunity to stand out from the crowd.

I suggest a merge of these ideas: a platform that allows researchers to meet each other and that provide the tools to collaborate and create/share reports/datas...researchers will publish on the platform (so there will be a public area).
The last step could be no mandatory: researchers will be able to choose where to publish their researches...
We could insert the Kickstarter model too, though it may ruin the OpenAccess model.

This way there will be the opportunity to create links between related researches.

I see a complex project, made of many slots.

Daniel Mietchen

April 07, 2014, 19:24PM
Hi Ted and Ivan,

of course, we could just use some existing publishing platform like OJS or the code from eLife to start yet another journal, but there is a profound difference between existing journals and our proposed project: while the former only concern research that has already been performed, we want to publish research from early on, i.e. in the idea and grant proposal stage. In the long run, we hope that this helps pave the way for researchers to actually share their entire research workflows.

Our project would also differ from existing crowdfunding initiatives in that finding suitable funding for an idea would only be one of several goals, with others being the refinement of ideas in response to public comments, and the assembling of a team on the basis of interactions over the initial proposal, which may quite possibly lead to teaming up with people who have proposed similar projects.

The main issue here is not the technical one of how to ingest research proposals and how to present them to users of our envisaged platform, though this is important too: the main issue is to make researchers want to share their ideas and proposals early on, and suitable parameters for a pilot on that have yet to be identified, though cross-disciplinarity seems to be a key ingredient.

Ivan Ferrero

March 26, 2014, 10:06AM
It's some years I'd like an Open Psychology, so I had to applaude to this Open Science project!
My compliments!
Have you found the team yet?
BTW: what team? :-)

Daniel Mietchen

March 26, 2014, 11:11AM
Thanks - some people have expressed an interest in joining the team, but none have actually done so. Still a month to go, though. Would you like to join in? How would that make the project stronger?

Ivan Ferrero

March 30, 2014, 12:31PM
Hi Daniel I'm sorry for my delay: I had a great Hangout to set up and it kept me very busy!
I never participated to Open Access projects, and I'd like to.
I'm a Psychologist, I study new medias and new technologies from lot of time: how they affect oujr lives and our minds.

I'm very active on Social Medias, most of all Google+, where I have good followers and co-moderate a Psychology Community with more than 70000 members.

I have lots of ideas about Open Access projects, and I think they would be useful to this project.
BTW: do you have a project structure yet? I'm reading a lot about data journalism and I believe this project would benefit its principles.

Plus: I can give some more exposure to the project via my Social channels (i.e.: my last hangout post had been seen by more than 5000 followers, though about 30 declared the participation.

Here some link...

My G+ business profile:
https://plus.google.com/+IvanferreroIt/posts

The last hangout (the post had been viewed by more than 5000 followers:
https://plus.google.com/108685563399602479492/posts/4WaQXgnBpzz

Please let me know what you think about it!

PS: I'm planning several Hangouts, and I'd like to talk about Open Access and Open Data too.
Maybe you could be my guest?
Consider this offer as indipendent from you approval (or not approval).

Daniel Mietchen

April 01, 2014, 21:28PM
Hi Ivan,

thanks for all the information. As mentioned above, I am at a conference these days, and only now that my talk is over, I actually found the time to look at all the stuff you put in here.

I actually have a soft spot for psychology - did a bit of music psychology and animal cognition during my first postdoc, e.g.
http://dx.doi.org/10.1111/j.1469-8986.2007.00517.x
and
http://dx.doi.org/10.1016/j.cub.2012.09.022 . Unfortunately, both are not really open.

What are your ideas about Open Access? Perhaps we should have a Skype call one of these days (or Hangout, if you prefer)?

I would certainly appreciate an opportunity to bring up the topics of Open Access and Open Data in front of such a community of psychologists and to find out what they think about Myron Brender's proposal of a Journal of Research proposals, or whether they could provide feedback on the related ideas discussed on this page.

As for a structure of the project, that is not set in stone yet, and I agree that it provides great opportunities for journalism - Mike Linksvayer has made similar comments below.

Ivan Ferrero

April 02, 2014, 09:52AM
Hey Daniel it's ok for a Hangout (I'm experiencing some technical issues about Skype!).
When? Send me some date/time proposals.
There we'll be able to talk about everything we need.

How about bringing up these topics to the Community, I'd like to involve other guests.
Do you have any idea?

Daniel Mietchen

April 07, 2014, 21:20PM
For those following along here, Ivan and I are now planning a public Hangout on the topic of opening research proposals, tentatively scheduled for 19:00 UTC on Friday 11. We will confirm the precise time on Tuesday and provide some further details.

Daniel Mietchen

April 08, 2014, 22:37PM
The hangout is set up at https://plus.google.com/u/0/events/ce1t7snttl29082nmpa44joqsdc and has been provisionally linked from the proposal above.

Mike Linksvayer

March 24, 2014, 14:13PM
*Applause*

But: "I want to provide researchers with incentives to share their research ideas and proposals early on, so that they can be worked on collaboratively to the benefit of society. [...] A public collection of actionable topical research proposals would be a success at basically any size, but a goal of getting a few dozens in the first year would seem realistic, with the first 1000 reached on a five-year frame."

You want to provide incentives, but your outcome is a public collection of research proposals. The obvious question is: what incentives do you propose to put in place to such that the outcome is feasible and scalable/sustainable beyond the first 1000? Or what infrastructure do you propose such that with additional funding, well aligned incentives would be feasible? Some kind of contribution metric?

Daniel Mietchen

March 24, 2014, 14:15PM
Certainly a week point so far, unless some funder says hooray (working on that too, but won't hold my breath).

But getting credit may be an incentive under some circumstances - there are a good number of funding schemes where you can apply only once, or only until 35, or 5 years after PhD, or some such.

A more general incentive that I am considering is to provide a drafting environment for grant proposals that allows easy conversion into data papers.

Pawel Szczesny

March 26, 2014, 11:16AM
Incentives part is weak, but what if you look for another source of incentives, otuside of the system? Corporate Social Responsibility (CSR) is an emerging trend in the private sector, but many companies are struggling to put their money into a good use (this is frequently outsourced to consulting industry). I can imagine that open repository of research proposals has an intrinsic value for CSR efforts, as the companies on their own can decide that certain research are cool enough to be with funding. Will you be interestted in exploring that line of thought?

Daniel Mietchen

March 26, 2014, 11:38AM
Yes, incentives from outside the system have been on my mind, but I hadn't thought the CSR option through, so thanks for bringing it up here.

Given your long-standing interest in open science approaches, would you like to join the team?

Pawel Szczesny

March 26, 2014, 11:44AM
Yes, I'd be interested. Let me in.

Daniel Mietchen

March 26, 2014, 11:54AM
Done. Phrasing is provisional at the moment - suggestions welcome.

Daniel Mietchen

March 26, 2014, 11:57AM
@Mike, what about you?

Mike Linksvayer

March 27, 2014, 14:38PM
Hmm, not sure I have any "suggestions as to how [my] involvement would make the project stronger."

But I've been thinking about making the connection to incentives stronger; some ideas:
* Include a research (which would also serve as outreach) component to the project: survey researchers concerning barriers to, incentives needed for, potential usefulness of open research proposals.
* Plan to include an opt-in trial of Brenner's idea. Posters of research proposals on the site could opt-in to committing 1% of research grant obtained for proposal to site fund which on periodic basis would go to posted proposal deemed most interesting but unfundable by traditional funders. This trial could be informed by survey above.
* Plan to include some kind of evaluation system in the site, resulting in metrics that would at least enable selecting winners in previous point, and also informed by survey above.

Mike Linksvayer

March 27, 2014, 14:42PM
Also, an evaluation like "most interesting but unfundable" could connect with corporate or other funders mentioned by Pawel. Such a funder would probably be interested in a particular subfield, and could use "most interesting but unfundable" proposals in that subfield as way to direct their funding in most impactful way.

Mike Linksvayer

March 27, 2014, 14:45PM
One more suggestion, somewhat related to incentives, but also news/journalism (hence this challenge?) -- the site would be a *great* resource for science journalists.

Daniel Mietchen

March 28, 2014, 08:46AM
Thanks, Mike - all good suggestions. Thinking about factoring them in.

Daniel Mietchen

April 07, 2014, 23:26PM
I think I have now incorporated all of your suggestions, Mike. Thanks again!

Mike Linksvayer

April 08, 2014, 00:33AM
Indeed, clearly and concisely.

I also enjoy the two references to this challenge/platform. :)

Ethan White

March 24, 2014, 13:47PM
Very cool idea. I've been maintaining a list of publicly available grant proposals in the biological sciences that might serve as a good starting point for putting together contributions:
http://jabberwocky.weecology.org/2012/08/10/a-list-of-publicly-available-grant-proposals-in-the-biological-sciences/

Daniel Mietchen

March 24, 2014, 14:14PM
Yes, that is useful indeed. I may well be contacting some or all of these people to see what they think of a public repository for proposals, and of open licensing in that context.

Daniel Mietchen

March 26, 2014, 11:58AM
Ethan, could you imagine being part of the project?

Ethan White

March 28, 2014, 08:58AM
I really like the idea, and part of me says "Yes! Yes! Do it!", but the other part of me realizes that I'm already way over-committed at the moment so I have to say no. That said, I'm happy to help out going forward by seeding proposals, providing introductions to other folks who would be happy to contribute, etc.

Daniel Mietchen

March 28, 2014, 17:07PM
I fully understand that and would be happy about any contributions from your end. Right now, the focus is on identifying weaknesses in the proposal, so that we can address them as long as it remains editable.

Entertaining some ideas on how this all would work out on a practical level (e.g. seeding proposals) sounds like a promising approach too.
Login
Close
Login to News Challenge
 
or