More than just a checklist: protocols.io and GENETICS breathe new life into methods

This week, GENETICS and protocols.io are launching a partnership to improve the materials & methods sections of published papers. The journal GENETICS, published by the Genetics Society of America, is encouraging authors to publish detailed methods on protocols.io, in parallel with their article’s publication in GENETICS. Also, as part of the partnership, several accepted manuscripts or articles will be selected each month by the editors of GENETICS, and protocols.io will then digitize these protocols. See the initial set of GENETICS protocols here.)

Tracey A. DePellegrin, Executive Editor of GSA Journals, talks with Lenny Teytelman, founder protocols.io, about this technology innovation.

TAD: Our integration with protocols.io means that protocols – detailed methods – will no longer be simply static PDFs or a series of printed steps. It seems to me that they become multi-dimensional!

Lenny Teytelman

LT: Exactly. Scientists will be able to execute the protocols on the web or on our mobile apps, step-by-step. Most importantly, the authors and readers can keep the methods up-to-date and share tweaks and optimizations on our open access platform.

TAD: Though GENETICS is turning 100 in a few months, a lot of people are surprised that we’ve been early adopters (in many cases, the first – like model organism database linking to articles or publishing early online) of some great technological innovations in communicating science – many of them created by geneticists.

LT: As a geneticist, having gone to countless GSA meetings as a graduate student and postdoc, it’s hard for me to imagine a better journal than GENETICS as the first partnership between protocols.io and a publisher. Through our partnership, many of your methods will be available on mobile phones, and I think this is one of several great examples of efforts on your part to improve the publishing experience and the published results. Together with the recent announcement that references do not have to be formatted on submission and partnership with bioRxiv as examples – I really do think that the GENETICS editors are actively pushing to improve science communication.

TAD: Tell us more about the impetus behind protocols.io. What’s your scientific muse?

LT: In a nutshell, I literally spent the first year and a half of my postdoc at MIT, correcting a single step of the RNA-FISH microscopy method that I was using. Instead of a microliter of an enzyme, we needed 5, and instead of 15 minutes incubation with this chemical, we needed an hour. I discussed this in a lab meeting, and researchers in our lab had to redo their experiments. I called Arjun Raj, the inventor of the method, and he put this info on his website’s FAQ. But the crazy part is that I had no effective way of letting anyone else using this technique know about my discovery. It’s not a new method, but a correction of a previously-published one, and neither I nor Arjun can go to the original Nature Methods publication and put a post-it note on that step, warning people to follow the new instructions. So I get zero credit for this year and a half, but more importantly, everyone else is either getting misleading data or has to spend a year or two rediscovering what I know. My example may be extreme, but it’s an experience familiar to most researchers when something simple that should take a week takes months due to some important tweak of a protocol.

TAD: A big part of making a difference in scientific communication I think does involve connecting scientists with what they need, when they need it. So putting up-to-date information ‘out there’ is important, but you mentioned before that’s only half the battle. Tell me about your approach here.

LT: What we didn’t realize is that more often than not, this knowledge is never even published in the first place. Materials and methods sections are full of “as reported elsewhere”, “we used a slightly modified version”, “detailed protocol available upon e-mail request.” So, I hope that our partnership will shift invisible supplementary protocols into a beautiful web/mobile user interface and will move many valuable protocols out of personal notebooks and into the public domain.

TAD: Two important areas in science right now – transparency and reproducibility – you’ve discussed as being improved with protocols.io. Can you elaborate?

LT: Over the last few years, there has been a lot of attention paid to this issue in academic and industry circles, as well as in the mainstream media. And we have all realized that reproducing work that others have published is very hard. I am glad the issue is in the spotlight, but frankly, some of the proposed solutions scare me. So instead of extra burdens on the scientists who are already overwhelmed, I am a big fan of leveraging technology to both help the researchers and to improve reproducibility smartly. (More thoughts from me on this topic here.) In the case of protocols.io, sharing the methods in fine detail will help others to reproduce the experiments in their lab. And while we all make mistakes, having a central place where we can correct them and share improvements and modifications should also ensure greater efficiency and reproducibility (more on the importance of protocols here and here).

TAD: What’s a typical use case?

LT: Today, we have about a thousand protocols that are private or shared within groups, and 200 publicly-shared methods. So at this point, most of the utility comes from the ability to “run” the public and private protocols, following them step-by-step as one performs the experiment. Many researchers run their protocols on a phone or tablet device directly at the bench, noting any tweaks and deviations in the app. We record a snapshot of exactly how they executed the protocol on that day in a cloud-synchronized journal, so it’s easy to instantly look up the details later, when you are writing the paper or answering the reviewer’s questions. However, the most important use case will happen when we reach critical mass in a year or so.

TAD:  Tell me about that future use case. What have you got up your sleeve for protocols.io in say a year?

LT: We are growing very quickly in terms of content and users, but we are still in the content-seeding stage. It’s critical for us to seed protocols.io with the commonly-used protocols. About 1,000 core techniques in molecular biology cover 70-80% of all the work at the bench. Once we get this initial set of widely-used methods into the platform, the barrier to adoption will be infinitely lower. Scientists will be able to immediately start using and commenting on the protocols, without having to first spend time on entering them in the first place. Getting content has been one of the challenges at the heart of getting our repository off the ground, which is why we are so excited about this partnership with GENETICS. I genuinely believe this partnership is a key step on our path to the ultimate use case – saving somebody a week, month or a year of unnecessary effort because they discovered the correction or optimization on protocols.io.