SciScore to check for adherence to experimental rigor and reporting requirements in several American Heart Association journals 

San Diego, 19 September 2024

An earlier pilot project found that the use of SciScore, an AI-based tool, to evaluate submitted manuscripts was effective in improving several aspects of rigor of research articles, prompting the expansion of SciScore to more titles.

The reproducibility crisis in scientific research is a many-sided problem involving ingrained practices, both in laboratory and publication practices. Fortunately, some of the root causes of irreproducibility are known and being addressed by scientists and editors alike through a steadily increasing number of recommendations and requirements used to address different aspects of rigor and reproducibility in the scientific literature, e.g., MDAR, ARRIVE, CONSORT, and RRID standards.

SciScore, a product of SciCrunch Inc., provides authors and reviewers with both a score and a supporting report of the submitted manuscript to identify if key areas of reproducibility and transparency have been addressed or not in the manuscript. It will look for evidence of randomization, blinded conduct of the experiment, sample size estimation, whether sex is included as a biological characteristic, animal/cell line authentication or contamination, verify the identity of the antibodies used, and various other guideline-specific items.

SciScore, an automated and multifaceted tool based on artificial intelligence, evaluates scientific manuscripts for adherence to several key reporting criteria for rigor and reproducibility. To improve the reproducibility of the research published, SciScore is now integrated into the American Heart Association’s journal submission workflow. Results from the initial pilot, which ran over a year’s time for the Association’s Circulation Research journal, showed that the average monthly scores for submitted manuscripts rose dramatically by 4% month-over-month, which helped the journal to improve its rigor and reproducibility.

“SciScore helps editors point out the places where authors may have omitted a bit of detail about their study. I know that most researchers, when asked about methodological details, are happy to supply needed information. It is great that SciScore can alert authors and reviewers in advance where things might be missing. It is like spell check but for reagents.”

Anita Bandrowski, CEO of SciScore

Moving forward, the following American Heart Association journals will have SciScore integrated in their journal submission process: Arteriosclerosis, Thrombosis, and Vascular Biology (ATVB), Circulation Research, Stroke, Hypertension, and Stroke: Vascular and Interventional Neurology.

Further reading on reproducibility crisis

• https://journals.plos.org/plosbiology/article?id=10.1371/journal.pbio.1002165

• https://blogs.scientificamerican.com/observations/to-fix-the-reproducibility-crisis-rethink-how-we-do-experiments/

• https://www.ahajournals.org/doi/full/10.1161/STROKEAHA.108.515957

Further results with SciScore

• https://www.nature.com/articles/d41586-020-00104-6 & https://www.biorxiv.org/content/10.1101/2020.01.15.908111v2 

• https://www.jmir.org/2022/6/e37324 

• https://www.csescienceeditor.org/article/4257/

About SciCrunch

SciCrunch Inc. is home to platforms like the Resource Identification Initiative, the Antibody Registry, and SciScore. We know the “ingredients” that go into your scientific paper and we work closely with the research community. Our platforms, tools, and expertise enable scientists, resource providers, and companies to track research reagents and verify rigor and transparency in scientific methods.

SciScore was awarded the 2022 Vesalius Innovation Award runner-up prize and was a finalist for the 2020 ALPSP Innovation Award.

For more information, go to: https://www.scicrunch.com/about-us 

Contact for Researchers: Anita Bandrowski | anita@scicrunch.com

Contact for Media/Publishers: Martijn Roelandse | martijn@martijnroelandse.dev