World-first framework for assessing research integrity in clinical guidelines, developed by Monash researchers, published in the prominent international medical journal, The Lancet’s eClinical Medicine.

The number of retractions issued for scientific research articles in 2023 exceeded 10,000 — smashing annual records. To date, publishers have struggled to clean up a slew of papers with serious integrity concerns.

Now a world-first framework for assessing research integrity in clinical guidelines, has been published today in the prominent international medical journal, The Lancet’s eClinical Medicine.

Monash University researchers in Australia, world leaders in National and International Clinical Guidelines have developed the Research Integrity in Guidelines and evIDence synthesis (RIGID) framework.

Clinical trials which lack integrity or present concerns over trustworthiness, can compromise patient care, both directly through unnecessary or harmful treatments, or indirectly through wasted resources and misdirected future medical research. “It is estimated that at least 25% of clinical trials informing clinical guidelines may not be trustworthy” according to RIGID co-lead author and research integrity expert, Professor Ben Mol.

In 2023 the RIGID framework was applied to the International Evidence-based Guidelines for Polycystic Ovary Syndrome (PCOS) in collaboration with 39 national and international societies and with input from 80 multidisciplinary experts, and consumers. The PCOS Guidelines have been downloaded over 35,000 times and the framework is now being applied in several other national and international guidelines.

According to first author, Dr Aya Mousa from Monash University, “integrity issues may arise unintentionally through honest error, incorrect analyses or naïve oversight due to inexperience, or intentionally through research misconduct,” she said.

The RIGID framework and accompanying checklist uses a simple six-step approach to determine integrity risk, giving guideline developers, policy-makers, clinicians and scientists a roadmap to assess research integrity, and to exclude untrustworthy research during evidence synthesis and clinical guideline development.

In 2009, the UK-based Committee on Publication Ethics (COPE) released a now widely-adopted policy for handling publication retractions. However, according to Professor Mol, “these policies relate to editorial and publishing processes for retraction and do not provide guidance for researchers and guideline developers on how to handle suspicious or untrustworthy evidence which remains in the public domain, unretracted.” Retraction practices are also often insufficient, relying on whistleblowers to flag integrity concerns and then authors, their institutions and publishers to respond to these queries, where there is little incentive, processes or systems to do so. “Queries to journals and authors’ institutions are frequently met with silence or defensive threats of legal action, and official conclusions are seldom reached or formulated euphemistically, with limited retractions.”

According to Professor Helena Teede, the other co-lead author “the perpetuation of problematic research is underpinned by complex systemic shortcomings, including inadequate application of quality research reporting processes or detection systems; lack of time and resources to investigate claims; lack of incentives for journals, institutions and whistle-blowers; and barriers around reputational or legal implications. Most importantly, there is a lack of standardized procedures or protocols with appropriate oversight to manage integrity concerns.”

Key to the RIGID framework is transparency, where studies ranked by the integrity committee as having a moderate or high risk for integrity concerns are clearly documented, and authors contacted to highlight the identified concerns. This limit their inclusion in the evidence synthesis, pending clarification. Authors are provided with the opportunity to engage in processes to address these issues, generally within two weeks of contact. This is usually met with little to no response, and the research evidence in question is then not considered in formulating conclusions or guideline recommendations.

In the 2023 International Evidence-based Guideline for PCOS, the framework weeded out problematic studies to ensure only trustworthy evidence informed clinical practice. With the use of RIGID, it turned out that no less than 45% of the RCTs assessed had moderate or high risk of integrity concerns and these could not be trusted to guide practice.

The lead developer of the PCOS guideline, Professor Helena Teede, noted “it has been approved by the National Health and Medical Research Council of Australia, including the RIGID process, and has been viewed over 130,000 times and presented at over 100 conferences globally. During guideline development, patient representatives and health professionals highly prioritised the importance of only relying on trustworthy research to guide clinical practice. The RIGID framework is now being applied to other international guidelines, including the Premature Ovarian Insufficiency (POI) International Guideline and the Australian adaptation of the European Society of Human Reproduction and Embryology (ESHRE) Unexplained Infertility Guideline,”  she said.