Signals x JMIR: Research Integrity Screening in Practice

Submission volumes are rising rapidly across scholarly publishing due to widespread availability of generative artificial intelligence (AI), persistent system incentives driving volume over quality or contribution, and even ill-intended actors exploiting these systems (e.g., papermills). Automated integrity screening, such as the indicators identified by Signals, offer a supportive check that helps to surface issues that are then examined by journal staff and editors. At JMIR Publications, “all applications are in use only with human oversight and decisions are made exclusively by people.”

Bringing in Signals: How it Fits Into JMIR’s Workflow

In late 2025, Signals and JMIR Publications built an integration between Signals Manuscript Checks and JMIR's editorial management system. Since January 2026, every new submission across the JMIR portfolio has been screened automatically, with results delivered directly into the editorial workflow.

Using a traffic-light system of three evaluation categories – Looks Good, Caution, or Alert – which the JMIR editorial team reviews during desk triage. Most manuscripts are evaluated as Looks Good and continue through editorial processing. All Caution and Alert evaluations warrant examination by the research integrity team, with the Alert being the only hard stop before an editorial assessment proceeds. The research integrity team investigates each potential issue and uses it to help recommend the next action, passing along their findings and recommendations to the handling editor for assessment and communication with the authors if necessary.

Signals Manuscript Checks look for a range of indicators. Among them, authors' publication and retraction history, institutional retraction rates, the validity of cited references, patterns of self-citation or citation clustering, and some automated checks of manuscript content. Signals does not make editorial decisions: it surfaces potential issues and supporting evidence, and JMIR Publications’ research integrity team and editors decide what to do with that information. Automated evaluations are intended as a prompt for human investigation to mitigate the influence of false positives. Where clearly problematic submissions are identified earlier, this helps to free up editors’ and peer reviewers' limited time and attention to focus on submissions that are more likely to be genuine and original contributions.

Early Impact and Iteration

Signals analyses thousands of JMIR submissions every month. In the last month alone, there have been over 150 Alerts, which the research integrity team investigated. Those newly submitted manuscripts that receive an Alert are more than twice as likely as the Caution and Looks Good submissions to ultimately result in rejection on verified grounds. Some individual signals have been particularly helpful to JMIR, such as those related to authorship. Since the beginning for 2026, the Author History signal flagged over 900 manuscripts by authors with a history of retractions or articles flagged by the Problematic Paper Screener.

Scaling an automated screening tool has understandably identified areas of collaborative learning and improvement opportunities. This has also helped identify where the most salient and useful signals are in live workflows, for example, JMIR has found signals related to authorship to be especially informative. Over the course of 2026, several updates to Signals' models introduced temporary increases in false positives, and the rollout of new signal types (including the Author History signal) initially increased the volume of alerts that resulted in unintended consequences for live workflows and workload.

These issues were addressed with detailed feedback from JMIR's research integrity team, which Signals acted on in order to refine the automated checks. For instance, JMIR was the first to identify author name disambiguation problems, which fed into the Author History signal. As a result, Signals then refined author matching to reduce these false positives. Signals and JMIR also worked together to adjust signal scoring and evaluation thresholds so the tool's output better matched the realities of JMIR’s editorial workflow.

Lessons Learned

The clearest lesson from this collaboration is that integrity-screening thresholds aren't one-size-fits-all. JMIR has been an important partner in helping Signals develop a default set of scores for individual signals. Provided feedback helped Signals develop a more considered default scoring approach aimed at flagging genuinely concerning combinations of signals without overflagging manuscripts in ways that would make the workflow unsustainable.

As a result, Signals is now working on functionality to calibrate signal scores for specific journals. This means that evaluation thresholds and the pace at which new signal types are rolled out can be tailored to a publisher's own workflow needs. For any publisher considering integrity screening, our shared takeaway is to expect an extended period of tuning rather than an out-of-the-box fit.

Treating the publisher-vendor relationship as an ongoing partnership rather than a one-time or one-and-done implementation ensures that the tools evolve as the scholarly publishing environment evolves.

What's Next

JMIR has recently started using Signals also as a component of final manuscript quality checks before proceeding to a final decision of acceptance. This gives JMIR and Signals the chance to work together on bringing insight into more parts of the publishing workflow. JMIR's feedback has shaped parts of Signals' roadmap, including further customization options and development of Signals Author Profiles, intended to give editorial teams a clearer picture of the authors submitting to their journals.