This study evaluated three AI-based literature search platforms (Elicit, Scispace, Scite) for their ability to identify primary studies in evidence synthesis, using allergic rhinitis and respirology case studies as benchmarks, queried in November 2024.
The main AI strategy identified 85.7% of published RCTs with a DOI but only 56.3% of all eligible RCTs overall (missing all unpublished/grey-literature trials); meta-analytical estimates were similar to the full systematic review, but with higher imprecision and heterogeneity for some outcomes. AI platforms identified only 17/48 (35%) of observational studies in a secondary case study.
- Searches capped at 50 results per query per platform, potentially underestimating true recall. - Evaluation is retrospective (November 2024) using platforms that have since evolved; results may not reflect current AI capabilities. - All case studies were in allergy/respirology, limiting generalizability to other fields.
AI-based search platforms can complement—but should not replace—traditional database searches, especially for RCT-based rapid reviews; they consistently miss unpublished and grey-literature trials, risking dissemination bias. Clinicians conducting evidence synthesis should still search trial registries and use AI tools as a supplementary or query-testing step, not as a standalone method.
Explore related topics
