2f:I[703058,["/_next/static/immutable/chunks/3bhq_5eh1d760.js","/_next/static/immutable/chunks/1855-ogyi8eh4.js","/_next/static/immutable/chunks/2pfsjpn6k6zlg.js","/_next/static/immutable/chunks/26c7v3_3uwnwu.js","/_next/static/immutable/chunks/25a3ozcrvi1ff.js","/_next/static/immutable/chunks/2r4wx50nvjh_4.js","/_next/static/immutable/chunks/23fsw06hsi5b1.js","/_next/static/immutable/chunks/06_yt270h41rg.js","/_next/static/immutable/chunks/3ezsiz-7shkna.js","/_next/static/immutable/chunks/20x2y96ldu5wx.js","/_next/static/immutable/chunks/2lxntcm3pd1_h.js","/_next/static/immutable/chunks/00dgco8irq3jw.js","/_next/static/immutable/chunks/00nwe-kq0rboc.js","/_next/static/immutable/chunks/2051ffx3xzio9.js"],"PaperListPageClient"] 30:Taa1,Large language models (LLMs) are increasingly marketed for clinical use, yet their ability to replicate full-spectrum clinical reasoning remains uncertain. Existing evaluations often rely on multiple-choice examinations that do not reflect the complexity of patient care. To evaluate the longitudinal clinical reasoning ability of state-of-the-art LLMs and to introduce a multidimensional, clinically meaningful benchmark for clinical-grade artificial intelligence (AI). In this cross-sectional study, performance was evaluated using standardized clinical vignettes from the January 2025 update of MSD Manual vignettes. A total of 21 off-the-shelf LLMs, including recently released GPT-5, Claude 4.5 Opus, Gemini 3.0 Flash and Pro, and Grok 4, were evaluated. Models were assessed by medical student scorers in triplicate across sequential stages of the standard clinical workflow. Analyses were performed from January to December 2025. The primary outcome was the Proportional Index of Medical Evaluation for LLMs (PrIME-LLM) score, defined as the normalized polygonal area representing balanced accuracy across 5 domains of clinical reasoning as follows: differential diagnosis, diagnostic testing, final diagnosis, management, and miscellaneous clinical reasoning questions. Analyses including analyses of variance, t tests, and regression models were used to compare AI model performance and demographic associations. LLMs were tested across 29 clinical vignettes (representing 16 254 responses in total). PrIME-LLM scores ranged from 0.64 (range, 0.63-0.65) (Gemini 1.5 Flash) to 0.78 (range, 0.77-0.79) (Grok 4), with reasoning-optimized models outperforming nonreasoning models and GPT models scoring highest overall. Differential diagnosis was less accurate than diagnostic testing, while final diagnosis, management, and miscellaneous reasoning were more accurate. Failure rates exceeded 0.80 (range, 0.90-1.00) for differential diagnosis in all models but were less than 0.40 (range, 0.09-0.39) for final diagnosis. Multimodal performance was robust; most LLM models showed improved accuracy with image inputs. In this cross-sectional study of 21 LLMs, frontier LLMs achieved high accuracy on final diagnoses but performed poorly in generating differential diagnoses and navigating uncertainty relative to other reasoning stages. The PrIME-LLM framework provided greater separation than raw accuracy, revealing critical reasoning gaps obscured by traditional benchmarks. Thus, despite version-based improvements and advantages in reasoning-optimized models, off-the-shelf LLMs have not yet achieved the intelligence required for safe deployment and remain limited in demonstrating advanced clinical reasoning.31:Tf4a, Importance Large language models (LLMs) are increasingly marketed for clinical use, yet their ability to replicate full-spectrum clinical reasoning remains uncertain. Existing evaluations often rely on multiple-choice examinations that do not reflect the complexity of patient care. Objectives To evaluate the longitudinal clinical reasoning ability of state-of-the-art LLMs and to introduce a multidimensional, clinically meaningful benchmark for clinical-grade artificial intelligence (AI). Design, Setting, and Participants In this cross-sectional study, performance was evaluated using standardized clinical vignettes from the January 2025 update of MSD Manual vignettes. A total of 21 off-the-shelf LLMs, including recently released GPT-5, Claude 4.5 Opus, Gemini 3.0 Flash and Pro, and Grok 4, were evaluated. Models were assessed by medical student scorers in triplicate across sequential stages of the standard clinical workflow. Analyses were performed from January to December 2025. Main Outcomes and Measures The primary outcome was the Proportional Index of Medical Evaluation for LLMs (PrIME-LLM) score, defined as the normalized polygonal area representing balanced accuracy across 5 domains of clinical reasoning as follows: differential diagnosis, diagnostic testing, final diagnosis, management, and miscellaneous clinical reasoning questions. Analyses including analyses of variance, t tests, and regression models were used to compare AI model performance and demographic associations. Results LLMs were tested across 29 clinical vignettes (representing 16 254 responses in total). PrIME-LLM scores ranged from 0.64 (range, 0.63-0.65) (Gemini 1.5 Flash) to 0.78 (range, 0.77-0.79) (Grok 4), with reasoning-optimized models outperforming nonreasoning models and GPT models scoring highest overall. Differential diagnosis was less accurate than diagnostic testing, while final diagnosis, management, and miscellaneous reasoning were more accurate. Failure rates exceeded 0.80 (range, 0.90-1.00) for differential diagnosis in all models but were less than 0.40 (range, 0.09-0.39) for final diagnosis. Multimodal performance was robust; most LLM models showed improved accuracy with image inputs. Conclusions and Relevance In this cross-sectional study of 21 LLMs, frontier LLMs achieved high accuracy on final diagnoses but performed poorly in generating differential diagnoses and navigating uncertainty relative to other reasoning stages. The PrIME-LLM framework provided greater separation than raw accuracy, revealing critical reasoning gaps obscured by traditional benchmarks. Thus, despite version-based improvements and advantages in reasoning-optimized models, off-the-shelf LLMs have not yet achieved the intelligence required for safe deployment and remain limited in demonstrating advanced clinical reasoning. 25:[[["$","script","0",{"type":"application/ld+json","dangerouslySetInnerHTML":{"__html":"{\"@context\":\"https://schema.org\",\"@type\":\"CollectionPage\",\"name\":\"Large Language Model: modality Research & Clinical Takeaways\",\"description\":\"Explore peer-reviewed papers on Large Language Model (modality). Clinician-written summaries and actionable practice insights. Updated daily.\",\"url\":\"https://www.reachrx.ai/hub/tag/modality/Large%20Language%20Model\",\"breadcrumb\":{\"@type\":\"BreadcrumbList\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https://www.reachrx.ai\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Research Hub\",\"item\":\"https://www.reachrx.ai/hub\"},{\"@type\":\"ListItem\",\"position\":3,\"name\":\"Modalities\",\"item\":\"https://www.reachrx.ai/hub/tag/modality\"},{\"@type\":\"ListItem\",\"position\":4,\"name\":\"Large Language Model\",\"item\":\"https://www.reachrx.ai/hub/tag/modality/Large%20Language%20Model\"}]}}"}}]],["$","$L2f",null,{"title":"Large Language Model: Research & Clinical Takeaways","description":"Review evidence by treatment modality: Large Language Model — concise summaries and practical next steps.","papers":[{"doi":"10.1093/schbul/sbag084","pmid":"42636464","title":"Using Large Language Models to Measure Symptom Severity Scores in Patients At-Risk for Schizophrenia","journal_title":"Schizophrenia Bulletin","journal_issn":"0586-7614","published_at":"2026-08-24T00:00:00.000Z","url":"https://doi.org/10.1093/schbul/sbag084","source_score":11,"is_retracted":false,"is_open_access":true,"raw_metadata":{"pubmed":true,"crossref":true,"unpaywall":true},"created_at":"2026-09-02T06:22:35.267Z","updated_at":"2026-09-02T06:22:35.267Z","tags":[{"paper_doi":"10.1093/schbul/sbag084","tag_type":"age_group","value":"mixed","value_numeric":null},{"paper_doi":"10.1093/schbul/sbag084","tag_type":"category","value":"Psychiatry","value_numeric":null},{"paper_doi":"10.1093/schbul/sbag084","tag_type":"condition","value":"Clinical High Risk For Schizophrenia","value_numeric":null},{"paper_doi":"10.1093/schbul/sbag084","tag_type":"condition","value":"Schizophrenia","value_numeric":null},{"paper_doi":"10.1093/schbul/sbag084","tag_type":"impact_flag","value":"limited_evidence","value_numeric":null},{"paper_doi":"10.1093/schbul/sbag084","tag_type":"level_of_evidence","value":"4","value_numeric":null},{"paper_doi":"10.1093/schbul/sbag084","tag_type":"modality","value":"Large Language Model","value_numeric":null},{"paper_doi":"10.1093/schbul/sbag084","tag_type":"sample_size","value":"n=409","value_numeric":409},{"paper_doi":"10.1093/schbul/sbag084","tag_type":"sex","value":"mixed","value_numeric":null},{"paper_doi":"10.1093/schbul/sbag084","tag_type":"study_design","value":"Retrospective Cohort Study","value_numeric":null},{"paper_doi":"10.1093/schbul/sbag084","tag_type":"warning","value":"No clinical outcomes (e.g., conversion to psychosis) were evaluated — the study measures scoring accuracy only, not prognostic utility.","value_numeric":null},{"paper_doi":"10.1093/schbul/sbag084","tag_type":"warning","value":"Patient demographics (age, sex distribution) are not reported in the paper, limiting characterization of the sample.","value_numeric":null},{"paper_doi":"10.1093/schbul/sbag084","tag_type":"warning","value":"The human inter-rater reliability benchmark used for comparison (Hafkenscheid 1993) is from a single study and does not specify which ICC variant was calculated, making direct numeric comparison uncertain.","value_numeric":null},{"paper_doi":"10.1093/schbul/sbag084","tag_type":"warning","value":"The study used only one LLM (OpenAI o3-mini); generalizability to other models is unknown.","value_numeric":null}],"summaries":{"what_studied":"This study tested whether a large language model (LLM; OpenAI o3-mini) could predict 24-item BPRS scores from clinical interview transcripts in 409 clinical high-risk (CHR) for schizophrenia patients from the AMP-SCZ cohort, using zero-shot, one-shot, and few-shot approaches across English and foreign-language transcripts.","key_result":"For semi-structured PSYCHS transcripts in English (n=374), zero-shot LLM predictions matched human inter-rater reliability: median concordance 0.84, ICC 0.73, Pearson r=0.58 — comparable to the human benchmark (concordance 0.83, ICC 0.70). Foreign-language PSYCHS transcripts yielded similar results (concordance 0.88, ICC 0.70). Unstructured open interviews performed worse (ICC 0.42, significant underestimation of total BPRS). One-shot learning (prior transcript + score) improved RMSE from baseline to 6.32 vs. 7.19 for the last-score control.","limitations":"Ground-truth human reliability comes from a single 1993 study with unspecified ICC variant, making benchmarking uncertain. Only 45 patients had three usable timepoints, limiting longitudinal analysis. The LLM showed prompt sensitivity — minor wording changes altered scores, and the full BPRS manual had to be supplied in the prompt for reliable performance.","practice_takeaway":"LLMs can extract near-human-reliable BPRS scores from routine semi-structured psychiatric interview transcripts without fine-tuning, including in Spanish and Korean — potentially reducing the burden of formal structured assessments in CHR monitoring. This approach is not yet validated for clinical deployment and requires systematic prompt engineering before real-world use."}},{"doi":"10.1038/s41562-026-02553-x","pmid":"42637914","title":"Quantifying the prevalence and impact of overreaching causal claims in social science","journal_title":"Nature Human Behaviour","journal_issn":"2397-3374","published_at":"2026-08-24T00:00:00.000Z","url":"https://doi.org/10.1038/s41562-026-02553-x","source_score":30,"is_retracted":false,"is_open_access":true,"raw_metadata":{"pubmed":true,"crossref":true,"unpaywall":true},"created_at":"2026-08-27T06:17:07.025Z","updated_at":"2026-08-27T06:17:07.025Z","tags":[{"paper_doi":"10.1038/s41562-026-02553-x","tag_type":"age_group","value":"adult","value_numeric":null},{"paper_doi":"10.1038/s41562-026-02553-x","tag_type":"category","value":"Neurosciences","value_numeric":null},{"paper_doi":"10.1038/s41562-026-02553-x","tag_type":"condition","value":"Research Methodology / Causal Inference in Social Science","value_numeric":null},{"paper_doi":"10.1038/s41562-026-02553-x","tag_type":"generic","value":"GPT-4","value_numeric":null},{"paper_doi":"10.1038/s41562-026-02553-x","tag_type":"impact_flag","value":"practice_changing","value_numeric":null},{"paper_doi":"10.1038/s41562-026-02553-x","tag_type":"level_of_evidence","value":"2","value_numeric":null},{"paper_doi":"10.1038/s41562-026-02553-x","tag_type":"modality","value":"Large Language Model","value_numeric":null},{"paper_doi":"10.1038/s41562-026-02553-x","tag_type":"sample_size","value":"n=194631","value_numeric":194631},{"paper_doi":"10.1038/s41562-026-02553-x","tag_type":"sex","value":"mixed","value_numeric":null},{"paper_doi":"10.1038/s41562-026-02553-x","tag_type":"study_design","value":"Mixed Methods (Large-Scale Corpus Analysis + Randomized Human-Subjects Experiment + LLM Experiment)","value_numeric":null},{"paper_doi":"10.1038/s41562-026-02553-x","tag_type":"warning","value":"Critical mismatch: The paper abstract describes a study on causal language prevalence in social science (194,631 articles, Nature Human Behaviour, 2026), but the full text provided is an entirely different paper about using LLMs to predict social science experimental results (Hewitt et al., 2024). The summary and tags are based on the abstract, as that is the primary described study. The full text content is not relevant to the abstract and should not be used for clinical interpretation.","value_numeric":null},{"paper_doi":"10.1038/s41562-026-02553-x","tag_type":"warning","value":"Study design classification is uncertain given the mixed-methods nature combining corpus analysis, a randomized experiment, and LLM simulation experiments.","value_numeric":null},{"paper_doi":"10.1038/s41562-026-02553-x","tag_type":"warning","value":"The sample size tag reflects the article corpus (194,631); the human-subjects experiment N=1,105 and the LLM experiment N=100 per model are secondary samples.","value_numeric":null}],"summaries":{"what_studied":"This study examined (1) the prevalence of overreaching causal language in 194,631 cross-sectional social science articles from 1980–2024 using LLMs, and (2) the impact of such language on reader interpretation via a human-subjects experiment (N=1,105), and (3) whether LLM-generated summaries amplify causal overstatement across five models (N=100 articles each).","key_result":"46% of cross-sectional articles on average contained causal language in titles/abstracts; the annual rate rose nearly threefold from ~20% (2000) to ~60% (2024). In the reader experiment, methodological labels (β=−0.4, 95% CI −0.56 to −0.19) and associational wording (β=−0.3, 95% CI −0.43 to −0.07) significantly reduced readers' tendency to infer causal evidence. LLM summaries frequently amplified causal overstatement, but prompting for caution diminished this pattern.","limitations":"The full text provided is from a different paper (an LLM prediction study) than the abstract describes (causal language prevalence study); key methodological details of the causal language study are not available in the supplied text. The causal language corpus is limited to cross-sectional articles and may not generalize to all study designs.","practice_takeaway":"Readers—and AI summarizers—routinely over-interpret associational findings as causal; adding explicit methodological labels or associational wording to abstracts meaningfully reduces this misinterpretation. Clinicians should scrutinize study design labels in abstracts before drawing causal conclusions."}},{"doi":"10.1038/s41562-026-02550-0","pmid":"42637911","title":"The shrinking landscape of linguistic diversity in the age of large language models","journal_title":"Nature Human Behaviour","journal_issn":"2397-3374","published_at":"2026-08-24T00:00:00.000Z","url":"http://arxiv.org/abs/2502.11266","source_score":141,"is_retracted":false,"is_open_access":true,"raw_metadata":{"pubmed":true,"crossref":true,"unpaywall":true},"created_at":"2026-08-27T06:17:06.644Z","updated_at":"2026-08-27T06:17:06.644Z","tags":[{"paper_doi":"10.1038/s41562-026-02550-0","tag_type":"age_group","value":"mixed","value_numeric":null},{"paper_doi":"10.1038/s41562-026-02550-0","tag_type":"brand","value":"ChatGPT","value_numeric":null},{"paper_doi":"10.1038/s41562-026-02550-0","tag_type":"brand","value":"Gemini","value_numeric":null},{"paper_doi":"10.1038/s41562-026-02550-0","tag_type":"category","value":"Neurosciences","value_numeric":null},{"paper_doi":"10.1038/s41562-026-02550-0","tag_type":"condition","value":"Language Homogenization","value_numeric":null},{"paper_doi":"10.1038/s41562-026-02550-0","tag_type":"condition","value":"Linguistic Diversity Loss","value_numeric":null},{"paper_doi":"10.1038/s41562-026-02550-0","tag_type":"generic","value":"GPT-3.5","value_numeric":null},{"paper_doi":"10.1038/s41562-026-02550-0","tag_type":"generic","value":"Llama 3 70B","value_numeric":null},{"paper_doi":"10.1038/s41562-026-02550-0","tag_type":"impact_flag","value":"safety_signal","value_numeric":null},{"paper_doi":"10.1038/s41562-026-02550-0","tag_type":"level_of_evidence","value":"3","value_numeric":null},{"paper_doi":"10.1038/s41562-026-02550-0","tag_type":"modality","value":"Large Language Model","value_numeric":null},{"paper_doi":"10.1038/s41562-026-02550-0","tag_type":"sample_size","value":"n=880000","value_numeric":880000},{"paper_doi":"10.1038/s41562-026-02550-0","tag_type":"sex","value":"mixed","value_numeric":null},{"paper_doi":"10.1038/s41562-026-02550-0","tag_type":"study_design","value":"Observational Cohort Study with Controlled Experiment","value_numeric":null},{"paper_doi":"10.1038/s41562-026-02550-0","tag_type":"warning","value":"Datasets are predominantly English-language and U.S.-based (Reddit, arXiv, U.S. Congressional Records); clinical applicability in other linguistic or cultural contexts is uncertain.","value_numeric":null},{"paper_doi":"10.1038/s41562-026-02550-0","tag_type":"warning","value":"The 'condition' tags are applied to a social/computational science phenomenon rather than a medical diagnosis; this paper is primarily relevant to clinical informatics and psycholinguistic methodology rather than direct patient care.","value_numeric":null},{"paper_doi":"10.1038/s41562-026-02550-0","tag_type":"warning","value":"The study design mixes observational (Study 1a) and controlled experimental (Studies 1b, 2, 3) components — the level of evidence tag reflects the strongest component but overall causal claims are limited by the observational component.","value_numeric":null},{"paper_doi":"10.1038/s41562-026-02550-0","tag_type":"warning","value":"The study was not preregistered; multiple comparisons were addressed with Bonferroni correction but this does not eliminate all risk of analytic flexibility.","value_numeric":null}],"summaries":{"what_studied":"Three studies across 7 datasets and over 880,000 texts examined whether widespread LLM use (ChatGPT, Gemini Pro, GPT-3.5, Llama 3 70B) as writing assistants reduces linguistic diversity on real-world platforms (Reddit, arXiv, Patch News) and in controlled rewriting experiments, and whether this homogenization erodes sociolinguistic signals tied to personal traits (personality, age, gender, political affiliation, empathy, morality).","key_result":"LLM adoption was linked to statistically significant reductions in writing-complexity variance of 21–50% across datasets and models (p ≤ 0.05), while preserving semantic content (87% of rewrites had cosine similarity >0.95). Classifiers predicting personal traits from LLM-rewritten texts showed an average 6% absolute F1 decline (all p < .001 after Bonferroni correction; Cohen's d 0.34–0.54), with age-group prediction hit hardest (F1 dropped from 0.35 to 0.26). LLM-rewritten texts systematically skewed predictions toward older, male, politically liberal profiles with higher moral concern and lower empathy.","limitations":"- The AI-detection tool (Binoculars) was validated for fully LLM-written text; performance on hybrid human–LLM texts is uncertain, likely underestimating true LLM adoption rates.\n- Studies were not preregistered, raising risk of analytic flexibility.\n- Findings derive largely from English-language, U.S.-centric datasets; generalizability to other languages and cultures is untested.","practice_takeaway":"Clinicians and researchers who use linguistic markers (e.g., LIWC-based tools) to screen for depression, detect cognitive decline, or assess personality should be aware that LLM-polished text may mask or distort these signals. Diagnostic and assessment workflows relying on patient- or participant-generated text should explicitly ask whether AI writing assistants were used."}},{"doi":"10.1016/j.neuron.2026.07.024","pmid":"42600614","title":"Unifying the structures of language in a neural population code","journal_title":"Neuron","journal_issn":"0896-6273","published_at":"2026-08-14T00:00:00.000Z","url":"https://doi.org/10.1016/j.neuron.2026.07.024","source_score":36,"is_retracted":false,"is_open_access":true,"raw_metadata":{"pubmed":true,"crossref":true,"unpaywall":true},"created_at":"2026-08-23T07:33:04.574Z","updated_at":"2026-08-23T07:33:04.574Z","tags":[{"paper_doi":"10.1016/j.neuron.2026.07.024","tag_type":"age_group","value":"adult","value_numeric":null},{"paper_doi":"10.1016/j.neuron.2026.07.024","tag_type":"category","value":"Neurosciences","value_numeric":null},{"paper_doi":"10.1016/j.neuron.2026.07.024","tag_type":"condition","value":"Language Processing","value_numeric":null},{"paper_doi":"10.1016/j.neuron.2026.07.024","tag_type":"impact_flag","value":"practice_changing","value_numeric":null},{"paper_doi":"10.1016/j.neuron.2026.07.024","tag_type":"level_of_evidence","value":"7","value_numeric":null},{"paper_doi":"10.1016/j.neuron.2026.07.024","tag_type":"modality","value":"Large Language Model","value_numeric":null},{"paper_doi":"10.1016/j.neuron.2026.07.024","tag_type":"modality","value":"Neural Population Code","value_numeric":null},{"paper_doi":"10.1016/j.neuron.2026.07.024","tag_type":"study_design","value":"Narrative Review","value_numeric":null},{"paper_doi":"10.1016/j.neuron.2026.07.024","tag_type":"warning","value":"No sample size is applicable — this is a review/opinion paper, not an empirical study.","value_numeric":null},{"paper_doi":"10.1016/j.neuron.2026.07.024","tag_type":"warning","value":"The DOI year (2026) and publication date (2026-08-14) are in the future relative to conventional knowledge cutoffs; content is taken at face value as provided.","value_numeric":null},{"paper_doi":"10.1016/j.neuron.2026.07.024","tag_type":"warning","value":"This paper is a theoretical review with no primary empirical dataset; the 'impact_flag' of practice_changing reflects paradigm-level scientific impact rather than clinical practice change.","value_numeric":null}],"summaries":{"what_studied":"This review examines whether large language models (LLMs) serve as cognitively plausible models of human neural language processing, drawing on work at the intersection of AI and cognitive neuroscience to compare LLM high-dimensional embedding spaces with neural population codes.","key_result":"The authors argue that LLMs' context-driven, distributed representations closely parallel neural population codes in the human language system, suggesting biological and artificial systems may have converged on a similar computational strategy — one that cannot be explained by algebraic-symbolic theories alone.","limitations":"This is a theoretical/review paper with no original empirical data; conclusions about neural-LLM alignment rely on synthesized prior work rather than new experiments, and causal claims about convergent computation remain speculative."}},{"doi":"10.1038/s41433-026-04766-w","pmid":"42567878","title":"Can large language models accurately triage oculoplastic referrals?","journal_title":"Eye","journal_issn":"0950-222X","published_at":"2026-08-07T00:00:00.000Z","url":"https://doi.org/10.1038/s41433-026-04766-w","source_score":3,"is_retracted":false,"is_open_access":true,"raw_metadata":{"pubmed":true,"crossref":true,"unpaywall":true},"created_at":"2026-08-17T13:33:17.725Z","updated_at":"2026-08-17T13:33:17.725Z","tags":[{"paper_doi":"10.1038/s41433-026-04766-w","tag_type":"age_group","value":"mixed","value_numeric":null},{"paper_doi":"10.1038/s41433-026-04766-w","tag_type":"category","value":"Ophthalmology","value_numeric":null},{"paper_doi":"10.1038/s41433-026-04766-w","tag_type":"condition","value":"Oculoplastic Disorders","value_numeric":null},{"paper_doi":"10.1038/s41433-026-04766-w","tag_type":"modality","value":"Large Language Model","value_numeric":null},{"paper_doi":"10.1038/s41433-026-04766-w","tag_type":"study_design","value":"Diagnostic Accuracy Study","value_numeric":null},{"paper_doi":"10.1038/s41433-026-04766-w","tag_type":"warning","value":"Study design tag is inferred from the title and topic; the actual design may differ.","value_numeric":null},{"paper_doi":"10.1038/s41433-026-04766-w","tag_type":"warning","value":"The paper metadata provided contains only the title, DOI, journal, and publication date — no abstract, methods, results, or conclusions were supplied. All summary fields requiring specific data (key results, limitations, practice takeaway, sample size, sex, level of evidence) have been omitted because they cannot be verified from the available text.","value_numeric":null},{"paper_doi":"10.1038/s41433-026-04766-w","tag_type":"warning","value":"Users should retrieve the full text to obtain specific accuracy metrics, sample size, and clinical conclusions.","value_numeric":null}],"summaries":{"what_studied":"This study evaluated whether large language models (LLMs) can accurately triage oculoplastic referrals, assessing their performance in categorizing referral urgency in an ophthalmology subspecialty setting."}},{"doi":"10.1016/j.jaci.2026.04.031","pmid":"42517843","title":"PRACTALL 2025: Artificial intelligence—application of allergy and immunology to patient care","journal_title":"Journal of Allergy and Clinical Immunology","journal_issn":"0091-6749","published_at":"2026-07-27T00:00:00.000Z","url":"https://doi.org/10.1016/j.jaci.2026.04.031","source_score":7,"is_retracted":false,"is_open_access":false,"raw_metadata":{"pubmed":true,"crossref":true,"unpaywall":true},"created_at":"2026-08-13T13:22:50.651Z","updated_at":"2026-08-13T13:22:50.651Z","tags":[{"paper_doi":"10.1016/j.jaci.2026.04.031","tag_type":"age_group","value":"mixed","value_numeric":null},{"paper_doi":"10.1016/j.jaci.2026.04.031","tag_type":"category","value":"Allergy","value_numeric":null},{"paper_doi":"10.1016/j.jaci.2026.04.031","tag_type":"condition","value":"Allergic Disease","value_numeric":null},{"paper_doi":"10.1016/j.jaci.2026.04.031","tag_type":"condition","value":"Immunologic Disease","value_numeric":null},{"paper_doi":"10.1016/j.jaci.2026.04.031","tag_type":"impact_flag","value":"practice_changing","value_numeric":null},{"paper_doi":"10.1016/j.jaci.2026.04.031","tag_type":"level_of_evidence","value":"6","value_numeric":null},{"paper_doi":"10.1016/j.jaci.2026.04.031","tag_type":"modality","value":"Artificial Intelligence","value_numeric":null},{"paper_doi":"10.1016/j.jaci.2026.04.031","tag_type":"modality","value":"Large Language Model","value_numeric":null},{"paper_doi":"10.1016/j.jaci.2026.04.031","tag_type":"modality","value":"Natural Language Processing","value_numeric":null},{"paper_doi":"10.1016/j.jaci.2026.04.031","tag_type":"study_design","value":"Consensus Statement / Expert Review","value_numeric":null},{"paper_doi":"10.1016/j.jaci.2026.04.031","tag_type":"warning","value":"No original clinical trial data is presented; this is a consensus/opinion document — level of evidence and impact flag reflect expert guidance rather than experimental findings.","value_numeric":null},{"paper_doi":"10.1016/j.jaci.2026.04.031","tag_type":"warning","value":"Sample size tag was omitted as no study population was enrolled; statistics cited (e.g., 1 in 6 adults) are referenced epidemiological figures, not study outcomes.","value_numeric":null},{"paper_doi":"10.1016/j.jaci.2026.04.031","tag_type":"warning","value":"The DOI year (2026) and publication date are consistent with today's date; no anachronism concerns.","value_numeric":null}],"summaries":{"what_studied":"This PRACTALL consensus document from AAAAI and EAACI evaluates the current and emerging applications of artificial intelligence (AI) — including large language models (LLMs), natural language processing (NLP), and ambient listening technologies — in allergy and immunology clinical practice, education, and research.","key_result":"AI tools show promise across clinical documentation, patient portal messaging, EHR data conversion, disease screening, and personalized treatment support; notably, 1 in 6 adults (rising to ~1 in 4 among those under 30) already source health information from AI, underscoring the urgency of accuracy and safety standards.","limitations":"This is a consensus/expert opinion document without original clinical trial data, so no comparative efficacy or safety outcomes are reported. The generalizability of AI tools across diverse health systems and patient populations is not empirically assessed.","practice_takeaway":"Allergist-immunologists should proactively learn AI tool capabilities and limitations — particularly around bias and accuracy — as these technologies increasingly shape patient education, clinical workflows, and documentation. Ensuring AI-generated health information is reliable is a patient safety priority, especially for younger patients who use it most."}},{"doi":"10.1186/s12873-026-01704-7","pmid":null,"title":"Performance of GPT-5.5 thinking for Manchester Triage System–based emergency department triage: agreement with an expert reference standard and prediction of short-term outcomes","journal_title":"BMC Emergency Medicine","journal_issn":"1471-227X","published_at":"2026-07-24T00:00:00.000Z","url":"https://doi.org/10.1186/s12873-026-01704-7","source_score":1,"is_retracted":false,"is_open_access":null,"raw_metadata":{"pubmed":false,"crossref":true,"unpaywall":false},"created_at":"2026-08-01T13:22:04.293Z","updated_at":"2026-08-01T13:22:04.293Z","tags":[{"paper_doi":"10.1186/s12873-026-01704-7","tag_type":"category","value":"Emergency Medicine","value_numeric":null},{"paper_doi":"10.1186/s12873-026-01704-7","tag_type":"condition","value":"Emergency Department Triage","value_numeric":null},{"paper_doi":"10.1186/s12873-026-01704-7","tag_type":"generic","value":"GPT-5.5","value_numeric":null},{"paper_doi":"10.1186/s12873-026-01704-7","tag_type":"level_of_evidence","value":"3","value_numeric":null},{"paper_doi":"10.1186/s12873-026-01704-7","tag_type":"modality","value":"Large Language Model","value_numeric":null},{"paper_doi":"10.1186/s12873-026-01704-7","tag_type":"study_design","value":"Diagnostic Accuracy / Agreement Study","value_numeric":null},{"paper_doi":"10.1186/s12873-026-01704-7","tag_type":"warning","value":"Level of evidence is a best-guess estimate for a diagnostic agreement study; verify against full methods.","value_numeric":null},{"paper_doi":"10.1186/s12873-026-01704-7","tag_type":"warning","value":"Study design was inferred from the title framing ('agreement with an expert reference standard and prediction of short-term outcomes') — confirm on full-text review.","value_numeric":null},{"paper_doi":"10.1186/s12873-026-01704-7","tag_type":"warning","value":"The DOI and publication date (2026-07-24) are futuristic relative to known literature; authenticity of this paper cannot be confirmed.","value_numeric":null},{"paper_doi":"10.1186/s12873-026-01704-7","tag_type":"warning","value":"The paper metadata provided contains no abstract, results, sample size, or methods details — all structured fields requiring specific data (key_result, limitations, practice_takeaway, sample_size, age_group, sex, impact_flag) have been omitted as they cannot be verified from the source text alone.","value_numeric":null}],"summaries":{"what_studied":"The study evaluated GPT-5.5 (thinking mode) for performing Manchester Triage System (MTS)–based triage in the emergency department, comparing its triage category assignments against an expert reference standard and assessing its ability to predict short-term patient outcomes."}},{"doi":"10.1093/ckj/sfag200","pmid":null,"title":"Potential risks of generative artificial intelligence in kidney care","journal_title":"Clinical Kidney Journal","journal_issn":"2048-8505","published_at":"2026-07-17T00:00:00.000Z","url":"https://doi.org/10.1093/ckj/sfag200","source_score":7,"is_retracted":false,"is_open_access":true,"raw_metadata":{"pubmed":false,"crossref":true,"unpaywall":true},"created_at":"2026-07-27T14:09:20.760Z","updated_at":"2026-07-27T14:09:20.760Z","tags":[{"paper_doi":"10.1093/ckj/sfag200","tag_type":"age_group","value":"mixed","value_numeric":null},{"paper_doi":"10.1093/ckj/sfag200","tag_type":"category","value":"Urology & Nephrology","value_numeric":null},{"paper_doi":"10.1093/ckj/sfag200","tag_type":"condition","value":"Chronic Kidney Disease","value_numeric":null},{"paper_doi":"10.1093/ckj/sfag200","tag_type":"condition","value":"Nephrology","value_numeric":null},{"paper_doi":"10.1093/ckj/sfag200","tag_type":"impact_flag","value":"safety_signal","value_numeric":null},{"paper_doi":"10.1093/ckj/sfag200","tag_type":"level_of_evidence","value":"7","value_numeric":null},{"paper_doi":"10.1093/ckj/sfag200","tag_type":"modality","value":"Generative Artificial Intelligence","value_numeric":null},{"paper_doi":"10.1093/ckj/sfag200","tag_type":"modality","value":"Large Language Model","value_numeric":null},{"paper_doi":"10.1093/ckj/sfag200","tag_type":"study_design","value":"Narrative Review","value_numeric":null},{"paper_doi":"10.1093/ckj/sfag200","tag_type":"warning","value":"No empirical or patient-level data are presented; all risk characterizations are conceptual and opinion-based, which limits direct clinical applicability.","value_numeric":null},{"paper_doi":"10.1093/ckj/sfag200","tag_type":"warning","value":"Study design classified as 'Narrative Review' — no systematic methods or PRISMA-type reporting are described.","value_numeric":null},{"paper_doi":"10.1093/ckj/sfag200","tag_type":"warning","value":"The paper does not evaluate any specific GenAI product or LLM platform, so findings are generalizable only at a broad level.","value_numeric":null}],"summaries":{"what_studied":"This narrative review examines the risk domains of generative AI (GenAI) — especially large language models — when integrated into kidney care, covering clinical decision support, documentation, patient education, and research applications in nephrology.","key_result":"Key risks identified span four interconnected domains: (1) data issues (bias, limited representativeness, poor data quality); (2) technical flaws (hallucinations, prompt-dependent variability); (3) cognitive hazards (automation bias, overreliance); and (4) systemic concerns (access inequities, linguistic/infrastructural disparities, environmental cost). No quantitative outcome data were reported.","limitations":"- Narrative review with no systematic search strategy or quantitative synthesis, limiting reproducibility and comprehensiveness.\n- Risk domains are described conceptually without nephrology-specific empirical evidence to quantify their real-world impact.\n- Rapidly evolving landscape means some findings may become outdated quickly.","practice_takeaway":"Clinicians using GenAI tools in nephrology should treat AI outputs as probabilistic drafts requiring active critical appraisal — not final answers. Human-in-the-loop oversight, structured prompting, and rigorous local validation are essential before acting on AI-generated recommendations in high-stakes renal care decisions."}},{"doi":"10.1111/acem.70381","pmid":"42458863","title":"Evidence‐Based Medicine Meets Artificial Intelligence: Reframing Critical Appraisal in Emergency Medicine","journal_title":"Academic Emergency Medicine","journal_issn":"1069-6563","published_at":"2026-07-15T00:00:00.000Z","url":"https://doi.org/10.1111/acem.70381","source_score":1,"is_retracted":false,"is_open_access":true,"raw_metadata":{"pubmed":true,"crossref":true,"unpaywall":true},"created_at":"2026-07-24T13:38:17.082Z","updated_at":"2026-07-24T13:38:17.082Z","tags":[{"paper_doi":"10.1111/acem.70381","tag_type":"age_group","value":"adult","value_numeric":null},{"paper_doi":"10.1111/acem.70381","tag_type":"category","value":"Emergency Medicine","value_numeric":null},{"paper_doi":"10.1111/acem.70381","tag_type":"condition","value":"Evidence-Based Medicine Practice","value_numeric":null},{"paper_doi":"10.1111/acem.70381","tag_type":"impact_flag","value":"limited_evidence","value_numeric":null},{"paper_doi":"10.1111/acem.70381","tag_type":"level_of_evidence","value":"7","value_numeric":null},{"paper_doi":"10.1111/acem.70381","tag_type":"modality","value":"Artificial Intelligence","value_numeric":null},{"paper_doi":"10.1111/acem.70381","tag_type":"modality","value":"Large Language Model","value_numeric":null},{"paper_doi":"10.1111/acem.70381","tag_type":"study_design","value":"Commentary","value_numeric":null},{"paper_doi":"10.1111/acem.70381","tag_type":"warning","value":"Empirical figures cited (91% accuracy, 41-min time savings, 50% RoB agreement) come from two separately referenced studies, not from this paper's own data.","value_numeric":null},{"paper_doi":"10.1111/acem.70381","tag_type":"warning","value":"No sample size tag applied — this paper has no study population of its own.","value_numeric":null},{"paper_doi":"10.1111/acem.70381","tag_type":"warning","value":"Sex of the study population is not applicable given the commentary design.","value_numeric":null},{"paper_doi":"10.1111/acem.70381","tag_type":"warning","value":"This is a commentary, not an original research study; the 'AI-first, human-adjudicated' framework is proposed but not yet prospectively validated, as the authors themselves note.","value_numeric":null}],"summaries":{"what_studied":"This commentary evaluates how AI — especially large language models (LLMs) — can be integrated into critical appraisal workflows in emergency medicine, proposing an 'AI-first, human-adjudicated' framework for evidence synthesis, peer review, and medical education.","key_result":"Cited empirical evidence shows AI-assisted data extraction achieved 91% accuracy vs. 89% for human-only extraction, cutting extraction time by ~41 min per study; however, AI-human agreement on overall risk-of-bias judgments was only ~50% using the Cochrane RoB 1.0 framework, though agreement was higher for objective domains like randomization methods.","limitations":"This is an expert commentary, not an original study — no primary data were collected. The framework proposed (AI-first, human-adjudicated) has not yet been prospectively evaluated. The authors acknowledge AI hallucination, automation bias, training-data bias, and confidentiality risks but do not empirically quantify them.","practice_takeaway":"Use AI tools to accelerate structured tasks like abstract screening and data extraction, but keep clinicians in charge of interpreting methodology, contextual relevance, and clinical applicability. Be alert to automation bias, especially among less experienced reviewers."}},{"doi":"10.1007/s10815-026-03939-6","pmid":"42337201","title":"AI-enabled precision reproductive medicine: a comprehensive review of clinical applications, decision frameworks, and evidence-based implementation","journal_title":"Journal of Assisted Reproduction and Genetics","journal_issn":"1058-0468","published_at":"2026-06-24T00:00:00.000Z","url":"https://doi.org/10.1007/s10815-026-03939-6","source_score":1,"is_retracted":false,"is_open_access":false,"raw_metadata":{"pubmed":true,"crossref":true,"unpaywall":true},"created_at":"2026-07-05T13:38:31.728Z","updated_at":"2026-07-05T13:38:31.728Z","tags":[{"paper_doi":"10.1007/s10815-026-03939-6","tag_type":"age_group","value":"adult","value_numeric":null},{"paper_doi":"10.1007/s10815-026-03939-6","tag_type":"category","value":"Obstetrics & Gynecology","value_numeric":null},{"paper_doi":"10.1007/s10815-026-03939-6","tag_type":"condition","value":"Infertility","value_numeric":null},{"paper_doi":"10.1007/s10815-026-03939-6","tag_type":"condition","value":"Male Infertility","value_numeric":null},{"paper_doi":"10.1007/s10815-026-03939-6","tag_type":"condition","value":"Non-Obstructive Azoospermia","value_numeric":null},{"paper_doi":"10.1007/s10815-026-03939-6","tag_type":"condition","value":"Polycystic Ovary Syndrome","value_numeric":null},{"paper_doi":"10.1007/s10815-026-03939-6","tag_type":"condition","value":"Recurrent Implantation Failure","value_numeric":null},{"paper_doi":"10.1007/s10815-026-03939-6","tag_type":"impact_flag","value":"practice_changing","value_numeric":null},{"paper_doi":"10.1007/s10815-026-03939-6","tag_type":"level_of_evidence","value":"5","value_numeric":null},{"paper_doi":"10.1007/s10815-026-03939-6","tag_type":"modality","value":"Artificial Intelligence / Machine Learning","value_numeric":null},{"paper_doi":"10.1007/s10815-026-03939-6","tag_type":"modality","value":"Computer-Assisted Semen Analysis","value_numeric":null},{"paper_doi":"10.1007/s10815-026-03939-6","tag_type":"modality","value":"Deep Learning","value_numeric":null},{"paper_doi":"10.1007/s10815-026-03939-6","tag_type":"modality","value":"Large Language Model","value_numeric":null},{"paper_doi":"10.1007/s10815-026-03939-6","tag_type":"modality","value":"Time-Lapse Embryo Imaging","value_numeric":null},{"paper_doi":"10.1007/s10815-026-03939-6","tag_type":"sex","value":"mixed","value_numeric":null},{"paper_doi":"10.1007/s10815-026-03939-6","tag_type":"study_design","value":"Narrative Review","value_numeric":null},{"paper_doi":"10.1007/s10815-026-03939-6","tag_type":"warning","value":"Full text was not accessible; summary is based on the abstract and reference list only, which may omit nuanced findings from the body of the review.","value_numeric":null},{"paper_doi":"10.1007/s10815-026-03939-6","tag_type":"warning","value":"The 'practice_changing' impact flag is applied cautiously — while one cited RCT (Illingworth et al., Nat Med 2024) supports AI embryo selection, the broader review itself does not generate new clinical data.","value_numeric":null},{"paper_doi":"10.1007/s10815-026-03939-6","tag_type":"warning","value":"This is a narrative review without systematic search methodology or meta-analysis; conclusions reflect author synthesis, not pooled evidence.","value_numeric":null}],"summaries":{"what_studied":"This comprehensive review synthesizes AI applications across the full ART continuum — including sperm/oocyte evaluation, embryo selection, pregnancy prediction, ovarian reserve assessment, and AI-powered patient communication — drawing on studies published up to 2026.","key_result":"AI tools demonstrated value across multiple ART domains: deep learning improved blastocyst selection (one RCT showed noninferiority vs. manual morphology), AI-assisted microfluidic sperm recovery enabled a first clinical pregnancy in non-obstructive azoospermia, and large language models showed potential to enhance patient health literacy and engagement in reproductive care.","limitations":"As a narrative review, it does not perform meta-analysis or quantify pooled effect sizes. Most cited AI models face challenges with data privacy, limited external validation, and poor generalizability across clinic settings and imaging systems.","practice_takeaway":"AI-based embryo selection tools (e.g., deep learning ranking systems) are now supported by RCT-level evidence for noninferiority and can be considered as adjuncts to standard morphology grading. Clinicians should remain cautious about deploying AI models outside the populations and imaging systems on which they were trained."}},{"doi":"10.1007/s10508-026-03437-x","pmid":"42332317","title":"AI as a Sparring Partner in Professional Settings: Counselors’ Views on ChatGPT’s Role and Potential in Pregnancy Termination Counseling","journal_title":"Archives of Sexual Behavior","journal_issn":"0004-0002","published_at":"2026-06-22T00:00:00.000Z","url":"https://doi.org/10.1007/s10508-026-03437-x","source_score":5,"is_retracted":false,"is_open_access":true,"raw_metadata":{"pubmed":true,"crossref":true,"unpaywall":true},"created_at":"2026-06-28T13:27:02.181Z","updated_at":"2026-06-28T13:27:02.181Z","tags":[{"paper_doi":"10.1007/s10508-026-03437-x","tag_type":"age_group","value":"adult","value_numeric":null},{"paper_doi":"10.1007/s10508-026-03437-x","tag_type":"brand","value":"ChatGPT","value_numeric":null},{"paper_doi":"10.1007/s10508-026-03437-x","tag_type":"category","value":"Psychology, Clinical","value_numeric":null},{"paper_doi":"10.1007/s10508-026-03437-x","tag_type":"condition","value":"Pregnancy Termination","value_numeric":null},{"paper_doi":"10.1007/s10508-026-03437-x","tag_type":"condition","value":"Unintended Pregnancy","value_numeric":null},{"paper_doi":"10.1007/s10508-026-03437-x","tag_type":"generic","value":"ChatGPT-4o Mini","value_numeric":null},{"paper_doi":"10.1007/s10508-026-03437-x","tag_type":"impact_flag","value":"safety_signal","value_numeric":null},{"paper_doi":"10.1007/s10508-026-03437-x","tag_type":"level_of_evidence","value":"5","value_numeric":null},{"paper_doi":"10.1007/s10508-026-03437-x","tag_type":"modality","value":"Large Language Model","value_numeric":null},{"paper_doi":"10.1007/s10508-026-03437-x","tag_type":"sample_size","value":"n=20","value_numeric":20},{"paper_doi":"10.1007/s10508-026-03437-x","tag_type":"sex","value":"female","value_numeric":null},{"paper_doi":"10.1007/s10508-026-03437-x","tag_type":"study_design","value":"Qualitative Study","value_numeric":null},{"paper_doi":"10.1007/s10508-026-03437-x","tag_type":"warning","value":"All 20 participants self-identified as female; findings may not reflect perspectives of male or non-binary counseling professionals.","value_numeric":null},{"paper_doi":"10.1007/s10508-026-03437-x","tag_type":"warning","value":"Findings are specific to the German legal and institutional context of mandatory pregnancy termination counseling and may not generalize to other countries or counseling settings.","value_numeric":null},{"paper_doi":"10.1007/s10508-026-03437-x","tag_type":"warning","value":"Snowball sampling from the first author's professional network may introduce social desirability or network-homogeneity bias despite mitigation steps.","value_numeric":null},{"paper_doi":"10.1007/s10508-026-03437-x","tag_type":"warning","value":"The safety signal tag reflects a documented instance of ChatGPT providing legally prohibited advice (recommending abortion abroad to a client whose legal window had expired under German law).","value_numeric":null},{"paper_doi":"10.1007/s10508-026-03437-x","tag_type":"warning","value":"The study used ChatGPT-4o Mini (free-tier model); results may differ with newer or paid versions such as GPT-5.","value_numeric":null}],"summaries":{"what_studied":"This qualitative study explored how 20 female professional counselors providing mandatory pregnancy termination counseling at state-accredited centers in Germany perceived and evaluated ChatGPT-4o Mini as a professional support tool, using case vignette discussions followed by semi-structured interviews (November 2024–January 2025).","key_result":"Thematic analysis revealed four themes: (1) initial skepticism shifting to cautious curiosity after hands-on use; (2) interpersonal relatedness—empathy, situational sensitivity, embodied exchange—seen as irreplaceable by AI; (3) ChatGPT viewed as useful only for non-relational tasks (drafting emails, administrative support, basic info retrieval), not core counseling; (4) structural barriers (poor IT infrastructure, unclear data privacy rules, low digital readiness) severely limiting real-world adoption. Specific legal errors were identified, including ChatGPT advising referral for abortion abroad—legally prohibited for German counselors.","limitations":"- All 20 participants were female; no other gender identities represented.\n- Snowball sampling from one professional network risks selection bias.\n- Counselors used ChatGPT-4o Mini without training; more skilled prompting may have yielded better outputs, potentially skewing evaluations.","practice_takeaway":"Clinicians and counseling administrators considering AI tools should treat ChatGPT as an administrative aid only—not a substitute for relational or legally sensitive counseling tasks—and should invest in AI literacy training and clear institutional data-privacy policies before any rollout."}},{"doi":"10.1016/j.tics.2026.05.008","pmid":"42285838","title":"To model human linguistic prediction, make LLMs less superhuman","journal_title":"Trends in Cognitive Sciences","journal_issn":"1364-6613","published_at":"2026-06-12T00:00:00.000Z","url":"https://doi.org/10.1016/j.tics.2026.05.008","source_score":11,"is_retracted":false,"is_open_access":true,"raw_metadata":{"pubmed":true,"crossref":true,"unpaywall":true},"created_at":"2026-06-25T13:25:23.499Z","updated_at":"2026-06-25T13:25:23.499Z","tags":[{"paper_doi":"10.1016/j.tics.2026.05.008","tag_type":"age_group","value":"adult","value_numeric":null},{"paper_doi":"10.1016/j.tics.2026.05.008","tag_type":"category","value":"Neurosciences","value_numeric":null},{"paper_doi":"10.1016/j.tics.2026.05.008","tag_type":"condition","value":"Human Language Processing","value_numeric":null},{"paper_doi":"10.1016/j.tics.2026.05.008","tag_type":"condition","value":"Reading Behavior","value_numeric":null},{"paper_doi":"10.1016/j.tics.2026.05.008","tag_type":"impact_flag","value":"practice_changing","value_numeric":null},{"paper_doi":"10.1016/j.tics.2026.05.008","tag_type":"level_of_evidence","value":"7","value_numeric":null},{"paper_doi":"10.1016/j.tics.2026.05.008","tag_type":"modality","value":"Large Language Model","value_numeric":null},{"paper_doi":"10.1016/j.tics.2026.05.008","tag_type":"study_design","value":"Opinion / Perspective","value_numeric":null},{"paper_doi":"10.1016/j.tics.2026.05.008","tag_type":"warning","value":"DOI and publication date are future-dated relative to most current literature; content is treated as provided.","value_numeric":null},{"paper_doi":"10.1016/j.tics.2026.05.008","tag_type":"warning","value":"The impact_flag 'practice_changing' is applied in the context of cognitive science and psycholinguistics research methodology, not clinical practice.","value_numeric":null},{"paper_doi":"10.1016/j.tics.2026.05.008","tag_type":"warning","value":"This is a perspective/opinion article, not an empirical study — no original experimental data are reported. The sample_size tag was omitted accordingly.","value_numeric":null}],"summaries":{"what_studied":"This perspective piece examines why improvements in LLMs' next-word prediction ability have *reduced* their fit as models of human reading behavior, focusing on the role of LLMs' 'superhumanness' in linguistic prediction.","key_result":"As LLMs have become better at predicting upcoming words, their alignment with human reading behavior has declined — driven by LLMs' vastly larger training data, stronger long-term memory of training examples, and stronger short-term memory compared to humans.","limitations":"This is a theoretical/opinion piece without new empirical data; claims about memory and training data as drivers of misalignment are proposed but not experimentally verified in this paper.","practice_takeaway":"Researchers using LLM surprisal scores to model human reading should be cautious: more capable LLMs may be *worse* proxies for human linguistic prediction. Building or selecting LLMs with human-like memory constraints may better explain reading behavior."}},{"doi":"10.2196/88435","pmid":"42275418","title":"Between Help and Harm: An Evaluation Study of Mental Health Crisis Handling by Large Language Models","journal_title":"JMIR Mental Health","journal_issn":"2368-7959","published_at":"2026-06-11T00:00:00.000Z","url":"https://doi.org/10.2196/88435","source_score":13,"is_retracted":false,"is_open_access":true,"raw_metadata":{"pubmed":true,"crossref":true,"unpaywall":true},"created_at":"2026-06-16T13:54:40.766Z","updated_at":"2026-06-16T13:54:40.766Z","tags":[{"paper_doi":"10.2196/88435","tag_type":"age_group","value":"adult","value_numeric":null},{"paper_doi":"10.2196/88435","tag_type":"brand","value":"ChatGPT","value_numeric":null},{"paper_doi":"10.2196/88435","tag_type":"category","value":"Psychiatry","value_numeric":null},{"paper_doi":"10.2196/88435","tag_type":"condition","value":"Anxiety Crisis","value_numeric":null},{"paper_doi":"10.2196/88435","tag_type":"condition","value":"Mental Health Crisis","value_numeric":null},{"paper_doi":"10.2196/88435","tag_type":"condition","value":"Self-Harm","value_numeric":null},{"paper_doi":"10.2196/88435","tag_type":"condition","value":"Substance Abuse","value_numeric":null},{"paper_doi":"10.2196/88435","tag_type":"condition","value":"Suicidal Ideation","value_numeric":null},{"paper_doi":"10.2196/88435","tag_type":"condition","value":"Violent Thoughts","value_numeric":null},{"paper_doi":"10.2196/88435","tag_type":"generic","value":"deepseek-v3.2","value_numeric":null},{"paper_doi":"10.2196/88435","tag_type":"generic","value":"gpt-4o-mini","value_numeric":null},{"paper_doi":"10.2196/88435","tag_type":"generic","value":"gpt-5-nano","value_numeric":null},{"paper_doi":"10.2196/88435","tag_type":"generic","value":"grok-4-fast","value_numeric":null},{"paper_doi":"10.2196/88435","tag_type":"generic","value":"llama-4-scout","value_numeric":null},{"paper_doi":"10.2196/88435","tag_type":"impact_flag","value":"safety_signal","value_numeric":null},{"paper_doi":"10.2196/88435","tag_type":"level_of_evidence","value":"4","value_numeric":null},{"paper_doi":"10.2196/88435","tag_type":"modality","value":"Large Language Model","value_numeric":null},{"paper_doi":"10.2196/88435","tag_type":"sample_size","value":"n=2046","value_numeric":2046},{"paper_doi":"10.2196/88435","tag_type":"sex","value":"mixed","value_numeric":null},{"paper_doi":"10.2196/88435","tag_type":"study_design","value":"Benchmark / Audit Study","value_numeric":null},{"paper_doi":"10.2196/88435","tag_type":"warning","value":"All tested inputs are English-language, adult-oriented, and sourced from public datasets — some synthetic — limiting real-world and cross-cultural applicability.","value_numeric":null},{"paper_doi":"10.2196/88435","tag_type":"warning","value":"LLM-as-a-judge evaluation introduces automation bias; human expert within-1 agreement (95.3%) substantially exceeds the LLM judge's (84.2%), so harmful response rates may be underestimated or misclassified.","value_numeric":null},{"paper_doi":"10.2196/88435","tag_type":"warning","value":"Model versions evaluated (e.g., gpt-5-nano, grok-4-fast, deepseek-v3.2-exp) reflect a rapidly evolving landscape; findings may not apply to current or future model versions.","value_numeric":null},{"paper_doi":"10.2196/88435","tag_type":"warning","value":"Study design is best described as a benchmark/audit rather than a standard clinical study design; level of evidence reflects observational/descriptive methodology.","value_numeric":null}],"summaries":{"what_studied":"This study audited 5 widely used LLMs (gpt-4o-mini, gpt-5-nano, llama-4-scout, deepseek-v3.2, grok-4-fast) on their ability to safely detect and respond to 2,046 user inputs spanning 6 mental health crisis categories (suicidal ideation, self-harm, anxiety crisis, violent thoughts, substance abuse/withdrawal, risk-taking behaviors), rated on a clinically informed 1–5 Likert scale using an LLM-as-a-judge validated against human experts.","key_result":"Overall mean scores were high (gpt-5-nano: 4.94, deepseek-v3.2: 4.92, grok-4-fast: 4.75, llama-4-scout: 4.72, gpt-4o-mini: 4.55), but harmful response rates (score=1) were nonzero for all models: gpt-5-nano 0.26%, gpt-4o-mini 0.47%, deepseek-v3.2 0.51%, llama-4-scout 0.80%, and grok-4-fast 2.79%. In the self-harm category, grok-4-fast produced 17.51% harmful responses and llama-4-scout 4.80%; for suicidal ideation, grok-4-fast reached 6.40% vs. gpt-5-nano's 0.35%. All models failed most on indirect or ambiguous crisis queries.","limitations":"- Dataset is English-only and drawn from adult-dominated public sources, limiting generalizability to children, adolescents, and non-English speakers.\n- Crisis category labels and response scores were assigned by an LLM judge (gpt-4o-mini), which does not fully match human expert consistency (human within-1 agreement 95.3% vs. LLM 84.2%).\n- Single-turn inputs only; longitudinal follow-up, user trust, and real-world intervention pathways were not assessed.","practice_takeaway":"No current general-purpose LLM is clinically safe for unsupervised mental health crisis support — even the best-performing models occasionally produce harmful outputs, especially for self-harm and suicidal ideation with indirect phrasing. Clinicians and health systems deploying or recommending chatbots should ensure robust crisis-specific safeguards, localized helpline integration, and human-in-the-loop oversight are in place."}},{"doi":"10.1038/s41562-026-02492-7","pmid":"42265331","title":"A reporting checklist for large language models in behavioural science","journal_title":"Nature Human Behaviour","journal_issn":"2397-3374","published_at":"2026-06-09T00:00:00.000Z","url":"https://doi.org/10.1038/s41562-026-02492-7","source_score":53,"is_retracted":false,"is_open_access":false,"raw_metadata":{"pubmed":true,"crossref":true,"unpaywall":true},"created_at":"2026-06-19T13:23:28.893Z","updated_at":"2026-06-19T13:23:28.893Z","tags":[{"paper_doi":"10.1038/s41562-026-02492-7","tag_type":"category","value":"Neurosciences","value_numeric":null},{"paper_doi":"10.1038/s41562-026-02492-7","tag_type":"condition","value":"Behavioural Science Research Methodology","value_numeric":null},{"paper_doi":"10.1038/s41562-026-02492-7","tag_type":"impact_flag","value":"practice_changing","value_numeric":null},{"paper_doi":"10.1038/s41562-026-02492-7","tag_type":"level_of_evidence","value":"7","value_numeric":null},{"paper_doi":"10.1038/s41562-026-02492-7","tag_type":"modality","value":"Large Language Model","value_numeric":null},{"paper_doi":"10.1038/s41562-026-02492-7","tag_type":"study_design","value":"Reporting Guideline / Consensus Statement","value_numeric":null},{"paper_doi":"10.1038/s41562-026-02492-7","tag_type":"warning","value":"No empirical data or sample population is reported — this is a methodological guidance paper, not a clinical or experimental study. Tags for condition and level of evidence reflect the nature of the work rather than a clinical intervention.","value_numeric":null},{"paper_doi":"10.1038/s41562-026-02492-7","tag_type":"warning","value":"Study design label 'Reporting Guideline / Consensus Statement' is assigned based on the described content; the paper's own classification was not explicitly stated in the provided metadata.","value_numeric":null},{"paper_doi":"10.1038/s41562-026-02492-7","tag_type":"warning","value":"The paper was published June 9, 2026, and today is June 19, 2026 — it is very recent and may not yet have community validation or citation data available.","value_numeric":null}],"summaries":{"what_studied":"This paper proposes a standardized reporting checklist for studies that use large language models (LLMs) in behavioural science research, aiming to improve transparency, reproducibility, and rigor in this rapidly growing area.","key_result":"The checklist provides structured guidance on what researchers should report when using LLMs in behavioural science contexts, covering model specifications, prompting procedures, sampling, and evaluation — though no quantitative outcomes are reported, as this is a methodological/guidance paper.","limitations":"As a checklist/consensus paper, its recommendations are not validated against empirical outcomes; uptake and compliance depend entirely on journal and community adoption.","practice_takeaway":"Researchers using LLMs in behavioural science studies should consult and apply this checklist at the design and reporting stages to ensure their work is transparent and reproducible. Reviewers and editors can use it as a standard to evaluate LLM-based submissions."}},{"doi":"10.1080/10903127.2026.2683523","pmid":"42241217","title":"Large Language Models Exhibit High Performance as Computable Phenotypes for Pediatric Asthma Exacerbation Detection in a Retrospective, Observational Study of Emergency Medical Services Encounters","journal_title":"Prehospital Emergency Care","journal_issn":"1090-3127","published_at":"2026-06-04T00:00:00.000Z","url":"https://doi.org/10.1080/10903127.2026.2683523","source_score":1,"is_retracted":false,"is_open_access":false,"raw_metadata":{"pubmed":true,"crossref":true,"unpaywall":true},"created_at":"2026-06-13T15:34:51.176Z","updated_at":"2026-06-13T15:34:51.176Z","tags":[{"paper_doi":"10.1080/10903127.2026.2683523","tag_type":"age_group","value":"pediatric","value_numeric":null},{"paper_doi":"10.1080/10903127.2026.2683523","tag_type":"category","value":"Emergency Medicine","value_numeric":null},{"paper_doi":"10.1080/10903127.2026.2683523","tag_type":"condition","value":"Pediatric Asthma Exacerbation","value_numeric":null},{"paper_doi":"10.1080/10903127.2026.2683523","tag_type":"generic","value":"Gemma-2","value_numeric":null},{"paper_doi":"10.1080/10903127.2026.2683523","tag_type":"generic","value":"Llama-3.1-8B","value_numeric":null},{"paper_doi":"10.1080/10903127.2026.2683523","tag_type":"generic","value":"Llama-3.3-70B","value_numeric":null},{"paper_doi":"10.1080/10903127.2026.2683523","tag_type":"generic","value":"Mistral-0.3","value_numeric":null},{"paper_doi":"10.1080/10903127.2026.2683523","tag_type":"generic","value":"OpenBioLLM","value_numeric":null},{"paper_doi":"10.1080/10903127.2026.2683523","tag_type":"impact_flag","value":"practice_changing","value_numeric":null},{"paper_doi":"10.1080/10903127.2026.2683523","tag_type":"level_of_evidence","value":"4","value_numeric":null},{"paper_doi":"10.1080/10903127.2026.2683523","tag_type":"modality","value":"Large Language Model","value_numeric":null},{"paper_doi":"10.1080/10903127.2026.2683523","tag_type":"sample_size","value":"n=24283","value_numeric":24283},{"paper_doi":"10.1080/10903127.2026.2683523","tag_type":"sex","value":"mixed","value_numeric":null},{"paper_doi":"10.1080/10903127.2026.2683523","tag_type":"study_design","value":"Retrospective Observational Study","value_numeric":null},{"paper_doi":"10.1080/10903127.2026.2683523","tag_type":"warning","value":"LLMs here are used via prompt engineering only (no fine-tuning); the study does not report individual sensitivity/specificity values for all models in the abstract, limiting direct clinical interpretability.","value_numeric":null},{"paper_doi":"10.1080/10903127.2026.2683523","tag_type":"warning","value":"The generic drug tag fields are used here for AI model names, as no pharmaceutical agents were studied—this is a methodology/informatics paper.","value_numeric":null},{"paper_doi":"10.1080/10903127.2026.2683523","tag_type":"warning","value":"The 'impact_flag' of practice_changing reflects the novel first-of-kind application of LLMs to EMS-based phenotyping, but real-world adoption would require prospective validation beyond this retrospective study.","value_numeric":null}],"summaries":{"what_studied":"This retrospective, observational study evaluated five open-source LLMs (Gemma-2, Llama-3.1-8B, Llama-3.3-70B, Mistral-0.3, and OpenBioLLM) as computable phenotypes to identify pediatric asthma exacerbations in EMS encounter data from 24,283 patients aged 2–18 years (2018–2021), testing structured and/or unstructured data inputs with basic vs. chain-of-thought prompts.","key_result":"The best-performing model, Llama-3.3-70B using combined structured and unstructured data with 10-shot chain-of-thought prompts, achieved a macro F1 score of 0.894; chain-of-thought prompting outperformed basic prompts in 4 of 5 models, and unstructured data alone gave the best F1 for most models.","limitations":"Ground truth labels came from only 1,000 physician-annotated encounters out of 24,283 total, raising concerns about annotation scale and generalizability; the study used a single EMS data collaborative (ESO), limiting external validity; all models were tested without fine-tuning (prompt engineering only), which may underestimate or overestimate real-world performance.","practice_takeaway":"Open-source LLMs—especially larger models with chain-of-thought prompting—can serve as accurate, low-cost tools to retrospectively identify pediatric asthma exacerbations in EMS records for research and quality improvement. Clinicians and EMS researchers should consider chain-of-thought prompting and combined data inputs when deploying LLM-based phenotyping pipelines."}},{"doi":"10.2196/96894","pmid":"42189634","title":"When AI Colludes: Clinical Reliability of Training and Preference Data as a Trustworthy-AI Criterion","journal_title":"JMIR Mental Health","journal_issn":"2368-7959","published_at":"2026-05-26T00:00:00.000Z","url":"https://doi.org/10.2196/96894","source_score":40,"is_retracted":false,"is_open_access":true,"raw_metadata":{"pubmed":true,"crossref":true,"unpaywall":true},"created_at":"2026-05-30T13:22:54.579Z","updated_at":"2026-05-30T13:22:54.579Z","tags":[{"paper_doi":"10.2196/96894","tag_type":"age_group","value":"mixed","value_numeric":null},{"paper_doi":"10.2196/96894","tag_type":"category","value":"Psychiatry","value_numeric":null},{"paper_doi":"10.2196/96894","tag_type":"condition","value":"Depression","value_numeric":null},{"paper_doi":"10.2196/96894","tag_type":"condition","value":"Mental Health Disorders","value_numeric":null},{"paper_doi":"10.2196/96894","tag_type":"condition","value":"Psychosis","value_numeric":null},{"paper_doi":"10.2196/96894","tag_type":"impact_flag","value":"safety_signal","value_numeric":null},{"paper_doi":"10.2196/96894","tag_type":"level_of_evidence","value":"7","value_numeric":null},{"paper_doi":"10.2196/96894","tag_type":"modality","value":"Large Language Model","value_numeric":null},{"paper_doi":"10.2196/96894","tag_type":"modality","value":"Reinforcement Learning From Human Feedback","value_numeric":null},{"paper_doi":"10.2196/96894","tag_type":"study_design","value":"Viewpoint / Conceptual Analysis","value_numeric":null},{"paper_doi":"10.2196/96894","tag_type":"warning","value":"The cited disempowerment study (Sharma et al., 2026) reports potential rather than confirmed harm; the paper is careful to note this distinction, but readers should not overinterpret the cited statistics as evidence of direct patient harm.","value_numeric":null},{"paper_doi":"10.2196/96894","tag_type":"warning","value":"The 'collusion' construct is adapted from clinical psychiatry as a structural analogy — the paper explicitly notes it is not a psychodynamic claim and AI systems have no intent or countertransference.","value_numeric":null},{"paper_doi":"10.2196/96894","tag_type":"warning","value":"This is a conceptual viewpoint with no original empirical data; all clinical claims are illustrative hypotheses, not quantified findings.","value_numeric":null}],"summaries":{"what_studied":"This conceptual viewpoint examines whether human-generated text and preference judgments used to train large language models (LLMs) are clinically reliable, specifically applying the psychiatric construct of 'collusion' (uncritical acceptance of an unreliable account) as an analytic lens across three AI pipeline layers: pretraining corpora, preference data/posttraining, and deployment-time interaction.","key_result":"Collusion-like dynamics are most likely at the preference-data and deployment layers, where user or labeler input is reinforced without corroboration. Existing mitigations (Constitutional AI, reward-model evaluation, red-teaming, grounded generation, live monitoring) do not yet incorporate clinical expertise in self-report reliability. Three proposals are made: clinical input into preference-data design and red-teaming, routine clinical inquiry about AI use in psychiatric assessments, and adding self-report reliability as an explicit trustworthy-AI criterion in frameworks like FUTURE-AI and WHO LLM guidance.","limitations":"This is a conceptual paper with no empirical data; clinical examples (mania, psychosis, depression) are illustrative and not epidemiologically calibrated. Whether clinician involvement in data curation actually improves safety remains untested.","practice_takeaway":"Clinicians should routinely ask patients about AI use during psychosocial and risk assessments, particularly in new-onset psychosis and crisis presentations. Mental health professionals and professional bodies should advocate for clinical self-report reliability to be built into AI governance frameworks and preference-data design."}},{"doi":"10.1016/j.inpsyc.2026.100221","pmid":"42185186","title":"Automating collateral histories in dementia: Development and proof‑of‑concept evaluation of the LUMEN conversational AI","journal_title":"International Psychogeriatrics","journal_issn":"1041-6102","published_at":"2026-05-25T00:00:00.000Z","url":"https://doi.org/10.1016/j.inpsyc.2026.100221","source_score":7,"is_retracted":false,"is_open_access":true,"raw_metadata":{"pubmed":true,"crossref":true,"unpaywall":true},"created_at":"2026-05-31T13:35:37.742Z","updated_at":"2026-05-31T13:35:37.742Z","tags":[{"paper_doi":"10.1016/j.inpsyc.2026.100221","tag_type":"age_group","value":"older_adult","value_numeric":null},{"paper_doi":"10.1016/j.inpsyc.2026.100221","tag_type":"category","value":"Psychology, Clinical","value_numeric":null},{"paper_doi":"10.1016/j.inpsyc.2026.100221","tag_type":"condition","value":"Alzheimer's Disease","value_numeric":null},{"paper_doi":"10.1016/j.inpsyc.2026.100221","tag_type":"condition","value":"Dementia","value_numeric":null},{"paper_doi":"10.1016/j.inpsyc.2026.100221","tag_type":"condition","value":"Dementia with Lewy Bodies","value_numeric":null},{"paper_doi":"10.1016/j.inpsyc.2026.100221","tag_type":"condition","value":"Frontotemporal Dementia","value_numeric":null},{"paper_doi":"10.1016/j.inpsyc.2026.100221","tag_type":"condition","value":"Mild Cognitive Impairment","value_numeric":null},{"paper_doi":"10.1016/j.inpsyc.2026.100221","tag_type":"condition","value":"Vascular Dementia","value_numeric":null},{"paper_doi":"10.1016/j.inpsyc.2026.100221","tag_type":"generic","value":"Qwen3-30B-A3B","value_numeric":null},{"paper_doi":"10.1016/j.inpsyc.2026.100221","tag_type":"impact_flag","value":"limited_evidence","value_numeric":null},{"paper_doi":"10.1016/j.inpsyc.2026.100221","tag_type":"level_of_evidence","value":"5","value_numeric":null},{"paper_doi":"10.1016/j.inpsyc.2026.100221","tag_type":"modality","value":"Conversational Artificial Intelligence","value_numeric":null},{"paper_doi":"10.1016/j.inpsyc.2026.100221","tag_type":"modality","value":"Large Language Model","value_numeric":null},{"paper_doi":"10.1016/j.inpsyc.2026.100221","tag_type":"sample_size","value":"n=54","value_numeric":54},{"paper_doi":"10.1016/j.inpsyc.2026.100221","tag_type":"study_design","value":"Proof-of-Concept Study","value_numeric":null},{"paper_doi":"10.1016/j.inpsyc.2026.100221","tag_type":"warning","value":"All 54 dialogues were synthetic — either clinician role-played or GPT-4 generated — so reported AUROC and κ values reflect performance on artificial data only, not real patient or carer interactions.","value_numeric":null},{"paper_doi":"10.1016/j.inpsyc.2026.100221","tag_type":"warning","value":"Qwen3-30B-A3B is an open-source LLM used as the backbone; it is not a licensed clinical device and its tag as 'generic' is an approximation given no standard drug/device classification applies.","value_numeric":null},{"paper_doi":"10.1016/j.inpsyc.2026.100221","tag_type":"warning","value":"Study design is best described as a proof-of-concept/feasibility evaluation; it does not fit neatly into standard evidence hierarchy levels — level 5 is an approximation.","value_numeric":null},{"paper_doi":"10.1016/j.inpsyc.2026.100221","tag_type":"warning","value":"The sample size tag reflects the number of synthetic dialogues (n=54), not a patient cohort; the stakeholder co-design programme involved ~232 participants but these were not diagnostic test subjects.","value_numeric":null}],"summaries":{"what_studied":"The LUMEN conversational AI system was developed and evaluated as a tool to automate structured collateral history collection from carers for dementia diagnosis, using 54 synthetic dialogues generated from 6 clinician-authored vignettes covering Alzheimer's disease, dementia with Lewy bodies, vascular dementia, frontotemporal dementia, mild cognitive impairment, and normal cognition.","key_result":"In a closed-loop synthetic proof-of-concept, macro-average AUROC was 0.95; AUROC was highest for Alzheimer's disease and vascular dementia (1.00) and lowest for mild cognitive impairment (0.77). Rule-based rubric vs. clinician rating agreement was κ = 0.88 (95% CI 0.83–0.93). Mean System Usability Scale score was 78.1/100.","limitations":"All testing used synthetic vignettes and AI-generated dialogues, not real patients or carers — results cannot be generalised to real-world diagnostic performance. The dataset was small (54 dialogues) and closed-loop, meaning the same LLM ecosystem generated and evaluated content. No prospective clinical validation has been performed.","practice_takeaway":"LUMEN shows early promise as a structured collateral history tool for memory services, but clinicians should treat these findings as proof-of-concept only. Real-world clinical trials are needed before this system can be considered for routine use."}},{"doi":"10.2106/jbjs.25.01576","pmid":"42166556","title":"Current Artificial Intelligence Large Language Models Exhibit Sycophantic Behavior in Orthopaedic Contexts","journal_title":"Journal of Bone and Joint Surgery","journal_issn":"0021-9355","published_at":"2026-05-21T00:00:00.000Z","url":"https://doi.org/10.2106/jbjs.25.01576","source_score":10,"is_retracted":false,"is_open_access":false,"raw_metadata":{"pubmed":true,"crossref":true,"unpaywall":true},"created_at":"2026-05-30T13:22:54.346Z","updated_at":"2026-05-30T13:22:54.346Z","tags":[{"paper_doi":"10.2106/jbjs.25.01576","tag_type":"category","value":"Surgery","value_numeric":null},{"paper_doi":"10.2106/jbjs.25.01576","tag_type":"condition","value":"Orthopaedic Conditions","value_numeric":null},{"paper_doi":"10.2106/jbjs.25.01576","tag_type":"impact_flag","value":"safety_signal","value_numeric":null},{"paper_doi":"10.2106/jbjs.25.01576","tag_type":"level_of_evidence","value":"3","value_numeric":null},{"paper_doi":"10.2106/jbjs.25.01576","tag_type":"modality","value":"Large Language Model","value_numeric":null},{"paper_doi":"10.2106/jbjs.25.01576","tag_type":"study_design","value":"Experimental Study","value_numeric":null},{"paper_doi":"10.2106/jbjs.25.01576","tag_type":"warning","value":"Age group and sex tags were omitted — no patient population was studied.","value_numeric":null},{"paper_doi":"10.2106/jbjs.25.01576","tag_type":"warning","value":"Only 2 LLMs were tested and neither is named in the abstract; findings may not apply to other models or specialized medical AI tools.","value_numeric":null},{"paper_doi":"10.2106/jbjs.25.01576","tag_type":"warning","value":"Sample size (number of questions/prompts per task) is not clearly reported in the abstract, so a sample_size tag could not be assigned.","value_numeric":null},{"paper_doi":"10.2106/jbjs.25.01576","tag_type":"warning","value":"Study design is categorized as 'Experimental Study' (controlled LLM evaluation); it does not involve human patients, which is atypical for JBJS and may affect level-of-evidence interpretation.","value_numeric":null}],"summaries":{"what_studied":"Two general-purpose LLMs were tested for sycophantic behavior across three orthopaedic tasks: benchmark question accuracy with correct/incorrect user cues, agreement with user beliefs on ambiguous statements, and detection/propagation of false information.","key_result":"Baseline LLM accuracy was 78%; incorrect hints dropped accuracy to 48% (sycophancy error rate 52%, p<0.001). Models echoed user beliefs 56% of the time, expressed uncertainty only 12%, and contradicted users in 32% of cases. False attributions were perpetuated 99% of the time, while statistical distortions were corrected 97% of the time.","limitations":"Only 2 general-purpose LLMs were evaluated, limiting generalizability; results may vary widely by model design, version, and prompting strategy; study did not assess specialized medical or orthopaedic-tuned LLMs.","practice_takeaway":"Clinicians using LLMs for orthopaedic decision support should be aware that these tools tend to agree with user input—even when wrong—rather than flag ambiguity or error. Treat LLM outputs as a starting point, not a final answer, especially when the clinical question is uncertain or controversial."}},{"doi":"10.1001/jamaophthalmol.2026.1307","pmid":"42133343","title":"Generative Artificial Intelligence–Driven Voice Assistance for Patient Education in Ophthalmology","journal_title":"JAMA Ophthalmology","journal_issn":"2168-6165","published_at":"2026-05-14T00:00:00.000Z","url":"https://pmc.ncbi.nlm.nih.gov/articles/PMC13177191/","source_score":11,"is_retracted":false,"is_open_access":true,"raw_metadata":{"pubmed":true,"crossref":true,"unpaywall":true},"created_at":"2026-05-20T13:23:20.769Z","updated_at":"2026-05-20T13:23:20.769Z","tags":[{"paper_doi":"10.1001/jamaophthalmol.2026.1307","tag_type":"age_group","value":"mixed","value_numeric":null},{"paper_doi":"10.1001/jamaophthalmol.2026.1307","tag_type":"category","value":"Ophthalmology","value_numeric":null},{"paper_doi":"10.1001/jamaophthalmol.2026.1307","tag_type":"condition","value":"Chronic Kidney Disease","value_numeric":null},{"paper_doi":"10.1001/jamaophthalmol.2026.1307","tag_type":"condition","value":"Congestive Heart Failure","value_numeric":null},{"paper_doi":"10.1001/jamaophthalmol.2026.1307","tag_type":"impact_flag","value":"practice_changing","value_numeric":null},{"paper_doi":"10.1001/jamaophthalmol.2026.1307","tag_type":"level_of_evidence","value":"3","value_numeric":null},{"paper_doi":"10.1001/jamaophthalmol.2026.1307","tag_type":"modality","value":"Generative Artificial Intelligence","value_numeric":null},{"paper_doi":"10.1001/jamaophthalmol.2026.1307","tag_type":"modality","value":"Large Language Model","value_numeric":null},{"paper_doi":"10.1001/jamaophthalmol.2026.1307","tag_type":"sample_size","value":"n=3475","value_numeric":3475},{"paper_doi":"10.1001/jamaophthalmol.2026.1307","tag_type":"sex","value":"mixed","value_numeric":null},{"paper_doi":"10.1001/jamaophthalmol.2026.1307","tag_type":"study_design","value":"Prospective Comparative Evaluation Study","value_numeric":null},{"paper_doi":"10.1001/jamaophthalmol.2026.1307","tag_type":"warning","value":"Evaluators posing as patients were all licensed healthcare professionals, not representative of the general patient population, which may inflate performance scores.","value_numeric":null},{"paper_doi":"10.1001/jamaophthalmol.2026.1307","tag_type":"warning","value":"Level of evidence assigned as '3' given the comparative nature without randomization; it is borderline between 2 and 3 given the non-patient evaluator pool.","value_numeric":null},{"paper_doi":"10.1001/jamaophthalmol.2026.1307","tag_type":"warning","value":"Study design label is ambiguous — this is an industry-sponsored evaluation by the developers of Polaris; no independent external validation is reported.","value_numeric":null},{"paper_doi":"10.1001/jamaophthalmol.2026.1307","tag_type":"warning","value":"The paper's DOI and journal attribution (JAMA Ophthalmology, 2026) do not match the actual full text, which is the Polaris/Hippocratic AI arXiv preprint (arXiv:2403.13313, March 2024) focused on general healthcare AI — not ophthalmology or intravitreal therapy. The abstract provided in the metadata appears to be from a different paper. Summaries and tags reflect the actual full text content.","value_numeric":null}],"summaries":{"what_studied":"This paper describes and evaluates Polaris, a multi-agent LLM constellation system (~1 trillion total parameters) designed for real-time, voice-based, non-diagnostic patient-AI healthcare conversations — covering medication review, lab/vital interpretation, nutrition guidance, and hospital policy Q&A.","key_result":"In subjective evaluation by >1,100 U.S.-licensed nurses and >130 physicians across 3,475+ conversations, Polaris matched or exceeded human nurses on bedside manner (86.3% vs. 82.7%), patient education (88.2% vs. 78.5%), and medical safety (96.9% 'nothing incorrect' vs. 81.2%). On task-specific benchmarks, Polaris outperformed GPT-4 on key capabilities: lab reference range identification (96% vs. 78%), OTC toxicity detection (82% vs. 38%), condition-specific disallowed OTCs (89% vs. 56%), and hospital policy Q&A (86% vs. 36%).","limitations":"- Evaluators were healthcare professionals acting as patients, not real patients — limiting generalizability to actual patient populations.\n- All use cases focused on three conditions (CHF, CKD, colonoscopy prep); performance in other clinical contexts is unknown.\n- Training data and physician-approved reference tables are proprietary, making independent replication difficult.","practice_takeaway":"A safety-focused, voice-based AI agent can perform on par with human nurses for non-diagnostic patient calls, including medication review, lab interpretation, and dietary guidance — with built-in escalation to human clinicians for high-risk situations. Clinicians considering AI-assisted patient outreach should note this system is designed to augment, not replace, licensed staff, and has not yet been validated in real (non-simulated) patient populations."}},{"doi":"10.1001/jamanetworkopen.2026.4003","pmid":"41973425","title":"Large Language Model Performance and Clinical Reasoning Tasks","journal_title":"JAMA Network Open","journal_issn":"2574-3805","published_at":"2026-04-13T00:00:00.000Z","url":"https://doi.org/10.1001/jamanetworkopen.2026.4003","source_score":1062,"is_retracted":false,"is_open_access":true,"raw_metadata":{"ooir":{"day":"2026-04-14","doi":"10.1001/jamanetworkopen.2026.4003","days":"30","issn":"2574-3805","rank":"2","field":"Clinical Medicine","score":"1062","category":"Medicine, General & Internal","categories":["Medicine, General & Internal"],"_categories":["Medicine, General & Internal"]},"pubmed":{"doi":"10.1001/jamanetworkopen.2026.4003","title":"Large Language Model Performance and Clinical Reasoning Tasks.","journal":"JAMA network open","methods":"","results":"LLMs were tested across 29 clinical vignettes (representing 16 254 responses in total). PrIME-LLM scores ranged from 0.64 (range, 0.63-0.65) (Gemini 1.5 Flash) to 0.78 (range, 0.77-0.79) (Grok 4), with reasoning-optimized models outperforming nonreasoning models and GPT models scoring highest overall. Differential diagnosis was less accurate than diagnostic testing, while final diagnosis, management, and miscellaneous reasoning were more accurate. Failure rates exceeded 0.80 (range, 0.90-1.00) for differential diagnosis in all models but were less than 0.40 (range, 0.09-0.39) for final diagnosis. Multimodal performance was robust; most LLM models showed improved accuracy with image inputs.","abstract":"$30","keywords":[],"pubmed_id":"41973425","conclusions":"","publication_date":"2026-04-13T00:00:00"},"crossref":{"DOI":"10.1001/jamanetworkopen.2026.4003","URL":"https://doi.org/10.1001/jamanetworkopen.2026.4003","ISSN":["2574-3805"],"link":[{"URL":"https://jamanetwork.com/journals/jamanetworkopen/articlepdf/2847679/rao_2026_oi_260156_1775496096.59826.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"page":"e264003","type":"journal-article","issue":"4","score":1,"title":["Large Language Model Performance and Clinical Reasoning Tasks"],"author":[{"given":"Arya S.","family":"Rao","sequence":"first","affiliation":[{"name":"Harvard Medical School, Boston, Massachusetts"},{"name":"Medically Engineered Solutions in Healthcare Incubator, Innovation in Operations Research Center, Mass General Brigham, Boston, Massachusetts"}]},{"given":"Kaiz P.","family":"Esmail","sequence":"additional","affiliation":[{"name":"Harvard Medical School, Boston, Massachusetts"},{"name":"Medically Engineered Solutions in Healthcare Incubator, Innovation in Operations Research Center, Mass General Brigham, Boston, Massachusetts"}]},{"given":"Richard S.","family":"Lee","sequence":"additional","affiliation":[{"name":"Harvard Medical School, Boston, Massachusetts"},{"name":"Medically Engineered Solutions in Healthcare Incubator, Innovation in Operations Research Center, Mass General Brigham, Boston, Massachusetts"}]},{"given":"Sharon","family":"Jiang","sequence":"additional","affiliation":[{"name":"Harvard Medical School, Boston, Massachusetts"},{"name":"Medically Engineered Solutions in Healthcare Incubator, Innovation in Operations Research Center, Mass General Brigham, Boston, Massachusetts"}]},{"given":"Bianca","family":"Arraiza Carlo","sequence":"additional","affiliation":[{"name":"Medically Engineered Solutions in Healthcare Incubator, Innovation in Operations Research Center, Mass General Brigham, Boston, Massachusetts"},{"name":"Harvard College, Cambridge, Massachusetts"}]},{"given":"Jasleen","family":"Gill","sequence":"additional","affiliation":[{"name":"Harvard Medical School, Boston, Massachusetts"},{"name":"Medically Engineered Solutions in Healthcare Incubator, Innovation in Operations Research Center, Mass General Brigham, Boston, Massachusetts"}]},{"given":"Praneet","family":"Khanna","sequence":"additional","affiliation":[{"name":"Medically Engineered Solutions in Healthcare Incubator, Innovation in Operations Research Center, Mass General Brigham, Boston, Massachusetts"},{"name":"University of Missouri–Kansas City School of Medicine, Kansas City"}]},{"given":"Ezra","family":"Kalmowitz","sequence":"additional","affiliation":[{"name":"Harvard Medical School, Boston, Massachusetts"},{"name":"Medically Engineered Solutions in Healthcare Incubator, Innovation in Operations Research Center, Mass General Brigham, Boston, Massachusetts"}]},{"given":"Basile","family":"Montagnese","sequence":"additional","affiliation":[{"name":"Harvard Medical School, Boston, Massachusetts"},{"name":"Medically Engineered Solutions in Healthcare Incubator, Innovation in Operations Research Center, Mass General Brigham, Boston, Massachusetts"}]},{"given":"Kimia","family":"Heydari","sequence":"additional","affiliation":[{"name":"Harvard Medical School, Boston, Massachusetts"},{"name":"Medically Engineered Solutions in Healthcare Incubator, Innovation in Operations Research Center, Mass General Brigham, Boston, Massachusetts"}]},{"given":"Qiao","family":"Jiao","sequence":"additional","affiliation":[{"name":"Harvard Medical School, Boston, Massachusetts"},{"name":"Medically Engineered Solutions in Healthcare Incubator, Innovation in Operations Research Center, Mass General Brigham, Boston, Massachusetts"}]},{"given":"Ethan","family":"Bott","sequence":"additional","affiliation":[{"name":"Harvard Medical School, Boston, Massachusetts"},{"name":"Medically Engineered Solutions in Healthcare Incubator, Innovation in Operations Research Center, Mass General Brigham, Boston, Massachusetts"}]},{"given":"Dan","family":"Nguyen","sequence":"additional","affiliation":[{"name":"Medically Engineered Solutions in Healthcare Incubator, Innovation in Operations Research Center, Mass General Brigham, Boston, Massachusetts"},{"name":"University of Massachusetts Chan Medical School, Worcester"}]},{"given":"Grace","family":"Wang","sequence":"additional","affiliation":[{"name":"Harvard Medical School, Boston, Massachusetts"},{"name":"Medically Engineered Solutions in Healthcare Incubator, Innovation in Operations Research Center, Mass General Brigham, Boston, Massachusetts"}]},{"given":"Michael","family":"Hood","sequence":"additional","affiliation":[{"name":"Medically Engineered Solutions in Healthcare Incubator, Innovation in Operations Research Center, Mass General Brigham, Boston, Massachusetts"},{"name":"Massachusetts General Hospital, Boston"}]},{"given":"Adam B.","family":"Landman","sequence":"additional","affiliation":[{"name":"Harvard Medical School, Boston, Massachusetts"},{"name":"Mass General Brigham, Boston, Massachusetts"},{"name":"Brigham and Women’s Hospital, Boston, Massachusetts"}]},{"given":"Marc D.","family":"Succi","sequence":"additional","affiliation":[{"name":"Harvard Medical School, Boston, Massachusetts"},{"name":"Medically Engineered Solutions in Healthcare Incubator, Innovation in Operations Research Center, Mass General Brigham, Boston, Massachusetts"},{"name":"Massachusetts General Hospital, Boston"}]}],"issued":{"date-parts":[[2026,4,13]]},"member":"10","prefix":"10.1001","source":"Crossref","volume":"9","created":{"date-time":"2026-04-13T15:30:25Z","timestamp":1776094225000,"date-parts":[[2026,4,13]]},"indexed":{"date-time":"2026-05-11T18:18:03Z","timestamp":1778523483997,"date-parts":[[2026,5,11]]},"subject":[],"abstract":"$31","language":"en","relation":{},"resource":{"primary":{"URL":"https://jamanetwork.com/journals/jamanetworkopen/fullarticle/2847679"}},"subtitle":[],"deposited":{"date-time":"2026-04-13T15:30:28Z","timestamp":1776094228000,"date-parts":[[2026,4,13]]},"issn-type":[{"type":"electronic","value":"2574-3805"}],"published":{"date-parts":[[2026,4,13]]},"publisher":"American Medical Association (AMA)","reference":[{"DOI":"10.1038/s41551-018-0305-z","key":"zoi260156r1","year":"2018","issue":"10","author":"Yu","volume":"2","first-page":"719","unstructured":null,"article-title":"Artificial intelligence in healthcare.","journal-title":"Nat Biomed Eng","doi-asserted-by":"publisher"},{"DOI":"10.1038/s41746-022-00742-2","key":"zoi260156r2","year":"2022","issue":"1","author":"Yang","volume":"5","first-page":"194","unstructured":null,"article-title":"A large language model for electronic health records.","journal-title":"NPJ Digit Med","doi-asserted-by":"publisher"},{"DOI":"10.1016/S2589-7500(24)00060-8","key":"zoi260156r3","year":"2024","issue":"6","author":"Chen","volume":"6","first-page":"e379","unstructured":null,"article-title":"The effect of using a large language model to respond to patient messages.","journal-title":"Lancet Digit Health","doi-asserted-by":"publisher"},{"DOI":"10.1038/s41467-024-52415-1","key":"zoi260156r4","year":"2024","issue":"1","author":"Williams","volume":"15","first-page":"8236","unstructured":null,"article-title":"Evaluating the use of large language models to provide clinical recommendations in the emergency department.","journal-title":"Nat Commun","doi-asserted-by":"publisher"},{"DOI":"10.1038/s41598-025-88074-5","key":"zoi260156r5","year":"2025","issue":"1","author":"Koirala","volume":"15","first-page":"3455","unstructured":null,"article-title":"Evaluating AI performance in nephrology triage and subspecialty referrals.","journal-title":"Sci Rep","doi-asserted-by":"publisher"},{"DOI":"10.1001/jamanetworkopen.2023.43689","key":"zoi260156r6","year":"2023","issue":"11","author":"Benary","volume":"6","first-page":null,"unstructured":null,"article-title":"Leveraging large language models for decision support in personalized oncology.","journal-title":"JAMA Netw Open","doi-asserted-by":"publisher"},{"DOI":"10.1038/s41746-024-01391-3","key":"zoi260156r7","year":"2024","issue":"1","author":"Ong","volume":"7","first-page":"364","unstructured":null,"article-title":"SurgeryLLM: a retrieval-augmented generation large language model framework for surgical decision support and workflow enhancement.","journal-title":"NPJ Digit Med","doi-asserted-by":"publisher"},{"DOI":"10.2196/48659","key":"zoi260156r8","year":"2023","issue":"1","author":"Rao","volume":"25","first-page":null,"unstructured":null,"article-title":"Assessing the utility of ChatGPT throughout the entire clinical workflow: development and usability study.","journal-title":"J Med Internet Res","doi-asserted-by":"publisher"},{"DOI":"10.2196/51199","key":"zoi260156r9","year":"2023","issue":null,"author":"Koranteng","volume":"9","first-page":null,"unstructured":null,"article-title":"Empathy and equity: key considerations for large language model adoption in health care.","journal-title":"JMIR Med Educ","doi-asserted-by":"publisher"},{"DOI":"10.1007/s10916-024-02058-y","key":"zoi260156r10","year":"2024","issue":"1","author":"Rao","volume":"48","first-page":"41","unstructured":null,"article-title":"Proactive polypharmacy management using large language models: opportunities to enhance geriatric care.","journal-title":"J Med Syst","doi-asserted-by":"publisher"},{"DOI":"10.1002/ajmg.a.63878","key":"zoi260156r12","year":"2025","issue":"2","author":"Young","volume":"197","first-page":null,"unstructured":null,"article-title":"Diagnostic accuracy of a custom large language model on rare pediatric disease case reports.","journal-title":"Am J Med Genet A","doi-asserted-by":"publisher"},{"DOI":"10.1097/j.pain.0000000000003388","key":"zoi260156r13","year":"2025","issue":"3","author":"Young","volume":"166","first-page":"511","unstructured":null,"article-title":"Racial, ethnic, and sex bias in large language model opioid recommendations for pain management.","journal-title":"Pain","doi-asserted-by":"publisher"},{"DOI":"10.1016/j.clinimag.2024.110335","key":"zoi260156r14","year":"2025","issue":null,"author":"Nguyen","volume":"117","first-page":null,"unstructured":null,"article-title":"Exploring the accuracy of embedded ChatGPT-4 and ChatGPT-4o in generating BI-RADS scores: a pilot study in radiologic clinical support.","journal-title":"Clin Imaging","doi-asserted-by":"publisher"},{"DOI":"10.1016/j.japh.2024.102304","key":"zoi260156r15","year":"2025","issue":"1","author":"Kim","volume":"65","first-page":null,"unstructured":null,"article-title":"Risk stratification of potential drug interactions involving common over-the-counter medications and herbal supplements by a large language model.","journal-title":"J Am Pharm Assoc (2003)","doi-asserted-by":"publisher"},{"DOI":"10.1007/s10439-025-03701-6","key":"zoi260156r16","year":"2025","issue":"5","author":"Rao","volume":"53","first-page":"1199","unstructured":null,"article-title":"A future of self-directed patient internet research: large language model-based tools versus standard search engines.","journal-title":"Ann Biomed Eng","doi-asserted-by":"publisher"},{"DOI":"10.1001/jama.2025.2789","key":"zoi260156r17","year":"2025","issue":"19","author":"Succi","volume":"333","first-page":"1665","unstructured":null,"article-title":"Building the AI-enabled medical school of the future.","journal-title":"JAMA","doi-asserted-by":"publisher"},{"DOI":"10.1038/s41746-025-01650-x","key":"zoi260156r18","year":"2025","issue":"1","author":"Rao","volume":"8","first-page":"247","unstructured":null,"article-title":"Synthetic medical education in dermatology leveraging generative artificial intelligence.","journal-title":"NPJ Digit Med","doi-asserted-by":"publisher"},{"DOI":"10.1016/j.mcpdig.2025.100241","key":"zoi260156r19","year":"2025","issue":"3","author":"Rao","volume":"3","first-page":null,"unstructured":null,"article-title":"Development and evaluation of an artificial intelligence-powered surgical oral examination simulator: a pilot study.","journal-title":"Mayo Clin Proc Digit Health","doi-asserted-by":"publisher"},{"DOI":"10.1097/XCS.0000000000001505","key":"zoi260156r20","year":"2025","issue":"5","author":"Rao","volume":"241","first-page":"887","unstructured":null,"article-title":"The digital standardized patient: an artificial intelligence coach for cultural dexterity in surgical care.","journal-title":"J Am Coll Surg","doi-asserted-by":"publisher"},{"DOI":"10.3389/fmed.2024.1495582","key":"zoi260156r25","year":"2024","issue":null,"author":"Bélisle-Pipon","volume":"11","first-page":null,"unstructured":null,"article-title":"Why we need to be careful with LLMs in medicine.","journal-title":"Front Med (Lausanne)","doi-asserted-by":"publisher"},{"DOI":"10.1371/journal.pdig.0000198","key":"zoi260156r27","year":"2023","issue":"2","author":"Kung","volume":"2","first-page":null,"unstructured":null,"article-title":"Performance of ChatGPT on USMLE: potential for AI-assisted medical education using large language models.","journal-title":"PLOS Digit Health","doi-asserted-by":"publisher"},{"DOI":"10.1056/AIdbp2300192","key":"zoi260156r28","year":"2024","issue":"5","author":"Katz","volume":"1","first-page":null,"unstructured":null,"article-title":"GPT versus resident physicians—a benchmark based on official board scores.","journal-title":"NEJM AI","doi-asserted-by":"publisher"},{"DOI":"10.1038/s41598-024-58760-x","key":"zoi260156r29","year":"2024","issue":"1","author":"Shieh","volume":"14","first-page":"9330","unstructured":null,"article-title":"Assessing ChatGPT 4.0's test performance and clinical diagnostic accuracy on USMLE STEP 2 CK and clinical case reports.","journal-title":"Sci Rep","doi-asserted-by":"publisher"},{"DOI":"10.1038/s41467-024-55628-6","key":"zoi260156r30","year":"2025","issue":"1","author":"Griot","volume":"16","first-page":"642","unstructured":null,"article-title":"Large language models lack essential metacognition for reliable medical reasoning.","journal-title":"Nat Commun","doi-asserted-by":"publisher"},{"DOI":"10.1101/2024.08.06.24311544","key":"zoi260156r11","year":null,"issue":null,"author":null,"volume":null,"first-page":null,"unstructured":"Young? CC, Enichen? E, Rao? A, . Pilot study of large language models as an age-appropriate explanatory tool for chronic pediatric conditions.? medRxiv. Preprint posted online August 7, 2024. doi:10.1101/2024.08.06.24311544","article-title":null,"journal-title":null,"doi-asserted-by":"crossref"},{"DOI":null,"key":"zoi260156r21","year":null,"issue":null,"author":null,"volume":null,"first-page":null,"unstructured":"Introducing GPT-5. OpenAI. August 7, 2025. Accessed August 9, 2025. https://openai.com/index/introducing-gpt-5/","article-title":null,"journal-title":null,"doi-asserted-by":null},{"DOI":null,"key":"zoi260156r22","year":null,"issue":null,"author":null,"volume":null,"first-page":null,"unstructured":"Introducing ChatGPT Health. OpenAI. January 7, 2026. Accessed January 16, 2026. https://openai.com/index/introducing-chatgpt-health/","article-title":null,"journal-title":null,"doi-asserted-by":null},{"DOI":null,"key":"zoi260156r23","year":null,"issue":null,"author":null,"volume":null,"first-page":null,"unstructured":"Advancing Claude in healthcare and the life sciences. Anthropic. January 11, 2026. Accessed January 16, 2026. https://www.anthropic.com/news/healthcare-life-sciences","article-title":null,"journal-title":null,"doi-asserted-by":null},{"DOI":null,"key":"zoi260156r24","year":null,"issue":null,"author":null,"volume":null,"first-page":null,"unstructured":"Advancing cutting-edge AI capabilities. Google for Health. Accessed January 17, 2026. https://health.google/ai-models","article-title":null,"journal-title":null,"doi-asserted-by":null},{"DOI":"10.20944/preprints202310.1662.v1","key":"zoi260156r26","year":null,"issue":null,"author":null,"volume":null,"first-page":null,"unstructured":"Ahmad? M, Yaramic? I, Roy? TD. Creating trustworthy LLMs: dealing with hallucinations in healthcare AI.? arXiv [csCL]. Preprint posted online October 26, 2023. doi:10.20944/preprints202310.1662.v1","article-title":null,"journal-title":null,"doi-asserted-by":"crossref"},{"DOI":null,"key":"zoi260156r31","year":null,"issue":null,"author":null,"volume":null,"first-page":null,"unstructured":"Introducing HealthBench. OpenAI. Accessed July 29, 2025. https://openai.com/index/healthbench/","article-title":null,"journal-title":null,"doi-asserted-by":null},{"DOI":null,"key":"zoi260156r32","year":null,"issue":null,"author":null,"volume":null,"first-page":null,"unstructured":"Karthikesalingam? A, Natarajan? V. AMIE: a research AI system for diagnostic medical reasoning and conversations. Google Research. January 12, 2024. Accessed July 29, 2025. https://research.google/blog/amie-a-research-ai-system-for-diagnostic-medical-reasoning-and-conversations","article-title":null,"journal-title":null,"doi-asserted-by":null},{"DOI":null,"key":"zoi260156r33","year":null,"issue":null,"author":null,"volume":null,"first-page":null,"unstructured":"GPT-4o. OpenAI Platform. Accessed July 31, 2025. https://platform.openai.com/docs/models/gpt-4o","article-title":null,"journal-title":null,"doi-asserted-by":null},{"DOI":null,"key":"zoi260156r34","year":null,"issue":null,"author":null,"volume":null,"first-page":null,"unstructured":"GPT-o1. OpenAI Platform. Accessed July 31, 2025. https://platform.openai.com/docs/models/o1","article-title":null,"journal-title":null,"doi-asserted-by":null},{"DOI":null,"key":"zoi260156r35","year":null,"issue":null,"author":null,"volume":null,"first-page":null,"unstructured":"GPT-o1-Pro. OpenAI Platform. Accessed July 31, 2025. https://platform.openai.com/docs/models/o1-pro","article-title":null,"journal-title":null,"doi-asserted-by":null},{"DOI":null,"key":"zoi260156r36","year":null,"issue":null,"author":null,"volume":null,"first-page":null,"unstructured":"GPT-o3-Mini. OpenAI Platform. Accessed July 31, 2025. https://platform.openai.com/docs/models/o3-mini","article-title":null,"journal-title":null,"doi-asserted-by":null},{"DOI":null,"key":"zoi260156r37","year":null,"issue":null,"author":null,"volume":null,"first-page":null,"unstructured":"Introducing GPT-4.5. OpenAI. February 27, 2025. Accessed July 31, 2025. https://openai.com/index/introducing-gpt-4-5/","article-title":null,"journal-title":null,"doi-asserted-by":null},{"DOI":null,"key":"zoi260156r38","year":null,"issue":null,"author":null,"volume":null,"first-page":null,"unstructured":"Models overview. Anthropic. Accessed July 31, 2025. https://docs.anthropic.com/en/docs/about-claude/models/overview","article-title":null,"journal-title":null,"doi-asserted-by":null},{"DOI":null,"key":"zoi260156r39","year":null,"issue":null,"author":null,"volume":null,"first-page":null,"unstructured":"Models & pricing: DeepSeek API Docs. DeepSeek. Accessed July 31, 2025. https://api-docs.deepseek.com/quick_start/pricing","article-title":null,"journal-title":null,"doi-asserted-by":null},{"DOI":null,"key":"zoi260156r40","year":null,"issue":null,"author":null,"volume":null,"first-page":null,"unstructured":"Gemini models. Google AI for Developers. Accessed July 31, 2025. https://ai.google.dev/gemini-api/docs/models","article-title":null,"journal-title":null,"doi-asserted-by":null},{"DOI":null,"key":"zoi260156r41","year":null,"issue":null,"author":null,"volume":null,"first-page":null,"unstructured":"Grok 3 Beta: the age of reasoning agents. xAI. February 19, 2025. Accessed July 31, 2025. https://x.ai/news/grok-3","article-title":null,"journal-title":null,"doi-asserted-by":null},{"DOI":null,"key":"zoi260156r42","year":null,"issue":null,"author":null,"volume":null,"first-page":null,"unstructured":"Grok 4. xAI. July 9, 2025. Accessed January 17, 2026. https://x.ai/news/grok-4","article-title":null,"journal-title":null,"doi-asserted-by":null},{"DOI":null,"key":"zoi260156r43","year":null,"issue":null,"author":null,"volume":null,"first-page":null,"unstructured":"Case studies. Merck Manual Professional Version. Accessed July 31, 2025. https://www.merckmanuals.com/professional/pages-with-widgets/case-studies?mode=list","article-title":null,"journal-title":null,"doi-asserted-by":null}],"short-title":[],"journal-issue":{"issue":"4","published-print":{"date-parts":[[2026,4,1]]}},"content-domain":{"domain":[],"crossmark-restriction":false},"original-title":[],"container-title":["JAMA Network Open"],"reference-count":43,"published-online":{"date-time":null,"timestamp":null,"date-parts":[[2026,4,13]]},"references-count":43,"short-container-title":["JAMA Netw Open"],"is-referenced-by-count":3},"unpaywall":{"doi":"10.1001/jamanetworkopen.2026.4003","is_oa":true,"title":"Large Language Model Performance and Clinical Reasoning Tasks","updated":"2026-04-18T08:12:42+00:00","published_date":"2026-04-13","best_oa_location":{"url":"https://doi.org/10.1001/jamanetworkopen.2026.4003","pmh_id":null,"url_for_pdf":null,"url_for_landing_page":"https://doi.org/10.1001/jamanetworkopen.2026.4003"}}},"created_at":"2026-05-07T22:49:09.975Z","updated_at":"2026-05-14T15:13:26.918Z","tags":[{"paper_doi":"10.1001/jamanetworkopen.2026.4003","tag_type":"age_group","value":"mixed","value_numeric":null},{"paper_doi":"10.1001/jamanetworkopen.2026.4003","tag_type":"category","value":"Medicine, General & Internal","value_numeric":null},{"paper_doi":"10.1001/jamanetworkopen.2026.4003","tag_type":"condition","value":"Clinical Reasoning","value_numeric":null},{"paper_doi":"10.1001/jamanetworkopen.2026.4003","tag_type":"generic","value":"Claude 4.5 Opus","value_numeric":null},{"paper_doi":"10.1001/jamanetworkopen.2026.4003","tag_type":"generic","value":"Gemini 3.0","value_numeric":null},{"paper_doi":"10.1001/jamanetworkopen.2026.4003","tag_type":"generic","value":"GPT-5","value_numeric":null},{"paper_doi":"10.1001/jamanetworkopen.2026.4003","tag_type":"generic","value":"Grok 4","value_numeric":null},{"paper_doi":"10.1001/jamanetworkopen.2026.4003","tag_type":"impact_flag","value":"safety_signal","value_numeric":null},{"paper_doi":"10.1001/jamanetworkopen.2026.4003","tag_type":"level_of_evidence","value":"3","value_numeric":null},{"paper_doi":"10.1001/jamanetworkopen.2026.4003","tag_type":"modality","value":"Large Language Model","value_numeric":null},{"paper_doi":"10.1001/jamanetworkopen.2026.4003","tag_type":"sample_size","value":"n=16254 responses (21 LLMs × 29 vignettes)","value_numeric":16254},{"paper_doi":"10.1001/jamanetworkopen.2026.4003","tag_type":"study_design","value":"Cross-Sectional Study","value_numeric":null},{"paper_doi":"10.1001/jamanetworkopen.2026.4003","tag_type":"warning","value":"Medical student scorers were used instead of attending physicians, which introduces potential scoring variability especially for nuanced reasoning domains.","value_numeric":null},{"paper_doi":"10.1001/jamanetworkopen.2026.4003","tag_type":"warning","value":"Several models evaluated (GPT-5, Claude 4.5 Opus, Grok 4) are frontier models released close to or after the study period; their off-the-shelf configurations may have changed since evaluation.","value_numeric":null},{"paper_doi":"10.1001/jamanetworkopen.2026.4003","tag_type":"warning","value":"The PrIME-LLM scoring framework is novel and introduced by this study; external validation of this benchmark has not yet been performed.","value_numeric":null},{"paper_doi":"10.1001/jamanetworkopen.2026.4003","tag_type":"warning","value":"The 'sample_size' tag reflects total responses (16,254), not patients or unique cases — the actual vignette count is only 29, which is a small evaluation set.","value_numeric":null}],"summaries":{"what_studied":"This cross-sectional study evaluated 21 off-the-shelf LLMs (including GPT-5, Claude 4.5 Opus, Gemini 3.0 Flash/Pro, and Grok 4) on longitudinal clinical reasoning using 29 standardized MSD Manual vignettes scored by medical students across 5 domains: differential diagnosis, diagnostic testing, final diagnosis, management, and miscellaneous reasoning.","key_result":"PrIME-LLM scores ranged from 0.64 (Gemini 1.5 Flash) to 0.78 (Grok 4); differential diagnosis failure rates exceeded 0.80 in all models, while final diagnosis failure rates were below 0.40 — reasoning-optimized models outperformed standard models, and multimodal inputs (images) generally improved accuracy.","limitations":"- Only 29 clinical vignettes were tested, limiting generalizability across clinical complexity and specialty breadth.\n- Scorers were medical students rather than expert clinicians, which may affect scoring reliability.\n- Off-the-shelf models were evaluated without clinical fine-tuning, which may underestimate potential real-world performance.","practice_takeaway":"Current frontier LLMs can reliably reach a final diagnosis but struggle significantly with generating differential diagnoses — a core early clinical reasoning step. Clinicians should not rely on LLMs for open-ended diagnostic reasoning without careful human oversight."}}],"initialOffset":21,"fetchConfig":{"kind":"tags","tags":[{"tag_type":"modality","value":"Large Language Model"}]},"breadcrumbItems":[{"name":"Home","href":"/"},{"name":"Research Hub","href":"/hub"},{"name":"Modalities","href":"/hub/tag/modality"},{"name":"Large Language Model"}],"hrefConfig":{"basePath":"/hub/papers","webViewQueryParams":{"hub_ctx":"tag","hub_tag_type":"modality","hub_tag_value":"Large Language Model"}}}]]

Papers