2f:I[703058,["/_next/static/immutable/chunks/3bhq_5eh1d760.js","/_next/static/immutable/chunks/2iy5wruuedsh6.js","/_next/static/immutable/chunks/2pfsjpn6k6zlg.js","/_next/static/immutable/chunks/26c7v3_3uwnwu.js","/_next/static/immutable/chunks/00yc7rcvua6a7.js","/_next/static/immutable/chunks/37ktb71aq3480.js","/_next/static/immutable/chunks/2d50b-7eu4558.js","/_next/static/immutable/chunks/06_yt270h41rg.js","/_next/static/immutable/chunks/2d6noqpp6ehas.js","/_next/static/immutable/chunks/1sx186c93bpzg.js","/_next/static/immutable/chunks/3dc-1n866s65i.js","/_next/static/immutable/chunks/20x2y96ldu5wx.js","/_next/static/immutable/chunks/2w1bvggzb0ga8.js","/_next/static/immutable/chunks/2051ffx3xzio9.js"],"PaperListPageClient"] 30:Taa1,Large language models (LLMs) are increasingly marketed for clinical use, yet their ability to replicate full-spectrum clinical reasoning remains uncertain. Existing evaluations often rely on multiple-choice examinations that do not reflect the complexity of patient care. To evaluate the longitudinal clinical reasoning ability of state-of-the-art LLMs and to introduce a multidimensional, clinically meaningful benchmark for clinical-grade artificial intelligence (AI). In this cross-sectional study, performance was evaluated using standardized clinical vignettes from the January 2025 update of MSD Manual vignettes. A total of 21 off-the-shelf LLMs, including recently released GPT-5, Claude 4.5 Opus, Gemini 3.0 Flash and Pro, and Grok 4, were evaluated. Models were assessed by medical student scorers in triplicate across sequential stages of the standard clinical workflow. Analyses were performed from January to December 2025. The primary outcome was the Proportional Index of Medical Evaluation for LLMs (PrIME-LLM) score, defined as the normalized polygonal area representing balanced accuracy across 5 domains of clinical reasoning as follows: differential diagnosis, diagnostic testing, final diagnosis, management, and miscellaneous clinical reasoning questions. Analyses including analyses of variance, t tests, and regression models were used to compare AI model performance and demographic associations. LLMs were tested across 29 clinical vignettes (representing 16 254 responses in total). PrIME-LLM scores ranged from 0.64 (range, 0.63-0.65) (Gemini 1.5 Flash) to 0.78 (range, 0.77-0.79) (Grok 4), with reasoning-optimized models outperforming nonreasoning models and GPT models scoring highest overall. Differential diagnosis was less accurate than diagnostic testing, while final diagnosis, management, and miscellaneous reasoning were more accurate. Failure rates exceeded 0.80 (range, 0.90-1.00) for differential diagnosis in all models but were less than 0.40 (range, 0.09-0.39) for final diagnosis. Multimodal performance was robust; most LLM models showed improved accuracy with image inputs. In this cross-sectional study of 21 LLMs, frontier LLMs achieved high accuracy on final diagnoses but performed poorly in generating differential diagnoses and navigating uncertainty relative to other reasoning stages. The PrIME-LLM framework provided greater separation than raw accuracy, revealing critical reasoning gaps obscured by traditional benchmarks. Thus, despite version-based improvements and advantages in reasoning-optimized models, off-the-shelf LLMs have not yet achieved the intelligence required for safe deployment and remain limited in demonstrating advanced clinical reasoning.31:Tf4a, Importance Large language models (LLMs) are increasingly marketed for clinical use, yet their ability to replicate full-spectrum clinical reasoning remains uncertain. Existing evaluations often rely on multiple-choice examinations that do not reflect the complexity of patient care. Objectives To evaluate the longitudinal clinical reasoning ability of state-of-the-art LLMs and to introduce a multidimensional, clinically meaningful benchmark for clinical-grade artificial intelligence (AI). Design, Setting, and Participants In this cross-sectional study, performance was evaluated using standardized clinical vignettes from the January 2025 update of MSD Manual vignettes. A total of 21 off-the-shelf LLMs, including recently released GPT-5, Claude 4.5 Opus, Gemini 3.0 Flash and Pro, and Grok 4, were evaluated. Models were assessed by medical student scorers in triplicate across sequential stages of the standard clinical workflow. Analyses were performed from January to December 2025. Main Outcomes and Measures The primary outcome was the Proportional Index of Medical Evaluation for LLMs (PrIME-LLM) score, defined as the normalized polygonal area representing balanced accuracy across 5 domains of clinical reasoning as follows: differential diagnosis, diagnostic testing, final diagnosis, management, and miscellaneous clinical reasoning questions. Analyses including analyses of variance, t tests, and regression models were used to compare AI model performance and demographic associations. Results LLMs were tested across 29 clinical vignettes (representing 16 254 responses in total). PrIME-LLM scores ranged from 0.64 (range, 0.63-0.65) (Gemini 1.5 Flash) to 0.78 (range, 0.77-0.79) (Grok 4), with reasoning-optimized models outperforming nonreasoning models and GPT models scoring highest overall. Differential diagnosis was less accurate than diagnostic testing, while final diagnosis, management, and miscellaneous reasoning were more accurate. Failure rates exceeded 0.80 (range, 0.90-1.00) for differential diagnosis in all models but were less than 0.40 (range, 0.09-0.39) for final diagnosis. Multimodal performance was robust; most LLM models showed improved accuracy with image inputs. Conclusions and Relevance In this cross-sectional study of 21 LLMs, frontier LLMs achieved high accuracy on final diagnoses but performed poorly in generating differential diagnoses and navigating uncertainty relative to other reasoning stages. The PrIME-LLM framework provided greater separation than raw accuracy, revealing critical reasoning gaps obscured by traditional benchmarks. Thus, despite version-based improvements and advantages in reasoning-optimized models, off-the-shelf LLMs have not yet achieved the intelligence required for safe deployment and remain limited in demonstrating advanced clinical reasoning. 25:[[["$","script","0",{"type":"application/ld+json","dangerouslySetInnerHTML":{"__html":"{\"@context\":\"https://schema.org\",\"@type\":\"CollectionPage\",\"name\":\"Claude 4.5 Opus: generic Research & Clinical Takeaways\",\"description\":\"Explore peer-reviewed papers on Claude 4.5 Opus (generic). Clinician-written summaries and actionable practice insights. Updated daily.\",\"url\":\"https://www.reachrx.ai/hub/tag/generic/Claude%204.5%20Opus\",\"breadcrumb\":{\"@type\":\"BreadcrumbList\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https://www.reachrx.ai\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Research Hub\",\"item\":\"https://www.reachrx.ai/hub\"},{\"@type\":\"ListItem\",\"position\":3,\"name\":\"Generics\",\"item\":\"https://www.reachrx.ai/hub/tag/generic\"},{\"@type\":\"ListItem\",\"position\":4,\"name\":\"Claude 4.5 Opus\",\"item\":\"https://www.reachrx.ai/hub/tag/generic/Claude%204.5%20Opus\"}]}}"}}]],["$","$L2f",null,{"title":"Claude 4.5 Opus: Research & Clinical Takeaways","description":"Explore research on Claude 4.5 Opus (generic) — efficacy, safety, and comparative data.","papers":[{"doi":"10.1001/jamanetworkopen.2026.4003","pmid":"41973425","title":"Large Language Model Performance and Clinical Reasoning Tasks","journal_title":"JAMA Network Open","journal_issn":"2574-3805","published_at":"2026-04-13T00:00:00.000Z","url":"https://doi.org/10.1001/jamanetworkopen.2026.4003","source_score":1062,"is_retracted":false,"is_open_access":true,"raw_metadata":{"ooir":{"day":"2026-04-14","doi":"10.1001/jamanetworkopen.2026.4003","days":"30","issn":"2574-3805","rank":"2","field":"Clinical Medicine","score":"1062","category":"Medicine, General & Internal","categories":["Medicine, General & Internal"],"_categories":["Medicine, General & Internal"]},"pubmed":{"doi":"10.1001/jamanetworkopen.2026.4003","title":"Large Language Model Performance and Clinical Reasoning Tasks.","journal":"JAMA network open","methods":"","results":"LLMs were tested across 29 clinical vignettes (representing 16 254 responses in total). PrIME-LLM scores ranged from 0.64 (range, 0.63-0.65) (Gemini 1.5 Flash) to 0.78 (range, 0.77-0.79) (Grok 4), with reasoning-optimized models outperforming nonreasoning models and GPT models scoring highest overall. Differential diagnosis was less accurate than diagnostic testing, while final diagnosis, management, and miscellaneous reasoning were more accurate. Failure rates exceeded 0.80 (range, 0.90-1.00) for differential diagnosis in all models but were less than 0.40 (range, 0.09-0.39) for final diagnosis. Multimodal performance was robust; most LLM models showed improved accuracy with image inputs.","abstract":"$30","keywords":[],"pubmed_id":"41973425","conclusions":"","publication_date":"2026-04-13T00:00:00"},"crossref":{"DOI":"10.1001/jamanetworkopen.2026.4003","URL":"https://doi.org/10.1001/jamanetworkopen.2026.4003","ISSN":["2574-3805"],"link":[{"URL":"https://jamanetwork.com/journals/jamanetworkopen/articlepdf/2847679/rao_2026_oi_260156_1775496096.59826.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"page":"e264003","type":"journal-article","issue":"4","score":1,"title":["Large Language Model Performance and Clinical Reasoning Tasks"],"author":[{"given":"Arya S.","family":"Rao","sequence":"first","affiliation":[{"name":"Harvard Medical School, Boston, Massachusetts"},{"name":"Medically Engineered Solutions in Healthcare Incubator, Innovation in Operations Research Center, Mass General Brigham, Boston, Massachusetts"}]},{"given":"Kaiz P.","family":"Esmail","sequence":"additional","affiliation":[{"name":"Harvard Medical School, Boston, Massachusetts"},{"name":"Medically Engineered Solutions in Healthcare Incubator, Innovation in Operations Research Center, Mass General Brigham, Boston, Massachusetts"}]},{"given":"Richard S.","family":"Lee","sequence":"additional","affiliation":[{"name":"Harvard Medical School, Boston, Massachusetts"},{"name":"Medically Engineered Solutions in Healthcare Incubator, Innovation in Operations Research Center, Mass General Brigham, Boston, Massachusetts"}]},{"given":"Sharon","family":"Jiang","sequence":"additional","affiliation":[{"name":"Harvard Medical School, Boston, Massachusetts"},{"name":"Medically Engineered Solutions in Healthcare Incubator, Innovation in Operations Research Center, Mass General Brigham, Boston, Massachusetts"}]},{"given":"Bianca","family":"Arraiza Carlo","sequence":"additional","affiliation":[{"name":"Medically Engineered Solutions in Healthcare Incubator, Innovation in Operations Research Center, Mass General Brigham, Boston, Massachusetts"},{"name":"Harvard College, Cambridge, Massachusetts"}]},{"given":"Jasleen","family":"Gill","sequence":"additional","affiliation":[{"name":"Harvard Medical School, Boston, Massachusetts"},{"name":"Medically Engineered Solutions in Healthcare Incubator, Innovation in Operations Research Center, Mass General Brigham, Boston, Massachusetts"}]},{"given":"Praneet","family":"Khanna","sequence":"additional","affiliation":[{"name":"Medically Engineered Solutions in Healthcare Incubator, Innovation in Operations Research Center, Mass General Brigham, Boston, Massachusetts"},{"name":"University of Missouri–Kansas City School of Medicine, Kansas City"}]},{"given":"Ezra","family":"Kalmowitz","sequence":"additional","affiliation":[{"name":"Harvard Medical School, Boston, Massachusetts"},{"name":"Medically Engineered Solutions in Healthcare Incubator, Innovation in Operations Research Center, Mass General Brigham, Boston, Massachusetts"}]},{"given":"Basile","family":"Montagnese","sequence":"additional","affiliation":[{"name":"Harvard Medical School, Boston, Massachusetts"},{"name":"Medically Engineered Solutions in Healthcare Incubator, Innovation in Operations Research Center, Mass General Brigham, Boston, Massachusetts"}]},{"given":"Kimia","family":"Heydari","sequence":"additional","affiliation":[{"name":"Harvard Medical School, Boston, Massachusetts"},{"name":"Medically Engineered Solutions in Healthcare Incubator, Innovation in Operations Research Center, Mass General Brigham, Boston, Massachusetts"}]},{"given":"Qiao","family":"Jiao","sequence":"additional","affiliation":[{"name":"Harvard Medical School, Boston, Massachusetts"},{"name":"Medically Engineered Solutions in Healthcare Incubator, Innovation in Operations Research Center, Mass General Brigham, Boston, Massachusetts"}]},{"given":"Ethan","family":"Bott","sequence":"additional","affiliation":[{"name":"Harvard Medical School, Boston, Massachusetts"},{"name":"Medically Engineered Solutions in Healthcare Incubator, Innovation in Operations Research Center, Mass General Brigham, Boston, Massachusetts"}]},{"given":"Dan","family":"Nguyen","sequence":"additional","affiliation":[{"name":"Medically Engineered Solutions in Healthcare Incubator, Innovation in Operations Research Center, Mass General Brigham, Boston, Massachusetts"},{"name":"University of Massachusetts Chan Medical School, Worcester"}]},{"given":"Grace","family":"Wang","sequence":"additional","affiliation":[{"name":"Harvard Medical School, Boston, Massachusetts"},{"name":"Medically Engineered Solutions in Healthcare Incubator, Innovation in Operations Research Center, Mass General Brigham, Boston, Massachusetts"}]},{"given":"Michael","family":"Hood","sequence":"additional","affiliation":[{"name":"Medically Engineered Solutions in Healthcare Incubator, Innovation in Operations Research Center, Mass General Brigham, Boston, Massachusetts"},{"name":"Massachusetts General Hospital, Boston"}]},{"given":"Adam B.","family":"Landman","sequence":"additional","affiliation":[{"name":"Harvard Medical School, Boston, Massachusetts"},{"name":"Mass General Brigham, Boston, Massachusetts"},{"name":"Brigham and Women’s Hospital, Boston, Massachusetts"}]},{"given":"Marc D.","family":"Succi","sequence":"additional","affiliation":[{"name":"Harvard Medical School, Boston, Massachusetts"},{"name":"Medically Engineered Solutions in Healthcare Incubator, Innovation in Operations Research Center, Mass General Brigham, Boston, Massachusetts"},{"name":"Massachusetts General Hospital, Boston"}]}],"issued":{"date-parts":[[2026,4,13]]},"member":"10","prefix":"10.1001","source":"Crossref","volume":"9","created":{"date-time":"2026-04-13T15:30:25Z","timestamp":1776094225000,"date-parts":[[2026,4,13]]},"indexed":{"date-time":"2026-05-11T18:18:03Z","timestamp":1778523483997,"date-parts":[[2026,5,11]]},"subject":[],"abstract":"$31","language":"en","relation":{},"resource":{"primary":{"URL":"https://jamanetwork.com/journals/jamanetworkopen/fullarticle/2847679"}},"subtitle":[],"deposited":{"date-time":"2026-04-13T15:30:28Z","timestamp":1776094228000,"date-parts":[[2026,4,13]]},"issn-type":[{"type":"electronic","value":"2574-3805"}],"published":{"date-parts":[[2026,4,13]]},"publisher":"American Medical Association (AMA)","reference":[{"DOI":"10.1038/s41551-018-0305-z","key":"zoi260156r1","year":"2018","issue":"10","author":"Yu","volume":"2","first-page":"719","unstructured":null,"article-title":"Artificial intelligence in healthcare.","journal-title":"Nat Biomed Eng","doi-asserted-by":"publisher"},{"DOI":"10.1038/s41746-022-00742-2","key":"zoi260156r2","year":"2022","issue":"1","author":"Yang","volume":"5","first-page":"194","unstructured":null,"article-title":"A large language model for electronic health records.","journal-title":"NPJ Digit Med","doi-asserted-by":"publisher"},{"DOI":"10.1016/S2589-7500(24)00060-8","key":"zoi260156r3","year":"2024","issue":"6","author":"Chen","volume":"6","first-page":"e379","unstructured":null,"article-title":"The effect of using a large language model to respond to patient messages.","journal-title":"Lancet Digit Health","doi-asserted-by":"publisher"},{"DOI":"10.1038/s41467-024-52415-1","key":"zoi260156r4","year":"2024","issue":"1","author":"Williams","volume":"15","first-page":"8236","unstructured":null,"article-title":"Evaluating the use of large language models to provide clinical recommendations in the emergency department.","journal-title":"Nat Commun","doi-asserted-by":"publisher"},{"DOI":"10.1038/s41598-025-88074-5","key":"zoi260156r5","year":"2025","issue":"1","author":"Koirala","volume":"15","first-page":"3455","unstructured":null,"article-title":"Evaluating AI performance in nephrology triage and subspecialty referrals.","journal-title":"Sci Rep","doi-asserted-by":"publisher"},{"DOI":"10.1001/jamanetworkopen.2023.43689","key":"zoi260156r6","year":"2023","issue":"11","author":"Benary","volume":"6","first-page":null,"unstructured":null,"article-title":"Leveraging large language models for decision support in personalized oncology.","journal-title":"JAMA Netw Open","doi-asserted-by":"publisher"},{"DOI":"10.1038/s41746-024-01391-3","key":"zoi260156r7","year":"2024","issue":"1","author":"Ong","volume":"7","first-page":"364","unstructured":null,"article-title":"SurgeryLLM: a retrieval-augmented generation large language model framework for surgical decision support and workflow enhancement.","journal-title":"NPJ Digit Med","doi-asserted-by":"publisher"},{"DOI":"10.2196/48659","key":"zoi260156r8","year":"2023","issue":"1","author":"Rao","volume":"25","first-page":null,"unstructured":null,"article-title":"Assessing the utility of ChatGPT throughout the entire clinical workflow: development and usability study.","journal-title":"J Med Internet Res","doi-asserted-by":"publisher"},{"DOI":"10.2196/51199","key":"zoi260156r9","year":"2023","issue":null,"author":"Koranteng","volume":"9","first-page":null,"unstructured":null,"article-title":"Empathy and equity: key considerations for large language model adoption in health care.","journal-title":"JMIR Med Educ","doi-asserted-by":"publisher"},{"DOI":"10.1007/s10916-024-02058-y","key":"zoi260156r10","year":"2024","issue":"1","author":"Rao","volume":"48","first-page":"41","unstructured":null,"article-title":"Proactive polypharmacy management using large language models: opportunities to enhance geriatric care.","journal-title":"J Med Syst","doi-asserted-by":"publisher"},{"DOI":"10.1002/ajmg.a.63878","key":"zoi260156r12","year":"2025","issue":"2","author":"Young","volume":"197","first-page":null,"unstructured":null,"article-title":"Diagnostic accuracy of a custom large language model on rare pediatric disease case reports.","journal-title":"Am J Med Genet A","doi-asserted-by":"publisher"},{"DOI":"10.1097/j.pain.0000000000003388","key":"zoi260156r13","year":"2025","issue":"3","author":"Young","volume":"166","first-page":"511","unstructured":null,"article-title":"Racial, ethnic, and sex bias in large language model opioid recommendations for pain management.","journal-title":"Pain","doi-asserted-by":"publisher"},{"DOI":"10.1016/j.clinimag.2024.110335","key":"zoi260156r14","year":"2025","issue":null,"author":"Nguyen","volume":"117","first-page":null,"unstructured":null,"article-title":"Exploring the accuracy of embedded ChatGPT-4 and ChatGPT-4o in generating BI-RADS scores: a pilot study in radiologic clinical support.","journal-title":"Clin Imaging","doi-asserted-by":"publisher"},{"DOI":"10.1016/j.japh.2024.102304","key":"zoi260156r15","year":"2025","issue":"1","author":"Kim","volume":"65","first-page":null,"unstructured":null,"article-title":"Risk stratification of potential drug interactions involving common over-the-counter medications and herbal supplements by a large language model.","journal-title":"J Am Pharm Assoc (2003)","doi-asserted-by":"publisher"},{"DOI":"10.1007/s10439-025-03701-6","key":"zoi260156r16","year":"2025","issue":"5","author":"Rao","volume":"53","first-page":"1199","unstructured":null,"article-title":"A future of self-directed patient internet research: large language model-based tools versus standard search engines.","journal-title":"Ann Biomed Eng","doi-asserted-by":"publisher"},{"DOI":"10.1001/jama.2025.2789","key":"zoi260156r17","year":"2025","issue":"19","author":"Succi","volume":"333","first-page":"1665","unstructured":null,"article-title":"Building the AI-enabled medical school of the future.","journal-title":"JAMA","doi-asserted-by":"publisher"},{"DOI":"10.1038/s41746-025-01650-x","key":"zoi260156r18","year":"2025","issue":"1","author":"Rao","volume":"8","first-page":"247","unstructured":null,"article-title":"Synthetic medical education in dermatology leveraging generative artificial intelligence.","journal-title":"NPJ Digit Med","doi-asserted-by":"publisher"},{"DOI":"10.1016/j.mcpdig.2025.100241","key":"zoi260156r19","year":"2025","issue":"3","author":"Rao","volume":"3","first-page":null,"unstructured":null,"article-title":"Development and evaluation of an artificial intelligence-powered surgical oral examination simulator: a pilot study.","journal-title":"Mayo Clin Proc Digit Health","doi-asserted-by":"publisher"},{"DOI":"10.1097/XCS.0000000000001505","key":"zoi260156r20","year":"2025","issue":"5","author":"Rao","volume":"241","first-page":"887","unstructured":null,"article-title":"The digital standardized patient: an artificial intelligence coach for cultural dexterity in surgical care.","journal-title":"J Am Coll Surg","doi-asserted-by":"publisher"},{"DOI":"10.3389/fmed.2024.1495582","key":"zoi260156r25","year":"2024","issue":null,"author":"Bélisle-Pipon","volume":"11","first-page":null,"unstructured":null,"article-title":"Why we need to be careful with LLMs in medicine.","journal-title":"Front Med (Lausanne)","doi-asserted-by":"publisher"},{"DOI":"10.1371/journal.pdig.0000198","key":"zoi260156r27","year":"2023","issue":"2","author":"Kung","volume":"2","first-page":null,"unstructured":null,"article-title":"Performance of ChatGPT on USMLE: potential for AI-assisted medical education using large language models.","journal-title":"PLOS Digit Health","doi-asserted-by":"publisher"},{"DOI":"10.1056/AIdbp2300192","key":"zoi260156r28","year":"2024","issue":"5","author":"Katz","volume":"1","first-page":null,"unstructured":null,"article-title":"GPT versus resident physicians—a benchmark based on official board scores.","journal-title":"NEJM AI","doi-asserted-by":"publisher"},{"DOI":"10.1038/s41598-024-58760-x","key":"zoi260156r29","year":"2024","issue":"1","author":"Shieh","volume":"14","first-page":"9330","unstructured":null,"article-title":"Assessing ChatGPT 4.0's test performance and clinical diagnostic accuracy on USMLE STEP 2 CK and clinical case reports.","journal-title":"Sci Rep","doi-asserted-by":"publisher"},{"DOI":"10.1038/s41467-024-55628-6","key":"zoi260156r30","year":"2025","issue":"1","author":"Griot","volume":"16","first-page":"642","unstructured":null,"article-title":"Large language models lack essential metacognition for reliable medical reasoning.","journal-title":"Nat Commun","doi-asserted-by":"publisher"},{"DOI":"10.1101/2024.08.06.24311544","key":"zoi260156r11","year":null,"issue":null,"author":null,"volume":null,"first-page":null,"unstructured":"Young? CC, Enichen? E, Rao? A, . Pilot study of large language models as an age-appropriate explanatory tool for chronic pediatric conditions.? medRxiv. Preprint posted online August 7, 2024. doi:10.1101/2024.08.06.24311544","article-title":null,"journal-title":null,"doi-asserted-by":"crossref"},{"DOI":null,"key":"zoi260156r21","year":null,"issue":null,"author":null,"volume":null,"first-page":null,"unstructured":"Introducing GPT-5. OpenAI. August 7, 2025. Accessed August 9, 2025. https://openai.com/index/introducing-gpt-5/","article-title":null,"journal-title":null,"doi-asserted-by":null},{"DOI":null,"key":"zoi260156r22","year":null,"issue":null,"author":null,"volume":null,"first-page":null,"unstructured":"Introducing ChatGPT Health. OpenAI. January 7, 2026. Accessed January 16, 2026. https://openai.com/index/introducing-chatgpt-health/","article-title":null,"journal-title":null,"doi-asserted-by":null},{"DOI":null,"key":"zoi260156r23","year":null,"issue":null,"author":null,"volume":null,"first-page":null,"unstructured":"Advancing Claude in healthcare and the life sciences. Anthropic. January 11, 2026. Accessed January 16, 2026. https://www.anthropic.com/news/healthcare-life-sciences","article-title":null,"journal-title":null,"doi-asserted-by":null},{"DOI":null,"key":"zoi260156r24","year":null,"issue":null,"author":null,"volume":null,"first-page":null,"unstructured":"Advancing cutting-edge AI capabilities. Google for Health. Accessed January 17, 2026. https://health.google/ai-models","article-title":null,"journal-title":null,"doi-asserted-by":null},{"DOI":"10.20944/preprints202310.1662.v1","key":"zoi260156r26","year":null,"issue":null,"author":null,"volume":null,"first-page":null,"unstructured":"Ahmad? M, Yaramic? I, Roy? TD. Creating trustworthy LLMs: dealing with hallucinations in healthcare AI.? arXiv [csCL]. Preprint posted online October 26, 2023. doi:10.20944/preprints202310.1662.v1","article-title":null,"journal-title":null,"doi-asserted-by":"crossref"},{"DOI":null,"key":"zoi260156r31","year":null,"issue":null,"author":null,"volume":null,"first-page":null,"unstructured":"Introducing HealthBench. OpenAI. Accessed July 29, 2025. https://openai.com/index/healthbench/","article-title":null,"journal-title":null,"doi-asserted-by":null},{"DOI":null,"key":"zoi260156r32","year":null,"issue":null,"author":null,"volume":null,"first-page":null,"unstructured":"Karthikesalingam? A, Natarajan? V. AMIE: a research AI system for diagnostic medical reasoning and conversations. Google Research. January 12, 2024. Accessed July 29, 2025. https://research.google/blog/amie-a-research-ai-system-for-diagnostic-medical-reasoning-and-conversations","article-title":null,"journal-title":null,"doi-asserted-by":null},{"DOI":null,"key":"zoi260156r33","year":null,"issue":null,"author":null,"volume":null,"first-page":null,"unstructured":"GPT-4o. OpenAI Platform. Accessed July 31, 2025. https://platform.openai.com/docs/models/gpt-4o","article-title":null,"journal-title":null,"doi-asserted-by":null},{"DOI":null,"key":"zoi260156r34","year":null,"issue":null,"author":null,"volume":null,"first-page":null,"unstructured":"GPT-o1. OpenAI Platform. Accessed July 31, 2025. https://platform.openai.com/docs/models/o1","article-title":null,"journal-title":null,"doi-asserted-by":null},{"DOI":null,"key":"zoi260156r35","year":null,"issue":null,"author":null,"volume":null,"first-page":null,"unstructured":"GPT-o1-Pro. OpenAI Platform. Accessed July 31, 2025. https://platform.openai.com/docs/models/o1-pro","article-title":null,"journal-title":null,"doi-asserted-by":null},{"DOI":null,"key":"zoi260156r36","year":null,"issue":null,"author":null,"volume":null,"first-page":null,"unstructured":"GPT-o3-Mini. OpenAI Platform. Accessed July 31, 2025. https://platform.openai.com/docs/models/o3-mini","article-title":null,"journal-title":null,"doi-asserted-by":null},{"DOI":null,"key":"zoi260156r37","year":null,"issue":null,"author":null,"volume":null,"first-page":null,"unstructured":"Introducing GPT-4.5. OpenAI. February 27, 2025. Accessed July 31, 2025. https://openai.com/index/introducing-gpt-4-5/","article-title":null,"journal-title":null,"doi-asserted-by":null},{"DOI":null,"key":"zoi260156r38","year":null,"issue":null,"author":null,"volume":null,"first-page":null,"unstructured":"Models overview. Anthropic. Accessed July 31, 2025. https://docs.anthropic.com/en/docs/about-claude/models/overview","article-title":null,"journal-title":null,"doi-asserted-by":null},{"DOI":null,"key":"zoi260156r39","year":null,"issue":null,"author":null,"volume":null,"first-page":null,"unstructured":"Models & pricing: DeepSeek API Docs. DeepSeek. Accessed July 31, 2025. https://api-docs.deepseek.com/quick_start/pricing","article-title":null,"journal-title":null,"doi-asserted-by":null},{"DOI":null,"key":"zoi260156r40","year":null,"issue":null,"author":null,"volume":null,"first-page":null,"unstructured":"Gemini models. Google AI for Developers. Accessed July 31, 2025. https://ai.google.dev/gemini-api/docs/models","article-title":null,"journal-title":null,"doi-asserted-by":null},{"DOI":null,"key":"zoi260156r41","year":null,"issue":null,"author":null,"volume":null,"first-page":null,"unstructured":"Grok 3 Beta: the age of reasoning agents. xAI. February 19, 2025. Accessed July 31, 2025. https://x.ai/news/grok-3","article-title":null,"journal-title":null,"doi-asserted-by":null},{"DOI":null,"key":"zoi260156r42","year":null,"issue":null,"author":null,"volume":null,"first-page":null,"unstructured":"Grok 4. xAI. July 9, 2025. Accessed January 17, 2026. https://x.ai/news/grok-4","article-title":null,"journal-title":null,"doi-asserted-by":null},{"DOI":null,"key":"zoi260156r43","year":null,"issue":null,"author":null,"volume":null,"first-page":null,"unstructured":"Case studies. Merck Manual Professional Version. Accessed July 31, 2025. https://www.merckmanuals.com/professional/pages-with-widgets/case-studies?mode=list","article-title":null,"journal-title":null,"doi-asserted-by":null}],"short-title":[],"journal-issue":{"issue":"4","published-print":{"date-parts":[[2026,4,1]]}},"content-domain":{"domain":[],"crossmark-restriction":false},"original-title":[],"container-title":["JAMA Network Open"],"reference-count":43,"published-online":{"date-time":null,"timestamp":null,"date-parts":[[2026,4,13]]},"references-count":43,"short-container-title":["JAMA Netw Open"],"is-referenced-by-count":3},"unpaywall":{"doi":"10.1001/jamanetworkopen.2026.4003","is_oa":true,"title":"Large Language Model Performance and Clinical Reasoning Tasks","updated":"2026-04-18T08:12:42+00:00","published_date":"2026-04-13","best_oa_location":{"url":"https://doi.org/10.1001/jamanetworkopen.2026.4003","pmh_id":null,"url_for_pdf":null,"url_for_landing_page":"https://doi.org/10.1001/jamanetworkopen.2026.4003"}}},"created_at":"2026-05-07T22:49:09.975Z","updated_at":"2026-05-14T15:13:26.918Z","tags":[{"paper_doi":"10.1001/jamanetworkopen.2026.4003","tag_type":"age_group","value":"mixed","value_numeric":null},{"paper_doi":"10.1001/jamanetworkopen.2026.4003","tag_type":"category","value":"Medicine, General & Internal","value_numeric":null},{"paper_doi":"10.1001/jamanetworkopen.2026.4003","tag_type":"condition","value":"Clinical Reasoning","value_numeric":null},{"paper_doi":"10.1001/jamanetworkopen.2026.4003","tag_type":"generic","value":"Claude 4.5 Opus","value_numeric":null},{"paper_doi":"10.1001/jamanetworkopen.2026.4003","tag_type":"generic","value":"Gemini 3.0","value_numeric":null},{"paper_doi":"10.1001/jamanetworkopen.2026.4003","tag_type":"generic","value":"GPT-5","value_numeric":null},{"paper_doi":"10.1001/jamanetworkopen.2026.4003","tag_type":"generic","value":"Grok 4","value_numeric":null},{"paper_doi":"10.1001/jamanetworkopen.2026.4003","tag_type":"impact_flag","value":"safety_signal","value_numeric":null},{"paper_doi":"10.1001/jamanetworkopen.2026.4003","tag_type":"level_of_evidence","value":"3","value_numeric":null},{"paper_doi":"10.1001/jamanetworkopen.2026.4003","tag_type":"modality","value":"Large Language Model","value_numeric":null},{"paper_doi":"10.1001/jamanetworkopen.2026.4003","tag_type":"sample_size","value":"n=16254 responses (21 LLMs × 29 vignettes)","value_numeric":16254},{"paper_doi":"10.1001/jamanetworkopen.2026.4003","tag_type":"study_design","value":"Cross-Sectional Study","value_numeric":null},{"paper_doi":"10.1001/jamanetworkopen.2026.4003","tag_type":"warning","value":"Medical student scorers were used instead of attending physicians, which introduces potential scoring variability especially for nuanced reasoning domains.","value_numeric":null},{"paper_doi":"10.1001/jamanetworkopen.2026.4003","tag_type":"warning","value":"Several models evaluated (GPT-5, Claude 4.5 Opus, Grok 4) are frontier models released close to or after the study period; their off-the-shelf configurations may have changed since evaluation.","value_numeric":null},{"paper_doi":"10.1001/jamanetworkopen.2026.4003","tag_type":"warning","value":"The PrIME-LLM scoring framework is novel and introduced by this study; external validation of this benchmark has not yet been performed.","value_numeric":null},{"paper_doi":"10.1001/jamanetworkopen.2026.4003","tag_type":"warning","value":"The 'sample_size' tag reflects total responses (16,254), not patients or unique cases — the actual vignette count is only 29, which is a small evaluation set.","value_numeric":null}],"summaries":{"what_studied":"This cross-sectional study evaluated 21 off-the-shelf LLMs (including GPT-5, Claude 4.5 Opus, Gemini 3.0 Flash/Pro, and Grok 4) on longitudinal clinical reasoning using 29 standardized MSD Manual vignettes scored by medical students across 5 domains: differential diagnosis, diagnostic testing, final diagnosis, management, and miscellaneous reasoning.","key_result":"PrIME-LLM scores ranged from 0.64 (Gemini 1.5 Flash) to 0.78 (Grok 4); differential diagnosis failure rates exceeded 0.80 in all models, while final diagnosis failure rates were below 0.40 — reasoning-optimized models outperformed standard models, and multimodal inputs (images) generally improved accuracy.","limitations":"- Only 29 clinical vignettes were tested, limiting generalizability across clinical complexity and specialty breadth.\n- Scorers were medical students rather than expert clinicians, which may affect scoring reliability.\n- Off-the-shelf models were evaluated without clinical fine-tuning, which may underestimate potential real-world performance.","practice_takeaway":"Current frontier LLMs can reliably reach a final diagnosis but struggle significantly with generating differential diagnoses — a core early clinical reasoning step. Clinicians should not rely on LLMs for open-ended diagnostic reasoning without careful human oversight."}}],"initialOffset":1,"fetchConfig":{"kind":"tags","tags":[{"tag_type":"generic","value":"Claude 4.5 Opus"}]},"breadcrumbItems":[{"name":"Home","href":"/"},{"name":"Research Hub","href":"/hub"},{"name":"Generics","href":"/hub/tag/generic"},{"name":"Claude 4.5 Opus"}],"hrefConfig":{"basePath":"/hub/papers","webViewQueryParams":{"hub_ctx":"tag","hub_tag_type":"generic","hub_tag_value":"Claude 4.5 Opus"}}}]]

Papers