MediAgent: A Multi-Agent Framework for Reliable Clinical Decision Support with Tool-Grounded Verification
Abstract
Large language models (LLMs) are increasingly explored for clinical decision support, yet their tendency to hallucinate medical facts and inability to provide verifiable evidence pose significant safety risks in healthcare settings. We present MediAgent, a multi-agent framework built on LangGraph that decomposes clinical reasoning into four specialized, tool-grounded agents—patient intake structuring, drug-drug interaction (DDI) checking, clinical guideline retrieval, and evidence-grounded synthesis—each consulting external medical databases rather than relying on parametric memory. The Patient Intake Agent extracts structured patient profiles from unstructured clinical notes via LLM-based extraction with Pydantic schema enforcement. The DDI Agent normalizes drug names via the RxNorm API and queries the DDInter 2.0 database for known interactions with severity levels; critical-severity findings trigger a human-in-the-loop interrupt for clinician review. The Clinical Guidelines RAG Agent retrieves relevant passages using hybrid BM25 + dense retrieval fused via Reciprocal Rank Fusion, re-scored by a cross-encoder re-ranker. The Synthesis Agent generates structured clinical reports enforcing a citation format ([REF:chunk_id]) for every recommendation; post-generation validation flags ungrounded claims, ensuring the system declines rather than hallucinating unsupported evidence. We evaluate MediAgent on 50 clinical cases derived from the MTSamples corpus, each annotated with expected conditions, drug interactions, and guideline topics. MediAgent achieves 81.8% condition extraction recall, 100% guideline topic retrieval relevance, and 92.2% citation coverage across generated recommendations, with a median end-to-end latency of 23 seconds. Our results demonstrate that tool-grounded agent decomposition with citation enforcement yields more reliable clinical analyses compared to monolithic LLM prompting, providing a replicable blueprint for trustworthy agentic AI in safety-critical healthcare domains.