Workshop

AIM-FM: Advancements In Medical Foundation Models: Explainability, Robustness, Security, and Beyond

Yixuan Yuan · Yao Qin · Xiang Li · Ying Wei · Bulat Ibragimov · Linda Petzold

[ OpenReview]

Abstract

Towards next-generation medical analysis: Unlock the potential of medical foundation models for more explainable, robust, secure diagnosis solutions. Workshop Homepage: https://wymancv.github.io/AIM-FM-NeurIPS-Workshop/

Video

Chat is not available.

Schedule

Timezone: America/Los_Angeles

8:20 AM

8:20 - 8:30 AM: Introduction Introduction

Yixuan Yuan

Video

8:30 AM

8:30 - 9:10 AM: Speaker: Pearse Keane Invited talk

Pearse Keane

Video

9:10 AM

9:10 - 9:50 AM: Speaker: Faisal Mahmood Invited talk

Faisal Mahmood

Video

9:50 AM

09:50 - 10:00 AM: Break (10 minutes)

10:00 AM

10:00 - 10:40 AM: Speaker: Sheng Wang Invited talk

Sheng Wang

Video

10:40 AM

10:40 - 11:20 AM: Panel Discussion Discussion Panel

Pearse Keane · James Zou · Tianming Liu · Faisal Mahmood · Shekoofeh Azizi

Video

11:20 AM

11:20 AM - 12:00 PM: Poster Session Poster

12:00 PM

12:00 - 1:00 PM Lunch Break

1:00 PM

1:00 - 1:40 PM: Speaker: Tianming Liu Invited talk

Tianming Liu

Video

1:40 PM

1:40 - 2:20 PM: Speaker: James Zou Invited talk

James Zou

Video

2:20 PM

2:20 - 2:30 PM: Break (10 minutes)

2:30 PM

2:30 - 3:30 PM: Oral Session Oral

Video

2:30 PM

2:30 - 3:30 PM: M3H: Multimodal Multitask ML for Healthcare Online Oral

Video

Link

2:30 PM

2:30 - 3:30 PM: Deep Generative Models Unveil Patterns in Medical Images Through Vision-Language Conditioning Online Oral

Video

Link

2:30 PM

2:30 - 3:30 PM: Applying SAM to Few-shot Medical Image Segmentation using Mask Propagation and Auto-prompting Online Oral

Video

Link

3:30 PM

3:30 - 4:10 PM: Speaker: Animashree Anandkumar Invited talk

Animashree Anandkumar

Video

4:10 PM

4:10 - 4:50 PM: Speaker: Karan Singhal Invited talk

Karan Singhal

Video

4:50 PM

4:50 - 5:00 PM: Break (10 minutes)

5:00 PM

5:00- 5:10 PM: Award Presentation and Summary

Video

5:10 PM

5:10- 5:30 PM: Poster Session Poster

Development and bilingual evaluation of Japanese medical large language model within reasonably low computational resources Poster

Issey Sukeda

Link

Anatomical 3D Style Transfer Enabling Efficient Federated Learning with Extremely Low Communication Costs Poster

Yuto Shibata · Yasunori Kudo · Yohei Sugawara

Link

VidLPRO: A Video-Language Pre-training Framework for Robotic and Laparoscopic Surgery Poster

Mohammadmahdi Honarmand · Muhammad Abdullah Jamal · Omid Mohareri

Link

Biomedical SAM-2: Segment Anything in Biomedical Images and Videos Poster

Zhiling Yan · Weixiang Sun · Rong Zhou · Zhengqing Yuan · Kai Zhang · Yiwei Li · Sekeun Kim · Sifan Song · Hui Ren · Tianming Liu · Quanzheng Li · Xiang Li · Lifang He · Lichao Sun

Link

Assessment of Medical Foundation Models for Survival Prediction with Whole Slide Images Poster

Elena Menand

Link

Universal Speech Disorder Recognition: Towards a Foundation Model for Cross-Pathology Generalisation Poster

Giulia Sanguedolce · Dragos-Cristian Gruia · Sophie Brook · Patrick Naylor · Fatemeh Geranmayeh

Link

DAug: Diffusion-based Channel Augmentation for Radiology Image Retrieval and Classification Poster

Zhuoran Zhou · Zhuoran Zhou · Haoquan Fang · Jenq-Neng Hwang

Link

Exploring Fairness in Long-Term Prediction of Type 2 Diabetes Microvascular Complications Poster

Elizabeth Remfry · Rafael Henkin · Zainab Awan · Rohini Mathur · Michael Barnes · Aakanksha Naik

Link

MediConfusion: Can you trust your AI radiologist? Probing the reliability of multimodal medical foundation models Poster

Mohammad Shahab Sepehri · Zalan Fabian · Maryam Soltanolkotabi · Mahdi Soltanolkotabi

Link

Going beyond H&E and Oncology: how do Histopathology Foundation Models perform for multi-stain IHC & Immunology? Poster

Amaya Gallagher-Syed · ELENA PONTARINI · Myles Lewis · Greg Slabaugh · Michael Barnes

Link

M3H: Multimodal Multitask Machine Learning for Healthcare Oral

Dimitris Bertsimas · Yu Ma

Link

Reliability in AI-Assisted Critical Care: Assessing Large Language Model Robustness and Instruction Following for Cardiac Arrest Identification Poster

Ugurcan Vurgun · Sy Hwang · Danielle Mowery

Link

Armadillo: Robust Secure Aggregation for Federated Learning with Input Validation Poster

Yiping Ma · Yue Guo · Harish Karthikeyan · Antigoni Polychroniadou

Link

DiversityMedQA: A Benchmark for Assessing Demographic Biases in Medical Diagnosis using Large Language Models Poster

Rajat Rawat · Hudson McBride · Rajarshi Ghosh · Dhiyaan Nirmal · Jong Moon · Dhruv Alamari · Kevin Zhu · Sean O'Brien

Link

Demographic Bias of Expert-Level Vision-Language Foundation Models in Medical Imaging Poster

Yuzhe Yang · Yujia Liu · Xin Liu · Wei Wu · Shwetak Patel

Link

Hetero-UNet: Heterogeneous Transformer with Mamba for Medical Image Segmentation Poster

Zhiling Yan · yixin liu · Xiang Li · Lichao Sun

Link

GPT Sonograpy: Hand Gesture Decoding from Forearm Ultrasound Images via VLM Poster

Keshav Bimbraw · Ye Wang · Jing Liu · Toshiaki Koike-Akino

Link

Deep Generative Models Unveil Patterns in Medical Images Through Vision- “Language” Conditioning Oral

Xiaodan Xing · Junzhi Ning · Yang Nan · Guang Yang

Link

Federated Self-Supervised Single-cell Clustering of scRNA-seq Data Poster

Shentong Mo

Link

Synthetic Multimodal Data Generation and Training Optimization For Computed Tomography Cardiac Imaging Applications Poster

Masaki Ikuta · Quanqi Hu · Ashok Addala · Ravi Soni · Avinash Gopal

Link

PATIENT-Ψ: Using Large Language Models to Simulate Patients for Training Mental Health Professionals Poster

Ruiyi Wang · Stephanie Milani · Jamie Chiu · Jiayin Zhi · Shaun Eack · Travis Labrum · Samuel Murphy · Nev Jones · Kate Hardy · Hong Shen · Fei Fang · Zhiyu Chen

Link

Learning biologically relevant features in a pathology foundation model using sparse autoencoders Poster

Nhat Le · Neel Patel · Ciyue Shen · Blake Martin · Alfred Eng · Chintan Shah · Sean Grullon · Dinkar Juyal

Link

Controlling for Unobserved Confounding with Large Language Model Classification of Patient Smoking Status Oral

Samuel Lee · Zach Wood-Doughty

Link

Necessity of Uncertainty Quantification for Audio-driven Healthcare Diagnosis Poster

Shubham Kulkarni · Hideaki Watanabe · Fuminori Homma

Link

ProMISe: Promptable Medical Image Segmentation using SAM Poster

Jinfeng Wang · Sifan Song · Xinkun Wang · yiyi wang · Yiyi Miao · Jionglong Su · S. Kevin Zhou

Link

Best of Both Worlds: Harmonizing LLM Capabilities in Decision-Making and Question-Answering for Treatment Regimes Poster

Hongxuan Liu · Zhiyao Luo · Tingting Zhu

Link

Do Large Language Models have Shared Weaknesses in Medical Question Answering? Poster

Andrew M. Bean · Karolina Korgul · Felix Krones · Robert McCraith · Adam Mahdi

Link

Med-FastSAM: Improving Transfer Efficiency of SAM to Domain-Generalised Medical Image Segmentation Poster

Yuxiang Luo · Qing Xu · Jinglei Feng · Guangwu Qian · Wenting Duan

Link

Sentiment Reasoning for Healthcare Poster

Khai-Nguyen Nguyen · Khai Le-Duc · Bach Phan Tat · Duy Le · Phuc H Ngo · Long Vo-Dang · Anh Nguyen · Truong Son Hy

Link

Surgical SAM 2: Real-time Segment Anything in Surgical Video by Efficient Frame Pruning Poster

Haofeng Liu · Erli Zhang · Junde Wu · Mingxuan Hong · Yueming Jin

Link

Do Histopathological Foundation Models Eliminate Batch Effects? A Comparative Study Poster

Jonah Kömen · Hannah Marienwald · Jonas Dippel · Julius Hense

Link

Integrating Image Interpretation and Textual Context for Improved Breast Imaging Classification Poster

Ibrahim Gulluk · Olivier Gevaert

Link

A Self-Supervised Model for Multi-modal Stroke Risk Prediction Poster

Camille Delgrange · Olga Demler · Samia Mora · bjoern menze · Ezequiel De la Rosa · Neda Davoudi

Link

Improving the Classification Effect of Clinical Images of Diseases for Multi-Source Privacy Protection Poster

Bowen Tian · Zhengyang Xu · Zihao Yin · JingyingWang · Yutao Yue

Link

DMM: Distributed Matrix Mechanism for Differentially-Private Federated Learning using Packed Secret Sharing Poster

Alexander Bienstock · Ujjwal Kumar · Antigoni Polychroniadou

Link

Contextual Evaluation of Large Language Models for Classifying Tropical and Infectious Diseases Poster

Mercy Asiedu · Nenad Tomasev · Chintan Ghate · Tiya Tiyasirichokchai · Awa Dieng · Geoffrey Siwo · Steve Adudans · Oluwatosin Akande · Katherine Heller

Link

A Textbook Remedy for Domain Shifts: Knowledge Priors for Medical Image Analysis Oral

Yue Yang · Mona Gandhi · Yufei Wang · Yifan Wu · Michael Yao · Chris Callison-Burch · James Gee · Mark Yatskar

Link

Research Journey of Generative Protein Modeling Poster

Xinhui Chen · Yiwen Yuan · Joseph Liu · Chak Tou Leong · Zhen Xie · Xiaoye Zhu · Ying Chen · Songyue Chen · Chenyi Wang · Kun Li · Jie Zhang · Zuchao Li · Jiaqi Chen

Link

An Autonomous Dual-channel Entity Recognition Method for Chinese Acupuncture Poster

Changshuai Zhang · Yuxin Zhang · Wenjun Tan

Link

MambaHealth: A Lightweight Foundation Model for Efficient Drug Recommendation Poster

Yuda Wang · Xuxin He · Shengxin Zhu

Link

Application of Contrastive Learning on ECG Data: Evaluating Performance in Japanese and Classification with Around 100 Labels Poster

Junichiro Takahashi · JingChuan Guan · Kaito Baba · satoshi kodera

Link

Towards Foundation Models for Critical Care Time Series Oral

Manuel Burger · Fedor Sergeev · Malte Londschien · Daphné Chopard · Hugo Yèche · Eike Gerdes · Polina Leshetkina · Alexander Morgenroth · Zeynep Babür · Jasmina Bogojeska · Martin Faltys · Rita Kuznetsova · Gunnar Rätsch

Link

How Does Diverse Interpretability of Textual Prompts Impact Medical Vision-Language Zero-Shot Tasks? Poster

Sicheng Wang · Che Liu · Rossella Arcucci

Link

Random Token Fusion for Multi-View Medical Diagnosis Poster

Jingyu Guo · Christos Matsoukas · Fredrik Strand · Kevin Smith

Link

A self-supervised framework for learning whole slide representations Poster

Xinhai Hou · Cheng Jiang · Akhil Kondepudi · Yiwei Lyu · Asadur Chowdury · Honglak Lee · Todd Hollon

Link

SAM-MPA: Applying SAM to Few-shot Medical Image Segmentation using Mask Propagation and Auto-prompting Oral

Jie Xu · LiXiaokang · Chengyuyue · Chen Ma · yi guo · Yuanyuan Wang

Link

TTT-UNet: Enhancing U-Net with Test-Time Training Layers for Biomedical Image Segmentation Poster

Rong Zhou · Zhengqing Yuan · Zhiling Yan · Weixiang Sun · Kai Zhang · Yiwei Li · Xiang Li · Quanzheng Li · Lifang He · Lichao Sun

Link

Promoting cross-modal representations to improve multimodal foundation models for physiological signals Poster

Ching Fang · Christopher Sandino · Behrooz Mahasseni · Juri Minxha · Hadi Pouransari · Erdrin Azemi · Ali Moin · Ellen Zippi

Link

JRadiEvo: A Japanese Radiology Report Generation Model Enhanced by Evolutionary Optimization of Model Merging Poster

Kaito Baba · Ryota Yagi · Junichiro Takahashi · Risa Kishikawa · satoshi kodera

Link

Surgical-LLaVA: Toward Surgical Scenario Understanding via Large Language and Vision Models Poster

Juseong Jin · Chang Jeong

Link

Fairness Of AI Models in vector embedded Chest X-ray representations Poster

Gebreyowhans H. Bahre · Hassan Hamidi · Francesco Calimeri · Andrew Sellergren · Leo Anthony Celi · Laleh Seyyed-Kalantari

Link

Precise Lens Status Classification via Projection Tuning for Efficient Adaptation to Data Shifts in Small Cataract Image Datasets Poster

Ji Young Byun · Jordan Shuff · Kunal S. Parikh · Rama Chellappa

Link

MSA-LM: Integrating DNA-level Inductive Biases into DNA Language Models Poster

Vishrut Thoutam

Link

Masked Modeling for Single-cell Clustering of scRNA‐seq Data Poster

Shentong Mo

Link

PICASO: Secure Aggregation for Federated Learning with Minimal Synchronization Poster

Harish Karthikeyan · Antigoni Polychroniadou

Link

Catching the Spikes: Heteroscedastic Uncertainty Quantification for Enhanced Malaria Prediction Poster

Feng Chen · Qi Qi · Jiayu Qiu · Kemeng Zhang · Xiang Li

Link

Multimodal Lego: Model Merging and Fine-Tuning Across Topologies and Modalities Poster

Konstantin Hemker · Nikola Simidjievski · Mateja Jamnik

Link

Neural machine translation of clinical procedure codes for medical diagnosis and uncertainty quantification Poster

Pei-Hung Chung · Shuhan He · Norawit Kijpaisalratana · Abdel Ariss · Byung-Jun Yoon

Link

A Benchmark for Long-Form Medical Question Answering Poster

Pedram Hosseini · Jessica Sin · Bing Ren · Bryceton Thomas · Elnaz Nouri · Ali Farahanchi · Saeed Hassanpour

Link

Gaze-Assisted Medical Image Segmentation Poster

Leila Khaertdinova · Ilya Pershin · Tatyana Shmykova · Bulat Ibragimov

Link

Cell ontology guided transcriptome foundation model Poster

XINYU YUAN · Zhihao Zhan · Zuobai Zhang · Manqi Zhou · Jianan Zhao · Boyu Han · Yue Li · Jian Tang

Link

Explaining Chest X-ray Pathology Models using Textual Concepts Poster

Vijay Sadashivaiah · Pingkun Yan · James Hendler

Link

RespLLM: Unifying Audio and Text with Multimodal LLMs for Generalized Respiratory Health Prediction Poster

Yuwei Zhang · Tong Xia · Aaqib Saeed · Cecilia Mascolo

Link

Towards a Personal Health Large Language Model Oral

Justin Cosentino · Anastasiya Belyaeva · Xin Liu · Zhun Yang · Yun Liu · Shyam Tailor · Tim Althoff · John Hernandez · Yossi Matias · Greg Corrado · Shwetak Patel · Shravya Shetty · Jiening Zhan · Daniel McDuff · Cory McLean

Link

ASTRID - An Automated and Scalable TRIaD for the Evaluation of RAG-based Clinical Question Answering Systems Poster

Mohita Chowdhury · Yajie He · Ernest Lim · Aisling Higham

Link