CS Paper Radar

Pretrained Medical Representations for the Practical Screening of Drug Repositioning Candidates

Yuhei Fujioka, Daitaro Misawa, Shingo Fukuma

cs.LG ·submitted 2026-09-17 08:19 UTC · v1
published

Abstract

Representation learning from medical code sequences in electronic health records and medical claims data has been successful in various clinical applications, such as those regarding disease prediction. However, significant challenges remain in extending this approach to the discovery of scientific hypotheses. One reason is that many existing BERT-based models fail to adequately capture the hierarchical structure of medical codes and the complex interactions between diagnoses and treatments. To address these limitations, we propose a new unified pre-training framework that explicitly integrates hierarchical sub-token aggregation, partial masking, and cross-reference mechanisms. The proposed model consistently outperformed existing methods on both pre-training objectives and downstream clinical event prediction tasks, including the onset of dementia and hospitalization. We also conducted an in silico drug repositioning case study targeting Alzheimer's disease. In the hypothesis generation step, our approach successfully rediscovered known promising drugs in a data-driven manner without relying on such external knowledge sources as the literature. Subsequently, in the hypothesis prioritization step, we introduced a Task-Adaptive Representation Approach to alleviate the over-encoding of historical prescription information within diagnostic vectors, enabling the robust prioritization of generated hypotheses. This study establishes an exploratory screening workflow for hypothesis generation and prioritization based on observational associations. Importantly, this framework is not intended to provide causal evidence, but rather to identify promising candidates for subsequent rigorous causal inference. Overall, this study demonstrates that domain-informed representation learning combined with task-adaptive representation control can enable a practical hypothesis discovery workflow.

Comment
Accepted at ICML 2026 AI for Science Workshop

Closest papers

Nearest neighbours by meaning across the whole index.

EHRAdapt: Adapting Pretrained Language Models to Electronic Health Records with Semantic Priors for Rare Clinical Events

cs.AI ·2026-09-27 ·Andre R Goncalves, Vincent Liu, Priyadip Ray
similarity 0.70
Abstract

Electronic health records (EHRs) encode clinical histories as (time, modality, code) tuples, whereas pretrained language models expect text tokens. Serializing tuples as text inflates sequence length and repeatedly encodes the same structure. We introduce EHRAdapt, an adapter that maps tuples directly into a frozen language model's embedding space. Modality and time gaps enter through a learned embedding and attention biases, and event codes receive dedicated vectors. Learning them is challenging because clinical vocabularies are long-tailed, leaving rare events with few observations. EHRAdapt therefore represents each event vector as the sum of a semantic prior and an evidence residual. The prior is a frozen, ontology-trained biomedical embedding of the event's clinical description, mapped into the input space by a shared learned projection, so it supplies clinical meaning even when observations are scarce. The residual is a learned low-rank, event-specific correction that refines it as evidence accumulates. We perform continued pretraining on records from about 4 million patients with three frozen LLM backbones, OLMo2 1B, Llama3.2 1B, and OLMo2 7B, training only the adapter (0.1--0.6% of all parameters). The full adapter outperforms all ablations in held-out next-event prediction on every backbone. Removing the semantic pathway hurts rare events most, over ten times more than the most frequent ones, whereas removing the residual hurts overall prediction but improves it for the rarest events. On reportable infectious-disease and syndromic classification, EHRAdapt outperforms text-based LLM and count-based baselines and the same LLMs adapted to the same records through next-event prediction, and both pathways improve rare-disease discrimination. The two pathways therefore play complementary roles, visible only when results are broken down by event frequency.

0.17

Learning Scientific Trajectories to Predict Future Biomedical Discoveries

cs.CL cs.AI ·2026-10-28 ·Chia-Hsuan Chang, Brian Ondov, Jiang Lv, Lan Jiang, Yunmai Wang +5 more
similarity 0.70
Abstract

AI co-scientists have expanded the use of large language models for generating scientific hypotheses and guiding experimental investigation. Here we investigate whether temporal patterns in the semantic representations of complete biomedical conclusions can guide hypothesis generation. We introduce ForeSCITE, a framework that identifies thematic clusters of published conclusions, estimates a direction separating earlier from more recent findings within each cluster, and projects existing conclusion embeddings along these directions. An embedding language model trained to reconstruct conclusions decodes the projected coordinates into natural-language hypotheses. Across 407 clusters derived from approximately three million pre-cutoff conclusions, we identify temporal structure and illustrate its retention after decoding. In retrospective, time-sliced evaluations, ForeSCITE increases semantic coverage beyond the historical corpus, biomedical entity-pair novelty and post-cutoff pair re-observation relative to five prompt-based and retrieval-augmented baselines. Expert review of prioritized CAR-T hypotheses identified 37 candidates as both novel and biologically plausible, and expert justifications for five selected hypotheses illustrate their biological rationale and subsequent literature support where available. These findings support temporal guidance in semantic space as a scalable mechanism for proposing biomedical hypotheses for subsequent scientific evaluation.

0.19

Disease-level drug approval forecasting from trial-agnostic public data

cs.LG q-bio.QM ·2026-09-29 ·Samir Townsley, Pagkratios Tagkopoulos, Ilias Tagkopoulos
similarity 0.68
Abstract

Drug development is an expensive and lengthy process requiring developers to select drugs for human trials and diseases to target. Computational tools that can estimate before testing which drug-disease candidates are most likely to reach FDA approval could reduce development time and costs through early program advancement and disease prioritization. Yet existing approaches predict outcomes at a single level or stage or do not distinguish between disease outcomes for the same drug. We ask (1) whether FDA approval can be forecast for a drug-disease pair using only pre-clinical trial information, and (2) whether these forecasts can help prioritize diseases for the same drug. We built a cohort of 14,134 drug-disease pairs by reconstructing programs from ClinicalTrials.gov, extracting disease-level approvals from FDA label documentation, and enriching each pair with molecular, disease, ADMET, target, and pathway features. We divided the cohort into pre- and post-2019 programs and trained a classifier on pre-2019 data to predict FDA approval. On the post-2019 cohort (2,637 programs), the model would have prioritized a program that ultimately gained approval over one that did not approximately three out of four times. For drugs tested across multiple diseases, selecting the model's highest-ranked disease would have identified an ultimately approved disease 61.3% of the time, compared with 40.7% under random selection. These results show that public pre-trial information contains signal for forecasting FDA approval at the drug-disease level and can help prioritize diseases for the same drug. We provide a public web tool for SMILES- and ICD-10-based drug-disease approval forecasting.

0.14

Explainable Transformer Models for Clinical Prediction Tasks on Structured Electronic Health Records

cs.LG ·2026-08-20 ·Jun Ni Du, Lukas Adamek, Maxim Kryukov, Flavio Dormont, Ziv Bar-Joseph +2 more
top venuebenchmark similarity 0.67
Abstract · Accepted at MLHC 2026; to appear in Proceedings of Machine Learning Research (PMLR)

Predictive models over structured electronic health records (EHRs) remain central to machine learning for healthcare, but few have jointly emphasized quantitative laboratory information and interpretability with respect to input medical events. We present BERT-LER, a BERT-style model for coded EHR timelines pretrained and fine-tuned from a de-identified EHR dataset of 75 million patients, that encodes laboratory test results as discrete tokens while retaining graded information through percentile-based binning, paired with Integrated Gradients for token-level attributions grounded in the input EHR sequence. We benchmark our approach on the public EHRShot benchmark suite and on an asthma severity progression study based on real-world data. This addresses a methodological gap in EHR foundation-style modeling by unifying laboratory value representation and explainability in a single framework, while assessing whether both predictive performance and explanations generalize beyond standard clinical prediction tasks. Across EHRShot and asthma tasks, BERT-LER achieves predictive performance that is competitive with, and on laboratory-related tasks often exceeds, publicly available benchmark models, and provides attributions that align with clinically known risk factors. Our architecture and explainability approach can be applied to many therapeutic areas and prediction tasks using language models trained on structured EHRs.

0.42

Coverage-Aware Reasoning with Medical Tokens for Diagnosis Prediction

cs.LG cs.AI ·2026-10-07 ·Kaisong Zhang, Haotian Fang, Junmeng Zhou, Hang Lv, Yulan Pan +1 more
code/data similarity 0.67
Abstract

Large language models (LLMs) offer promising potential for next-visit diagnosis prediction, owing to their ability to integrate longitudinal clinical evidence and reason over it in natural language. However, reinforcement learning for LLM reasoning commonly rewards each trajectory according to the correctness of its final answer. In next-visit diagnosis prediction, multiple diagnoses can be simultaneously valid, but independently rewarding one diagnosis per trajectory does not distinguish repeated hits from coverage of different diagnoses. The policy can therefore concentrate on a few correct diagnoses, leaving others uncovered. Meanwhile, LLM tokenizers can split ICD codes into several generic tokens with limited clinical meaning, requiring multiple decoding steps to predict each diagnosis and hindering reasoning over a large disease vocabulary. To address both challenges, we propose CARing, a framework that represents diagnoses with compositional Semantic IDs (SIDs) and optimizes reasoning trajectories for multi-label coverage. Concretely, we first encode ontology-enriched disease semantics into compact SIDs through residual quantization, and ground the resulting SID tokens in natural language and longitudinal EHR contexts through multi-task alignment and reasoning-enriched training to unlock transferable LLM reasoning. CARing further improves unordered multi-label prediction through a coverage reward for reinforcement learning and multi-positive supervision. At inference time, the model supports both efficient direct constrained decoding and multi-chain reasoning with rank fusion. On MIMIC-III and MIMIC-IV, CARing exceeds all EHR-trained baselines in weighted F1 and attains the highest top-k recall at every reported cutoff, including R@30 of 46.04% and 46.52% in reasoning mode. Our codes and logs are available at https://github.com/zmlxzyh/CARing-Codes-Logs.

0.19

CoLa-ICD: A Knowledge-Enhanced Framework for Long-Tail Automated Medical Coding

cs.AI ·2026-08-31 ·Yihang Cheng, Veronica Liesaputra, Andrew Trotman
top venuecode/data similarity 0.67
Abstract

Automatic medical coding assigns ICD codes to clinical notes, but it remains challenging due to long documents, imbalanced label distributions, and diverse terms. These challenges are especially severe for rare codes, which have limited training instances and are easily confused with semantically similar labels. We introduce CoLa-ICD, a knowledge-enhanced framework for long-tail prediction. CoLa-ICD enriches ICD labels with external terms, models dependencies among related codes, and learns stronger alignment between label semantics and clinical evidence for long-tail prediction. Experiments show that CoLa-ICD improves long-tail prediction with larger gains in larger and sparser label spaces and achieves state-of-the-art performance in AUC, F1, and P@k. Our code is available at https://github.com/youwillbethebest/Cola-ICD.

0.69

KREL: Automatic Medical Coding via Knowledge-Guided Reasoning over Clinical Evidence with LLMs

cs.CL cs.AI ·2026-08-21 ·Xubin Chen, Yipeng Zhou, Wen Sun, Chengkai Huang, Xiaoming Fu +1 more
top venuebenchmark similarity 0.66
Abstract · Accepted at Findings of EMNLP 2026. Camera-ready version

Automatic Medical Coding (AMC), which assigns standardized International Classification of Diseases (ICD) codes to clinical notes, is essential for medical reimbursement, quality reporting, and clinical research. Existing pre-trained language model (PLM)-based methods typically formulate AMC as an extreme multi-label classification problem over a predefined code set, while recent large language model (LLM)-based approaches instead frame it as generation or multi-step reasoning. However, key challenges remain, including the extreme length of clinical notes that hinders effective interpretation, the vast ICD label space, and complex coding rules that are not explicitly captured by LLMs. In this work, we propose Knowledge-Guided Reasoning over Clinical Evidence with LLMs (KREL), a framework that leverages LLMs for clinical text understanding and reasoning while integrating external ICD coding guidelines as structured knowledge. This design enables tight coupling between domain knowledge and LLM reasoning, reducing hallucinations and improving compliance with coding standards. Experiments on benchmark datasets show that KREL consistently outperforms strong PLM-based and state-of-the-art LLM-based baselines.

0.69

Foundation Agents Meet Agentic Deep Research: Evidence-Grounded Clinical Code Forecasting

cs.CL cs.AI ·2026-08-17 ·Junda Wang, Meysam Ghaffari, Akshat Choube, Mohsen Sharifi Renani, Hong Yu +1 more
hot direction similarity 0.66
Abstract

Next-encounter ICD forecasting predicts which standardized diagnosis codes will be documented at a future visit from the longitudinal record available beforehand. The task is prospective and multi-label: the target note does not yet exist, and several codes may be correct. Structured EHR foundation models capture recurrence and temporal progression, whereas language foundation models generate flexible diagnostic hypotheses. We introduce ICD-Deepresearch, a DeepResearch workflow that composes these predictive foundation models with medical search and ICD dictionaries. Because no source reveals the future code set, research evaluates candidate transitions by linking patient evidence, external clinical relations, and exact code semantics under a fixed top-K budget. Candidate Generation uses SparseEHR to produce an EHR Prior that initializes two bounded Research Expansion rounds; an independent GPT-5 Direct Forecast supplies complementary candidates. Final Selection validates, deduplicates, and jointly ranks both paths, after which a separate module writes rationales without changing predictions. Finally ICD-Deepresearch achieves patient-averaged precision/recall of 24.60/35.09% on MIMIC-III and 25.14/48.32% on MIMIC-IV. Physicians rate 51% and 68% of its retrieved documents useful, compared with 22% and 39% for standalone GPT-5 web search and 32% and 41% for Medical Deep Research. ICD-Deepresearch therefore improves over the registered local comparators while retrieving evidence with higher physician-rated usefulness than the standalone research systems

0.27