Model Choice Crucial for Causal Inference in Pharmacovigilance

Identifying causal adverse drug events (ADEs) from mere correlations is a persistent challenge in pharmacovigilance. The InferBERT framework, which combines transformer models with Do-calculus for causal inference, relies heavily on the performance of its underlying classification model. This study aimed to determine the optimal model choice within this framework. Researchers conducted a comparative analysis using two benchmarks: Analgesics-induced Acute Liver Failure (AILF) and Tramadol-related Mortalities (TRAM). They evaluated four distinct models: XGBoost as a baseline, ALBERT (the original InferBERT model), BioBERT (a transformer pre-trained on biomedical text), and Med-LLaMA (a medical large language model). The findings revealed that BioBERT consistently achieved the highest accuracy on both datasets, demonstrating a clear advantage for domain-specific pre-training. Surprisingly, Med-LLaMA, despite its larger size, underperformed. While post-hoc calibration improved calibration error, its effect on accuracy and causal discovery was mixed. The study concludes that investing in manageable, domain-aware models is more effective for computational pharmacovigilance than simply scaling model size, as BioBERT's superiority also led to stronger concordance with traditional pharmacovigilance signals.