Curriculum Boosts Relational PFN Pretraining Efficiency

Relational Prior-Data Fitted Networks (PFNs), such as RDB-PFN, are designed to approximate Bayesian inference over multi-table relational databases by pretraining on vast amounts of synthetic data. This study investigates whether a different synthetic data generator (PluRel) can substitute for RDB-PFN's prior, the impact of synthetic data presentation order (curriculum), and the extent of relational reasoning acquired from single-table pretraining alone.Using PluRel as the sole synthetic data source, the researchers found that a progressive single-table curriculum, gradually increasing schema complexity, achieved 0.703 average ROC-AUC on a 23-task tabular benchmark using approximately 45 times less synthetic data than RDB-PFN's warm-up recipe. In contrast, training with the same data all at once resulted in a significantly lower 0.541 ROC-AUC.Furthermore, a relational curriculum trained from scratch on about 220 times less relational synthetic data than RDB-PFN still recovered 88% of its reported performance on the RelBench/4DBInfer benchmark. Remarkably, the single-table curriculum model, without any relational adaptation, nearly matched the dedicated relational pipeline's performance when evaluated directly on the relational benchmark. These findings emphasize that thoughtful curriculum design and diverse synthetic data are more impactful than simply scaling up raw synthetic data for relational PFN pretraining.