Small-Data AI Predicts Cryocooler Lifetimes Without Foundation Models

While large-scale pretrained time-series models, often referred to as foundation models, excel with abundant and diverse data, many industrial and scientific applications lack such extensive datasets. To address this, a new paradigm called FSD-RM (Family of Small-Data Representation Models) is introduced, offering a practical alternative for scenarios with limited, domain-specific telemetry. Instead of relying on massive pretraining, FSD-RM focuses on capacity-controlled representation learning using established encoder architectures like CNN1D, LSTM, GRU, and Transformer, chosen for their suitability in small-data environments and their interpretability. These encoders are trained in an unsupervised manner on multivariate telemetry data and integrated into a two-stage pipeline for downstream tasks, specifically cryocooler lifetime prediction. A key innovation is the use of dimension-aware Neural Architecture Search (NAS), which jointly optimizes both the model's capacity and the input dimensionality. Experiments conducted on cryocooler telemetry data demonstrate that this combined approach achieves predictive performance comparable to larger models, but with significantly reduced training costs and model complexity. The core contribution lies in integrating proven representation learning techniques within a NAS-driven framework specifically designed for small-data regimes, emphasizing explicit parameter settings and design choices. The results suggest that effective representation learning is achievable without extensive pretraining when appropriate inductive biases and capacity control mechanisms are applied.