Hierarchy-Aware RoBERTa Improves Cybersecurity Vulnerability Classification

Classifying cybersecurity vulnerabilities using the Common Weakness Enumeration (CWE) taxonomy presents significant challenges due to severe class imbalance and the inherent hierarchical structure of weakness categories. Traditional oversampling methods like SMOTE and ADASYN, while common for imbalance, prove largely ineffective or even detrimental for deep learning models when applied to hierarchical CWE text classification. This paper introduces a Hierarchy-Aware RoBERTa framework that directly integrates CWE structural information. It achieves this by using learnable parent-class embeddings, which helps preserve taxonomic consistency. Experiments on a CWE Research Concept dataset show that this approach achieves a weighted F1-score of 0.76 without data augmentation, significantly outperforming all baselines. Notably, it improved the F1-score for the minority "Class" category from 0.40 to 0.60 over the BERT baseline, demonstrating that hierarchy-aware representation learning is a more effective strategy than oversampling for structured vulnerability classification.