Directional Sharpness Improves ML Model Generalization Certification

A new metric called "directional sharpness" has been introduced to improve the certification of machine learning models, particularly in assessing their ability to generalize to unseen data. While generalization is crucial for model quality, it's difficult to measure directly. Existing proxies like test accuracy can be misleading, and traditional sharpness metrics, though linked to generalization, are computationally expensive and unreliable when training deviates from standard procedures. Directional sharpness is designed to be both efficient to compute and reliable, even under training perturbations. Empirical and analytical evidence shows that this new metric correlates more strongly with generalization than existing methods and more dependably identifies models with poor generalization. Its efficient computability makes it suitable for model auditing scenarios where access to training data is available, and it can also be used with zero-knowledge proofs to certify quality without revealing sensitive training data.