New Platform for Visual-Aware Web Page ML

Applying machine learning to web pages presents unique challenges, primarily due to the need to interpret HTML alongside its associated resources and render it to capture meaningful visual and layout-aware information. This complexity has historically limited the exploration of machine learning over web content. This research addresses this gap by presenting a novel platform designed for visual-aware representation and machine learning applications on web pages.The platform leverages the open-source rendering tool FitLayout. It features a server capable of rendering web pages and explicitly capturing their visual and structural properties in an RDF-based representation, which is then stored in an integrated database. The entire processing pipeline is managed via a REST API, while SPARQL queries facilitate the retrieval of structured data, making it suitable as input for various machine learning algorithms.By explicitly modeling rendered web pages, including intricate layout details, the platform promotes dataset sharing and enhances the reproducibility of experimental results. Its architecture supports the complete dataset preparation workflow, from collecting and rendering web pages to preprocessing, annotating content elements, and executing downstream learning tasks. A complementary Python client library integrates the platform seamlessly with standard machine learning workflows. As a practical demonstration, the paper illustrates how rendered web pages can be transformed into graph-based representations to train graph neural networks for recognizing key content elements, showcasing both the applicability and reproducibility of the approach.