Giraffe Maps Text to Visual Embeddings for Graphic Design

Multimodal large language models (MLLMs) have advanced significantly in understanding multimedia, but their generation capabilities, especially for complex tasks like graphic design, remain limited. Existing methods often represent images using multiple specialized tokens, leading to excessively long input sequences, which is problematic for designs involving numerous visual and textual elements. The new "Giraffe" architecture addresses this by efficiently mapping hidden text representations to visual model embedding spaces, such as CLIP ViT-L/14, using only a single [IMG] token per image. This lightweight solution employs two shallow MLP blocks with compression and expansion modules, trained with six distinct loss functions. Giraffe demonstrates strong performance in both image-to-design and text-to-design generation tasks, offering a more scalable approach for complex visual content creation.