Image captioning and image generation are two directions of the same image–text mapping: one turns images into text, while the other turns text back into images. Historically, captions were short summaries of only a few dozen words, but region-wise and panoptic captioning now seek to preserve substantially more visual information. The same shift is occurring in text-to-image generation: systems such as DALL·E 3, Nano Banana, and GPT Image increasingly follow longer and more detailed descriptions, making text a more precise interface for controlling content, composition, style, and fine-grained details.
This talk asks: how much of an image can text preserve? Rather than claiming exact reconstruction of every visible detail, we focus on retaining the semantic and perceptually salient information needed for image understanding and controllable generation. We explore Holistic Visual Descriptions that combine global semantics, visual style, background context, and detailed descriptions of regions across multiple granularities. We discuss why making captions longer is not sufficient: coverage, grounding, organization, and information density also matter. We further examine whether explicit structures such as JSON are necessary, and show that natural-language linearizations of the same semantic content can be equally or even more effective. By treating text simultaneously as the output of image understanding and the input to image generation, this talk presents a path toward a high-fidelity textual bridge between visual perception and synthesis.

Speaker:
Yuhui Yuan is Research Director at Canva CORE and founder of Canva Research Lab China, leading work on graphic design foundation models—multimodal generation, multi-layer generation, and design editing. 8 years at Microsoft as Senior Researcher (CV, GenAI), contributing to Azure Form Recognizer and Microsoft Designer. PhD in CV, UCAS; MS CS, Peking University. Known for OCNet, OCRNet, HDETR, Glyph-ByT5, SPO, ART.