CS Paper Radar

Enabling Creative Exploration for Vibe Design Agents

Yifan Zhang, Nghi D. Q. Bui, Georgios Evangelopoulos, Arnaud Benard

cs.AI ·submitted 2026-09-14 05:49 UTC · v3
benchmark

Abstract

Vibe design agents translate natural-language requests into interfaces, yet repeated generations often converge on similar visual directions. We present an inference architecture that makes design exploration an explicit stage before implementation. A pre-pass proposes structured design specifications with typicality weights, an external policy samples a direction, and the generator implements it under fixed decoding settings. Candidate elicitation draws on Verbalized Sampling. Applied to UI themes and visual-asset prompts, this architecture provides direct control over aesthetic exploration without changing code-generation temperature. Across 168 benchmark prompts, theme sampling expands observed selection coverage from one option to nearly three and increases rendered visual variation. For visual-asset prompts, the strongest observed LLM-judge win/loss ratio is 1.32, with only a small change in whole-screen similarity. Localized aesthetic choices can thus influence judged preference even when the overall screen remains similar. We further evaluate joint theme and asset sampling in a randomized online study conducted over nearly two months, covering 3.39 million tasks and 87,416 labeled conversations. These studies show how explicit design specifications support exploration of alternative UI concepts and help us evaluate their effects on rendered interfaces, judged preference, and user interaction.

Comment
24 pages; revised exposition and evaluation, updated figures, and expanded appendix

Closest papers

Nearest neighbours by meaning across the whole index.

Towards Effective Generation of Interactive Visualizations with Vibe Coding: An Empirical Study

cs.HC ·2026-08-30 ·Yanshan Zeng, Ruixuan Tu, Zuo Xiang, Lijia Feng, Guozheng Li +1 more
similarity 0.72
Abstract · 5 pages, 4 figures

Constructing interactive visualizations has traditionally required substantial human effort, involving both technical implementation and design decision-making. Recently, vibe coding, a programming paradigm leveraging Large Language Models to generate, interpret, and refactor code from natural language specifications, has emerged as a promising approach to reduce the burden. However, the capabilities and limitations of vibe coding in building interactive visualizations remain unexplored. To address this gap, we conducted a user study with 78 participants that were tasked with constructing interactive visualizations using vibe coding. We further collected users feedback through questionnaires, interviews, and case analyses. Based on this study, we examine (1) the capabilities and (2) user experience of vibe coding in generating interactive visualizations, and (3) the practical human-agent collaboration strategies adopted. Our findings provide the first systematic assessment of vibe coding for interactive visualization construction, revealing both its strengths and limitations, explaining the shift in developer labor and identifying the hybrid collaboration strategies participants adopted. Furthermore, our study offers insights for more intuitive and robust vibe coding practices.

0.21

FocusGen: Expanding Visual Design Exploration with a Simulated Focus Group of Persona Agents

cs.HC cs.MA ·2026-08-28 ·Jaewon Choi, Helena Vasconcelos, Hyun Lee, Carolyn Zou, Tak Yeon Lee +1 more
similarity 0.70
Abstract · 18 pages, 7 figures, 6 tables

Creative professionals rarely design for themselves--they design for audiences whose preferences they must anticipate. Yet current text-to-image exploration tools derive diversity entirely from the designer's own input--their prompts, their chosen dimensions, their search queries--confining exploration to what the designer already knows to look for. We present FocusGen, an interactive system that introduces external perspectives into visual design exploration through a "virtual focus group" of simulated persona agents. In contrast to prior persona systems in which multiple agents converge as critics on a single evolving artifact, FocusGen uses personas as parallel generators: each agent--constructed from demographic data, a procedurally generated backstory, and aesthetic preferences elicited through interviews--independently drives an iterative generation loop that produces its own visual concept, transforming one design brief into a spectrum of audience-conditioned directions. With real human participants, we confirm that the iterative refinement loop produces outputs people prefer over zero-shot generation. With synthetic agents at scale, we show that persona conditioning yields higher visual diversity than a generic-assistant baseline--measured by CLIP distance and corroborated by human perceptual judgments--and that open-ended preference interviews yield more diverse outputs than structured ones for both human and synthetic cohorts, while also revealing that agent cohorts recover only part of the diversity of comparable human cohorts. A qualitative study with 16 creative professionals suggests FocusGen helps designers discover unanticipated directions, overcome fixation, and probe audience contexts--while surfacing stereotyping risks that we analyze. We position FocusGen as a divergence scaffold for early-stage ideation rather than a substitute for audience research.

0.51

Crossing the Chasm: Bridging Visual Augmentations and Designer Intent

cs.HC ·2026-10-28 ·Luke S. Snyder, Maureen C. Stone, Jeffrey Heer
published similarity 0.67
Abstract · 23 pages, 17 figures, 2 tables, appendix, IEEE TVCG 2026

To direct attention and communicate context, visualization designers often employ augmentations such as annotations, animated transitions, and stylistic layouts. While graphical perception principles can helpfully prescribe effective low-level representations of data, they lack guidance for augmentations that service higher-level communication goals. To bridge this gap, we contribute a design space that frames designer intent as viewer-oriented cognitive behaviors, grounding communicative aims in actionable visualization design techniques, including both visual encodings and augmentations. This design space consists of common augmentation tactics (annotation, animation, stylized encodings, etc.) that implement design strategies (spotlighting, sequencing, association, etc.) to achieve higher-level design goals (observe, interpret, introspect), often simultaneously. We demonstrate the analytic and generative value of our design space with examples across varied designer objectives. We discuss how our contributions help align designer intent with reader takeaways, pave the way for readers to learn more effectively, and enable future (semi-)automated systems to support visualization designers in achieving their communication goals.

0.58

AesCode: Aesthetic Code Generation with Decoupled Cross-Modal Rewards

cs.AI ·2026-10-13 ·Xin Gao, Yifan Yang, Xin Li, Pan Wang, Yan Li +11 more
benchmarkcommunity pick similarity 0.67
Abstract · 44 pages,18 figures

Generating information-rich visual artifacts such as slides, posters, and dashboards as code is appealing because the outputs are structured, editable, and verifiable. Yet code models cannot see how their choices of layout and color come together on the canvas, often limiting spatial planning and aesthetic quality. Image generators excel at visual design but can misrender requested content, making their outputs unreliable targets for direct reproduction. We therefore introduce AesCode, which generates code grounded in the content prompt while using an image generated from the same prompt as an aesthetic guidance. To do so, AesCode represents the target as a design graph that describes the whole canvas and makes each property attributable. From this graph, we derive complementary cross-modal rewards: deterministic verifiers score properties parsable from code and rendering, while a VLM judge assesses the non-parsable ones with a sample-specific Visual Graph Rubric (VGR) tied to graph elements and relations. We optimize these decoupled rewards through reinforcement learning with per-channel normalization to jointly improve spatial planning, aesthetic quality, and verifiable correctness. To evaluate this setting, we build a benchmark that scores verifiable correctness and visual quality separately, with visual scores highly consistent with human judgments. We find that reference images substantially improve the visual quality of code generated by existing models, and ablations show that AesCode benefits from references both as model input and as a source of visual rewards. Relative to same-scale Qwen3-VL baselines given the same references, AesCode-8B and AesCode-32B improve visual quality by 31.1 and 22.4 percentage points, reaching scores comparable to those of closed-source frontier models such as GPT-5.5 and Claude Opus 4.8.

0.40

Editable Visual Design

cs.CV cs.CL ·2026-09-03 ·Junyan Ye, Wei Liu, Dongzhi Jiang, Zichen Wen, HaoDong Li +7 more
similarity 0.65
Abstract

While diffusion base models such as GPT-Image-2 and Nano-Banana exhibit remarkable visual expressiveness, their end-to-end generation inherently yields flattened bitmaps with error-prone text, precluding layer-wise post-editing. Conversely, code-based visual generation via Coding Agents provides precise layout control and decoupled layers, yet remains constrained by a lack of global aesthetic intuition and the difficulty of coding complex visual assets. To address this, we propose Editable Visual Design, a new paradigm driven by a Coding Agent. We designate the VLM as the ``creative brain'' for requirement comprehension, task planning, and aesthetic judgment, while utilizing the image generation model as an on-demand ``visual world simulator'' to synthesize standalone visual assets. Operating under an ``imagine first, then act'' closed-loop workflow, the agent generates isolated assets, writes native HTML/CSS, and iteratively refines the design against visual rendering feedback. Furthermore, Agent Design Replay faithfully reproduces the creative and reasoning trajectory akin to that of professional human designers. Ultimately, the system delivers editable artifacts with decoupled layers and real text, enabling users to perform intuitive mouse dragging and layout adjustments on a graphical user interface. Validations on posters, infographics, and other scenarios show that this paradigm successfully achieves both refined aesthetics and production-grade editability.

0.35

From Style Replication to Style Exploration: Enabling Art Style Exploration with Analyze-Experiment-Resituate Framework

cs.HC cs.AI ·2026-08-14 ·Wen-Fan Wang, TsaiHsuan Lin, Chi-Lan Yang, An-Ru Cheng, Bing-Yu Chen
top venue similarity 0.65
Abstract · Accepted to UIST '26

Art style is a signature of professional digital artists that develops through repeated experimentation, reflection, and adaptation. While generative AI (GenAI) can reproduce styles with high fidelity, current tools provide limited support for exploring new stylistic directions and may encourage style replication over exploration. To address this gap, we propose Analyze-Experiment-Resituate (AER), a framework for AI-assisted style exploration derived from interviews with 10 professional digital artists. Rather than prioritizing visually appealing outputs alone, AER supports three core practices of style exploration, including interpreting references, trying out stylistic possibilities, and reflecting on how emerging styles may be received. Specifically, AER enabled artists to (1) analyze artworks into interpretable stylistic elements, (2) have controllable experimentation guided by their own choices, and (3) resituate emerging styles through simulated social perspectives. We implemented AER in a prototype system and evaluated it in a controlled study with 16 artists. Compared with a direct style-transfer workflow, AER increased artists' agency and reflection as they pursued new stylistic directions. A two-week field study with four artists revealed how the AER framework influenced daily style exploration, such as reflection, experimentation, and stylistic decision-making at each stage. We discuss opportunities and challenges in designing AI-assisted style-exploration workflows, and outline implications for future artistic support tools.

0.68

Exploring the Affordances of Generative Image AI for Supporting Early-stage Architect-client Communication

cs.HC ·2026-09-28 ·Chengzhi Zhang, Weijie Wang
similarity 0.64
Abstract · 18 pages

Text-to-image generative AI can produce renderings from natural-language prompts in near real time, making it increasingly popular for rapidly visualizing concepts in early-stage architectural design. Meanwhile, exchanging ideas efficiently and building shared understanding have long been central challenges in architect-client communication. How might the speed of generative image AI change this communication? To explore this question, we conducted a study with 11 architect-client pairs, in which each pair used generative image AI over video conference to collaboratively produce early-stage renderings of the client's "dream house." Our findings suggest that generative image AI helped pairs develop a solid shared understanding by providing concrete visual materials and supporting the exchange of ideas. It also shifted conversation dynamics, enabling clients to participate more actively in shaping design direction. However, challenges emerged, including a stylistic bias toward particular types of images and unpredictable shifts in design direction caused by variation across generations. We conclude with implications for the design of future generative image AI-based systems that support architect-client communication.

0.13

Beyond Explicit Intent: Discovering and Shaping Artistic Intent through Human-AI Co-Creation

cs.AI cs.CVcs.HC ·2026-10-15 ·Renye Yan, Jialun Zhong, Jikang Cheng, Yi Gao, Yanzeng Li +1 more
similarity 0.64
Abstract · Human-AI Co-Creation, Reinforcement Learning

Generative AI is becoming an important medium for artistic creation, yet existing interactions often assume that creators can specify intentions in advance and accurately communicate them through text prompts. Artistic intentions, however, may be vague, latent, or recognizable only after seeing concrete visual possibilities. We propose a human-AI co-creation paradigm beyond explicit intent expression. By expanding model exploration, we expose artists to diverse visual possibilities, allowing them to identify previously unarticulated preferences through observation, comparison, and selection. We then integrate artists' curatorial judgments into reinforcement learning, assigning different learning strengths based on preferences over generation trajectories, so recognized directions are more strongly reinforced and the model gradually adapts to individual aesthetics. This creates a continuous "explore-identify-shape-re-explore" loop in which artistic intent is progressively discovered, externalized, and shaped. Through three user studies, we examine latent-preference discovery, personalized preference alignment, and the externalization of vague, difficult-to-express creative intentions.

0.37