CS Paper Radar

PolyGandr: Steering Generative Models via Spatial Navigation through Latent Space

Mingwei S. G. Li, Susie S. Y. Li, Isabel Moreira de Oliveira, Ryan P. Adams, Remco Chang

cs.HC ·submitted 2026-10-27 15:52 UTC · v1

Abstract

Generative models support content creation across text, images, videos, and design disciplines, yet their interfaces rely primarily on natural language. While effective for specifying high-level intent, language can be inefficient and imprecise for fine-grained control, iterative refinement, and open-ended exploration. We present PolyGandr, a visual interface for exploring relationships between latent representations and generated outputs, complementing natural-language model steering. PolyGandr formulates generative control as high-dimensional navigation, enabling users to define starting points, steering directions, and step sizes through a unified map-and-mixing-palette interface. Combining global exploration with local refinement supports both broad design-space navigation and fine-grained output manipulation without repeatedly translating visual intent into prompts. We evaluate PolyGandr across three use cases: StyleGAN2 face generation, Stable Diffusion chair design, and rotational kirigami engineering. A formative study indicates that PolyGandr complements existing workflows by supporting real-time editing while simultaneously providing an overview and means to explore the broader design space.

Closest papers

Nearest neighbours by meaning across the whole index.

From Prompting to Composing: A Spatial Canvas Interface for Poster Generation

cs.CV cs.AI ·2026-10-08 ·Yitong Wang, Fangyun Wei, Jinjing Zhao, Sirui Zhang, Hongyang Zhang +3 more
benchmarkcode/data similarity 0.72
Abstract · Project page: https://snowflakewang.github.io/Compo-Page/ GitHub: https://github.com/snowflakewang/Compo

Text prompting is an indirect interface for poster generation, requiring users to encode inherently two-dimensional composition intent into a one-dimensional sequence of words. We introduce a Spatial Canvas Interface that enables users to directly compose generation intent in space through four complementary binding types: semantic, identity, text, and pixel, together with Text Specifications for individual elements and global appearance. Based on this interface, we develop Compo, a poster generation model adapted from a pretrained image editing model to understand Spatial Canvas inputs and Text Specifications. Compo supports both direct inference, where users explicitly construct the canvas, and agentic mode, where a high-level request is automatically translated into a planned Spatial Canvas. To train Compo, we develop a scalable pipeline that automatically constructs supervision data for different binding types and their combinations, enabling efficient adaptation without training a specialized poster generator from scratch. We further introduce a benchmark that evaluates adherence to individual binding types and their joint composition. Experiments show that Compo achieves stronger compositional controllability than both general-purpose image generation models and dedicated poster generation systems while maintaining high visual quality. By decoupling intent specification from visual generation, our work shifts poster generation from prompting toward composing.

0.26

A Design Space for Visual Interfaces for Generative Image Models

cs.HC ·2026-09-16 ·Susie S. Y. Li, Mingwei S. G. Li, Remco Chang
similarity 0.70
Abstract

Interactive visual interfaces have become an important means of controlling generative image models, enabling users to manipulate generation through prompts, direct manipulation, and a range of interactions. However, existing techniques are typically presented as independent systems, making it difficult to understand how they relate, compare their interaction mechanisms, or identify opportunities for new interface designs. We introduce a design space for interactive visual interfaces for generative image models derived from 51 research systems and practitioner tools. The framework decomposes each system into three complementary components: the user interface (U), the controllable model objects (Z), and the mapping function ($φ$) that translates user interaction into model operations. This decomposition provides a common representation for analyzing heterogeneous interaction techniques across model families, revealing recurring design patterns and underexplored regions of the design space. We further present an interactive corpus explorer that support comparative analysis, and discuss usage scenarios for both educational settings and HCI/AI practitioners identifying research and design opportunities.

0.07

SenseNova-U1.5: Towards Native Unified Visual Intelligence

cs.CV ·2026-09-10 ·Haiwen Diao, Jiahao Wang, Chenjing Ding, Hanming Deng, Jiangnan Chen +60 more
code/datahighly cited similarity 0.69
Abstract · Project page: https://github.com/OpenSenseNova/SenseNova-U1

We launch SenseNova-U1.5, an 8B-MoT native unified multimodal model that understands, reasons about, and generates visual content within an encoder-free and VAE-free architecture. We strengthen its visual interface through spatially coherent patch reconstruction and scale its training with carefully curated generation and editing data, improved task formulation, structural prompt enhancement, and native resolutions of up to 4K. For post-training, we optimize specialized experts for visual aesthetics, bilingual text rendering, infographic generation, and image editing, and consolidate their capabilities through multi-expert on-policy distillation. Across extensive evaluations, SenseNova-U1.5 largely advances image fidelity, text rendering, complex composition, multi-reference editing, and interleaved generation, while improving instruction following and preserving subject identity, geometry, and unmodified regions. Despite limited exposure to structured formats in its generation data, SenseNova-U1.5 generalizes effectively to long, complex, and structured visual instructions, further proving that multimodal understanding can transfer to visual planning and creation. Together, these findings position native unified modelling as a promising path towards systems that perceive, reason and create within a fully end-to-end framework. We will open-source training code, including supervised fine-tuning, reinforcement learning, and on-policy distillation.

0.44

reComposer: Editable Planning for Controllable Image Generation

cs.HC cs.AI ·2026-11-09 ·Susie S. Y. Li, Angie Boggust, Fred Hohman, Donghao Ren, Dominik Moritz +1 more
similarity 0.68
Abstract · 21 pages, 14 figures, 4 tables

Prompt-based image generation makes it easy to produce visual assets, but offers little support for engaging with the creative process. Prompting collapses interdependent creative decisions into a monolithic instruction and returns a finished image, leaving control limited to revising the prompt or correcting the model's output. We present reComposer, a process-oriented image-generation interface built around editable planning, which decomposes users' evolving intent into meaningful creative decisions - such as subject, composition, or palette - and collaborates with users to explore underspecified goals and alternatives. reComposer supports conversational and direct manipulation of the plan, moving between concept-level intent and visual-level control. We evaluated reComposer with 8 design and non-design professionals. Compared to prompting, participants felt greater control and ownership while developing visual assets, exploring alternatives, and iterating on details. Our findings suggest how shifting generative AI interaction from specifying outputs to shaping process can support creative thinking alongside asset production.

0.13

Exploring the Affordances of Generative Image AI for Supporting Early-stage Architect-client Communication

cs.HC ·2026-09-28 ·Chengzhi Zhang, Weijie Wang
similarity 0.68
Abstract · 18 pages

Text-to-image generative AI can produce renderings from natural-language prompts in near real time, making it increasingly popular for rapidly visualizing concepts in early-stage architectural design. Meanwhile, exchanging ideas efficiently and building shared understanding have long been central challenges in architect-client communication. How might the speed of generative image AI change this communication? To explore this question, we conducted a study with 11 architect-client pairs, in which each pair used generative image AI over video conference to collaboratively produce early-stage renderings of the client's "dream house." Our findings suggest that generative image AI helped pairs develop a solid shared understanding by providing concrete visual materials and supporting the exchange of ideas. It also shifted conversation dynamics, enabling clients to participate more actively in shaping design direction. However, challenges emerged, including a stylistic bias toward particular types of images and unpredictable shifts in design direction caused by variation across generations. We conclude with implications for the design of future generative image AI-based systems that support architect-client communication.

0.13

PixelBench: Evaluating Language Models for Pixel-Art Creation

cs.CV ·2026-11-16 ·Mutian Hong, Jianxuan Yao, Mingkun Yue, Changkai Li, Hower Yang +1 more
benchmark similarity 0.68
Abstract · 38 pages, 12 figures, 11 tables

Pixel art is widely used in games and digital content creation, requiring readable forms, coherent spatial relationships, and harmonious colors under limited resolution and palette constraints. Language models show growing promise in visual creation, yet their pixel art creation capabilities lack systematic evaluation. We introduce PixelBench, formalizing text-to-image generation and instruction-guided editing for pixel art and providing a benchmark for these tasks. We further present a creation environment in which language model agents invoke drawing tools through code to generate and edit pixel art. We assess the quality of generated artwork through human evaluation and employ a multimodal large language model as a judge, validating the agreement between its automated assessments and human preferences. We compare language models and configurations across diverse subjects and canvas sizes. In our experiments, GPT-6 Astra achieves both the highest average judge score and the best average human rank, highlighting the potential of general-purpose language models for pixel art creation. We release the tasks, all artworks, the human rankings, and the judge protocol.

0.12

ReVision: Supporting Designers' Interpretation and Exploration of Visuals in Concepts and Forms

cs.HC ·2026-09-27 ·Yaqing Yang, Mei-Xi Chia, Aniket Kittur
published similarity 0.67
Abstract · UIST'26 Poster

Visual designers get inspiration from references to expand their design space. They decompose what makes a reference evocative into conceptual and visual elements, ranging from explicit attributes such as objects and colors, to less readily articulated concepts and visual motifs. They then create different visual forms to explore how the selected elements could be combined differently. Novices often struggle with these moves, instead focusing on surface features or producing limited visual variation, thus becoming fixated on the reference. Existing tools support editable visual attributes and high-level themes, but provide limited control over how conceptual interpretations relate to expressive visual motifs or how their combinations can be systematically re-expressed. We present ReVision, an AI-based tool that decomposes visual and textual references into editable conceptual interpretations and visual motifs, enables their recombination across conceptual and visual spaces, and renders each direction as divergent visual-form variations, supporting more divergent exploration during the creation process.

0.18

InstructMesh: Selective Refinement of Generative 3D Models for Fabrication

cs.AI ·2026-08-28 ·Faraz Faruqi, Ahmed Katary, Demircan Tas, Theresa Hradilak, Ning Zhang +8 more
top venue similarity 0.67
Abstract

Recent advances in generative AI allow users to create 3D models from text or images. However, these models prioritize visual plausibility over geometric accuracy, often generating results with flaws that compromise their intended use post-fabrication. We present InstructMesh, an interactive post-generation refinement tool that enables selective repair of generative 3D models through region selection and targeted operations, such as opening or sealing voids, or adjusting local thickness. Users can invoke edit operations via natural language prompts or slider controls. By operating directly on the intermediate latent representation, InstructMesh allows users to apply robust geometric corrections without requiring expert modeling skills. To inform our design, we first analyze common fabrication-related failure modes in outputs from state-of-the-art generative tools. We then conduct two user studies, demonstrating that novices can identify and perform fabrication-relevant repairs on generative outputs using InstructMesh, and revealing user preference for hybrid interfaces that combine slider controls with natural language input.

0.37