CS Paper Radar

Live Artifacts: Authoring Dynamic Media via Live Layers Encapsulating Generative Specifications

Leixian Shen, Haotian Li, Hugo Romat, Fanny Chevalier, Nicolai Marquardt, Nathalie Riche

cs.HC ·submitted 2026-08-21 08:58 UTC · v1

Abstract

We frame Live Artifacts as a class of persistent generative media between static assets and interactive software. Unlike conventional generative outputs that collapse into static files, Live Artifacts retain their generative logic as a persistent media property, enabling continuous context-dependent regeneration. Time, location, or live data become part of their generative specifications, initiating coordinated updates across modalities (e.g., adapting text, visuals, and audio together) while preserving composition, semantics, identity, and cross-modal coherence. To facilitate experimentation with this medium, we present LiveCanvas, an authoring system that reconceptualizes visual layers as live generative specifications with explicit mutability and constrained dependencies. Creators orchestrate dynamic behaviors and manage generative persistence within a visual canvas rather than through programming, defining what remains stable, what can change, and how changes propagate. We evaluate Live Artifacts through a gallery of responsive examples and a qualitative study with six professionals, finding that LiveCanvas facilitates a shift from composing static outputs to crafting responsive generative artifacts while remaining aligned with familiar authoring practices.

Closest papers

Nearest neighbours by meaning across the whole index.

StreamStudio: Exploring Video Authoring with Streaming Autoregressive Generation

cs.HC ·2026-10-28 ·Boyu Li, Zhe Lin, Eli Shechtman, Ding Li, Hongbo Fu +1 more
similarity 0.68
Abstract

We present StreamStudio, a prototype system that explores a new authoring paradigm for streaming video generation. Instead of entering a prompt and waiting for a complete video before making changes, creators can interact with the generation as it unfolds: steering what comes next, returning to earlier moments, and exploring different continuations. In this way, StreamStudio shifts creative control from post-generation revision to continuous interaction with the generative process. Our contribution centers on this authoring paradigm and the interaction mechanisms that make streaming generation directly manipulable. StreamStudio is designed as a plug-and-play interface that can work with compatible generation and editing models, allowing its interaction model to evolve alongside advances in underlying video models. As streaming models become faster and easier to control in the future, such an interface may support more flexible and controllable video creation, including richer edits and multimodal interventions during generation.

0.05

LiveWorkspace: Externalizing Visual Understanding as Persistent Shared State for Screenshot-to-Code Agents

cs.SE ·2026-11-09 ·Shijun Xiao, Zonghui Yang, Xinrong Chen, Hao Liu, Wensheng Zhang +1 more
similarity 0.66
Abstract · 17 pages, including references and appendices

Screenshot-to-Code requires coding agents to sustain visual understanding across code generation, execution, and revision. We introduce LiveWorkspace, a visual coding harness that externalizes this understanding as Shared Visual State (Shared VS), an implementation-independent representation of visual hierarchy, node attributes, and layout relations. LiveWorkspace constructs and retains \textit{Target} from screenshots and recovers \textit{Current} from browser execution in the same schema. Structured Diff and implementation bindings guide code modifications; user edits update \textit{Target} and reuse the alignment loop. Using GPT-5.2 for both methods, LiveWorkspace improves end-to-end Avg by 1.8 points on Design2Code and 2.0 on ScreenBench over a control with matched model-call counts and stagewise budget limits. Further experiments demonstrate reuse of constructed Shared VS across coders and improvements in iterative repair. LiveWorkspace also achieves 93.0\% edit success across 100 controlled editing cases. These findings support treating visual understanding as a persistent shared working state throughout code generation, execution, and modification.

0.20

PLAYGROUND: Progressive Layout Generation with Render-Grounded Planning

cs.CV ·2026-10-13 ·Jaeho Lee, Junhwan Heo, Junghyun Park, WonJun Moon, Eunju Yang +3 more
similarity 0.65
Abstract

Generating editable graphic designs requires arranging visual and textual elements into a coherent composition while preserving their content and editability. Although recent MLLM-based approaches employ iterative generation and visual refinement to reflect the real-world design workflows, their high-level plans are predetermined prior to execution, limiting their adaptability as the design evolves. To address this limitation, we propose PlayGround, a progressive layout generation framework that dynamically adapts to the current rendered canvas. PlayGround establishes a progressive design loop comprising a Progressive Planner, a Layout Generator, and a Visual Verifier, guided by a Structure-Aware Retriever. Given an initial set of unordered elements, the Structure-Aware Retriever initially infers a coarse layout structure to retrieve structurally similar reference designs. Subsequently, at each turn, the Progressive Planner examines the current rendered canvas alongside the remaining elements, forms the next functional group, and plans its composition. The Layout Generator then predicts editable layout attributes, while the Visual Verifier evaluates rendered candidates and provides feedback when refinement is necessary. To train the Progressive Planner with step-wise supervision, we construct a two-stage data generation pipeline that extracts functional groups and turn-level planning trajectories from completed designs. Experiments on Crello and LICA demonstrate the effectiveness of PlayGround over existing methods. Our code and weights will be publicly released.

0.32

From Visual Widgets to UI Code: Efficient Tool-Grounded Generation

cs.CV cs.LG ·2026-08-12 ·Houston H. Zhang, Tao Zhang, Li Gu, Linfeng Ye, Yuanhao Yu +3 more
published similarity 0.64
Abstract · ECCV2026 (MUCG Workshop)

Existing screenshot-to-code systems face a trade-off between flexibility and controllability. Direct multimodal generation can hallucinate visible details, whereas structured pipelines reduce such errors through component-wise decomposition, predefined templates, and customized intermediate representations. These structures, however, introduce additional generative orchestration and restrict outputs to designs covered by the representation. We investigate whether selective tool grounding can improve the fidelity--efficiency trade-off of direct widget-to-code generation. We introduce \textbf{WidgetGen}, a lightweight tool-grounded framework that extracts observable text and color evidence, performs high-level layout and optional chart reasoning, and directly generates executable JavaScript XML (\emph{JSX}). This design reduces reliance on component-wise generation while avoiding a fixed UI schema. Across six multimodal models and \(1{,}000\) held-out widgets, WidgetGen outperforms direct prompting and the structured Widget2Code pipeline on most visual reconstruction metrics, with consistent gains in area, legibility, and style. Finally, reconstruction-derived image-code pairs improve six Qwen-family open-weight models across every reported metric through supervised fine-tuning. These results establish WidgetGen as a strong lightweight baseline and show that selective evidence grounding offers an effective alternative to extensive representation constraints.

0.49

CraftTrace: Unflattening Videos into Malleable, Creation-Inspired Structures for Generative Editing

cs.HC ·2026-09-24 ·Boyu Li, Yuqian Zhou, Duotun Wang, Ding Li, Zhe Lin +4 more
similarity 0.64
Abstract

Recent generative video editing models enable video content modification (e.g., changing a character) but target short clips. Extending them to full multi-shot videos requires tedious work to locate relevant content across shots, segment it into clips, craft context-aware editing prompts for each clip, and repeatedly articulate complex editing intent. To address this, we explore an interaction paradigm for editing through underlying video structures (e.g., scripts, scenes, characters, shots, and their relationships). We present CraftTrace, an interactive prototype that transforms a video into a malleable, multilevel structure for generative editing. Users work in task-centric workspaces to modify elements or reshape relationships, while an AI agent translates and propagates changes across the video. A user study and expert review show that this structure helps users understand videos, formulate and refine editing intent, and explore alternatives, supporting rapid prototyping during early-stage exploration and full video post-production.

0.30

PolyGandr: Steering Generative Models via Spatial Navigation through Latent Space

cs.HC ·2026-10-27 ·Mingwei S. G. Li, Susie S. Y. Li, Isabel Moreira de Oliveira, Ryan P. Adams, Remco Chang
similarity 0.63
Abstract

Generative models support content creation across text, images, videos, and design disciplines, yet their interfaces rely primarily on natural language. While effective for specifying high-level intent, language can be inefficient and imprecise for fine-grained control, iterative refinement, and open-ended exploration. We present PolyGandr, a visual interface for exploring relationships between latent representations and generated outputs, complementing natural-language model steering. PolyGandr formulates generative control as high-dimensional navigation, enabling users to define starting points, steering directions, and step sizes through a unified map-and-mixing-palette interface. Combining global exploration with local refinement supports both broad design-space navigation and fine-grained output manipulation without repeatedly translating visual intent into prompts. We evaluate PolyGandr across three use cases: StyleGAN2 face generation, Stable Diffusion chair design, and rotational kirigami engineering. A formative study indicates that PolyGandr complements existing workflows by supporting real-time editing while simultaneously providing an overview and means to explore the broader design space.

0.07

State-Aware Generation of Deployable Websites from Visual Prototypes and Requirements

cs.SE ·2026-10-13 ·Ruwei Pan, Junlei Shen, Linhao Wu, Zixiong Yang, Qisheng Zhang +5 more
hot direction similarity 0.63
Abstract

Generating deployable websites from visual prototypes and textual requirements is increasingly feasible with multimodal models, yet keeping generated projects consistent with their prototypes remains challenging. Prototypes specify layouts and visual elements, while requirements describe interaction workflows and deployment constraints. Existing UI-to-code methods and coding agents often treat layout reconstruction, interaction implementation, and project editing separately, allowing visual elements, files, workflows, dependencies, and render evidence to lose their correspondence across generation, verification, and update. We call this problem prototype-to-project state drift. We propose VisStateCoder, a state-aware framework that maintains these correspondences in a Project State Graph. The graph connects four types of state nodes through typed links and supports prototype grounding, graph-guided generation, and render-evidence-guided scoped updates that map failed checks to affected files while preserving files outside the update scope. We evaluate VisStateCoder on Vision2Web using five foundation models across Static Webpage, Interactive Frontend, and Full-Stack Website tasks. Averaged across models, VisStateCoder improves overall scores over the strongest baseline for each model by 26.1 and 22.6 points on Interactive Frontend and Full-Stack Website tasks, respectively. It also reduces verified-state drift, measured by checks that pass in one update round but fail later, compared with alternative state representations. An annotation study of 50 tasks finds that the constructed graphs achieve 95.6% edge-level F1.

0.30

MaLiang-Harness: A Programmable Path to Image and Video Generation

cs.CV ·2026-09-28 ·Haoyu Zhao, Zihao Zhang, Xudong Wang, Jiaxi Gu, Zuxuan Wu +2 more
code/data similarity 0.63
Abstract · 22 pages, 11 figures

Executable programs offer explicit control over how images and videos are constructed, but generating runnable code is only the beginning of visual creation. A program can execute correctly while violating the requested composition, appearance, or motion. We define this discrepancy as the Program-to-Visual (P2V) gap and introduce MaLiang-Harness, a unified framework for organizing MLLM-driven visual generation into a persistent process of construction, inspection, and revision. Its central design is to make the evolving visual program, its construction history, and its verification share a common revision reference. We define the Persistent Executable Generation (PEG) state as preserving programs and task context. Traceable Generation Process (TGP) connects edits to rendered evidence, and Revision-aware Editing and Verification (REV) supports restoration and checks the current revision before completion. Together, these mechanisms coordinate planning, execution, and visual feedback across rendering backends. We evaluate 11 powerful closed-source MLLMs on MaLiang-IBench and four on MaLiang-VBench, measuring generation success, visual quality, and computational cost. GPT-6-Astra achieves 100% generation success on both benchmarks, with 96.0% of image tasks and 76.9% of video tasks meeting all quality thresholds. The comparison also reveals a mismatch between general capability scores and visual generation performance, with similarly scored models differing substantially in their ability to satisfy visual requirements. MaLiang-Harness provides a systematic basis for studying how MLLMs translate executable code into visual outcomes, exposing both the potential of programmable generation and the limitations of general benchmarks as predictors of this ability. The project is available at https://github.com/gulucaptain/MaLiang-Harness.

0.70