CS Paper Radar

MARS - A foundational map auto-regressor

Qi Zhang, Suvam Bag, Rupanjali Kukal, Lucas Thai, Jacob Kovarskiy, Mike Bianco, Xiaojie Guo, Raphael Tang, Mikael Figueroa, Rishi Madhok, Nikolaos Karianakis, Fuxun Yu

cs.CV ·submitted 2026-10-14 19:28 UTC · v1
top venuedatasetcode/data

Abstract

Map generation tasks feature extensive non-structural vectorized data (e.g., points, polylines, and polygons) and thus pose significant challenges to common pixel-wise generative models. Conventional approaches use multiple stages, first segmenting these features at the pixel level and then performing vectorized post-processing, with errors and complexity compounding at each stage. Motivated by the recent success of auto-regressive language modeling, we propose the first map foundation model, named Map Auto-Regressor (MARS), that is capable of generating both multi-polyline road networks and polygon buildings in a unified manner. For training MARS, we collected to our knowledge the largest multi-class map extraction dataset totaling 3.4M examples, which we call MAP-3M. Across four road and building datasets, MARS outperforms or matches the performance of multistage baselines. Additionally, we develop a "Chat with MARS" capability that enables interactive human-in-the-loop map generation and correction, supported by the auto-regressive nature of our end-to-end approach. We release our MAP-3M dataset and project demo page at (1) https://huggingface.co/datasets/bag-lab/MAP-3M and (2) https://huggingface.co/spaces/bag-lab/MARS, respectively.

Closest papers

Nearest neighbours by meaning across the whole index.

From Fragments to Global Maps: Learning Vectorized Map Aggregation with Large Language Models

cs.CV cs.AI ·2026-10-01 ·Ziwei Li, Yi-Tang Chen, Xiaoqi Wang, Wenbin He, Han-Wei Shen +1 more
similarity 0.68
Abstract

Large-scale vectorized HD maps provide structured road information that is essential for perception, localization, and planning in autonomous driving. Constructing such maps requires aggregating noisy, fragmented, and overlapping local predictions collected along a vehicle trajectory into a coherent global map. Existing aggregation methods typically rely on hand-crafted rules for fragment association and refinement. However, a fixed set of thresholds cannot effectively handle variations in road structures and prediction errors, often requiring detector-specific tuning or manual adjustment. To address this limitation, we propose MapMergeLLM, a data-driven framework that formulates vectorized map aggregation as conditional sequence generation with a large language model. Given serialized local vectorized maps, our model directly predicts the aggregated global map polylines. To reduce dependence on any particular upstream detector, we train the model on synthetic local maps generated from clean vector maps using corruptions that simulate representative prediction errors. We further introduce a coordinate tokenizer with geometry-aware pretraining to precisely represent map coordinates. In addition, we propose a line-level association loss that explicitly supervises correspondences between local observations of the same map element. Experiments on Argoverse2 and nuScenes using multiple recent upstream detectors demonstrate that MapMergeLLM substantially outperforms heuristic and optimization-based aggregation baselines without detector-specific retraining.

0.13

Click2Poly: A VLM for vector mapping buildings and walls

cs.LG cs.CV ·2026-08-11 ·Nicolas Girard, Jawher Ben Abdallah, Arno Gobbin, Liuyun Duan, Sacha Lepretre
published similarity 0.64
Abstract

Accurate vector mapping of buildings and walls is critical for geospatial applications but remains a labor-intensive process. While recent deep learning methods have improved automatic extraction, in order to meet cartographic standards they always require a human to perform quality control and fix complex cases in the extraction. We present Click2Poly, a human-in-the-loop AI assistant designed to speed up this manual step. Extending the Florence-2 Vision Language Model (VLM), Click2Poly responds to user clicks by editing the building or wall vector layer directly. Implemented as a QGIS plugin, Click2Poly speeds up the manual editing of building and wall vector layers in a real-world production environment.

0.23

CityLift: Compositional 3D Urban Generation from a Single Image

cs.CV ·2026-10-23 ·Haoran Gong, Jie Feng, Chenxu Wang
code/data similarity 0.61
Abstract

Generating editable 3D urban scenes from a single image requires detailed building assets with a coherent spatial arrangement. Occlusion and perspective ambiguity make it difficult to infer complete building geometry together with consistent relative scales and positions. We present CityLift, a framework that generates individual textured building meshes from an urban image by coupling scene-level spatial modeling with building-level synthesis. CityLift first learns building shape and placement jointly through coupled scene and instance streams, producing coarse sparse structures in a common coordinate system. We then introduce generative spatial inheritance, which maps each building's coarse occupancy and scene-conditioned 3D feature field into a local volume to guide a flow model that generates finer structure. The same coordinate mapping places the generated meshes back into the scene, preserving their spatial relationship to the scaffold. To address the scarcity of urban data pairing scene images with individual 3D buildings, we construct CityLift-5K, comprising 4,997 synthetic scenes with instance masks, building meshes, and placement annotations. Experiments on CityLift-5K and scene blocks from UrbanScene3D and InstanceBuilding demonstrate improved building geometry and scene composition. The project page is available at https://ranhaogong.github.io/CityLift/.

0.22

A Scene Language Model for Open-Vocabulary Scene Mapping

cs.CV cs.RO ·2026-09-18 ·Adam Lilja, Fabio Hübel, Siming He, Junsheng Fu, Claire Tomlin +4 more
benchmarkcode/data similarity 0.61
Abstract

Open-vocabulary 3D scene mapping aims to build a persistent representation of the objects in an environment. Existing systems typically rely on engineered mapping pipelines to associate observations, merge information across views, and maintain a consistent scene representation over time. Many additionally store feature-rich object representations, such as embeddings or image crops, increasing the size and complexity of the persistent memory. We introduce SceneLM, a Scene-Language Model that directly maintains a textual scene map. The full scene is represented as a structured text list of objects, which serves as the model's only persistent memory. For each input image, the model reads the current scene state and updates the map by adding, editing, and removing objects. To learn this behavior, we introduce supervision tasks for iterative scene map maintenance together with an automatic annotation pipeline that generates training data from images without human labels. We evaluate SceneLM on both a language-grounded retrieval benchmark and a localization benchmark. Across both benchmarks, the model produces a scene map that achieves competitive performance with complete mapping systems built from dedicated perception and geometric modules while producing a scene representation that is 6-12x more compact. We further show that SceneLM can be run online on an edge device through experiments on a quadruped. These results show that a persistent open-vocabulary 3D scene map can be maintained directly by a single vision-language model using only a lightweight text representation. Training and inference code is available on https://goldengait.github.io/scenelm/.

0.31

nnMNet: Baseline for Martian Terrain Semantic Segmentation

cs.CV ·2026-08-30 ·Ming-Han Lee, Chi-Yeh Chen
benchmarkcode/data similarity 0.60
Abstract

Semantic segmentation is a crucial task for understanding Mars, the most Earth-like planet in our solar system. However, it is challenging because the Martian surface is highly unstructured and complex, making accurate pixel-level prediction and fine-grained annotation difficult. Recent advancements in deep learning have introduced numerous methods and datasets to address these challenges. Nevertheless, the field lacks a robust, publicly available, and reproducible baseline, as well as a unified benchmark to facilitate fair evaluations. In this work, we present nnMNet, a new baseline model designed for Martian terrain semantic segmentation. Building upon nnWNet, we integrate linear attention to better capture global context and employ lightweight convolutions to reduce computational overhead. To bridge the gap between local and global representations, we introduce the Spatially-Aware Fusion Block (SAFB), which augments and combines features with diverse characteristics. Furthermore, we establish a new benchmark by curating and standardizing three high-quality datasets for thorough evaluation. nnMNet achieves new state-of-the-art 86.61%, 83.25%, and 88.24% mIoU on SynMars-TW, SynMars-Air, and MarsScapes, respectively. Our code, models, and datasets are publicly available at https://github.com/dereklee0310/nnMNet.

0.29

MarsRecon: Self-Supervised and Multimodal Surface Representations for Mars

cs.CV cs.AI ·2026-09-17 ·Akshay Naik, Marius F. R. Juston, Jay Mahajan
similarity 0.60
Abstract · 27 pages, 22 figures

High-resolution orbital imagery offers a rich record of the Martian surface, but sparse geological labels limit supervised representation learning. We present MarsRecon, a geospatially aware pipeline for learning visual and multimodal representations from HiRISE observations of Olympus Mons. The pipeline calibrates NASA Planetary Data System products, extracts valid georeferenced patches, and trains a masked autoencoder on unlabeled imagery. Increasing input resolution and filtering invalid tokens reduced held-out reconstruction loss from 0.1751 to 0.1342 in the principal Stage A model series. We then freeze the visual encoder and align its features with observation text, coordinates, and local--global image context. The strongest current local-primary model achieves image-to-text recall@10 of 0.3787, text-to-image recall@10 of 0.9161, and local-to-global recall@10 of 0.4350 on the held-out test split. These results establish a working Mars-specific pretraining and retrieval pipeline; further crop-overlap controls and downstream geological evaluations are needed to assess the broader utility of its embeddings.

0.24

RoadState: A Domain-Prior-Enhanced Vision-Language System for Industrial-Grade Lane-Level Map Updating

cs.CV cs.AI ·2026-11-07 ·Deguo Xia, Zihan Li, Jiaqi Wang, Dong Xie, Xiyan Liu +9 more
published similarity 0.60
Abstract · 12 pages, 5 figures, including appendices. Accepted to AAAI-IAAI 2027, poster presentation

Lane-level map updating requires temporally consistent, physically valid, and auditable road-scene semantics despite occlusions, ambiguous lane markings, temporary obstacles, and viewpoint changes. We present RoadState, a production-deployed vision-language system that transforms sequential road-scene observations into structured lane-level semantics for Baidu Maps' map-updating pipeline. Its core model, RoadState-VLM, formulates road-scene understanding as autoregressive state estimation, predicting the current structured road state from the previous observation and symbolic state together with the current observation. We introduce Geometric Chain-of-Thought (Geo-CoT), which uses a directed acyclic graph of semantic dependencies as a domain prior to improve semantic consistency and reasoning fidelity across interdependent road-state dimensions. Extensive experiments on a real-world dataset show that RoadState-VLM achieves 94.33% mean accuracy across six road-state dimensions, outperforming the strongest fine-tuned VLM baseline. Ablation studies further demonstrate the complementary effectiveness of Geo-CoT and temporal transition context. A cascaded routing pipeline directs routine cases to lightweight models while reserving complex cases for RoadState-VLM. Since its deployment in March 2025, RoadState has supported lane-level map updating across more than 360 cities and reduced overall production costs by 30%. Together with human-in-the-loop auditing, the system maintains robust performance under stress-test conditions. These results demonstrate how domain-prior-enhanced reasoning, structured state interfaces, selective routing, and human quality control can bridge the gap between VLM-based road-scene understanding and large-scale lane-level map updating.

0.30

Consistency-Aware Map Generation at Multiple Zoom Levels Using Aerial Image

cs.CV ·2026-11-14 ·Linwei Chen, Zheng Fang, Ying Fu
publisheddatasethighly cited similarity 0.59
Abstract · 14 pages, 9 figures

The multi-level tiled map service is widely used and serves as a kind of digital infrastructure. These map tiles are usually rendered from vector data, whose update needs to walk or drive with professional equipment to check every points of interest. This leads to inconvenience and expensive cost in timely updating maps. Compared with vector data, aerial images are much easier and cheaper to obtain. In this paper, we propose a novel multi-level map generation framework that can automatically generate accurate and consistent maps with multiple zoom levels from aerial images. It consists of a level-aware map generator and a consistency-aware map generator. The level-aware map generator is able to generate accurate initial maps with realistic details for each zoom level. The consistency-aware map generator regards initial maps at each zoom level as a sequence and builds the connection between them, so as to guarantee content consistency between maps at different zoom levels. Furthermore, we collect a large-scale high-quality dataset called Multi-Level Map (MLM) for map generation at multiple zoom levels. Experiments on our MLM dataset show that our method outperforms previous state-of-the-art map generation methods on both comprehensive quantitative metrics and perceptual quality.

0.52