CS Paper Radar

Answer Correction with Verifiable Rewards: A Gemma-Family Study of Multi-View Driving QA

T N Rickesh, Pranav Vinodh, Sowmya Kamath S

cs.CV cs.LG ·submitted 2026-10-31 11:19 UTC · v2

Abstract

Multi-camera driving question answering provides a discrete test of vision-language models, but post-training gains can reflect changed selected answers or improved answer extraction. We investigate this distinction within the Gemma model family using archived, task-linked WaymoQA evaluations. A zero-shot diagnostic on 5,519 test questions spans effective-small, mixture-of-experts, unified, and dense checkpoints, with serving differences disclosed rather than interpreted as architecture effects. The strongest controlled evidence is a matched-harness Gemma-4-12B base/GRPO pair: accuracy rises from 73.78% to 75.43%, a gain of 1.65 percentage points with a scene-cluster bootstrap interval of [+0.84, +2.41]. There are 215 improvements and 124 regressions; 213 improvements start from an extractable but incorrect base answer, and only two recover previously unextractable responses. Crossing correctness with extraction state shows why correct noncompliant outputs cannot be counted as prediction errors. Category counts reveal a critical safety asymmetry: routine perception gains (+5.93 pp) mask a regression in safety-critical hazard prediction (-1.82 pp), demonstrating that unweighted outcome rewards can trade off rare edge cases for aggregate accuracy. Reward saturation describes sampled training dynamics rather than an architectural ceiling. A separate archived Gemma-31B thinking comparison yields no detected average improvement, with uncertainty that does not establish equivalence. These results support a modest answer-selection gain for the recorded tuning configuration and a reproducible diagnostic accounting, without attributing corrected labels to specific grounding mechanisms or claiming superiority over matched supervised tuning.

Comment
6 pages, 2 figures, 3 tables

Closest papers

Nearest neighbours by meaning across the whole index.

Fewer Tokens, More Answers: Verifiable-Reward Tuning of an Ornith-Style VLM for Multi-View Driving QA

cs.CV cs.CLcs.LG ·2026-10-31 ·T N Rickesh, Pranav Vinodh, Sowmya Kamath S
similarity 0.75
Abstract · 7 pages, 2 figures, 3 tables

Reasoning-enabled model derivatives must turn intermediate deliberation into a usable answer within a deployment budget. We study that transition in an Ornith checkpoint adapted to multi-camera driving question answering with verifiable rewards. Paired archived outputs show that, on a selected 400-question development screen with a 256-token cap, GRPO increases answer extraction from 78.25% to 90.25% and accuracy from 55.50% to 63.75%. Of 39 wrong-to-correct transitions, 33 begin with an unextractable base response. The base's 87 unextractable responses all reach the token cap. Checkpoint-verified reconstruction on those same questions gives a much smaller first-pass accuracy gain at 512 tokens: +0.75 percentage points, versus +8.25 at 256. A later answer-cued continuation is analyzed separately; it helps the base more and narrows the adapter difference. On the full 2,511-question development set, this recovery-enabled pipeline reduces mean generated tokens from 108.6 to 47.1, while the paired accuracy interval includes zero gain. We frame these findings as adaptation of answer delivery under finite budgets, with explicit selection and provenance limits. Ornith's self-scaffolding lineage and newer reasoning-enabled derivatives such as Bonsai motivate the problem, but our measured results concern Ornith alone and do not establish a shared architectural mechanism or cross-model transfer.

0.51

Reasoning Instructions Can Break Answer Decoding in Vision--Language Models

cs.CL cs.AI ·2026-09-24 ·Zeyan Li, Siyuan Qiu, Jianfeng Xu
similarity 0.73
Abstract

Chain-of-thought (CoT) instructions can distort multiple-choice VLM evaluation when a scorer appends a reasoning cue but reads answer-label logits before the model generates any rationale. We call this CoT-prefix scoring. On ScienceQA, Qwen2.5-VL-7B drops from 80.76% to 45.48%, and across five option-content permutations 93.54% of CoT-prefix predictions select the first slot. Condition-matched linear probes recover 78.94% from the same hidden states, while free generation restores 75.24%, showing that the answer often survives the prefix and the immediate readout fails. Vocabulary and layer diagnostics explain the mismatch: probability mass moves toward continuation tokens, while answer information remains linearly accessible in late layers. The effect recurs with varying severity across datasets and models, though not universally. These results show that CoT-prefix scoring can confound model knowledge with an evaluation-interface mismatch and should be avoided unless the requested and scored output events are aligned.

0.17

GroundSight at GroundLM 2026 Shared Tasks: GoldenViewVQA

cs.CV ·2026-10-08 ·Kun Wang, Yupeng Hu, Ruping Cao, Hao Liu, Zhiran Li +2 more
similarity 0.72
Abstract

GoldenViewVQA requires models to jointly answer driving-scene questions and identify the camera view containing the supporting visual evidence, making precise evidence localization as important as answer correctness. We present \textbf{CoVeR-VQA}, a training-free multi-stage verification and correction framework for grounded multi-view VQA. Starting from GPT-5.6 zero-shot predictions, CoVeR-VQA progressively applies view-specific verification with Gemini-3.6-Flash, prior-guided joint verification with Claude-Opus-5, and cross-split group-level verification that exploits semantically filtered question groups from shared multi-view scenes and validation-derived prior knowledge. On the official GoldenViewVQA test set, the four-stage CoVeR-VQA pipeline achieves 84.75\% Joint Accuracy, improving the GPT-5.6 zero-shot baseline by 13.56 percentage points, while reaching 94.92\% Answer Accuracy and 86.44\% View Accuracy. The final submitted run achieves 88.14\% Joint Accuracy after two additional evaluator-informed post-hoc corrections. Our analysis shows that supporting-view localization remains the primary source of residual errors, highlighting the importance of explicit evidence verification for reliable multi-view multimodal reasoning.

0.21

Answer with Evidence: Consistency-Aware Grounded Visual Question Answering for Roadside Traffic Scenes

cs.CV cs.HC ·2026-10-04 ·Runwei Guan, Rongsheng Hu, Shangshu Chen, Ningwei Ouyang, Shaofeng Liang +8 more
benchmarkcode/data similarity 0.72
Abstract · 14 pages, 7 figures

Roadside traffic reasoning requires every free-form textual claim to be backed by visual evidence. Existing grounded multimodal large language models (MLLMs) frequently exhibit say-point mismatch, in which the textual answer contradicts the bounding boxes the model localizes. Evaluation metrics that score answers and boxes separately leave this failure unpenalized. We trace the mismatch to the conventional answer-then-ground factorization, which commits to a numerical claim before any object is enumerated. To measure it, we build RoadSceneVQA-G, a benchmark of 34.7K question-answer pairs in which every free-form answer is linked to the set of boxes that witnesses it, and we propose the Answer-Grounding Consistency (AGC) evaluation suite. To address it, we introduce Enumerate-then-Answer (EtA), which reverses the generation order so that answer-evidence agreement becomes a property of the output structure, and Enumeration-Consistent Policy Optimization (ECPO), a reinforcement learning stage that uses the union of multiple rollouts as a recall teacher without ground-truth boxes. EtA raises say-point consistency from 26.6\% to 93.7\% and grounding F1 from 52.2\% to 73.0\%, and ECPO further increases F1 to 75.6\% without per-box supervision. On gRefCOCO, the same framework outperforms the strongest compared method, indicating that it transfers beyond traffic scenes. The project is available at https://github.com/GuanRunwei/RoadSceneVQA-G.

0.36

PRISM-BENCH: Diagnosing Grounded and Faithful Multimodal Reasoning Beyond Final-Answer Accuracy

cs.CV cs.CL ·2026-10-27 ·Vansh Bhatia, Mehdi Rezagholizadeh, Parsa Ashrafi Fashi, Igor Gilitschenski, Vikram Appia +1 more
publishedbenchmarkcode/data similarity 0.70
Abstract · Accepted as a poster at the NeurIPS 2026 Workshop on Grounded and Faithful Vision-Language Models for Real-World Deployment (VLM4RWD). 18 pa…

Final-answer accuracy reports whether a vision-language model was right, but not whether its answer was grounded in the image, whether its stated reasoning reflects that evidence, or what the answer cost to produce. Those are the properties that decide whether a system can be deployed. We present PRISM-BENCH, a benchmark of 1,000 expert-curated questions spanning five under-represented reasoning categories, in which every item carries an idealized reasoning trace written by one human annotator and validated by another. The traces support two diagnostics that accuracy cannot provide. A perception probe, in which the model answers from a description it wrote itself, attributes roughly 85% of frontier-model failures to misperception and places the bottleneck at grounding. Trace alignment predicts answer correctness for all 25 systems tested (macro AUROC 0.66) and survives length control, so it measures reasoning content and not verbosity. We further introduce CORE, an efficiency-aware metric that reorders 24.2% of model pairs relative to accuracy while remaining uncorrelated with response length. Across 25 open and closed systems, including GPT-5 and Claude Opus 4.6, the best accuracy is 66.0%. On the four categories with a human baseline the best system trails humans by 18.4 points, and recent frontier gains fall on recognition and not on spatial grounding.

0.58

Correctness Is a Direction: Geometric Answer Selection in Language Models

cs.CL ·2026-10-03 ·Marcus Armstrong, Navid Ayoobi, Pradham Mummaleti, Alexander Chulzhanov, Arjun Mukherjee
similarity 0.70
Abstract

Answer correctness is encoded as a recoverable geometric direction in the hidden states of language models. We show that the mean displacement from incorrect to correct answer representations, computed at approximately 70\% of model depth from fifty labeled examples with no parameter updates, yields a scoring direction that outperforms zero-shot log-probability scoring by up to +32.0 percentage points on factual benchmarks (ARC-Challenge and MMLU) and by +38.1 to +51.8 percentage points on TruthfulQA, across five models spanning 1B to 8B parameters in three architecture families (Llama, Qwen, Gemma). The method requires one forward pass and one dot product per candidate; no generation is performed at inference. Applied as a hallucination detector on individual (question, answer) pairs, the recovered direction achieves 0.693~AUROC versus 0.578 for log-probability scoring. We additionally find that correctness directions for factual reasoning, domain knowledge, and calibrated truthfulness are near-orthogonal in representation space, revealing that language models allocate geometrically independent subspaces to qualitatively distinct notions of correct answer, with architecture-dependent variation in the degree of separation. This structure explains the observed transfer pattern---the direction calibrated on factual questions transfers within task type but not across it---and suggests that LLM calibration failures may reflect a routing problem: the model's internal representation contains more correctness signal than its output behaviour exploits.

0.41

Do VLMs Really Look at the Road? A Diagnostic Evaluation for Autonomous Driving

cs.CV ·2026-11-14 ·Shankar Gangisetty, Muskan Singh, C V Jawahar
benchmarkcode/data similarity 0.70
Abstract

Vision-language models (VLMs) are increasingly being evaluated for autonomous driving through question-answering benchmarks, yet high benchmark accuracy does not necessarily reflect visually grounded reasoning. We present a diagnostic evaluation framework that systematically probes whether VLMs base safety-critical driving decisions on visual evidence or instead rely on language priors, answer biases, and dataset regularities. Our benchmark comprises two task categories, Adverse Weather and Junctions \& Intersections, with a nuScenes spatial reasoning extension, and evaluates open-source (general, driving, and spatial-reasoning) VLMs along four diagnostic protocols: image versus no-image evaluation, counterfactual weather sensitivity, linguistic robustness under paraphrasing and negation, and spatial reasoning. All ground-truth labels are derived programmatically from BDD100K and DriveLM annotations, enabling reproducible evaluation without manual labeling. Across all models, we find that competitive overall accuracy conceals systematic failure modes, including strong affirmation bias, limited sensitivity to counterfactual scene changes, near-chance performance on negated questions despite high paraphrase consistency, and confident driving decisions even without visual input. These results demonstrate that current open-source VLMs remain insufficiently grounded for safety-critical autonomous driving and underscore the need for diagnostic evaluation beyond aggregate accuracy.

0.23

Visual Dependence Does Not Imply Answerability: A Systematic Analysis of Answerability Signals in Frozen InstructBLIP

cs.CV cs.AI ·2026-10-13 ·Minjun Kim, Hyunmin Song, Changwon Lim
similarity 0.70
Abstract · 10 pages, 3 figures, 5 tables

Visual question answering models often produce plausible responses even when the available visual evidence is insufficient to support a valid answer. Although prior work has introduced benchmarks and abstention mechanisms for unanswerable visual questions, it remains unclear which signals derived from a frozen vision-language model reliably indicate answerability on unseen image-question pairs. We systematically evaluate answerability signals on a stratified 10K subset of UNK-VQA using frozen InstructBLIP, covering predictive uncertainty, cross-modal alignment, visual grounding, visual dependence, internal representations, and learned probes. Answerability discrimination remains weak, with test AUROC largely confined to 0.50-0.54. CLIP alignment yields the highest point estimate but does not significantly outperform generation confidence under paired bootstrap analysis. Targeted removal of question-relevant visual evidence causes substantially larger changes in answer confidence than same-size random masking, yet intervention sensitivity provides essentially no answerability discrimination. Increasing probe capacity, combining multiple signals, and adding reason-aware supervision do not provide robust improvements, while post-freeze cross-validation yields similarly weak discrimination. Selective-answering analysis shows limited usable coverage under low-risk operating criteria. Failure-mode analysis shows that the relative utility of individual signals varies across perturbation mechanisms. Taken together, these results show that, on our UNK-VQA subset with frozen InstructBLIP, visual dependence, uncertainty, semantic alignment, and static representations are not robust standalone proxies for answerability. Our findings emphasize the need to distinguish a model's reliance on visual evidence from whether that evidence is sufficient to support an answer.

0.20