CS Paper Radar

DICE: Tilting the Odds through Interface Attacks on System One Model

Ben Dong, Tianyi Lu, Zhonghao Guo, Qian Wang

cs.CR cs.SE ·submitted 2026-10-09 23:47 UTC · v2

Abstract

System One models offer typed judgments that applications can turn into decisions. Their answers, however, are conditional on the state and question supplied by surrounding components, and on how the application uses the response. We study this boundary in a Jev-based pipeline with bounded substitutions at three adapter outputs: state extraction, question generation, and output interpretation. Across three small synthetic studies, each adapter can change the final decision while Jev and its authentic API responses remain outside attacker control. The cases expose distinct failures of source fidelity, answer meaning, and response relevance. Checks that bind these relationships stopped the corresponding interventions we tested, although source verification also rejected some valid clean decisions. The pilot identifies where correct request-level judgments can lose their connection to the original task; it does not estimate the prevalence of these failures in deployed systems.

Closest papers

Nearest neighbours by meaning across the whole index.

JevBreak: Your Model, Their Decision

cs.CR cs.AI ·2026-11-02 ·Nitin Aravind Birur, Divyanshu Kumar, Sahil Agarwal, Prashanth Harshangi
similarity 0.70
Abstract

Restricting what a model can say is often mistaken for restricting an attacker's influence over its decision. An attacker can add text to the evidence and steer the model toward a wrong answer that still belongs to the allowed set. We evaluate seven decision models spanning hosted APIs, open-weight models and an encoder on 16 tasks in seven domains from Decision Bench, targeting every wrong option across 240 text examples (990 targets per model). We present Slot Attack, a black-box attack designed for decision models that appends text assembled from named components to steer predictions toward an attacker-selected incorrect option. The method relies solely on returned label probabilities, achieves success rates of 32--99\% on initially correct decisions, and identifies the effective component combinations with a median of 17--72 queries per successful attack. Accuracy and confidence do not reliably reveal this vulnerability. Models with near-identical accuracy on the sampled tasks differ by about 20\%age points in attack success. We demonstrate that an attacker can force not just one wrong answer but every available one: on about 20\% of correct decisions for the JEV models, 77\% for gpt-6-luna and 98\% for GLiNER2.5; a confidence threshold does not reliably screen the attack out. A fixed answer set constrains a decision's form, not its source; robustness must be measured per model, per task and per version rather than assumed from output format.

0.15

Decision Hijacking: Prompt Injection Attacks on Jev's Typed Probabilistic Decisions

cs.CR cs.AI ·2026-09-23 ·Tiantong Wu, Wei Yang Bryan Lim
highly cited similarity 0.70
Abstract

Most studies of prompt injection focus on generative agents, leaving their effects on models with schema-defined outputs unclear. We examine these effects in Jev, a non-generative decision model, using 510 reconstructed InjecAgent cases. Malicious content shifts action probabilities but rarely causes Jev to select the attacker's target. Override markers reduce this influence, while claims of contextual relatedness have small effects. Adaptive attacks using score feedback double the mean highest attacker-target probability found during optimization, while success on fresh validation calls rises from 1.8% to 3.5%. Exploratory analysis links these successes to small initial decision margins or greater attacker control over the observation. Together, these findings show that schema-defined outputs change but do not eliminate prompt-injection risk, highlighting the need to evaluate how untrusted content influences choices within the allowed action set.

0.49

Poisoned into Action: Context Attacks on Typed Decision Workflows

cs.CR cs.LG ·2026-10-11 ·Chenkai Wu, Xing Chang, Jun Yan
code/data similarity 0.70
Abstract · 13 pages, 3 figures. Code available at https://github.com/WuXixiong/jev-poisoning-bench

Typed decision interfaces restrict models to predefined actions, yet schema-valid choices can still trigger policy-violating state changes. Evaluating context poisoning thus requires connecting model decisions to execution outcomes while accounting for legitimate automation retained by defenses. We introduce an execution-aware protocol tracing poisoned evidence through retrieval, model choice, automated action, and persisted state. We evaluate it with 1,659 fresh calls to Jev-1.13.0 across 20 development tasks in two synthetic workflows, using three repetitions per condition. Under natural retrieval, a standalone payload reaches all 60 trials and produces five security-violating executions, while a carrier-payload pair exposes the payload in 57 trials and produces one violation. An initial source-exclusion gate blocks the pair's observed violation but also withholds ten otherwise correct attacked decisions and sixteen correct clean decisions. Correcting its empty-scope behavior restores twelve clean decisions without additional model calls. Factorial controls show that adding correct evidence reduces wrong executions from 119/120 to 55/120 in the semantic arm and from 60/120 to zero in the neutral arm, suggesting that much of the improvement reflects general evidence value rather than attack-specific protection. Three prespecified contrasts are not statistically supported after Holm correction, so the findings remain exploratory. The protocol provides an auditable framework for studying which poisoning-induced errors reach execution and how much correct automation defenses preserve. Code is available at https://github.com/WuXixiong/jev-poisoning-bench.

0.38

Nudge the Judge: Uncovering Program Decision Hijacking in Real-World Jev Applications

cs.CR cs.AI ·2026-10-13 ·Yang Luo, Tiantian Ji, Zifeng Kang, Yucheng Wang, Yong Liu +2 more
code/data similarity 0.69
Abstract

System One models such as Jev change how software makes decisions by mapping open-ended semantic contexts into structured judgments that directly drive program execution. However, this paradigm introduces a new security risk, which we term Jev Program Decision Hijacking (JPDH): attacker-controlled contexts can redirect program decisions through schema-valid Jev judgments. Existing evaluations focus primarily on model-level attacks or predefined injection channels, leaving attack reachability and downstream security consequences in real-world Jev applications underexplored. In this paper, we present JevProbe, the first automated framework to evaluate JPDH by tracing adversarial influence from reachable context surfaces through program-consumed decisions to downstream security consequences in real-world Jev applications. Our key insight is that application logic maps diverse Jev judgments to observable decisions, enabling systematic, program-grounded attack evaluation. Across 228 popular real-world Jev applications and 2,273 test cases, 81.5% exhibit decision flips, of which 98.6% lead to potential downstream security impacts. Compared with the strongest adapted baseline, JevProbe improves macro-average Decision Flip Rate (DFR) by 50.4 percentage points, while existing defenses offer limited protection. We further construct JevProbeBench to support systematic evaluation of JPDH in real-world Jev applications. We have responsibly disclosed our findings to Jev application maintainers, pending their responses. The source code of JevProbe and the JevProbeBench dataset are publicly available on GitHub (https://github.com/xiaoxiaohg8d/JevProbe-).

0.51

On Task, Off Track: Benchmarking LLM and Agent Robustness to Near-Target Prompt Injection

cs.CR ·2026-10-23 ·Yuyang Gong, Zihao Wang, Jiawei Liu, XiaoFeng Wang
benchmark similarity 0.67
Abstract

Prompt injection benchmarks assess whether large language models (LLMs) and agents resist malicious instructions embedded in external content. However, a system could perform well by rejecting attacks that obviously abandon the user's request, while still accepting changes that retain most of a legitimate answer but make it incorrect or harmful. Testing more instructions for unrelated outcomes would not necessarily reveal this weakness. We call attacks seeking responses designed to resemble legitimate answers while introducing incorrect or harmful changes near-target prompt injection. We introduce NTBench, the Near-Target Benchmark, which builds on legitimate tasks from existing datasets. For each task, a legitimate reference outcome guides the generation of target--instruction pairs, with separate tracks targeting incorrect and harmful outcomes in responses or agent tool invocations. Harmful targets can also accompany otherwise correct task completion. We combine LLM-based checks with human validation of the intended deviations, then assess whether injected instructions induce them using response judgments or tool-call checks, excluding deviations already observed without injection. We evaluate seven LLMs on ten datasets for language tasks and two agent benchmarks. Under a common budget of three attack executions per sample, both tracks achieve higher overall attack success rates than the fixed-target baseline for every evaluated model in each setting. On language tasks, the increases range from 9.3 to 35.8 percentage points for incorrect targets and 1.7 to 17.3 percentage points for harmful targets. These results show that resistance to the tested fixed-target attacks does not imply resistance to related incorrect or harmful outcomes, and that reference-guided targets broaden the failures revealed by robustness evaluation.

0.19

Hidden Risks of Jev: An Empirical Study of Security, Privacy, and Dual Use

cs.CR cs.AI ·2026-10-04 ·Shang Wang, Tianqing Zhu, Huajie Chen, Jiayang Li, Meng Yang +1 more
code/datahighly cited similarity 0.66
Abstract · 16 pages, 8 figures, 6 tables; The source code is available at \url{https://github.com/shihe98/Security_Privacy_Jev}

Jev turns natural-language questions into typed answers and probabilities with low latency and cost, enabling applications to route requests and select tools. While this interface allows Jev to integrate naturally into application workflows as a decision layer, the security and privacy implications of this emerging use remain largely unexplored. To address this gap, we conduct the first systematic study of these implications using the official Jev API and NanoJev, a local model with controllable training data and updates, focusing on three research questions: (1) What security threats arise when Jev is deployed as an application decision layer? (2) What private information can Jev reveal despite returning constrained typed outputs? (3) How can Jev's general-purpose decision capability be used for beneficial purposes or misused? Jev's decisions depend on application state and may be influenced by user-provided inputs. We therefore adapt prompt injection and adversarial suffixes to manipulate its decisions. Open-source Jev distribution and updates introduce supply-chain risks, which we examine by implanting backdoors in NanoJev through training data poisoning. Since Jev's outputs reflect both application state and information learned during training, we further adapt membership, private attribute, and internal knowledge inference attacks to recover sensitive information despite its constrained output format. Finally, Jev can serve as a general-purpose decision oracle for defensive and malicious workflows. We examine this dual use through four detection tasks covering prompt injection, jailbreak inputs, harmful content, and AI-generated text, alongside misuse scenarios involving jailbreak and model extraction. Our empirical evaluation shows that Jev remains vulnerable to the examined security and privacy threats, while its decision capability can support beneficial and malicious uses.

0.61

Can the Confidence and Decisions of System 1 Models Be Trusted? An Explainability and Perturbation Audit of Jev-Style Decision Models

cs.LG cs.CL ·2026-11-09 ·Julien Delaunay
benchmarkcode/data similarity 0.66
Abstract · 36 pages, 4 figures, 16 tables. Code and benchmark artifacts available at https://github.com/j2launay/jev-decision-models-xai

"System 1" decision models (pioneered by TypeSafe AI's Jev) replace autoregressive text generation with a single forward pass returning typed probability distributions over declared schemas (Choice, Score, Noul). While deployed for low-latency routing and automated triage, their statistical calibration and post-hoc explainability remain uncharacterized. We introduce an open, deterministic audit benchmark evaluating System 1 decision models across calibration under label-preserving perturbations, cross-question interference, and post-hoc attribution faithfulness. Evaluating five open-weight models (0.8B to 27B) alongside generative System 2 baselines -- both single-token logprob scoring and structured JSON/CoT decoding -- across NLI (e-SNLI), adversarial syntactic shortcuts (HANS), synthetic clinical triage (N=900), prompt injection defense (Tensor Trust, N=250), and an 86-case logic-gate trigger suite yields five findings: (1) provider confidence is an exact closed-form dispersion transform (1 - H(p)/ln K on sub-10B models; normalized margin on Clef-27B) rather than empirical accuracy; (2) bundling multiple questions in one request induces directional cross-question interference (11.7% to 22.4% binary flips on Tev1-0.8B, vs. 0.1% to 0.2% on Tev1-4B); (3) System 1 models with >= 4B parameters are faster and better calibrated under syntactic shifts (ECE_w = 0.030 to 0.087) than logprob (0.144) and JSON (0.378) baselines; (4) for prompt-injection defense (N=250), >= 4B models achieve 0.988 to 0.998 AUROC with up to 95.2% recall and 0.0% false alarms at 48 to 60 ms; and (5) single-pass inference makes 512-coalition XAI interactive (1.3 to 3.2 s/input), while KernelSHAP with a logit link (S1-SHAP) mitigates softmax saturation on disjunctive OR gates (89.3% to 100.0% gate recovery vs. 57.1% to 82.1% in prob space; p_adj = 0.004).

0.27

APIFlow-Bench: Measuring Whether Agents Survive Long, Dependent API Workflows

cs.AI cs.LGcs.SE ·2026-08-29 ·Zelin Wan, Arash Nourian, Xiaoxiao Li, Nihar Nandan, Kamalakannan Nandagopal
benchmarkcode/data similarity 0.65
Abstract · 15 pages, 8 figures. Under review at the NeurIPS 2026 Workshop on Evaluation of Interactive Agents (IAEval). Harness, frozen task bank, and …

Tool-using agents are commonly evaluated by a single bit: whether an end-to-end workflow completed. This metric fails to distinguish failures that matter in production, such as expired credentials, malformed payloads, or correct execution followed by incorrect final delivery. We introduce APIFlow-Bench, a fully auditable benchmark for long-horizon, dependent REST-API workflows that decomposes performance into seven engineering capabilities and requires agents to produce answers supported by the actual call path. We generate synthetic API worlds forward, subtask by subtask; each subtask is admitted only after a zero-LLM self-test triad verifies its grader and an oracle establishes solvability, and an adversarial audit identified and fixed six grader exploits. Grading is deterministic and provenance-sensitive: a state check traces a mock-minted canary through the API data flow to the response the answer must originate from, and a typed answer card is verified field by field. We release all answer keys and 44,362 unredacted execution transcripts. Across 19 frontier and open-weight models under one neutral scaffold, we find: (1) longer dependency chains degrade success, from 93% on individual subtasks to 74% on clean 20-subtask chains and 61% when including the 8% of chain trials that a model-consensus screen flags as passed by no model; (2) reliability separates models more than best-case capability, with best-of-five spanning seven points but all-five-of-five reliability spanning 44 points; (3) the independent-error account of compounding failure does not fit the data: pass rates on 20-subtask chains are 33 percentage points above the product of subtask-level rates, and on the clean slice 77% of failing runs reached the correct final state and failed only at delivery.

0.53