CS Paper Radar

VX-One: Typed, Calibrated Decision Heads for Real-Time Perimeter Triage

Juan S. Santillana

cs.CR cs.CLcs.LGcs.NI ·submitted 2026-10-21 17:39 UTC · v2
benchmark

Abstract

Perimeter triage faces a latency/semantics trade-off: signature engines are fast and format-safe but blind to unseen attacks, while generative models are semantic but slow and unparseable. We present VX-One, a small multi-head network (100K--10M parameters) with calibrated typed outputs: a binary gate, a closed-set choice with an explicit unknown option where the benchmark supports it (and abstention elsewhere), and an ordinal score. Heads are temperature-calibrated, with 0.7--1.7 ms single-event CPU p50 latency. We evaluate UNSW-NB15, CIC-IDS2017, and EMBER2024 against calibrated XGBoost, Random Forest, MLP, and Kitsune baselines, and measure a Suricata-to-VX-One cascade over five days of CIC-IDS2017. Results are deliberately double-sided: on UNSW the gate ties the best baseline (0.9837+/-0.0002 versus MLP 0.9835+/-0.0013 across three seeds), while VX-One wins temporal CIC detection (0.734+/-0.022 versus 0.639+/-0.027) but loses multi-class attribution to XGBoost. Under leave-one-family-out, the gate detects all nine UNSW families but drops below chance on CIC recon; CIC trees transfer better. Suricata resolves only about 0.1% of attacks on flow reconstructions and 0.2--2.2% on payload-bearing PCAPs; the strictly by-day Phase 2 recovers only the in-scope DoS/DDoS share on the wave day (recall 0.34, precision 0.97). Calibration collapses under temporal drift (ECE 0.04->0.40) for every method while ranking partly survives. We contribute a measured account of where typed calibrated decisions help perimeter triage, and where they fail.

Comment
14 pages, 7 tables, 1 figure

Closest papers

Nearest neighbours by meaning across the whole index.

XLOG: A CUDA-Native Engine for Neurosymbolic Integration

cs.AI ·2026-09-23 ·Levi Dubrovin, Nikita Pospelov, Kirill Sabitov
benchmark similarity 0.62
Abstract · 31 pages

xlog is a CUDA-native logic programming engine integrating neural perception with deterministic Datalog, probabilistic inference, and epistemic world views through a typed frontend and provider-owned CUDA runtime. Its reasoning modes share device data planes, but their execution boundaries differ: ordinary Datalog and exact inference are host-orchestrated, while certified resident recursive and Monte Carlo sampled cores record zero tracked host-device transfers before a bounded terminal receipt. The probabilistic path supports end-to-end gradients through GPU knowledge compilation from provenance to CNF to Decision-DNNF, exact weighted model counting, and backward gradients. A final smoothed circuit is certified against its source formula before caching or evaluation. Circuit caching yields a 2.74x MNIST-addition training speedup; a worst-case-optimal join subsystem yields a 27.96x geometric-mean gain over xlog's binary-join baseline. MNIST-addition accuracy matches Scallop's (0.9561 versus 0.9468), but no per-epoch speed claim is made because baseline epoch time varies with CPU quota. In five hub-skewed triangle-counting cases, the Souffle-to-fused-xlog execution-time ratio rises from 0.88x at 150k edges, where Souffle is faster, to 5.54x at 1.2M; fused peak device allocations are 85-1,033 MB versus 3,287-44,979 MB for the materializing arm. Exact inference is correctness-equivalent to but slower than ProbLog2. On a public video benchmark, a proximity predicate trained only through symbolic credit replaces hand-set geometry at unchanged held-out accuracy; within Event-Calculus rule search it fails ten-fold cross-validation and does not transfer on a leak-free split. On a maritime corpus, weighted clauses beat crisp selection by 0.065 F1, with the result reproduced by one chronological training pass.

0.32

Cross-Primitive Consistency in Jev

cs.CV cs.CL ·2026-10-28 ·Xu Liu, Guikun Chen, Kanzhi Wu, Wenguan Wang
benchmarkcode/data similarity 0.61
Abstract

Jev enables efficient decision-making with large language models (LLMs) through typed querying primitives, allowing the same semantic state to be expressed in different formats, such as binary and multiple-choice queries. However, the consistency of decisions across primitives has not been systematically evaluated. This work introduces PrimaCons, a four-axis benchmark for assessing cross-primitive consistency while preserving the semantics of the query. Evaluations across three Jev sizes reveal consistency violations across all four axes. Our analysis shows that these violations follow systematic patterns and that cross-primitive discrepancies predict decision failure with a ROC-AUC of 0.91. To mitigate these inconsistencies, we propose the Cross-Primitive Residual Calibrator (CPRC), which attaches a lightweight residual network to a frozen backbone and jointly optimizes predictive accuracy and cross-primitive consistency. Under a leave-one-domain-out protocol, CPRC reduces cross-primitive inconsistency by 58.57% and improves accuracy by +2.97%., demonstrating its effectiveness in both in-domain and out-of-domain settings. Our findings establish cross-primitive consistency as a key dimension for evaluating Jev and provide a practical approach to improving the reliability of its decisions. Code and benchmark is available at https://github.com/67L1/PrimaCons-CRPR.

0.48

COGNIT-Guard: Calibrated Standalone Direct-Decision Guardrails with Heterogeneous CPU-NPU Confidence Cascading under Explicit Latency and False-Positive Constraints

cs.CR cs.LG ·2026-09-27 ·Hao Chen
similarity 0.60
Abstract · 7 pages, 2 figures, 5 tables, IEEE double-column format

When must a foundation-model safety gateway generate tokens, and when should it directly output a calibrated decision? We study calibrated standalone direct-decision foundation models for real-time pre-ingestion safety guardrails, jointly addressing probability calibration, dual-use false-positive control, and heterogeneous CPU-NPU routing under explicit latency SLOs. Pre-ingestion guardrails must screen prompts prior to target-LLM prefill with low false alarms on benign compliance inquiries; however, shallow classifiers are brittle to phrasing shifts, hidden-state probes require coupling to a target LLM, and generative guards incur high decoding latency and dual-use false positives. We present COGNIT-Guard, coupling a validation-calibrated CPU fast gatekeeper with confidence-gated escalation to an NPU-resident 322M bidirectional direct-decision model (Laya-322M) under an asymmetric false-positive penalty. On the clean unseen DUCS-Bench test split ($N=607$), COGNIT-Guard achieves 98.85% accuracy (McNemar $p = 1.19 \times 10^{-4}$ vs. ML), reduces benign FPR to 0.42% ($1/238$; Fisher's exact $p = 8.23 \times 10^{-4}$ vs. ML), and attains 1.12% ECE and 0.0104 Brier score. On Huawei Ascend 910C NPUs, pure NPU inference runs in 21.77 ms mean latency (45.90 QPS), while the live serial CPU-NPU cascade ($θ^*_{\mathrm{deploy}}=0.70$) achieves 41.63 ms mean latency (P50: 39.47 ms, 99.23% accuracy, 0.00% FPR). Evaluation on SafetyBench-ZH ($N=2,100$) and comparison against a bi-encoder direct-decision baseline (CLM-8B) disentangle in-domain gains, OOD alignment tax (60.33% $\to$ 56.81% on Laya; 55.10% on domain CLM-8B), and experience replay recovery, restoring OOD accuracy to 64.10%-65.05% and reaching 99.67%-99.84% in-domain accuracy with 0.00%-0.42% FPR.

0.18

DFIP: Decision-Focused Inference Path for System One Models

cs.PF ·2026-10-27 ·Ismail Pelaseyed, Alan Zabihi
code/data similarity 0.59
Abstract · 11 pages, 1 figure. Code and reproducible results: https://github.com/superagent-ai/dfip

System One (decision) models return small structured answers, but can retain host dispatch, GPU launch overhead and full-vocabulary projection. We present DFIP (Decision-Focused Inference Path), an empirical systems study of established inference techniques in PPLX-Decider, Liquid d1-3B, Clef-Flash and GLiNER2.5-Decide. Selective CUDA graph replay, candidate-token projection and schema-compatible length-aware batching are applied at compatible model boundaries. We compare resident requests with stronger native baselines, audit exposed answers and measure changing arrivals. With optional native Qwen kernels installed, short Boolean resident speedups over strengthened native are 2.47x for Decider across five containers and 1.45x for Clef across four of five planned containers; Clef's fifth fails in native preflight. Liquid is effectively tied with released compilation; replay supplies most of its eager-baseline gain, while projection alone adds little full-request benefit. In a one-worker GLiNER follow-up, length-aware batching is 1.40x faster than compiled native FIFO batching on a diverse resident trace, but has worse arrival p95. Decider and Clef audits cover 3,024 decisions per model and preserve all 1,112 returned choices, while Clef changes scores, probabilities and serving confidence thresholds. Diverse external arrivals reverse fixed-shape replay gains and expose overload losses and a replay-path CUDA error. The results identify workload-specific acceleration and its limits, rather than a universal optimizer or production readiness.

0.45

Benchmarking System One decision models against trained classifiers and language models for automated decision gates

cs.LG ·2026-09-29 ·Amir Rafe, Subasish Das
benchmarkhighly cited similarity 0.59
Abstract

Software that hands branching decisions to a model needs a declared option and a probability it can threshold. Typed decision models, also called System One models, return such probabilities without generating text, while supervised classifiers and generative language models are the established alternatives. One harness sends eight decision-model checkpoints from six families, including the hosted model Jev, and four open generative models from three developers the same semantic requests, and scores trained and zero-shot classifiers on the same workflow, intent, emotion and social-science items. With task labels, a fine-tuned DeBERTa-v3-large has the highest observed accuracy on every labeled benchmark but one. Without labels, no decision model is significantly more accurate than Jev on workflows or intents, but Gemma-4-31B matches it on workflows and exceeds it on CLINC-150 at higher cost and latency. Stated probabilities of generative models become unreadable when replies miss the key format, whereas key likelihoods avoid this but can saturate. A guaranteed 5 percent risk leaves Jev 0.528 of the intent decisions, and an in-scope threshold still accepts 0.310 of out-of-scope requests. On typed-decisions, swapping yes and no flips 50.5 answers per hundred for Jev and at least 16.8 for every generative model tested, against at most 6.5 for four fine-tuned decision checkpoints. Exposure to a benchmark's training data explains the largest lead of an open checkpoint, which vanishes on rater-labeled emotions. An intent-trained first stage escalating to Gemma-4-31B reaches that model's accuracy at about Jev's price. The results yield condition-dependent design rules for automated decision gates.

0.48

A Measurement Study of LLM Inference Trade-offs Across Edge Continuum Hardware

cs.DC cs.AIcs.LG ·2026-09-08 ·Maysam Khatib, Moysis Symeonides, Demetris Trihinas, George Pallis, Marios D. Dikaiakos
published similarity 0.58
Abstract · Accepted for publication at WIMS 2026

Large language models (LLMs) are increasingly used as backends for intelligent web services, but serving them across the edge continuum requires balancing quality, latency, model footprint, and energy. This paper presents a controlled measurement study of self-hosted LLM inference across edge and near-edge deployment nodes: an NVIDIA Jetson AGX Orin and a near-edge server with CPU-only and GPU-enabled inference modes. We evaluate multiple open-weight LLMs and quantization variants using a fixed question-answering workload, and compare them against GPT-4o as a cloud-hosted accuracy and latency reference. Our benchmarking pipeline reports accuracy, model footprint, per-token decoding latency, prefill latency, and overall execution energy. The results show that GPU-enabled server execution provides the lowest compute-side latency, while Jetson Orin shows lower measured energy, consistent with its lower platform power under our setup. CPU-only execution is consistently dominated in latency for our workload and shows higher measured energy. We also show that parameter count and downloaded weight-file size alone do not reliably predict observed accuracy or latency. Finally, using Pareto-frontier analysis, we study how deployment decisions may change under possible streamed-token delivery overheads, highlighting that compute-side inference metrics alone can lead to suboptimal placement for latency-sensitive interactive web services.

0.26

HoliBench: A Cross-Platform Benchmarking and Deployment Toolkit for Foundation Models in CPS-IoT Applications

cs.DC cs.LGcs.PF ·2026-09-11 ·Inesh Chakrabarti, Zejun Xiong, Pragya Sharma, Mani Srivastava
code/data similarity 0.58
Abstract · 16 pages, 9 figures, 2 tables. Code: https://github.com/beesfleas/HoliBench

Foundation models, including large language models, vision-language models, and time-series foundation models, are increasingly deployed on embedded and edge platforms for CPS and IoT applications, where energy, latency, and memory are as critical as task accuracy. Existing benchmarking tools evaluate model capability in isolation, reporting accuracy assuming sufficient compute, while hardware profiling tools remain platform-specific and mutually incompatible. As a result, users lack a unified workflow for making deployment decisions across heterogeneous devices. We present HoliBench, a modular benchmarking and deployment toolkit that jointly characterizes accuracy, latency, and energy across platforms from single-board computers to GPU servers. Its platform abstraction layer calibrates cross-device measurement, and the toolkit supports multiple model modalities, inference engines, concurrencies, and existing evaluation harnesses. An interactive interface exposes constraint-aware configuration selection over a design space that is profiled once and reused across studies. Using HoliBench, we characterize 20 models across 7 device types, 3 quantization levels, 8 inference backends, and over 30 tasks, surfacing tradeoffs that existing tools miss: quantization reduces latency only on hardware with low-precision support, accuracy gains show diminishing returns relative to energy, and for autoregressive workloads, average inference power is approximately constant across output lengths. We further find that single-model profiles compose under sequential co-resident execution. In a multi-model CPS deployment, standalone profiles predict combined-pipeline latency and power within 1.2% and 2.5%, enabling deployment exploration without exhaustively profiling every pipeline configuration. We release HoliBench as open-source infrastructure for deployment-aware evaluation of foundation models.

0.36

Fast Models, Slow Evidence: A Paired and Self-Audited Evaluation of System-1 Decision Models for LLM Agent Harnesses

cs.AI cs.CLcs.CRcs.LG ·2026-10-01 ·Jiawei Li
code/datahighly cited similarity 0.58
Abstract · 11 pages, 7 figures. Code and data: https://github.com/David-DL-Space/sys1-eval

Agent harnesses make many small, typed decisions per task: which model to call, which tool to use, whether retrieved text is relevant, whether an input carries an injection. System-1 decision models answer such questions in a single forward pass with class probabilities, promising large cost and latency savings over LLM calls. We present a paired evaluation of an open-weight (Laya) and a hosted (Jev) System-1 model on 11 agent decision points built from 18 public sources: 7,283 base cases plus 6,640 robustness variants, with byte-identical inputs, paired tests, and cross-hardware and cross-day reproducibility checks. Jev is significantly more accurate on 9 of 11 decision points (+10.8 to +46.0 pp). Neither model beats chance on zero-shot model routing, and they tie on RAG relevance gating. Laya changes 30% of its answers when the option order is reversed and degrades sharply with many or similar candidates (31% at 50 nearest-neighbour tools, vs. 98% for Jev on items with a unique correct tool). We also audit our own pipeline. Three analysis errors and one design confound distorted headline deployment claims: an omitted pre-screen cost (reported 23.9% saving, actual 4.3%), gate accuracy reported as end-to-end quality (58% vs. 98%), in-sample thresholds (5% target, up to 17% held-out misses), and a "channel effect" on injection false positives that vanishes with channel-native content. Two other suspected confounds did not change the conclusions. All cases, raw outputs and analysis code are available at https://github.com/David-DL-Space/sys1-eval.

0.59