CS Paper Radar

NetInspector: Measuring and Improving LLM Capabilities for Reliable Intent-Based Networking Policy Generation

Yuxuan Zhang, Hongxin Hu, Guofei Gu

cs.CR cs.NI ·submitted 2026-09-17 21:26 UTC · v1
benchmark

Abstract

Modern networks are large in scale and heterogeneous in configuration, making manual policy management increasingly impractical. Intent-Based Networking (IBN) addresses this by automating the translation of high-level operator goals into low-level network configurations. Yet existing IBN systems rely on static heuristics and fixed-feature classifiers that generalize poorly to distribution shifts such as new service definitions or evolving phrasing in operator requests. Large Language Models (LLMs), with strong reasoning and translation capabilities demonstrated across many domains, are a natural candidate for IBN policy generation. However, it is unclear whether LLMs can be reliably applied to this task, nor whether their use mitigates or worsens the underlying security risk. In this work, we show that while fine-tuned LLMs excel at intent translation, they exhibit false negative rates when checking whether a proposed intent violates an existing security policy. The root cause is not a lack of logical reasoning capability, but LLMs lack of persistent grounding in network topology and group hierarchy. Motivated by this finding, we introduce NetInspector, a three-layer agentic framework that enforces a verify-then-act protocol, decoupling information retrieval from reasoning so that the LLM focuses on symbolic reasoning while every policy decision is grounded in verifiable network facts retrieved from a live Environment Layer before approval. On NetInspector-Bench, a 2,224-sample synthetic benchmark spanning campus, enterprise, and WAN topologies, NetInspector reduces FNR by over 30\% relative to ungrounded baselines and remains robust under linguistic distribution shifts.

Closest papers

Nearest neighbours by meaning across the whole index.

How Far Can Lightweight LLMs Go in Intent-to-SLO Translation?

cs.NI cs.CL ·2026-11-05 ·Sabbir Ahmed, Hazem Abbas, Salimur Choudhury
code/data similarity 0.71
Abstract · Under review at ICC 2027. Source code and data: https://github.com/ggck43/intent2slo. (6 pages. 7 figures, 3 tables)

Intent-based networking (IBN) promises QoS provisioning in which the network derives the configuration from operator-stated goals. Its first step, translating a natural-language intent into enforceable service-level objectives (SLOs) grounded in a live snapshot of the network state, increasingly relies on large language models (LLMs). These large models add cost and energy to the management loop and typically require multi-GPU, cloud-hosted infrastructure. Can lightweight open LLMs take over this step dependably? We answer this question with a five-dimensional evaluation of ten LLMs with <10B parameters from five families, under four prompting setups, on 716 placement intents. We evaluate translation performance, intent validation, hallucination, energy impact, and robustness to diverse phrasing. The best configurations translate intents with a strict F1 of 0.943 and reject up to 191 of 195 invalid intents. With few-shot reasoning, the strongest models hallucinate unsupported QoS requirements in only about one response in a hundred, whereas the zero-shot setup struggles. A translated intent costs 0.044-0.148 Wh of GPU energy without reasoning, and added energy cost for reasoning pays off for the weaker models. Furthermore, we show that realistic rephrasing lowers the rejection of invalid intents in 15 of 16 configurations for the best-performing models, leaving robust intent validation as a key open challenge for reliable IBN. Our code and data are available: https://github.com/ggck43/intent2slo.

0.36

Intent2Tc: Automated Intent-to-Traffic Control Translation with Language Models

cs.NI cs.AIcs.CL ·2026-09-25 ·Andrea Masini, Sudipta Acharya, Paolo Bellavista, Luca Foschini, Burak Kantarci
publishedcode/data similarity 0.69
Abstract · 6 pages, 6 figures, Accepted to IEEE Conference on Future Communications and Networks (FCN) 2026

Automated and highly usable Quality-of-Service (QoS) enforcement requires translating high-level service intents into deployable traffic-management policies. Although intent-based networking (IBN) has simplified policy specification, bridging the gap between business-level intents and executable network configurations remains complex, error-prone, and difficult to automate. This paper presents Intent2Tc, a closed-loop language-model-driven framework that translates business-level traffic-shaping intents into declarative sub-intents and subsequently into validated, executable Linux traffic control (tc) configurations. The framework integrates an Active Queue Management (AQM)-based digital twin (DT) semantic model, automated metadata extraction, critique-driven refinement, and Retrieval-Augmented Generation (RAG)-based knowledge reuse to improve semantic consistency and configuration reliability. We evaluate multiple open-source large language models (LLMs) and small language models (SLMs), together with Claude Sonnet-4.6, on 100 Request for Comments (RFC) 9315-compliant traffic-shaping intents. Across both translation stages, Intent2Tc achieves high semantic fidelity, configuration accuracy, and deployment readiness, with Claude Sonnet-4.6 reaching 0.98 semantic similarity, 1.0 semantic unit coverage, and 0.045 normalized edit distance. Furthermore, RAG reduces token consumption and inference latency while enabling compact models such as Phi-4-mini to approach the performance of substantially larger models. Linux tc serves as the target configuration platform, demonstrating the practical applicability of the proposed framework.

0.53

Semantic Routing Framework for Multi-Domain Telecom Intent-Based Networking

cs.NI ·2026-10-17 ·Marwa Tageldien, Bassant Selim, Lokman Sboui
benchmark similarity 0.67
Abstract

Intent-Based Networking (IBN) is a key enabler of autonomous network management in next-generation networks, as it allows operators to drive the network through high-level intents instead of low-level configuration commands. In this paper, we propose a semantic routing framework for telecom intent classification and translation across the radio access networks (RANs), transport, 5G core, and cloud/edge domains. This framework operates locally to preserve intent privacy and avoid the latency and cost of cloud-based Large Language Model (LLM) solutions. The proposed framework is composed of intent classification and translation. To accurately classify the intent, we propose fine-tuning sentence-transformer models using contrastive learning over a 3GPP TS 28.312-aligned taxonomy of five top-level routes and 22 branches, supported by a telecom-specific codebook of 1,100 intents. In addition, we develop a branchaware, template-driven rule-based module for intent translation that enforces deterministic, hallucination-free parameter extraction and explicit missing-field detection, ensuring a structured and verifiable output pipeline. The numerical results show that the best fine-tuned encoder achieves up to 99.02%/94.61% route/branch classification accuracy, outperforming both base encoders and a commercial embedding baseline. The proposed translator achieves up to 96.24% translation accuracy, compared to 85.37% for a cloud-based LLM benchmark. Moreover, since the proposed pipeline runs entirely on local hardware, an average translation latency of 4.81 ms is recorded against 4,163.58 ms for the cloud-based benchmark, i.e., a speedup of approximately 866x.

0.13

Multi-Agentic AI for Intent-Driven and Conflict-Aware rApp Policy Orchestration in Open RAN

eess.SY ·2026-10-17 ·Haiyuan Li, Yulei Wu, Luyu Qi, Dimitra Simeonidou
similarity 0.66
Abstract

Open radio access network (O-RAN) enables a multi-vendor ecosystem that accelerates xApp development for near-real-time (RT) network management. However, this openness shifts a combinatorial coordination burden to the Non-RT layer, where rApp policies compose xApps from a growing catalog to realize operator intents. Coexisting intents further compound this challenge, as conflicts may arise among xApp actions, rApp policies, and their cross-layer interactions. To maximize intent admission while maintaining key performance indicator (KPI) compliance across incoming and incumbent intents, we propose a large language model-based Multi-Agentic AI framework. A Reasoning Agent screens candidate xApps against KPI contracts and produces a joint intent admission and xApp assignment decision. On post-deployment KPI violations, a Diagnosis Agent identifies the root causes of failure, labels the conflict mechanism, and guides the Reasoning Agent in subsequent attempts. Experimental results show that the framework based on GPT-OSS 120B achieves 28.6% more KPI-compliant cases and admits over 15* as many incoming intents as state-of-the-art baselines. Under the Qwen 72B variant, Diagnosis increases compliant cases by 50.0%, quadruples admitted intents, and reduces deployment attempts by 32.6%. A live deployment conforming to O-RAN specifications further validates the feasibility of the proposed solution for conflict-aware rApp policy orchestration in real networks.

0.13

RFCLLM: Evaluating LLMs' Reasoning Ability of Network Protocol State Machines

cs.CL ·2026-09-11 ·Anqi Chen, Dan Goldwasser, Cristina Nita-Rotaru
top venue similarity 0.64
Abstract · Accepted at EMNLP 2026 (Findings)

Mapping textual specifications into formal representations is essential for ensuring the correctness of protocol designs and implementations. LLM-generated mappings, used for networking security or testing, are assumed to capture a perfect understanding of the specification, which may not hold in practice. The goal of this paper is to assess the extent to which LLMs can interpret the specification correctly. We examine the degree to which an LLM's implicit representation of a finite-state transition system-defined via natural language descriptions-aligns with a manually generated ground-truth model. We designed 4 tasks and 1482 task queries for 16 protocols. We evaluated different judge biases, observed the inherent difficulty gaps between tasks, looked into the effect of 4 context types, and the influence of protocol characteristics. Our work contributes to a step toward verifying whether LLMs can really be trusted in FSM (Finite State Machine) reasoning of protocol specifications.

0.37

NetConfArena: An Executable Benchmark for LLM Agents in Closed-Loop Network Configuration

cs.NI cs.AI ·2026-08-24 ·Chang Liu, Xiaohui Xie, Xinyi Chen, Yong Cui
benchmark similarity 0.64
Abstract

Large language model (LLM) agents are increasingly attractive for automating network configuration, yet their reliability and failure patterns are poorly understood. An essential prerequisite is to assess such agents in a realistic but risk-free environment. Existing benchmarks, however, fall short: they often treat configuration as static command generation or rely on overly simplified settings. Such evaluations understate the core challenges of network configuration, where correctness requires reasoning about protocol complexity and topology dependence. We present NetConfArena, an executable benchmark for evaluating LLM agents in closed-loop network configuration. NetConfArena places agents in emulated multi-device networks, provides a standardized and compact action interface for task execution, and evaluates the resulting network behavior with hidden task-specific executable test cases. The benchmark relies on an LLM-assisted, emulation-grounded pipeline, which converts human-oriented network materials into reusable parameterized task templates. We evaluate representative LLM agents on 480 task instances instantiated from 96 protocol-focused task templates, yielding 3840 execution trajectories, and show that failures are not limited to command errors. The failures also reveal gaps in task-specification adherence and robust planning and execution. These findings suggest two future directions: using validated trajectories as supervision signals to improve foundation models, and designing harness mechanisms that make agent execution more reliable and accountable.

0.38

Staying on the Attack Path: Structured State for Long-Horizon Automated Penetration Testing

cs.CR cs.AI ·2026-09-07 ·Weizhe Wang, Yitong Zhang, Yao Zhang, Xiaoqiang Di, Zhigang Li +2 more
benchmark similarity 0.63
Abstract

Large language model (LLM) based agents are increasingly applied to cybersecurity tasks such as vulnerability discovery and automated penetration testing. On long-horizon security tasks, however, such agents remain limited by context forgetting and intent drift: early critical facts and causal reasoning chains are lost over extended interactions, and the agent falls into aimless, repetitive exploration. This paper proposes Intentest, an intent-graph-guided automated penetration testing agent that externalizes long-horizon state from the LLM's context window onto a persistent fact-intent directed acyclic graph (DAG), thereby substantially reducing invalid transitions. We evaluate Intentest on automated penetration testing of web applications, a representative long-tail task in cybersecurity. In the DAG, verified network states are stored as immutable fact nodes, and exploration directions are constrained as intent edges bounded by predecessor facts. The system adopts a three-layer architecture, in which the fact-intent mapping layer maintains the global state, the task scheduling and allocation layer ensures execution stability through two-phase degradation recovery and multi-dimensional adaptive load balancing, and the intent retrieval and prediction layer provides tactical priors through a top-down five-stage filtering algorithm. On a benchmark of real CTF challenges covering more than ten vulnerability types across three difficulty levels, Intentest achieves an overall success rate of 88.2% and a success rate of 75.0% on hard tasks, improving over the baseline by approximately 44 and 50 percentage points. Ablation experiments further show that the intent retrieval and prediction reduce the average number of rounds on successful medium and hard tasks by about 33% and 48%, respectively, without changing the set of solvable tasks.

0.21

TopoIntent: Compiling Security Intent into Executable, Compliance-Checked Network Topologies

cs.AI cs.CRcs.NI ·2026-08-13 ·Xiaokang Qu, Jianliang Ma, Zao Fan, Tianshu Chu, Tianlong Fan +1 more
benchmark similarity 0.63
Abstract

Enterprise security topology design requires translating business intent, regulatory requirements, and risk assumptions into zones, boundary devices, inter-zone paths, and access-control policies. Existing NetOps automation tools mainly operate after this design is fixed, providing limited support for generating structured security topologies from underspecified natural-language requirements. We present TopoIntent, a system that compiles security intent into executable, compliance-checked network topologies. It uses a schema contract to constrain generation, retrieves reference architectures from a curated template library via dense-vector search, and applies staged fusion for intent-template alignment and security completion. The generated topology is checked against CIS Controls v8.1.2 safeguards visible at the topology layer, while unresolved cases are marked for manual review. Structural gaps are repaired through additive schema-preserving edits. The final topology is exported to Mininet scripts with kernel-level iptables ACLs, enabling executable reachability and allow/deny tests. Because no public benchmark exists for this requirement-to-topology task, we construct an evaluation set from reference security architecture diagrams. The retrieval set contains 22 templates and 44 synthetic intents across five scenarios, while the held-out set contains 7 templates and 14 intents from finance and government scenarios excluded from retrieval. On the held-out set, additive repair improves topology-visible CIS satisfaction from 0.78 to 1.00 in fewer than 1.5 rounds on average, and one feedback round raises the post-ACL policy pass rate from 0.78 to 0.88.

0.04