CS Paper Radar

Plug-and-Play Multi-Licensee Robust Watermarking for Mixture-of-Experts LLMs

Jianshuo Liu, Kang G. Shin, Haining Wang

cs.CR ·submitted 2026-11-18 04:45 UTC · v1
benchmark

Abstract

Mixture-of-Experts (MoE) architectures enable high-capacity large language models (LLMs) with high sparse computation, making them attractive for deployment on private hardware. However, distributing model weights exposes valuable intellectual property (IP) to leakage and unauthorized redistribution, creating an urgent need to both verify model ownership and trace leaked copies to individual licensees. Existing model-watermarking techniques are ill-suited for meeting this need because they are fragile under MoE's dynamic routing or require costly retraining to embed a distinct watermark for each licensee, thus making scalable distribution difficult. To address these challenges, we present PPMoE, the first plug-and-play multi-licensee watermarking framework for MoE LLMs. Rather than training a separate model per user, PPMoE trains a compact set of mergeable incremental weight modules that steer the MoE router toward distinct, mutually compatible routing patterns. During model distribution, the model owner selectively merges unique combinations of these pre-trained, mutually compatible modules into the base model on the fly, embedding a licensee-specific watermark without additional training. PPMoE supports reliable multi-licensee attribution under both white- and black-box threat models. In the white-box setting, attribution is done by inspecting triggered routing decisions or the merged weight modules responsible for these decisions, whereas in the black-box setting, attribution relies on analyzing output token likelihood ratios. Extensive experiments across five state-of-the-art MoE models, seven benchmark datasets, and varying number of licensees show that PPMoE achieves very high attribution accuracy while preserving base model utility and maintaining strong robustness to various adaptive watermark-removal attacks.

Closest papers

Nearest neighbours by meaning across the whole index.

Watermarkability: Measuring and Improving Language-Model Compatibility with Logit-Based Watermarks

cs.CL cs.CR ·2026-10-16 ·Zewei Deng, Mohamed Seif, Liyan Xie
code/data similarity 0.67
Abstract · 16 pages, 2 figures, 10 tables. Code available at https://github.com/zeweid1221/LLM_Watermarkability

Language-model watermarks can help identify model-generated text, encode provenance information, and detect post-generation edits. A broad class of logit-based watermarks increases the logits of a designated token set at each decoding step. Because a base model's native probability mass on this set varies across contexts, models, and watermark rules, the same logit bias can yield different adherence rates and increases in perplexity (PPL). We introduce watermarkability to characterize the compatibility between a model and the token sets requested by a watermark. At a target mass $τ$, we quantify this property through a watermarkability cost: the expected minimum KL divergence from the base-model distribution required to assign probability at least $τ$ to the requested set. A lower cost means that the model is more watermarkable for that request distribution. To reduce this cost, we propose a two-stage method. In the first stage, we train a target-set-conditioned low-rank adapter on unions drawn from a 16-way vocabulary partition to increase the requested set's probability mass. In the second stage, a closed-form controller applies the smallest remaining uniform logit bias needed to reach the target mass. We evaluate the method on six watermark schedules using OPT-125M and Qwen3-8B. Across the 12 model-schedule pairs, the pre-adaptation watermarkability cost correlates with the log-PPL increase under a near-hard fixed-bias baseline (pooled Pearson $r=0.929$). For every schedule, at least 99.80% of sampled tokens fall in the target set requested at their decoding step. At this reliability, the adapter and controller reduce average per-schedule prompt-conditional PPL by 46.3% on OPT-125M and 5.2% on Qwen3-8B relative to the fixed-bias baseline.

0.26

WoE Wrote It? Watermarking Mixture-of-Experts LLMs for Black-Box Text Provenance

cs.CR ·2026-08-29 ·Jona te Lintelo, Lichao Wu, Stjepan Picek
similarity 0.66
Abstract

Large Language Model (LLM) watermarks provide a mechanism for text provenance, enabling model owners to identify machine-generated content and attribute it to a specific watermarked model. However, current LLM watermarking approaches predominantly rely on inference-time sampler methods and focus their analysis on dense models. Inference-time methods are only effective when the text is explicitly generated via the model owner's controlled API; they fail in a post-compromise scenario. An adversary who steals or leaks the model weights gains complete control over inference and can simply run an unmodified sampler, bypassing the watermark and preventing post-theft attribution. In this work, we introduce Watermarking of Experts (WoE), a novel black-box text provenance method that leverages the unique structural properties of sparse Mixture-of-Experts (MoE) models. WoE biases the vocabulary of specific experts and shifts the watermark signal embedding away from unenforceable inference wrappers. This approach ensures the watermark remains intrinsic to the model parameters, enabling defenders to attribute text generated by stolen weights, leaked checkpoints, and secondary dense models distilled from the stolen architecture without needing access to the adversary's deployment or weights. We evaluate WoE across eight MoE models, demonstrating successful watermark detection from suspect text, achieving an average true positive rate of 90.1% at a 1% false positive rate, reaching up to 94.9%, while largely preserving general model utility. Furthermore, WoE remains detectable under adversarial supervised fine-tuning, model extraction, and output-level paraphrasing, forcing malicious actors into a trade-off in which weakening the attribution signal requires additional model adaptation or text-rewriting operations, or compromises the utility of the resulting output.

0.10

SAME: Semantic Watermark Embedding for Open-Source Large Pretrained Models

cs.LG ·2026-11-11 ·Xinyao Song, Jialiang Shen, Hui Kang, Huajie Chen, Ruiqi Liu +5 more
code/data similarity 0.66
Abstract

Open-source large pretrained models (LPMs) have become increasingly valuable assets for model development and downstream applications, but their accessibility also facilitates unauthorised copying and modification. To protect their intellectual property (IP), existing approaches typically embed tailored trigger-response patterns as watermarks for ownership verification. However, such trigger designs need to be tailored to task-specific data that constrain their transferability across downstream tasks, and are not durable under subsequent benign fine-tuning. To address these limitations, we propose SemAntic waterMark Embedding (SAME), which introduces a semantically meaningful private vocabulary that enables flexible composition of adaptive and persistent watermarks. Specifically, SAME introduces a fixed private vocabulary that assigns each native token a private embedding counterpart, while aligning the semantic expressiveness of the two vocabularies within the model's representation space. This alignment allows private tokens to be flexibly composed into semantic triggers that elicit meaningful responses, enabling adaptive watermarks beyond predefined trigger-response pairs. By grounding the private vocabulary in the model's native semantics, SAME couples the watermark with its evolving semantic capabilities, allowing this connection to be preserved throughout downstream fine-tuning. Extensive experiments demonstrate that SAME provides reliable and robust watermarks across LPMs of varying scales and architectures, enabling effective ownership verification for IP protection.

0.23

OpenStamp: A Watermark for Open-Source Language Models

cs.CL cs.AIcs.LG ·2026-08-28 ·Miroojin Bakshi, Saksham Rastogi, Danish Pruthi
publishedcode/data similarity 0.65
Abstract · Published at COLM 2026

With the growing prevalence of large language model (LLM) generated content, watermarking is considered a promising approach for attributing text to LLMs and distinguishing it from human-written content. A prominent class of techniques embeds subtle but detectable signals in generated text by modifying token sampling probabilities. However, such methods are unsuitable for open-source models, where users have white-box access and can easily disable watermarking during inference. In this work, we introduce OpenStamp, a watermarking technique that encodes the watermarking logic directly into the model weights by modifying only the final projection, or unembedding, layer. Through experiments across two models, we show that OpenStamp achieves superior detection performance, with minimal degradation in model capabilities compared to prior methods. The implanted watermark is explicitly designed, and empirically confirmed, to be more robust to paraphrasing attacks and harder to scrub off through post-hoc fine-tuning than prior open-source watermarks. To enable developers to watermark their models, we release our code alongside watermarked versions of 4 popular open-source models.

0.36

Decision Uncertainty-Oriented Watermarking (DUOW): Securing Multi-Agent Routing Against Policy Cloning

cs.CR ·2026-10-10 ·Ziming Zhang, Li Li, Guorui Feng, Hanzhou Wu, Xinpeng Zhang +1 more
similarity 0.65
Abstract

Routing policies in Multi-Agent Systems (MAS) represent valuable intellectual property, yet their exposure through execution logs creates significant risks of unauthorized policy cloning. Existing agent watermarking methods protect intellectual property by modifying internal generation processes, such as perturbing token sampling distributions or injecting structural mutations into action trajectories, which may compromise task fidelity or suffer from limited transferability across heterogeneous platforms. To address these limitations, we propose Decision Uncertainty-Oriented Watermarking (DUOW), a lightweight framework that operates at the external decision layer without modifying model parameters or generated outputs. DUOW introduces a plug-and-play scoring mechanism that evaluates candidate agents from semantic and operational perspectives, identifies functionally redundant routing decisions, and selectively redirects choices among equivalent alternatives to embed ownership signals. By coupling watermark behaviors with utility-preserving routing decisions, DUOW enables cloned policies to retain ownership signals during policy imitation while increasing the difficulty of watermark removal. A confidence-driven adaptive threshold dynamically determines embedding opportunities, balancing watermark capacity and task fidelity under diverse routing contexts. Extensive evaluations on four multi-agent routing benchmarks demonstrate that DUOW preserves high task utility, achieves reliable ownership verification under up to 90% log erasure, and introduces negligible computational overhead, with an average per-step latency of 0.043 ms and total embedding overhead below 0.54 ms.

0.32

Adaptive Co-Serving LLM Watermarking on Modern Inference Engines

cs.CR ·2026-10-02 ·Kieu Dang, Phung Lai, Ching-Yun Ko, Pin-Yu Chen
similarity 0.64
Abstract

Large language model (LLM) watermarking is important for ownership verification and intellectual property protection. However, existing approaches focus on algorithmic design while treating LLM inference engines as separate components. This separation often introduces auxiliary models or external tools, increasing latency and memory overhead while limiting the use of modern inference optimizations. As a result, a deployment gap remains: practical watermarking must preserve utility, detectability, and robustness while minimizing serving overhead. To bridge this gap, we propose SWIFT, a framework that co-designs LLM watermarking with modern inference infrastructure. SWIFT (i) integrates text generation and watermark construction within a shared LLM backend to reduce re-computation, improve cache reuse, and simplify system complexity; (ii) uses instruction-guided candidate generation with semantic protection to preserve factual consistency and meaning; (iii) executes watermarking concurrently with generation through asynchronous co-serving and adaptive scheduling to reduce latency and improve resource utilization; and (iv) leverages vLLM optimizations to enhance serving efficiency. Extensive experiments across domains and tasks show that SWIFT achieves strong utility, detectability, robustness, and downstream performance while substantially improving serving efficiency. Specifically, it achieves the highest utility score (4.87), 99.65% detection accuracy compared with 76.65% for the low-latency baseline SynthID, and up to 97.7% detection under watermark removal attacks, with 5.9x lower latency than the highest-utility baseline, SafeSeal. These results demonstrate the benefits of co-designing watermarking with inference infrastructure for practical LLM serving.

0.13

SkillMarker: A KGW-Based Watermarking Method for Agent Skill Ownership Verification

cs.CR cs.AI ·2026-10-11 ·Alan Tang, Lifeng Wang, Gang Wu, Tao Zhang, Yantao Jia +1 more
code/data similarity 0.63
Abstract · Project homepage: https://fuxi-ailabs.github.io/SkillMarker/

Agent skills encode operational procedures and domain expertise as reusable knowledge assets, yet their distribution in plaintext exposes them to unauthorized copying and leakage. However, practical mechanisms for skill ownership verification and leakage tracing remain underdeveloped. Watermark embedding before distribution offers a potential solution, but structured, compact skill instructions impose stringent requirements for factual and procedural accuracy. To address these challenges, we introduce SkillMarker, a KGW-based watermarking method that jointly targets functionality preservation and strong watermark embedding. SkillMarker unifies quality-constrained rewriting with signal-oriented candidate search. The former separates essential knowledge from flexible expression to preserve instruction fidelity; the latter explores alternative expressions to obtain a stronger watermark signal under preservation constraints. We release an open-source toolkit that integrates watermark embedding, quality auditing, and detection to support the protection of agent skills. SkillMarker targets ownership verification and leakage tracing in distributed skill ecosystems, where content may be copied in whole or in part, or incorporated into other skills. Across four datasets, SkillMarker negligibly affects downstream task performance on average. In our copy-paste tests, window-based detection consistently identifies copied passages containing as few as 1,536 consecutive tokens. Even after merging with one or three background skills, SkillMarker retains watermark detection rates of 72.2--77.8\%.

0.39

Watermarking for Byzantine Robust LLM Training

cs.CR ·2026-10-09 ·Leixu Huang, Zedian Shao, Teodora Baluta
similarity 0.63
Abstract · 19 pages, including appendix and references

LLM watermarking has emerged as a promising approach for detecting downstream model misuse in centralized settings, but extending it to federated learning (FL) presents unique challenges. In particular, FL systems often rely on strong robust aggregators to defend against poisoning attacks. We empirically show that a malicious server can exploit these same aggregators as effective watermark-removal mechanisms during training, causing representative existing LLM watermarking methods to fail detection in the final model. To address this limitation, we formalize the problem as a constrained optimization task and propose CamoMark, an aggregator-agnostic watermarking scheme for embedding robust watermarks in FL. CamoMark optimizes watermarked updates to become inliers, maintaining bounded deviation from the distribution of clean updates, while preserving sufficient watermark signal for reliable detection. Evaluated on Qwen fine-tuned on the Alpaca dataset, CamoMark achieves a detection confidence of $p$-value $= 1.82\times10^{-2}$ using only 3.3% watermarked data. The watermark also remains robust to a range of post-training removal attacks, including continued fine-tuning on clean data, pruning, and model distillation. At higher watermark ratios, detection confidence improves substantially, reaching $p$-value $= 5.11\times10^{-11}$ at a 13.3% watermark ratio.

0.29