CS Paper Radar

Maverick: Private and Verifiable LLM Inference Made Practical via Matrix-Vector Multiplication Delegation

Ben Merbaum, Mohammad Amin Raeisi, Wenhao Wang, Charalampos Papamanthou, Katerina Sotiraki, Fan Zhang

cs.CR cs.LG ·submitted 2026-09-09 14:53 UTC · v1
code/data

Abstract

Open-source large language models (LLMs) are increasingly competitive with closed-source models while offering transparency and the ability to run inference without exposing user inputs to a service provider. However, running large-scale models locally requires substantial computational resources. In practice, users may still resort to a third-party provider, giving rise to privacy and correctness concerns. Existing solutions that address these problems often impose substantial server overhead or introduce additional trust assumptions. In this paper, we present Maverick, a novel approach to private and verifiable LLM inference based on a protocol for delegating matrix-vector multiplication, a dominant operation in LLMs. At its core, Maverick provides, to our knowledge, the first information-theoretically sound verification protocol for matrix-vector multiplication delegation with transparent preprocessing, efficient (batch) verification, and virtually no server overhead. We combine this verification primitive with LPN-based pseudorandom masking to provide input privacy. We implement our matrix-vector delegation primitive and use it to build an end-to-end prototype of Maverick, which we evaluate on Qwen3-4B by measuring throughput in tokens per second. We evaluate client configurations with 1-8 threads. With one client thread and a CPU server using up to 128 threads, Maverick achieves throughput gains over local inference of up to 17x when privacy masks are generated online, 45x when they are precomputed, and 44x when only verification is required. With four client threads, the corresponding gains are 13x, 18x, and 17x. When server computation is no longer the bottleneck, client-side microbenchmarks with simulated network delay show speedups of 12x-20x, 34x-135x, and 38x-157x.

Closest papers

Nearest neighbours by meaning across the whole index.

Trusted Model Environment for Private Semantic Computations

cs.CR ·2026-09-24 ·Vasisht Duddu, Xi He
similarity 0.71
Abstract

A private semantic computation primitive enables parties to privately compute over structured and unstructured data that requires understanding its semantics, context, and relationships. Standard cryptographic primitives (e.g., multiparty computation) do not readily support such computation. Generative models are well suited for such tasks but typically process data in plaintext, while cryptographic private inference remains inefficient and difficult to scale. Thus, we need a new primitive for private semantic computation. We introduce trusted model environments (TME), the first such primitive that executes generative models inside trusted execution environments (TEEs) while controlling output leakage. TME is designed to be (i) effective (correctly performs the semantic task); (ii) confidential (protects computation and sensitive inputs); (iii) utility-preserving (retains utility on other tasks); (iv) verifiable (provides tamper-resistant evidence of the computations); (v) efficient (incurs low overhead compared to baseline model computations); and (vi) scalable (supports multiple participating parties). Effectiveness follows from the generative models, while TEEs provide confidential computation. For confidentiality of sensitive inputs, we combine adversarial training to resist verbatim leakage with an information flow control module to suppress semantic leakage. For verifiability, we introduce novel attestations that let parties verify TME operations on their data and queries, along with optimizations (e.g., batching) for efficiency and scalability. We design and evaluate the proof-of-concept for TME across three applications, showing that it meets all the requirements.

0.09

BlindStep: A Secure One-Step Kernel for Privacy-Preserving Collaborative Reasoning

cs.CR ·2026-10-30 ·Xiaoxian Liu, Xingyu Tan, Chen Chen, Xiaoyang Wang
published similarity 0.69
Abstract · Accepted by The Annual Computer Security Applications Conference 2026 (ACSAC, 2026)

A recurring step in two-party collaborative workflows combines a private candidate frontier with a partner's compatible evidence, scores the resulting candidates, and retains only a small frontier. Direct execution can expose intermediate candidates, local scores, or unauthorized evidence provenance. Existing MPC platforms and PSI protocols provide useful building blocks, but they do not define a unified post-alignment boundary for bounded-slot aggregation, fail-closed overflow handling, retained-frontier selection, and downstream disclosure. We present BlindStep, a fixed-shape MPC-online kernel for one such post-alignment step between two semi-honest parties. Given a frontend realizing the bounded-slot contract and an application enforcing the reveal policy $ρ$, BlindStep provides a per-invocation simulation-based guarantee in the $F_{\mathrm{align}}$-hybrid model: the kernel leaks only public setup parameters, fixed execution-shape metadata, a global overflow bit, and $ρ$-authorized Layer-2 disclosures. The guarantee excludes malicious security and multi-invocation composition, and treats frontend leakage and offline preprocessing separately. Within this kernel boundary, we implement six semi2k realizations in MP-SPDZ, crossing two sharing styles and three top-K strategies: H-variants compare over arithmetic shares, while B-variants convert packed ranking values to Boolean shares before selection. Under LAN, B-Queue is 6-7 times faster and uses about 40 times less communication than H-Queue at N = 256 and 2048. Against H-Sort on the same backend, B-Queue achieves speedups of 21 and 30 times and communication reductions of 204 and 290 times at N = 2048 and 8192, respectively. Under the evaluated N = 256, 100 ms RTT WAN profile, H-Sort becomes fastest, showing that round complexity can dominate backend choice for the measured MPC-online kernel.

0.43

Ask in the Crowd: Differentially Private LLM Inference via Dummy-Augmented Shuffling

cs.AI cs.CR ·2026-11-19 ·Zhihao Liu, Zixiong Guo, Shuo Shao, Yu He, Wenli Wang +3 more
similarity 0.68
Abstract

Large language models are increasingly deployed as LLM-as-a-Service (LLMaaS), where user queries are sent to remote LLMs for inference, raising severe privacy concerns about the exposure of sensitive user inputs. Differentially private inference mitigates these risks by injecting noise into input embeddings. However, existing methods struggle to balance privacy and utility, as stronger privacy protection often leads to substantial accuracy degradation in LLM inference. In this work, we propose DP-SPD, a differentially private LLM inference framework that protects user queries by hiding the real query among client-generated dummy queries. All queries are randomly shuffled before being sent to the server, preventing direct identification of the user's true intent. To improve inference efficiency, we exploit the shared-prefix structure among queries and reuse transformer key--value (KV) caches during decoding. We analyze the privacy guarantee of DP-SPD and evaluate its utility on classification and generation tasks, as well as its privacy protection under QIA, EIA, and GPT-based attacks. Experimental results show that our method achieves strong inference-time privacy protection and near-plaintext utility while improving inference efficiency through shared-prefix KV-cache reuse.

0.07

SpliTEE: Fast and Private LLM Inference by Coupling GPU-Assisted Trusted Execution Environments with Differential Privacy

cs.CR cs.AIcs.LG ·2026-09-14 ·Shashie Dilhara Batan Arachchige, Robin Carpentier, Hassan Jameel Asghar, Dali Kaafar
theory similarity 0.68
Abstract

User prompts provided to large language models (LLMs) may contain private information. One way to protect them is to execute the LLM inside a trusted execution environment (TEE). However, this results in slow inference times as current TEEs are significantly slower than GPUs for LLM inference. To circumvent this, Tramèr and Boneh (2019) proposed Slalom which splits neural network inference between a TEE and an untrusted GPU. They encrypt inputs to computations outsourced to the GPU. In this paper, we extend this split-inference architecture to LLM inference and instead protect intermediate inputs using differential privacy (DP). We first demonstrate that masking intermediate representations is necessary by showing an 80% accuracy on a prompt-reconstruction attack from these representations. Our main contribution is a global sensitivity analysis of key functions in LLM inference, which bounds the required scale of DP noise. Unlike encryption, DP avoids quantization, allowing the LLM to remain in the floating-point domain. We also derive an upper bound on the floating-point error from masking and subsequent noise cancellation as a function of the privacy parameter epsilon, keeping the same quality of the LLM response. We implement our architecture using the Intel TDX TEE and two LLMs: Llama-3.2-3B and Qwen3-4B. Our split execution is nearly twice as fast as fully TDX-based inference. Moreover, it is at most 43% faster than Slalom while achieving higher accuracy. Finally, we demonstrate that prompt reconstruction, even with knowledge of the DP mechanism, cannot recover more information than is contained in an unrelated prompt.

0.17

NOMAI: Scalable Federated Learning with Private Weights

cs.CR ·2026-10-15 ·Yiyi Sun, Mushtari Sadia, Amrita Roy Chowdhury
similarity 0.68
Abstract

The twin goals of privacy-preserving federated learning (FL) -- harnessing distributed data while protecting the parties who contribute it -- demands three properties: model privacy, data privacy, and integrity. Achieving all three in a realistic deployment -- with a single coordinator and millions of users -- remains an open challenge that prior work has yet to fully resolve. We present NOMAI, the first scalable MPC-based FL system to provide all three guarantees simultaneously. NOMAI operates under an honest-majority assumption in the semi-honest model, with an extension to the malicious-with-abort setting. At its core, NOMAI combines a multi-committee MPC architecture with packed Shamir secret sharing, enabling SIMD-style secure computation that scales far beyond what monolithic n-party protocols can support. We further introduce ML-aware optimizations and new cryptographic primitives that substantially reduce per-user overhead. Our evaluation shows NOMAI is practically deployable at unprecedented scale: it reduces per-user computation to just <350 ms (semi-honest) and <700 ms (malicious) for per iteration of training MNIST, achieving three orders-of-magnitude improvements over prior work while seamlessly supporting populations of over 106 users. This demonstrates, for the first time, that MPC-based FL can meet the scale demands of real-world systems without compromising on privacy or integrity.

0.13

An Open-Source End-to-End FHE Implementation for Privacy-Preserving Llama 3 8B Inference

cs.CR ·2026-09-11 ·Yuhang Fan, Yusi Chen, Kanyu Ye, Zhuoran Ji
code/data similarity 0.67
Abstract

Cloud LLM services typically require users to send prompts to a model provider, creating a privacy risk. Fully homomorphic encryption (FHE) lets a server perform inference without decrypting the input, but representing data as ciphertexts adds storage and computational overhead. In CKKS-based LLM inference, the packing scheme maps logical tensors to ciphertexts and slots. It therefore determines the ciphertext count and the homomorphic cost of linear layers, and it constrains how data pass between linear layers, attention, and nonlinear computation. As models and sequences grow, inefficient layouts accumulate encoding, compute, and layout-conversion overhead. We present Odin, an FHE inference system that co-designs ciphertext packing and model execution for Llama. Starting from a THOR-style baseline whose bottleneck is weight encoding, Odin uses a feature-major cross-layer layout to unify residual connections and layer interfaces, and builds transient intra-operator layouts for linear projections and attention. This reduces redundant plaintext encoding of weights in wide projections. Within attention, QK^T produces scores that Softmax can consume directly, and PV consumes the resulting probabilities, avoiding intermediate repacking. For nonlinear ops, we use minimax polynomial approximation with input-range control and joint error allocation guided by model quality, reducing polynomial degree and multiplicative depth. To our knowledge, Odin is the first open-source end-to-end GPU CKKS implementation of Llama-3. With Llama-3-8B weights and a 128-token input, Odin evaluates all 32 Transformer layers on a single NVIDIA H100 80 GB GPU. Server-side end-to-end FHE evaluation takes 366.4 s and 58.9 GiB peak device memory. Under the same model, input, CKKS parameters, and hardware, THOR takes 1651.9 s, a 4.51x speedup.

0.47

VeilTune: Secure Fine-Tuning of Private LoRA Adapters on Public Pretrained Language Models

cs.CR cs.AI ·2026-11-09 ·Huiyu Xie, Tanping Zhou, Tao Hu, Xinlei Liu, Hailong Ma
similarity 0.67
Abstract · 13 pages, 7 tables

Fine-tuning public pretrained language models with lightweight adapters such as LoRA has become the mainstream way to adapt them to downstream tasks. A service provider often trains its adapter on a data owner's samples, so two assets need protection: the data owner's samples and the provider's adapter. Existing secure fine-tuning protocols protect the samples but reveal to the data owner intermediate results of every training step, such as the hidden representations or logits of its inputs. We show that a single step suffices to copy the adapter: from the hidden representations of one mini-batch of 32 samples, the data owner fits an adapter that agrees with the provider's on 99.2% of test inputs, and logits alone yield about 94%. Protecting both assets therefore requires that no value that depends on the adapter be revealed throughout training. We present VeilTune, to the best of our knowledge the first protocol that fine-tunes a pretrained language model with backpropagation entirely in secure multi-party computation (MPC). VeilTune computes only what the loss depends on, replaces generic MPC operators with training-aware nonlinear operators that stay accurate on the values that training produces, and obliviously skips the steps corrupted by truncation failures. On four few-shot tasks with RoBERTa-large, VeilTune nearly matches plaintext training and is 5.3 points more accurate on average than SecP-Tuning, the state-of-the-art secure fine-tuning protocol. On SST-2, it achieves a 3.5x end-to-end speedup and a 3.6x reduction in communication over SecP-Tuning, and 26x and 17x over full fine-tuning in MPC.

0.24

Transparent Identity Verification Approach Using MPC and Efficient Credential Status Handling

cs.CR cs.ET ·2026-09-12 ·Istiaque Ahmed, Shoji Kasahara, Kentaroh Toyoda, Tadashi Nakano, Thi Hong Tran
similarity 0.67
Abstract · We will run more experiment and validate more to produce quality research work

A secure and privacy-preserving identity verification process is essential for digital ecosys- tems. Current eKYC frameworks that rely on Zero-Knowledge Proofs (ZKPs) face high computational cost, rigid circuit design, complex integration, and expensive on-chain verification. The W3C 2021 BitString- based credential status mechanism also suffers from inefficient updates and poor scalability in large- scale deployments. We propose a transparent and cost-effective identity verification framework based on Multi-Party Computation (MPC). It enables private off-chain code execution and produces runtime proofs anchored to a blockchain. The framework introduces a multidimensional bit-matrix model with efficient compression. Using ZSTD, the credential data is reduced to 76 bytes compared to 140 bytes with GZIP, cutting storage and bandwidth costs. The system also supports fine-grained status updates and Layer-2 blockchain anchoring for tamper-evident, low-cost verification. The system employs reusable verifiable presentations (VPs) with unique access tokens, enabling cost-free verification and stronger access control. Selective disclosure preserves user control and strengthens privacy. Finally, the system integrates SHA3 hashing and Falcon post-quantum signatures. This guarantees robustness against quantum attacks, transparency, and scalability. It is a future-proof solution for national-scale identity verification, as demonstrated by experimental findings and security studies that validate its robustness and applicability.

0.13