CS Paper Radar

Multi-View Fusion for Encrypted C2 Detection: A Leakage-Controlled Measurement Study of Evaluation Pitfalls

Hoang-Huy Nguyen-Huu, Van-Tri Phan, Khuong Nguyen-An

cs.CR cs.AI ·submitted 2026-09-23 03:44 UTC · v1
publishedhot direction

Abstract

Command-and-control (C2) traffic increasingly hides within TLS, so defenders now apply machine learning to traffic metadata. Many studies assume that combining two metadata views, namely flow statistics and TLS handshake fingerprints, improves both accuracy and robustness. We tested this assumption on 17,577 TLS flows from 62 real Cobalt Strike captures. Our evaluation removes the data leakage that leads to overly optimistic reported scores. We report three findings that matter more than the fusion result itself. First, an incorrect preprocessing step increases the F1 score by 0.28. This step computes the frequency encoding across the entire dataset rather than within each cross-validation fold. The increase is about ten times larger than any real effect we measured. Second, both the labels and the behavioral features depend on the destination address. Because of this, the 17,577 flows form only 2,132 independent groups, and the positive rate of 55.1\%, which looks balanced, drops to 4.2\%. Therefore, class balance is just a result of how we analyze the data, specifically whether we count flows or endpoints, and not a real feature of the task. Third, 20 of the 62 captures (32\%) have no TLS flows to any known C2 address, so they contain only benign samples. We checked these captures directly and confirmed that this is a gap in the ground truth, not a labeling error. In this context, fusion beats the best single view by only 0.022 in F1. When an attacker forges both feature surfaces simultaneously, every model performs worse than a simple baseline that always predicts positive (F1 = 0.711). For encrypted C2 detection, the evaluation design is not a preliminary step. It \emph{is} the main result.

Comment
Accepted for presentation at the 2026 ASIAN Conference on Communication and Networks (ASIANComNet 2026), Hanoi, Vietnam, October 11-14, 2026

Closest papers

Nearest neighbours by meaning across the whole index.

The Split Protocol Decides the Result: Re-Evaluating Encrypted Traffic Classification

cs.LG cs.AI ·2026-11-04 ·Mayan Sharma, Kriti Saini, Devansh Behl
code/data similarity 0.66
Abstract · 12 pages, 4 figures, 6 tables Code and data splits: https://github.com/Mayan10/Flowconx

Encrypted traffic classifiers are routinely reported at 95-99% accuracy. We show that on one of two public corpora, the same model and code score macro-F1 0.992 under the evaluation protocol standard in the literature and 0.567 under a protocol that keeps flows from one packet capture on one side of the split. Nothing changes but the partition (gap 0.426 over eight seeds, p = 0.008 after Holm-Bonferroni correction). The inflation is not a property of random splitting as such. On a month of backbone QUIC traffic, we evaluate five split protocols at eight seeds each: random, temporal and client-disjoint splitting are statistically indistinguishable from session-disjoint, while server-disjoint splitting costs 0.192 macro-F1 (d = 17.0, p = 0.008). Which axis matters depends on how the corpus was collected, in a way invisible from its label space. We give a diagnostic that identifies the right axis before any model is trained: single-identifier probes. A decision tree on the capture identifier alone reaches 0.983 under a random split and exactly 0.000 under session-disjoint splitting. Under this evaluation, a metric-trained flow embedding leads four deep baselines but does not beat gradient-boosted trees on twenty packet sizes. Five of our own pre-registered architectural claims fail: distance-based rejection of unknown applications beats softmax thresholding, but ablating every metric objective makes rejection better, because class separation correlates at r = -0.88 with rejection quality. Few-shot enrollment is flat within a corpus but, across corpora, five labelled flows per class recover 91% of the deficit left by failed zero-shot transfer. All splits, results, code and negative findings are released.

0.34

Unknown-Traffic Detection, Calibration and Shortcut Reliance in Distilled Encrypted-Traffic Classifiers over One Year

cs.NI cs.LG ·2026-09-25 ·Mahmoud Abbasi
code/data similarity 0.65
Abstract · 15 pages, 5 figures, 11 tables. Pre-registered at OSF (https://osf.io/rts6n) before any test-window result was computed. Code: https://githu…

Knowledge distillation is the standard way to compress encrypted-traffic classifiers for the edge, and almost all such work judges students by accuracy alone. We ask what else a student inherits: unknown-traffic detection, calibration, shortcut reliance, and whether any survives a year of drift. Resemblance proves little on its own, since soft targets also regularise. We therefore distil one 101k-parameter student from two teachers of equal accuracy but different construction, a five-member ensemble and a single wider model, so that following one rather than the other is attributable to it. The design was pre-registered before any test result was seen. We tested ten hypotheses on CESNET-TLS-Year22, a year of real TLS traffic, across 18 test windows over 35 weeks. Two are supported: a student's per-flow unknown-scores shift toward its own teacher, but only at a conventional temperature, not the accuracy-optimal one; and a shortcut-reliant teacher passes its over-confidence to a student that never sees the feature. The drift prediction is reversed under both scores, the gap narrowing rather than widening and the student overtaking under the energy score in two of three replicates, as is the prediction that such a teacher harms its student's detection, which improves slightly. Shortcut reliance is set by model size, not distillation. Under the logit-based scores nothing else transfers: distillation beats neither a temperature-scaled direct student nor label smoothing. Exploratory analysis shows this turns on the scoring rule: with a feature-space detector the teacher detects unknown traffic 0.073 AUROC better than the direct student, where the energy score sees 0.000, and the conventional-temperature student inherits most of it. Label smoothing, with no teacher, recovers more. Distillation transfers the teacher's habits; what looks like an inherited ability is available without one.

0.44

SoK: Where Do Flow Labels Come From? Auditing Label Provenance in Encrypted Traffic Benchmarks

cs.NI cs.LG ·2026-09-02 ·Sizhe Huang, Shujie Yang
surveybenchmark similarity 0.65
Abstract

Encrypted traffic classification infers semantics beyond the flow record from transport-layer observables, and supervised training rests on labels that hold for the individual flow they are attached to. Recent systematizations scrutinize model in- puts and data splits; we systematize the complementary label side. Across 14 audited benchmark entries, we identify two recurring label-side strategies: coarse inheritance, which risks labelling flows the evidence does not cover, and overstrict filtering, which keeps only self-attesting flows and risks dis- carding relevant ones. No audited entry exposes a countable pre-selection population, and the task objects downstream papers attach to the same labels disagree with the recovered record in 8 of 23 referenced cells. Under strict side-channel features we derive a representation-relative ceiling on bal- anced accuracy for any classifier restricted to those features: on the public benchmarks that inherit, it ranges from 0.56 to 0.76. On the filtering side, only 24.95% of connections in our fully captured corpus carry an observable SNI of their own; yet the discarded connections raise macro accuracy from 0.44 to 0.65 through same-run co-occurrence features. We end with recommendations for benchmark builders and users.

0.46

PERO: Efficient Robust Post-Training Foundation Models for Encrypted Traffic Classification

cs.LG stat.ML ·2026-08-16 ·Wumei Du, Jiarong Wen, Kaiyu Zhang, Zi Yang, Yiqin Lv +3 more
top venue similarity 0.64
Abstract · 16 pages, 6 figures, 6 tables, conference

Encrypted traffic classification is vital for network security, yet real-world deployments are inherently sensitive to rare but high-loss errors such as misclassification of malicious traffic. The encrypted traffic foundation model, as a promising general-purpose technique, can achieve impressive overall performance. However, employing standard objectives such as empirical risk minimization often overlooks high-risk tail events, and commonly used performance metrics hardly reflect robustness limitations in risk-sensitive scenarios. Directly applying robust optimization objectives, such as conditional value-at-risk, to post-training is computationally prohibitive for large models, as identifying high-loss samples exhausts substantial computation. To this end, we propose Pre-Evaluation Robust Optimization (PERO), an efficient robust post-training framework for encrypted traffic foundation models. PERO employs a lightweight proxy to estimate sample-wise risk and selects a subset of high-risk samples to update the foundation model, decoupling risk estimation from expensive large-model optimization. Extensive experiments on typical encrypted traffic datasets show that PERO achieves competitive or superior robustness and average performance compared to outstanding robust post-training methods, while significantly reducing computational and memory costs.

0.35

Matched-View Cross-Domain Evaluation of WireGuard VPN Traffic Classification Using Early-Flow Fingerprints

cs.NI ·2026-08-30 ·Yasameen Sajid Razooqi, Adrian Pekar
publishedbenchmark similarity 0.64
Abstract · Accepted at CNSM 2026

Classifying VPN-encrypted traffic by application category typically relies on datasets that collect non-VPN and VPN traffic in separate sessions, conflating encapsulation effects with session-level differences in user behavior, timing, and application mix. We use a recently published WireGuard tunnel dataset in which pre- and post-tunnel traffic is captured simultaneously, with a packet-level match ratio above 99.9%. This matched-capture design eliminates session-level confounds and enables a cross-domain benchmark: models are trained on non-VPN flows and tested on the VPN view of the same underlying flows. We compare whole-flow statistical aggregates (FlowFeatures) and Sequence of Packet Length and Time (SPLT) early-flow fingerprints across Random Forest, XGBoost, and a multi-scale CNN1D. Cross-domain transfer depends jointly on representation and model: tree ensembles achieve balanced accuracy of 0.84-0.93 with FlowFeatures but only 0.60-0.75 with flattened SPLT, whereas CNN1D processes the same SPLT fingerprint as a sequence and achieves the strongest transfer overall (balanced accuracy 0.98, macro F1 0.89) without any VPN data during training.

0.25

Honest Fusion Evaluation for Cross-Domain Deepfake Detection: Leakage Audits and Tail Robustness

cs.CV ·2026-11-11 ·Luke Collins, Chang-Tsun Li, Duc Thanh Nguyen
publishedtheorycode/data similarity 0.63
Abstract · Accepted at WACV 2027. 19 pages including supplementary material; 6 figures, 10 tables. Code: https://github.com/lukegcollins/WACV27-HFEFCDD

Cross-domain video deepfake detection increasingly uses frozen foundation-model experts, combined by calibrated score averaging, a per-video router picking one expert, or a learned head over concatenated features. Expressive routing and feature fusion are often reported to beat plain averaging. Across a 22-dataset frozen-expert testbed, we find those gains are largely evaluation artifacts. A corner theorem locates the gap routing aims to close: because AUC depends only on score ordering, the best any score-space router can do is an oracle reading the test label to pick each video's expert, a ceiling no realisable, label-blind router reaches. The theorem does not order real routers against a calibrated global average; the case against routing is empirical, and no router we test beats that average. Feature fusion reads raw features, falls outside the theorem, and is audited empirically. Reported gains trace to leakage: holding out one dataset leaks siblings sharing identities and reals (over-crediting routing), while holding out one manipulation keeps that dataset's reals in training (over-crediting feature fusion). A $2\times 2$ over fold and evaluation granularity pins the $+2.6$pp feature-fusion "win" to the fold alone: 12/12 significant under leaky folds vs. 0/12 under honest family-held-out folds, where both gains vanish. Fusion still pays off in the tail of hard videos and families: as experts are added, videos every expert misclassifies drop from 15.2% to 2.3%, and global calibrated fusion (InvB) beats the best single expert on worst-family AUC, selective risk (AURC), and low-false-positive recall, stably across protocols. Practitioners should calibrate and combine globally, hold out whole dataset families, and measure tail performance. For score-space routing under ranking metrics, honest evaluation and expert diversity matter more than combiner complexity.

0.47

When Handshakes Go Dark: An Adaptive Semantic-Sequential Fusion Framework for Encrypted Traffic Classification

cs.NI ·2026-11-09 ·Tong Guan, Yunchun Li, Minyuan Xiao, Mingyuan Xia, Yuchen Zhao +1 more
similarity 0.63
Abstract

Encrypted traffic classification is essential for network security monitoring and traffic management. Self-supervised pre-training can learn reusable representations from large-scale unlabeled traffic, but TLS 1.3, QUIC, and privacy mechanisms such as Encrypted Client Hello (ECH) increasingly conceal application, handshake, and service-identifying information. Observable upper-layer bytes are therefore dominated by ciphertext, weakening the link between raw-byte patterns and application semantics and limiting model generalization. Moreover, the availability and discriminative value of observable information vary across protocols, protection profiles, and tasks, complicating classification across diverse scenarios. To address these challenges, we propose FLARE, a Field-aware Learning and Adaptive Representation Fusion framework for encrypted traffic classification. FLARE pre-trains a BERT-based packet encoder through Masked Field Prediction, which reconstructs masked tokens from field-structured context. During downstream fine-tuning, the parameters of the pre-trained packet encoder remain frozen, and a packet-level gated module adaptively combines protocol-semantic representations with directional packet-length representations. A flow-level encoder then models inter-packet dependencies for classification. We evaluate FLARE on eight datasets spanning diverse encrypted-traffic tasks, including CipherSpectrum and VisQUIC for strongly encrypted scenarios. FLARE achieves superior performance over baselines across the eight datasets, exceeding the strongest pre-trained baseline by 10.92 percentage points on QUIC website fingerprinting. These results demonstrate the effectiveness of field-aware pre-training and adaptive semantic-sequential fusion under constrained observability.

0.04

Shortcut in Generalization: Rethinking Evaluation Protocol for Encrypted Traffic Classification

cs.NI ·2026-10-13 ·Qiming Yu, Chungang Lin, Zhengyi Ma, Zijia Song, Yuliang Lu +2 more
published similarity 0.61
Abstract · Accepted for publication at IEEE CSCloud 2026

Encrypted traffic classification is commonly evaluated under a range of experimental protocols, with classification performance reported as the primary measure. However, the reported performance may be influenced by shortcuts left uncontrolled in the evaluation protocol, such as shortcut features and cross-partition sharing, rather than reflect generalizable traffic patterns. And test-only feature transformations may further introduce train-test mismatch. In this paper, we study how current evaluation protocols are affected by shortcut-related factors, including field-level shortcuts, cross-partition sharing, and train-test processing. We compare feature importance, retrained feature removal, and field-only split sensitivity. We also vary Source-IP overlap and packets per flow, compare test-only IP transformations with matched train-test transformations. Experiments on three public datasets show that the performance of existing methods are affected by both cross-partition sharing and the consistency of train-test transformations. Specifically, increasing Source-IP overlap from 0% to 100% raises accuracy by about 32-33 points before identifier masking, compared with only about 8 points after masking. For two state-of-the-art pretrained models (i.e., YaTC and NetMamba), the 11-13 points loss caused by test-only IP zeroing is almost fully recovered when the same transformation is applied during both training and testing.

0.33