CS Paper Radar

Security and Privacy in the Musical Metaverse: Threat Analysis and Design Implications

Luca Turchet, Michał Kłosinski

cs.CR ·submitted 2026-09-03 10:56 UTC · v1
unusual

Abstract

The Musical Metaverse (MM) introduces immersive, real-time environments for collaborative musical interaction, characterized by ultra-low-latency constraints, continuous multimodal data streams, and heterogeneous devices. These properties create a distinctive security and privacy landscape that differs significantly from conventional XR or multimedia systems. This paper presents a multi-layer threat analysis of MM ecosystems, identifying key assets including live musical content, expressive interaction data, identity and session metadata, and intellectual property. Threats are analyzed across network, application, data/AI, device, intellectual property rights, and social layers, with particular attention to risks arising from expressive and neurophysiological data, which enable inference, re-identification, and potential privacy violations. We describe a stakeholder-driven survey involving 14 participants from 13 organizations, revealing that neurophysiological data leakage and real-time stream disruption are perceived as the most critical risks, followed by intellectual property infringement and avatar impersonation. We further evaluate the suitability of existing security protocols under strict latency constraints, showing that conventional approaches such as TLS over TCP are often incompatible with real-time musical interaction, while lightweight, stream-oriented mechanisms (e.g., SRTP, DTLS) provide a more suitable balance between security and performance. Based on these findings, we derive a set of design guidelines for MM systems, emphasizing latency-aware security, differentiation of interaction paths, data minimization, and edge-centric processing. The results support a security-by-design approach that enables trust and compliance without compromising real-time performance.

Comment
IEEE International Symposium on Emerging Metaverse 2026

Closest papers

Nearest neighbours by meaning across the whole index.

Omniscience for the Masses: New Threats in the Metaverse's Democratized World Creation

cs.CR ·2026-09-11 ·Andrea Mengascini, Ryan Aurelio, Jason Polakis, Giancarlo Pellegrino
top venuesurvey similarity 0.60
Abstract

Metaverse platforms increasingly derive their success from user-generated virtual worlds: self-contained social and interactive environments, which can be created by any ordinary user and scale to billions of visits. Platforms such as Roblox, Horizon Worlds, and VRChat now host millions of creator-built worlds that govern how users see, hear, and interact with one another. While this model enables rapid growth and creativity, it fundamentally delegates control over social interactions and world behavior to untrusted users. In this paper, we present the first systematic security and privacy assessment of metaverse world creators. We survey 25 platforms that support user-created worlds and analyze their world-creation capabilities. Guided by this analysis, we design and implement five novel attacks that exploit creator-provided tools to violate spatial, visual, and auditory constraints in immersive environments, enabling covert user surveillance and manipulation without software vulnerabilities or developer-level privileges. We further show that five previously-proposed attacks can be replicated using only standard world-creation features. Finally, we find that existing platform vetting, runtime protections, and creator policies are insufficient to mitigate malicious world-creator behavior, revealing a fundamental mismatch between users' privacy expectations and the powers granted to world creators.

0.75

Why Today's Internet Cannot Support Tomorrow's Immersive Experiences: Insights from the NSF Workshop

cs.NI cs.CRcs.HCcs.MM ·2026-09-23 ·Bo Han, Amitabh Varshney, Klara Nahrstedt, Songqing Chen, Sarita Adve +8 more
similarity 0.58
Abstract · 32 pages, 6 figures

Immersive computing is poised to become a major paradigm for future human-computer interaction (HCI), enabling spatial and embodied experiences across applications such as education, healthcare, manufacturing, emergency response, and scientific discovery. However, today's Internet and computing infrastructure were not designed for real-time, multi-user, sensor-rich, and privacy-sensitive immersive applications. They require coordination across sensing, representation, compression, networking, edge/cloud computing, rendering, security/privacy, and human experience. This report summarizes key findings from the NSF Workshop on Networking and Systems Challenges in Immersive Computing. The workshop brought together researchers from networking, systems, extended reality (XR), artificial intelligence (AI), graphics, HCI, security, privacy, and applications to identify challenges and opportunities. We conclude that immersive computing cannot be enabled by isolated advances in network bandwidth, device capabilities, AI models, or rendering alone. The field needs end-to-end architectures that adapt to user behavior, application requirements, resource constraints, and social expectations. We organize the agenda into six themes: networked systems for immersive computing; real-time edge-cloud orchestration for XR applications; AI for immersive experiences; privacy, security, and trust in XR systems; human-centered design and user experience in XR; and scalable infrastructure, testbeds, and open ecosystems for XR. We further identify key research questions, future trends, risks, ten-year success metrics, and action items for academia, industry, government, and research community. The report calls for coordinated investment in open platforms, reproducible benchmarks, privacy-preserving systems, and human-centered evaluation to build trustworthy, scalable, and broadly impactful infrastructure.

0.28

When Music Needs Meaning: Semantic Communication as a Transformative Enabler for the Musical Metaverse

eess.SY cs.AI ·2026-11-02 ·Ali Al Housseini, Jaime Llorca, Cristina Rottondi, Luca Turchet, Tiziano Leidi +1 more
published similarity 0.56
Abstract · 10 pages, 3 figures. Accepted and Presented at IEEE International Symposium on the Internet of Sounds (IS2) Conference 2026

The Musical Metaverse (MM) enables geographically distributed musicians, audiences, sound engineers, and AI agents to interact through synchronized audio, visual, kinematic, and haptic streams, imposing latency, synchronization, bandwidth, and scalability requirements that are difficult to satisfy with conventional bit-level communication. This paper argues that semantic communication, understood here as the transmission of musically meaningful information rather than raw audio-visual-haptic samples, provides a principled communication paradigm for the MM. We define the MM from a communication perspective and derive modality-specific require ments; propose a three-pillar framework for musical semantic communication comprising semantics extraction, semantic-aware orchestration, and cross-modal semantic synchronization; survey representative work across semantic communication, music AI and edge orchestration, identifying the gaps that prevent their integration; and propose a semantic-first reference architecture based on device-side semantic encoders, edge semantic routers, and cross-modal semantic synchronization. The resulting framework positions the MM as a concrete and demanding use case for 6G semantic communication.

0.44

Beyond the Headset: A Systematization of Knowledge on Extended Reality Privacy and Security in Healthcare

cs.CR cs.HC ·2026-09-29 ·Nafisa Anjum, M. Rasel Mahmud
publishedsurveybenchmarkhighly cited similarity 0.55
Abstract · Published in 31st ACM Symposium on Virtual Reality Software and Technology

Extended reality (XR) systems are increasingly used in healthcare applications ranging from surgical planning to remote rehabilitation and mental health support. However, the rich streams of sensor, biometric, behavioral, and environmental data that enable these applications also introduce substantial privacy and security risks. Adversaries may exploit insecure communication, sensor side channels, application-layer vulnerabilities, or data-processing pipelines to infer sensitive information or disrupt clinical workflows. Despite growing interest in XR security and privacy, the healthcare-specific literature remains fragmented. In this Systematization of Knowledge (SoK), we review 65 peer-reviewed studies published between 2017 and 2024 across XR, security, privacy, and healthcare venues. We develop a unified threat taxonomy spanning device, user, network, and cloud layers and introduce XR-PRISM, a quantitative Privacy and Risk Impact Scoring Metric for systematically characterizing security and privacy risks. Our analysis identifies several gaps in the literature: more than 70% of proposed countermeasures lack standardized risk evaluation, fewer than 15% of studied attacks require high attack prerequisites, and reproducibility is limited by the scarcity of publicly released artifacts and datasets. Based on these findings, we outline a research roadmap emphasizing shared benchmark datasets, stronger artifact-release practices, improved cloud-layer protections, and more comprehensive detection, mitigation, and recovery mechanisms. This SoK provides a structured and data-driven foundation for understanding existing risks and guiding the development of more secure, privacy-preserving, and usable XR healthcare systems.

0.57

From Acoustic Sensing to Physical Actions: A Comprehensive Survey of Security Risks in Edge-Deployed Audio LLM-Based Agents

cs.CR cs.CVcs.LG ·2026-10-11 ·Binhong Wang, Haidong Kang, Yanfeng Zhang, Hong-Ning Dai, Guo Yu +1 more
survey similarity 0.54
Abstract · 20 pages, 8 figures

Audio Large Language Models (Audio LLMs) are embedded in agents for smart homes, vehicles, wearables, and embodied systems, where acoustic inputs can propagate from perception into persistent state and consequential execution. This setting extends security analysis beyond model behavior to the system conditions that determine how acoustic inputs can influence agent behavior and external actions. Existing surveys examine acoustic security, LLM-based agents, and edge deployment in isolation, leaving their security interactions in deployed Audio LLM-based agents insufficiently characterized. This survey introduces an Acoustic Sensing-to-Action (S2A) system model that traces these security implications along the path from acoustic sensing to external execution, while accounting for edge-cloud runtime and lifecycle transformation. We use the model to synthesize attacks, defenses, and evaluation methods across the acoustic, agent, and edge-system literature. The synthesis shows that security gaps can emerge within components and at the transitions between stages, where protections developed for one stage do not necessarily provide end-to-end assurance in the deployed system. These findings motivate security evaluation that follows acoustic influence from sensing to action across persistent state, system boundaries, and changing deployment conditions in real-world agents.

0.18

CrowdForge: Effects of Audio and Visual Audience Participation Modalities on Agency, Collaboration, and Engagement in AI-Mediated MR Performances

cs.HC cs.AIcs.NIcs.PF+1 ·2026-11-02 ·Torin Hopkins, Shih-Yu Ma, Zander Trafton-Hancock, Ellen Yi-Luen Do, Luca Turchet
position similarity 0.53
Abstract

Technology-mediated audience participation (TMAP) enables audiences to influence live performances, yet balancing interaction with meaningful agency remains challenging. While AI-generated music and MR enable new forms of audio and visual audience participation, little is known about how these participation modalities influence audience experiences during live musical performance. In particular, it remains unclear how they shape perceived agency, collaboration, and engagement, and how they complement one another within TMAP systems. We present CrowdForge, an AI- and MR-mediated paradigm where audiences collectively control generative music and virtual environments during a live performance featuring a guitarist and DJ. Through a mixedmethods concert study evaluating three interaction conditions (audio, visual, and combined audio-visual), we demonstrate that observability is the primary driver of audience agency. While visual interaction provided easily observable feedback, audio parameters offered macro-structural control over the music. Audiences tended to use the control enabled by AI audio generation to test musical boundaries, introducing a productive adversarial tension between the crowd and the musicians. This dynamic pushed creative limits, deepened engagement, and generated moments of musical coherence when performers successfully resolved audience-induced challenges. We argue future TMAP systems should prioritize human-centered, highly observable interactions to build agency, while combining the expressiveness of music with the clarity of visual feedback.

0.19

The Unseen Threat: How Complementary Combinations of Acoustic Features Compromise Speech Privacy

eess.AS ·2026-11-12 ·Jule Pohlhausen, Jörg Bitzer
top venue similarity 0.52
Abstract · Accepted at SPSC 2026 - 6th Symposium on Security and Privacy in Speech Communication

The interaction of acoustic features in leaking private information remains underexplored. This work focuses on time-domain features, i.e., broadband RMS level and pitch, alone and combined with temporally smoothed and subsampled Mel spectrograms. Privacy is assessed using informed automatic speech recognition and speaker-verification attacks, while utility for conversational analysis is evaluated using voice activity detection and speaker diarisation. RMS level and pitch alone exposed limited information, but fusion with 500 ms-smoothed Mel spectrograms enhanced utility while substantially increasing speech-content leakage. Specifically, the combination with RMS level showed that fine-grained temporal dynamics can amplify leakage from coarsely sampled spectral features. Speaker-identity protection remained limited for all tested features. Overall, these findings indicate that privacy assessments performed on individual acoustic features may underestimate leakage after feature fusion.

0.24

Backdooring Acoustic Foundation Models for Physically Realizable Triggers

cs.SD cs.LG ·2026-10-07 ·Zebin Yun, Eyal Ronen, Mahmood Sharif
published similarity 0.52
Abstract · Accepted at RAID 2026

Acoustic foundation models (AFMs) have democratized acoustic applications, enabling powerful models for tasks ranging from speech recognition to speaker verification with minimal resources. However, the security of applications based on AFMs remains largely underexplored. Our work addresses this gap by proposing the Foundation Acoustic model Backdoor (FAB) attack, demonstrating that state-of-the-art AFMs are susceptible to backdooring under practical settings. Despite making minimal assumptions about adversary capabilities (e.g., no access to pre-training data), we show that FAB preserves benign performance while inducing backdoors that survive fine-tuning and cause significant degradation across diverse downstream tasks when activated. Notably, FAB utilizes task-agnostic, physically realizable, inconspicuous, and sync-free triggers (e.g., a background siren). We evaluate FAB using two leading AFMs, nine downstream tasks, and four different triggers. We further demonstrate its effectiveness against established defenses and across both digital and physical domains. While extensive end-to-end fine-tuning can mitigate FAB, such a defense is resource-intensive and task-specific. Our work highlights critical risks to AFMs and calls for advanced defenses.

0.52