CS Paper Radar

Drowning in Noise: On Signal Saliency in Self-Supervised Learning for Ocean Acoustics

Shikhar Bharadwaj, Samuele Cornell, Shinji Watanabe, Malcolm Slaney

cs.SD ·submitted 2026-11-04 16:14 UTC · v1
publishedbenchmark

Abstract

Self-supervised learning (SSL) for audio typically benefits from larger pre-training corpora, but this relationship assumes that training data carries meaningful learning signal. In ocean acoustics, this assumption is often violated: passive recordings are dominated by waves, wind, and silence, with biological vocalizations occupying a small fraction of the total duration. We show that this overrepresentation of noise causes SSL models to allocate representational capacity to acoustically trivial structure, degrading downstream performance. Using BEATs as a testbed, we demonstrate that pre-training on 30 hours of salient recordings (segments containing animal vocalizations) outperforms 1,240 hours of unfiltered data on a 31-class marine mammal classification benchmark. Under linear probing, the manually curated salient set achieves markedly better accuracy than the full corpus, despite using 41 times less data. Automatic saliency-based filtering improves over unfiltered pre-training, but still falls well short of the manually curated salient set. Our findings suggest that, in domains where informative content is temporally sparse, data curation matters more than data scale for SSL pre-training.

Comment
Accepted at DCASE 2026

Closest papers

Nearest neighbours by meaning across the whole index.

MAST: Label-Efficient, Robust, and Generalizable Sound Detection for Biodiversity Monitoring via Masked Audio Pretraining and Self-Training

cs.SD cs.LG ·2026-09-14 ·Tianyi Xu, Daniel Pimentel-Alarcón, Zuzana Buřivalová, Claudia Solís-Lemus
similarity 0.73
Abstract · 29 pages, 8 figures

Passive acoustic monitoring can measure biodiversity at larger scales, but time--frequency annotation of animal vocalizations is expensive, site-specific, and difficult to sustain at scale. We present a label-efficient sound detection framework that combines masked audio pretraining with a lightweight detector on mel spectrograms, then further improves robustness through iterative self-training on unlabeled audio. We first pretrain a ViT-based encoder on unlabeled recordings via masked reconstruction and transfer the encoder to a detection backbone. To better separate animal sounds from confounding background, we add a box-level contrastive loss that pulls matched event regions together while pushing noisy negatives apart. We then apply a two-stage pseudo-labeling curriculum to exploit large unlabeled pools without additional annotation. We evaluate the performance on two ecologically distinct domains: tropical rainforest soundscapes (Indonesia) and bird vocalizations in Mediterranean habitats (Spain). On both domains, masked audio pretraining and contrastive learning consistently improve time--frequency detection under temporal and cross-site distribution shift, and self-training yields further gains in out-of-distribution performance. On the rainforest domain, MAST with self-training achieves +0.22 mAP and +0.24 F1 over the strongest baseline under cross-site shift. On the bird domain, self-training achieves +0.12 mAP and +0.10 F1 over the strongest baseline under cross-site shift. Overall, our results show that MAST can effectively extend self-supervised audio representations from clip-level tasks to robust box-level localization across diverse bioacoustic settings, providing a practical path for biodiversity monitoring with limited labels.

0.51

MarineSet: Large-Scale Underwater Acoustic Data Curation and evaluation

cs.SD eess.AS ·2026-10-21 ·Hilde I. Hummel, Sandjai Bhulai, Rob D. van der Mei, Burooj Ghani
similarity 0.73
Abstract

The scarcity of labeled underwater acoustic data limits the development of data-driven models for marine acoustic monitoring, motivating the use of self-supervised learning to exploit the abundance of unlabeled recordings. However, large-scale data alone does not guarantee effective representation learning, because model performance is strongly influenced by the distribution of the training data. To facilitate such training data, this study introduces a scalable curation pipeline that curates 3,000 hours from 41 years of raw NOAA hydrophone recordings, producing MarineSet, a large-scale and diverse dataset designed for underwater acoustic learning. A baseline model, MarineNet, trained on MarineSet achieves competitive performance on marine mammal call and ship-type classification, highlighting the value of systematic data curation for developing self-supervised learning models in underwater acoustics.

0.20

OpenWhistle: A Large-Scale Longitudinal Dataset and Benchmark of Bottlenose Dolphin Vocalizations

cs.CL ·2026-09-28 ·Faadil Mustun, Chiara Semenzin, Roberto Dessi, Pablo Robin Guerrero, Pierre Orhan +5 more
top venuebenchmark similarity 0.71
Abstract · Accepted as a Spotlight at the NeurIPS 2026 Datasets & Evaluations Track

Recent advances in bioacoustics have been driven by large-scale corpora and standardized benchmarks, yet existing resources are overwhelmingly bird-centric and shallow per species, limiting their use for studying the structure of a single species' communication system. This gap is particularly acute for cetaceans: despite bottlenose dolphins (Tursiops truncatus) being a compelling case of complex vocal communication among non-human mammals, existing dolphin datasets are small, fragmented, and largely closed. We introduce OpenWhistle, the largest publicly available dataset of dolphin vocalizations. It comprises approximately 180,000 whistles (114 hours) recorded over five years from a stable pod of five individuals in a semi-natural environment, paired with a curated subset of 8,354 expert-annotated whistles and reproducible evaluation protocols for whistle-type detection and classification. We further release the full processing pipeline for whistle detection, segmentation, and categorization. To demonstrate its utility, we pretrain a Wav2Vec2.0 model adapted to dolphin acoustics on the OpenWhistle corpus and show that it learns effective representations, outperforming general-purpose bioacoustic models such as AVES and BioLingual on both tasks while leaving meaningful headroom for future work. By releasing the dataset, pipeline, and evaluation protocol, we provide the first open dolphin whistle dataset tailored for training self-supervised models, laying the groundwork for advancing dolphin communication research and developing models that capture fine-grained acoustic structure within species.

0.29

UniqueShip: Mitigating Data Leakage in Acoustic Ship Classification Benchmark Datasets

cs.SD cs.CVeess.AS ·2026-09-12 ·Connor Hashemi, Trevor Stout, Anthony Hoogs, Jason Parham
top venuebenchmark similarity 0.67
Abstract · 9 pages paper, 4 pages supplementary at the end, accepted/will be published at Oceans 2026. Note that while this is underwater acoustics, we…

Underwater Acoustic Target Recognition (UATR) of ships is well-suited for machine learning, yet its progress is hindered by the lack of large, diverse, and publicly available labeled datasets. In this work, we introduce UniqueShip, a machine learning-ready benchmark dataset for UATR applications sourced from the open Ocean Networks Canada (ONC) repository. Unlike previous datasets, we explicitly control for "data leakage" between the training and evaluation sets to ensure more reliable and generalizable model evaluation that does not encourage the model to memorize individual ships. We demonstrate that typical, random data partitioning in two prominent UATR datasets leads to falsely optimistic test performance, increasing accuracy by 10-48 percentage points compared to our more careful partitioning. Ablations on UniqueShip further show that doubling the number of unique vessels improves accuracy by 2.4-2.6 percentage points, while doubling total audio duration improves only by 0.8-1.3 points, indicating that vessel diversity should drive dataset curation more than total hours. We provide baselines with convolutional and transformer backbones, and analyze how ship metadata correlates with classification performance, finding that individual vessel characteristics predict classification difficulty far better than distance to the hydrophone alone. Overall, UniqueShip contains 2,460 hours of ship-radiated audio from 4,218 unique vessels (3,437 hours including background). We publish the dataset, code, and easy-to-download splits at uniqueshipdata.org to foster further UATR research.

0.70

ONCship: A Large-Scale Underwater Acoustic Corpus for Ship Recognition

cs.SD cs.AIeess.SP ·2026-10-14 ·Zhe Cao, Jichao Zhang, Xiao-Lei Zhang
similarity 0.66
Abstract

Recent advances in deep learning have significantly improved underwater acoustic vessel recognition. However, existing publicly available datasets are typically limited in scale, hindering large-scale representation learning and downstream task evaluation. To address this limitation, we introduce ONCship, a large-scale underwater acoustic dataset comprising 4521 hours of curated recordings from Ocean Networks Canada (ONC) hydrophones and Automatic Identification System (AIS) data. We evaluate the proposed dataset across three tasks: self-supervised representation learning, vessel-type classification, and MMSI-Level Vessel Classification. Experiments on DeepShip and ShipsEar demonstrate the effectiveness of ONCship for acoustic representation learning, with joint pre-training on ONCship and AudioSet achieving a classification accuracy of 62.45% on DeepShip through linear probing. Furthermore, incorporating ONCship as auxiliary training data improves downstream vessel-type classification performance. Beyond conventional vessel-type classification, we further investigate MMSI-Level Vessel Classification, treating each vessel as a distinct identity. Experimental results show that models fine-tuned with different loss functions achieve Top-1 accuracies exceeding 53% and Top-5 accuracies approaching 69%, demonstrating the feasibility of distinguishing individual vessels through MMSI-Level Vessel Classification. Overall, ONCship provides a valuable resource for underwater acoustic representation learning and vessel recognition, while its rich vessel-level metadata enables further exploration of MMSI-Level Vessel Classification.

0.18

A Perch out of Water: Few-Shot Detection of Bryde's Whale Vocalisations in False Bay

eess.AS ·2026-10-21 ·G. Tonitz, C. M. Geldenhuys, T. R. Niesler
similarity 0.65
Abstract · 10 pages, 5 figures, conference

The automated detection of baleen whale calls generally requires large annotated datasets, which are not available for many species and populations. We investigate whether pretrained bioacoustic embeddings, combined with a prototypical few-shot classifier, allow such calls to be detected and classified from only a handful of labelled examples. Two embedding networks are compared: Perch, a general-purpose bioacoustic model trained predominantly on birdsong, and WhaleVAD, a task-specific blue- and fin-whale call detector. Neither model is fine-tuned, and at most ten labelled examples per call type are assumed. Evaluating on Antarctic blue- and fin-whale recordings, we find that the default Perch input representation is poorly matched to low-frequency whale calls, and that increasing the frame length while restricting the mel filterbank to the frequency range occupied by the calls improves the AUC-ROC by 5.8% absolute when ten examples per call type are available. Allowing the classifier to detect each call type independently, so that segments may contain several calls, affords a further 3.3% improvement and a macro AUC-ROC of 0.739, representing a combined gain of 9.1% absolute. Both embeddings are applied to Bryde's whale vocalisations recorded in False Bay, South Africa, which neither model has encountered during pre-training. Here Perch outperforms WhaleVAD for every call type, achieving a macro AUC-ROC of 0.987 against 0.859, despite having a predominantly terrestrial training set. These results indicate that general-purpose bioacoustic embeddings, paired with an input representation matched to the calls of interest, can support whale call detection for species and populations for which large annotated datasets do not exist.

0.22

Behavior-Conditioned Animal Vocalization Generation: Event-Aware Modeling and Multidimensional Evaluation

cs.SD ·2026-10-12 ·Xinlong Deng, Yong Tian, Jian Guan, Qiuqiang Kong, Jie Jiang +1 more
datasetcode/data similarity 0.64
Abstract · Submitted to ICASSP 2027. Project page: https://persimmontian.github.io/BCAVG

Behavior-conditioned animal vocalization generation aims to synthesize vocalizations from natural-language descriptions of animal behaviors, social interactions, and contexts. This task remains largely unexplored, with limited behavioral text supervision and substantial variation in vocalization duration. We construct the Behavior-Vocalization Dataset (BeVo) by integrating public bioacoustic data from six species with natural-language behavioral conditions derived from species-specific ethological and bioacoustic literature, totaling 31,191 events. We then adapt a rectified-flow-based generator with Event-Aware variable-length modeling, which encodes vocalizations at their actual durations and uses masks to prevent attention to padded latent positions and exclude these positions from the training loss. We evaluate vocalization discriminability, acoustic distribution fidelity, and language-aligned representation fidelity. Experiments show that task-specific adaptation improves all evaluated metrics compared with its zero-shot baseline, while Event-Aware modeling further increases vocalization-anchor macro accuracy from 59.86% to 61.99% and reduces Frechet Audio Distance from 1.40 to 1.22. Conditioning analyses further show that behavioral conditions improve target-anchor discriminability, while swapping behavior shifts generations toward the corresponding vocalization anchors. Demos are available on our project page.

0.30

Dolphin broadband transient vocalization detection in impulsive ambient noise

cs.SD eess.ASeess.SP ·2026-10-26 ·Hari Vishnu, Jin Xinhui, Mandar Chitre, Simen Hexeberg, Koay Teong Beng
similarity 0.64
Abstract

Passive acoustic monitoring in tropical waters such as those in Southeast Asia is hampered by the persistent, high-level impulsive noise of snapping shrimp. This is a particularly daunting challenge for detecting broadband transient vocalizations of dolphins - clicks, buzzes and burst-pulses. These vocalizations, unlike frequency-modulated whistles, share the broadband, impulsive character of the shrimp snaps and cannot be easily separated from them by frequency-localized or energy-based means. We develop TRIDENT, a lightweight waveform-based detector for these vocalizations, and apply it to 54.2 recorder-months of data from ten island sites around Singapore. We show that the training-data pipeline and the architecture are both important: controlled signal-to-noise-ratio based mixing to generate training data, appropriate normalization and hard negative mining, together steer the detector toward learning transient temporal structure rather than energy shortcuts. A single joint detector handles all three signal types better than specialized models for each, and outperforms established baselines. Applied to long-term recordings, it reveals tightly clustered, predominantly daytime activity, motivating days-with-detections as a robust indicator of cetacean presence.

0.15