CS Paper Radar

Audiovisual diarization of overlapping click trains in sperm whale (Physeter macrocephalus) vocal sparring using a three-hydrophone array

Lara Berkenbaum, Hervé Glotin, François Sarano, Walter M X Zimmer, Véronique Sarano, Olivier Adam, Pascale Giraudet

eess.SP q-bio.QM ·submitted 2026-09-11 08:46 UTC · v1

Abstract

The individual attribution of sperm whale (Physeter macrocephalus) vocalizations during surface interactions constitutes a methodological challenge due to acoustic overlaps, multipath propagation, body shadowing, and indistinguishable inter-pulse intervals among similar-sized individuals. A multimodal audio-visual workflow is presented to deinterleave and attribute uncharacterized click trains produced by immature males during ``vocal sparring'' using a portable three-hydrophone array coupled with synchronized video. The approach combines spectro-temporal tracking, based on inter-click interval dynamics and the Constant-Q Transform, with spatial time-difference-of-arrival modeling projected onto the image plane for optical validation. Analysis of 2,655 manually validated clicks shows that purely acoustic clustering diarization errors remain low, peaking at 25.79% only during extreme temporal superpositions. Integrating the optical modality resolves residual spatial indeterminacies; although near-planar array geometry induces vertical ambiguities, the system achieves up to 100% horizontal visual concordance for primary emitters. Crucially, this framework successfully reconstructed and assigned 11 distinct, intertwined click trains totaling 882 clicks to specific focal individuals despite near-field tactile constraints. Because ethological descriptions remain incomplete without identifying the emitter, explicitly correlating these emissions with physical kinematics provides the fine-scale resolution required to define this socio-acoustic behavior.

Closest papers

Nearest neighbours by meaning across the whole index.

Dual-Channel Contrastive Encoding of Sperm Whale Coda

cs.SD cs.AIcs.LG ·2026-10-21 ·João Victor Quintanilha, Emma Virnelli, Shuo Zhang
code/data similarity 0.66
Abstract

Sperm whales (Physeter macrocephalus) communicate through short click sequences called codas, whose structure includes the rhythmic pattern of inter-click intervals (ICI) and the spectral composition of the clicks. In this paper, we propose the Dual-Channel Contrastive Encoder (DCCE), which explicitly encodes rhythm and spectral information with separate encoders before fusing them. We evaluate DCCE on three coda-understanding tasks: social unit, coda type, and individual identification. We compare DCCE against strong baselines from literature under a common linear-probe protocol and analyze the role of each channel and training objective through ablations. On 1,383 codas, shared partitions and validation-only layer selection support three random splits and three recording-date-disjoint folds. DCCE with unit pairing and type supervision achieves mean random-split macro-F1 of 0.901/0.865/0.597 for unit/type/identity. Raw ICI remains strongest for type (0.987), and ICI plus temporal mel statistics is a strong identity reference (0.724). An auxiliary identity objective improves spectral-branch identity prediction but does not improve the mean fused-identity score in this configuration. Date-grouped evaluation and four year holdouts expose generalization limits. The results support task-dependent combinations of timing and spectral information, without establishing independent biological channels or general architectural superiority. Code, notebooks, and complete results are available at https://dcce-research-site.vercel.app.

0.50

Bridging Echolocation Gaps in Automated Beaked Whale Tracking

eess.SP cs.SD ·2026-09-09 ·Clair Ma, Thomas Kropfreiter, Lauren Baggett, Simone Baumann-Pickering, Florian Meyer
similarity 0.65
Abstract · 43 pages

Passive acoustic monitoring (PAM) is an effective and widely used tool for tracking marine mammals, particularly beaked whales, which are infrequently observed visually because of their deep-diving behavior. However, the large data sets generated by PAM methods often require time-consuming hand labeling to identify whale trajectories in the recorded audio. Automated multi-target tracking (MTT) methods could significantly reduce human workload, but current methods have difficulty forming continuous tracks because of the irregularity of beaked whale echolocation clicks. More precisely, regular sequences of clicks are often interrupted by longer pauses that occur when whales face away from the sensors or stop clicking. Consequently, the probability of detection is difficult to model accurately, and MTT trajectories become fragmented at these pauses. In this paper, we propose a multistage target-estimation method aimed at bridging large gaps in click sequences by combining belief propagation-based MTT with track smoothing and stitching. We validate our method using acoustic recordings of clicks from goose-beaked whales (Ziphius cavirostris), and demonstrate that it improves track estimates and reduces fragmentation in the presence of consecutive missed detections. When evaluated with the generalized optimal subpattern assignment (GOSPA) metric, our method outperforms existing MTT reference methods through reductions in missed-target errors.

0.32

Music of the Deep: An acoustic blueprint of sperm whale codas

cs.SD ·2026-10-25 ·Mudit Sinha, Sanika Chavan
similarity 0.64
Abstract · 10 pages, 3 figures, 4 tables

Sperm-whale codas are usually described by the number and timing of their clicks. Earlier acoustic studies showed that individual clicks and longer coda sequences contain additional organization. We ask how that organization compares with music, whose smaller sounds also form larger units and ordered sequences. We compare codas with performed violin music, drum grooves, and sung melodies. Their sequences resemble music in some measures of predictability, yet codas show a different relationship between the smaller sounds within a unit and the order of units across a sequence. This combination was not observed in the musical references tested. Coda timing also varies coherently and carries information about individual callers. The recordings show another striking property: broad, reproducible variation along acoustic dimensions associated with human ratings of emotion, with a range comparable to human audio in most eligible tests. These findings motivate an acoustic blueprint that describes natural codas beyond their click timing. Its importance is apparent in synthetic audio: WhAM produces recognizable coda-like recordings, yet the tested clips remain distinguishable from natural codas in five of six acoustic comparisons after strict matching. The blueprint provides a fuller way to study natural codas and to evaluate how faithfully synthetic recordings represent them.

0.23

Structure Across Voices: Comparing acoustic-event type accumulation and sequence dependence across four vocal corpora using frozen audio encoders

cs.SD cs.AI ·2026-09-15 ·Mudit Sinha, Sanika Chavan
similarity 0.64
Abstract · 12 pages, 4 figures, 2 tables

Comparing acoustic structure across very different vocal corpora is difficult because the recordings use different native event units, contain different amounts of sequence, and were collected under different conditions. We develop a matched procedure for comparing such corpora without forcing them into the same event definition. Frozen audio encoders provide a common acoustic representation for recorded events, while matched sampling gives each corpus the same amount of evidence for acoustic and sequence analysis. We apply this procedure to sperm-whale codas, human speech phones, Bengalese-finch syllables, and common-marmoset calls. The comparison reveals different patterns across the four corpora. Sperm-whale recordings stand out broadly: new acoustic forms continue to appear, while their sequences also show strong organization and repetition. Finch recordings show the strongest organization of sounds over time, especially in how neighboring sounds and repeated patterns are arranged. Direct temporal and spectral measurements show that these differences are also present in the recorded sounds themselves. The comparison provides a practical way to measure acoustic diversity and temporal organization across very different vocal corpora by combining a common acoustic representation with matched observation opportunity, without treating their native sounds as equivalent.

0.32

Dolphin broadband transient vocalization detection in impulsive ambient noise

cs.SD eess.ASeess.SP ·2026-10-26 ·Hari Vishnu, Jin Xinhui, Mandar Chitre, Simen Hexeberg, Koay Teong Beng
similarity 0.59
Abstract

Passive acoustic monitoring in tropical waters such as those in Southeast Asia is hampered by the persistent, high-level impulsive noise of snapping shrimp. This is a particularly daunting challenge for detecting broadband transient vocalizations of dolphins - clicks, buzzes and burst-pulses. These vocalizations, unlike frequency-modulated whistles, share the broadband, impulsive character of the shrimp snaps and cannot be easily separated from them by frequency-localized or energy-based means. We develop TRIDENT, a lightweight waveform-based detector for these vocalizations, and apply it to 54.2 recorder-months of data from ten island sites around Singapore. We show that the training-data pipeline and the architecture are both important: controlled signal-to-noise-ratio based mixing to generate training data, appropriate normalization and hard negative mining, together steer the detector toward learning transient temporal structure rather than energy shortcuts. A single joint detector handles all three signal types better than specialized models for each, and outperforms established baselines. Applied to long-term recordings, it reveals tightly clustered, predominantly daytime activity, motivating days-with-detections as a robust indicator of cetacean presence.

0.15

A Perch out of Water: Few-Shot Detection of Bryde's Whale Vocalisations in False Bay

eess.AS ·2026-10-21 ·G. Tonitz, C. M. Geldenhuys, T. R. Niesler
similarity 0.58
Abstract · 10 pages, 5 figures, conference

The automated detection of baleen whale calls generally requires large annotated datasets, which are not available for many species and populations. We investigate whether pretrained bioacoustic embeddings, combined with a prototypical few-shot classifier, allow such calls to be detected and classified from only a handful of labelled examples. Two embedding networks are compared: Perch, a general-purpose bioacoustic model trained predominantly on birdsong, and WhaleVAD, a task-specific blue- and fin-whale call detector. Neither model is fine-tuned, and at most ten labelled examples per call type are assumed. Evaluating on Antarctic blue- and fin-whale recordings, we find that the default Perch input representation is poorly matched to low-frequency whale calls, and that increasing the frame length while restricting the mel filterbank to the frequency range occupied by the calls improves the AUC-ROC by 5.8% absolute when ten examples per call type are available. Allowing the classifier to detect each call type independently, so that segments may contain several calls, affords a further 3.3% improvement and a macro AUC-ROC of 0.739, representing a combined gain of 9.1% absolute. Both embeddings are applied to Bryde's whale vocalisations recorded in False Bay, South Africa, which neither model has encountered during pre-training. Here Perch outperforms WhaleVAD for every call type, achieving a macro AUC-ROC of 0.987 against 0.859, despite having a predominantly terrestrial training set. These results indicate that general-purpose bioacoustic embeddings, paired with an input representation matched to the calls of interest, can support whale call detection for species and populations for which large annotated datasets do not exist.

0.22

Interpreting Dolphin Vocal Sequences via Multiple Sequence Alignment

eess.AS ·2026-09-08 ·Daniel Kohlsdorf, Denise Herzing, Thad Starner
unusual similarity 0.58
Abstract

Dolphin communication understanding is essential for uncovering the linguistic complexity and social structures of wild pods. We adapt the ClustalW bioinformatics algorithm to analyze continuous acoustic data, treating vocalizations as high-dimensional spectral feature vectors. By replacing discrete scoring with a continuous Gaussian kernel similarity measure, our framework generates Multiple Sequence Alignment (MSA) visualizations that reveal shared structural patterns. These alignments highlight temporal motifs such as synchronized burst pulses in aggressive contexts that are difficult to detect through standard spectrogram inspection.

0.40

OpenWhistle: A Large-Scale Longitudinal Dataset and Benchmark of Bottlenose Dolphin Vocalizations

cs.CL ·2026-09-28 ·Faadil Mustun, Chiara Semenzin, Roberto Dessi, Pablo Robin Guerrero, Pierre Orhan +5 more
top venuebenchmark similarity 0.57
Abstract · Accepted as a Spotlight at the NeurIPS 2026 Datasets & Evaluations Track

Recent advances in bioacoustics have been driven by large-scale corpora and standardized benchmarks, yet existing resources are overwhelmingly bird-centric and shallow per species, limiting their use for studying the structure of a single species' communication system. This gap is particularly acute for cetaceans: despite bottlenose dolphins (Tursiops truncatus) being a compelling case of complex vocal communication among non-human mammals, existing dolphin datasets are small, fragmented, and largely closed. We introduce OpenWhistle, the largest publicly available dataset of dolphin vocalizations. It comprises approximately 180,000 whistles (114 hours) recorded over five years from a stable pod of five individuals in a semi-natural environment, paired with a curated subset of 8,354 expert-annotated whistles and reproducible evaluation protocols for whistle-type detection and classification. We further release the full processing pipeline for whistle detection, segmentation, and categorization. To demonstrate its utility, we pretrain a Wav2Vec2.0 model adapted to dolphin acoustics on the OpenWhistle corpus and show that it learns effective representations, outperforming general-purpose bioacoustic models such as AVES and BioLingual on both tasks while leaving meaningful headroom for future work. By releasing the dataset, pipeline, and evaluation protocol, we provide the first open dolphin whistle dataset tailored for training self-supervised models, laying the groundwork for advancing dolphin communication research and developing models that capture fine-grained acoustic structure within species.

0.29