Talk2NatureEgyptian fruit bat (Rousettus aegyptiacus) / Machine-learning study
A 2024 model study predicts bat context—with important limits
EUSIPCO 2024 proceedings; arXiv manuscript v1 2024 · Note reviewed 2026-10-24
Triantafyllopoulos et al. trained models to predict 11 video-coded interaction contexts from Egyptian fruit-bat calls. Their best reported result was 33.3% unweighted average recall; this predicts observed context, not message meaning or receiver comprehension. A fresh checksum-verified audit of the paper-described subset matches 35,074 annotations. Strict source-file-plus-participant graphs remain highly connected (largest 95.56%), but a role-aware caller diagnostic excluding 122 mixed-caller source files retains 34,815 annotations (99.26%) and leaves every context represented by 8–41 callers. This makes a caller-held-out test plausible for review, not validated: the paper already reports subject-independent folds, exact membership is unverified, caller identities can appear as addressees in development, and session/chamber/treatment factors remain unjoined. No split or model was created.
What we reviewed · Paper, dataset, split-feasibility and reviewer-independence review
Reviewed all five pages of the official EUSIPCO 2024 proceedings paper and the official arXiv v1 record/manuscript, focusing on dataset filtering, evaluation protocol, model setup, metrics, class errors, data source and license. Also reviewed the first author’s linked public GitHub profile and the pinned Earth Species Library snapshot at commit 6382c52bf553569528e0a790efe16e5ae42f0094: root inventory, bat README and displayed task/split/license sections from notebooks 01, 02, 99a and 99b. No code, split files, raw media, model weights or supplements were downloaded or run. On 2026-10-23, the first author’s live GitHub owner-repository API again listed 13 public repositories (newest update 2025-10-20), and a GitHub repository search for the exact paper title returned no match. This remains a bounded search, not proof that code or splits are unavailable elsewhere. A Zenodo API search was inaccessible from this environment and is not treated as a negative result. The acquired five-page Figshare Metadata.pdf item 4555900 was visually reviewed across Tables 1–4. It lists 20 treatment periods with date ranges, group composition and channels; subject IDs, mother/date-of-birth fields; and an annotation schema with both emitter and addressee. The already-acquired 91,080-row Annotations.csv item was rechecked with an aggregate-only participant-pair audit. No raw annotation rows or IDs were exported. The 31.6 MB FileInfo.csv item and the paper’s exact fold files remain unacquired/unavailable. On 2026-10-23, the checksum-verified Annotations.csv was filtered using the paper-described context, emitter and duration rules. The aggregate-only script matches the reported 35,074-row count and summarizes file-plus-emitter and file-plus-all-known-participant group topology; it exports no row or identity values. No audio, FileInfo.csv, paper fold files, model code or model weights were acquired or run. On 2026-10-23, the official St Andrews group page was reviewed for bat vocal-communication expertise, and the TAU profile plus Nature 2025 author list were checked solely to screen fit and disclose a separate Vernes–Yovel coauthorship. No reviewer was contacted or selected. On October 24, 2026, the official Figshare API and exact CC0-declared annotation file size/MD5 were rechecked, then the aggregate audit was regenerated. The new caller-support diagnostic excludes 122 mixed-caller files and retains 99.26% of the filtered annotations; it creates no split.
The published study is an important prior method; this Talk2Nature assessment has not received independent review by a bat bioacoustics specialist.
Study at a glance
- Data and labels
- The study reuses captive Egyptian fruit-bat ultrasound calls recorded at 250 kHz with infrared video. Context labels were manually assigned from video; 35,074 utterances remained after the stated filters.
- Reported split
- The paper describes three subject-independent development/test folds. Within each development fold, the 70/30 train/validation split was stratified random and did not account for identity.
- Reported results
- UAR: SVM 22.4%, ResNet-50 27.7%, EfficientNet-B0 33.3%. The paper gives 95% intervals from 1,000 bootstrap resamples of examples, not animal/session clusters.
- Error pattern
- The best model had zero recall for grooming and often confused rare or potentially ambiguous classes. The authors discuss possible effects of class imbalance, annotation ambiguity, pup development and chamber acoustics.
- Source-table grouping fields
- The separate five-page Metadata.pdf lists 20 date-bounded treatments, social composition, channels, subject IDs and mother/date-of-birth fields. Annotation rows include emitter and addressee; negative IDs preserve a known participant pair when roles are uncertain, while zero means unknown. The full-table aggregate graph is not the paper subset or its reported folds.
- Talk2Nature role-aware split diagnostics
- The aggregate-only audit matches the paper filter count, not exact rows/folds. Strict file-plus-caller grouping yields 10 connected groups (largest 95.31%); file-plus-all-participant grouping yields 9 (largest 95.56%). Separately, excluding 122 files with multiple known callers retains 34,815/35,074 annotations (99.26%); all 11 contexts remain represented by at least eight callers. This is descriptive support, not validated folds or a split. Caller/addressee roles, exact published folds, sessions, channels, treatments and chambers remain unresolved.
Reading location: EUSIPCO 2024 paper, Sections II Dataset, III Methodology (Cross-validation, SVM and Deep learning), IV Results & Discussion, Tables I–II and confusion matrix; official arXiv v1 submission/version/license record; first author personal site and linked public GitHub repository list; Earth Species Library commit 6382c52: root file inventory, Egyptian-fruit-bat README; notebook 02 task and StratifiedKFold cells; notebook 99a license, FileInfo mapping and no-split statements; notebook 99b emitter subset selection; 2017 dataset descriptor for source context.; local aggregate report: research/figshare_bat_annotation_report.json, generated from the checksum-verified Figshare item 4555903 without exporting row values or IDs; 2026-10-23 GitHub owner-repository API inventory and exact-title repository search; St Andrews Neurogenetics of Vocal Communication group page, TAU Yossi Yovel profile, and Nature 2025 author list, reviewed only for expertise-fit and relationship disclosure
What this does — and doesn’t — establish
The paper’s subject-independent test split is a positive reported safeguard, but Talk2Nature has not inspected the partitions or verified which identities, roles, sessions, dates, microphones, chambers or treatments cross folds. The training/validation split inside each development set ignores identity. Confidence intervals bootstrap individual predictions rather than clustered animals or sessions. Class frequencies are unequal and grooming recall is zero. Context is an observer-coded group/action category, not a test of receiver interpretation. The 2026-10-23 public-code search covered the first author’s 13-repository owner inventory and an exact-title GitHub repository search, not coauthor/lab/archive locations; a Zenodo API search was inaccessible and yielded no negative finding. The pinned Earth Species notebook snapshot predates the EUSIPCO paper and therefore is not its promised release; a separate coauthor/lab/archive release may still exist. The repository root inventory has no visible LICENSE, and notebook 99a states code license “to be linked”; it declares dataset CC BY 4.0, which is not reconciled with Talk2Nature’s direct item-level Figshare records or derived archive-copy provenance. No independent reproduction or specialist review exists. The paper reports ImageNet-pretrained vision weights, not a bioacoustic foundation model. The arXiv v1 manuscript declares CC BY-NC-ND 4.0; proceedings-copy reuse terms were not separately reviewed. The identity graph covers the complete annotation table, not the 35,074 records used by the 2024 paper, and cannot establish whether its three folds isolate all interacting bats, emitters only, sessions, channels, treatments or chamber backgrounds. Metadata.pdf provides treatment and subject context, but the necessary per-file timestamp/channel join is in a separate 31,574,701-byte FileInfo.csv item with no upstream MD5; it was not fetched. The aggregate graph is not an independent split or leakage test. The 2026-10-24 filtered-subset audit reproduces the published row count, not the authors' exact selected-row manifest. Under source-FileID plus positive-known-participant grouping, nine components remain and the largest contains 95.56% of the filtered annotations. Under source-FileID plus positive-emitter grouping, ten components remain and the largest contains 95.31%. These strict component groupings are too concentrated for a balanced whole-component holdout. A separate caller-support diagnostic excludes 122 files with multiple known callers and retains 34,815 of 35,074 annotations (99.26%); it is descriptive support for review, not a split. It does not prevent the same animal appearing as an addressee in development data. Unknown participant roles may hide more links, and the available annotation table does not resolve session, date, channel, treatment or chamber grouping. No model was run.
What we can build on
The reported animal-held-out test split is a useful design target, but must be inspectable and audited by all caller/addressee identities, sessions, files, dates, channels and treatments. A 2021 archive-preparation notebook can clarify FileID-to-file joins without proving that FileInfo.csv is necessary for an unrelated later paper split audit. The Earth Species loader’s shuffled row-level split evaluates emitter identity rather than unseen-animal context prediction; the mini archive drops context labels. Compare acoustic models with class-prior, context-only and background/chamber baselines, use group-aware uncertainty, and keep prediction distinct from translation or comprehension. Because an interaction has two participant fields and uncertainty markers, a subject-independent claim should specify whether the split holds out emitters, both interacting animals, or another unit; it should also account for recording period, treatment/chamber and channel. Source tables make those checks possible in principle but do not prove the reported folds. The count-level reconstruction shows that the existing annotation table is sufficient to verify the reported filtered sample count without acquiring audio or FileInfo.csv. Strict file-plus-emitter graph components are highly imbalanced. A separate role-aware check finds descriptive support for excluding mixed-caller files, but creates no split and does not hold the same animal out when it appears as an addressee in development. An independent reviewer must decide whether this narrower estimand adds value beyond the paper's reported subject-independent folds and whether role/session/chamber effects invalidate it; otherwise defer this archive as a benchmark.
A practical starting point
The video-derived Annotations.csv component was used for a checksum-verified aggregate structure report only; training_admitted remains false. Metadata.pdf was inspected locally and not republished.
Available starting point: The EUSIPCO paper, arXiv v1 manuscript, Annotations.csv, Metadata.pdf, FileInfo.csv, audiovisual archive, code and model weights are separate assets.
Reuse limits: Figshare items 4555903 and 4555900 individually declare CC0 1.0; this is the repository declaration for those items, not legal advice or blanket permission for the article, 31.6 MB FileInfo item, audiovisual files, code or weights. No raw rows or PDF pages are published.
Our proposed experiment: Use aggregates only for source-structure review. Any benchmark requires exact split artifacts, independently reviewed identity/session/channel grouping, artifact-specific rights, specialist review and the remaining frozen-evaluation gates.
OPEN METHODS QUESTION
Independent methods review requested
We invite an independent bat-bioacoustics or animal-behavior researcher to assess whether a caller-held-out, whole-source-file test would support a useful new-caller claim beyond the paper’s already reported subject-independent folds. The aggregate audit retains 99.26% of the filtered annotations after excluding files with multiple known callers, but it creates no split and does not hold out the same bat when it appears as an addressee in development data. Please assess role uncertainty, session/chamber/treatment effects and the exact fold evidence needed. This is a review of published context prediction, not a Talk2Nature model or claim about call meaning. Please disclose authorship of the source dataset or target result, and relevant collaborations, so independence can be characterized accurately.
- Would a caller-held-out, whole-source-file evaluation test generalization to new callers within this controlled cohort, given that a held-out caller could appear as an addressee in development? Which stricter grouping is needed for unseen receivers, pairs or social groups?
- What evidence is needed to verify that all relevant participant roles and repeated recordings are isolated across the reported development/test folds? Which dates, channels, treatments, chambers or recording sessions could confound generalization?
- Are context-only, class-prior, recording-channel/background and shuffled-label baselines adequate for this question? What uncertainty estimate respects repeated animals, pairs and sessions?
- What is the smallest lawful, technically sound metadata check that could resolve the key uncertainty? Is the separate FileInfo.csv actually necessary, given its size and missing upstream checksum?
- The paper reports subject-independent development/test folds, but exact membership is not verified. Would a new-caller evaluation add scientific value beyond that published result, and what smallest artifact would resolve this? Please disclose whether you authored the source dataset or 2024 prediction result, or have a relevant recent collaboration; this is used to characterize review independence, not as an automatic disqualification.
Review status: No independent specialist review has been received. A checksum-verified metadata audit matches the paper’s 35,074-example filter count and finds descriptive support for a caller-held-out question after excluding mixed-caller source files. No split is validated; exact published fold membership and session/chamber effects remain unresolved.
Do not attach recordings, private research material, personal information or sensitive locations. A review is not an endorsement or partnership.
See the related tools and reading paths →
The next reading task
Obtain independent bat-behavior and ML/statistical review of whether a caller-held-out, whole-source-file test adds value beyond the paper's reported subject-independent folds. The aggregate support check excludes 122 mixed-caller files and retains 34,815 of 35,074 filtered annotations, but does not prevent held-out callers appearing as addressees in development or control session, channel, treatment and chamber effects. Ask which exact published fold artifact and minimum metadata join would resolve these risks; defer if no non-duplicative, defensible estimand remains. Only consider FileInfo.csv if a reviewer explains how it could change that decision; then write a bounded integrity plan before any acquisition. Keep model training and AutoResearch blocked until exact evaluation artifacts and the other readiness gates are satisfied. Before counting a review as independent, check authorship of the source dataset and target result and disclose relevant collaboration with their authors; source-author replies can clarify provenance or methods but do not satisfy independent review.
Go to the sources
- Egyptian fruit bat audiovisual vocalization data descriptor ↗
Prat et al. (2017), Scientific Data. Reviewed abstract, methods, data records, annotation quality check and rights statement. The descriptor reports 293,238 250 kHz WAV files and 91,080 video-derived annotations; no files were acquired.
- An automatic analysis of ultrasound vocalisations for the prediction of interaction context in captive Egyptian fruit bats ↗
Peer-reviewed EUSIPCO 2024 study comparing acoustic-feature and spectrogram models on video-context-labelled bat calls.
- Author manuscript: Egyptian fruit-bat context prediction (arXiv v1) ↗
Versioned author manuscript of the EUSIPCO 2024 context-prediction study; arXiv records acceptance at EUSIPCO.
- Andreas Triantafyllopoulos public GitHub profile and repository list ↗
Author-linked GitHub profile and owner repository inventory rechecked as a bounded search for the promised EUSIPCO bat-context code and split files.
- Earth Species Library Egyptian fruit-bat pipeline (pinned 2021 snapshot) ↗
Pinned public repository snapshot containing the bat README, data-download instructions, preprocessing notebooks and PyTorch data-loader notebook.
Earth Species Library Egyptian-fruit-bat folder at the reviewed immutable commit ↗
- Earth Species Library: Egyptian fruit-bat PyTorch data-loader notebook ↗
Pinned notebook constructs a spectrogram dataset for emitter/speaker identification and uses a shuffled, row-level stratified fold.
PyTorch loader notebook at commit 6382c52 (speaker-identity task and row split) ↗
- Earth Species Library: Egyptian fruit-bat preprocessing notebook ↗
Pinned preprocessing notebook maps annotation FileIDs to filenames and records that it did not define a train/validation split.
Preprocessing notebook at commit 6382c52 (FileInfo join and no split) ↗
- Earth Species Library: Egyptian fruit-bat mini-dataset notebook ↗
Pinned notebook builds a smaller emitter-identity subset rather than a context-prediction benchmark.
Mini-dataset notebook at commit 6382c52 (ten-emitter identity subset) ↗
- Egyptian fruit bat subject and annotation metadata (Figshare item 4555900) ↗
The five-page Metadata.pdf item declares CC0 1.0. Tables 1–4 describe treatment periods, dates, group composition, recording channels, subject and maternal fields, annotation columns and label definitions. Talk2Nature reviewed the file locally; no PDF pages or subject IDs were republished.
- Egyptian fruit bat video-derived annotations (Figshare item 4555903) ↗
The item API declares CC0 1.0 for Annotations.csv. Its 91,080-row table contains FileID, emitter, addressee, context, pre/post actions and sample bounds. A local aggregate structural and participant-pair report is available in the repository; raw rows and IDs are not published and training admission remains false.
- University of St Andrews Neurogenetics of Vocal Communication group ↗
The official group profile describes research on mammalian vocal communication and bat vocal-learning and communication behavior. Reviewed for possible subject-expertise fit only; it does not establish availability, interest or independent-review status.
- Bat genomes illuminate adaptations to viral tolerance and disease resistance ↗
The Nature 2025 author list includes Sonja C. Vernes and Yossi Yovel. Checked only to disclose a separate coauthorship relationship during reviewer-independence screening; the genomics paper was not reviewed as evidence about vocal communication.
- Tel Aviv University profile: Yossi Yovel ↗
The official university profile identifies Yossi Yovel as a zoology professor and describes research interests including vocal communication, bat behavior and neuro-ecology.
Original editorial note by Talk2Nature contributors. AI-assisted, source-grounded synthesis; not a peer review or an independent replication. Linked works retain their own rights.