Talk2NatureMammals / Text-to-video retrieval benchmark
Natural-language prompts can retrieve annotated wildlife videos
arXiv v1 posted July 10, 2026; official code README lists ECCV 2026 2026 · Note reviewed 2026-10-11
Prompting-MammAlps tests whether natural-language queries can retrieve camera-trap videos whose structured, visible-behavior annotations satisfy each query. The 135 ecology-informed queries include multi-attribute, multi-animal and temporal relations. This is a text interface to a video-label schema, not translation of animal signals or a two-way communication system.
What we reviewed · Targeted paper, benchmark, release and rights review
Targeted review of arXiv v1 sections 1, 3.1–3.2, 4–7 and supplement S1.1–S1.3; official EPFL project page; pinned Hugging Face dataset card; official GitHub README, pyproject, SALMA README, repository root inventory and checkpoint issue. No dataset, code, article or model weights were acquired; no code or result was run.
Not independently reviewed by a wildlife behavior, ethology or bioacoustics specialist.
Study at a glance
- Question and target
- Can text queries retrieve clips whose structured visual annotations satisfy a visible species, behavior, weather or temporal relation? The target is video retrieval, not acoustic meaning.
- Data
- MammAlps-S2 contains 2,865 camera-trap video clips, 18.4 hours across 15 views, from three sites and two field seasons (2023–2024). The benchmark has 135 queries across seven mammal species, 23 actions and 12 activities; 775 videos form the test candidates.
- Query and annotation construction
- The paper says queries were developed with a behavioral ecologist and matched to video sets using the ground-truth annotations. Exact matching excludes partial matches and retains queries with no matching video. Label reliability was not independently audited in this review.
- Split and evaluation
- The published split groups by acquisition day, not verified individual identity. Set-based F1 is reported per query. The 0.34 SALMA+agent result covers 135 queries; the 0.18 zero-shot InternVideo2.0 comparison covers 113 queries without video-comparison prompts. The paper also reports 0.87 for an oracle query agent applied to ground-truth annotations. None of these results was reproduced by Talk2Nature.
- Component rights and access
- The dataset card declares CC BY-NC 4.0. The code repository has no GitHub license metadata or root LICENSE file; code use is unresolved. The SALMA checkpoint package was not found and a public issue asks about a future release. No dataset, code or weights were acquired.
- Project decision
- Keep as a methods comparator for transparent retrieval over explicit labels. Do not call it animal communication, put it in a commercial-capable corpus, or treat its video-only output as audio meaning.
Reading location: arXiv v1 sections 1, 3.1–3.2, 4, 5, 6 and 7; supplementary S1.1–S1.3; EPFL project page; Hugging Face dataset card revision 04d96ea01312aab5727a39491ef1d736f1df2728; official GitHub main commit 9627d90b3917e9cc78e27fb08eba18d0fdaaa3c4; GitHub issue 1. No full-code audit or independent result reproduction.
What this does — and doesn’t — establish
The benchmark is video-only and covers seven Alpine mammal species, three sites in one Swiss national park, 15 camera views and two non-overlapping seasons (2023 and 2024); it is not a parrot or audio benchmark. The published split is by acquisition day, not verified individual identity, and cross-season distribution shift is reported. Query/video matches are built from ground-truth annotations; independent label-reliability review was not performed here. The authors report set-based F1: SALMA plus the query agent scores 0.34 across all 135 queries and 0.37 on the 113 without video-comparison prompts; zero-shot InternVideo2.0 scores 0.18 only on those 113. The commonly repeated 34% vs 18% figures therefore do not compare identical query sets. An oracle query agent on ground-truth annotations scores 0.87, so retrieval quality depends strongly on the upstream perception/annotation layer. No result was independently reproduced. Dataset terms are CC BY-NC 4.0, the code repository has no declared license, and no SALMA checkpoint release was found.
What we can build on
Natural-language retrieval can make an explicit behavior schema easier to explore, but it does not turn labels into an animal's message. For Talk2Nature, measure label extraction, prompt interpretation and retrieval separately; retain uncertainty, compare against simple baselines and test held-out individuals/sites before considering model training.
See the related tools and reading paths →
The next reading task
Keep metadata-only as a non-commercial retrieval-method comparator. Before any academic reproduction, resolve the dataset download handle, audit the exact split and label provenance, confirm code and model-weight rights, and test cross-year/site generalization. For the first Talk2Nature benchmark, continue seeking rights-cleared synchronized audio and independently timed behavior labels. AutoResearch remains NOT READY.
Go to the sources
- Prompting-MammAlps: fine-grained text-to-video retrieval for camera-trap data ↗
arXiv v1 posted July 10, 2026. Targeted review of benchmark, retrieval method, experiments, results and discussion. The headline 34% vs 18% comparison needs qualification: the SALMA+agent result is 0.34 across 135 queries, while the zero-shot VLM comparison excludes 22 video-comparison queries and reports 0.18 on the remaining 113. No result was reproduced.
- Prompting-MammAlps official EPFL project page ↗
Official project page describes MammAlps-S2, the camera-trap seasons and the 135-query text-to-video retrieval benchmark. It links the paper, code and dataset.
- MammAlps-S2 and Prompting-MammAlps Hugging Face dataset card ↗
Dataset card revision 04d96ea01312aab5727a39491ef1d736f1df2728 describes 2,865 camera-trap video clips, two field seasons, 135 retrieval queries, and a declared CC BY-NC 4.0 license. No data files were downloaded.
- Official Prompting-MammAlps code repository ↗
Official repository main commit 9627d90b3917e9cc78e27fb08eba18d0fdaaa3c4. README and SALMA setup were reviewed. GitHub reports no repository license and the root inventory contains no LICENSE file; code was not cloned or run.
- SALMA checkpoint release request, official repository issue 1 ↗
Issue was open on October 11, 2026 and asks the authors about releasing SALMA checkpoints. No SALMA checkpoint package or model license was located in the reviewed repository.
Original editorial note by Talk2Nature contributors. AI-assisted, source-grounded synthesis; not a peer review or an independent replication. Linked works retain their own rights.